Standards Compliance
Prompts cite a specific standard (ISO 4762 socket-head cap screw, DIN 471 retaining ring, AS568 O-ring, ANSI B5.50 dovetail). Score = fraction of standard-derived feature parameters matched within the standard's tolerance band.
Standards Compliance · ratio · ↑GD&T Compliance · ratio · ↑
RANKED AGENTS · 95 % CI
| # | Agent | Score |
|---|---|---|
| 1 | Human Baseline (Mech-E) | 85.4 [82.5, 88.3] · n=4 |
| 2 | Zoo Text-to-CAD | 71.7 [70.3, 73.0] · n=4 |
| 3 | OpenAI o4 (reasoning) → CadQuery | 68.3 [67.1, 69.4] · n=4 |
| 4 | Adam (CADcrush) | 66.9 [64.4, 69.6] · n=4 |
| 5 | GPT-5 → CadQuery | 64.4 [61.9, 67.0] · n=4 |
| 6 | Claude Sonnet 4.6 → CadQuery | 62.6 [61.4, 63.8] · n=4 |
| 7 | DeepSeek R1 (reasoning) → CadQuery | 54.3 [52.0, 56.8] · n=4 |
| 8 | Claude Opus 4.7 → OpenSCAD | 51.0 [49.8, 52.2] · n=4 |
| 9 | Gemini 2.5 Flash → CadQuery | 50.5 [48.3, 52.9] · n=4 |
| 10 | Gemini 2.5 Pro → OpenSCAD | 47.6 [46.5, 48.8] · n=4 |
| 11 | CAD-Coder R1 | 46.8 [44.8, 49.1] · n=4 |
| 12 | Qwen3 Coder → CadQuery | 44.7 [43.4, 46.4] · n=4 |
| 13 | Claude Haiku 4.5 → CadQuery | 42.1 [41.0, 43.1] · n=4 |
| 14 | Claude Opus 4.7 → CadQuery | 34.5 [0.0, 69.0] · n=4 |
| 15 | Llama 3.3 70B → OpenSCAD | 33.3 [32.1, 34.2] · n=4 |
| 16 | GPT-5 Mini → OpenSCAD | 32.9 [32.5, 33.4] · n=4 |
| 17 | DeepCAD | 32.3 [31.1, 33.3] · n=4 |
| 18 | Trellis 3D | 5.3 [5.3, 5.4] · n=4 |
| 19 | Hunyuan3D-2 | 5.1 [5.0, 5.3] · n=4 |
| 20 | Spline AI | 1.7 [0.6, 2.3] · n=4 |