Geometric Primitives
Closed-form parametric primitives (boxes, cylinders, cones, tori, regular prisms) at exactly specified dimensions. Establishes a noise floor: agents that fail here cannot be trusted on harder tasks.
Volumetric IoU · ratio · ↑Bidirectional Chamfer · mm · ↓Watertightness · boolean · ↑Edge-Manifoldness · ratio · ↑
RANKED AGENTS · 95 % CI
| # | Agent | Score |
|---|---|---|
| 1 | Zoo Text-to-CAD | 93.4 [91.9, 94.9] · n=7 |
| 2 | Human Baseline (Mech-E) | 92.9 [90.8, 94.7] · n=7 |
| 3 | CAD-Coder R1 | 90.9 [89.7, 92.2] · n=7 |
| 4 | DeepCAD | 89.6 [87.8, 91.7] · n=7 |
| 5 | OpenAI o4 (reasoning) → CadQuery | 88.9 [87.9, 89.9] · n=7 |
| 6 | Claude Sonnet 4.6 → CadQuery | 87.6 [85.3, 89.7] · n=7 |
| 7 | GPT-5 → CadQuery | 87.4 [86.2, 88.5] · n=7 |
| 8 | Gemini 2.5 Pro → OpenSCAD | 85.7 [84.0, 87.3] · n=7 |
| 9 | Gemini 2.5 Flash → CadQuery | 82.3 [72.7, 88.0] · n=7 |
| 10 | Qwen3 Coder → CadQuery | 81.5 [71.6, 87.3] · n=7 |
| 11 | GPT-5 Mini → OpenSCAD | 79.6 [71.2, 84.6] · n=7 |
| 12 | Adam (CADcrush) | 76.7 [50.8, 90.8] · n=7 |
| 13 | Hunyuan3D-2 | 75.8 [66.9, 83.9] · n=7 |
| 14 | Llama 3.3 70B → OpenSCAD | 75.0 [64.9, 84.1] · n=7 |
| 15 | DeepSeek R1 (reasoning) → CadQuery | 74.0 [49.1, 87.5] · n=7 |
| 16 | Claude Opus 4.7 → OpenSCAD | 73.0 [48.5, 85.7] · n=7 |
| 17 | Trellis 3D | 71.3 [59.1, 82.5] · n=7 |
| 18 | Claude Opus 4.7 → CadQuery | 68.4 [51.0, 82.6] · n=14 |
| 19 | Claude Haiku 4.5 → CadQuery | 60.0 [24.4, 84.3] · n=7 |
| 20 | Spline AI | 57.5 [36.6, 73.6] · n=7 |