2-D Sketch Constraints
Closed planar profiles defined purely by geometric constraints (tangency, equal-length, perpendicular, coincident). Score is fraction of constraints the agent honors after sketch resolution.
Constraint Solve Rate · ratio · ↑Bidirectional Chamfer · mm · ↓
RANKED AGENTS · 95 % CI
| # | Agent | Score |
|---|---|---|
| 1 | OpenAI o4 (reasoning) → CadQuery | 89.9 [89.4, 90.5] · n=2 |
| 2 | Claude Opus 4.7 → CadQuery | 89.3 [88.5, 90.0] · n=2 |
| 3 | Human Baseline (Mech-E) | 87.3 [86.8, 87.8] · n=2 |
| 4 | GPT-5 → CadQuery | 85.9 [85.3, 86.5] · n=2 |
| 5 | DeepSeek R1 (reasoning) → CadQuery | 84.8 [82.8, 86.8] · n=2 |
| 6 | Claude Sonnet 4.6 → CadQuery | 83.8 [81.5, 86.2] · n=2 |
| 7 | Adam (CADcrush) | 83.6 [83.6, 83.6] · n=2 |
| 8 | Zoo Text-to-CAD | 82.8 [81.9, 83.8] · n=2 |
| 9 | Qwen3 Coder → CadQuery | 80.0 [78.5, 81.6] · n=2 |
| 10 | CAD-Coder R1 | 77.1 [76.3, 77.8] · n=2 |
| 11 | Claude Opus 4.7 → OpenSCAD | 76.8 [74.9, 78.8] · n=2 |
| 12 | Gemini 2.5 Flash → CadQuery | 75.5 [74.9, 76.1] · n=2 |
| 13 | Gemini 2.5 Pro → OpenSCAD | 73.4 [73.3, 73.5] · n=2 |
| 14 | Claude Haiku 4.5 → CadQuery | 71.0 [70.7, 71.3] · n=2 |
| 15 | GPT-5 Mini → OpenSCAD | 68.5 [67.8, 69.3] · n=2 |
| 16 | Llama 3.3 70B → OpenSCAD | 64.2 [64.1, 64.3] · n=2 |
| 17 | DeepCAD | 55.5 [52.7, 58.4] · n=2 |
| 18 | Trellis 3D | 12.2 [12.0, 12.4] · n=2 |
| 19 | Spline AI | 11.9 [11.7, 12.1] · n=2 |
| 20 | Hunyuan3D-2 | 0.0 [0.0, 0.0] · n=2 |