Assembly & Mating
Two- and three-body assemblies with pin-in-hole, dovetail, and threaded mate constraints. Scoring requires the candidate to mate the held-out reference partner within the prescribed clearance band.
Mating Clearance · ratio · ↑Fit-Class Compliance · ratio · ↑Feature Recall · ratio · ↑
RANKED AGENTS · 95 % CI
| # | Agent | Score |
|---|---|---|
| 1 | Human Baseline (Mech-E) | 84.2 [81.4, 87.8] · n=5 |
| 2 | Zoo Text-to-CAD | 66.1 [63.9, 68.1] · n=5 |
| 3 | OpenAI o4 (reasoning) → CadQuery | 63.8 [62.3, 65.7] · n=5 |
| 4 | Claude Sonnet 4.6 → CadQuery | 54.7 [53.4, 56.6] · n=5 |
| 5 | DeepSeek R1 (reasoning) → CadQuery | 49.4 [48.2, 50.1] · n=5 |
| 6 | Adam (CADcrush) | 48.3 [24.1, 61.3] · n=5 |
| 7 | CAD-Coder R1 | 48.1 [47.2, 48.7] · n=5 |
| 8 | Qwen3 Coder → CadQuery | 43.7 [42.4, 44.9] · n=5 |
| 9 | GPT-5 → CadQuery | 43.6 [21.2, 55.7] · n=5 |
| 10 | Claude Opus 4.7 → OpenSCAD | 43.5 [42.8, 44.6] · n=5 |
| 11 | Gemini 2.5 Flash → CadQuery | 43.1 [42.3, 43.9] · n=5 |
| 12 | Claude Opus 4.7 → CadQuery | 36.9 [12.0, 61.8] · n=5 |
| 13 | DeepCAD | 34.0 [33.4, 34.7] · n=5 |
| 14 | Gemini 2.5 Pro → OpenSCAD | 31.3 [15.3, 39.8] · n=5 |
| 15 | Llama 3.3 70B → OpenSCAD | 30.3 [29.5, 31.1] · n=5 |
| 16 | GPT-5 Mini → OpenSCAD | 29.2 [28.1, 30.3] · n=5 |
| 17 | Claude Haiku 4.5 → CadQuery | 28.3 [14.1, 35.9] · n=5 |
| 18 | Trellis 3D | 8.1 [4.0, 10.4] · n=5 |
| 19 | Hunyuan3D-2 | 8.0 [4.0, 10.2] · n=5 |
| 20 | Spline AI | 3.7 [1.8, 4.7] · n=5 |