ASM-011 · Assembly & Mating · difficulty 4/5
Dovetail slide (60° flanks)
sha256:773bb55ecd0c1a2e…
§1Prompt verbatim
Male dovetail per ANSI B5.50: 30 mm tall, 50 mm wide at the base, 60° flank angle, 100 mm long. Held-out female slot is 0.04 mm wider on each flank for free running. Assembled clearance must be 0.04 ± 0.01 mm normal to each flank.
§2Ground-truth spec
shells1
watertighttrue
manifoldtrue
acceptance ε±0.01 mm
clearance[0.03, 0.05] mm
§3Reference render
canonical reference · drag to orbit, scroll to zoom
Visualisation is rebuilt in-browser from the canonical parametric description. Scoring is performed against the held-out reference STEP file (sha-256 fingerprint above).
§4Per-agent renders
reference + 10 agent outputs · scored against the held-out STEP
vol IoU · BREP · manifold
canonical reference
REFERENCE
canonical · ground truth
1.000100✓
Human Baseline (Mech-E)
Human Baseline (Mech-E)
n=4 senior engineers
0.8588✓
Zoo Text-to-CAD
Zoo Text-to-CAD
Zoo (KittyCAD)
0.6379✗
Claude Opus 4.7 → CadQuery
Claude Opus 4.7 → CadQuery
Anthropic + CadQuery 2.4
0.57510✗
Adam (CADcrush)
Adam (CADcrush)
CADcrush
0.52510✗
GPT-5 → CadQuery
GPT-5 → CadQuery
OpenAI + CadQuery 2.4
0.50713✗
Claude Opus 4.7 → OpenSCAD
Claude Opus 4.7 → OpenSCAD
Anthropic + OpenSCAD 2024.06
0.4610✗
DeepCAD
DeepCAD
Wu et al. 2021 (research)
0.37214✗
Gemini 2.5 Pro → OpenSCAD
Gemini 2.5 Pro → OpenSCAD
Google + OpenSCAD 2024.06
0.2790✗
Trellis 3D
Trellis 3D
Microsoft Research
0.0400✗
Spline AI
Spline AI
Spline.design
0.0170✗
Each tile is rebuilt from the canonical parametric description and degraded to match the agent's scored profile (tessellation, non-manifold face removal, dimension scale jitter, missing features). Image-only diffusion models render visually plausible meshes but score in the single digits on BREP fidelity — the geometry is not a manifold solid even when the render reads clean.
§5Per-agent metrics
ranked by Vol IoU · same data as the leaderboard, restricted to this task
| Agent | Watert. | Manif. | FeatRec | Mating Clearance | Fit-Class Compliance | P@1 | p50 | latency | cost |
|---|---|---|---|---|---|---|---|---|---|
| Human Baseline (Mech-E) | ✓ | 0.971 | 0.935 | 0.745 | 0.824 | 1.000 | — | 782.9s | $5.630 |
| Zoo Text-to-CAD | ✓ | 0.948 | 0.790 | 0.720 | 0.566 | 0.000 | — | 6.2s | $0.208 |
| Claude Opus 4.7 → CadQuery | ✓ | 0.942 | 0.708 | 0.589 | 0.433 | 0.000 | — | 42.2s | $0.390 |
| Adam (CADcrush) | ✓ | 0.931 | 0.686 | 0.664 | 0.461 | 0.000 | — | 9.0s | $0.305 |
| GPT-5 → CadQuery | ✓ | 0.923 | 0.657 | 0.544 | 0.375 | 0.000 | — | 45.5s | $0.211 |
| Claude Opus 4.7 → OpenSCAD | ✓ | 0.917 | 0.582 | 0.460 | 0.259 | 0.000 | — | 41.4s | $0.372 |
| DeepCAD | × | 0.902 | 0.486 | 0.385 | 0.173 | 0.000 | — | 5.2s | $0.022 |
| Gemini 2.5 Pro → OpenSCAD | × | 0.894 | 0.510 | 0.454 | 0.196 | 0.000 | — | 27.1s | $0.095 |
| Trellis 3D | × | 0.856 | 0.190 | 0.085 | 0.005 | 0.000 | — | 14.0s | $0.041 |
| Spline AI | × | 0.853 | 0.098 | 0.043 | 0.001 | 0.000 | — | 6.1s | $0.039 |