DFMFDM-008 · DFM · FDM 3D Print · difficulty 3/5
FDM-printable hinge (no support)
sha256:44e92cd1aa7f3380…
§1Prompt verbatim
Living-hinge 80 × 30 × 6 mm with two 40 × 30 × 6 mm leaves and a 0.5 mm hinge web. All overhangs ≤ 45° from build plate, single body, FDM-printable without support material on a 0.4 mm nozzle.
§2Ground-truth spec
shells1
watertighttrue
manifoldtrue
acceptance ε±0.1 mm
featureshinge_web_0.5, no_overhang_gt45
§3Reference render
canonical reference · drag to orbit, scroll to zoom
Visualisation is rebuilt in-browser from the canonical parametric description. Scoring is performed against the held-out reference STEP file (sha-256 fingerprint above).
§4Per-agent renders
reference + 10 agent outputs · scored against the held-out STEP
vol IoU · BREP · manifold
canonical reference
REFERENCE
canonical · ground truth
1.000100✓
GPT-5 → CadQuery
GPT-5 → CadQuery
OpenAI + CadQuery 2.4
0.7079✓
Human Baseline (Mech-E)
Human Baseline (Mech-E)
n=4 senior engineers
0.6927✓
Claude Opus 4.7 → OpenSCAD
Claude Opus 4.7 → OpenSCAD
Anthropic + OpenSCAD 2024.06
0.6240✗
Claude Sonnet 4.6 → CadQuery
Claude Sonnet 4.6 → CadQuery
Anthropic + CadQuery 2.4
0.60013✗
Qwen3 Coder → CadQuery
Qwen3 Coder → CadQuery
Alibaba + CadQuery 2.4
0.58512✗
OpenAI o4 (reasoning) → CadQuery
OpenAI o4 (reasoning) → CadQuery
OpenAI + CadQuery 2.4
0.55613✗
Gemini 2.5 Pro → OpenSCAD
Gemini 2.5 Pro → OpenSCAD
Google + OpenSCAD 2024.06
0.5520✗
Llama 3.3 70B → OpenSCAD
Llama 3.3 70B → OpenSCAD
Meta + OpenSCAD 2024.06
0.51010✗
Zoo Text-to-CAD
Zoo Text-to-CAD
Zoo (KittyCAD)
0.5019✗
Claude Opus 4.7 → CadQuery
Claude Opus 4.7 → CadQuery
Anthropic + CadQuery 2.4
0.47310✗
Adam (CADcrush)
Adam (CADcrush)
CADcrush
0.43412✗
GPT-5 Mini → OpenSCAD
GPT-5 Mini → OpenSCAD
OpenAI + OpenSCAD 2024.06
0.41911✗
Gemini 2.5 Flash → CadQuery
Gemini 2.5 Flash → CadQuery
Google + CadQuery 2.4
0.40314✗
CAD-Coder R1
CAD-Coder R1
CAD-Coder Labs (research)
0.39016✗
Claude Haiku 4.5 → CadQuery
Claude Haiku 4.5 → CadQuery
Anthropic + CadQuery 2.4
0.33118✗
DeepCAD
DeepCAD
Wu et al. 2021 (research)
0.23822✗
Spline AI
Spline AI
Spline.design
0.1750✗
no manifold solid produced
Trellis 3D
Trellis 3D
Microsoft Research
—4✗
no manifold solid produced
DeepSeek R1 (reasoning) → CadQuery
DeepSeek R1 (reasoning) → CadQuery
DeepSeek + CadQuery 2.4
—61✗
no manifold solid produced
Hunyuan3D-2
Hunyuan3D-2
Tencent
—5✗
Each tile is rebuilt from the canonical parametric description and degraded to match the agent's scored profile (tessellation, non-manifold face removal, dimension scale jitter, missing features). Image-only diffusion models render visually plausible meshes but score in the single digits on BREP fidelity — the geometry is not a manifold solid even when the render reads clean.
§5Per-agent metrics
ranked by Vol IoU · same data as the leaderboard, restricted to this task
| Agent | Watert. | Manif. | DFM | Min-Wall Compliance | Support-Volume Ratio | P@1 | p50 | latency | cost |
|---|---|---|---|---|---|---|---|---|---|
| GPT-5 → CadQuery | ✓ | 0.950 | 81.1 | 0.617 | 0.291 | 0.000 | — | 50.8s | $0.202 |
| Human Baseline (Mech-E) | ✓ | 0.958 | 89.1 | 0.842 | 0.295 | 0.000 | — | 557.1s | $5.246 |
| Claude Opus 4.7 → OpenSCAD | ✓ | 0.943 | 76.5 | 0.611 | 0.494 | 0.000 | — | 39.1s | $0.332 |
| Claude Sonnet 4.6 → CadQuery | ✓ | 0.936 | 78.3 | 0.595 | 0.422 | 0.000 | — | 20.3s | $0.077 |
| Qwen3 Coder → CadQuery | ✓ | 0.934 | 73.0 | 0.474 | 0.377 | 0.000 | — | 17.1s | $0.035 |
| OpenAI o4 (reasoning) → CadQuery | ✓ | 0.930 | 82.8 | 0.678 | 0.268 | 0.000 | — | 128.0s | $1.130 |
| Gemini 2.5 Pro → OpenSCAD | ✓ | 0.935 | 74.9 | 0.554 | 0.441 | 0.000 | — | 31.8s | $0.107 |
| Llama 3.3 70B → OpenSCAD | ✓ | 0.928 | 74.1 | 0.400 | 0.611 | 0.000 | — | 25.6s | $0.018 |
| Zoo Text-to-CAD | ✓ | 0.929 | 78.3 | 0.682 | 0.455 | 0.000 | — | 6.3s | $0.216 |
| Claude Opus 4.7 → CadQuery | ✓ | 0.925 | 79.4 | 0.612 | 0.495 | 0.000 | — | 31.9s | $0.289 |
| Adam (CADcrush) | ✓ | 0.915 | 76.3 | 0.626 | 0.423 | 0.000 | — | 12.0s | $0.310 |
| GPT-5 Mini → OpenSCAD | ✓ | 0.914 | 75.1 | 0.465 | 0.409 | 0.000 | — | 12.2s | $0.009 |
| Gemini 2.5 Flash → CadQuery | × | 0.910 | 75.0 | 0.538 | 0.304 | 0.000 | — | 13.5s | $0.021 |
| CAD-Coder R1 | ✓ | 0.914 | 71.9 | 0.539 | 0.372 | 0.000 | — | 7.4s | $0.004 |
| Claude Haiku 4.5 → CadQuery | × | 0.904 | 70.6 | 0.420 | 0.504 | 0.000 | — | 6.2s | $0.024 |
| DeepCAD | × | 0.888 | 64.1 | 0.367 | 0.564 | 0.000 | — | 4.1s | $0.024 |
| Spline AI | × | 0.878 | 50.9 | 0.102 | 1.000 | 0.000 | — | 6.2s | $0.044 |
| Trellis 3D kernel error: BRepCheck_NotClosed | × | 0.000 | — | — | — | 0.000 | — | 14.2s | $0.057 |
| DeepSeek R1 (reasoning) → CadQuery kernel error: BRepCheck_NotClosed | × | 0.000 | — | — | — | 0.000 | — | 118.0s | $0.043 |
| Hunyuan3D-2 kernel error: BRepCheck_NotClosed | × | 0.000 | — | — | — | 0.000 | — | 35.9s | $0.059 |