SHEET-007 · Sheet-Metal Bodies · difficulty 4/5

3-bend electronics chassis

sha256:4cb01ef02d3a0091…

§1Prompt verbatim

U-shaped electronics chassis from 1.5 mm Al-5052, three 90° bends, internal volume 200 × 120 × 60 mm. Two louvered vent slots 60 × 8 mm on each long side, six M3 PEM nut clearance holes (Ø 4.2) on the base. Bend radius 1.5 mm, k-factor 0.40.

§2Ground-truth spec

shells1

watertighttrue

manifoldtrue

acceptance ε±0.1 mm

featuresbend_x3, louver_x4, PEM_M3_clearance_x6

§3Reference render

canonical reference · drag to orbit, scroll to zoom

Visualisation is rebuilt in-browser from the canonical parametric description. Scoring is performed against the held-out reference STEP file (sha-256 fingerprint above).

§4Per-agent renders

reference + 10 agent outputs · scored against the held-out STEP

vol IoU · BREP · manifold

canonical reference

REFERENCE

canonical · ground truth

1.000100✓

Human Baseline (Mech-E)

Claude Opus 4.7 → CadQuery

Anthropic + CadQuery 2.4

0.67210✗

OpenAI o4 (reasoning) → CadQuery

OpenAI + CadQuery 2.4

0.60110✗

Qwen3 Coder → CadQuery

Alibaba + CadQuery 2.4

0.58912✗

Gemini 2.5 Flash → CadQuery

Google + CadQuery 2.4

0.56210✗

GPT-5 → CadQuery

OpenAI + CadQuery 2.4

0.54512✗

Claude Opus 4.7 → OpenSCAD

Anthropic + OpenSCAD 2024.06

0.4710✗

CAD-Coder R1

CAD-Coder Labs (research)

Claude Sonnet 4.6 → CadQuery

Anthropic + CadQuery 2.4

0.43414✗

Claude Haiku 4.5 → CadQuery

Anthropic + CadQuery 2.4

0.43216✗

GPT-5 Mini → OpenSCAD

OpenAI + OpenSCAD 2024.06

0.41613✗

DeepCAD

Wu et al. 2021 (research)

0.36716✗

Llama 3.3 70B → OpenSCAD

Meta + OpenSCAD 2024.06

0.34917✗

Gemini 2.5 Pro → OpenSCAD

Google + OpenSCAD 2024.06

no manifold solid produced

DeepSeek R1 (reasoning) → CadQuery

DeepSeek + CadQuery 2.4

—61✗

Each tile is rebuilt from the canonical parametric description and degraded to match the agent's scored profile (tessellation, non-manifold face removal, dimension scale jitter, missing features). Image-only diffusion models render visually plausible meshes but score in the single digits on BREP fidelity — the geometry is not a manifold solid even when the render reads clean.

§5Per-agent metrics

ranked by Vol IoU · same data as the leaderboard, restricted to this task

Agent	Watert.	Manif.	Named-Dimension RMSE	FeatRec	Wall-Thickness Uniformity	p50	latency	cost
Human Baseline (Mech-E)	✓	0.969	0.080	0.875	0.816	—	895.1s	$6.401
Zoo Text-to-CAD	✓	0.961	0.179	0.695	0.630	—	8.0s	$0.187
Claude Opus 4.7 → CadQuery	✓	0.944	0.197	0.705	0.656	—	36.4s	$0.404
OpenAI o4 (reasoning) → CadQuery	✓	0.949	0.185	0.787	0.665	—	99.6s	$1.005
Qwen3 Coder → CadQuery	✓	0.934	0.288	0.575	0.486	—	18.1s	$0.028
Gemini 2.5 Flash → CadQuery	✓	0.930	0.280	0.582	0.524	—	8.5s	$0.018
GPT-5 → CadQuery	✓	0.935	0.270	0.697	0.578	—	44.2s	$0.173
Claude Opus 4.7 → OpenSCAD	✓	0.919	0.248	0.547	0.579	—	39.9s	$0.339
CAD-Coder R1	✓	0.925	0.306	0.683	0.527	—	6.1s	$0.005
Adam (CADcrush)	✓	0.922	0.182	0.638	0.664	—	7.9s	$0.232
Claude Sonnet 4.6 → CadQuery	✓	0.914	0.193	0.714	0.574	—	12.7s	$0.067
Claude Haiku 4.5 → CadQuery	×	0.912	0.302	0.535	0.438	—	10.0s	$0.017
GPT-5 Mini → OpenSCAD	×	0.912	0.310	0.396	0.413	—	9.8s	$0.009
DeepCAD	×	0.907	0.310	0.469	0.403	—	5.3s	$0.020
Llama 3.3 70B → OpenSCAD	×	0.900	0.304	0.445	0.360	—	27.6s	$0.018
Gemini 2.5 Pro → OpenSCAD	×	0.897	0.319	0.515	0.530	—	30.7s	$0.084
Trellis 3D	×	0.870	0.465	0.208	0.131	—	13.0s	$0.047
Hunyuan3D-2	×	0.867	0.509	0.195	0.135	—	35.1s	$0.072
Spline AI	×	0.850	0.598	0.100	0.062	—	9.5s	$0.033
DeepSeek R1 (reasoning) → CadQuery kernel error: BRepCheck_NotClosed	×	0.000	—	—	—	—	73.7s	$0.038