PRIM-003 · Geometric Primitives · difficulty 1/5

Right truncated cone (frustum)

sha256:2c3fa9001ee04d80…

§1Prompt verbatim

Right circular frustum: bottom Ø 60 mm, top Ø 30 mm, height 50 mm. Origin at the centroid of the bottom face. Solid, manifold.

§2Ground-truth spec

bbox60.0 × 60.0 × 50.0 mm

shells1

V−E+F2

genus0

watertighttrue

manifoldtrue

acceptance ε±0.03 mm

§3Reference render

canonical reference · drag to orbit, scroll to zoom

Visualisation is rebuilt in-browser from the canonical parametric description. Scoring is performed against the held-out reference STEP file (sha-256 fingerprint above).

§4Per-agent renders

reference + 10 agent outputs · scored against the held-out STEP

vol IoU · BREP · manifold

canonical reference

REFERENCE

canonical · ground truth

1.000100✓

Human Baseline (Mech-E)

Gemini 2.5 Pro → OpenSCAD

Google + OpenSCAD 2024.06

0.7070✓

CAD-Coder R1

CAD-Coder Labs (research)

0.67511✓

DeepSeek R1 (reasoning) → CadQuery

DeepSeek + CadQuery 2.4

0.64212✗

DeepCAD

Wu et al. 2021 (research)

0.63312✗

Qwen3 Coder → CadQuery

Alibaba + CadQuery 2.4

0.6319✓

Claude Opus 4.7 → CadQuery

Anthropic + CadQuery 2.4

0.61610✗

OpenAI o4 (reasoning) → CadQuery

OpenAI + CadQuery 2.4

0.6157✗

Claude Sonnet 4.6 → CadQuery

Anthropic + CadQuery 2.4

0.5779✗

GPT-5 → CadQuery

OpenAI + CadQuery 2.4

0.5738✗

Gemini 2.5 Flash → CadQuery

Google + CadQuery 2.4

0.5479✗

Claude Haiku 4.5 → CadQuery

Anthropic + CadQuery 2.4

Llama 3.3 70B → OpenSCAD

Meta + OpenSCAD 2024.06

GPT-5 Mini → OpenSCAD

OpenAI + OpenSCAD 2024.06

no manifold solid produced

Claude Opus 4.7 → OpenSCAD

Anthropic + OpenSCAD 2024.06

—18✗

Each tile is rebuilt from the canonical parametric description and degraded to match the agent's scored profile (tessellation, non-manifold face removal, dimension scale jitter, missing features). Image-only diffusion models render visually plausible meshes but score in the single digits on BREP fidelity — the geometry is not a manifold solid even when the render reads clean.

§5Per-agent metrics

ranked by Vol IoU · same data as the leaderboard, restricted to this task

Agent	Vol IoU	Bidirectional Chamfer	Watert.	Manif.	p50	latency	cost
Human Baseline (Mech-E)	0.802	0.107	✓	0.975	—	698.4s	$6.222
Adam (CADcrush)	0.773	0.144	✓	0.959	—	10.5s	$0.312
Zoo Text-to-CAD	0.762	0.127	✓	0.969	—	6.2s	$0.172
Gemini 2.5 Pro → OpenSCAD	0.707	0.150	✓	0.951	—	34.6s	$0.102
CAD-Coder R1	0.675	0.152	✓	0.950	—	6.5s	$0.006
DeepSeek R1 (reasoning) → CadQuery	0.642	0.129	✓	0.943	—	96.6s	$0.036
DeepCAD	0.633	0.157	✓	0.949	—	4.1s	$0.024
Qwen3 Coder → CadQuery	0.631	0.130	✓	0.950	—	22.3s	$0.032
Claude Opus 4.7 → CadQuery	0.616	0.148	✓	0.943	—	41.0s	$0.299
OpenAI o4 (reasoning) → CadQuery	0.615	0.122	✓	0.945	—	130.8s	$0.901
Claude Sonnet 4.6 → CadQuery	0.577	0.165	✓	0.945	—	21.7s	$0.080
GPT-5 → CadQuery	0.573	0.171	✓	0.938	—	31.4s	$0.220
Gemini 2.5 Flash → CadQuery	0.547	0.128	✓	0.939	—	12.6s	$0.022
Claude Haiku 4.5 → CadQuery	0.537	0.146	✓	0.927	—	9.1s	$0.023
Trellis 3D	0.519	0.159	✓	0.929	—	15.5s	$0.046
Llama 3.3 70B → OpenSCAD	0.472	0.182	✓	0.916	—	18.8s	$0.021
Spline AI	0.470	0.214	✓	0.917	—	9.8s	$0.040
GPT-5 Mini → OpenSCAD	0.464	0.202	✓	0.918	—	11.3s	$0.009
Hunyuan3D-2	0.422	0.178	✓	0.915	—	29.1s	$0.076
Claude Opus 4.7 → OpenSCAD kernel error: BRepCheck_NotClosed	0.000	—	×	0.000	—	37.7s	$0.341