PRIM-001 · Geometric Primitives · difficulty 1/5
Hollow cylinder (60 × 40 × 100)
sha256:e3b0c44298fc1c14…
§1Prompt verbatim
Model a hollow cylinder with outer diameter 60 mm, inner diameter 40 mm, height 100 mm. Origin at the centroid of the bottom face. Output a watertight solid.
§2Ground-truth spec
volume157080.0 mm³
surface area34558.0 mm²
bbox60.0 × 60.0 × 100.0 mm
shells1
V−E+F0
genus1
watertighttrue
manifoldtrue
acceptance ε±0.05 mm
§3Reference render
canonical reference · drag to orbit, scroll to zoom
Visualisation is rebuilt in-browser from the canonical parametric description. Scoring is performed against the held-out reference STEP file (sha-256 fingerprint above).
§4Per-agent renders
reference + 10 agent outputs · scored against the held-out STEP
vol IoU · BREP · manifold
canonical reference
REFERENCE
canonical · ground truth
1.000100✓
Human Baseline (Mech-E)
Human Baseline (Mech-E)
n=4 senior engineers
0.84310✓
Zoo Text-to-CAD
Zoo Text-to-CAD
Zoo (KittyCAD)
0.8347✓
DeepCAD
DeepCAD
Wu et al. 2021 (research)
0.7999✓
Adam (CADcrush)
Adam (CADcrush)
CADcrush
0.77511✓
Claude Opus 4.7 → CadQuery
Claude Opus 4.7 → CadQuery
Anthropic + CadQuery 2.4
0.7559✓
GPT-5 → CadQuery
GPT-5 → CadQuery
OpenAI + CadQuery 2.4
0.6788✗
Trellis 3D
Trellis 3D
Microsoft Research
0.5160✗
Gemini 2.5 Pro → OpenSCAD
Gemini 2.5 Pro → OpenSCAD
Google + OpenSCAD 2024.06
0.4990✗
no manifold solid produced
Claude Opus 4.7 → OpenSCAD
Claude Opus 4.7 → OpenSCAD
Anthropic + OpenSCAD 2024.06
—18✗
no manifold solid produced
Spline AI
Spline AI
Spline.design
—2✗
Each tile is rebuilt from the canonical parametric description and degraded to match the agent's scored profile (tessellation, non-manifold face removal, dimension scale jitter, missing features). Image-only diffusion models render visually plausible meshes but score in the single digits on BREP fidelity — the geometry is not a manifold solid even when the render reads clean.
§5Per-agent metrics
ranked by Vol IoU · same data as the leaderboard, restricted to this task
| Agent | Vol IoU | Bidirectional Chamfer | Watert. | Manif. | P@1 | p50 | latency | cost |
|---|---|---|---|---|---|---|---|---|
| Human Baseline (Mech-E) | 0.843 | 0.138 | ✓ | 0.974 | 0.000 | — | 792.5s | $7.170 |
| Zoo Text-to-CAD | 0.834 | 0.127 | ✓ | 0.969 | 0.000 | — | 6.3s | $0.146 |
| DeepCAD | 0.799 | 0.117 | ✓ | 0.973 | 0.000 | — | 5.4s | $0.024 |
| Adam (CADcrush) | 0.775 | 0.147 | ✓ | 0.960 | 0.000 | — | 11.1s | $0.226 |
| Claude Opus 4.7 → CadQuery | 0.755 | 0.143 | ✓ | 0.960 | 0.000 | — | 31.0s | $0.339 |
| GPT-5 → CadQuery | 0.678 | 0.157 | ✓ | 0.949 | 0.000 | — | 31.6s | $0.208 |
| Trellis 3D | 0.516 | 0.178 | ✓ | 0.926 | 0.000 | — | 9.7s | $0.051 |
| Gemini 2.5 Pro → OpenSCAD | 0.499 | 0.189 | ✓ | 0.926 | 0.000 | — | 35.6s | $0.094 |
| Claude Opus 4.7 → OpenSCAD kernel error: BRepCheck_NotClosed | 0.000 | — | × | 0.000 | 0.000 | — | 39.4s | $0.318 |
| Spline AI kernel error: BRepCheck_NotClosed | 0.000 | — | × | 0.000 | 0.000 | — | 8.3s | $0.045 |