BREP-001 · BREP Fidelity · difficulty 4/5

Periodic-spline cylinder (closed in U)

sha256:44a01ef0bcb117e0…

§1Prompt verbatim

Right cylinder Ø 40 × 60 mm whose lateral face is a single B-spline surface that closes periodically in U (no seam edge). 12 control points per ring × 3 rings, knot vector clamped in V only. Export AP242.

§2Ground-truth spec

shells1

watertighttrue

manifoldtrue

acceptance ε±0.02 mm

featuresperiodic_spline_lateral

Requires the kernel to honour periodicity — a closed spline with seam still scores 0 on this task.

§3Reference render

canonical reference · drag to orbit, scroll to zoom

Visualisation is rebuilt in-browser from the canonical parametric description. Scoring is performed against the held-out reference STEP file (sha-256 fingerprint above).

§4Per-agent renders

reference + 10 agent outputs · scored against the held-out STEP

vol IoU · BREP · manifold

canonical reference

REFERENCE

canonical · ground truth

1.000100✓

Human Baseline (Mech-E)

CAD-Coder Labs (research)

0.61911✗

Claude Opus 4.7 → CadQuery

Anthropic + CadQuery 2.4

0.61310✗

Qwen3 Coder → CadQuery

Alibaba + CadQuery 2.4

0.6049✗

DeepCAD

Wu et al. 2021 (research)

0.55411✗

DeepSeek R1 (reasoning) → CadQuery

DeepSeek + CadQuery 2.4

0.51411✗

Claude Sonnet 4.6 → CadQuery

Anthropic + CadQuery 2.4

OpenAI o4 (reasoning) → CadQuery

OpenAI + CadQuery 2.4

0.47711✗

GPT-5 → CadQuery

OpenAI + CadQuery 2.4

0.43312✗

Claude Haiku 4.5 → CadQuery

Anthropic + CadQuery 2.4

0.40516✗

Gemini 2.5 Flash → CadQuery

Google + CadQuery 2.4

0.39816✗

Claude Opus 4.7 → OpenSCAD

Anthropic + OpenSCAD 2024.06

0.0190✗

GPT-5 Mini → OpenSCAD

OpenAI + OpenSCAD 2024.06

0.004105✗

Gemini 2.5 Pro → OpenSCAD

Google + OpenSCAD 2024.06

Llama 3.3 70B → OpenSCAD

Meta + OpenSCAD 2024.06

Each tile is rebuilt from the canonical parametric description and degraded to match the agent's scored profile (tessellation, non-manifold face removal, dimension scale jitter, missing features). Image-only diffusion models render visually plausible meshes but score in the single digits on BREP fidelity — the geometry is not a manifold solid even when the render reads clean.

§5Per-agent metrics

ranked by Vol IoU · same data as the leaderboard, restricted to this task

Agent	Watert.	Manif.	Euler-Poincaré Compliance	STEP RT	FeatRec	P@1	p50	latency	cost
Human Baseline (Mech-E)	✓	0.973	✓	0.067	0.951	1.000	—	761.7s	$6.340
Zoo Text-to-CAD	✓	0.967	✓	0.069	0.796	1.000	—	5.6s	$0.180
CAD-Coder R1	✓	0.937	✓	0.114	0.644	0.000	—	7.1s	$0.005
Claude Opus 4.7 → CadQuery	✓	0.948	✓	0.099	0.697	0.000	—	42.8s	$0.282
Qwen3 Coder → CadQuery	✓	0.937	✓	0.090	0.632	0.000	—	17.0s	$0.033
DeepCAD	✓	0.939	✓	0.111	0.470	0.000	—	5.9s	$0.023
DeepSeek R1 (reasoning) → CadQuery	✓	0.924	×	0.105	0.675	0.000	—	118.1s	$0.039
Claude Sonnet 4.6 → CadQuery	✓	0.931	✓	0.113	0.634	0.000	—	22.0s	$0.077
Adam (CADcrush)	✓	0.925	×	0.098	0.717	0.000	—	11.7s	$0.323
OpenAI o4 (reasoning) → CadQuery	✓	0.922	×	0.113	0.757	0.000	—	105.2s	$1.203
GPT-5 → CadQuery	✓	0.914	×	0.124	0.717	0.000	—	51.4s	$0.176
Claude Haiku 4.5 → CadQuery	×	0.911	×	0.159	0.530	0.000	—	9.3s	$0.020
Gemini 2.5 Flash → CadQuery	×	0.909	×	0.155	0.599	0.000	—	13.5s	$0.018
Claude Opus 4.7 → OpenSCAD	×	0.853	×	—	0.573	0.000	—	26.3s	$0.367
GPT-5 Mini → OpenSCAD	×	0.851	×	1.045	0.434	0.000	—	12.8s	$0.009
Gemini 2.5 Pro → OpenSCAD	×	0.850	×	—	0.520	0.000	—	34.1s	$0.106
Trellis 3D	×	0.850	×	—	0.206	0.000	—	8.8s	$0.051
Spline AI	×	0.850	×	—	0.090	0.000	—	7.8s	$0.039
Llama 3.3 70B → OpenSCAD	×	0.850	×	1.005	0.414	0.000	—	25.8s	$0.019
Hunyuan3D-2	×	0.850	×	1.011	0.212	0.000	—	36.1s	$0.080