Claude Opus 4.7 → CadQuery
A · Engineering-Capablecomposite 72.5 [68.9, 76.6]p5 57.7IRT θ 97.8
Anthropic + CadQuery 2.4· vopus-4.7 + cadquery 2.4· released 2026-01-22· LLM+CadQuery· emits CadQuery· proprietary· 1000k ctx· $0.015/$0.075 per 1k tok
Few-shot scaffold (8 exemplars from the OCC tutorial set), self-repair loop with up to 3 OCC error feedbacks. Executes in a Vercel Sandbox per call.
solid: agent · dashed: human baseline
PER-CATEGORY SCORE (95 % CI)
Geometric Primitives
89.7
Boolean Robustness
87.5
BREP Fidelity
90.7
Free-form Surfaces
80.6
Parametric Mechanical Parts
72.3
Assembly & Mating
58.7
Standards Compliance
69.5
Sheet-Metal Bodies
70.1
Sealing-Groove Design
73.2
Kinematic Mechanisms
73.1
DFM · 3-Axis CNC
71.0
DFM · Injection Mould
64.3
DFM · FDM 3D Print
75.7
CAM Toolpath Validity
72.2
Constraint Solving & Editability
79.8
Reverse Engineering
70.8
2-D Sketch Constraints
84.8
Functional Intent · FEA-Gated
62.6
Paraphrase Robustness
85.0
Confidence Calibration
39.5
MEAN METRICS BY LAYER
L1 · Geometry
| Category | Vol IoU | Chamfer | Hausdorff | NormCons | Watert. | Manif. | Euler | STEP RT | P@1 |
|---|---|---|---|---|---|---|---|---|---|
| Geometric Primitives | 0.704 | 0.140 | — | — | — | 0.953 | — | — | — |
| Boolean Robustness | 0.560 | — | — | — | — | 0.940 | — | — | — |
| BREP Fidelity | — | — | — | — | — | 0.930 | — | 0.088 | — |
| Free-form Surfaces | — | 0.302 | 1.359 | 0.731 | — | — | — | — | — |
L2 · Engineering
| Category | Dim RMSE | GD&T | FeatRec | Mating | Fit-Cls | Std |
|---|---|---|---|---|---|---|
| Parametric Mechanical Parts | 0.223 | 0.624 | 0.768 | — | — | — |
| Assembly & Mating | — | — | 0.742 | 0.595 | 0.423 | — |
| Standards Compliance | — | 0.669 | — | — | — | 0.720 |
| Sheet-Metal Bodies | 0.264 | — | 0.685 | — | — | — |
| Sealing-Groove Design | 0.238 | — | — | — | — | 0.701 |
| Kinematic Mechanisms | — | — | 0.715 | 0.667 | — | — |
L3 · Manufacturing
| Category | DFM | Draft | Min-Wall | CAM-Reach | Supp | Wall-Unif |
|---|---|---|---|---|---|---|
| DFM · 3-Axis CNC | 82.400 | — | 0.622 | 0.683 | — | — |
| DFM · Injection Mould | — | 0.594 | 0.708 | — | — | 0.628 |
| DFM · FDM 3D Print | 81.500 | — | 0.643 | — | 0.376 | — |
| CAM Toolpath Validity | — | — | — | 0.691 | — | — |
L4 · Cognition
| Category | ParamEdit | ConSolve | P-Range | FEA-σ | Para σ | Seed σ | Brier | EditLat |
|---|---|---|---|---|---|---|---|---|
| Constraint Solving & Editability | 0.796 | 0.842 | 0.757 | — | — | — | — | — |
| Reverse Engineering | — | — | — | — | — | — | — | — |
| 2-D Sketch Constraints | — | 0.753 | — | — | — | — | — | — |
| Functional Intent · FEA-Gated | — | — | — | 0.530 | — | — | — | — |
| Paraphrase Robustness | — | — | — | — | 0.021 | 0.039 | — | — |
| Confidence Calibration | — | — | — | — | — | — | 0.105 | — |
TASK-BY-TASK ARTIFACTS
PRIM-001Hollow cylinder (60 × 40 × 100)0.7550.00PRIM-007Right hexagonal prism with pitch fillet0.6520.00BOOL-003Coplanar-face union (knife-edge stress)0.5920.00BOOL-009High-genus subtraction (lattice block)0.5280.00BREP-004NURBS-handle goblet (G2 swept loft)0.5030.00SURF-002Compressor blade (NACA 65-(12)10)0.3160.00MECH-014L-bracket with M6 + slotted hole0.6060.00MECH-022Stepped shaft with retaining-ring groove0.5100.00MECH-027Planetary-gear carrier plate0.6070.00ASM-005Pin-in-hole, H7/g6 sliding fit0.6010.00ASM-011Dovetail slide (60° flanks)0.5750.00STD-002ISO 4762 M8×30 socket-head cap screw0.5780.00SHEET-003U-channel bracket, 1.5 mm Al, k=0.400.6140.00SEAL-001AS568-214 face-seal groove0.7611.00KIN-002Four-bar linkage, Grashof crank-rocker0.5910.00DFMCNC-0023-ax-machinable manifold block0.5940.00DFMMOLD-002Injection-mouldable enclosure half0.4290.00DFMFDM-008FDM-printable hinge (no support)0.6070.00CAM-0015-pocket plate, Ø3 endmill finish0.5050.00PARAM-006Editable flange (bolt circle param sweep)0.8351.00PARAM-013Editable bracket (length+30 %, hole→M8)0.6570.00REVENG-002Three-view ortho → bracket0.6760.00REVENG-009Three-view ortho → housing with cores0.5180.00SKETCH-003Tangent-arc transition profile0.7611.00FUNC-001Cantilever bracket — 250 N tip load, 6061-T6, SF≥40.3830.00FUNC-007Heat-sink fin array for 25 W TO-2200.4780.00PARA-0015× paraphrased L-bracket0.7050.00CAL-003Confidence-calibrated planetary carrier0.4280.00