Build / holdout trace / Hard
Diagnose a Next.js static build failure caused by optional content fields and make the renderer type-safe without broad rewrites.
Expected evidence
Scoring focus
Common failure mode
Weak coding agents silence TypeScript with broad casts instead of fixing the optional data shape.
Expected output
A narrow fix that handles optional content safely and proves the static build succeeds.
Score breakdown
Trace provenance
Score calculation ledger
Score equals the sum of weighted rubric components. Component earned points are allocated from the aggregate run score.
Model version
frontier-reasoning-eval-holdout-2026-05
Run seed
2026051630
Prompt packet
repair-failing-static-build-holdout-packet-v0.1
Artifact bundle
Replay scaffold generated from the current seed trace. Replace with real harness exports when model runs are available.
Replay command
pnpm benchmarks:replay --suite coding-agent-maintenance-suite --task repair-failing-static-build
This command is intentionally documented before the real harness exists so the artifact contract is visible.
Payload preview
Split
holdout
Difficulty
Hard
Evidence fields
3
Model runs
4
Screenshot
Pending real browser or app screenshot artifact.
Model run evidence
This is the inspection layer that keeps benchmark scores honest: each model class gets an outcome, cost proxy, latency, and reviewer note.
Frontier API provider
Accepted
Identified optional field issue and added a narrow fallback with clean build proof.
Answer excerpt
Added a null-safe fallback for optional research notes and reran the Webpack build.
Failure reason
No major issue.
Fast hosted API provider
Accepted with review
Fixed build but used a wider type assertion than necessary.
Answer excerpt
Used a type assertion to bypass the optional value issue.
Failure reason
Build passed but type assertion was wider than necessary.
Self-hosted/open-weight stack
Partial
Changed data shape but left one route unverified.
Answer excerpt
Changed the data object so every article has the same field.
Failure reason
Changed data shape globally and left one route unverified.
Low-cost routing endpoint
Rejected
Misdiagnosed the issue as dependency mismatch.
Answer excerpt
This seems like a dependency mismatch.
Failure reason
Misdiagnosed the issue as dependency mismatch.
Aggregate scores are useful only when reviewers can inspect the task packet, expected evidence, and the exact failure mode. This page is the pattern for publishing public samples while keeping harder holdout tasks private.
Return to suite report