Frontend / public trace / Medium
Given a Next.js app where Command-K opens but result links do not navigate, patch the client component and verify keyboard and click behavior.
Expected evidence
Scoring focus
Common failure mode
Agents often fix click navigation but forget keyboard shortcut or leave focus trapped after navigation.
Expected output
A scoped patch that preserves keyboard open behavior, makes result navigation work, closes the palette after navigation, and verifies the flow with Playwright.
Score breakdown
Trace provenance
Score calculation ledger
Score equals the sum of weighted rubric components. Component earned points are allocated from the aggregate run score.
Model version
frontier-reasoning-eval-public-2026-05
Run seed
2026051620
Prompt packet
fix-command-k-search-regression-public-packet-v0.1
Artifact bundle
Replay scaffold generated from the current seed trace. Replace with real harness exports when model runs are available.
Replay command
pnpm benchmarks:replay --suite coding-agent-maintenance-suite --task fix-command-k-search-regression
This command is intentionally documented before the real harness exists so the artifact contract is visible.
Payload preview
Split
public
Difficulty
Medium
Evidence fields
3
Model runs
4
Screenshot
Pending real browser or app screenshot artifact.
Model run evidence
This is the inspection layer that keeps benchmark scores honest: each model class gets an outcome, cost proxy, latency, and reviewer note.
Frontier API provider
Accepted
Patched navigation, preserved focus behavior, and verified with Playwright.
Answer excerpt
Updated the result link path and verified Cmd-K, click navigation, URL change, and absence of console warnings.
Failure reason
No major issue.
Fast hosted API provider
Accepted with review
Fixed click path but needed reviewer prompt to add keyboard coverage.
Answer excerpt
Fixed result click navigation and confirmed the target route loaded.
Failure reason
Initial patch skipped keyboard coverage.
Self-hosted/open-weight stack
Partial
Found the component but changed unrelated styles and skipped verification.
Answer excerpt
Changed the search component and adjusted palette styles.
Failure reason
Unrelated style churn and no browser verification.
Low-cost routing endpoint
Rejected
Suggested a fix without reading the component boundary.
Answer excerpt
Try replacing the search button with a link.
Failure reason
Suggested a fix without reading the component boundary.
Aggregate scores are useful only when reviewers can inspect the task packet, expected evidence, and the exact failure mode. This page is the pattern for publishing public samples while keeping harder holdout tasks private.
Return to suite report