Smart Suggestions: Three Next-Actions, Not Eleven Dashboards

Winnow ships eleven eval capabilities — judge calibration drift, context health, rubric editor, scaffold history, agent rollup, harness pill, and the rest. Each one surfaces its own state. Stitched together, you get a suite-home page that looks like an air traffic control panel: every light is true, every light is loud, and the user has no obvious next step.

The Smart Suggestions strip is the prioritization layer. It lives at the top of the Eval Builder suite home, between the back link and the suite info card. It renders up to three rows. Each row is one question with one or two one-click answers. When no rule fires, the strip renders nothing — zero height, no placeholder, no "you're all caught up" copy.

What lights up today

One rule reads live data:

Judge calibration baseline staleness. When any of a suite's judge criteria has a calibration baseline older than three days (read directly from GET /scoring/calibration-baselines/{suite_id}/{dimension}), a row appears:

Judge redact-pii baseline is 5 days old. Recalibrate or fold into rubric?

The Recalibrate action lands on the Experiment Health tab where the user can paste a fresh labelled sample and re-run the drift check. The Fold into rubric action returns to the Eval Builder so the user can promote the criterion into a single-call rubric judge.

What is registered but invisible

Two more rules ship registered in the rule registry but return null until their backends land:

• Task regression ("Task Y has failed 6 of 8 recent runs — demote to frontier or fix?") needs the typed task_set column on datasets and the per-task recent-run rollup — both follow-ups from the S5 lane.
• Scaffold alias stale ("Scaffold alias production hasn't moved in 14 days — review pending experiments?") needs the scaffold alias-move audit log endpoint — a follow-up from the B5 lane.

The stub rules carry explicit TODO(P2) deferral comments in frontend/src/lib/smart-suggestions.ts. The reverse shape — registering placeholder UI rows for unbuilt backends — is the empty-state anti-pattern we're trying to avoid. When the S5 and B5 backends ship, the rules flip from return null to a real evaluate body and the strip starts surfacing them. No frontend slice required.

Why the strip is contextual

Dismissing a row writes smart-suggestions-dismissed:{suite_id}:{rule_id}=1 to browser localStorage. Per-suite, per-rule. Dismissing "drift" on suite A doesn't silence "drift" on suite B, or silence "regression" on suite A. The dismissal is not synced across users or devices — a team-wide dismissal setting would be a heavyweight surface for what is meant as a quiet, contextual nudge. If a user wants to re-enable a dismissed rule, they clear the matching localStorage key in browser DevTools.

The rule registry shape

Each rule is a small typed object:

export type SuggestionRule = {
  id: string
  evaluate: (ctx: SuggestionContext) => Promise<Suggestion | null>
}

The orchestrator (evaluateRules) runs every rule with Promise.allSettled — one rule throwing does not silence the others. The first three non-null returns render; the rest are dropped. Future rules add a row to DEFAULT_RULES. The renderer never changes.

Honesty about routing

The mockup's "Recalibrate" CTA targets the Scorecard Dimensions sub-tab where A5's drift pills live. The Scorecard component is internally tabbed with no query-param plumbing, and the file is a high-activity shared component owned by other lanes — adding the deep link belongs in its own slice. Today the action routes to the closest live destination (/experiments/{suite_id}?tab=health). The deep-link follow-up is tracked in docs/UNBUILT-FEATURES.md under the P6 deep-link entry for smart suggestions to Scorecard Dimensions.

The strip is meant to disappear when there's nothing useful to say. Most days, on a healthy suite, it does.