Linj router exp#61
Conversation
WalkthroughAdds an LLM-judged escalation router with strong-tier session latching, YAML and Python configuration, CLI dispatch, tests, documentation, benchmark profiles, run comparison tooling, and expanded model pricing. ChangesEscalation Router
Estimated code review effort: 5 (Critical) | ~120 minutes Poem
🚥 Pre-merge checks | ✅ 3 | ❌ 2❌ Failed checks (1 warning, 1 inconclusive)
✅ Passed checks (3 passed)
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 10
🧹 Nitpick comments (1)
docs/routing_algorithms/escalation_router_routing.md (1)
64-75: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winConfig table omits several valid
judge.*keys andenable_stats.Per
_ESCALATION_JUDGE_KEYSinswitchyard/cli/route_bundle.pyandEscalationRouterConfig,judge.confirmations,judge.confirmation_window,judge.disable_reasoning,judge.window_message_chars, and top-levelenable_statsare all accepted config keys with documented defaults in the config docstring, but none appear in this table. Users tuning confirmation/latch behavior have no doc reference for these.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/routing_algorithms/escalation_router_routing.md` around lines 64 - 75, Update the configuration table to document the accepted keys judge.confirmations, judge.confirmation_window, judge.disable_reasoning, judge.window_message_chars, and top-level enable_stats. Use the documented defaults and meanings from EscalationRouterConfig and its config docstring, keeping the existing table format and entries unchanged.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@benchmark/compare_runs.py`:
- Around line 72-73: Update the trial-count extraction in the run comparison
rows to safely handle missing harbor-level n_completed_trials and
n_errored_trials fields. Reuse the available stats.evals.*.n_trials data to
derive a sensible completed-trials count when the aggregate completed field is
absent, while preserving existing values when present and avoiding
direct-subscript KeyErrors.
- Around line 25-28: Update the result-loading logic around results in the
benchmark comparison flow so it does not silently use only results[0] when
multiple job result files exist. Either aggregate all matched result.json files
into the comparison input or explicitly raise an error when more than one file
is present, while preserving the existing missing-results failure behavior.
In `@benchmark/routing-profiles/tb-lite-escalation-opus-deepseek-gemini.yaml`:
- Around line 12-29: Add pricing-table aliases for the exact model IDs
anthropic/claude-opus-4.7 and google/gemini-3.5-flash, mapping each to its
corresponding existing cost entry so strong and judge usage is reported instead
of unknown. Leave the profile configuration unchanged.
In `@docs/routing_algorithms/escalation_router_routing.md`:
- Line 73: Update the session_key_depth link to use the exact generated anchor
for the “Repeated-trial benchmarking (k>1)” heading, verifying the slug with
MkDocs or rendered HTML before finalizing the fragment.
In `@switchyard/lib/processors/escalation_judge_request_processor.py`:
- Around line 738-745: Update the request assembly and cap enforcement around
_assemble so exceeding config.max_request_chars never uses a head-only slice
that can remove the task or newest evidence. Re-budget the available length by
trimming system/developer anchors first, then remove the oldest window lines as
needed, while preserving the first-user task and latest evidence in the final
assembled text.
- Around line 280-289: Update the dump_verdicts_to_stderr configuration to be
opt-in by default, and ensure the escalation verdict stderr payload excludes or
explicitly redacts user-derived fields such as task_hint, judge-generated
reason, and fail-open exception details. Preserve dumping only when callers
explicitly enable the setting.
- Around line 479-488: After the successful await of _affinity.pin in the
confirmed branch of the escalation processor, remove the corresponding session
key from _escalate_streaks before updating routing metadata and returning.
Preserve the existing behavior for unconfirmed verdicts and only clear the
streak after pin succeeds.
In `@switchyard/lib/profiles/escalation_router_config.py`:
- Line 84: Update the judge configuration flow in build() to use judge_timeout_s
as the fallback whenever judge_target.endpoint.timeout_secs is unset, while
preserving the explicit endpoint timeout when provided. Add or update a test
covering both precedence cases.
- Line 90: Update EscalationRouterProfileConfig.build() to consume enable_stats
and thread its value through profile composition so the documented stats
processors and tier wrappers are enabled when true and omitted when false. Add
coverage for both enable_stats values, or remove the option if it cannot be
wired to those behaviors.
In `@switchyard/lib/profiles/escalation_router_profile_config.py`:
- Around line 126-130: Update the ComponentChainProfile construction in the
escalation router profile configuration to translate the validated
fallback_target_on_evict ID into its corresponding deterministic tier label,
"strong" or "weak", before passing it as fallback_target_on_evict. Preserve the
existing behavior for standard IDs and add coverage for custom strong.id and
weak.id values.
---
Nitpick comments:
In `@docs/routing_algorithms/escalation_router_routing.md`:
- Around line 64-75: Update the configuration table to document the accepted
keys judge.confirmations, judge.confirmation_window, judge.disable_reasoning,
judge.window_message_chars, and top-level enable_stats. Use the documented
defaults and meanings from EscalationRouterConfig and its config docstring,
keeping the existing table format and entries unchanged.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 22a2f7fd-102a-4b09-9fa8-179ab4009a53
📒 Files selected for processing (21)
.agents/skills/switchyard-lib-core/SKILL.mdbenchmark/compare_runs.pybenchmark/routing-profiles/tb-lite-escalation-opus-deepseek-gemini.yamlbenchmark/routing-profiles/tb-lite-escalation-opus-deepseek-nvidia.yamlbenchmark/routing-profiles/tb-lite-single-deepseek-v4-pro-nvidia.yamlbenchmark/routing-profiles/tb-lite-single-opus-4-7-nvidia.yamlbenchmark/routing-profiles/tb-lite-single-opus-4-8-nvidia.yamlcrates/switchyard-components/src/stats/cost.rsdocs/routing_algorithms/escalation_router_routing.mddocs/routing_algorithms/overview.mdmkdocs.ymlswitchyard/__init__.pyswitchyard/cli/route_bundle.pyswitchyard/lib/cost_estimator.pyswitchyard/lib/processors/escalation_judge_request_processor.pyswitchyard/lib/profiles/__init__.pyswitchyard/lib/profiles/escalation_router_config.pyswitchyard/lib/profiles/escalation_router_profile_config.pytests/test_escalation_judge_request_processor.pytests/test_escalation_router_profile.pytests/test_route_bundle.py
| results = sorted(run_dir.glob("jobs/*/result.json")) | ||
| if not results: | ||
| raise FileNotFoundError(f"no jobs/*/result.json under {run_dir}") | ||
| harbor = json.loads(results[0].read_text()) |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | 🏗️ Heavy lift
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
fd -a '^result\.json$' benchmark -x sh -c '
case "$1" in
*/jobs/*/result.json) printf "%s\n" "$1" ;;
esac
' sh {} | head -100
rg -n -C3 'jobs/\*/result\.json|n_completed_trials|reward_stats' benchmark testsRepository: NVIDIA-NeMo/Switchyard
Length of output: 1966
Do not ignore additional job results in benchmark/compare_runs.py:25-28. The glob can match multiple jobs/*/result.json files, but only results[0] is read, so multi-job runs will report a single lexicographically first job instead of the run as a whole. Aggregate the results or fail fast when more than one file is present.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@benchmark/compare_runs.py` around lines 25 - 28, Update the result-loading
logic around results in the benchmark comparison flow so it does not silently
use only results[0] when multiple job result files exist. Either aggregate all
matched result.json files into the comparison input or explicitly raise an error
when more than one file is present, while preserving the existing
missing-results failure behavior.
| row("completed trials", [str(r["harbor"]["n_completed_trials"]) for r in runs.values()]) | ||
| row("errored trials", [str(r["harbor"]["n_errored_trials"]) for r in runs.values()]) |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟠 Major | ⚡ Quick win
Handle Harbor results without run-level trial counters.
The baseline fixture in tests/test_run_baseline_script.py contains stats.evals.*.n_trials but no stats.n_completed_trials or stats.n_errored_trials; these direct subscripts therefore crash the utility. Use safe reads and derive a sensible completed count when the aggregate fields are absent.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@benchmark/compare_runs.py` around lines 72 - 73, Update the trial-count
extraction in the run comparison rows to safely handle missing harbor-level
n_completed_trials and n_errored_trials fields. Reuse the available
stats.evals.*.n_trials data to derive a sensible completed-trials count when the
aggregate completed field is absent, while preserving existing values when
present and avoiding direct-subscript KeyErrors.
| model: anthropic/claude-opus-4.7 | ||
| api_key: ${OPENROUTER_API_KEY} | ||
| base_url: https://openrouter.ai/api/v1 | ||
| format: openai | ||
| timeout_secs: 600 | ||
| extra_body: | ||
| reasoning: | ||
| effort: high | ||
| weak: | ||
| model: deepseek/deepseek-v4-pro | ||
| api_key: ${OPENROUTER_API_KEY} | ||
| base_url: https://openrouter.ai/api/v1 | ||
| format: openai | ||
| timeout_secs: 600 | ||
| judge: | ||
| model: google/gemini-3.5-flash | ||
| api_key: ${OPENROUTER_API_KEY} | ||
| base_url: https://openrouter.ai/api/v1 |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '\n== profile file ==\n'
git ls-files 'benchmark/routing-profiles/tb-lite-escalation-opus-deepseek-gemini.yaml' && \
sed -n '1,120p' benchmark/routing-profiles/tb-lite-escalation-opus-deepseek-gemini.yaml
printf '\n== search pricing/lookup references ==\n'
rg -n "claude-opus-4\.7|gemini-3\.5-flash|enable_stats|pricing|alias|unknown IDs|zero" -S .
printf '\n== locate likely pricing files ==\n'
git ls-files | rg 'pricing|cost|model.*price|models?.*yaml|models?.*json|bench'Repository: NVIDIA-NeMo/Switchyard
Length of output: 50378
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '\n== cost table around gemini/opus mappings ==\n'
sed -n '1,340p' crates/switchyard-components/src/stats/cost.rs
printf '\n== tests around unknown/default behavior ==\n'
sed -n '1,120p' tests/test_cost_estimator_gemini.py 2>/dev/null || true
sed -n '1,360p' crates/switchyard-components/src/stats/cost.rs | sed -n '240,320p'Repository: NVIDIA-NeMo/Switchyard
Length of output: 12882
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '\n== cost table around gemini/opus mappings ==\n'
sed -n '1,340p' crates/switchyard-components/src/stats/cost.rs
printf '\n== tests around unknown/default behavior ==\n'
sed -n '1,120p' tests/test_cost_estimator_gemini.py 2>/dev/null || trueRepository: NVIDIA-NeMo/Switchyard
Length of output: 11870
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '\n== opus/gemini pricing entries ==\n'
rg -n -A4 -B4 'claude-opus-4\.7|gemini-3\.5-flash|unknown_model_defaults_to_zero|ModelPrice' crates/switchyard-components/src/stats/cost.rs
printf '\n== any other pricing tables or normalization helpers ==\n'
rg -n 'claude-opus-4\.7|gemini-3\.5-flash|normalize.*model|pricing.*alias|model.*alias' crates switchyard tests -SRepository: NVIDIA-NeMo/Switchyard
Length of output: 12339
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '\n== python cost estimator pricing table ==\n'
sed -n '1,260p' switchyard/lib/cost_estimator.py
printf '\n== live stats model normalization ==\n'
sed -n '1,140p' switchyard/lib/live_stats_collector.py
printf '\n== exact references around claude/gemini pricing in python ==\n'
rg -n -A3 -B3 'claude-opus-4\.7|gemini-3\.5-flash|MODEL_PRICING|_PRICE_PER_1M|normalize_model_name' switchyard/lib switchyard/tests tests -SRepository: NVIDIA-NeMo/Switchyard
Length of output: 35836
🏁 Script executed:
#!/bin/bash
set -euo pipefail
rg -n -A6 -B6 'MODEL_PRICING|claude-opus-4\.7|gemini-3\.5-flash|_normalize_model_name|_PRICE_PER_1M' switchyard/lib/cost_estimator.py switchyard/lib/live_stats_collector.pyRepository: NVIDIA-NeMo/Switchyard
Length of output: 11983
🏁 Script executed:
#!/bin/bash
set -euo pipefail
sed -n '1,240p' switchyard/lib/cost_estimator.pyRepository: NVIDIA-NeMo/Switchyard
Length of output: 11369
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '\n== cost estimator call sites ==\n'
rg -n -A4 -B4 'estimate_cost\(|MODEL_PRICING|estimated_cost_usd|to_dict\(\)|routing/stats' switchyard crates tests -S
printf '\n== model name normalization in stats accumulation ==\n'
rg -n -A4 -B4 '_normalize_model_name|estimated_cost_usd|record\(|snapshot\(|to_dict\(' switchyard/lib/live_stats_collector.py switchyard/lib/stats* crates/switchyard-components/src/stats -S
printf '\n== route/profile model names used for stats ==\n'
rg -n -A3 -B3 'model: anthropic/claude-opus-4\.7|model: google/gemini-3\.5-flash|anthropic/claude-opus-4\.7|google/gemini-3\.5-flash' benchmark switchyard crates tests -SRepository: NVIDIA-NeMo/Switchyard
Length of output: 50378
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '\n== search for model-id rewrite/normalization rules ==\n'
rg -n -A4 -B4 'claude-opus-4\.7|claude-opus-4-7|google/gemini-3\.5-flash|gcp/google/gemini-3\.5-flash|replace\(|normalize.*model|alias.*model' switchyard crates tests docs -S
printf '\n== route/stats serialization path ==\n'
rg -n -A4 -B4 'routing/stats|estimated_cost_usd|estimated_cost|ModelStats|model_id|model_name' crates/switchyard-components/src switchyard/lib switchyard/server -SRepository: NVIDIA-NeMo/Switchyard
Length of output: 50379
🏁 Script executed:
#!/bin/bash
set -euo pipefail
sed -n '140,320p' switchyard/lib/live_stats_collector.pyRepository: NVIDIA-NeMo/Switchyard
Length of output: 5875
Add pricing aliases for the exact model IDs used here. anthropic/claude-opus-4.7 and google/gemini-3.5-flash don’t match the current cost-table keys, so this profile’s strong/judge spend will show up as unknown and underreported in routing stats.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@benchmark/routing-profiles/tb-lite-escalation-opus-deepseek-gemini.yaml`
around lines 12 - 29, Add pricing-table aliases for the exact model IDs
anthropic/claude-opus-4.7 and google/gemini-3.5-flash, mapping each to its
corresponding existing cost entry so strong and judge usage is reported instead
of unknown. Leave the profile configuration unchanged.
| | `judge.prompt` | built-in | Judge system-prompt override. | | ||
| | `judge.max_request_chars` | `12000` | Cap on the judge transcript; oldest window messages are dropped first. | | ||
| | `fallback_target_on_evict` | required | `strong` or `weak`; context-window-eviction reroute target. | | ||
| | `session_key_depth` | `0` | See [Repeated-trial benchmarking](#repeated-trial-benchmarking) below. | |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
Broken link fragment to “Repeated-trial benchmarking (k>1)”.
MkDocs slugifies headings by converting disallowed characters (including (, ), >) to dashes and collapsing consecutive dashes, so this heading's actual anchor is not #repeated-trial-benchmarking. The link will 404/fail to scroll to the section.
📝 Proposed fix
-| `session_key_depth` | `0` | See [Repeated-trial benchmarking](`#repeated-trial-benchmarking`) below. |
+| `session_key_depth` | `0` | See [Repeated-trial benchmarking](`#repeated-trial-benchmarking-k1`) below. |Verify the exact generated anchor (e.g. via mkdocs build --strict or inspecting rendered HTML) since punctuation slugification can vary.
📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| | `session_key_depth` | `0` | See [Repeated-trial benchmarking](#repeated-trial-benchmarking) below. | | |
| | `session_key_depth` | `0` | See [Repeated-trial benchmarking](`#repeated-trial-benchmarking-k1`) below. | |
🧰 Tools
🪛 markdownlint-cli2 (0.23.0)
[warning] 73-73: Link fragments should be valid
(MD051, link-fragments)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/routing_algorithms/escalation_router_routing.md` at line 73, Update the
session_key_depth link to use the exact generated anchor for the “Repeated-trial
benchmarking (k>1)” heading, verifying the slug with MkDocs or rendered HTML
before finalizing the fragment.
Source: Linters/SAST tools
| dump_verdicts_to_stderr: bool = True | ||
| """Emit one ``escalation_verdict={...}`` JSON line to ``sys.stderr`` per | ||
| judge call (verdict or fail-open). | ||
|
|
||
| Written directly (not via the logging module) so it lands in the | ||
| benchmark server's captured log regardless of uvicorn's logger config — | ||
| mirrors :attr:`LLMClassifierConfig.dump_signals_to_stderr`. Grep | ||
| ``escalation_verdict=`` to reconstruct per-turn judge decisions and | ||
| escalation timing for a run. Disable for callers that share stderr | ||
| with an interactive TUI.""" |
There was a problem hiding this comment.
🔒 Security & Privacy | 🟠 Major | ⚡ Quick win
Do not log user-derived verdict data by default.
The default stderr dump includes task_hint, judge-generated reason, and fail-open exception details. These can expose prompts, secrets, or provider data in benchmark/server logs. Make dumping opt-in and omit or explicitly redact user-derived fields.
Also applies to: 548-556
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@switchyard/lib/processors/escalation_judge_request_processor.py` around lines
280 - 289, Update the dump_verdicts_to_stderr configuration to be opt-in by
default, and ensure the escalation verdict stderr payload excludes or explicitly
redacts user-derived fields such as task_hint, judge-generated reason, and
fail-open exception details. Preserve dumping only when callers explicitly
enable the setting.
| if confirmed: | ||
| log.info( | ||
| "EscalationJudgeRequestProcessor: escalating to strong tier " | ||
| "(turn %d): %s", | ||
| turn, | ||
| verdict.reason, | ||
| ) | ||
| await self._affinity.pin(ctx, request, TIER_STRONG) | ||
| ctx.metadata[CTX_DETERMINISTIC_ROUTING_TIER] = TIER_STRONG | ||
| return request |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
Clear the confirmation streak after successfully latching.
The confirmed streak remains in _escalate_streaks. If the bounded affinity pin is later evicted, the same session key can resume with an already-confirmed count and immediately re-escalate. Remove the entry after pin() succeeds.
Proposed fix
if confirmed:
await self._affinity.pin(ctx, request, TIER_STRONG)
+ self._escalate_streaks.pop(self._streak_key(ctx, request), None)
ctx.metadata[CTX_DETERMINISTIC_ROUTING_TIER] = TIER_STRONG📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| if confirmed: | |
| log.info( | |
| "EscalationJudgeRequestProcessor: escalating to strong tier " | |
| "(turn %d): %s", | |
| turn, | |
| verdict.reason, | |
| ) | |
| await self._affinity.pin(ctx, request, TIER_STRONG) | |
| ctx.metadata[CTX_DETERMINISTIC_ROUTING_TIER] = TIER_STRONG | |
| return request | |
| if confirmed: | |
| log.info( | |
| "EscalationJudgeRequestProcessor: escalating to strong tier " | |
| "(turn %d): %s", | |
| turn, | |
| verdict.reason, | |
| ) | |
| await self._affinity.pin(ctx, request, TIER_STRONG) | |
| self._escalate_streaks.pop(self._streak_key(ctx, request), None) | |
| ctx.metadata[CTX_DETERMINISTIC_ROUTING_TIER] = TIER_STRONG | |
| return request |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@switchyard/lib/processors/escalation_judge_request_processor.py` around lines
479 - 488, After the successful await of _affinity.pin in the confirmed branch
of the escalation processor, remove the corresponding session key from
_escalate_streaks before updating routing metadata and returning. Preserve the
existing behavior for unconfirmed verdicts and only clear the streak after pin
succeeds.
| text = _assemble() | ||
| # Drop oldest window lines first: for a trajectory judge the newest | ||
| # evidence is strictly the most valuable. | ||
| while len(text) > config.max_request_chars and window_lines: | ||
| window_lines.pop(0) | ||
| text = _assemble() | ||
| if len(text) > config.max_request_chars: | ||
| text = text[: config.max_request_chars - 15] + "...<truncated>" |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
Preserve the task and newest evidence when enforcing the global cap.
After dropping window lines, the final head-only slice can discard the first-user task and all recent evidence when system/developer anchors exceed the cap. Re-budget or trim system anchors first instead of truncating the assembled transcript from the end.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@switchyard/lib/processors/escalation_judge_request_processor.py` around lines
738 - 745, Update the request assembly and cap enforcement around _assemble so
exceeding config.max_request_chars never uses a head-only slice that can remove
the task or newest evidence. Re-budget the available length by trimming
system/developer anchors first, then remove the oldest window lines as needed,
while preserving the first-user task and latest evidence in the final assembled
text.
| judge_window_message_chars: int = Field(default=300, ge=50) | ||
| judge_max_request_chars: int = Field(default=12_000, ge=1_000) | ||
| judge_system_prompt: str | None = Field(default=None, min_length=1) | ||
| judge_timeout_s: float = Field(default=5.0, gt=0.0) |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟠 Major | ⚡ Quick win
Wire judge_timeout_s into the judge configuration.
This setting is never read: build() uses judge_target.endpoint.timeout_secs or 5.0, so a configured judge_timeout_s is ignored. Use it as the default when the judge target has no explicit timeout, and cover that precedence in a test.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@switchyard/lib/profiles/escalation_router_config.py` at line 84, Update the
judge configuration flow in build() to use judge_timeout_s as the fallback
whenever judge_target.endpoint.timeout_secs is unset, while preserving the
explicit endpoint timeout when provided. Add or update a test covering both
precedence cases.
| default=DEFAULT_DETERMINISTIC_TIER_TIMEOUT_S, | ||
| gt=0.0, | ||
| ) | ||
| enable_stats: bool = True |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | 🏗️ Heavy lift
Implement or remove the inert enable_stats option.
EscalationRouterProfileConfig.build() never reads this field, so it cannot enable or disable the documented stats processors and tier wrappers. Thread it through the profile composition and test both values.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@switchyard/lib/profiles/escalation_router_config.py` at line 90, Update
EscalationRouterProfileConfig.build() to consume enable_stats and thread its
value through profile composition so the documented stats processors and tier
wrappers are enabled when true and omitted when false. Add coverage for both
enable_stats values, or remove the option if it cannot be wired to those
behaviors.
| return ComponentChainProfile( | ||
| request_processors=request_processors, | ||
| backend=backend, | ||
| fallback_target_on_evict=config.fallback_target_on_evict, | ||
| ) |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
Translate fallback target IDs to deterministic tier labels.
The config accepts strong.id / weak.id, but the backend recognizes only "strong" / "weak". With custom IDs, eviction writes an unrecognized ctx.selected_target, then routing falls back to stale metadata or weak instead of the requested fallback. Map the validated target ID to its tier label before passing it here, and test custom IDs.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@switchyard/lib/profiles/escalation_router_profile_config.py` around lines 126
- 130, Update the ComponentChainProfile construction in the escalation router
profile configuration to translate the validated fallback_target_on_evict ID
into its corresponding deterministic tier label, "strong" or "weak", before
passing it as fallback_target_on_evict. Preserve the existing behavior for
standard IDs and add coverage for custom strong.id and weak.id values.
…nd bar, varied-exploration carve-out
1 participant
What
Short summary of what this PR changes.
Why
The motivation — what problem does this solve, or which ticket does it close?
Closes #
How tested
uv run ruff check .cleanuv run mypy switchyardcleanuv run pytest tests/greenChecklist
snake_caseof the primary class.switchyard/__init__.py.__all__if intended for downstream use.--helpupdated if customer-facing surface changed.Signed-off-by: Your Name <email>) per the DCO.Notes for reviewers
Anything reviewers should pay extra attention to — risky paths, follow-up tickets, intentional trade-offs.
Summary by CodeRabbit
New Features
Documentation
Bug Fixes