feat: add Relay routing Decision API#8
Open
bbednarski9 wants to merge 4 commits into
Open
Conversation
Signed-off-by: Bryan Bednarski <bbednarski@nvidia.com>
This was referenced Jul 1, 2026
Signed-off-by: Bryan Bednarski <bbednarski@nvidia.com>
Signed-off-by: Bryan Bednarski <bbednarski@nvidia.com>
Signed-off-by: Bryan Bednarski <bbednarski@nvidia.com>
sabhatinas
reviewed
Jul 1, 2026
|
|
||
| - `run()` is the authoritative serving path. It owns the complete request lifecycle and is the | ||
| method a Switchyard server should call. | ||
| - `decide()` is the decision-only path. Supporting profiles reuse their request-side policy but |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Summary
Adds the first, independently functional layer of the NeMo Relay integration:
POST /v1/routing/decisionwith distinct malformed, unknown-profile, and unsupported-profile errors;backend_id.This is the Decision API slice of a previous end-to-end implementation, split into a focused public-repository change.
Why
Relay needs Switchyard to select a configured target without dispatching that target itself. Keeping this contract and the two existing v2 routing profiles in a focused first PR makes the wire API, profile capability boundary, and server error behavior reviewable before ATOF state or Cascade routing are introduced.
How tested
Testing summary
cargo fmt --all --checkcargo clippy --workspace --all-targets -- -D warningscargo test -p switchyard-components-v2(46 unit tests plus profile integration suites)cargo test -p switchyard-server(21 server tests plus migration coverage)cargo test --workspace --exclude switchyard-pycargo check -p switchyard-pyuv run ruff check .— not applicable; no Python source changed.uv run mypy switchyard— not applicable; no Python source changed.uv run pytest tests/— not applicable; the affected API is covered by Rust component/server tests.Checklist
snake_caseof the primary class. (N/A: Rust API change.)switchyard/__init__.py.__all__if intended for downstream use. (N/A: symbols are exported from the Rust crate root.)--helpupdated if customer-facing surface changed. (DESIGN.mddocuments the runtime contract.)Signed-off-by: Your Name <email>) per the DCO.Notes for reviewers
Review guidelines
Suggested review order:
decision.rsfor the wire contract, protocol/materialization validation, and response conversion.profile.rsandregistry.rsfor the single-runtime capability path.The legacy Relay payload still includes
decision_profile.router. Serde intentionally ignores that additive field;profile_idis authoritative. The response still includesrouterand all five route fields required by Relay.Previous feedback addressed
DecisionRuntimeRegistrydesign withProfile::decide()on the sameArc<dyn Profile>already used for serving; configured runtimes are built once.DecisionProfile.routerwhile retaining legacy wire compatibility for the extra field.400even when the profile ID is unknown..expect()usage with typed error propagation.400), unknown profile (404), and unsupported profile (422) failures distinct.ATOF ingestion and Relay snapshots are intentionally deferred to #9; Relay-aware Cascade routing and unified session metadata are deferred to #11.
Follow-up fixes