FE-1162: Python optimisation script (Optuna) driving the Petrinaut CLI#8989
FE-1162: Python optimisation script (Optuna) driving the Petrinaut CLI#8989YannisZa wants to merge 19 commits into
Conversation
…n a petrinaut specification
…ut-cli params and init_states based on optimisation specification
PR SummaryMedium Risk Overview
Reviewed by Cursor Bugbot for commit b3ace77. Bugbot is set up for automated code reviews on this repo. Configure here. |
|
The latest updates on your projects. Learn more about Vercel for GitHub.
1 Skipped Deployment
|
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
github-actions
Bot
added
area/infra
|
|
| break | ||
| finally: | ||
| stop_flag.set() | ||
| self.lock.release() |
There was a problem hiding this comment.
Disconnect releases lock early
High Severity
When an SSE client disconnects, stream_all / stream_best exit their receive loop and the finally block releases the per-session lock while the daemon thread may still be inside study.optimize and PetrinautModel.run. A second stream on the same session_id can then start another optimization on the same Study and share one socket stream, causing overlapping Optuna runs and interleaved socket I/O.
Additional Locations (1)
Reviewed by Cursor Bugbot for commit b5e10c1. Configure here.
| async def teardown(session_id: str): | ||
| if app.state.sessions.pop(session_id, None) is None: | ||
| raise HTTPException(404, "unknown session_id") | ||
| return {"status": "deleted"} |
There was a problem hiding this comment.
DELETE ignores running optimization
Medium Severity
DELETE /optimize/{session_id} only removes the session from app.state.sessions and does not signal the optimizer to stop or wait for its background thread. An in-flight stream or Optuna worker keeps using the PetrinautModel socket until trials finish.
Reviewed by Cursor Bugbot for commit b5e10c1. Configure here.
YannisZa
changed the title
petrinaut-cli parameter and initial state optimisationpetrinaut-cli parameter and initial state optimisation
YannisZa
changed the title
petrinaut-cli parameter and initial state optimisation
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 2 potential issues.
There are 4 total unresolved issues (including 2 from previous reviews).
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit b3ace77. Configure here.
| try: | ||
| self.study.optimize( | ||
| self.objective, n_trials=n_trials, callbacks=[callback] | ||
| ) |
There was a problem hiding this comment.
Same session adds more trials
Medium Severity
Each call to the SSE stream endpoints invokes study.optimize on the single Optuna study created at /init. A second stream on the same session_id runs another full n_trials batch and appends to the existing study instead of starting a fresh run of the configured trial count.
Additional Locations (1)
Reviewed by Cursor Bugbot for commit b3ace77. Configure here.
| yield f"data: {json.dumps(item)}\n\n" | ||
| if await request.is_disconnected(): | ||
| stop_flag.set() | ||
| break |
There was a problem hiding this comment.
Disconnect checked after queue wait
Medium Severity
Client disconnect is only handled after await q.get() returns and a frame is yielded. While blocked waiting for the next trial, a disconnected client does not set stop_flag, so the background Optuna worker can start or finish additional expensive Petrinaut evaluations after the client has gone.
Additional Locations (1)
Reviewed by Cursor Bugbot for commit b3ace77. Configure here.
|
Unresolved |
2 participants
🌟 What is the purpose of this PR?
Adds @hashintel/petrinaut-opt, a standalone Python service for black-box optimization of Petri-net executions. Given a set of Petri-net parameters and initial states (markings), it uses Optuna to search the input space and maximise/minimise a single metric computed by the Petri-net model. Each candidate is evaluated by running the net through the
petrinaut-clisocket server, and results stream back one trial at a time over Server-Sent Events so a UI can watch the optimization live.This is the Python-side counterpart to the Petrinaut CLI server: the CLI executes the net, this package decides which inputs to try and orchestrates the search.
🔗 Related links
Depends on the Petrinaut CLI socket server (petrinaut serve) — libs/@hashintel/petrinaut-cli
🚫 Blocked by
None
🔍 What does this change?
Adds a new Python (uv) package petrinaut-opt under libs/@hashintel/.
petrinaut_client.py — PetrinautModel connects to the Petrinaut CLI over a Unix socket (/tmp/petrinaut.sock) and, per trial, sends a run request (parameters, initialState, metrics, maxSteps, dt, seed) and reads back the named metric. PetrinautModelSpec (Pydantic) configures the execution.
petrinaut_optimizer.py — PetrinautOptimizer drives an Optuna study: suggest proposes inputs (fixed inputs held constant, the rest searched over BOUNDS), objective runs the model and prunes failed/errored trials, and stream_all / stream_best emit SSE frames (one per finished trial / running-best) via a background thread and asyncio.Queue. OptimizationSpec configures a run.
optimization_api.py — FastAPI service exposing POST /init (create a session), GET /optimize/{session_id}/stream/all, GET /optimize/{session_id}/stream/best, DELETE /optimize/{session_id}, GET /status, and GET /.
Concurrency is scoped per session: each optimizer holds its own lock (one session can't be driven by two concurrent streams), and there is no global single-run guard — independent /init sessions run independently.
stream_best only emits once at least one trial has completed, avoiding an Optuna error when opening trials are all pruned.
Adds README.md, pyproject.toml, requirements.txt, and .gitignore.
Pre-Merge Checklist 🚀
🚢 Has this modified a publishable library?
This PR:
📜 Does this require a change to the docs?
The changes in this PR:
🕸️ Does this require a change to the Turbo Graph?
The changes in this PR:
turbo.json's have been updated to reflect thisSocket path /tmp/petrinaut.sock is hard-coded on both the client and the CLI side; they must be started to match.
If an entire input group (parameters or initial_states) is fully fixed, the SSE callback can raise a KeyError — each group should leave at least one input free to optimize.
structure, outpath, store, and eval_timeout on PetrinautModelSpec are accepted but currently unused.
Not wired into monorepo tooling (no package.json), so it isn't picked up by turbo lint/test.
🐾 Next steps
TBD
🛡 What tests cover this?
None yet — no automated tests are included in this PR.
❓ How to test this?
Check out the branch.
Start the Petrinaut CLI server: petrinaut serve --model <path/to/model.json> --socket /tmp/petrinaut.sock (from libs/@hashintel/petrinaut-cli).
From libs/@hashintel/petrinaut-opt, run uv sync, then start the API: uv run uvicorn src.optimization_api:app --reload.
POST /init with an opt_spec + pn_spec body (see README) and grab the session_id.
curl -sN localhost:8000/optimize/<session_id>/stream/all and confirm one SSE frame per trial, ending in event: done. Repeat with /stream/best to confirm best-so-far frames.
Alternatively run the optimizer directly: uv run python -m src.petrinaut_optimizer.
📹 Demo