Skip to content

H-6664: Python monorepo infrastructure (uv workspace + turbo integration) — experimental spike#8994

Draft
claude[bot] wants to merge 5 commits into
mainfrom
claude/h-6664-python-monorepo-spike
Draft

H-6664: Python monorepo infrastructure (uv workspace + turbo integration) — experimental spike#8994
claude[bot] wants to merge 5 commits into
mainfrom
claude/h-6664-python-monorepo-spike

Conversation

@claude

@claude claude Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Requested by Tim Diekmann · Slack thread

Warning

Experimental spike for H-6664 "Create Python monorepo infrastructure" (internal), opened so the approach can be picked apart before the real implementation. Totally fine to close this and start over.

🌟 What is the purpose of this PR?

Demonstrate proper Python monorepo infrastructure, borrowing internal-infra's uv-workspace pattern and wiring it into turborepo.

Before: Python packages are invisible to turbo and CI. There is no shared lint/type-check/test configuration and no lockfile policy — e.g. #8989 adds a uv package with no package.json (so turbo never sees it), no CI, and a gitignored uv.lock.

After: turbo run lint:ruff lint:basedpyright test:unit --filter '@local/python-example' runs ruff, basedpyright, and pytest for Python packages via uv — with shared config and one committed uv.lock at the repo root — and the existing Lint/Test CI pipelines pick Python packages up like any other workspace package.

How it works

  • uv workspace: the new root pyproject.toml is a virtual workspace root ([tool.uv] package = false) listing Python packages as members, with requires-python = ">=3.14". A single uv.lock is committed at the repo root (do not gitignore it). Shared tooling lives in one root dev dependency group: ruff, basedpyright, pytest, pytest-asyncio. Per org Semgrep policy (uv-missing-dependency-cooldown), [tool.uv] exclude-newer = "7 days" applies a dependency cooldown — releases younger than a week are ignored during resolution (recorded in uv.lock as exclude-newer-span = "P7D"; adding it changed no resolved versions).
  • Centralized tool config in the root pyproject.toml:
    • ruff: select = ["ALL"] with internal-infra's short, documented ignore list (COM812, D203, D213, FIX002, TD003) plus per-file ignores for tests; line-length = 100 (matches the Rust width used here); target-version = "py314".
    • basedpyright: typeCheckingMode = "all".
    • pytest: --import-mode=importlib, asyncio_mode = "auto".
  • The package.json shim (the key part): each Python package carries a package.json whose scripts wrap uv, e.g. "test:unit": "uv run --frozen --all-packages pytest". Turbo then treats the package like any other workspace member — task graph, caching, --filter, remote cache all work. Two subtleties:
    • --all-packages is required: uv run from a member directory only syncs that member's dependencies, so without it the root dev group tools would be missing from the shared venv (verified with a fresh .venv).
    • basedpyright is pointed at the root config via --project ../../.. because pyright, unlike ruff and pytest, does not search parent directories for configuration.
  • Example packages (temporary scaffolding): libs/@local/python-example (hatchling build backend, src layout, py.typed, fully typed module, pytest suite incl. an async test for the pytest-asyncio wiring) and libs/@local/python-example-two, which depends on the first via [tool.uv.sources] hash-python-example = { workspace = true } (mirrored as a workspace:* dependency in its package.json so turbo's graph matches) and imports from it in module + tests — so cross-package uv workspace dependencies and the prune path are exercised on every CI run, not just coexistence. Both are marked as temporary in their READMEs: they validate the infra and should be deleted once the first real Python package (e.g. petrinaut-opt) joins the workspace.

CI (mostly the zero-new-workflow path)

  • test.yml: zero changes needed. It discovers packages via turbo query on TASK_NAME == "test:unit"; verified locally that @local/python-example shows up in exactly that query. Runners get uv 0.11.18 via mise (install-tools), and uv run --frozen syncs the venv on first use.
  • lint.yml: small addition. It enumerates tools per package (lint:eslint, lint:tsc, clippy, …) rather than running a generic lint task, so it genuinely can't pick Python up unaided. Added lint:ruff / lint:basedpyright detection + run steps mirroring the existing ESLint/TSC pattern.
  • prune-repository action: the action keeps the root pyproject.toml + uv.lock (they aren't part of turbo prune's output), and prune.py treats every uv workspace member as an always-included prune scope (directory-copy fallback for members without a package.json) — Python is effectively exempt from pruning, since uv resolves the whole workspace (see open question 4).
  • check-license-in-workspaces now ignores .venv/ (uv creates it at the repo root).
  • turbo.json already listed pyproject.toml and *.lock in globalDependencies, so Python task caching invalidates correctly with no changes.

🔗 Related links

🔍 What does this change?

  • New root pyproject.toml (uv workspace + shared ruff/basedpyright/pytest config) and committed root uv.lock
  • Two temporary example packages, libs/@local/python-example and libs/@local/python-example-two (pyproject.toml, package.json shim, LICENSE.md, typed module, tests each; the second depends on the first through the uv workspace)
  • turbo.json: new lint:ruff, lint:basedpyright, fix:ruff tasks
  • Root package.json: matching lint:ruff / lint:basedpyright / fix:ruff scripts, so yarn lint includes Python
  • .github/workflows/lint.yml: ruff/basedpyright detection + run steps
  • .github/actions/prune-repository/: action.yml keeps pyproject.toml + uv.lock; prune.py always preserves uv workspace members
  • libs/@local/repo-chores license check: ignore .venv/
  • yarn.lock: registers the new workspace

❓ Open questions for Bilal

  1. Python version floor — resolved: requires-python = ">=3.14" per Bilal, matching internal-infra (initially shipped as >=3.13). Note a workspace resolves on the intersection of all members, so FE-1162: Python optimisation script (Optuna) driving the Petrinaut CLI #8989's >=3.10 package could only join by raising its own floor.

  2. Placement/naming: libs/@local/python-example with dist name hash-python-example / module hash_python_example. Should Python packages live under a different convention (e.g. alongside @hashintel), and should the npm-scope name mirror the dist name?

  3. petrinaut-opt (FE-1162: Python optimisation script (Optuna) driving the Petrinaut CLI #8989): should it join the workspace (member entry + package.json shim + un-gitignore its lock into the root uv.lock) when cf-petrinaut-python lands?

  4. CI pruning — reproduced; interim fix shipped (the remove-prune-entirely discussion stays open). With a temporary second workspace member (libs/@local/python-example-two, not committed) and the prune-repository action's steps run locally scoped to @local/python-example, the pruned tree lacks the sibling member and:

    • uv sync --all-packages --locked exits 1 — The lockfile at 'uv.lock' needs to be updated, but '--locked' was provided. (uv re-discovers the workspace from members and resolves 12 packages against a 13-package lock)
    • uv run --frozen --all-packages pytest exits 2 — error: Failed to determine installation plan / Caused by: Distribution not found at: file:///…/out/libs/@local/python-example-two

    i.e. the first additional Python package would have broken every pruned Python CI job.

    Interim fix (in this PR): prune.py resolves the uv workspace members from the root pyproject.toml and adds them to the turbo prune scopes, so every pruned tree keeps the full Python workspace — including the members' entries in the pruned yarn.lock (a plain directory-copy after pruning is not enough: yarn then fails with This package doesn't seem to be present in your lockfile). Members without a package.json fall back to a directory copy. The two-package validation is now committed (python-example-two), so every pruned CI job carries both members and this path is exercised continuously instead of only in a local reproduction. Verified on the committed tree: pruned to a Python scope and to an unrelated scope (@local/advanced-types), both pruned trees pass uv sync --all-packages --locked and turbo run lint:ruff lint:basedpyright test:unit for both packages (6/6 tasks).

    For the wider debate on dropping turbo prune from CI: the prune-repository action (custom prune.py wrapping turbo prune) is a CI-only checkout optimizationlint.yml (×1), test.yml (×2), bench.yml (×4), codspeed.yml (×1), deploy.yml (×1, sourcemaps job) — while the five docker image builds (hash-graph, hash-frontend, hash-api, hash-ai-worker-ts, hash-integration-worker) call turbo's native turbo prune --scope=… --docker directly in their Dockerfiles and never use prune.py, so dropping the CI action would not leave the docker builds' prune path unexercised — they never shared code with it. Separately: no uv download cache in CI yet; worth adding?

  5. Line length: picked 100 (Rust width); repo conventions are split (JS 80, SQL 120, ruff default 88).

🚫 Blocked by

Nothing — but this is a spike; the real implementation is Bilal's roadmap item.

Pre-Merge Checklist 🚀

🚢 Has this modified a publishable library?

This PR:

  • does not modify any publishable blocks or libraries, or modifications do not need publishing

📜 Does this require a change to the docs?

The changes in this PR:

  • are internal and do not require a docs change

🕸️ Does this require a change to the Turbo Graph?

The changes in this PR:

  • affected the execution graph, and the turbo.json's have been updated to reflect this

⚠️ Known issues

  • uv run --frozen without --all-packages silently falls back to a ruff/pytest found on PATH if the tool isn't in the venv — the scripts always pass --all-packages to avoid this.

🛡 What tests cover this?

  • libs/@local/python-example/tests/test_greeting.py (3 tests, incl. async) and libs/@local/python-example-two/tests/test_team_greetings.py (2 tests, imports across the workspace dependency) — run by turbo run test:unit.

Verified locally (uv 0.11.18, turbo 2.6.3, yarn 4.16.0, CPython 3.14.6): uv lock, uv sync --all-packages --locked, and for both packages uv run --frozen --all-packages ruff check ., … ruff format --check ., … basedpyright --project ../../.. . (strict mode confirmed to really analyze files via an intentional error), … pytest (3 + 2 passed), turbo run lint:ruff lint:basedpyright test:unit filtered to each package and to both (6/6 tasks, FULL TURBO cache hits on re-run), the pruned-tree validation described in open question 4, yarn constraints, yarn lint:license-in-workspaces, taplo/oxfmt/markdownlint on the changed files.

Not verified in this environment: the actual GitHub Actions runs (incl. mise-provisioned uv on runners) and the Rust sort-package-json check (mise run lint:package-json needs a cargo build) — watch the checks on this PR.

❓ How to test this?

  1. Checkout the branch, uv sync --all-packages
  2. turbo run lint:ruff lint:basedpyright test:unit --filter '@local/python-example'
  3. Confirm ruff, basedpyright, and pytest all execute via uv and pass (and hit the turbo cache on the second run)

@vercel

vercel Bot commented Jul 9, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
hash Ready Ready Preview, Comment Jul 13, 2026 2:17pm
2 Skipped Deployments
Project Deployment Actions Updated (UTC)
hashdotdesign-tokens Ignored Ignored Preview Jul 13, 2026 2:17pm
petrinaut Skipped Skipped Comment Jul 13, 2026 2:17pm

@github-actions github-actions Bot added area/deps Relates to third-party dependencies (area) area/infra Relates to version control, CI, CD or IaC (area) area/libs Relates to first-party libraries/crates/packages (area) type/eng > backend Owned by the @backend team area/tests New or updated tests type/legal Owned by the @legal team labels Jul 9, 2026
Comment thread pyproject.toml
Comment on lines +16 to +18
[tool.uv]
package = false

@semgrep-code-hashintel semgrep-code-hashintel Bot Jul 9, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This pyproject.toml configures uv but does not set a dependency cooldown. Newly published packages can be malicious or unstable. Add exclude-newer = "7 days" under [tool.uv] to wait 7 days before resolving newly published package versions. Added in: 0.9.17 Reference: https://docs.astral.sh/uv/concepts/resolution/#dependency-cooldowns

🍰 Fixed in commit d55400d 🍰

Comment thread pyproject.toml Fixed
@codecov

codecov Bot commented Jul 9, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 59.73%. Comparing base (6736174) to head (22a66ed).
⚠️ Report is 5 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #8994   +/-   ##
=======================================
  Coverage   59.73%   59.73%           
=======================================
  Files        1371     1371           
  Lines      135461   135461           
  Branches     6066     6066           
=======================================
  Hits        80911    80911           
  Misses      53618    53618           
  Partials      932      932           
Flag Coverage Δ
rust.hash-graph-api 7.41% <ø> (ø)
rust.hashql-syntax-jexpr 94.04% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@codspeed-hq

codspeed-hq Bot commented Jul 9, 2026

Copy link
Copy Markdown

Merging this PR will degrade performance by 15.38%

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

❌ 2 regressed benchmarks
✅ 96 untouched benchmarks

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Benchmark BASE HEAD Efficiency
bit_matrix/dense/iter_row[64] 140.8 ns 170 ns -17.16%
bit_matrix/dense/iter_row[200] 185.8 ns 215 ns -13.57%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing claude/h-6664-python-monorepo-spike (22a66ed) with main (b6fb59e)

Open in CodSpeed

claude added 3 commits July 13, 2026 13:38
…o integration)

- root pyproject.toml: uv workspace with shared ruff/basedpyright/pytest
  config and a single dev dependency group; committed root uv.lock
- libs/@local/python-example: minimal example package (hatchling, typed,
  tested) whose package.json scripts wrap uv so turbo runs lint:ruff,
  lint:basedpyright and test:unit like any other workspace package
- turbo.json: lint:ruff, lint:basedpyright and fix:ruff tasks
- lint.yml: detect and run ruff/basedpyright per package, mirroring the
  existing ESLint/TSC steps; test.yml picks up test:unit automatically
- prune-repository action: keep root pyproject.toml and uv.lock
- check-license-in-workspaces: ignore .venv
- requires-python >=3.14 across the workspace (matching internal-infra)
  and ruff target-version py314
- [tool.uv] exclude-newer = "7 days" per the uv-missing-dependency-cooldown
  Semgrep policy; re-locked (no resolved versions changed)
uv resolves the whole workspace, so a pruned tree missing any member
listed in the root pyproject.toml fails uv sync --locked / uv run
--frozen. prune.py now adds every uv workspace member to the turbo
prune scopes (keeping their pruned yarn.lock entries intact) and falls
back to copying member directories that have no package.json.
…the prune path

libs/@local/python-example-two depends on hash-python-example via
[tool.uv.sources] { workspace = true } (mirrored as workspace:* in its
package.json for turbo's graph) and imports it in module and tests, so
cross-package resolution and the prune exemption are validated on every
CI run instead of only in a local reproduction. Both example packages
are temporary scaffolding, marked for deletion in their READMEs once
the first real Python package joins the workspace.
oxfmt (yarn lint:format) normalizes *emphasis* to _emphasis_ in
markdown; the Global lint job caught the discrepancy.
@github-actions

Copy link
Copy Markdown
Contributor

Benchmark results

@rust/hash-graph-benches – Integrations

policy_resolution_large

Function Value Mean Flame graphs
resolve_policies_for_actor user: empty, selectivity: high, policies: 2002 $$28.6 \mathrm{ms} \pm 193 \mathrm{μs}\left({\color{gray}-3.240 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: empty, selectivity: low, policies: 1 $$3.46 \mathrm{ms} \pm 17.9 \mathrm{μs}\left({\color{lightgreen}-6.070 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: empty, selectivity: medium, policies: 1002 $$12.7 \mathrm{ms} \pm 104 \mathrm{μs}\left({\color{lightgreen}-10.047 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: seeded, selectivity: high, policies: 3314 $$44.0 \mathrm{ms} \pm 578 \mathrm{μs}\left({\color{gray}-2.283 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: seeded, selectivity: low, policies: 1 $$14.3 \mathrm{ms} \pm 113 \mathrm{μs}\left({\color{lightgreen}-12.391 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: seeded, selectivity: medium, policies: 1527 $$24.6 \mathrm{ms} \pm 227 \mathrm{μs}\left({\color{gray}-4.598 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: high, policies: 2078 $$29.2 \mathrm{ms} \pm 244 \mathrm{μs}\left({\color{gray}-3.212 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: low, policies: 1 $$3.80 \mathrm{ms} \pm 26.1 \mathrm{μs}\left({\color{gray}-3.886 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: medium, policies: 1033 $$14.2 \mathrm{ms} \pm 110 \mathrm{μs}\left({\color{gray}1.33 \mathrm{\%}}\right) $$ Flame Graph

policy_resolution_medium

Function Value Mean Flame graphs
resolve_policies_for_actor user: empty, selectivity: high, policies: 102 $$3.88 \mathrm{ms} \pm 27.6 \mathrm{μs}\left({\color{gray}-1.996 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: empty, selectivity: low, policies: 1 $$3.07 \mathrm{ms} \pm 20.1 \mathrm{μs}\left({\color{gray}-0.611 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: empty, selectivity: medium, policies: 52 $$3.44 \mathrm{ms} \pm 18.8 \mathrm{μs}\left({\color{gray}-2.258 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: seeded, selectivity: high, policies: 269 $$5.34 \mathrm{ms} \pm 42.2 \mathrm{μs}\left({\color{gray}-2.265 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: seeded, selectivity: low, policies: 1 $$3.70 \mathrm{ms} \pm 25.2 \mathrm{μs}\left({\color{gray}0.190 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: seeded, selectivity: medium, policies: 108 $$4.20 \mathrm{ms} \pm 24.5 \mathrm{μs}\left({\color{gray}-3.741 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: high, policies: 133 $$4.48 \mathrm{ms} \pm 28.7 \mathrm{μs}\left({\color{gray}-2.948 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: low, policies: 1 $$3.52 \mathrm{ms} \pm 18.6 \mathrm{μs}\left({\color{gray}-2.124 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: medium, policies: 63 $$4.20 \mathrm{ms} \pm 28.0 \mathrm{μs}\left({\color{gray}-2.630 \mathrm{\%}}\right) $$ Flame Graph

policy_resolution_none

Function Value Mean Flame graphs
resolve_policies_for_actor user: empty, selectivity: high, policies: 2 $$2.71 \mathrm{ms} \pm 16.2 \mathrm{μs}\left({\color{gray}-1.924 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: empty, selectivity: low, policies: 1 $$2.55 \mathrm{ms} \pm 15.3 \mathrm{μs}\left({\color{gray}-1.397 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: empty, selectivity: medium, policies: 2 $$2.69 \mathrm{ms} \pm 14.9 \mathrm{μs}\left({\color{gray}-1.669 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: high, policies: 8 $$2.95 \mathrm{ms} \pm 16.2 \mathrm{μs}\left({\color{gray}-3.460 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: low, policies: 1 $$2.77 \mathrm{ms} \pm 17.7 \mathrm{μs}\left({\color{gray}-1.852 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: medium, policies: 3 $$2.99 \mathrm{ms} \pm 20.1 \mathrm{μs}\left({\color{gray}-1.615 \mathrm{\%}}\right) $$ Flame Graph

policy_resolution_small

Function Value Mean Flame graphs
resolve_policies_for_actor user: empty, selectivity: high, policies: 52 $$3.11 \mathrm{ms} \pm 19.1 \mathrm{μs}\left({\color{gray}-1.216 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: empty, selectivity: low, policies: 1 $$2.82 \mathrm{ms} \pm 17.9 \mathrm{μs}\left({\color{gray}0.057 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: empty, selectivity: medium, policies: 26 $$3.05 \mathrm{ms} \pm 18.4 \mathrm{μs}\left({\color{gray}-0.838 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: seeded, selectivity: high, policies: 94 $$3.54 \mathrm{ms} \pm 22.3 \mathrm{μs}\left({\color{gray}-0.348 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: seeded, selectivity: low, policies: 1 $$3.11 \mathrm{ms} \pm 19.7 \mathrm{μs}\left({\color{gray}0.742 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: seeded, selectivity: medium, policies: 27 $$3.45 \mathrm{ms} \pm 22.3 \mathrm{μs}\left({\color{gray}1.30 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: high, policies: 66 $$3.46 \mathrm{ms} \pm 20.9 \mathrm{μs}\left({\color{gray}0.188 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: low, policies: 1 $$2.99 \mathrm{ms} \pm 17.9 \mathrm{μs}\left({\color{gray}-2.010 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: medium, policies: 29 $$3.42 \mathrm{ms} \pm 22.5 \mathrm{μs}\left({\color{gray}-0.575 \mathrm{\%}}\right) $$ Flame Graph

read_scaling_complete

Function Value Mean Flame graphs
entity_by_id;one_depth 1 entities $$44.3 \mathrm{ms} \pm 310 \mathrm{μs}\left({\color{gray}1.28 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;one_depth 10 entities $$34.4 \mathrm{ms} \pm 219 \mathrm{μs}\left({\color{gray}1.90 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;one_depth 25 entities $$37.5 \mathrm{ms} \pm 218 \mathrm{μs}\left({\color{red}5.46 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;one_depth 5 entities $$33.9 \mathrm{ms} \pm 178 \mathrm{μs}\left({\color{gray}2.69 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;one_depth 50 entities $$43.1 \mathrm{ms} \pm 240 \mathrm{μs}\left({\color{gray}1.84 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;two_depth 1 entities $$50.8 \mathrm{ms} \pm 354 \mathrm{μs}\left({\color{gray}1.86 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;two_depth 10 entities $$41.6 \mathrm{ms} \pm 255 \mathrm{μs}\left({\color{gray}1.35 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;two_depth 25 entities $$97.3 \mathrm{ms} \pm 547 \mathrm{μs}\left({\color{red}5.34 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;two_depth 5 entities $$35.3 \mathrm{ms} \pm 223 \mathrm{μs}\left({\color{gray}4.17 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;two_depth 50 entities $$285 \mathrm{ms} \pm 968 \mathrm{μs}\left({\color{lightgreen}-6.826 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;zero_depth 1 entities $$11.1 \mathrm{ms} \pm 59.1 \mathrm{μs}\left({\color{gray}1.69 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;zero_depth 10 entities $$11.2 \mathrm{ms} \pm 78.0 \mathrm{μs}\left({\color{gray}1.57 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;zero_depth 25 entities $$10.7 \mathrm{ms} \pm 64.0 \mathrm{μs}\left({\color{gray}-3.816 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;zero_depth 5 entities $$11.1 \mathrm{ms} \pm 63.6 \mathrm{μs}\left({\color{gray}1.61 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;zero_depth 50 entities $$10.8 \mathrm{ms} \pm 73.9 \mathrm{μs}\left({\color{gray}-3.815 \mathrm{\%}}\right) $$ Flame Graph

read_scaling_linkless

Function Value Mean Flame graphs
entity_by_id 1 entities $$10.7 \mathrm{ms} \pm 58.2 \mathrm{μs}\left({\color{gray}-2.632 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 10 entities $$10.9 \mathrm{ms} \pm 62.9 \mathrm{μs}\left({\color{gray}-3.224 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 100 entities $$10.9 \mathrm{ms} \pm 74.8 \mathrm{μs}\left({\color{gray}-2.463 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 1000 entities $$10.8 \mathrm{ms} \pm 63.1 \mathrm{μs}\left({\color{gray}-4.346 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 10000 entities $$11.1 \mathrm{ms} \pm 68.1 \mathrm{μs}\left({\color{gray}-3.226 \mathrm{\%}}\right) $$ Flame Graph

representative_read_entity

Function Value Mean Flame graphs
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/block/v/1 $$11.5 \mathrm{ms} \pm 74.7 \mathrm{μs}\left({\color{gray}1.34 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/book/v/1 $$11.5 \mathrm{ms} \pm 87.7 \mathrm{μs}\left({\color{gray}0.378 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/building/v/1 $$11.1 \mathrm{ms} \pm 59.7 \mathrm{μs}\left({\color{gray}-2.145 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/organization/v/1 $$11.3 \mathrm{ms} \pm 79.6 \mathrm{μs}\left({\color{gray}-3.478 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/page/v/2 $$11.1 \mathrm{ms} \pm 64.8 \mathrm{μs}\left({\color{gray}1.09 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/person/v/1 $$11.1 \mathrm{ms} \pm 68.9 \mathrm{μs}\left({\color{gray}-1.011 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/playlist/v/1 $$11.1 \mathrm{ms} \pm 59.3 \mathrm{μs}\left({\color{gray}-3.092 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/song/v/1 $$11.2 \mathrm{ms} \pm 66.1 \mathrm{μs}\left({\color{gray}-0.939 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/uk-address/v/1 $$11.4 \mathrm{ms} \pm 81.5 \mathrm{μs}\left({\color{gray}-0.144 \mathrm{\%}}\right) $$ Flame Graph

representative_read_entity_type

Function Value Mean Flame graphs
get_entity_type_by_id Account ID: bf5a9ef5-dc3b-43cf-a291-6210c0321eba $$8.68 \mathrm{ms} \pm 48.4 \mathrm{μs}\left({\color{gray}-1.502 \mathrm{\%}}\right) $$ Flame Graph

representative_read_multiple_entities

Function Value Mean Flame graphs
entity_by_property traversal_paths=0 0 $$58.1 \mathrm{ms} \pm 423 \mathrm{μs}\left({\color{gray}1.92 \mathrm{\%}}\right) $$
entity_by_property traversal_paths=255 1,resolve_depths=inherit:1;values:255;properties:255;links:127;link_dests:126;type:true $$114 \mathrm{ms} \pm 595 \mathrm{μs}\left({\color{gray}1.04 \mathrm{\%}}\right) $$
entity_by_property traversal_paths=2 1,resolve_depths=inherit:0;values:0;properties:0;links:0;link_dests:0;type:false $$64.4 \mathrm{ms} \pm 540 \mathrm{μs}\left({\color{gray}0.748 \mathrm{\%}}\right) $$
entity_by_property traversal_paths=2 1,resolve_depths=inherit:0;values:0;properties:0;links:1;link_dests:0;type:true $$75.0 \mathrm{ms} \pm 559 \mathrm{μs}\left({\color{gray}2.58 \mathrm{\%}}\right) $$
entity_by_property traversal_paths=2 1,resolve_depths=inherit:0;values:0;properties:2;links:1;link_dests:0;type:true $$85.2 \mathrm{ms} \pm 470 \mathrm{μs}\left({\color{gray}1.33 \mathrm{\%}}\right) $$
entity_by_property traversal_paths=2 1,resolve_depths=inherit:0;values:2;properties:2;links:1;link_dests:0;type:true $$90.7 \mathrm{ms} \pm 592 \mathrm{μs}\left({\color{gray}1.17 \mathrm{\%}}\right) $$
link_by_source_by_property traversal_paths=0 0 $$45.9 \mathrm{ms} \pm 343 \mathrm{μs}\left({\color{gray}1.95 \mathrm{\%}}\right) $$
link_by_source_by_property traversal_paths=255 1,resolve_depths=inherit:1;values:255;properties:255;links:127;link_dests:126;type:true $$76.2 \mathrm{ms} \pm 369 \mathrm{μs}\left({\color{gray}0.556 \mathrm{\%}}\right) $$
link_by_source_by_property traversal_paths=2 1,resolve_depths=inherit:0;values:0;properties:0;links:0;link_dests:0;type:false $$52.4 \mathrm{ms} \pm 372 \mathrm{μs}\left({\color{gray}0.114 \mathrm{\%}}\right) $$
link_by_source_by_property traversal_paths=2 1,resolve_depths=inherit:0;values:0;properties:0;links:1;link_dests:0;type:true $$61.5 \mathrm{ms} \pm 388 \mathrm{μs}\left({\color{gray}-0.270 \mathrm{\%}}\right) $$
link_by_source_by_property traversal_paths=2 1,resolve_depths=inherit:0;values:0;properties:2;links:1;link_dests:0;type:true $$64.8 \mathrm{ms} \pm 369 \mathrm{μs}\left({\color{gray}0.992 \mathrm{\%}}\right) $$
link_by_source_by_property traversal_paths=2 1,resolve_depths=inherit:0;values:2;properties:2;links:1;link_dests:0;type:true $$65.0 \mathrm{ms} \pm 406 \mathrm{μs}\left({\color{gray}2.41 \mathrm{\%}}\right) $$

scenarios

Function Value Mean Flame graphs
full_test query-limited $$123 \mathrm{ms} \pm 653 \mathrm{μs}\left({\color{gray}-4.989 \mathrm{\%}}\right) $$ Flame Graph
full_test query-unlimited $$136 \mathrm{ms} \pm 857 \mathrm{μs}\left({\color{gray}-1.636 \mathrm{\%}}\right) $$ Flame Graph
linked_queries query-limited $$23.7 \mathrm{ms} \pm 194 \mathrm{μs}\left({\color{red}25.6 \mathrm{\%}}\right) $$ Flame Graph
linked_queries query-unlimited $$560 \mathrm{ms} \pm 1.41 \mathrm{ms}\left({\color{red}5.16 \mathrm{\%}}\right) $$ Flame Graph

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/deps Relates to third-party dependencies (area) area/infra Relates to version control, CI, CD or IaC (area) area/libs Relates to first-party libraries/crates/packages (area) area/tests New or updated tests type/eng > backend Owned by the @backend team type/legal Owned by the @legal team

Development

Successfully merging this pull request may close these issues.

3 participants