Skip to content

Speed up Python GraphBinary deserialization (3.7)#3504

Open
kirill-stepanishin wants to merge 1 commit into
apache:3.7-devfrom
kirill-stepanishin:python-graphbinary-int-dispatch-3.7
Open

Speed up Python GraphBinary deserialization (3.7)#3504
kirill-stepanishin wants to merge 1 commit into
apache:3.7-devfrom
kirill-stepanishin:python-graphbinary-int-dispatch-3.7

Conversation

@kirill-stepanishin

Copy link
Copy Markdown
Contributor

The GraphBinary reader built a DataType enum member from the type byte for every object it decoded. That per-object enum construction heavily degrades deserialization performance on large result sets.

The reader now builds a {type code: deserializer} lookup table once up front and dispatches on the raw integer instead, avoiding per-object enum construction. Behavior is unchanged: an unknown type code still raises ValueError("... is not a valid DataType").

Performance

Benchmarked on two cross-region EC2 instances (server in US-EAST-2, client in US-WEST-2) to capture realistic network latency, against the Modern graph over WebSocket on Python 3.11. Each query was run with and without this change, alternating back to back across 3 sweeps, reporting the median.

Query Before After Change
g.V().repeat(both()).times(12) (~200k results) 6.81 s 5.36 s 21% faster
g.V() (6 results) 0.181 s 0.181 s no change

The improvement is significant on large result sets, where per-object deserialization cost dominates, and scales with the number of objects returned.

Assisted-by: Claude Code:claude-opus-4-8
@codecov-commenter

codecov-commenter commented Jul 2, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 75.52%. Comparing base (4722890) to head (d4e2af1).
⚠️ Report is 14 commits behind head on 3.7-dev.

Additional details and impacted files
@@              Coverage Diff              @@
##             3.7-dev    #3504      +/-   ##
=============================================
+ Coverage      75.49%   75.52%   +0.02%     
- Complexity     13161    13175      +14     
=============================================
  Files           1092     1093       +1     
  Lines          67208    67226      +18     
  Branches        7391     7396       +5     
=============================================
+ Hits           50742    50770      +28     
+ Misses         13837    13833       -4     
+ Partials        2629     2623       -6     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants