Skip to content

[core][python] Improve earliest snapshot retry handling#8242

Open
JingsongLi wants to merge 5 commits into
apache:masterfrom
JingsongLi:codex/python-snapshot-earliest-retry
Open

[core][python] Improve earliest snapshot retry handling#8242
JingsongLi wants to merge 5 commits into
apache:masterfrom
JingsongLi:codex/python-snapshot-earliest-retry

Conversation

@JingsongLi

@JingsongLi JingsongLi commented Jun 15, 2026

Copy link
Copy Markdown
Contributor

Summary

When the earliest snapshot is expired concurrently, callers may observe an earliest snapshot id whose snapshot file is already gone. This PR makes Java and Python retry forward through a bounded window before giving up, while preserving empty-table handling for callers that have no available latest snapshot.

Changes

  • Share Java earliest-snapshot retry handling through SnapshotManager.retryEarliestSnapshot, reused by both SnapshotManager and TimeTravelUtil.
  • Align Python SnapshotManager with the Java behavior by retrying from the discovered earliest snapshot id instead of stopping at the first missing snapshot.
  • Increase the default earliest-snapshot retry window from 3 to 300 snapshot ids to better tolerate batch expiration races.
  • Throw RuntimeException / RuntimeError when an identified earliest snapshot range cannot be read after retry, instead of silently returning null / None.
  • Remove repeated WARN logs from the Java retry loop.
  • Add regression coverage for missing earliest snapshots, retry exhaustion, and retrying past more than one hundred missing Python snapshots.

Testing

  • GitHub Actions checks for this PR passed.
  • python -m pytest paimon-python/pypaimon/tests/snapshot_manager_test.py paimon-python/pypaimon/tests/time_travel_util_test.py
  • git diff --check

@JingsongLi JingsongLi changed the title [python] Retry missing earliest snapshot in time travel [core][python] Improve earliest snapshot retry handling Jun 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant