Skip to content

ci(showcase): replace hermetic build golden tests with Librarian#13569

Open
zhumin8 wants to merge 8 commits into
googleapis:mainfrom
zhumin8:ci-showcase-librarian
Open

ci(showcase): replace hermetic build golden tests with Librarian#13569
zhumin8 wants to merge 8 commits into
googleapis:mainfrom
zhumin8:ci-showcase-librarian

Conversation

@zhumin8

@zhumin8 zhumin8 commented Jun 29, 2026

Copy link
Copy Markdown
Contributor

Replace the hermetic build code generation and verification workflow for the showcase client with direct Librarian execution.

- Update `generate_showcase.sh` to run `librarian generate showcase` directly, with the Librarian version defined in the
  workspace's `librarian.yaml` configuration.
- Simplify `verify.sh` to generate the code in-place and use `git status` on the generated source directories to check for drift, eliminating the complex temporary directory and symlink mirroring setups.
- Add Go, Python, protoc, and Librarian setup steps to the showcase workflow runner in `showcase.yaml` so tests can execute librarian generate.
- Update `java-showcase/README.md` to document Go, protoc, and Python virtual environment setup instructions.

Choose to put environment setups outside the script to prevent it from intruding on developers' local configurations, keep execution fast. Note that there are on-going work to simplify these setups (googleapis/librarian#6558 to make protoc part of install; https://github.com/googleapis/librarian/milestone/139 to migrate away from python owlbot.py)

Fixes googleapis/librarian#6393

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request replaces the Docker-based generation and verification of the Java Showcase library with a local setup using a Python virtual environment and the librarian tool. Key changes include updating the README with the new setup steps, simplifying generate_showcase.sh to run librarian in-place, and updating verify.sh to check for git diffs directly. The review feedback suggests several robust improvements: extracting the librarian version locally from librarian.yaml to avoid network latency, scoping the pip configuration to the virtual environment using the --venv flag, and switching verify.sh to Bash with array variables to prevent word-splitting bugs and fix an invalid option in set -oxe.

Comment thread java-showcase/scripts/generate_showcase.sh
Comment thread java-showcase/README.md Outdated
Comment thread java-showcase/scripts/verify.sh Outdated
@zhumin8

zhumin8 commented Jun 29, 2026

Copy link
Copy Markdown
Contributor Author

showcase test failure: https://paste.googleplex.com/5262905682558976
Root cause: hermetic build uses docker for code generation, now configuring librarian directly with the test matrix Java 25 and 26 tries to use it to generate code and fails because formatter version in librarian.yaml does not support these versions.
Fix: set Java 17 for generation, then switch back for tests.

@zhumin8 zhumin8 marked this pull request as ready for review June 29, 2026 21:37
@zhumin8 zhumin8 requested review from a team as code owners June 29, 2026 21:37
@zhumin8

zhumin8 commented Jun 29, 2026

Copy link
Copy Markdown
Contributor Author

@blakeli0 @JoeWang1127 Two point worth noting and curious on what do you think:

  • I kept env setup steps and librarian install step outside of java-showcase/scripts/generate_showcase.sh. I can potentially move librarian install step inside the script, but it feels wrong separation of concerns, and install takes around 5 mins. If I am a developer running this locally, I wouldn't want to redo this step within mvn verify -P enable-golden-tests
  • I removed the "replace" flag and simplified the scripts, with the cost that test needs be run on clean git tree. This requires developers to commit code before running test. Which is a change but isn't terrible to me. But I also don't feel strongly on this approach, I can do a followup to restore a "replace flag" if that is desired. Curious about your take on this?

cc. @lqiu96 who I think may recently be running these tests locally.

Comment thread java-showcase/README.md

```shell
# Setup and activate Python virtual environment
python -m venv .venv

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume Python is only needed for post-processing, and we should be able to remove it after we migrate it to Go?

- name: Install protoc
run: |
set -e
VERSION="33.2"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How do we specify protoc version with Librarian now? Is it configurable or we just use the available protoc on the machine?

-P enable-golden-tests \
--batch-mode \
--no-transfer-progress
- name: Restore Matrix Java

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this extra step needed?

@blakeli0

Copy link
Copy Markdown
Contributor

I removed the "replace" flag and simplified the scripts, with the cost that test needs be run on clean git tree

Is it because hermetic build used to only modify selective files, but Librarian is a replace-all approach?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

google-cloud-java: update showcase golden test to use librarian generate

2 participants