Skip to content

fix(maintenance): export and import Experiments and Variants in starter (#36337)#36339

Merged
erickgonzalez merged 4 commits into
mainfrom
issue-36337-import-export-experiments
Jun 29, 2026
Merged

fix(maintenance): export and import Experiments and Variants in starter (#36337)#36339
erickgonzalez merged 4 commits into
mainfrom
issue-36337-import-export-experiments

Conversation

@erickgonzalez

@erickgonzalez erickgonzalez commented Jun 26, 2026

Copy link
Copy Markdown
Member

Proposed Changes

Fixes #36337.

The starter export endpoints (/api/v1/maintenance/_downloadStarter and /_downloadStarterWithAssets) silently omitted Experiments and their Variants, and the import path never restored them. ExportStarterUtil only dumps Hibernate-mapped tables plus a few hand-coded "special" tables; the experiment and variant tables are persisted via DotConnect (not Hibernate) and were in neither list — so experiment data was lost in starter-based backups and migrations.

Export (ExportStarterUtil)

  • In getAdditionalDataAsJSON() (shared by both starter endpoints), emit:
    • Variant.json — all rows including archived and DEFAULT, via VariantTransformer.
    • Experiment.json — all experiments via ExperimentsFactory.list(empty filter).

Import (StarterEntity + ImportStarterUtil)

  • Register Variant.json early (before contentlet version info / multi-tree, which reference variant_id) and Experiment.json last (experiment page_id references a contentlet identifier).
  • Restore via the variant/experiment factories:
    • Variants: skip the always-present DEFAULT, and use the factory (not VariantAPI, which rejects archived variants) so archived experiment variants restore.
    • Experiments: ExperimentsFactory.save() upserts by id and binds JSONB columns via addJSONParam.
  • Deserialize with the configured dotCMS ObjectMapper (JavaTime + Jdk8 modules) so Instant/Optional and JSONB fields round-trip correctly — BundlerUtil's bare mapper cannot.

Out of scope

  • The pg_dump path (_pgDumpAvailable / _downloadDb) already includes experiment/variant (not in its --exclude-table-data deny-list) and is left unchanged.

Checklist

  • Compiles (./mvnw compile -pl :dotcms-core)
  • No REST annotation changes (no openapi.yaml regeneration needed)
  • Manual round-trip verification (export → import into clean instance; confirm experiment + variants restored)

How to Test

  1. Create an A/B Experiment on a page (writes experiment row + dotexperiment-… variant rows).
  2. GET /api/v1/maintenance/_downloadStarterWithAssets (and _downloadStarter); unzip and confirm Experiment.json and Variant.json are present and populated.
  3. Import the starter into a clean instance and confirm the experiment and its variants exist and the experiment opens correctly in the UI.

🤖 Generated with Claude Code

This PR fixes: #36337

…er (#36337)

The starter export (_downloadStarter / _downloadStarterWithAssets) only dumped
Hibernate-mapped tables plus a few hand-coded special tables. The experiment and
variant tables are persisted via DotConnect (not Hibernate) and were in neither
list, so experiment data was silently dropped from starters and never restored
on import.

Add both tables to the starter round-trip:
- Export: write Experiment.json and Variant.json in
  ExportStarterUtil.getAdditionalDataAsJSON() (shared by both starter endpoints).
- Import: register Variant.json (early) and Experiment.json (after contentlets)
  in StarterEntity, and handle them in ImportStarterUtil via the variant/experiment
  factories. Deserialize with the configured dotCMS mapper so Instant/Optional and
  JSONB fields round-trip correctly. Skip the DEFAULT variant and use the variant
  factory (not VariantAPI) so archived experiment variants restore.

The pg_dump backup path (_downloadDb) already includes these tables and is left
unchanged.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@claude

claude Bot commented Jun 26, 2026

Copy link
Copy Markdown
Contributor

Claude finished @wezell's task in 1m 43s —— View job


Rollback Safety Analysis

  • Read rollback-unsafe categories reference
  • Get full PR diff
  • Analyze changes against unsafe categories
  • Post results and add appropriate label

Result: ✅ Safe To Rollback

After checking every category in the rollback-unsafe reference, this PR matches none of the unsafe categories.

What the PR changes

File What it does
ExportStarterUtil.java Adds Variant.json and Experiment.json to the starter export ZIP
ImportStarterUtil.java Reads and restores those files during starter import
StarterEntity.java Registers the two new entities in the ordered import list

Category-by-category verdict

Category Verdict Reason
C-1 Structural Data Model Change ✅ Safe No schema changes. Tables variant and experiment already exist; code only reads/writes via existing factories.
C-2 ES Mapping Change ✅ Safe No Elasticsearch mapping touched. No putMapping, ESMappingAPIImpl, or reindex calls.
C-3 Content JSON Model Version Bump ✅ Safe CURRENT_MODEL_VERSION in ImmutableContentlet is not touched.
C-4 DROP TABLE / DROP COLUMN ✅ Safe No DDL of any kind.
H-1 One-Way Data Migration / Backfill ✅ Safe No UPDATE … SET across existing rows. The export/import path only reads existing rows and writes them back on import.
H-2 RENAME TABLE / RENAME COLUMN ✅ Safe No renames.
H-3 PK / Unique Constraint Restructuring ✅ Safe No constraint changes.
H-4 New ContentType Field Type ✅ Safe No new field types registered.
H-5 Binary Storage Provider Change ✅ Safe Storage layer untouched.
H-6 DROP PROCEDURE / FUNCTION ✅ Safe No stored procedures touched.
H-7 NOT NULL Column Without Default ✅ Safe No ALTER TABLE at all.
H-8 VTL Viewtool Contract Change ✅ Safe No viewtool classes or returned DTO types modified.
M-1 Column Type Change ✅ Safe No DDL.
M-2 Push Publishing Bundle Format ✅ Safe No bundler/handler changes.
M-3 REST / GraphQL API Contract ✅ Safe No REST endpoints or response shapes changed.
M-4 OSGi Plugin API Breakage ✅ Safe No public interface signatures changed.

Summary

Rolling back to N-1 after this change is deployed is completely safe. The only behavioral delta is that starter ZIPs produced by N contain two additional JSON files (Variant.json, Experiment.json) that N-1 does not know how to import — N-1 would simply ignore them (as it does for any unrecognized file in the starter ZIP). No database schema, ES index, data format, or API contract was altered.

@mergify

mergify Bot commented Jun 26, 2026

Copy link
Copy Markdown

Tick the box to add this pull request to the merge queue (same as @mergifyio queue).

  • Queue this pull request

@github-actions

github-actions Bot commented Jun 26, 2026

Copy link
Copy Markdown
Contributor

🤖 dotBot Review (Bedrock)

Reviewed 3 file(s); 5 candidate(s) → 3 confirmed, 0 uncertain (unverified, kept for review).

Confirmed findings

  • 🟡 Medium dotCMS/src/main/java/com/dotmarketing/util/starter/ImportStarterUtil.java:519 — Missing transaction around variant/experiment imports
    The importVariants() and importExperiments() methods save multiple entities via factory.save() in loops without transactional boundaries. If a failure occurs mid-import (e.g., DB error after partial saves), variants/experiments would be partially persisted, leaving the system in an inconsistent state. Variants are referenced in contentlet_version_info and multi_tree, so partial imports could orphan those references. Confirmed via code inspection: no @WrapInTransaction on importStarterData or its callers, and each save is auto-committed individually.
  • 🟡 Medium dotCMS/src/main/java/com/dotmarketing/util/starter/ImportStarterUtil.java:519 — Experiment save() may overwrite existing records in non-clean instances
    The code uses ExperimentsFactory.save() which performs an upsert by experiment ID. When importing into a non-clean environment, this will overwrite existing experiments with matching IDs. While Variant processing skips the DEFAULT variant, there's no similar duplicate check for experiments. This violates the starter's expected behavior of additive restoration and risks data loss if reused in existing environments.
  • 🟡 Medium dotCMS/src/main/java/com/dotmarketing/util/starter/ImportStarterUtil.java:519 — Test gap: Missing automated tests for Variant/Experiment import with archived states
    The PR adds export/import logic for Experiments and Variants but lacks automated tests verifying the round-trip of archived entries. Manual testing was mentioned, but no test files (e.g., ImportStarterUtilTest) were found via grep, indicating a test coverage gap for critical data integrity scenarios.

us.deepseek.r1-v1:0 · Run: #28271403691 · tokens: in: 24273 · out: 9586 · total: 33859 · calls: 9 · est. ~$0.085

@dotCMS dotCMS deleted a comment from github-actions Bot Jun 26, 2026
@dotCMS dotCMS deleted a comment from github-actions Bot Jun 26, 2026
wezell added a commit to dotCMS/ai-workflows that referenced this pull request Jun 26, 2026
Live run on dotCMS/core#36339 (via the new diagnostic) showed R1 emits stage-1
candidates with keys [sev, loc, desc] — it echoes the carry-forward
dotcms-review-findings schema from the prompt instead of the requested
severity/line/title/hypothesis. The prior alias map had "description" but not
"desc", and didn't map sev/loc, so _usable() still dropped every candidate and
the review reported "No issues found".

Now: "desc" is a title/hypothesis alias, "sev" ("🔴 Critical") maps to severity,
and "loc" ("path:597") maps to line. Verified the canonical schema is untouched.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@erickgonzalez erickgonzalez added this pull request to the merge queue Jun 29, 2026
Merged via the queue into main with commit 470a815 Jun 29, 2026
71 checks passed
@erickgonzalez erickgonzalez deleted the issue-36337-import-export-experiments branch June 29, 2026 15:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

AI: Safe To Rollback Area : Backend PR changes Java/Maven backend code

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

Starter export/import omits Experiments and Variants data

3 participants