Skip to content

chore: modernize testing infrastructure#5160

Merged
timdeschryver merged 19 commits into
mainfrom
test/vitest-type-tests
Jun 13, 2026
Merged

chore: modernize testing infrastructure#5160
timdeschryver merged 19 commits into
mainfrom
test/vitest-type-tests

Conversation

@rainerhahnekamp

@rainerhahnekamp rainerhahnekamp commented Jun 5, 2026

Copy link
Copy Markdown
Contributor

This PR unifies Vitest into a root workspace, adds native type-test modes (runtime / type-only / combi), and aligns per-project configs. Modules already ran Vitest via vite.config.mts; this centralizes that setup and enables IDE tooling.

What this PR does

  • Introduces a shared Vitest workspace via /vitest.config.mts and per-project vitest.config.mts files (14 config files: 12 modules/*, example-app, www)
  • Replaces per-project vite.config.mts with vitest.config.mts extending shared baseConfig via mergeConfig (except www, which keeps a standalone Analog config)
  • Enables pnpm vitest --project <project-name> and Vitest IDE tooling across the monorepo (13 projects in the root workspace — see Projects with customized test config)
  • Configures three test modes: runtime, type-only (.test-d.ts), and combi (types/*.spec.ts with Vitest type-check)
  • Streamlines example-app: moves test-setup.ts to the project root and updates tsconfig.spec.json (files path, drops strict: false)
  • Fixes schematic and migration tests to use dist/modules/... paths (relies on existing dependsOn: ["build"] when running through Nx)
  • Applies common test fixes: mock_actionsasync/await, local strip-json-comments for eslint-plugin schematics

The main file is /vitest.config.mts. It exposes shared baseConfig for most projects and registers workspace projects under modules/* and example-app — the repo root is not a project itself. Each standard project extends baseConfig via mergeConfig; exceptions are noted below.

Current state

Runtime tests already run through Vitest; type suites are not migrated yet.

  • 29 spec files under types/ts-snippet (25), expectTypeOf (1), custom helpers (2), runtime regression (1)
  • Naming is mixed: 16 *.types.spec.ts, 13 plain *.spec.ts; no *.test-d.ts files yet
  • standalone-app is still on Jest; everything else in scope uses Vitest

The configuration below is the basis for follow-up PRs that migrate those suites to native type-only or combi tests (see Follow-up tasks).

Projects with customized test config

  • www: Standalone Analog config (not baseConfig). Renamed vite.config.tsvitest.config.mts; project.json configFile updated for build/serve. Explicit resolve.alias for @ngrx-io (Vitest was resolving to /src/app/...). Not part of the root Vitest workspace. Use Nx: nx run www:test, nx run www:build, nx serve www.
  • effects: include: ['spec/**/*.spec.ts'] and testTimeout: 15000 (same include as main — see Excluded specs).
  • entity: include: ['spec/**/*.spec.ts'] and testTimeout: 15000 (same include as main; timeout raised for slow ts-snippet combi tests on CI).
  • schematics: include: ['src/**/*.spec.ts'] (same as main).
  • eslint-plugin: testTimeout: 8000.

Running tests

Nx (CI and recommended): nx run <project>:test or nx affected -t test. Module targets use existing dependsOn: ["build"], so dist/modules/... is populated before schematic and migration specs run.

Native Vitest (IDE / pnpm vitest): The root workspace enables pnpm vitest and pnpm vitest --project <name> for modules and example-app, but it does not trigger Nx builds. Schematic and migration tests load compiled factories from dist/modules/... via Node require() — the same requirement as on main. Run a build first:

nx build <project>
# or all modules:
nx run-many -t build -p component,component-store,data,effects,entity,eslint-plugin,operators,router-store,schematics,signals,store,store-devtools

Without that step, schematic/migration specs fail with missing dist/modules/... modules. Runtime-only specs (e.g. under spec/) are unaffected.

For www, use nx run www:test — it is not registered in the root workspace.

Excluded specs (dead on main)

Three modules keep the narrower include patterns from main. These specs exist in the repo but are not executed by Vitest or nx test — they were never part of CI on main either:

Module include Excluded specs
effects spec/**/*.spec.ts migrations/**, schematics/ng-add, testing/spec/mock_actions
entity spec/**/*.spec.ts migrations/6_0_0, schematics/ng-add
schematics src/**/*.spec.ts migrations/6_0_0

9 migration/schematic specs in total (plus mock_actions on effects). Enabling them is a follow-up (see Enable excluded migration and schematic tests).

Folder Structure

With Vitest we have three types of tests:

  • Runtime Tests: The default. Those verify via expect, etc. They are not type-safe — Vitest strips types and runs the code. Fast, but no type safety.
  • Type Tests: Type-only tests using expectTypeOf or assertType. Vitest only runs the compiler on them; they are not executed at runtime.
  • Combi Tests: Tests with @ts-expect-error are not precise enough to count as type tests alone. Example:
it('should fail', () => {
  const state = signalState({ foo: 'bar' });

  // @ts-expect-error foo is not a number
  patchState(state, { foo: 1 });
});

This is a valid type test, but @ts-expect-error does not differentiate between a type-only error and a syntax error. For example, the test would also succeed for:

it('should fail', () => {
  const state = signalState({ foo: 'bar' });

  // @ts-expect-error foo is not a number
  patchStore(state, { foo: 1 });
});

Therefore, these tests need to run both as runtime and type tests.

With the new configuration, combi tests belong in a types/ subfolder. Files matching **/*.test-d.ts run as type-tests only (anywhere in the project); **/{spec,test}.ts files under types/ run as both runtime and type tests.

Sidenote: With typecheck.enabled: true, Vitest prints an experimental-feature warning at startup: "Testing types with tsc and vue-tsc is an experimental feature. Breaking changes might not follow SemVer, please pin Vitest's version when using it." This is expected and harmless — type-checking works as intended. Pin the Vitest version in package.json until the feature stabilizes.

project-root/
├── modules/
│   └── mypackage/
│       ├── src/
│       ├── spec/
│       │   ├── foo.spec.ts         # Runtime Test
│       │   ├── bar.test.ts         # Runtime Test
│       │   └── types/
│       │       ├── baz.test-d.ts   # Type Test (only type-checked)
│       │       └── qux.spec.ts     # Combi Test (e.g., uses @ts-expect-error)

Test migration patterns

Patterns applied when enabling schematic, migration, and legacy tests under Vitest.

Schematic and migration paths: __dirnamedist/

SchematicTestRunner loads factories via Node require(), which expects compiled .js files. Tests should point at the built output, not source JSON next to .ts files.

// before
const collectionPath = path.join(__dirname, '../migration.json');

// after
const collectionPath = path.join(
  process.cwd(),
  'dist/modules/effects/migrations/migration.json'
);

The same applies to schematic collections:

// before
path.join(__dirname, '../collection.json');

// after
path.join(
  process.cwd(),
  'dist/modules/effects/schematics/collection.json'
);

Module test targets rely on existing dependsOn: ["build"] so dist/ exists when tests run through Nx. That does not apply to native pnpm vitest (see Running tests). Project apps (example-app, www) have no schematic/migration specs that require a pre-test build.

Replacing deprecated done() callbacks

Vitest 4 deprecates Jasmine-style done() in favor of async/await:

// before
it('should provide Actions from source', (done) => {
  const actions$ = TestBed.inject(Actions);
  actions$.subscribe((action) => {
    expect(action.type).toBe('foo');
    done();
  });
});

// after
it('should provide Actions from source', async () => {
  const actions$ = TestBed.inject(Actions);
  const action = await firstValueFrom(actions$);
  expect(action.type).toBe('foo');
});

CJS schematic dependencies under Vitest

Schematics compiled to CommonJS cannot require() ESM-only packages resolved from the workspace root. For @ngrx/eslint-plugin, strip-json-comments was replaced with a local helper bundled into the schematic output:

// before
import stripJsonComments from 'strip-json-comments';

// after
import { stripJsonComments } from '../strip-json-comments';

Slow type-check tests: testTimeout

Type tests that run tsc via ts-snippet can exceed the default timeout. Projects with heavy type suites set a higher limit in vitest.config.mts:

// effects
test: {
  name: 'effects',
  setupFiles: ['test-setup.ts'],
  include: ['spec/**/*.spec.ts'],
  testTimeout: 15000,
},

// entity
test: {
  name: 'entity',
  setupFiles: ['test-setup.ts'],
  include: ['spec/**/*.spec.ts'],
  testTimeout: 15000,
},

// eslint-plugin
test: {
  name: 'eslint-plugin',
  setupFiles: ['test-setup.ts'],
  testTimeout: 8000,
},

Combi specs under spec/types/ can exceed 8s on CI even when the type-check pass succeeds.

Follow-up tasks

Outstanding work after this PR (verified June 2026). Planned as separate PRs on top of the configuration introduced here.

Enable excluded migration and schematic tests

Planned. Widen include in effects, entity, and schematics to match other modules (**/*.{spec,test}.ts), then ensure those specs pass under Vitest with dependsOn: ["build"] (and document the manual nx build step for native pnpm vitest). The 9 currently excluded migration/schematic specs are listed in Excluded specs.

Migrate type suites to native Vitest type tests

Planned. Convert the 29 existing types/ spec files from ts-snippet (and related helpers) to native .test-d.ts type-only tests or combi tests. Start with smaller packages, then tackle heavy suites (signals, store, effects). Once complete, drop the custom typecheck.include pattern (**/types/**/*.{spec,test}.ts) in vitest.config.mts and standardize on *.test-d.ts naming.

Refactor deprecated done() callbacks

Planned. modules/effects/testing/spec/mock_actions.spec.ts is refactored to async/await but currently excluded via effects's include pattern (dead on main). Vitest 4 still warns on Jasmine-style done() in data, router-store, store, and store-devtools (roughly 70 usages across 16 spec files). These should move to Promises or async/await (see Replacing deprecated done() callbacks).

Fix source type errors (ignoreSourceErrors)

Planned. Shared baseConfig in vitest.config.mts sets typecheck.ignoreSourceErrors: true so Vitest type-checking only reports errors in test files, not in library source. Without it, 17 type errors in source code surface during pnpm vitest. Fix those errors and remove ignoreSourceErrors from baseConfig so type-check runs also guard production code. This does not apply to www, which uses its own standalone config and sets ignoreSourceErrors separately.

Performance: long-running type tests

Partially addressed. with-entities.types.spec.ts and the rest of the signals type suite pass with the default timeout. effects, entity (15s), and eslint-plugin (8s) use raised testTimeout values as a stopgap (see Slow type-check tests). Optimization may become less urgent as suites migrate off ts-snippet.

Remove legacy test framework artifacts

Planned. jasmine-marbles remains in a few modules/ specs (store, operators, data), Jasmine/Jest devDependencies remain in package.json, and projects/standalone-app is still on Jest/Karma. Migrate or remove where Vitest (or @analogjs/vitest-angular) equivalents exist.

Browser runtime instead of JSDOM

Future. Shared config sets environment: 'jsdom'. Moving runtime tests to a real browser environment is a larger follow-up for closer DOM fidelity.

Does this PR introduce a breaking change?

[ ] Yes
[x] No

Native pnpm vitest does not auto-build; schematic/migration tests need nx build first. www is not in the root workspace — use Nx for www tests and builds. No breaking changes to published packages.

@netlify

netlify Bot commented Jun 5, 2026

Copy link
Copy Markdown

Deploy Preview for ngrx-io ready!

Name Link
🔨 Latest commit ad3b722
🔍 Latest deploy log https://app.netlify.com/projects/ngrx-io/deploys/6a26a566aa24e7000842a3d8
😎 Deploy Preview https://deploy-preview-5160--ngrx-io.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.
🤖 Make changes Run an agent on this branch

To edit notification comments on pull requests, go to your Netlify project configuration.

@rainerhahnekamp rainerhahnekamp force-pushed the test/vitest-type-tests branch from ad3b722 to 9c205f5 Compare June 8, 2026 19:25
@rainerhahnekamp rainerhahnekamp changed the title DRAFT tests: modernization of testing infrastructure tests: modernization of testing infrastructure Jun 8, 2026
@rainerhahnekamp rainerhahnekamp marked this pull request as ready for review June 8, 2026 20:37
Comment thread modules/eslint-plugin/schematics/ng-add/index.ts
@markostanimirovic markostanimirovic changed the title tests: modernization of testing infrastructure chore: modernize testing infrastructure Jun 9, 2026
Comment thread modules/component-store/vitest.config.mts Outdated

@timdeschryver timdeschryver left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me!
(Sorry for the conflicts)

We can also add to include Analog's snapshot serializers to clean up some html snapshots and to remove the dependency to jest-preset-angular.

rainerhahnekamp and others added 15 commits June 13, 2026 02:18
- Establish workspace-level root vitest.config.mts config file.
- Standardize sub-project configs by renaming *vite.config.mts/ts to vitest.config.mts and merging shared baseConfig.
- Exclude schematic and migration test pipelines temporarily to keep CI green.
- Add FOLLOW_UP_TASKS.md to document identified migration tasks and technical debt.
- Include **/*.test-d.ts in Signals specification tsconfig.
Drop the explicit test configFile and tsconfig path aliases since the
workspace tsconfig and vitest config already resolve @ngrx-io imports.

Co-authored-by: Cursor <cursoragent@cursor.com>
- modules/effects/vitest.config.mts: Merged imports and timeout config.
- modules/effects/migrations/6_0_0/index.spec.ts & modules/entity/migrations/6_0_0/index.spec.ts: Removed redundant top-level collectionPath variable declarations.
- modules/effects/testing/spec/mock_actions.spec.ts: Standardized on firstValueFrom + async/await.
- projects/www/project.json: Merged dependsOn and removed redundant configFile options.
- package.json & pnpm-lock.yaml: Kept shiki v1.29.2 from origin/main and removed unused strip-json-comments dependency.
@rainerhahnekamp rainerhahnekamp force-pushed the test/vitest-type-tests branch from 9fa7f37 to 0750521 Compare June 13, 2026 09:01
@rainerhahnekamp rainerhahnekamp marked this pull request as draft June 13, 2026 09:06
By default, Vitest runs tests in the 'threads' pool (using worker_threads).
However, schematics/migrations tests (e.g. using @angular-devkit/schematics and ora) fail with 'SyntaxError: Cannot use import statement outside a module' inside worker threads.
This happens because worker threads have strict module loader limitations and do not fully support process-level hooks (like exit listeners) required by these packages.
Setting `pool: 'forks'` spawns tests in OS-level child processes (child_process.fork), restoring native Node.js ESM/CJS loader behavior and process API support.
@rainerhahnekamp rainerhahnekamp marked this pull request as ready for review June 13, 2026 16:00
@rainerhahnekamp

Copy link
Copy Markdown
Contributor Author

@markostanimirovic, @timdeschryver, @brandonroberts if you want you can take a look but the conflicts are gone and we are ready to go from my side.

@timdeschryver timdeschryver left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @rainerhahnekamp !

@timdeschryver timdeschryver merged commit ccb37f1 into main Jun 13, 2026
6 checks passed
@timdeschryver timdeschryver deleted the test/vitest-type-tests branch June 13, 2026 18:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants