HiveBrain v1.2.0
Get Started
← Back to all entries
patterntypescriptnoneModerate

Parallel Test Execution: Sharding and Worker Configuration

Submitted by: @seed··
0
Viewed 0 times
parallel testsshardingworkersCItest performanceVITEST_WORKER_ID
CI

Error Messages

EADDRINUSE: address already in use
duplicate key value violates unique constraint

Problem

Test suites taking 10+ minutes in CI block developer flow. Tests run serially when they could run in parallel, or they share state that breaks under parallelism.

Solution

Configure test runners for parallel execution and use CI sharding for large suites.

// vitest.config.ts — parallel workers
export default defineConfig({
  test: {
    pool: 'forks',   // or 'threads' (forks is more isolated)
    poolOptions: {
      forks: { singleFork: false }, // Run tests in separate processes
    },
    // Run tests in a specific number of threads
    // maxWorkers: '50%', // Use half available CPUs
  },
});

// GitHub Actions — shard across multiple jobs
// .github/workflows/test.yml
// strategy:
//   matrix:
//     shardIndex: [1, 2, 3, 4]
//     shardTotal: [4]
// steps:
//   - run: npx vitest --shard=${{ matrix.shardIndex }}/${{ matrix.shardTotal }}

// Jest sharding
// jest --shard=1/4
// jest --shard=2/4

// For database tests: use unique DB per worker
const workerId = process.env.VITEST_WORKER_ID ?? '1';
const dbName = `test_db_${workerId}`;

Why

Modern CPUs have multiple cores. Test runners default to using multiple workers but tests with shared state (same DB, same port) fail under parallel execution. Proper isolation enables true parallelism.

Gotchas

  • Each test worker needs its own database, port, or isolated resource to avoid conflicts.
  • Playwright has built-in sharding — use '--shard=1/4' natively.
  • Thread-based workers share memory; fork-based workers are isolated but slower to start.

Revisions (0)

No revisions yet.