flyer-crawler.projectium.com/docs/research-e2e-test-separation.md

Research: Separating E2E Tests from Integration Tests

Date: 2026-01-19 Status: In Progress Context: E2E tests exist with their own config but are not being run separately

Current State

Test Structure

• Unit tests: src/tests/unit/ (but most are co-located with source files)
• Integration tests: src/tests/integration/ (28 test files)
• E2E tests: src/tests/e2e/ (11 test files) ← NOT CURRENTLY RUNNING

Configurations

Config File	Project Name	Environment	Port	Include Pattern
vite.config.ts	unit	jsdom	N/A	Component/hook tests
vitest.config.integration.ts	integration	node	3099	src/tests/integration/**/*.test.{ts,tsx}
vitest.config.e2e.ts	e2e	node	3098	src/tests/e2e/**/*.e2e.test.ts

Workspace Configuration

vitest.workspace.ts currently includes:

export default [
  'vite.config.ts', // Unit tests
  'vitest.config.integration.ts', // Integration tests
  // ❌ vitest.config.e2e.ts is NOT included!
];

NPM Scripts

{
  "test": "node scripts/check-linux.js && cross-env NODE_ENV=test tsx ./node_modules/vitest/vitest.mjs run",
  "test:unit": "... --project unit ...",
  "test:integration": "... --project integration ..."
  // ❌ NO test:e2e script exists!
}

CI/CD Status

.gitea/workflows/deploy-to-test.yml runs:

• ✅ npm run test:unit -- --coverage
• ✅ npm run test:integration -- --coverage
• ❌ E2E tests are NOT run in CI

Key Findings

1. E2E Tests Are Orphaned

• 11 E2E test files exist but are never executed
• E2E config file exists (vitest.config.e2e.ts) but is not referenced anywhere
• No npm script to run E2E tests
• Not included in vitest workspace
• Not run in CI/CD pipeline

2. When Were E2E Tests Created?

Git history shows E2E config was added in commit e66027d ("fix e2e and deploy to prod"), but:

• It was never added to the workspace
• It was never added to CI
• No test:e2e script was created

This suggests the E2E separation was started but never completed.

3. How Are Tests Currently Run?

Locally:

• npm test → runs workspace (unit + integration only)
• npm run test:unit → runs only unit tests
• npm run test:integration → runs only integration tests
• E2E tests: Not accessible via any command

In CI:

• Only test:unit and test:integration are run
• E2E tests are never executed

4. Port Allocation

• Integration tests: Port 3099
• E2E tests: Port 3098 (configured but never used)
• No conflicts if both run sequentially

E2E Test Files (11 total)

1. admin-authorization.e2e.test.ts
2. admin-dashboard.e2e.test.ts
3. auth.e2e.test.ts
4. budget-journey.e2e.test.ts
5. deals-journey.e2e.test.ts ← Just fixed URL constraint issue
6. error-reporting.e2e.test.ts
7. flyer-upload.e2e.test.ts
8. inventory-journey.e2e.test.ts
9. receipt-journey.e2e.test.ts
10. upc-journey.e2e.test.ts
11. user-journey.e2e.test.ts

Problems to Solve

Immediate Issues

1. E2E tests are not running - Code exists but is never executed
2. No way to run E2E tests - No npm script or CI job
3. Coverage gaps - E2E scenarios are untested in practice
4. False sense of security - Team may think E2E tests are running

Implementation Challenges

1. Adding E2E to Workspace

Option A: Add to workspace

// vitest.workspace.ts
export default [
  'vite.config.ts',
  'vitest.config.integration.ts',
  'vitest.config.e2e.ts', // ← Add this
];

Impact: E2E tests would run with npm test, increasing test time significantly

Option B: Keep separate

• E2E remains outside workspace
• Requires explicit npm run test:e2e command
• CI would need separate step for E2E tests

2. Adding NPM Script

{
  "test:e2e": "node scripts/check-linux.js && cross-env NODE_ENV=test tsx --max-old-space-size=8192 ./node_modules/vitest/vitest.mjs run --project e2e -c vitest.config.e2e.ts"
}

Dependencies:

• Uses same global setup pattern as integration tests
• Requires server to be stopped first (like integration tests)
• Port 3098 must be available

3. CI/CD Integration

Add to .gitea/workflows/deploy-to-test.yml:

- name: Run E2E Tests
  run: |
    npm run test:e2e -- --coverage \
      --reporter=verbose \
      --includeTaskLocation \
      --testTimeout=120000 \
      --silent=passed-only

Questions:

• Should E2E run before or after integration tests?
• Should E2E failures block deployment?
• Should E2E have separate coverage reports?

4. Test Organization Questions

• Are current "integration" tests actually E2E tests?
• Should some E2E tests be moved to integration?
• What's the distinction between integration and E2E in this project?

5. Coverage Implications

• E2E tests have separate coverage directory: .coverage/e2e
• Integration tests: .coverage/integration
• How to merge coverage from all test types?
• Do we need combined coverage reports?

Recommended Approach

Phase 1: Quick Fix (Enable E2E Tests)

1. ✅ Fix any failing E2E tests (like URL constraints)
2. Add test:e2e npm script
3. Document how to run E2E tests manually
4. Do NOT add to workspace yet (keep separate)

Phase 2: CI Integration

1. Add E2E test step to .gitea/workflows/deploy-to-test.yml
2. Run after integration tests pass
3. Allow failures initially (monitor results)
4. Make blocking once stable

Phase 3: Optimize

1. Review test categorization (integration vs E2E)
2. Consider adding to workspace if test time is acceptable
3. Merge coverage reports if needed
4. Document test strategy in testing docs

Next Steps

1. Create test:e2e script in package.json
2. Run E2E tests manually to verify they work
3. Fix any failing E2E tests
4. Document E2E testing in TESTING.md
5. Add to CI once stable
6. Consider workspace integration after CI is stable

Questions for Team

1. Why were E2E tests never fully integrated?
2. Should E2E tests run on every commit or separately?
3. What's the acceptable test time for local development?
4. Should we run E2E tests in parallel or sequentially with integration?

Related Files

• vitest.workspace.ts - Workspace configuration
• vitest.config.e2e.ts - E2E test configuration
• src/tests/setup/e2e-global-setup.ts - E2E global setup
• .gitea/workflows/deploy-to-test.yml - CI pipeline
• package.json - NPM scripts