flyer-crawler.projectium.com/docs/adr/0010-testing-strategy-and-standards.md

ADR-010: Testing Strategy and Standards

Date: 2025-12-12

Status: Accepted

Context

The project has a good foundation of unit and integration tests. However, there is no formal document defining the scope, purpose, and expected coverage for different types of tests. This can lead to inconsistent testing quality and gaps in coverage.

Decision

We will formalize the testing pyramid for the project, defining the role of each testing layer:

1. Unit Tests (Vitest): For isolated functions, components, and repository methods with mocked dependencies. High coverage is expected.
2. Integration Tests (Supertest): For API routes, testing the interaction between controllers, services, and mocked database layers. Focus on contract and middleware correctness.
3. End-to-End (E2E) Tests (Vitest + Supertest): For critical user flows (e.g., login, flyer upload, checkout), running against a real test server and database to ensure the entire system works together.

Consequences

Positive: Ensures a consistent and comprehensive approach to quality assurance. Gives developers confidence when refactoring or adding new features. Clearly defines "done" for a new feature. Negative: May require investment in setting up and maintaining the E2E testing environment. Can slightly increase the time required to develop a feature if all test layers are required.

Implementation Details

Testing Framework Stack

Tool	Version	Purpose
Vitest	4.0.15	Test runner for all test types
@testing-library/react	16.3.0	React component testing
@testing-library/jest-dom	6.9.1	DOM assertion matchers
supertest	7.1.4	HTTP assertion library for API testing
msw	2.12.3	Mock Service Worker for network mocking
testcontainers	11.8.1	Database containerization (optional)
c8 + nyc	10.1.3 / 17.1.0	Coverage reporting

Test File Organization

src/
├── components/
│   └── *.test.tsx        # Component unit tests (colocated)
├── hooks/
│   └── *.test.ts         # Hook unit tests (colocated)
├── services/
│   └── *.test.ts         # Service unit tests (colocated)
├── routes/
│   └── *.test.ts         # Route handler unit tests (colocated)
├── utils/
│   └── *.test.ts         # Utility function tests (colocated)
└── tests/
    ├── setup/            # Test configuration and setup files
    ├── utils/            # Test utilities, factories, helpers
    ├── assets/           # Test fixtures (images, files)
    ├── integration/      # Integration test files (*.test.ts)
    └── e2e/              # End-to-end test files (*.e2e.test.ts)

Naming Convention: {filename}.test.ts or {filename}.test.tsx for unit/integration, {filename}.e2e.test.ts for E2E.

Configuration Files

Config	Environment	Purpose
vite.config.ts	jsdom	Unit tests (React components, hooks)
vitest.config.integration.ts	node	Integration tests (API routes)
vitest.config.e2e.ts	node	E2E tests (full user flows)
vitest.workspace.ts	-	Orchestrates all test projects

Test Pyramid

                    ┌─────────────┐
                    │    E2E      │  5 test files
                    │   Tests     │  Critical user flows
                    ├─────────────┤
                    │ Integration │  17 test files
                    │   Tests     │  API contracts + middleware
                ┌───┴─────────────┴───┐
                │     Unit Tests      │  185 test files
                │  Components, Hooks, │  Isolated functions
                │  Services, Utils    │  Mocked dependencies
                └─────────────────────┘

Unit Tests

Purpose: Test isolated functions, components, and modules with mocked dependencies.

Environment: jsdom (browser-like)

Key Patterns:

// Component testing with providers
import { renderWithProviders, screen } from '@/tests/utils/renderWithProviders';

describe('MyComponent', () => {
  it('renders correctly', () => {
    renderWithProviders(<MyComponent />);
    expect(screen.getByText('Hello')).toBeInTheDocument();
  });
});

// Hook testing
import { renderHook, waitFor } from '@testing-library/react';
import { useMyHook } from './useMyHook';

describe('useMyHook', () => {
  it('returns expected value', async () => {
    const { result } = renderHook(() => useMyHook());
    await waitFor(() => expect(result.current.data).toBeDefined());
  });
});

Global Mocks (automatically applied via tests-setup-unit.ts):

• Database connections (pg.Pool)
• AI services (@google/genai)
• Authentication (jsonwebtoken, bcrypt)
• Logging (logger.server, logger.client)
• Notifications (notificationService)

Integration Tests

Purpose: Test API routes with real service interactions and database.

Environment: node

Setup: Real Express server on port 3001, real PostgreSQL database

// API route testing pattern
import supertest from 'supertest';
import { createAndLoginUser } from '@/tests/utils/testHelpers';

describe('Auth API', () => {
  let request: ReturnType<typeof supertest>;
  let authToken: string;

  beforeAll(async () => {
    const app = (await import('../../../server')).default;
    request = supertest(app);
    const { token } = await createAndLoginUser(request);
    authToken = token;
  });

  it('GET /api/auth/me returns user profile', async () => {
    const response = await request.get('/api/auth/me').set('Authorization', `Bearer ${authToken}`);

    expect(response.status).toBe(200);
    expect(response.body.user.email).toBeDefined();
  });
});

Database Cleanup:

import { cleanupDb } from '@/tests/utils/cleanup';

afterAll(async () => {
  await cleanupDb({ users: [testUserId] });
});

E2E Tests

Purpose: Test complete user journeys through the application.

Timeout: 120 seconds (for long-running flows)

Current E2E Tests:

• auth.e2e.test.ts - Registration, login, password reset
• flyer-upload.e2e.test.ts - Complete flyer upload pipeline
• user-journey.e2e.test.ts - Full user workflow
• admin-authorization.e2e.test.ts - Admin-specific flows
• admin-dashboard.e2e.test.ts - Admin dashboard functionality

Mock Factories

The project uses comprehensive mock factories (src/tests/utils/mockFactories.ts, 1553 lines) for creating test data:

import {
  createMockUser,
  createMockFlyer,
  createMockFlyerItem,
  createMockRecipe,
  resetMockIds,
} from '@/tests/utils/mockFactories';

beforeEach(() => {
  resetMockIds(); // Ensure deterministic IDs
});

it('creates flyer with items', () => {
  const flyer = createMockFlyer({ store_name: 'TestMart' });
  const items = [createMockFlyerItem({ flyer_id: flyer.flyer_id })];
  // ...
});

Factory Coverage: 90+ factory functions for all domain entities including users, flyers, recipes, shopping lists, budgets, achievements, etc.

Test Utilities

Utility	Purpose
renderWithProviders()	Wrap components with AppProviders + Router
createAndLoginUser()	Create user and return auth token
cleanupDb()	Database cleanup respecting FK constraints
createTestApp()	Create Express app for route testing
poll()	Polling utility for async operations

Coverage Configuration

Coverage Provider: v8 (built-in Vitest)

Report Directories:

• .coverage/unit/ - Unit test coverage
• .coverage/integration/ - Integration test coverage
• .coverage/e2e/ - E2E test coverage

Excluded from Coverage:

• src/index.tsx, src/main.tsx (entry points)
• src/tests/** (test files themselves)
• src/**/*.d.ts (type declarations)
• src/components/icons/** (icon components)
• src/db/seed*.ts (database seeding scripts)

npm Scripts

# Run all tests
npm run test

# Run by level
npm run test:unit          # Unit tests only (jsdom)
npm run test:integration   # Integration tests only (node)

# With coverage
npm run test:coverage      # Unit + Integration with reports

# Clean coverage directories
npm run clean

Test Timeouts

Test Type	Timeout	Rationale
Unit	5 seconds	Fast, isolated tests
Integration	60 seconds	AI service calls, DB operations
E2E	120 seconds	Full user flow with multiple API calls

Best Practices

When to Write Each Test Type

1. Unit Tests (required):

   • Pure functions and utilities
   • React components (rendering, user interactions)
   • Custom hooks
   • Service methods with mocked dependencies
   • Repository methods

2. Integration Tests (required for API changes):

   • New API endpoints
   • Authentication/authorization flows
   • Middleware behavior
   • Database query correctness

3. E2E Tests (for critical paths):

   • User registration and login
   • Core business flows (flyer upload, shopping lists)
   • Admin operations

Test Isolation Guidelines

1. Reset mock IDs: Call resetMockIds() in beforeEach()
2. Unique test data: Use timestamps or UUIDs for emails/usernames
3. Clean up after tests: Use cleanupDb() in afterAll()
4. Don't share state: Each test should be independent

Mocking Guidelines

1. Unit tests: Mock external dependencies (DB, APIs, services)
2. Integration tests: Mock only external APIs (AI services)
3. E2E tests: Minimal mocking, use real services where possible

Testing Code Smells

When testing requires any of the following patterns, treat it as a code smell indicating the production code needs refactoring:

1. Capturing callbacks through mocks: If you need to capture a callback passed to a mock and manually invoke it to test behavior, the code under test likely has poor separation of concerns.

2. Complex module resets: If tests require vi.resetModules(), vi.doMock(), or careful ordering of mock setup to work correctly, the module likely has problematic initialization or hidden global state.

3. Indirect verification: If you can only verify behavior by checking that internal mocks were called with specific arguments (rather than asserting on direct outputs), the code likely lacks proper return values or has side effects that should be explicit.

4. Excessive mock setup: If setting up mocks requires more lines than the actual test assertions, consider whether the code under test has too many dependencies or responsibilities.

The Fix: Rather than writing complex test scaffolding, refactor the production code to be more testable:

• Extract pure functions that can be tested with simple input/output assertions
• Use dependency injection to make dependencies explicit and easily replaceable
• Return values from functions instead of relying on side effects
• Split modules with complex initialization into smaller, focused units
• Make async flows explicit and controllable rather than callback-based

Example anti-pattern:

// BAD: Capturing callback to test behavior
const capturedCallback = vi.fn();
mockService.onEvent.mockImplementation((cb) => {
  capturedCallback = cb;
});
await initializeModule();
capturedCallback('test-data'); // Manually triggering to test
expect(mockOtherService.process).toHaveBeenCalledWith('test-data');

Example preferred pattern:

// GOOD: Direct input/output testing
const result = await processEvent('test-data');
expect(result).toEqual({ processed: true, data: 'test-data' });

Known Code Smell Violations (Technical Debt)

The following files contain acknowledged code smell violations that are deferred for future refactoring:

File	Violations	Rationale for Deferral
src/services/queueService.workers.test.ts	Callback capture, vi.resetModules(), excessive setup	BullMQ workers instantiate at module load; business logic is tested via service classes
src/services/workers.server.test.ts	vi.resetModules()	Same as above - worker wiring tests
src/services/queues.server.test.ts	vi.resetModules()	Queue instantiation at module load
src/App.test.tsx	Callback capture, excessive setup	Component integration test; refactoring would require significant UI architecture changes
src/features/voice-assistant/VoiceAssistant.test.tsx	Multiple callback captures	WebSocket/audio APIs are inherently callback-based
src/services/aiService.server.test.ts	Multiple vi.resetModules()	AI service initialization complexity

Policy: New code should follow the code smell guidelines. These existing violations are tracked here and will be addressed when the underlying modules are refactored or replaced.

Key Files

• vite.config.ts - Unit test configuration
• vitest.config.integration.ts - Integration test configuration
• vitest.config.e2e.ts - E2E test configuration
• vitest.workspace.ts - Workspace orchestration
• src/tests/setup/tests-setup-unit.ts - Global mocks (488 lines)
• src/tests/setup/integration-global-setup.ts - Server + DB setup
• src/tests/utils/mockFactories.ts - Mock factories (1553 lines)
• src/tests/utils/testHelpers.ts - Test utilities

Future Enhancements

1. Browser E2E Tests: Consider adding Playwright for actual browser testing
2. Visual Regression: Screenshot comparison for UI components
3. Performance Testing: Add benchmarks for critical paths
4. Mutation Testing: Verify test quality with mutation testing tools
5. Coverage Thresholds: Define minimum coverage requirements per module