Torben Sorensen
4022768c03
All checks were successful
Deploy to Test Environment / deploy-to-test (push) Successful in 18m39s
233 lines
6.6 KiB
Markdown
233 lines
6.6 KiB
Markdown
# Research: Separating E2E Tests from Integration Tests
|
|
|
|
**Date:** 2026-01-19
|
|
**Status:** In Progress
|
|
**Context:** E2E tests exist with their own config but are not being run separately
|
|
|
|
## Current State
|
|
|
|
### Test Structure
|
|
|
|
- **Unit tests**: `src/tests/unit/` (but most are co-located with source files)
|
|
- **Integration tests**: `src/tests/integration/` (28 test files)
|
|
- **E2E tests**: `src/tests/e2e/` (11 test files) **← NOT CURRENTLY RUNNING**
|
|
|
|
### Configurations
|
|
|
|
| Config File | Project Name | Environment | Port | Include Pattern |
|
|
| ------------------------------ | ------------- | ----------- | ---- | ------------------------------------------ |
|
|
| `vite.config.ts` | `unit` | jsdom | N/A | Component/hook tests |
|
|
| `vitest.config.integration.ts` | `integration` | node | 3099 | `src/tests/integration/**/*.test.{ts,tsx}` |
|
|
| `vitest.config.e2e.ts` | `e2e` | node | 3098 | `src/tests/e2e/**/*.e2e.test.ts` |
|
|
|
|
### Workspace Configuration
|
|
|
|
**`vitest.workspace.ts` currently includes:**
|
|
|
|
```typescript
|
|
export default [
|
|
'vite.config.ts', // Unit tests
|
|
'vitest.config.integration.ts', // Integration tests
|
|
// ❌ vitest.config.e2e.ts is NOT included!
|
|
];
|
|
```
|
|
|
|
### NPM Scripts
|
|
|
|
```json
|
|
{
|
|
"test": "node scripts/check-linux.js && cross-env NODE_ENV=test tsx ./node_modules/vitest/vitest.mjs run",
|
|
"test:unit": "... --project unit ...",
|
|
"test:integration": "... --project integration ..."
|
|
// ❌ NO test:e2e script exists!
|
|
}
|
|
```
|
|
|
|
### CI/CD Status
|
|
|
|
**`.gitea/workflows/deploy-to-test.yml` runs:**
|
|
|
|
- ✅ `npm run test:unit -- --coverage`
|
|
- ✅ `npm run test:integration -- --coverage`
|
|
- ❌ E2E tests are NOT run in CI
|
|
|
|
## Key Findings
|
|
|
|
### 1. E2E Tests Are Orphaned
|
|
|
|
- 11 E2E test files exist but are never executed
|
|
- E2E config file exists (`vitest.config.e2e.ts`) but is not referenced anywhere
|
|
- No npm script to run E2E tests
|
|
- Not included in vitest workspace
|
|
- Not run in CI/CD pipeline
|
|
|
|
### 2. When Were E2E Tests Created?
|
|
|
|
Git history shows E2E config was added in commit `e66027d` ("fix e2e and deploy to prod"), but:
|
|
|
|
- It was never added to the workspace
|
|
- It was never added to CI
|
|
- No test:e2e script was created
|
|
|
|
This suggests the E2E separation was **started but never completed**.
|
|
|
|
### 3. How Are Tests Currently Run?
|
|
|
|
**Locally:**
|
|
|
|
- `npm test` → runs workspace (unit + integration only)
|
|
- `npm run test:unit` → runs only unit tests
|
|
- `npm run test:integration` → runs only integration tests
|
|
- E2E tests: **Not accessible via any command**
|
|
|
|
**In CI:**
|
|
|
|
- Only `test:unit` and `test:integration` are run
|
|
- E2E tests are never executed
|
|
|
|
### 4. Port Allocation
|
|
|
|
- Integration tests: Port 3099
|
|
- E2E tests: Port 3098 (configured but never used)
|
|
- No conflicts if both run sequentially
|
|
|
|
## E2E Test Files (11 total)
|
|
|
|
1. `admin-authorization.e2e.test.ts`
|
|
2. `admin-dashboard.e2e.test.ts`
|
|
3. `auth.e2e.test.ts`
|
|
4. `budget-journey.e2e.test.ts`
|
|
5. `deals-journey.e2e.test.ts` ← Just fixed URL constraint issue
|
|
6. `error-reporting.e2e.test.ts`
|
|
7. `flyer-upload.e2e.test.ts`
|
|
8. `inventory-journey.e2e.test.ts`
|
|
9. `receipt-journey.e2e.test.ts`
|
|
10. `upc-journey.e2e.test.ts`
|
|
11. `user-journey.e2e.test.ts`
|
|
|
|
## Problems to Solve
|
|
|
|
### Immediate Issues
|
|
|
|
1. **E2E tests are not running** - Code exists but is never executed
|
|
2. **No way to run E2E tests** - No npm script or CI job
|
|
3. **Coverage gaps** - E2E scenarios are untested in practice
|
|
4. **False sense of security** - Team may think E2E tests are running
|
|
|
|
### Implementation Challenges
|
|
|
|
#### 1. Adding E2E to Workspace
|
|
|
|
**Option A: Add to workspace**
|
|
|
|
```typescript
|
|
// vitest.workspace.ts
|
|
export default [
|
|
'vite.config.ts',
|
|
'vitest.config.integration.ts',
|
|
'vitest.config.e2e.ts', // ← Add this
|
|
];
|
|
```
|
|
|
|
**Impact:** E2E tests would run with `npm test`, increasing test time significantly
|
|
|
|
**Option B: Keep separate**
|
|
|
|
- E2E remains outside workspace
|
|
- Requires explicit `npm run test:e2e` command
|
|
- CI would need separate step for E2E tests
|
|
|
|
#### 2. Adding NPM Script
|
|
|
|
```json
|
|
{
|
|
"test:e2e": "node scripts/check-linux.js && cross-env NODE_ENV=test tsx --max-old-space-size=8192 ./node_modules/vitest/vitest.mjs run --project e2e -c vitest.config.e2e.ts"
|
|
}
|
|
```
|
|
|
|
**Dependencies:**
|
|
|
|
- Uses same global setup pattern as integration tests
|
|
- Requires server to be stopped first (like integration tests)
|
|
- Port 3098 must be available
|
|
|
|
#### 3. CI/CD Integration
|
|
|
|
**Add to `.gitea/workflows/deploy-to-test.yml`:**
|
|
|
|
```yaml
|
|
- name: Run E2E Tests
|
|
run: |
|
|
npm run test:e2e -- --coverage \
|
|
--reporter=verbose \
|
|
--includeTaskLocation \
|
|
--testTimeout=120000 \
|
|
--silent=passed-only
|
|
```
|
|
|
|
**Questions:**
|
|
|
|
- Should E2E run before or after integration tests?
|
|
- Should E2E failures block deployment?
|
|
- Should E2E have separate coverage reports?
|
|
|
|
#### 4. Test Organization Questions
|
|
|
|
- Are current "integration" tests actually E2E tests?
|
|
- Should some E2E tests be moved to integration?
|
|
- What's the distinction between integration and E2E in this project?
|
|
|
|
#### 5. Coverage Implications
|
|
|
|
- E2E tests have separate coverage directory: `.coverage/e2e`
|
|
- Integration tests: `.coverage/integration`
|
|
- How to merge coverage from all test types?
|
|
- Do we need combined coverage reports?
|
|
|
|
## Recommended Approach
|
|
|
|
### Phase 1: Quick Fix (Enable E2E Tests)
|
|
|
|
1. ✅ Fix any failing E2E tests (like URL constraints)
|
|
2. Add `test:e2e` npm script
|
|
3. Document how to run E2E tests manually
|
|
4. Do NOT add to workspace yet (keep separate)
|
|
|
|
### Phase 2: CI Integration
|
|
|
|
1. Add E2E test step to `.gitea/workflows/deploy-to-test.yml`
|
|
2. Run after integration tests pass
|
|
3. Allow failures initially (monitor results)
|
|
4. Make blocking once stable
|
|
|
|
### Phase 3: Optimize
|
|
|
|
1. Review test categorization (integration vs E2E)
|
|
2. Consider adding to workspace if test time is acceptable
|
|
3. Merge coverage reports if needed
|
|
4. Document test strategy in testing docs
|
|
|
|
## Next Steps
|
|
|
|
1. **Create `test:e2e` script** in package.json
|
|
2. **Run E2E tests manually** to verify they work
|
|
3. **Fix any failing E2E tests**
|
|
4. **Document E2E testing** in TESTING.md
|
|
5. **Add to CI** once stable
|
|
6. **Consider workspace integration** after CI is stable
|
|
|
|
## Questions for Team
|
|
|
|
1. Why were E2E tests never fully integrated?
|
|
2. Should E2E tests run on every commit or separately?
|
|
3. What's the acceptable test time for local development?
|
|
4. Should we run E2E tests in parallel or sequentially with integration?
|
|
|
|
## Related Files
|
|
|
|
- `vitest.workspace.ts` - Workspace configuration
|
|
- `vitest.config.e2e.ts` - E2E test configuration
|
|
- `src/tests/setup/e2e-global-setup.ts` - E2E global setup
|
|
- `.gitea/workflows/deploy-to-test.yml` - CI pipeline
|
|
- `package.json` - NPM scripts
|