set up local e2e tests, and some e2e test fixes + docs on more db fixin - ugh
All checks were successful
Deploy to Test Environment / deploy-to-test (push) Successful in 18m39s
All checks were successful
Deploy to Test Environment / deploy-to-test (push) Successful in 18m39s
This commit is contained in:
@@ -0,0 +1,352 @@
|
|||||||
|
# ADR-023: Database Normalization and Referential Integrity
|
||||||
|
|
||||||
|
**Date:** 2026-01-19
|
||||||
|
**Status:** Accepted
|
||||||
|
**Context:** API design violates database normalization principles
|
||||||
|
|
||||||
|
## Problem Statement
|
||||||
|
|
||||||
|
The application's API layer currently accepts string-based references (category names) instead of numerical IDs when creating relationships between entities. This violates database normalization principles and creates a brittle, error-prone API contract.
|
||||||
|
|
||||||
|
**Example of Current Problem:**
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// API accepts string:
|
||||||
|
POST /api/users/watched-items
|
||||||
|
{ "itemName": "Milk", "category": "Dairy & Eggs" } // ❌ String reference
|
||||||
|
|
||||||
|
// But database uses normalized foreign keys:
|
||||||
|
CREATE TABLE master_grocery_items (
|
||||||
|
category_id BIGINT REFERENCES categories(category_id) -- ✅ Proper FK
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
|
This mismatch forces the service layer to perform string lookups on every request:
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// Service must do string matching:
|
||||||
|
const categoryRes = await client.query(
|
||||||
|
'SELECT category_id FROM categories WHERE name = $1',
|
||||||
|
[categoryName], // ❌ Error-prone string matching
|
||||||
|
);
|
||||||
|
```
|
||||||
|
|
||||||
|
## Database Normal Forms (In Order of Importance)
|
||||||
|
|
||||||
|
### 1. First Normal Form (1NF) ✅ Currently Satisfied
|
||||||
|
|
||||||
|
**Rule:** Each column contains atomic values; no repeating groups.
|
||||||
|
|
||||||
|
**Status:** ✅ **Compliant**
|
||||||
|
|
||||||
|
- All columns contain single values
|
||||||
|
- No arrays or delimited strings in columns
|
||||||
|
- Each row is uniquely identifiable
|
||||||
|
|
||||||
|
**Example:**
|
||||||
|
|
||||||
|
```sql
|
||||||
|
-- ✅ Good: Atomic values
|
||||||
|
CREATE TABLE master_grocery_items (
|
||||||
|
master_grocery_item_id BIGINT PRIMARY KEY,
|
||||||
|
name TEXT,
|
||||||
|
category_id BIGINT
|
||||||
|
);
|
||||||
|
|
||||||
|
-- ❌ Bad: Non-atomic values (violates 1NF)
|
||||||
|
CREATE TABLE items (
|
||||||
|
id BIGINT,
|
||||||
|
categories TEXT -- "Dairy,Frozen,Snacks" (comma-delimited)
|
||||||
|
);
|
||||||
|
```
|
||||||
|
|
||||||
|
### 2. Second Normal Form (2NF) ✅ Currently Satisfied
|
||||||
|
|
||||||
|
**Rule:** No partial dependencies; all non-key columns depend on the entire primary key.
|
||||||
|
|
||||||
|
**Status:** ✅ **Compliant**
|
||||||
|
|
||||||
|
- All tables use single-column primary keys (no composite keys)
|
||||||
|
- All non-key columns depend on the entire primary key
|
||||||
|
|
||||||
|
**Example:**
|
||||||
|
|
||||||
|
```sql
|
||||||
|
-- ✅ Good: All columns depend on full primary key
|
||||||
|
CREATE TABLE flyer_items (
|
||||||
|
flyer_item_id BIGINT PRIMARY KEY,
|
||||||
|
flyer_id BIGINT, -- Depends on flyer_item_id
|
||||||
|
master_item_id BIGINT, -- Depends on flyer_item_id
|
||||||
|
price_in_cents INT -- Depends on flyer_item_id
|
||||||
|
);
|
||||||
|
|
||||||
|
-- ❌ Bad: Partial dependency (violates 2NF)
|
||||||
|
CREATE TABLE flyer_items (
|
||||||
|
flyer_id BIGINT,
|
||||||
|
item_id BIGINT,
|
||||||
|
store_name TEXT, -- Depends only on flyer_id, not (flyer_id, item_id)
|
||||||
|
PRIMARY KEY (flyer_id, item_id)
|
||||||
|
);
|
||||||
|
```
|
||||||
|
|
||||||
|
### 3. Third Normal Form (3NF) ⚠️ VIOLATED IN API LAYER
|
||||||
|
|
||||||
|
**Rule:** No transitive dependencies; non-key columns depend only on the primary key, not on other non-key columns.
|
||||||
|
|
||||||
|
**Status:** ⚠️ **Database is compliant, but API layer violates this principle**
|
||||||
|
|
||||||
|
**Database Schema (Correct):**
|
||||||
|
|
||||||
|
```sql
|
||||||
|
-- ✅ Categories are normalized
|
||||||
|
CREATE TABLE categories (
|
||||||
|
category_id BIGINT PRIMARY KEY,
|
||||||
|
name TEXT NOT NULL UNIQUE
|
||||||
|
);
|
||||||
|
|
||||||
|
CREATE TABLE master_grocery_items (
|
||||||
|
master_grocery_item_id BIGINT PRIMARY KEY,
|
||||||
|
name TEXT,
|
||||||
|
category_id BIGINT REFERENCES categories(category_id) -- Direct reference
|
||||||
|
);
|
||||||
|
```
|
||||||
|
|
||||||
|
**API Layer (Violates 3NF Principle):**
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// ❌ API accepts category name instead of ID
|
||||||
|
POST /api/users/watched-items
|
||||||
|
{
|
||||||
|
"itemName": "Milk",
|
||||||
|
"category": "Dairy & Eggs" // String! Should be category_id
|
||||||
|
}
|
||||||
|
|
||||||
|
// Service layer must denormalize by doing lookup:
|
||||||
|
SELECT category_id FROM categories WHERE name = $1
|
||||||
|
```
|
||||||
|
|
||||||
|
This creates a **transitive dependency** in the application layer:
|
||||||
|
|
||||||
|
- `watched_item` → `category_name` → `category_id`
|
||||||
|
- Instead of direct: `watched_item` → `category_id`
|
||||||
|
|
||||||
|
### 4. Boyce-Codd Normal Form (BCNF) ✅ Currently Satisfied
|
||||||
|
|
||||||
|
**Rule:** Every determinant is a candidate key (stricter version of 3NF).
|
||||||
|
|
||||||
|
**Status:** ✅ **Compliant**
|
||||||
|
|
||||||
|
- All foreign key references use primary keys
|
||||||
|
- No non-trivial functional dependencies where determinant is not a superkey
|
||||||
|
|
||||||
|
### 5. Fourth Normal Form (4NF) ✅ Currently Satisfied
|
||||||
|
|
||||||
|
**Rule:** No multi-valued dependencies; a record should not contain independent multi-valued facts.
|
||||||
|
|
||||||
|
**Status:** ✅ **Compliant**
|
||||||
|
|
||||||
|
- Junction tables properly separate many-to-many relationships
|
||||||
|
- Examples: `user_watched_items`, `shopping_list_items`, `recipe_ingredients`
|
||||||
|
|
||||||
|
### 6. Fifth Normal Form (5NF) ✅ Currently Satisfied
|
||||||
|
|
||||||
|
**Rule:** No join dependencies; tables cannot be decomposed further without loss of information.
|
||||||
|
|
||||||
|
**Status:** ✅ **Compliant** (as far as schema design goes)
|
||||||
|
|
||||||
|
## Impact of API Violation
|
||||||
|
|
||||||
|
### 1. Brittleness
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// Test fails because of exact string matching:
|
||||||
|
addWatchedItem('Milk', 'Dairy'); // ❌ Fails - not exact match
|
||||||
|
addWatchedItem('Milk', 'Dairy & Eggs'); // ✅ Works - exact match
|
||||||
|
addWatchedItem('Milk', 'dairy & eggs'); // ❌ Fails - case sensitive
|
||||||
|
```
|
||||||
|
|
||||||
|
### 2. No Discovery Mechanism
|
||||||
|
|
||||||
|
- No API endpoint to list available categories
|
||||||
|
- Frontend cannot dynamically populate dropdowns
|
||||||
|
- Clients must hardcode category names
|
||||||
|
|
||||||
|
### 3. Performance Penalty
|
||||||
|
|
||||||
|
```sql
|
||||||
|
-- Current: String lookup on every request
|
||||||
|
SELECT category_id FROM categories WHERE name = $1; -- Full table scan or index scan
|
||||||
|
|
||||||
|
-- Should be: Direct ID reference (no lookup needed)
|
||||||
|
INSERT INTO master_grocery_items (name, category_id) VALUES ($1, $2);
|
||||||
|
```
|
||||||
|
|
||||||
|
### 4. Impossible Localization
|
||||||
|
|
||||||
|
- Cannot translate category names without breaking API
|
||||||
|
- Category names are hardcoded in English
|
||||||
|
|
||||||
|
### 5. Maintenance Burden
|
||||||
|
|
||||||
|
- Renaming a category breaks all API clients
|
||||||
|
- Must coordinate name changes across frontend, tests, and documentation
|
||||||
|
|
||||||
|
## Decision
|
||||||
|
|
||||||
|
**We adopt the following principles for all API design:**
|
||||||
|
|
||||||
|
### 1. Use Numerical IDs for All Foreign Key References
|
||||||
|
|
||||||
|
**Rule:** APIs MUST accept numerical IDs when creating relationships between entities.
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// ✅ CORRECT: Use IDs
|
||||||
|
POST /api/users/watched-items
|
||||||
|
{
|
||||||
|
"itemName": "Milk",
|
||||||
|
"category_id": 3 // Numerical ID
|
||||||
|
}
|
||||||
|
|
||||||
|
// ❌ INCORRECT: Use strings
|
||||||
|
POST /api/users/watched-items
|
||||||
|
{
|
||||||
|
"itemName": "Milk",
|
||||||
|
"category": "Dairy & Eggs" // String name
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### 2. Provide Discovery Endpoints
|
||||||
|
|
||||||
|
**Rule:** For any entity referenced by ID, provide a GET endpoint to list available options.
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// Required: Category discovery endpoint
|
||||||
|
GET / api / categories;
|
||||||
|
Response: [
|
||||||
|
{ category_id: 1, name: 'Fruits & Vegetables' },
|
||||||
|
{ category_id: 2, name: 'Meat & Seafood' },
|
||||||
|
{ category_id: 3, name: 'Dairy & Eggs' },
|
||||||
|
];
|
||||||
|
```
|
||||||
|
|
||||||
|
### 3. Support Lookup by Name (Optional)
|
||||||
|
|
||||||
|
**Rule:** If convenient, provide query parameters for name-based lookup, but use IDs internally.
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// Optional: Convenience endpoint
|
||||||
|
GET /api/categories?name=Dairy%20%26%20Eggs
|
||||||
|
Response: { "category_id": 3, "name": "Dairy & Eggs" }
|
||||||
|
```
|
||||||
|
|
||||||
|
### 4. Return Full Objects in Responses
|
||||||
|
|
||||||
|
**Rule:** API responses SHOULD include denormalized data for convenience, but inputs MUST use IDs.
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// ✅ Response includes category details
|
||||||
|
GET / api / users / watched - items;
|
||||||
|
Response: [
|
||||||
|
{
|
||||||
|
master_grocery_item_id: 42,
|
||||||
|
name: 'Milk',
|
||||||
|
category_id: 3,
|
||||||
|
category: {
|
||||||
|
// ✅ Include full object in response
|
||||||
|
category_id: 3,
|
||||||
|
name: 'Dairy & Eggs',
|
||||||
|
},
|
||||||
|
},
|
||||||
|
];
|
||||||
|
```
|
||||||
|
|
||||||
|
## Affected Areas
|
||||||
|
|
||||||
|
### Immediate Violations (Must Fix)
|
||||||
|
|
||||||
|
1. **User Watched Items** ([src/routes/user.routes.ts:76](../../src/routes/user.routes.ts))
|
||||||
|
- Currently: `category: string`
|
||||||
|
- Should be: `category_id: number`
|
||||||
|
|
||||||
|
2. **Service Layer** ([src/services/db/personalization.db.ts:175](../../src/services/db/personalization.db.ts))
|
||||||
|
- Currently: `categoryName: string`
|
||||||
|
- Should be: `categoryId: number`
|
||||||
|
|
||||||
|
3. **API Client** ([src/services/apiClient.ts:436](../../src/services/apiClient.ts))
|
||||||
|
- Currently: `category: string`
|
||||||
|
- Should be: `category_id: number`
|
||||||
|
|
||||||
|
4. **Frontend Hooks** ([src/hooks/mutations/useAddWatchedItemMutation.ts:9](../../src/hooks/mutations/useAddWatchedItemMutation.ts))
|
||||||
|
- Currently: `category?: string`
|
||||||
|
- Should be: `category_id: number`
|
||||||
|
|
||||||
|
### Potential Violations (Review Required)
|
||||||
|
|
||||||
|
1. **UPC/Barcode System** ([src/types/upc.ts:85](../../src/types/upc.ts))
|
||||||
|
- Uses `category: string | null`
|
||||||
|
- May be appropriate if category is free-form user input
|
||||||
|
|
||||||
|
2. **AI Extraction** ([src/types/ai.ts:21](../../src/types/ai.ts))
|
||||||
|
- Uses `category_name: z.string()`
|
||||||
|
- AI extracts category names, needs mapping to IDs
|
||||||
|
|
||||||
|
3. **Flyer Data Transformer** ([src/services/flyerDataTransformer.ts:40](../../src/services/flyerDataTransformer.ts))
|
||||||
|
- Uses `category_name: string`
|
||||||
|
- May need category matching/creation logic
|
||||||
|
|
||||||
|
## Migration Strategy
|
||||||
|
|
||||||
|
See [research-category-id-migration.md](../research-category-id-migration.md) for detailed migration plan.
|
||||||
|
|
||||||
|
**High-level approach:**
|
||||||
|
|
||||||
|
1. **Phase 1: Add category discovery endpoint** (non-breaking)
|
||||||
|
- `GET /api/categories`
|
||||||
|
- No API changes yet
|
||||||
|
|
||||||
|
2. **Phase 2: Support both formats** (non-breaking)
|
||||||
|
- Accept both `category` (string) and `category_id` (number)
|
||||||
|
- Deprecate string format with warning logs
|
||||||
|
|
||||||
|
3. **Phase 3: Remove string support** (breaking change, major version bump)
|
||||||
|
- Only accept `category_id`
|
||||||
|
- Update all clients and tests
|
||||||
|
|
||||||
|
## Consequences
|
||||||
|
|
||||||
|
### Positive
|
||||||
|
|
||||||
|
- ✅ API matches database schema design
|
||||||
|
- ✅ More robust (no typo-based failures)
|
||||||
|
- ✅ Better performance (no string lookups)
|
||||||
|
- ✅ Enables localization
|
||||||
|
- ✅ Discoverable via REST API
|
||||||
|
- ✅ Follows REST best practices
|
||||||
|
|
||||||
|
### Negative
|
||||||
|
|
||||||
|
- ⚠️ Breaking change for existing API consumers
|
||||||
|
- ⚠️ Requires client updates
|
||||||
|
- ⚠️ More complex migration path
|
||||||
|
|
||||||
|
### Neutral
|
||||||
|
|
||||||
|
- Frontend must fetch categories before displaying form
|
||||||
|
- Slightly more initial API calls (one-time category fetch)
|
||||||
|
|
||||||
|
## References
|
||||||
|
|
||||||
|
- [Database Normalization (Wikipedia)](https://en.wikipedia.org/wiki/Database_normalization)
|
||||||
|
- [REST API Design Best Practices](https://stackoverflow.blog/2020/03/02/best-practices-for-rest-api-design/)
|
||||||
|
- [PostgreSQL Foreign Keys](https://www.postgresql.org/docs/current/ddl-constraints.html#DDL-CONSTRAINTS-FK)
|
||||||
|
|
||||||
|
## Related Decisions
|
||||||
|
|
||||||
|
- [ADR-001: Database Schema Design](./0001-database-schema-design.md) (if exists)
|
||||||
|
- [ADR-014: Containerization and Deployment Strategy](./0014-containerization-and-deployment-strategy.md)
|
||||||
|
|
||||||
|
## Approval
|
||||||
|
|
||||||
|
- **Proposed by:** Claude Code (via user observation)
|
||||||
|
- **Date:** 2026-01-19
|
||||||
|
- **Status:** Accepted (pending implementation)
|
||||||
1029
docs/research-category-id-migration.md
Normal file
1029
docs/research-category-id-migration.md
Normal file
File diff suppressed because it is too large
Load Diff
232
docs/research-e2e-test-separation.md
Normal file
232
docs/research-e2e-test-separation.md
Normal file
@@ -0,0 +1,232 @@
|
|||||||
|
# Research: Separating E2E Tests from Integration Tests
|
||||||
|
|
||||||
|
**Date:** 2026-01-19
|
||||||
|
**Status:** In Progress
|
||||||
|
**Context:** E2E tests exist with their own config but are not being run separately
|
||||||
|
|
||||||
|
## Current State
|
||||||
|
|
||||||
|
### Test Structure
|
||||||
|
|
||||||
|
- **Unit tests**: `src/tests/unit/` (but most are co-located with source files)
|
||||||
|
- **Integration tests**: `src/tests/integration/` (28 test files)
|
||||||
|
- **E2E tests**: `src/tests/e2e/` (11 test files) **← NOT CURRENTLY RUNNING**
|
||||||
|
|
||||||
|
### Configurations
|
||||||
|
|
||||||
|
| Config File | Project Name | Environment | Port | Include Pattern |
|
||||||
|
| ------------------------------ | ------------- | ----------- | ---- | ------------------------------------------ |
|
||||||
|
| `vite.config.ts` | `unit` | jsdom | N/A | Component/hook tests |
|
||||||
|
| `vitest.config.integration.ts` | `integration` | node | 3099 | `src/tests/integration/**/*.test.{ts,tsx}` |
|
||||||
|
| `vitest.config.e2e.ts` | `e2e` | node | 3098 | `src/tests/e2e/**/*.e2e.test.ts` |
|
||||||
|
|
||||||
|
### Workspace Configuration
|
||||||
|
|
||||||
|
**`vitest.workspace.ts` currently includes:**
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
export default [
|
||||||
|
'vite.config.ts', // Unit tests
|
||||||
|
'vitest.config.integration.ts', // Integration tests
|
||||||
|
// ❌ vitest.config.e2e.ts is NOT included!
|
||||||
|
];
|
||||||
|
```
|
||||||
|
|
||||||
|
### NPM Scripts
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"test": "node scripts/check-linux.js && cross-env NODE_ENV=test tsx ./node_modules/vitest/vitest.mjs run",
|
||||||
|
"test:unit": "... --project unit ...",
|
||||||
|
"test:integration": "... --project integration ..."
|
||||||
|
// ❌ NO test:e2e script exists!
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### CI/CD Status
|
||||||
|
|
||||||
|
**`.gitea/workflows/deploy-to-test.yml` runs:**
|
||||||
|
|
||||||
|
- ✅ `npm run test:unit -- --coverage`
|
||||||
|
- ✅ `npm run test:integration -- --coverage`
|
||||||
|
- ❌ E2E tests are NOT run in CI
|
||||||
|
|
||||||
|
## Key Findings
|
||||||
|
|
||||||
|
### 1. E2E Tests Are Orphaned
|
||||||
|
|
||||||
|
- 11 E2E test files exist but are never executed
|
||||||
|
- E2E config file exists (`vitest.config.e2e.ts`) but is not referenced anywhere
|
||||||
|
- No npm script to run E2E tests
|
||||||
|
- Not included in vitest workspace
|
||||||
|
- Not run in CI/CD pipeline
|
||||||
|
|
||||||
|
### 2. When Were E2E Tests Created?
|
||||||
|
|
||||||
|
Git history shows E2E config was added in commit `e66027d` ("fix e2e and deploy to prod"), but:
|
||||||
|
|
||||||
|
- It was never added to the workspace
|
||||||
|
- It was never added to CI
|
||||||
|
- No test:e2e script was created
|
||||||
|
|
||||||
|
This suggests the E2E separation was **started but never completed**.
|
||||||
|
|
||||||
|
### 3. How Are Tests Currently Run?
|
||||||
|
|
||||||
|
**Locally:**
|
||||||
|
|
||||||
|
- `npm test` → runs workspace (unit + integration only)
|
||||||
|
- `npm run test:unit` → runs only unit tests
|
||||||
|
- `npm run test:integration` → runs only integration tests
|
||||||
|
- E2E tests: **Not accessible via any command**
|
||||||
|
|
||||||
|
**In CI:**
|
||||||
|
|
||||||
|
- Only `test:unit` and `test:integration` are run
|
||||||
|
- E2E tests are never executed
|
||||||
|
|
||||||
|
### 4. Port Allocation
|
||||||
|
|
||||||
|
- Integration tests: Port 3099
|
||||||
|
- E2E tests: Port 3098 (configured but never used)
|
||||||
|
- No conflicts if both run sequentially
|
||||||
|
|
||||||
|
## E2E Test Files (11 total)
|
||||||
|
|
||||||
|
1. `admin-authorization.e2e.test.ts`
|
||||||
|
2. `admin-dashboard.e2e.test.ts`
|
||||||
|
3. `auth.e2e.test.ts`
|
||||||
|
4. `budget-journey.e2e.test.ts`
|
||||||
|
5. `deals-journey.e2e.test.ts` ← Just fixed URL constraint issue
|
||||||
|
6. `error-reporting.e2e.test.ts`
|
||||||
|
7. `flyer-upload.e2e.test.ts`
|
||||||
|
8. `inventory-journey.e2e.test.ts`
|
||||||
|
9. `receipt-journey.e2e.test.ts`
|
||||||
|
10. `upc-journey.e2e.test.ts`
|
||||||
|
11. `user-journey.e2e.test.ts`
|
||||||
|
|
||||||
|
## Problems to Solve
|
||||||
|
|
||||||
|
### Immediate Issues
|
||||||
|
|
||||||
|
1. **E2E tests are not running** - Code exists but is never executed
|
||||||
|
2. **No way to run E2E tests** - No npm script or CI job
|
||||||
|
3. **Coverage gaps** - E2E scenarios are untested in practice
|
||||||
|
4. **False sense of security** - Team may think E2E tests are running
|
||||||
|
|
||||||
|
### Implementation Challenges
|
||||||
|
|
||||||
|
#### 1. Adding E2E to Workspace
|
||||||
|
|
||||||
|
**Option A: Add to workspace**
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// vitest.workspace.ts
|
||||||
|
export default [
|
||||||
|
'vite.config.ts',
|
||||||
|
'vitest.config.integration.ts',
|
||||||
|
'vitest.config.e2e.ts', // ← Add this
|
||||||
|
];
|
||||||
|
```
|
||||||
|
|
||||||
|
**Impact:** E2E tests would run with `npm test`, increasing test time significantly
|
||||||
|
|
||||||
|
**Option B: Keep separate**
|
||||||
|
|
||||||
|
- E2E remains outside workspace
|
||||||
|
- Requires explicit `npm run test:e2e` command
|
||||||
|
- CI would need separate step for E2E tests
|
||||||
|
|
||||||
|
#### 2. Adding NPM Script
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"test:e2e": "node scripts/check-linux.js && cross-env NODE_ENV=test tsx --max-old-space-size=8192 ./node_modules/vitest/vitest.mjs run --project e2e -c vitest.config.e2e.ts"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Dependencies:**
|
||||||
|
|
||||||
|
- Uses same global setup pattern as integration tests
|
||||||
|
- Requires server to be stopped first (like integration tests)
|
||||||
|
- Port 3098 must be available
|
||||||
|
|
||||||
|
#### 3. CI/CD Integration
|
||||||
|
|
||||||
|
**Add to `.gitea/workflows/deploy-to-test.yml`:**
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
- name: Run E2E Tests
|
||||||
|
run: |
|
||||||
|
npm run test:e2e -- --coverage \
|
||||||
|
--reporter=verbose \
|
||||||
|
--includeTaskLocation \
|
||||||
|
--testTimeout=120000 \
|
||||||
|
--silent=passed-only
|
||||||
|
```
|
||||||
|
|
||||||
|
**Questions:**
|
||||||
|
|
||||||
|
- Should E2E run before or after integration tests?
|
||||||
|
- Should E2E failures block deployment?
|
||||||
|
- Should E2E have separate coverage reports?
|
||||||
|
|
||||||
|
#### 4. Test Organization Questions
|
||||||
|
|
||||||
|
- Are current "integration" tests actually E2E tests?
|
||||||
|
- Should some E2E tests be moved to integration?
|
||||||
|
- What's the distinction between integration and E2E in this project?
|
||||||
|
|
||||||
|
#### 5. Coverage Implications
|
||||||
|
|
||||||
|
- E2E tests have separate coverage directory: `.coverage/e2e`
|
||||||
|
- Integration tests: `.coverage/integration`
|
||||||
|
- How to merge coverage from all test types?
|
||||||
|
- Do we need combined coverage reports?
|
||||||
|
|
||||||
|
## Recommended Approach
|
||||||
|
|
||||||
|
### Phase 1: Quick Fix (Enable E2E Tests)
|
||||||
|
|
||||||
|
1. ✅ Fix any failing E2E tests (like URL constraints)
|
||||||
|
2. Add `test:e2e` npm script
|
||||||
|
3. Document how to run E2E tests manually
|
||||||
|
4. Do NOT add to workspace yet (keep separate)
|
||||||
|
|
||||||
|
### Phase 2: CI Integration
|
||||||
|
|
||||||
|
1. Add E2E test step to `.gitea/workflows/deploy-to-test.yml`
|
||||||
|
2. Run after integration tests pass
|
||||||
|
3. Allow failures initially (monitor results)
|
||||||
|
4. Make blocking once stable
|
||||||
|
|
||||||
|
### Phase 3: Optimize
|
||||||
|
|
||||||
|
1. Review test categorization (integration vs E2E)
|
||||||
|
2. Consider adding to workspace if test time is acceptable
|
||||||
|
3. Merge coverage reports if needed
|
||||||
|
4. Document test strategy in testing docs
|
||||||
|
|
||||||
|
## Next Steps
|
||||||
|
|
||||||
|
1. **Create `test:e2e` script** in package.json
|
||||||
|
2. **Run E2E tests manually** to verify they work
|
||||||
|
3. **Fix any failing E2E tests**
|
||||||
|
4. **Document E2E testing** in TESTING.md
|
||||||
|
5. **Add to CI** once stable
|
||||||
|
6. **Consider workspace integration** after CI is stable
|
||||||
|
|
||||||
|
## Questions for Team
|
||||||
|
|
||||||
|
1. Why were E2E tests never fully integrated?
|
||||||
|
2. Should E2E tests run on every commit or separately?
|
||||||
|
3. What's the acceptable test time for local development?
|
||||||
|
4. Should we run E2E tests in parallel or sequentially with integration?
|
||||||
|
|
||||||
|
## Related Files
|
||||||
|
|
||||||
|
- `vitest.workspace.ts` - Workspace configuration
|
||||||
|
- `vitest.config.e2e.ts` - E2E test configuration
|
||||||
|
- `src/tests/setup/e2e-global-setup.ts` - E2E global setup
|
||||||
|
- `.gitea/workflows/deploy-to-test.yml` - CI pipeline
|
||||||
|
- `package.json` - NPM scripts
|
||||||
@@ -14,6 +14,7 @@
|
|||||||
"test:coverage": "npm run clean && npm run test:unit -- --coverage && npm run test:integration -- --coverage",
|
"test:coverage": "npm run clean && npm run test:unit -- --coverage && npm run test:integration -- --coverage",
|
||||||
"test:unit": "node scripts/check-linux.js && cross-env NODE_ENV=test tsx --max-old-space-size=8192 ./node_modules/vitest/vitest.mjs run --project unit -c vite.config.ts",
|
"test:unit": "node scripts/check-linux.js && cross-env NODE_ENV=test tsx --max-old-space-size=8192 ./node_modules/vitest/vitest.mjs run --project unit -c vite.config.ts",
|
||||||
"test:integration": "node scripts/check-linux.js && cross-env NODE_ENV=test tsx --max-old-space-size=8192 ./node_modules/vitest/vitest.mjs run --project integration -c vitest.config.integration.ts",
|
"test:integration": "node scripts/check-linux.js && cross-env NODE_ENV=test tsx --max-old-space-size=8192 ./node_modules/vitest/vitest.mjs run --project integration -c vitest.config.integration.ts",
|
||||||
|
"test:e2e": "node scripts/check-linux.js && cross-env NODE_ENV=test tsx --max-old-space-size=8192 ./node_modules/vitest/vitest.mjs run --config vitest.config.e2e.ts",
|
||||||
"format": "prettier --write .",
|
"format": "prettier --write .",
|
||||||
"lint": "eslint . --ext ts,tsx --report-unused-disable-directives --max-warnings 0",
|
"lint": "eslint . --ext ts,tsx --report-unused-disable-directives --max-warnings 0",
|
||||||
"type-check": "tsc --noEmit",
|
"type-check": "tsc --noEmit",
|
||||||
|
|||||||
@@ -78,7 +78,7 @@ describe('ReceiptRepository', () => {
|
|||||||
const receiptRow = {
|
const receiptRow = {
|
||||||
receipt_id: 2,
|
receipt_id: 2,
|
||||||
user_id: 'user-1',
|
user_id: 'user-1',
|
||||||
store_id: null,
|
store_location_id: null,
|
||||||
receipt_image_url: '/uploads/receipts/receipt-2.jpg',
|
receipt_image_url: '/uploads/receipts/receipt-2.jpg',
|
||||||
transaction_date: null,
|
transaction_date: null,
|
||||||
total_amount_cents: null,
|
total_amount_cents: null,
|
||||||
|
|||||||
@@ -60,11 +60,13 @@ describe('E2E Deals and Price Tracking Journey', () => {
|
|||||||
await pool.query('DELETE FROM public.user_watched_items WHERE user_id = $1', [userId]);
|
await pool.query('DELETE FROM public.user_watched_items WHERE user_id = $1', [userId]);
|
||||||
}
|
}
|
||||||
|
|
||||||
// Clean up flyer items
|
// Clean up flyer items (disable trigger to avoid issues with NULL master_item_id)
|
||||||
if (createdFlyerIds.length > 0) {
|
if (createdFlyerIds.length > 0) {
|
||||||
|
await pool.query('ALTER TABLE public.flyer_items DISABLE TRIGGER ALL');
|
||||||
await pool.query('DELETE FROM public.flyer_items WHERE flyer_id = ANY($1::bigint[])', [
|
await pool.query('DELETE FROM public.flyer_items WHERE flyer_id = ANY($1::bigint[])', [
|
||||||
createdFlyerIds,
|
createdFlyerIds,
|
||||||
]);
|
]);
|
||||||
|
await pool.query('ALTER TABLE public.flyer_items ENABLE TRIGGER ALL');
|
||||||
}
|
}
|
||||||
|
|
||||||
// Clean up flyers
|
// Clean up flyers
|
||||||
@@ -166,7 +168,7 @@ describe('E2E Deals and Price Tracking Journey', () => {
|
|||||||
|
|
||||||
const flyer1Result = await pool.query(
|
const flyer1Result = await pool.query(
|
||||||
`INSERT INTO public.flyers (store_id, file_name, image_url, icon_url, valid_from, valid_to, status)
|
`INSERT INTO public.flyers (store_id, file_name, image_url, icon_url, valid_from, valid_to, status)
|
||||||
VALUES ($1, 'e2e-flyer-1.jpg', '/uploads/flyers/e2e-flyer-1.jpg', '/uploads/flyers/e2e-flyer-1-icon.jpg', $2, $3, 'processed')
|
VALUES ($1, 'e2e-flyer-1.jpg', 'http://localhost:3000/uploads/flyers/e2e-flyer-1.jpg', 'http://localhost:3000/uploads/flyers/e2e-flyer-1-icon.jpg', $2, $3, 'processed')
|
||||||
RETURNING flyer_id`,
|
RETURNING flyer_id`,
|
||||||
[store1Id, validFrom, validTo],
|
[store1Id, validFrom, validTo],
|
||||||
);
|
);
|
||||||
@@ -175,7 +177,7 @@ describe('E2E Deals and Price Tracking Journey', () => {
|
|||||||
|
|
||||||
const flyer2Result = await pool.query(
|
const flyer2Result = await pool.query(
|
||||||
`INSERT INTO public.flyers (store_id, file_name, image_url, icon_url, valid_from, valid_to, status)
|
`INSERT INTO public.flyers (store_id, file_name, image_url, icon_url, valid_from, valid_to, status)
|
||||||
VALUES ($1, 'e2e-flyer-2.jpg', '/uploads/flyers/e2e-flyer-2.jpg', '/uploads/flyers/e2e-flyer-2-icon.jpg', $2, $3, 'processed')
|
VALUES ($1, 'e2e-flyer-2.jpg', 'http://localhost:3000/uploads/flyers/e2e-flyer-2.jpg', 'http://localhost:3000/uploads/flyers/e2e-flyer-2-icon.jpg', $2, $3, 'processed')
|
||||||
RETURNING flyer_id`,
|
RETURNING flyer_id`,
|
||||||
[store2Id, validFrom, validTo],
|
[store2Id, validFrom, validTo],
|
||||||
);
|
);
|
||||||
@@ -184,26 +186,26 @@ describe('E2E Deals and Price Tracking Journey', () => {
|
|||||||
|
|
||||||
// Add items to flyers with prices (Store 1 - higher prices)
|
// Add items to flyers with prices (Store 1 - higher prices)
|
||||||
await pool.query(
|
await pool.query(
|
||||||
`INSERT INTO public.flyer_items (flyer_id, master_item_id, sale_price_cents, page_number)
|
`INSERT INTO public.flyer_items (flyer_id, master_item_id, price_in_cents, item, price_display, quantity)
|
||||||
VALUES
|
VALUES
|
||||||
($1, $2, 599, 1), -- Milk at $5.99
|
($1, $2, 599, 'Milk', '$5.99', 'each'), -- Milk at $5.99
|
||||||
($1, $3, 349, 1), -- Bread at $3.49
|
($1, $3, 349, 'Bread', '$3.49', 'each'), -- Bread at $3.49
|
||||||
($1, $4, 1299, 2), -- Coffee at $12.99
|
($1, $4, 1299, 'Coffee', '$12.99', 'each'), -- Coffee at $12.99
|
||||||
($1, $5, 299, 2), -- Bananas at $2.99
|
($1, $5, 299, 'Bananas', '$2.99', 'lb'), -- Bananas at $2.99
|
||||||
($1, $6, 899, 3) -- Chicken at $8.99
|
($1, $6, 899, 'Chicken', '$8.99', 'lb') -- Chicken at $8.99
|
||||||
`,
|
`,
|
||||||
[flyer1Id, ...createdMasterItemIds],
|
[flyer1Id, ...createdMasterItemIds],
|
||||||
);
|
);
|
||||||
|
|
||||||
// Add items to flyers with prices (Store 2 - better prices)
|
// Add items to flyers with prices (Store 2 - better prices)
|
||||||
await pool.query(
|
await pool.query(
|
||||||
`INSERT INTO public.flyer_items (flyer_id, master_item_id, sale_price_cents, page_number)
|
`INSERT INTO public.flyer_items (flyer_id, master_item_id, price_in_cents, item, price_display, quantity)
|
||||||
VALUES
|
VALUES
|
||||||
($1, $2, 499, 1), -- Milk at $4.99 (BEST PRICE)
|
($1, $2, 499, 'Milk', '$4.99', 'each'), -- Milk at $4.99 (BEST PRICE)
|
||||||
($1, $3, 299, 1), -- Bread at $2.99 (BEST PRICE)
|
($1, $3, 299, 'Bread', '$2.99', 'each'), -- Bread at $2.99 (BEST PRICE)
|
||||||
($1, $4, 1099, 2), -- Coffee at $10.99 (BEST PRICE)
|
($1, $4, 1099, 'Coffee', '$10.99', 'each'), -- Coffee at $10.99 (BEST PRICE)
|
||||||
($1, $5, 249, 2), -- Bananas at $2.49 (BEST PRICE)
|
($1, $5, 249, 'Bananas', '$2.49', 'lb'), -- Bananas at $2.49 (BEST PRICE)
|
||||||
($1, $6, 799, 3) -- Chicken at $7.99 (BEST PRICE)
|
($1, $6, 799, 'Chicken', '$7.99', 'lb') -- Chicken at $7.99 (BEST PRICE)
|
||||||
`,
|
`,
|
||||||
[flyer2Id, ...createdMasterItemIds],
|
[flyer2Id, ...createdMasterItemIds],
|
||||||
);
|
);
|
||||||
@@ -214,7 +216,7 @@ describe('E2E Deals and Price Tracking Journey', () => {
|
|||||||
token: authToken,
|
token: authToken,
|
||||||
body: JSON.stringify({
|
body: JSON.stringify({
|
||||||
itemName: 'E2E Milk 2%',
|
itemName: 'E2E Milk 2%',
|
||||||
category: 'Dairy',
|
category: 'Dairy & Eggs',
|
||||||
}),
|
}),
|
||||||
});
|
});
|
||||||
|
|
||||||
@@ -224,7 +226,7 @@ describe('E2E Deals and Price Tracking Journey', () => {
|
|||||||
|
|
||||||
// Add more items to watch list
|
// Add more items to watch list
|
||||||
const itemsToWatch = [
|
const itemsToWatch = [
|
||||||
{ itemName: 'E2E Bread White', category: 'Bakery' },
|
{ itemName: 'E2E Bread White', category: 'Bakery & Bread' },
|
||||||
{ itemName: 'E2E Coffee Beans', category: 'Beverages' },
|
{ itemName: 'E2E Coffee Beans', category: 'Beverages' },
|
||||||
];
|
];
|
||||||
|
|
||||||
@@ -252,7 +254,7 @@ describe('E2E Deals and Price Tracking Journey', () => {
|
|||||||
(item: { item_name: string }) => item.item_name === 'E2E Milk 2%',
|
(item: { item_name: string }) => item.item_name === 'E2E Milk 2%',
|
||||||
);
|
);
|
||||||
expect(watchedMilk).toBeDefined();
|
expect(watchedMilk).toBeDefined();
|
||||||
expect(watchedMilk.category).toBe('Dairy');
|
expect(watchedMilk.category).toBe('Dairy & Eggs');
|
||||||
|
|
||||||
// Step 6: Get best prices for watched items
|
// Step 6: Get best prices for watched items
|
||||||
const bestPricesResponse = await authedFetch('/users/deals/best-watched-prices', {
|
const bestPricesResponse = await authedFetch('/users/deals/best-watched-prices', {
|
||||||
|
|||||||
Reference in New Issue
Block a user