torbo/flyer-crawler.projectium.com

Files

Torben Sorensen d2efca8339 massive fixes to stores and addresses

2026-01-19 00:33:09 -08:00

17 KiB

Raw Blame History

Store Address Normalization Implementation Plan

Executive Summary

Problem: The database schema has a properly normalized structure for stores and addresses (stores → store_locations → addresses), but the application code does NOT fully utilize this structure. Currently:

TypeScript types exist (Store, Address, StoreLocation) ✅
AddressRepository exists for basic CRUD ✅
E2E tests now create data using normalized structure ✅
BUT: No functionality to CREATE/MANAGE stores with addresses in the application
BUT: No API endpoints to handle store location data
BUT: No frontend forms to input address data when creating stores
BUT: Queries don't join stores with their addresses for display

Impact: Users see stores without addresses, making features like "deals near me", "store finder", and location-based features impossible.

Current State Analysis

✅ What EXISTS and WORKS:

Database Schema: Properly normalized (stores, addresses, store_locations)
TypeScript Types (src/types.ts):
- Store type (lines 2-9)
- Address type (lines 712-724)
- StoreLocation type (lines 704-710)
AddressRepository (src/services/db/address.db.ts):
- getAddressById()
- upsertAddress()
Test Helpers (src/tests/utils/storeHelpers.ts):
- createStoreWithLocation() - for test data creation
- cleanupStoreLocations() - for test cleanup

❌ What's MISSING:

No StoreRepository/StoreService - No database layer for stores
No StoreLocationRepository - No functions to link stores to addresses
No API endpoints for:
- POST /api/stores - Create store with address
- GET /api/stores/:id - Get store with address(es)
- PUT /api/stores/:id - Update store details
- POST /api/stores/:id/locations - Add location to store
- etc.
No frontend components for:
- Store creation form (with address fields)
- Store editing form
- Store location display
Queries don't join - Existing queries (admin.db.ts, flyer.db.ts) join stores but don't include address data
No store management UI - Admin dashboard doesn't have store management

Detailed Investigation Findings

Places Where Stores Are Used (Need Address Data):

Flyer Display (src/features/flyer/FlyerDisplay.tsx)
- Shows store name, but could show "Store @ 123 Main St, Toronto"
Deal Listings (deals.db.ts queries)
- deal_store_name field exists (line 691 in types.ts)
- Should show "Milk $4.99 @ Store #123 (456 Oak Ave)"
Receipt Processing (receipt.db.ts)
- Receipts link to store_id
- Could show "Receipt from Store @ 789 Budget St"
Admin Dashboard (admin.db.ts)
- Joins stores for flyer review (line 720)
- Should show store address in admin views
Flyer Item Analysis (admin.db.ts line 334)
- Joins stores for unmatched items
- Address context would help with store identification

Test Files That Need Updates:

Unit Tests (may need store+address mocks):

src/services/db/flyer.db.test.ts
src/services/db/receipt.db.test.ts
src/services/aiService.server.test.ts
src/features/flyer/*.test.tsx (various component tests)

Integration Tests (create stores):

src/tests/integration/admin.integration.test.ts (line 164: INSERT INTO stores)
src/tests/integration/flyer.integration.test.ts (line 28: INSERT INTO stores)
src/tests/integration/price.integration.test.ts (line 48: INSERT INTO stores)
src/tests/integration/public.routes.integration.test.ts (line 66: INSERT INTO stores)
src/tests/integration/receipt.integration.test.ts (line 252: INSERT INTO stores)

E2E Tests (already fixed):

✅ src/tests/e2e/deals-journey.e2e.test.ts
✅ src/tests/e2e/budget-journey.e2e.test.ts
✅ src/tests/e2e/receipt-journey.e2e.test.ts

Implementation Plan (NO CODE YET - APPROVAL REQUIRED)

Phase 1: Database Layer (Foundation)

1.1 Create StoreRepository (src/services/db/store.db.ts)

Functions needed:

getStoreById(storeId) - Returns Store (basic)
getStoreWithLocations(storeId) - Returns Store + Address[]
getAllStores() - Returns Store[] (basic)
getAllStoresWithLocations() - Returns Array<Store & {locations: Address[]}>
createStore(name, logoUrl?, createdBy?) - Returns storeId
updateStore(storeId, updates) - Updates name/logo
deleteStore(storeId) - Cascades to store_locations
searchStoresByName(query) - For autocomplete

Test file: src/services/db/store.db.test.ts

1.2 Create StoreLocationRepository (src/services/db/storeLocation.db.ts)

Functions needed:

createStoreLocation(storeId, addressId) - Links store to address
getLocationsByStoreId(storeId) - Returns StoreLocation[] with Address data
deleteStoreLocation(storeLocationId) - Unlinks
updateStoreLocation(storeLocationId, newAddressId) - Changes address

Test file: src/services/db/storeLocation.db.test.ts

1.3 Enhance AddressRepository (src/services/db/address.db.ts)

Add functions:

searchAddressesByText(query) - For autocomplete
getAddressesByStoreId(storeId) - Convenience method

Files to modify:

Phase 2: TypeScript Types & Validation

2.1 Add Extended Types (src/types.ts)

// Store with address data for API responses
export interface StoreWithLocation {
  ...Store;
  locations: Array<{
    store_location_id: number;
    address: Address;
  }>;
}

// For API requests when creating store
export interface CreateStoreRequest {
  name: string;
  logo_url?: string;
  address?: {
    address_line_1: string;
    city: string;
    province_state: string;
    postal_code: string;
    country?: string;
  };
}

2.2 Add Zod Validation Schemas

Create src/schemas/store.schema.ts:

createStoreSchema - Validates POST /stores body
updateStoreSchema - Validates PUT /stores/:id body
addLocationSchema - Validates POST /stores/:id/locations body

Phase 3: API Routes

3.1 Create Store Routes (src/routes/store.routes.ts)

Endpoints:

GET /api/stores - List all stores (with pagination)
- Query params: ?includeLocations=true, ?search=name
GET /api/stores/:id - Get single store with locations
POST /api/stores - Create store (optionally with address)
PUT /api/stores/:id - Update store name/logo
DELETE /api/stores/:id - Delete store (admin only)
POST /api/stores/:id/locations - Add location to store
DELETE /api/stores/:id/locations/:locationId - Remove location

Test file: src/routes/store.routes.test.ts

Permissions:

Create/Update/Delete: Admin only
Read: Public (for store listings in flyers/deals)

3.2 Update Existing Routes to Include Address Data

Files to modify:

src/routes/flyer.routes.ts - GET /flyers should include store address
src/routes/deals.routes.ts - GET /deals should include store address
src/routes/receipt.routes.ts - GET /receipts/:id should include store address

Phase 4: Update Database Queries

4.1 Modify Existing Queries to JOIN Addresses

Files to modify:

src/services/db/admin.db.ts
- Line 334: JOIN store_locations and addresses for unmatched items
- Line 720: JOIN store_locations and addresses for flyers needing review
src/services/db/flyer.db.ts
- Any query that returns flyers with store data
src/services/db/deals.db.ts
- Add address fields to deal queries

Pattern to use:

SELECT
  s.*,
  json_agg(
    json_build_object(
      'store_location_id', sl.store_location_id,
      'address', row_to_json(a.*)
    )
  ) FILTER (WHERE sl.store_location_id IS NOT NULL) as locations
FROM stores s
LEFT JOIN store_locations sl ON s.store_id = sl.store_id
LEFT JOIN addresses a ON sl.address_id = a.address_id
GROUP BY s.store_id

Phase 5: Frontend Components

5.1 Admin Store Management

Create src/pages/admin/components/AdminStoreManager.tsx:

Table listing all stores with locations
Create store button → opens modal/form
Edit store button → opens modal with store+address data
Delete store button (with confirmation)

5.2 Store Form Component

Create src/features/store/StoreForm.tsx:

Store name input
Logo URL input
Address section:
- Address line 1 (required)
- City (required)
- Province/State (required)
- Postal code (required)
- Country (default: Canada)
Reusable for create & edit

5.3 Store Display Components

Create src/features/store/StoreCard.tsx:

Shows store name + logo
Shows primary address (if exists)
"View all locations" link (if multiple)

Update existing components to use StoreCard:

Flyer listings
Deal listings
Receipt displays

5.4 Location Selector Component

Create src/features/store/LocationSelector.tsx:

Dropdown or map view
Filter stores by proximity (future: use lat/long)
Used in "Find deals near me" feature

Phase 6: Update Integration Tests

All integration tests that create stores need to use createStoreWithLocation():

Files to update (5 files):

src/tests/integration/admin.integration.test.ts (line 164)
src/tests/integration/flyer.integration.test.ts (line 28)
src/tests/integration/price.integration.test.ts (line 48)
src/tests/integration/public.routes.integration.test.ts (line 66)
src/tests/integration/receipt.integration.test.ts (line 252)

Change pattern:

// OLD:
const storeResult = await pool.query('INSERT INTO stores (name) VALUES ($1) RETURNING store_id', [
  'Test Store',
]);

// NEW:
import { createStoreWithLocation } from '../utils/storeHelpers';
const store = await createStoreWithLocation(pool, {
  name: 'Test Store',
  address: '123 Test St',
  city: 'Test City',
  province: 'ON',
  postalCode: 'M5V 1A1',
});
const storeId = store.storeId;

Phase 7: Update Unit Tests & Mocks

7.1 Update Mock Factories

src/tests/utils/mockFactories.ts - Add:

createMockStore(overrides?): Store
createMockAddress(overrides?): Address
createMockStoreLocation(overrides?): StoreLocation
createMockStoreWithLocation(overrides?): StoreWithLocation

7.2 Update Component Tests

Files that display stores need updated mocks:

src/features/flyer/FlyerDisplay.test.tsx
src/features/flyer/FlyerList.test.tsx
Any other components that show store data

Phase 8: Schema Migration (IF NEEDED)

Check: Do we need to migrate existing data?

If production has stores without addresses, we need to handle this
Options:
1. Make addresses optional (store can exist without location)
2. Create "Unknown Location" placeholder addresses
3. Manual data entry for existing stores

Migration file: sql/migrations/XXX_add_store_locations_data.sql (if needed)

Phase 9: Documentation & Cache Invalidation

9.1 Update API Documentation

Add store endpoints to API docs
Document request/response formats
Add examples

9.2 Cache Invalidation

src/services/cacheService.server.ts:

Add invalidateStores() method
Add invalidateStoreLocations(storeId) method
Call after create/update/delete operations

Files Summary

New Files to Create (12 files):

src/services/db/store.db.ts - Store repository
src/services/db/store.db.test.ts - Store repository tests
src/services/db/storeLocation.db.ts - StoreLocation repository
src/services/db/storeLocation.db.test.ts - StoreLocation tests
src/schemas/store.schema.ts - Validation schemas
src/routes/store.routes.ts - API endpoints
src/routes/store.routes.test.ts - Route tests
src/pages/admin/components/AdminStoreManager.tsx - Admin UI
src/features/store/StoreForm.tsx - Store creation/edit form
src/features/store/StoreCard.tsx - Display component
src/features/store/LocationSelector.tsx - Location picker
STORE_ADDRESS_IMPLEMENTATION_PLAN.md - This document

Files to Modify (20+ files):

Database Layer (3):

src/services/db/address.db.ts - Add search functions
src/services/db/admin.db.ts - Update JOINs
src/services/db/flyer.db.ts - Update JOINs
src/services/db/deals.db.ts - Update queries
src/services/db/receipt.db.ts - Update queries

API Routes (3):

src/routes/flyer.routes.ts - Include address in responses
src/routes/deals.routes.ts - Include address in responses
src/routes/receipt.routes.ts - Include address in responses

Types (1):

src/types.ts - Add StoreWithLocation and CreateStoreRequest types

Tests (10+):

src/tests/integration/admin.integration.test.ts
src/tests/integration/flyer.integration.test.ts
src/tests/integration/price.integration.test.ts
src/tests/integration/public.routes.integration.test.ts
src/tests/integration/receipt.integration.test.ts
src/tests/utils/mockFactories.ts
src/features/flyer/FlyerDisplay.test.tsx
src/features/flyer/FlyerList.test.tsx
Component tests for new store UI

Frontend (2+):

src/pages/admin/Dashboard.tsx - Add store management link
Any components displaying store data

Services (1):

src/services/cacheService.server.ts - Add store cache methods

Estimated Complexity

Low Complexity (Well-defined, straightforward):

Phase 1: Database repositories (patterns exist)
Phase 2: Type definitions (simple)
Phase 6: Update integration tests (mechanical)

Medium Complexity (Requires design decisions):

Phase 3: API routes (standard REST)
Phase 4: Update queries (SQL JOINs)
Phase 7: Update mocks (depends on types)
Phase 9: Cache invalidation (pattern exists)

High Complexity (Requires UX design, edge cases):

Phase 5: Frontend components (UI/UX decisions)
Phase 8: Data migration (if needed)
Multi-location handling (one store, many addresses)

Dependencies & Risks

Critical Dependencies:

Address data quality - garbage in, garbage out
Google Maps API integration (future) - for geocoding/validation
Multi-location handling - some stores have 100+ locations

Risks:

Breaking changes: Existing queries might break if address data is required
Performance: Joining 3 tables (stores+store_locations+addresses) could be slow
Data migration: Existing production stores have no addresses
Scope creep: "Find stores near me" leads to mapping features

Mitigation:

Make addresses OPTIONAL initially
Add database indexes on foreign keys
Use caching aggressively
Implement in phases (can stop after Phase 3 and assess)

Questions for Approval

Scope: Implement all 9 phases, or start with Phase 1-3 (backend only)?
Addresses required: Should stores REQUIRE an address, or is it optional?
Multi-location: How to handle store chains with many locations?
- Option A: One "primary" location
- Option B: All locations equal
- Option C: User selects location when viewing deals
Existing data: How to handle production stores without addresses?
Priority: Is this blocking other features, or can it wait?
Frontend design: Do we have mockups for store management UI?

Approval Checklist

Before starting implementation, confirm:

Plan reviewed and approved by project lead
Scope defined (which phases to implement)
Multi-location strategy decided
Data migration plan approved (if needed)
Frontend design approved (if doing Phase 5)
Testing strategy approved
Estimated timeline acceptable

Next Steps After Approval

Create feature branch: feature/store-address-integration
Start with Phase 1.1 (StoreRepository)
Write tests first (TDD approach)
Implement phase by phase
Request code review after each phase
Merge only after ALL tests pass

17 KiB Raw Blame History

Store Address Normalization Implementation Plan

Executive Summary

Current State Analysis

✅ What EXISTS and WORKS:

❌ What's MISSING:

Detailed Investigation Findings

Places Where Stores Are Used (Need Address Data):

Test Files That Need Updates:

Implementation Plan (NO CODE YET - APPROVAL REQUIRED)

Phase 1: Database Layer (Foundation)

1.1 Create StoreRepository (src/services/db/store.db.ts)

1.2 Create StoreLocationRepository (src/services/db/storeLocation.db.ts)

1.3 Enhance AddressRepository (src/services/db/address.db.ts)

Phase 2: TypeScript Types & Validation

2.1 Add Extended Types (src/types.ts)

2.2 Add Zod Validation Schemas

Phase 3: API Routes

3.1 Create Store Routes (src/routes/store.routes.ts)

3.2 Update Existing Routes to Include Address Data

Phase 4: Update Database Queries

4.1 Modify Existing Queries to JOIN Addresses

Phase 5: Frontend Components

5.1 Admin Store Management

5.2 Store Form Component

5.3 Store Display Components

5.4 Location Selector Component

Phase 6: Update Integration Tests

Phase 7: Update Unit Tests & Mocks

7.1 Update Mock Factories

7.2 Update Component Tests

Phase 8: Schema Migration (IF NEEDED)

Phase 9: Documentation & Cache Invalidation

9.1 Update API Documentation

9.2 Cache Invalidation

Files Summary

New Files to Create (12 files):

Files to Modify (20+ files):

Estimated Complexity

Dependencies & Risks

Questions for Approval

Approval Checklist

Next Steps After Approval

17 KiB

Raw Blame History