flyer-crawler.projectium.com/docs/SCHEMA_RELATIONSHIP_ANALYSIS.md

Database Schema Relationship Analysis

Executive Summary

This document analyzes the database schema to identify missing table relationships and JOINs that aren't properly implemented in the codebase. This analysis was triggered by discovering that WatchedItemDeal was using a store_name string instead of a proper store object with nested locations.

Key Findings

✅ CORRECTLY IMPLEMENTED

1. Store → Store Locations → Addresses (3-table normalization)

Schema:

stores (store_id) → store_locations (store_location_id) → addresses (address_id)

Implementation:

• src/services/db/storeLocation.db.ts properly JOINs all three tables
• src/types.ts defines StoreWithLocations interface with nested address objects
• Recent fixes corrected WatchedItemDeal to use store object instead of store_name string

Queries:

// From storeLocation.db.ts
FROM public.stores s
LEFT JOIN public.store_locations sl ON s.store_id = sl.store_id
LEFT JOIN public.addresses a ON sl.address_id = a.address_id

2. Shopping Trips → Shopping Trip Items

Schema:

shopping_trips (shopping_trip_id) → shopping_trip_items (shopping_trip_item_id) → master_grocery_items

Implementation:

• src/services/db/shopping.db.ts:513-518 properly JOINs shopping_trips → shopping_trip_items → master_grocery_items
• Uses json_agg to nest items array within trip object
• src/types.ts:639-647 ShoppingTrip interface includes nested items: ShoppingTripItem[]

Queries:

FROM public.shopping_trips st
LEFT JOIN public.shopping_trip_items sti ON st.shopping_trip_id = sti.shopping_trip_id
LEFT JOIN public.master_grocery_items mgi ON sti.master_item_id = mgi.master_grocery_item_id

3. Receipts → Receipt Items

Schema:

receipts (receipt_id) → receipt_items (receipt_item_id)

Implementation:

• src/types.ts:649-662 Receipt interface includes optional items?: ReceiptItem[]
• Receipt items are fetched separately via repository methods
• Proper foreign key relationship maintained

---

❌ MISSING / INCORRECT IMPLEMENTATIONS

1. CRITICAL: Flyers → Flyer Locations → Store Locations (Many-to-Many)

Schema:

CREATE TABLE IF NOT EXISTS public.flyer_locations (
    flyer_id BIGINT NOT NULL REFERENCES public.flyers(flyer_id) ON DELETE CASCADE,
    store_location_id BIGINT NOT NULL REFERENCES public.store_locations(store_location_id) ON DELETE CASCADE,
    PRIMARY KEY (flyer_id, store_location_id),
    ...
);
COMMENT: 'A linking table associating a single flyer with multiple store locations where its deals are valid.'

Problem:

• The schema defines a many-to-many relationship - a flyer can be valid at multiple store locations
• Current implementation in src/services/db/flyer.db.ts IGNORES the flyer_locations table entirely
• Queries JOIN flyers directly to stores via store_id foreign key
• This means flyers can only be associated with ONE store, not multiple locations

Current (Incorrect) Queries:

// From flyer.db.ts:315-362
FROM public.flyers f
JOIN public.stores s ON f.store_id = s.store_id  // ❌ Wrong - ignores flyer_locations

Expected (Correct) Queries:

// Should be:
FROM public.flyers f
JOIN public.flyer_locations fl ON f.flyer_id = fl.flyer_id
JOIN public.store_locations sl ON fl.store_location_id = sl.store_location_id
JOIN public.stores s ON sl.store_id = s.store_id
JOIN public.addresses a ON sl.address_id = a.address_id

TypeScript Type Issues:

• src/types.ts Flyer interface has store object, but it should have locations: StoreLocation[] array
• Current structure assumes one store per flyer, not multiple locations

Files Affected:

• src/services/db/flyer.db.ts - All flyer queries
• src/types.ts - Flyer interface definition
• Any component displaying flyer locations

---

2. User Submitted Prices → Store Locations (MIGRATED)

Status: ✅ FIXED - Migration created

Schema:

CREATE TABLE IF NOT EXISTS public.user_submitted_prices (
    ...
    store_id BIGINT NOT NULL REFERENCES public.stores(store_id) ON DELETE CASCADE,
    ...
);

Solution Implemented:

• Created migration sql/migrations/005_add_store_location_to_user_submitted_prices.sql
• Added store_location_id column to table (NOT NULL after migration)
• Migrated existing data: linked each price to first location of its store
• Updated TypeScript interface src/types.ts:270-282 to include both fields
• Kept store_id for backward compatibility during transition

Benefits:

• Prices are now specific to individual store locations
• "Walmart Toronto" and "Walmart Vancouver" prices are tracked separately
• Improves geographic specificity for price comparisons
• Enables proximity-based price recommendations

Next Steps:

• Application code needs to be updated to use store_location_id when creating new prices
• Once all code is migrated, can drop the legacy store_id column
• User-submitted prices feature is not yet implemented in the UI

---

3. Receipts → Store Locations (MIGRATED)

Status: ✅ FIXED - Migration created

Schema:

CREATE TABLE IF NOT EXISTS public.receipts (
    ...
    store_id BIGINT REFERENCES public.stores(store_id) ON DELETE CASCADE,
    store_location_id BIGINT REFERENCES public.store_locations(store_location_id) ON DELETE SET NULL,
    ...
);

Solution Implemented:

• Created migration sql/migrations/006_add_store_location_to_receipts.sql
• Added store_location_id column to table (nullable - receipts may not have matched store)
• Migrated existing data: linked each receipt to first location of its store
• Updated TypeScript interface src/types.ts:661-675 to include both fields
• Kept store_id for backward compatibility during transition

Benefits:

• Receipts can now be tied to specific store locations
• "Loblaws Queen St" and "Loblaws Bloor St" are tracked separately
• Enables location-specific shopping pattern analysis
• Improves receipt matching accuracy with address data

Next Steps:

• Receipt scanning code needs to determine specific store_location_id from OCR text
• May require address parsing/matching logic in receipt processing
• Once all code is migrated, can drop the legacy store_id column
• OCR confidence and pattern matching should prefer location-specific data

---

4. Item Price History → Store Locations (Already Correct!)

Schema:

CREATE TABLE IF NOT EXISTS public.item_price_history (
    ...
    store_location_id BIGINT REFERENCES public.store_locations(store_location_id) ON DELETE CASCADE,
    ...
);

Status:

• ✅ CORRECTLY IMPLEMENTED - This table already uses store_location_id
• Properly tracks price history per location
• Good example of how other tables should be structured

---

Summary Table

Table	Foreign Key	Should Use	Status	Priority
flyer_locations	flyer_id, store_location_id	Many-to-many link	✅ FIXED	✅ Done
flyers	store_id	store_id Now uses flyer_locations	✅ FIXED	✅ Done
user_submitted_prices	store_id	store_location_id	✅ MIGRATED	✅ Done
receipts	store_id	store_location_id	✅ MIGRATED	✅ Done
item_price_history	store_location_id	✅ Already correct	✅ Correct	✅ Good
shopping_trips	(no store ref)	N/A	✅ Correct	✅ Good
store_locations	store_id, address_id	✅ Already correct	✅ Correct	✅ Good

---

Impact Assessment

Critical (Must Fix)

1. Flyer Locations Many-to-Many
   • Impact: Flyers can't be associated with multiple store locations
   • User Impact: Users can't see which specific store locations have deals
   • Business Logic: Breaks core assumption that one flyer can be valid at multiple stores
   • Fix Complexity: High - requires schema migration, type changes, query rewrites

Medium (Should Consider)

2. User Submitted Prices & Receipts
   • Impact: Loss of location-specific data
   • User Impact: Can't distinguish between different locations of same store chain
   • Business Logic: Reduces accuracy of proximity-based recommendations
   • Fix Complexity: Medium - requires migration and query updates

---

Recommended Actions

Phase 1: Fix Flyer Locations (Critical)

1. Create migration to properly use flyer_locations table
2. Update Flyer TypeScript interface to support multiple locations
3. Rewrite all flyer queries in src/services/db/flyer.db.ts
4. Update flyer creation/update endpoints to manage flyer_locations entries
5. Update frontend components to display multiple locations per flyer
6. Update tests to use new structure

Phase 2: Consider Store Location Specificity (Optional)

1. Evaluate if location-specific receipts and prices provide value
2. If yes, create migrations to change store_id → store_location_id
3. Update repository queries
4. Update TypeScript interfaces
5. Update tests

---

Related Documents

• ADR-013: Store Address Normalization
• STORE_ADDRESS_IMPLEMENTATION_PLAN.md
• TESTING.md

---

Analysis Methodology

This analysis was conducted by:

1. Extracting all foreign key relationships from sql/master_schema_rollup.sql
2. Comparing schema relationships against TypeScript interfaces in src/types.ts
3. Auditing database queries in src/services/db/ for proper JOIN usage
4. Identifying gaps where schema relationships exist but aren't used in queries

Commands used:

# Extract all foreign keys
podman exec -it flyer-crawler-dev bash -c "grep -n 'REFERENCES' sql/master_schema_rollup.sql"

# Check specific table structures
podman exec -it flyer-crawler-dev bash -c "grep -A 15 'CREATE TABLE.*table_name' sql/master_schema_rollup.sql"

# Verify query patterns
podman exec -it flyer-crawler-dev bash -c "grep -n 'JOIN.*table_name' src/services/db/*.ts"

---

Last Updated: 2026-01-19 Analyzed By: Claude Code (via user request after discovering store_name → store bug)