flyer-crawler.projectium.com/CLAUDE.md

Claude Code Project Instructions

CRITICAL RULES (READ FIRST)

Platform: Linux Only (ADR-014)

ALL tests MUST run in dev container - Windows results are unreliable.

Test Result	Container	Windows	Status
Pass	Fail	=	BROKEN (must fix)
Fail	Pass	=	PASSING (acceptable)

# Always test in container
podman exec -it flyer-crawler-dev npm test
podman exec -it flyer-crawler-dev npm run type-check

Database Schema Sync

CRITICAL: Keep these files synchronized:

• sql/master_schema_rollup.sql (test DB, complete reference)
• sql/initial_schema.sql (fresh install, identical to rollup)
• sql/migrations/*.sql (production ALTER TABLE statements)

Out-of-sync = test failures.

Communication Style

Ask before assuming. Never assume:

• Steps completed / User knowledge / External services configured / Secrets created

---

Session Startup Checklist

1. Memory: mcp__memory__read_graph - Recall project context, credentials, known issues
2. Git: git log --oneline -10 - Recent changes
3. Containers: mcp__podman__container_list - Running state

---

Quick Reference

Essential Commands

Command	Description
podman exec -it flyer-crawler-dev npm test	Run all tests
podman exec -it flyer-crawler-dev npm run test:unit	Unit tests only
podman exec -it flyer-crawler-dev npm run type-check	TypeScript check
podman exec -it flyer-crawler-dev npm run test:integration	Integration tests

Key Patterns (with file locations)

Pattern	ADR	Implementation	File
Error Handling	ADR-001	handleDbError(), throw NotFoundError	src/services/db/errors.db.ts
Repository Methods	ADR-034	get* (throws), find* (null), list* (array)	src/services/db/*.db.ts
API Responses	ADR-028	sendSuccess(), sendPaginated(), sendError()	src/utils/apiResponse.ts
Transactions	ADR-002	withTransaction(async (client) => {...})	src/services/db/transaction.db.ts

Key Files Quick Access

Purpose	File
Express app	server.ts
Environment	src/config/env.ts
Routes	src/routes/*.routes.ts
Repositories	src/services/db/*.db.ts
Workers	src/services/workers.server.ts
Queues	src/services/queues.server.ts

---

Application Overview

Flyer Crawler - AI-powered grocery deal extraction and analysis platform.

Data Flow: Upload → AI extraction (Gemini) → PostgreSQL → Cache (Redis) → API → React display

Architecture (ADR-035):

Routes → Services → Repositories → Database
         ↓
    External APIs (*.server.ts)

Key Entities: Flyers, FlyerItems, Stores, StoreLocations, Users, Watchlists, ShoppingLists, Recipes, Achievements

Full Architecture: See docs/architecture/OVERVIEW.md

---

Common Workflows

Adding a New API Endpoint

1. Add route in src/routes/{domain}.routes.ts
2. Use validateRequest(schema) middleware for input validation
3. Call service layer (never access DB directly from routes)
4. Return via sendSuccess() or sendPaginated()
5. Add tests in *.routes.test.ts

Example Pattern: See docs/development/CODE-PATTERNS.md

Adding a New Database Operation

1. Add method to src/services/db/{domain}.db.ts
2. Follow naming: get* (throws), find* (returns null), list* (array)
3. Use handleDbError() for error handling
4. Accept optional PoolClient for transaction support
5. Add unit test

Adding a Background Job

1. Define queue in src/services/queues.server.ts
2. Add worker in src/services/workers.server.ts
3. Call queue.add() from service layer

---

Subagent Delegation Guide

When to Delegate: Complex work, specialized expertise, multi-domain tasks

Decision Matrix

Task Type	Subagent	Key Docs
Write production code	coder	CODER-GUIDE.md
Database changes	db-dev	DATABASE-GUIDE.md
Create tests	testwriter	TESTER-GUIDE.md
Fix failing tests	tester	TESTER-GUIDE.md
Container/deployment	devops	DEVOPS-GUIDE.md
UI components	frontend-specialist	FRONTEND-GUIDE.md
External APIs	integrations-specialist	INTEGRATIONS-GUIDE.md
Security review	security-engineer	SECURITY-DEBUG-GUIDE.md
Production errors	log-debug	SECURITY-DEBUG-GUIDE.md
AI/Gemini issues	ai-usage	AI-USAGE-GUIDE.md
Planning features	planner	DOCUMENTATION-GUIDE.md

All Subagents: See docs/subagents/OVERVIEW.md

Launch Pattern:

Use Task tool with subagent_type: "coder", "db-dev", "tester", etc.

---

Known Issues & Gotchas

Integration Test Issues (Summary)

Common issues with solutions:

1. Vitest globalSetup context isolation - Mocks/spies don't share instances → Mark .todo() or use Redis-based flags
2. Cleanup queue interference - Worker processes jobs during tests → cleanupQueue.drain() and .pause()
3. Cache staleness - Direct SQL bypasses cache → cacheService.invalidateFlyers() after inserts
4. Filename collisions - Multer predictable names → Use ${Date.now()}-${Math.round(Math.random() * 1e9)}
5. Response format mismatches - API format changes → Log response bodies, update assertions
6. External service failures - PM2/Redis unavailable → try/catch with graceful degradation

Full Details: See test issues section at end of this document or docs/development/TESTING.md

Git Bash Path Conversion (Windows)

Git Bash auto-converts Unix paths, breaking container commands.

Solutions:

# Use sh -c with single quotes
podman exec container sh -c '/usr/local/bin/script.sh'

# Use MSYS_NO_PATHCONV=1
MSYS_NO_PATHCONV=1 podman exec container /path/to/script

# Use Windows paths for host files
podman cp "d:/path/file" container:/tmp/file

---

Configuration & Environment

Environment Variables

See: docs/getting-started/ENVIRONMENT.md for complete reference.

Quick Overview:

• Production: Gitea CI/CD secrets only (no .env file)
• Test: Gitea secrets + .env.test overrides
• Dev: .env.local file (overrides compose.dev.yml)

Key Variables: DB_HOST, DB_USER, DB_PASSWORD, JWT_SECRET, VITE_GOOGLE_GENAI_API_KEY, REDIS_URL

Adding Variables: Update src/config/env.ts, Gitea Secrets, workflows, ecosystem.config.cjs, .env.example

MCP Servers

See: docs/tools/MCP-CONFIGURATION.md for setup.

Quick Overview:

Server	Purpose	Config
gitea-projectium/torbonium	Gitea API	Global settings.json
podman	Container management	Global settings.json
memory	Knowledge graph	Global settings.json
redis	Cache access	Global settings.json
bugsink	Prod error tracking	Global settings.json
localerrors	Dev Bugsink	Project .mcp.json
devdb	Dev PostgreSQL	Project .mcp.json

Note: Localhost servers use project .mcp.json due to Windows/loader issues.

Bugsink Error Tracking

See: docs/tools/BUGSINK-SETUP.md for setup.

Quick Access:

• Dev: https://localhost:8443 (admin@localhost/admin)
• Prod: https://bugsink.projectium.com

Token Creation (required for MCP):

# Dev container
MSYS_NO_PATHCONV=1 podman exec -e DATABASE_URL=postgresql://bugsink:bugsink_dev_password@postgres:5432/bugsink -e SECRET_KEY=dev-bugsink-secret-key-minimum-50-characters-for-security flyer-crawler-dev sh -c 'cd /opt/bugsink/conf && DJANGO_SETTINGS_MODULE=bugsink_conf PYTHONPATH=/opt/bugsink/conf:/opt/bugsink/lib/python3.10/site-packages /opt/bugsink/bin/python -m django create_auth_token'

# Production (via SSH)
ssh root@projectium.com "cd /opt/bugsink && bugsink-manage create_auth_token"

Logstash

See: docs/operations/LOGSTASH-QUICK-REF.md

Log aggregation: PostgreSQL + PM2 + Redis + NGINX → Bugsink (ADR-050)

---

Documentation Quick Links

Topic	Document
Getting Started	QUICKSTART.md
Architecture	OVERVIEW.md
Code Patterns	CODE-PATTERNS.md
Testing	TESTING.md
Debugging	DEBUGGING.md
Database	DATABASE.md
Deployment	DEPLOYMENT.md
Monitoring	MONITORING.md
ADRs	docs/adr/index.md
All Docs	docs/README.md

---

Appendix: Integration Test Issues (Full Details)

1. Vitest globalSetup Context Isolation

Vitest's globalSetup runs in separate Node.js context. Singletons, spies, mocks do NOT share instances with test files.

Affected: BullMQ worker service mocks (AI/DB failure tests)

Solutions: Mark .todo(), create test-only API endpoints, use Redis-based mock flags

// DOES NOT WORK - different instances
const { flyerProcessingService } = await import('../../services/workers.server');
flyerProcessingService._getAiProcessor()._setExtractAndValidateData(mockFn);

2. Cleanup Queue Deletes Before Verification

Cleanup worker processes jobs in globalSetup context, ignoring test spies.

Solution: Drain and pause queue:

const { cleanupQueue } = await import('../../services/queues.server');
await cleanupQueue.drain();
await cleanupQueue.pause();
// ... test ...
await cleanupQueue.resume();

3. Cache Stale After Direct SQL

Direct pool.query() inserts bypass cache invalidation.

Solution: await cacheService.invalidateFlyers(); after inserts

4. Test Filename Collisions

Multer predictable filenames cause race conditions.

Solution: Use unique suffix: ${Date.now()}-${Math.round(Math.random() * 1e9)}

5. Response Format Mismatches

API formats change: data.jobId vs data.job.id, nested vs flat, string vs number IDs.

Solution: Log response bodies, update assertions

6. External Service Availability

PM2/Redis health checks fail when unavailable.

Solution: try/catch with graceful degradation or mock