flyer-crawler.projectium.com/docs/adr/0007-configuration-and-secrets-management.md

# ADR-007: Configuration and Secrets Management

**Date**: 2025-12-12

**Status**: Accepted

**Implemented**: 2026-01-09

## Context

The application currently accesses environment variables directly via `process.env`. This can lead to missing variables at runtime, inconsistent naming, and a lack of type safety. It is difficult to know at a glance which environment variables are required for the application to run.

## Decision

We will introduce a centralized, schema-validated configuration service. We will use a library like `zod` to define a schema for all required environment variables. A singleton service will parse `process.env` at application startup, validate it against the schema, and provide a type-safe configuration object to the rest of the app. The application will fail fast on startup if any required configuration is missing or invalid.

## Consequences

**Positive**: Improves application reliability and developer experience by catching configuration errors at startup rather than at runtime. Provides a single source of truth for all required configuration.
**Negative**: Adds a small amount of boilerplate for defining the configuration schema. Requires a one-time effort to refactor all `process.env` access points to use the new configuration service.

## Implementation Status

### What's Implemented

- ✅ **Centralized Configuration Schema** - Zod-based validation in `src/config/env.ts`
- ✅ **Type-Safe Access** - Full TypeScript types for all configuration
- ✅ **Fail-Fast Startup** - Clear error messages for missing/invalid config
- ✅ **Environment Helpers** - `isProduction`, `isTest`, `isDevelopment` exports
- ✅ **Service Configuration Helpers** - `isSmtpConfigured`, `isAiConfigured`, etc.

### Migration Status

- ⏳ Gradual migration of `process.env` access to `config.*` in progress
- Legacy `process.env` access still works during transition

## Implementation Details

### Configuration Schema

The configuration is organized into logical groups:

```typescript
import { config, isProduction, isTest } from './config/env';

// Database
config.database.host; // DB_HOST
config.database.port; // DB_PORT (default: 5432)
config.database.user; // DB_USER
config.database.password; // DB_PASSWORD
config.database.name; // DB_NAME

// Redis
config.redis.url; // REDIS_URL
config.redis.password; // REDIS_PASSWORD (optional)

// Authentication
config.auth.jwtSecret; // JWT_SECRET (min 32 chars)
config.auth.jwtSecretPrevious; // JWT_SECRET_PREVIOUS (for rotation)

// SMTP (all optional - email degrades gracefully)
config.smtp.host; // SMTP_HOST
config.smtp.port; // SMTP_PORT (default: 587)
config.smtp.user; // SMTP_USER
config.smtp.pass; // SMTP_PASS
config.smtp.secure; // SMTP_SECURE (default: false)
config.smtp.fromEmail; // SMTP_FROM_EMAIL

// AI Services
config.ai.geminiApiKey; // GEMINI_API_KEY
config.ai.geminiRpm; // GEMINI_RPM (default: 5)
config.ai.priceQualityThreshold; // AI_PRICE_QUALITY_THRESHOLD (default: 0.5)

// Google Services
config.google.mapsApiKey; // GOOGLE_MAPS_API_KEY (optional)
config.google.clientId; // GOOGLE_CLIENT_ID (optional)
config.google.clientSecret; // GOOGLE_CLIENT_SECRET (optional)

// Worker Configuration
config.worker.concurrency; // WORKER_CONCURRENCY (default: 1)
config.worker.lockDuration; // WORKER_LOCK_DURATION (default: 30000)
config.worker.emailConcurrency; // EMAIL_WORKER_CONCURRENCY (default: 10)
config.worker.analyticsConcurrency; // ANALYTICS_WORKER_CONCURRENCY (default: 1)
config.worker.cleanupConcurrency; // CLEANUP_WORKER_CONCURRENCY (default: 10)
config.worker.weeklyAnalyticsConcurrency; // WEEKLY_ANALYTICS_WORKER_CONCURRENCY (default: 1)

// Server
config.server.nodeEnv; // NODE_ENV (development/production/test)
config.server.port; // PORT (default: 3001)
config.server.frontendUrl; // FRONTEND_URL
config.server.baseUrl; // BASE_URL
config.server.storagePath; // STORAGE_PATH (default: /var/www/.../flyer-images)
```

### Convenience Helpers

```typescript
import { isProduction, isTest, isDevelopment, isSmtpConfigured } from './config/env';

// Environment checks
if (isProduction) {
  // Production-only logic
}

// Service availability checks
if (isSmtpConfigured) {
  await sendEmail(...);
} else {
  logger.warn('Email not configured, skipping notification');
}
```

### Fail-Fast Error Messages

When configuration is invalid, the application exits with a clear error:

```text
╔════════════════════════════════════════════════════════════════╗
║           CONFIGURATION ERROR - APPLICATION STARTUP            ║
╚════════════════════════════════════════════════════════════════╝

The following environment variables are missing or invalid:

  - database.host: DB_HOST is required
  - auth.jwtSecret: JWT_SECRET must be at least 32 characters for security

Please check your .env file or environment configuration.
See ADR-007 for the complete list of required environment variables.
```

### Usage Example

```typescript
// Before (direct process.env access)
const pool = new Pool({
  host: process.env.DB_HOST,
  port: parseInt(process.env.DB_PORT || '5432', 10),
  user: process.env.DB_USER,
  password: process.env.DB_PASSWORD,
  database: process.env.DB_NAME,
});

// After (type-safe config access)
import { config } from './config/env';

const pool = new Pool({
  host: config.database.host,
  port: config.database.port,
  user: config.database.user,
  password: config.database.password,
  database: config.database.name,
});
```

## Required Environment Variables

### Critical (Application will not start without these)

| Variable      | Description                                           |
| ------------- | ----------------------------------------------------- |
| `DB_HOST`     | PostgreSQL database host                              |
| `DB_USER`     | PostgreSQL database user                              |
| `DB_PASSWORD` | PostgreSQL database password                          |
| `DB_NAME`     | PostgreSQL database name                              |
| `REDIS_URL`   | Redis connection URL (e.g., `redis://localhost:6379`) |
| `JWT_SECRET`  | JWT signing secret (minimum 32 characters)            |

### Optional with Defaults

| Variable                     | Default                   | Description                     |
| ---------------------------- | ------------------------- | ------------------------------- |
| `DB_PORT`                    | 5432                      | PostgreSQL port                 |
| `PORT`                       | 3001                      | Server HTTP port                |
| `NODE_ENV`                   | development               | Environment mode                |
| `STORAGE_PATH`               | /var/www/.../flyer-images | File upload directory           |
| `SMTP_PORT`                  | 587                       | SMTP server port                |
| `SMTP_SECURE`                | false                     | Use TLS for SMTP                |
| `GEMINI_RPM`                 | 5                         | Gemini API requests per minute  |
| `AI_PRICE_QUALITY_THRESHOLD` | 0.5                       | AI extraction quality threshold |
| `WORKER_CONCURRENCY`         | 1                         | Flyer processing concurrency    |
| `WORKER_LOCK_DURATION`       | 30000                     | Worker lock duration (ms)       |

### Optional (Feature-specific)

| Variable              | Description                                 |
| --------------------- | ------------------------------------------- |
| `GEMINI_API_KEY`      | Google Gemini API key (enables AI features) |
| `GOOGLE_MAPS_API_KEY` | Google Maps API key (enables geocoding)     |
| `SMTP_HOST`           | SMTP server (enables email notifications)   |
| `SMTP_USER`           | SMTP authentication username                |
| `SMTP_PASS`           | SMTP authentication password                |
| `SMTP_FROM_EMAIL`     | Sender email address                        |
| `FRONTEND_URL`        | Frontend URL for email links                |
| `JWT_SECRET_PREVIOUS` | Previous JWT secret for rotation (ADR-029)  |

## Key Files

- `src/config/env.ts` - Configuration schema and validation
- `.env.example` - Template for required environment variables

## Migration Guide

To migrate existing `process.env` usage:

1. Import the config:

   ```typescript
   import { config, isProduction } from '../config/env';
   ```

2. Replace direct access:

   ```typescript
   // Before
   process.env.DB_HOST;
   process.env.NODE_ENV === 'production';
   parseInt(process.env.PORT || '3001', 10);

   // After
   config.database.host;
   isProduction;
   config.server.port;
   ```

3. Use service helpers for optional features:

   ```typescript
   import { isSmtpConfigured, isAiConfigured } from '../config/env';

   if (isSmtpConfigured) {
     // Email is available
   }
   ```