flyer-crawler.projectium.com/docs/adr/0007-configuration-and-secrets-management.md

ADR-007: Configuration and Secrets Management

Date: 2025-12-12

Status: Accepted

Implemented: 2026-01-09

Context

The application currently accesses environment variables directly via process.env. This can lead to missing variables at runtime, inconsistent naming, and a lack of type safety. It is difficult to know at a glance which environment variables are required for the application to run.

Decision

We will introduce a centralized, schema-validated configuration service. We will use a library like zod to define a schema for all required environment variables. A singleton service will parse process.env at application startup, validate it against the schema, and provide a type-safe configuration object to the rest of the app. The application will fail fast on startup if any required configuration is missing or invalid.

Consequences

Positive: Improves application reliability and developer experience by catching configuration errors at startup rather than at runtime. Provides a single source of truth for all required configuration. Negative: Adds a small amount of boilerplate for defining the configuration schema. Requires a one-time effort to refactor all process.env access points to use the new configuration service.

Implementation Status

What's Implemented

• ✅ Centralized Configuration Schema - Zod-based validation in src/config/env.ts
• ✅ Type-Safe Access - Full TypeScript types for all configuration
• ✅ Fail-Fast Startup - Clear error messages for missing/invalid config
• ✅ Environment Helpers - isProduction, isTest, isDevelopment exports
• ✅ Service Configuration Helpers - isSmtpConfigured, isAiConfigured, etc.

Migration Status

• ⏳ Gradual migration of process.env access to config.* in progress
• Legacy process.env access still works during transition

Implementation Details

Configuration Schema

The configuration is organized into logical groups:

import { config, isProduction, isTest } from './config/env';

// Database
config.database.host; // DB_HOST
config.database.port; // DB_PORT (default: 5432)
config.database.user; // DB_USER
config.database.password; // DB_PASSWORD
config.database.name; // DB_NAME

// Redis
config.redis.url; // REDIS_URL
config.redis.password; // REDIS_PASSWORD (optional)

// Authentication
config.auth.jwtSecret; // JWT_SECRET (min 32 chars)
config.auth.jwtSecretPrevious; // JWT_SECRET_PREVIOUS (for rotation)

// SMTP (all optional - email degrades gracefully)
config.smtp.host; // SMTP_HOST
config.smtp.port; // SMTP_PORT (default: 587)
config.smtp.user; // SMTP_USER
config.smtp.pass; // SMTP_PASS
config.smtp.secure; // SMTP_SECURE (default: false)
config.smtp.fromEmail; // SMTP_FROM_EMAIL

// AI Services
config.ai.geminiApiKey; // GEMINI_API_KEY
config.ai.geminiRpm; // GEMINI_RPM (default: 5)
config.ai.priceQualityThreshold; // AI_PRICE_QUALITY_THRESHOLD (default: 0.5)

// Google Services
config.google.mapsApiKey; // GOOGLE_MAPS_API_KEY (optional)
config.google.clientId; // GOOGLE_CLIENT_ID (optional)
config.google.clientSecret; // GOOGLE_CLIENT_SECRET (optional)

// Worker Configuration
config.worker.concurrency; // WORKER_CONCURRENCY (default: 1)
config.worker.lockDuration; // WORKER_LOCK_DURATION (default: 30000)
config.worker.emailConcurrency; // EMAIL_WORKER_CONCURRENCY (default: 10)
config.worker.analyticsConcurrency; // ANALYTICS_WORKER_CONCURRENCY (default: 1)
config.worker.cleanupConcurrency; // CLEANUP_WORKER_CONCURRENCY (default: 10)
config.worker.weeklyAnalyticsConcurrency; // WEEKLY_ANALYTICS_WORKER_CONCURRENCY (default: 1)

// Server
config.server.nodeEnv; // NODE_ENV (development/production/test)
config.server.port; // PORT (default: 3001)
config.server.frontendUrl; // FRONTEND_URL
config.server.baseUrl; // BASE_URL
config.server.storagePath; // STORAGE_PATH (default: /var/www/.../flyer-images)

Convenience Helpers

import { isProduction, isTest, isDevelopment, isSmtpConfigured } from './config/env';

// Environment checks
if (isProduction) {
  // Production-only logic
}

// Service availability checks
if (isSmtpConfigured) {
  await sendEmail(...);
} else {
  logger.warn('Email not configured, skipping notification');
}

Fail-Fast Error Messages

When configuration is invalid, the application exits with a clear error:

╔════════════════════════════════════════════════════════════════╗
║           CONFIGURATION ERROR - APPLICATION STARTUP            ║
╚════════════════════════════════════════════════════════════════╝

The following environment variables are missing or invalid:

  - database.host: DB_HOST is required
  - auth.jwtSecret: JWT_SECRET must be at least 32 characters for security

Please check your .env file or environment configuration.
See ADR-007 for the complete list of required environment variables.

Usage Example

// Before (direct process.env access)
const pool = new Pool({
  host: process.env.DB_HOST,
  port: parseInt(process.env.DB_PORT || '5432', 10),
  user: process.env.DB_USER,
  password: process.env.DB_PASSWORD,
  database: process.env.DB_NAME,
});

// After (type-safe config access)
import { config } from './config/env';

const pool = new Pool({
  host: config.database.host,
  port: config.database.port,
  user: config.database.user,
  password: config.database.password,
  database: config.database.name,
});

Required Environment Variables

Critical (Application will not start without these)

Variable	Description
DB_HOST	PostgreSQL database host
DB_USER	PostgreSQL database user
DB_PASSWORD	PostgreSQL database password
DB_NAME	PostgreSQL database name
REDIS_URL	Redis connection URL (e.g., redis://localhost:6379)
JWT_SECRET	JWT signing secret (minimum 32 characters)

Optional with Defaults

Variable	Default	Description
DB_PORT	5432	PostgreSQL port
PORT	3001	Server HTTP port
NODE_ENV	development	Environment mode
STORAGE_PATH	/var/www/.../flyer-images	File upload directory
SMTP_PORT	587	SMTP server port
SMTP_SECURE	false	Use TLS for SMTP
GEMINI_RPM	5	Gemini API requests per minute
AI_PRICE_QUALITY_THRESHOLD	0.5	AI extraction quality threshold
WORKER_CONCURRENCY	1	Flyer processing concurrency
WORKER_LOCK_DURATION	30000	Worker lock duration (ms)

Optional (Feature-specific)

Variable	Description
GEMINI_API_KEY	Google Gemini API key (enables AI features)
GOOGLE_MAPS_API_KEY	Google Maps API key (enables geocoding)
SMTP_HOST	SMTP server (enables email notifications)
SMTP_USER	SMTP authentication username
SMTP_PASS	SMTP authentication password
SMTP_FROM_EMAIL	Sender email address
FRONTEND_URL	Frontend URL for email links
JWT_SECRET_PREVIOUS	Previous JWT secret for rotation (ADR-029)

Key Files

• src/config/env.ts - Configuration schema and validation
• .env.example - Template for required environment variables

Migration Guide

To migrate existing process.env usage:

1. Import the config:

   import { config, isProduction } from '../config/env';
   

2. Replace direct access:

   // Before
   process.env.DB_HOST;
   process.env.NODE_ENV === 'production';
   parseInt(process.env.PORT || '3001', 10);
   
   // After
   config.database.host;
   isProduction;
   config.server.port;
   

3. Use service helpers for optional features:

   import { isSmtpConfigured, isAiConfigured } from '../config/env';
   
   if (isSmtpConfigured) {
     // Email is available
   }