flyer-crawler.projectium.com/DEPLOYMENT.md

Deployment Guide

This guide covers deploying Flyer Crawler to a production server.

Prerequisites

• Ubuntu server (22.04 LTS recommended)
• PostgreSQL 14+ with PostGIS extension
• Redis
• Node.js 20.x
• NGINX (reverse proxy)
• PM2 (process manager)

---

Server Setup

Install Node.js

curl -sL https://deb.nodesource.com/setup_20.x | sudo bash -
sudo apt-get install -y nodejs

Install PM2

sudo npm install -g pm2

---

Application Deployment

Clone and Install

git clone <repository-url>
cd flyer-crawler.projectium.com
npm install

Build for Production

npm run build

Start with PM2

npm run start:prod

This starts three PM2 processes:

• flyer-crawler-api - Main API server
• flyer-crawler-worker - Background job worker
• flyer-crawler-analytics-worker - Analytics processing worker

---

Environment Variables (Gitea Secrets)

For deployments using Gitea CI/CD workflows, configure these as repository secrets:

Secret	Description
DB_HOST	PostgreSQL server hostname
DB_USER	PostgreSQL username
DB_PASSWORD	PostgreSQL password
DB_DATABASE_PROD	Production database name
REDIS_PASSWORD_PROD	Production Redis password
REDIS_PASSWORD_TEST	Test Redis password
JWT_SECRET	Long, random string for signing auth tokens
VITE_GOOGLE_GENAI_API_KEY	Google Gemini API key
GOOGLE_MAPS_API_KEY	Google Maps Geocoding API key

---

NGINX Configuration

Reverse Proxy Setup

Create a site configuration at /etc/nginx/sites-available/flyer-crawler.projectium.com:

server {
    listen 80;
    server_name flyer-crawler.projectium.com;

    location / {
        proxy_pass http://localhost:5173;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection 'upgrade';
        proxy_set_header Host $host;
        proxy_cache_bypass $http_upgrade;
    }

    location /api {
        proxy_pass http://localhost:3001;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection 'upgrade';
        proxy_set_header Host $host;
        proxy_cache_bypass $http_upgrade;
    }
}

Enable the site:

sudo ln -s /etc/nginx/sites-available/flyer-crawler.projectium.com /etc/nginx/sites-enabled/
sudo nginx -t
sudo systemctl reload nginx

MIME Types Fix for .mjs Files

If JavaScript modules (.mjs files) aren't loading correctly, add the proper MIME type.

Option 1: Edit the site configuration file directly:

# Add inside the server block
types {
    application/javascript js mjs;
}

Option 2: Edit /etc/nginx/mime.types globally:

# Change this line:
application/javascript js;

# To:
application/javascript js mjs;

After changes:

sudo nginx -t
sudo systemctl reload nginx

---

PM2 Log Management

Install and configure pm2-logrotate to manage log files:

pm2 install pm2-logrotate
pm2 set pm2-logrotate:max_size 10M
pm2 set pm2-logrotate:retain 14
pm2 set pm2-logrotate:compress false
pm2 set pm2-logrotate:dateFormat YYYY-MM-DD_HH-mm-ss

---

Rate Limiting

The application respects the Gemini AI service's rate limits. You can adjust the GEMINI_RPM (requests per minute) environment variable in production as needed without changing the code.

---

CI/CD Pipeline

The project includes Gitea workflows at .gitea/workflows/deploy.yml that:

1. Run tests against a test database
2. Build the application
3. Deploy to production on successful builds

The workflow automatically:

• Sets up the test database schema before tests
• Tears down test data after tests complete
• Deploys to the production server

---

Monitoring

Check PM2 Status

pm2 status
pm2 logs
pm2 logs flyer-crawler-api --lines 100

Restart Services

pm2 restart all
pm2 restart flyer-crawler-api

---

Error Tracking with Bugsink (ADR-015)

Bugsink is a self-hosted Sentry-compatible error tracking system. See docs/adr/0015-application-performance-monitoring-and-error-tracking.md for the full architecture decision.

Creating Bugsink Projects and DSNs

After Bugsink is installed and running, you need to create projects and obtain DSNs:

1. Access Bugsink UI: Navigate to http://localhost:8000

2. Log in with your admin credentials

3. Create Backend Project:

   • Click "Create Project"
   • Name: flyer-crawler-backend
   • Platform: Node.js
   • Copy the generated DSN (format: http://<key>@localhost:8000/<project_id>)

4. Create Frontend Project:

   • Click "Create Project"
   • Name: flyer-crawler-frontend
   • Platform: React
   • Copy the generated DSN

5. Configure Environment Variables:

   # Backend (server-side)
   export SENTRY_DSN=http://<backend-key>@localhost:8000/<backend-project-id>
   
   # Frontend (client-side, exposed to browser)
   export VITE_SENTRY_DSN=http://<frontend-key>@localhost:8000/<frontend-project-id>
   
   # Shared settings
   export SENTRY_ENVIRONMENT=production
   export VITE_SENTRY_ENVIRONMENT=production
   export SENTRY_ENABLED=true
   export VITE_SENTRY_ENABLED=true
   

Testing Error Tracking

Verify Bugsink is receiving events:

npx tsx scripts/test-bugsink.ts

This sends test error and info events. Check the Bugsink UI for:

• BugsinkTestError in the backend project
• Info message "Test info message from test-bugsink.ts"

Sentry SDK v10+ HTTP DSN Limitation

The Sentry SDK v10+ enforces HTTPS-only DSNs by default. Since Bugsink runs locally over HTTP, our implementation uses the Sentry Store API directly instead of the SDK's built-in transport. This is handled transparently by the sentry.server.ts and sentry.client.ts modules.

---

Related Documentation

• Database Setup - PostgreSQL and PostGIS configuration
• Authentication Setup - OAuth provider configuration
• Installation Guide - Local development setup
• Bare-Metal Server Setup - Manual server installation guide