Torben Sorensen
11aeac5edd
Some checks failed
Deploy to Test Environment / deploy-to-test (push) Failing after 1m10s
272 lines
6.5 KiB
Markdown
272 lines
6.5 KiB
Markdown
# Deployment Guide
|
|
|
|
This guide covers deploying Flyer Crawler to a production server.
|
|
|
|
## Prerequisites
|
|
|
|
- Ubuntu server (22.04 LTS recommended)
|
|
- PostgreSQL 14+ with PostGIS extension
|
|
- Redis
|
|
- Node.js 20.x
|
|
- NGINX (reverse proxy)
|
|
- PM2 (process manager)
|
|
|
|
---
|
|
|
|
## Server Setup
|
|
|
|
### Install Node.js
|
|
|
|
```bash
|
|
curl -sL https://deb.nodesource.com/setup_20.x | sudo bash -
|
|
sudo apt-get install -y nodejs
|
|
```
|
|
|
|
### Install PM2
|
|
|
|
```bash
|
|
sudo npm install -g pm2
|
|
```
|
|
|
|
---
|
|
|
|
## Application Deployment
|
|
|
|
### Clone and Install
|
|
|
|
```bash
|
|
git clone <repository-url>
|
|
cd flyer-crawler.projectium.com
|
|
npm install
|
|
```
|
|
|
|
### Build for Production
|
|
|
|
```bash
|
|
npm run build
|
|
```
|
|
|
|
### Start with PM2
|
|
|
|
```bash
|
|
npm run start:prod
|
|
```
|
|
|
|
This starts three PM2 processes:
|
|
|
|
- `flyer-crawler-api` - Main API server
|
|
- `flyer-crawler-worker` - Background job worker
|
|
- `flyer-crawler-analytics-worker` - Analytics processing worker
|
|
|
|
---
|
|
|
|
## Environment Variables (Gitea Secrets)
|
|
|
|
For deployments using Gitea CI/CD workflows, configure these as **repository secrets**:
|
|
|
|
| Secret | Description |
|
|
| --------------------------- | ------------------------------------------- |
|
|
| `DB_HOST` | PostgreSQL server hostname |
|
|
| `DB_USER` | PostgreSQL username |
|
|
| `DB_PASSWORD` | PostgreSQL password |
|
|
| `DB_DATABASE_PROD` | Production database name |
|
|
| `REDIS_PASSWORD_PROD` | Production Redis password |
|
|
| `REDIS_PASSWORD_TEST` | Test Redis password |
|
|
| `JWT_SECRET` | Long, random string for signing auth tokens |
|
|
| `VITE_GOOGLE_GENAI_API_KEY` | Google Gemini API key |
|
|
| `GOOGLE_MAPS_API_KEY` | Google Maps Geocoding API key |
|
|
|
|
---
|
|
|
|
## NGINX Configuration
|
|
|
|
### Reverse Proxy Setup
|
|
|
|
Create a site configuration at `/etc/nginx/sites-available/flyer-crawler.projectium.com`:
|
|
|
|
```nginx
|
|
server {
|
|
listen 80;
|
|
server_name flyer-crawler.projectium.com;
|
|
|
|
location / {
|
|
proxy_pass http://localhost:5173;
|
|
proxy_http_version 1.1;
|
|
proxy_set_header Upgrade $http_upgrade;
|
|
proxy_set_header Connection 'upgrade';
|
|
proxy_set_header Host $host;
|
|
proxy_cache_bypass $http_upgrade;
|
|
}
|
|
|
|
location /api {
|
|
proxy_pass http://localhost:3001;
|
|
proxy_http_version 1.1;
|
|
proxy_set_header Upgrade $http_upgrade;
|
|
proxy_set_header Connection 'upgrade';
|
|
proxy_set_header Host $host;
|
|
proxy_cache_bypass $http_upgrade;
|
|
}
|
|
}
|
|
```
|
|
|
|
Enable the site:
|
|
|
|
```bash
|
|
sudo ln -s /etc/nginx/sites-available/flyer-crawler.projectium.com /etc/nginx/sites-enabled/
|
|
sudo nginx -t
|
|
sudo systemctl reload nginx
|
|
```
|
|
|
|
### MIME Types Fix for .mjs Files
|
|
|
|
If JavaScript modules (`.mjs` files) aren't loading correctly, add the proper MIME type.
|
|
|
|
**Option 1**: Edit the site configuration file directly:
|
|
|
|
```nginx
|
|
# Add inside the server block
|
|
types {
|
|
application/javascript js mjs;
|
|
}
|
|
```
|
|
|
|
**Option 2**: Edit `/etc/nginx/mime.types` globally:
|
|
|
|
```
|
|
# Change this line:
|
|
application/javascript js;
|
|
|
|
# To:
|
|
application/javascript js mjs;
|
|
```
|
|
|
|
After changes:
|
|
|
|
```bash
|
|
sudo nginx -t
|
|
sudo systemctl reload nginx
|
|
```
|
|
|
|
---
|
|
|
|
## PM2 Log Management
|
|
|
|
Install and configure pm2-logrotate to manage log files:
|
|
|
|
```bash
|
|
pm2 install pm2-logrotate
|
|
pm2 set pm2-logrotate:max_size 10M
|
|
pm2 set pm2-logrotate:retain 14
|
|
pm2 set pm2-logrotate:compress false
|
|
pm2 set pm2-logrotate:dateFormat YYYY-MM-DD_HH-mm-ss
|
|
```
|
|
|
|
---
|
|
|
|
## Rate Limiting
|
|
|
|
The application respects the Gemini AI service's rate limits. You can adjust the `GEMINI_RPM` (requests per minute) environment variable in production as needed without changing the code.
|
|
|
|
---
|
|
|
|
## CI/CD Pipeline
|
|
|
|
The project includes Gitea workflows at `.gitea/workflows/deploy.yml` that:
|
|
|
|
1. Run tests against a test database
|
|
2. Build the application
|
|
3. Deploy to production on successful builds
|
|
|
|
The workflow automatically:
|
|
|
|
- Sets up the test database schema before tests
|
|
- Tears down test data after tests complete
|
|
- Deploys to the production server
|
|
|
|
---
|
|
|
|
## Monitoring
|
|
|
|
### Check PM2 Status
|
|
|
|
```bash
|
|
pm2 status
|
|
pm2 logs
|
|
pm2 logs flyer-crawler-api --lines 100
|
|
```
|
|
|
|
### Restart Services
|
|
|
|
```bash
|
|
pm2 restart all
|
|
pm2 restart flyer-crawler-api
|
|
```
|
|
|
|
---
|
|
|
|
## Error Tracking with Bugsink (ADR-015)
|
|
|
|
Bugsink is a self-hosted Sentry-compatible error tracking system. See [docs/adr/0015-application-performance-monitoring-and-error-tracking.md](docs/adr/0015-application-performance-monitoring-and-error-tracking.md) for the full architecture decision.
|
|
|
|
### Creating Bugsink Projects and DSNs
|
|
|
|
After Bugsink is installed and running, you need to create projects and obtain DSNs:
|
|
|
|
1. **Access Bugsink UI**: Navigate to `http://localhost:8000`
|
|
|
|
2. **Log in** with your admin credentials
|
|
|
|
3. **Create Backend Project**:
|
|
- Click "Create Project"
|
|
- Name: `flyer-crawler-backend`
|
|
- Platform: Node.js
|
|
- Copy the generated DSN (format: `http://<key>@localhost:8000/<project_id>`)
|
|
|
|
4. **Create Frontend Project**:
|
|
- Click "Create Project"
|
|
- Name: `flyer-crawler-frontend`
|
|
- Platform: React
|
|
- Copy the generated DSN
|
|
|
|
5. **Configure Environment Variables**:
|
|
|
|
```bash
|
|
# Backend (server-side)
|
|
export SENTRY_DSN=http://<backend-key>@localhost:8000/<backend-project-id>
|
|
|
|
# Frontend (client-side, exposed to browser)
|
|
export VITE_SENTRY_DSN=http://<frontend-key>@localhost:8000/<frontend-project-id>
|
|
|
|
# Shared settings
|
|
export SENTRY_ENVIRONMENT=production
|
|
export VITE_SENTRY_ENVIRONMENT=production
|
|
export SENTRY_ENABLED=true
|
|
export VITE_SENTRY_ENABLED=true
|
|
```
|
|
|
|
### Testing Error Tracking
|
|
|
|
Verify Bugsink is receiving events:
|
|
|
|
```bash
|
|
npx tsx scripts/test-bugsink.ts
|
|
```
|
|
|
|
This sends test error and info events. Check the Bugsink UI for:
|
|
|
|
- `BugsinkTestError` in the backend project
|
|
- Info message "Test info message from test-bugsink.ts"
|
|
|
|
### Sentry SDK v10+ HTTP DSN Limitation
|
|
|
|
The Sentry SDK v10+ enforces HTTPS-only DSNs by default. Since Bugsink runs locally over HTTP, our implementation uses the Sentry Store API directly instead of the SDK's built-in transport. This is handled transparently by the `sentry.server.ts` and `sentry.client.ts` modules.
|
|
|
|
---
|
|
|
|
## Related Documentation
|
|
|
|
- [Database Setup](DATABASE.md) - PostgreSQL and PostGIS configuration
|
|
- [Authentication Setup](AUTHENTICATION.md) - OAuth provider configuration
|
|
- [Installation Guide](INSTALL.md) - Local development setup
|
|
- [Bare-Metal Server Setup](docs/BARE-METAL-SETUP.md) - Manual server installation guide
|