Compare commits
2 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
4f08238698 | ||
| 38b35f87aa |
@@ -117,7 +117,8 @@
|
||||
"Bash(git -C \"C:\\\\Users\\\\games3\\\\.claude\\\\plugins\\\\marketplaces\\\\claude-plugins-official\" fetch --dry-run -v)",
|
||||
"mcp__localerrors__get_project",
|
||||
"mcp__localerrors__get_issue",
|
||||
"mcp__localerrors__get_event"
|
||||
"mcp__localerrors__get_event",
|
||||
"mcp__localerrors__list_teams"
|
||||
]
|
||||
},
|
||||
"enabledMcpjsonServers": [
|
||||
|
||||
17
.env.example
17
.env.example
@@ -94,11 +94,18 @@ WORKER_LOCK_DURATION=120000
|
||||
# Error Tracking (ADR-015)
|
||||
# ===================
|
||||
# Sentry-compatible error tracking via Bugsink (self-hosted)
|
||||
# DSNs are created in Bugsink UI at http://localhost:8000 (dev) or your production URL
|
||||
# Backend DSN - for Express/Node.js errors
|
||||
SENTRY_DSN=
|
||||
# Frontend DSN - for React/browser errors (uses VITE_ prefix)
|
||||
VITE_SENTRY_DSN=
|
||||
# DSNs are created in Bugsink UI at https://localhost:8443 (dev) or your production URL
|
||||
#
|
||||
# Dev container projects:
|
||||
# - Project 1: Backend API (Dev) - receives Pino, PostgreSQL errors
|
||||
# - Project 2: Frontend (Dev) - receives browser errors via Sentry SDK
|
||||
# - Project 4: Infrastructure (Dev) - receives Redis, NGINX, Vite errors
|
||||
#
|
||||
# Backend DSN - for Express/Node.js errors (internal container URL)
|
||||
SENTRY_DSN=http://<key>@localhost:8000/1
|
||||
# Frontend DSN - for React/browser errors (uses nginx proxy for browser access)
|
||||
# Note: Browsers cannot reach localhost:8000 directly, so we use nginx proxy at /bugsink-api/
|
||||
VITE_SENTRY_DSN=https://<key>@localhost/bugsink-api/2
|
||||
# Environment name for error grouping (defaults to NODE_ENV)
|
||||
SENTRY_ENVIRONMENT=development
|
||||
VITE_SENTRY_ENVIRONMENT=development
|
||||
|
||||
27
CLAUDE.md
27
CLAUDE.md
@@ -123,23 +123,30 @@ The dev container now matches production by using PM2 for process management.
|
||||
|
||||
### Log Aggregation (ADR-050)
|
||||
|
||||
All logs flow to Bugsink via Logstash:
|
||||
All logs flow to Bugsink via Logstash with 3-project routing:
|
||||
|
||||
| Source | Log Location | Status |
|
||||
| ----------------- | --------------------------------- | ------ |
|
||||
| Backend (Pino) | `/var/log/pm2/api-*.log` | Active |
|
||||
| Worker (Pino) | `/var/log/pm2/worker-*.log` | Active |
|
||||
| Vite | `/var/log/pm2/vite-*.log` | Active |
|
||||
| PostgreSQL | `/var/log/postgresql/*.log` | Active |
|
||||
| Redis | `/var/log/redis/redis-server.log` | Active |
|
||||
| NGINX | `/var/log/nginx/*.log` | Active |
|
||||
| Frontend (Sentry) | Browser -> Bugsink SDK | Active |
|
||||
| Source | Log Location | Bugsink Project |
|
||||
| ----------------- | --------------------------------- | ------------------ |
|
||||
| Backend (Pino) | `/var/log/pm2/api-*.log` | Backend API (1) |
|
||||
| Worker (Pino) | `/var/log/pm2/worker-*.log` | Backend API (1) |
|
||||
| PostgreSQL | `/var/log/postgresql/*.log` | Backend API (1) |
|
||||
| Vite | `/var/log/pm2/vite-*.log` | Infrastructure (4) |
|
||||
| Redis | `/var/log/redis/redis-server.log` | Infrastructure (4) |
|
||||
| NGINX | `/var/log/nginx/*.log` | Infrastructure (4) |
|
||||
| Frontend (Sentry) | Browser -> nginx proxy | Frontend (2) |
|
||||
|
||||
**Bugsink Projects (Dev Container)**:
|
||||
|
||||
- Project 1: Backend API (Dev) - Application errors
|
||||
- Project 2: Frontend (Dev) - Browser errors via nginx proxy
|
||||
- Project 4: Infrastructure (Dev) - Redis, NGINX, Vite errors
|
||||
|
||||
**Key Files**:
|
||||
|
||||
- `ecosystem.dev.config.cjs` - PM2 development configuration
|
||||
- `scripts/dev-entrypoint.sh` - Container startup script
|
||||
- `docker/logstash/bugsink.conf` - Logstash pipeline configuration
|
||||
- `docker/nginx/dev.conf` - NGINX config with Bugsink API proxy
|
||||
|
||||
**Full Dev Container Guide**: See [docs/development/DEV-CONTAINER.md](docs/development/DEV-CONTAINER.md)
|
||||
|
||||
|
||||
@@ -174,6 +174,21 @@ BUGSINK = {\n\
|
||||
}\n\
|
||||
\n\
|
||||
ALLOWED_HOSTS = deduce_allowed_hosts(BUGSINK["BASE_URL"])\n\
|
||||
# Also allow 127.0.0.1 access (both localhost and 127.0.0.1 should work)\n\
|
||||
if "127.0.0.1" not in ALLOWED_HOSTS:\n\
|
||||
ALLOWED_HOSTS.append("127.0.0.1")\n\
|
||||
if "localhost" not in ALLOWED_HOSTS:\n\
|
||||
ALLOWED_HOSTS.append("localhost")\n\
|
||||
\n\
|
||||
# CSRF Trusted Origins (Django 4.0+ requires full origin for HTTPS POST requests)\n\
|
||||
# This fixes "CSRF verification failed" errors when accessing Bugsink via HTTPS\n\
|
||||
# Both localhost and 127.0.0.1 must be trusted to support different access patterns\n\
|
||||
CSRF_TRUSTED_ORIGINS = [\n\
|
||||
"https://localhost:8443",\n\
|
||||
"https://127.0.0.1:8443",\n\
|
||||
"http://localhost:8000",\n\
|
||||
"http://127.0.0.1:8000",\n\
|
||||
]\n\
|
||||
\n\
|
||||
# Console email backend for dev\n\
|
||||
EMAIL_BACKEND = "bugsink.email_backends.QuietConsoleEmailBackend"\n\
|
||||
|
||||
@@ -12,9 +12,18 @@
|
||||
# - NGINX logs (/var/log/nginx/*.log) - Access and error logs
|
||||
# - Redis logs (/var/log/redis/*.log) - Via shared volume (ADR-050)
|
||||
#
|
||||
# Bugsink Projects:
|
||||
# - Project 1: Backend API (Dev) - Pino errors, PostgreSQL errors
|
||||
# Bugsink Projects (3-project architecture):
|
||||
# - Project 1: Backend API (Dev) - Pino/PM2 app errors, PostgreSQL errors
|
||||
# DSN Key: cea01396c56246adb5878fa5ee6b1d22
|
||||
# - Project 2: Frontend (Dev) - Configured via Sentry SDK in browser
|
||||
# DSN Key: d92663cb73cf4145b677b84029e4b762
|
||||
# - Project 4: Infrastructure (Dev) - Redis, NGINX, PM2 operational logs
|
||||
# DSN Key: 14e8791da3d347fa98073261b596cab9
|
||||
#
|
||||
# Routing Logic:
|
||||
# - Backend logs (type: pm2_api, pm2_worker, pino, postgres) -> Project 1
|
||||
# - Infrastructure logs (type: redis, nginx_error, nginx_5xx) -> Project 4
|
||||
# - Vite errors (type: pm2_vite with errors) -> Project 4 (build tooling)
|
||||
#
|
||||
# Related Documentation:
|
||||
# - docs/adr/0050-postgresql-function-observability.md
|
||||
@@ -344,26 +353,56 @@ filter {
|
||||
}
|
||||
|
||||
# ============================================================================
|
||||
# Generate Sentry Event ID for all errors
|
||||
# Generate Sentry Event ID and Ensure Required Fields for all errors
|
||||
# ============================================================================
|
||||
# CRITICAL: sentry_level MUST be set for all errors before output.
|
||||
# Bugsink's PostgreSQL schema limits level to varchar(7), so valid values are:
|
||||
# fatal, error, warning, info, debug (all <= 7 chars)
|
||||
# If sentry_level is not set, the literal "%{sentry_level}" (16 chars) is sent,
|
||||
# causing PostgreSQL insertion failures.
|
||||
# ============================================================================
|
||||
if "error" in [tags] {
|
||||
# Use Ruby for robust field handling - handles all edge cases
|
||||
ruby {
|
||||
code => '
|
||||
require "securerandom"
|
||||
event.set("sentry_event_id", SecureRandom.hex(16))
|
||||
'
|
||||
}
|
||||
|
||||
# Ensure error_message has a fallback value
|
||||
if ![error_message] {
|
||||
mutate { add_field => { "error_message" => "%{message}" } }
|
||||
# Generate unique event ID for Sentry
|
||||
event.set("sentry_event_id", SecureRandom.hex(16))
|
||||
|
||||
# =====================================================================
|
||||
# CRITICAL: Validate and set sentry_level
|
||||
# =====================================================================
|
||||
# Valid Sentry levels (max 7 chars for Bugsink PostgreSQL schema):
|
||||
# fatal, error, warning, info, debug
|
||||
# Default to "error" if missing, empty, or invalid.
|
||||
# =====================================================================
|
||||
valid_levels = ["fatal", "error", "warning", "info", "debug"]
|
||||
current_level = event.get("sentry_level")
|
||||
|
||||
if current_level.nil? || current_level.to_s.strip.empty? || !valid_levels.include?(current_level.to_s.downcase)
|
||||
event.set("sentry_level", "error")
|
||||
else
|
||||
# Normalize to lowercase
|
||||
event.set("sentry_level", current_level.to_s.downcase)
|
||||
end
|
||||
|
||||
# =====================================================================
|
||||
# Ensure error_message has a fallback value
|
||||
# =====================================================================
|
||||
error_msg = event.get("error_message")
|
||||
if error_msg.nil? || error_msg.to_s.strip.empty?
|
||||
fallback_msg = event.get("message") || event.get("msg") || "Unknown error"
|
||||
event.set("error_message", fallback_msg.to_s)
|
||||
end
|
||||
'
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
output {
|
||||
# ============================================================================
|
||||
# Forward Errors to Bugsink (Backend API Project)
|
||||
# Forward Errors to Bugsink (Project Routing)
|
||||
# ============================================================================
|
||||
# Bugsink uses Sentry-compatible API. Events must include:
|
||||
# - event_id: 32 hex characters (UUID without dashes)
|
||||
@@ -373,9 +412,50 @@ output {
|
||||
# - platform: "node" for backend, "javascript" for frontend
|
||||
#
|
||||
# Authentication via X-Sentry-Auth header with project's public key.
|
||||
# Dev container DSN: http://cea01396c56246adb5878fa5ee6b1d22@localhost:8000/1
|
||||
#
|
||||
# Project Routing:
|
||||
# - Project 1 (Backend): Pino app logs, PostgreSQL errors
|
||||
# - Project 4 (Infrastructure): Redis, NGINX, Vite build errors
|
||||
# ============================================================================
|
||||
if "error" in [tags] {
|
||||
|
||||
# ============================================================================
|
||||
# Infrastructure Errors -> Project 4
|
||||
# ============================================================================
|
||||
# Redis warnings/errors, NGINX errors, and Vite build errors go to
|
||||
# the Infrastructure project for separation from application code errors.
|
||||
if "error" in [tags] and ([type] == "redis" or [type] == "nginx_error" or [type] == "nginx_access" or [type] == "pm2_vite") {
|
||||
http {
|
||||
url => "http://localhost:8000/api/4/store/"
|
||||
http_method => "post"
|
||||
format => "json"
|
||||
headers => {
|
||||
"X-Sentry-Auth" => "Sentry sentry_key=14e8791da3d347fa98073261b596cab9, sentry_version=7"
|
||||
"Content-Type" => "application/json"
|
||||
}
|
||||
mapping => {
|
||||
"event_id" => "%{sentry_event_id}"
|
||||
"timestamp" => "%{@timestamp}"
|
||||
"level" => "%{sentry_level}"
|
||||
"platform" => "other"
|
||||
"logger" => "%{type}"
|
||||
"message" => "%{error_message}"
|
||||
"extra" => {
|
||||
"hostname" => "%{[host][name]}"
|
||||
"source_type" => "%{type}"
|
||||
"tags" => "%{tags}"
|
||||
"original_message" => "%{message}"
|
||||
"project" => "infrastructure"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
# ============================================================================
|
||||
# Backend Application Errors -> Project 1
|
||||
# ============================================================================
|
||||
# Pino application logs (API, Worker), PostgreSQL function errors, and
|
||||
# native PostgreSQL errors go to the Backend API project.
|
||||
else if "error" in [tags] and ([type] in ["pm2_api", "pm2_worker", "pino", "postgres"]) {
|
||||
http {
|
||||
url => "http://localhost:8000/api/1/store/"
|
||||
http_method => "post"
|
||||
@@ -384,7 +464,6 @@ output {
|
||||
"X-Sentry-Auth" => "Sentry sentry_key=cea01396c56246adb5878fa5ee6b1d22, sentry_version=7"
|
||||
"Content-Type" => "application/json"
|
||||
}
|
||||
# Transform event to Sentry format using regular fields (not @metadata)
|
||||
mapping => {
|
||||
"event_id" => "%{sentry_event_id}"
|
||||
"timestamp" => "%{@timestamp}"
|
||||
@@ -397,6 +476,38 @@ output {
|
||||
"source_type" => "%{type}"
|
||||
"tags" => "%{tags}"
|
||||
"original_message" => "%{message}"
|
||||
"project" => "backend"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
# ============================================================================
|
||||
# Fallback: Any other errors -> Project 1
|
||||
# ============================================================================
|
||||
# Catch-all for any errors that don't match specific routing rules.
|
||||
else if "error" in [tags] {
|
||||
http {
|
||||
url => "http://localhost:8000/api/1/store/"
|
||||
http_method => "post"
|
||||
format => "json"
|
||||
headers => {
|
||||
"X-Sentry-Auth" => "Sentry sentry_key=cea01396c56246adb5878fa5ee6b1d22, sentry_version=7"
|
||||
"Content-Type" => "application/json"
|
||||
}
|
||||
mapping => {
|
||||
"event_id" => "%{sentry_event_id}"
|
||||
"timestamp" => "%{@timestamp}"
|
||||
"level" => "%{sentry_level}"
|
||||
"platform" => "node"
|
||||
"logger" => "%{type}"
|
||||
"message" => "%{error_message}"
|
||||
"extra" => {
|
||||
"hostname" => "%{[host][name]}"
|
||||
"source_type" => "%{type}"
|
||||
"tags" => "%{tags}"
|
||||
"original_message" => "%{message}"
|
||||
"project" => "backend-fallback"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -60,6 +60,37 @@ server {
|
||||
proxy_set_header X-Forwarded-Proto $scheme;
|
||||
}
|
||||
|
||||
# ============================================================================
|
||||
# Bugsink Sentry API Proxy (for frontend error reporting)
|
||||
# ============================================================================
|
||||
# The frontend Sentry SDK cannot reach localhost:8000 directly from the browser
|
||||
# because port 8000 is only accessible within the container network.
|
||||
# This proxy allows the browser to send errors to https://localhost/bugsink-api/
|
||||
# which NGINX forwards to the Bugsink container on port 8000.
|
||||
#
|
||||
# Frontend DSN format: https://localhost/bugsink-api/<project_id>
|
||||
# Example: https://localhost/bugsink-api/2 for Frontend (Dev) project
|
||||
#
|
||||
# The Sentry SDK sends POST requests to /bugsink-api/<project>/store/
|
||||
# This proxy strips /bugsink-api and forwards to http://localhost:8000/api/
|
||||
# ============================================================================
|
||||
location /bugsink-api/ {
|
||||
proxy_pass http://localhost:8000/api/;
|
||||
proxy_http_version 1.1;
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
proxy_set_header X-Forwarded-Proto $scheme;
|
||||
|
||||
# Allow large error payloads with stack traces
|
||||
client_max_body_size 10M;
|
||||
|
||||
# Timeouts for error reporting (should be fast)
|
||||
proxy_connect_timeout 10s;
|
||||
proxy_send_timeout 30s;
|
||||
proxy_read_timeout 30s;
|
||||
}
|
||||
|
||||
# Proxy WebSocket connections for real-time notifications
|
||||
location /ws {
|
||||
proxy_pass http://localhost:3001;
|
||||
|
||||
@@ -28,9 +28,28 @@ The `.env.local` file uses `localhost` while `compose.dev.yml` uses `127.0.0.1`.
|
||||
## HTTPS Setup
|
||||
|
||||
- Self-signed certificates auto-generated with mkcert on container startup
|
||||
- CSRF Protection: Django configured with `SECURE_PROXY_SSL_HEADER` to trust `X-Forwarded-Proto` from nginx
|
||||
- CSRF Protection: Django configured with `CSRF_TRUSTED_ORIGINS` for both `localhost` and `127.0.0.1` (see below)
|
||||
- HTTPS proxy: nginx on port 8443 proxies to Bugsink on port 8000
|
||||
- HTTPS is for UI access only - Sentry SDK uses HTTP directly
|
||||
|
||||
### CSRF Configuration
|
||||
|
||||
Django 4.0+ requires `CSRF_TRUSTED_ORIGINS` for HTTPS POST requests. The Bugsink configuration (`Dockerfile.dev`) includes:
|
||||
|
||||
```python
|
||||
CSRF_TRUSTED_ORIGINS = [
|
||||
"https://localhost:8443",
|
||||
"https://127.0.0.1:8443",
|
||||
"http://localhost:8000",
|
||||
"http://127.0.0.1:8000",
|
||||
]
|
||||
SECURE_PROXY_SSL_HEADER = ("HTTP_X_FORWARDED_PROTO", "https")
|
||||
```
|
||||
|
||||
**Both hostnames are required** because browsers treat `localhost` and `127.0.0.1` as different origins.
|
||||
|
||||
If you get "CSRF verification failed" errors, see [BUGSINK-SETUP.md](tools/BUGSINK-SETUP.md#csrf-verification-failed) for troubleshooting.
|
||||
|
||||
## Isolation Benefits
|
||||
|
||||
- Dev errors stay local, don't pollute production/test dashboards
|
||||
|
||||
@@ -175,29 +175,30 @@ npm run dev:pm2:logs
|
||||
|
||||
### Log Flow Architecture (ADR-050)
|
||||
|
||||
All application logs flow through Logstash to Bugsink:
|
||||
All application logs flow through Logstash to Bugsink using a 3-project architecture:
|
||||
|
||||
```
|
||||
```text
|
||||
+------------------+ +------------------+ +------------------+
|
||||
| PM2 Logs | | PostgreSQL | | Redis Logs |
|
||||
| PM2 Logs | | PostgreSQL | | Redis/NGINX |
|
||||
| /var/log/pm2/ | | /var/log/ | | /var/log/redis/ |
|
||||
+--------+---------+ | postgresql/ | +--------+---------+
|
||||
| +--------+---------+ |
|
||||
| (API + Worker) | | postgresql/ | | /var/log/nginx/ |
|
||||
+--------+---------+ +--------+---------+ +--------+---------+
|
||||
| | |
|
||||
v v v
|
||||
+------------------------------------------------------------------------+
|
||||
| LOGSTASH |
|
||||
| /etc/logstash/conf.d/bugsink.conf |
|
||||
| (Routes by log type) |
|
||||
+------------------------------------------------------------------------+
|
||||
| | |
|
||||
| +---------+---------+ |
|
||||
| | | |
|
||||
v v v v
|
||||
+------------------+ +------------------+ +------------------+
|
||||
| Errors -> | | Operational -> | | NGINX Logs -> |
|
||||
| Bugsink API | | /var/log/ | | /var/log/ |
|
||||
| (Project 1) | | logstash/*.log | | logstash/*.log |
|
||||
+------------------+ +------------------+ +------------------+
|
||||
v v v
|
||||
+------------------+ +------------------+ +------------------+
|
||||
| Backend API | | Frontend (Dev) | | Infrastructure |
|
||||
| (Project 1) | | (Project 2) | | (Project 4) |
|
||||
| - Pino errors | | - Browser SDK | | - Redis warnings |
|
||||
| - PostgreSQL | | (not Logstash) | | - NGINX errors |
|
||||
+------------------+ +------------------+ | - Vite errors |
|
||||
+------------------+
|
||||
```
|
||||
|
||||
### Log Sources
|
||||
@@ -231,8 +232,11 @@ podman exec flyer-crawler-dev curl -s localhost:9600/_node/stats/pipelines?prett
|
||||
- **URL**: `https://localhost:8443`
|
||||
- **Login**: `admin@localhost` / `admin`
|
||||
- **Projects**:
|
||||
- Project 1: Backend API (errors from Pino, PostgreSQL, Redis)
|
||||
- Project 2: Frontend (errors from Sentry SDK in browser)
|
||||
- Project 1: Backend API (Dev) - Pino app errors, PostgreSQL errors
|
||||
- Project 2: Frontend (Dev) - Browser errors via Sentry SDK
|
||||
- Project 4: Infrastructure (Dev) - Redis warnings, NGINX errors, Vite build errors
|
||||
|
||||
**Note**: Frontend DSN uses nginx proxy (`/bugsink-api/`) because browsers cannot reach `localhost:8000` directly. See [BUGSINK-SETUP.md](../tools/BUGSINK-SETUP.md#frontend-nginx-proxy) for details.
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -49,18 +49,24 @@ Bugsink is a lightweight, self-hosted error tracking platform that is fully comp
|
||||
| Web UI | `https://localhost:8443` (nginx proxy) |
|
||||
| Internal URL | `http://localhost:8000` (direct) |
|
||||
| Credentials | `admin@localhost` / `admin` |
|
||||
| Backend Project | Project ID 1 - `flyer-crawler-dev-backend` |
|
||||
| Frontend Project | Project ID 2 - `flyer-crawler-dev-frontend` |
|
||||
| Backend Project | Project ID 1 - `Backend API (Dev)` |
|
||||
| Frontend Project | Project ID 2 - `Frontend (Dev)` |
|
||||
| Infra Project | Project ID 4 - `Infrastructure (Dev)` |
|
||||
| Backend DSN | `http://<key>@localhost:8000/1` |
|
||||
| Frontend DSN | `http://<key>@localhost:8000/2` |
|
||||
| Frontend DSN | `https://<key>@localhost/bugsink-api/2` (via nginx proxy) |
|
||||
| Infra DSN | `http://<key>@localhost:8000/4` (Logstash only) |
|
||||
| Database | `postgresql://bugsink:bugsink_dev_password@postgres:5432/bugsink` |
|
||||
|
||||
**Important:** The Frontend DSN uses an nginx proxy (`/bugsink-api/`) because the browser cannot reach `localhost:8000` directly (container-internal port). See [Frontend Nginx Proxy](#frontend-nginx-proxy) for details.
|
||||
|
||||
**Configuration Files:**
|
||||
|
||||
| File | Purpose |
|
||||
| ----------------- | ----------------------------------------------------------------- |
|
||||
| `compose.dev.yml` | Initial DSNs using `127.0.0.1:8000` (container startup) |
|
||||
| `.env.local` | **OVERRIDES** compose.dev.yml with `localhost:8000` (app runtime) |
|
||||
| File | Purpose |
|
||||
| ------------------------------ | ------------------------------------------------------- |
|
||||
| `compose.dev.yml` | Initial DSNs using `127.0.0.1:8000` (container startup) |
|
||||
| `.env.local` | **OVERRIDES** compose.dev.yml (app runtime) |
|
||||
| `docker/nginx/dev.conf` | Nginx proxy for Bugsink API (frontend error reporting) |
|
||||
| `docker/logstash/bugsink.conf` | Log routing to Backend/Infrastructure projects |
|
||||
|
||||
**Note:** `.env.local` takes precedence over `compose.dev.yml` environment variables.
|
||||
|
||||
@@ -360,75 +366,127 @@ const config = {
|
||||
|
||||
---
|
||||
|
||||
## Frontend Nginx Proxy
|
||||
|
||||
The frontend Sentry SDK runs in the browser, which cannot directly reach `localhost:8000` (the Bugsink container-internal port). To solve this, we use an nginx proxy.
|
||||
|
||||
### How It Works
|
||||
|
||||
```text
|
||||
Browser --HTTPS--> https://localhost/bugsink-api/2/store/
|
||||
|
|
||||
v (nginx proxy)
|
||||
http://localhost:8000/api/2/store/
|
||||
|
|
||||
v
|
||||
Bugsink (internal)
|
||||
```
|
||||
|
||||
### Nginx Configuration
|
||||
|
||||
Location: `docker/nginx/dev.conf`
|
||||
|
||||
```nginx
|
||||
# Proxy Bugsink Sentry API for frontend error reporting
|
||||
location /bugsink-api/ {
|
||||
proxy_pass http://localhost:8000/api/;
|
||||
proxy_http_version 1.1;
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
proxy_set_header X-Forwarded-Proto $scheme;
|
||||
|
||||
# Allow large error payloads with stack traces
|
||||
client_max_body_size 10M;
|
||||
|
||||
# Timeouts for error reporting
|
||||
proxy_connect_timeout 10s;
|
||||
proxy_send_timeout 30s;
|
||||
proxy_read_timeout 30s;
|
||||
}
|
||||
```
|
||||
|
||||
### Frontend DSN Format
|
||||
|
||||
```bash
|
||||
# .env.local
|
||||
# Uses nginx proxy path instead of direct port
|
||||
VITE_SENTRY_DSN=https://<key>@localhost/bugsink-api/2
|
||||
```
|
||||
|
||||
### Testing Frontend Error Reporting
|
||||
|
||||
1. Open browser console at `https://localhost`
|
||||
|
||||
2. Trigger a test error:
|
||||
|
||||
```javascript
|
||||
throw new Error('Test frontend error from browser');
|
||||
```
|
||||
|
||||
3. Check Bugsink Frontend (Dev) project for the error
|
||||
|
||||
4. Verify browser console shows Sentry SDK activity (if VITE_SENTRY_DEBUG=true)
|
||||
|
||||
---
|
||||
|
||||
## Logstash Integration
|
||||
|
||||
Logstash aggregates logs from multiple sources and forwards error patterns to Bugsink.
|
||||
|
||||
**Note:** See [ADR-015](../adr/0015-application-performance-monitoring-and-error-tracking.md) for the full architecture.
|
||||
|
||||
### 3-Project Architecture
|
||||
|
||||
Logstash routes errors to different Bugsink projects based on log source:
|
||||
|
||||
| Project | ID | Receives |
|
||||
| -------------------- | --- | --------------------------------------------- |
|
||||
| Backend API (Dev) | 1 | Pino app errors, PostgreSQL errors |
|
||||
| Frontend (Dev) | 2 | Browser errors (via Sentry SDK, not Logstash) |
|
||||
| Infrastructure (Dev) | 4 | Redis warnings, NGINX errors, Vite errors |
|
||||
|
||||
### Log Sources
|
||||
|
||||
| Source | Log Path | Error Detection |
|
||||
| ---------- | ---------------------- | ------------------------- |
|
||||
| Pino (app) | `/app/logs/*.log` | level >= 50 (error/fatal) |
|
||||
| Redis | `/var/log/redis/*.log` | WARNING/ERROR log levels |
|
||||
| PostgreSQL | (future) | ERROR/FATAL log levels |
|
||||
| Source | Log Path | Project Destination | Error Detection |
|
||||
| ---------- | --------------------------- | ------------------- | ------------------------- |
|
||||
| PM2 API | `/var/log/pm2/api-*.log` | Backend (1) | level >= 50 (error/fatal) |
|
||||
| PM2 Worker | `/var/log/pm2/worker-*.log` | Backend (1) | level >= 50 (error/fatal) |
|
||||
| PM2 Vite | `/var/log/pm2/vite-*.log` | Infrastructure (4) | error keyword patterns |
|
||||
| PostgreSQL | `/var/log/postgresql/*.log` | Backend (1) | ERROR/FATAL log levels |
|
||||
| Redis | `/var/log/redis/*.log` | Infrastructure (4) | WARNING level (`#`) |
|
||||
| NGINX | `/var/log/nginx/error.log` | Infrastructure (4) | error/crit/alert/emerg |
|
||||
|
||||
### Pipeline Configuration
|
||||
|
||||
**Location:** `/etc/logstash/conf.d/bugsink.conf`
|
||||
**Location:** `/etc/logstash/conf.d/bugsink.conf` (or `docker/logstash/bugsink.conf` in project)
|
||||
|
||||
```conf
|
||||
# === INPUTS ===
|
||||
input {
|
||||
file {
|
||||
path => "/app/logs/*.log"
|
||||
codec => json
|
||||
type => "pino"
|
||||
tags => ["app"]
|
||||
}
|
||||
The configuration:
|
||||
|
||||
file {
|
||||
path => "/var/log/redis/*.log"
|
||||
type => "redis"
|
||||
tags => ["redis"]
|
||||
}
|
||||
1. **Inputs**: Reads from PM2 logs, PostgreSQL logs, Redis logs, NGINX logs
|
||||
2. **Filters**: Detects errors and assigns tags based on log type
|
||||
3. **Outputs**: Routes to appropriate Bugsink project based on log source
|
||||
|
||||
**Key Routing Logic:**
|
||||
|
||||
```ruby
|
||||
# Infrastructure logs -> Project 4
|
||||
if "error" in [tags] and ([type] == "redis" or [type] == "nginx_error" or [type] == "pm2_vite") {
|
||||
http { url => "http://localhost:8000/api/4/store/" ... }
|
||||
}
|
||||
|
||||
# === FILTERS ===
|
||||
filter {
|
||||
if [type] == "pino" and [level] >= 50 {
|
||||
mutate { add_tag => ["error"] }
|
||||
}
|
||||
|
||||
if [type] == "redis" {
|
||||
grok {
|
||||
match => { "message" => "%{POSINT:pid}:%{WORD:role} %{MONTHDAY} %{MONTH} %{TIME} %{WORD:loglevel} %{GREEDYDATA:redis_message}" }
|
||||
}
|
||||
if [loglevel] in ["WARNING", "ERROR"] {
|
||||
mutate { add_tag => ["error"] }
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
# === OUTPUT ===
|
||||
output {
|
||||
if "error" in [tags] {
|
||||
http {
|
||||
url => "http://localhost:8000/api/store/"
|
||||
http_method => "post"
|
||||
format => "json"
|
||||
}
|
||||
}
|
||||
# Backend logs -> Project 1
|
||||
else if "error" in [tags] and ([type] in ["pm2_api", "pm2_worker", "pino", "postgres"]) {
|
||||
http { url => "http://localhost:8000/api/1/store/" ... }
|
||||
}
|
||||
```
|
||||
|
||||
### Benefits
|
||||
|
||||
1. **Secondary Capture Path**: Catches errors before SDK initialization
|
||||
2. **Log-Based Errors**: Captures errors that don't throw exceptions
|
||||
3. **Infrastructure Monitoring**: Redis connection issues, slow commands
|
||||
4. **Historical Analysis**: Process existing log files
|
||||
1. **Separation of Concerns**: Application errors separate from infrastructure issues
|
||||
2. **Secondary Capture Path**: Catches errors before SDK initialization
|
||||
3. **Log-Based Errors**: Captures errors that don't throw exceptions
|
||||
4. **Infrastructure Monitoring**: Redis, NGINX, build tooling issues
|
||||
5. **Historical Analysis**: Process existing log files
|
||||
|
||||
---
|
||||
|
||||
@@ -743,6 +801,76 @@ podman exec flyer-crawler-dev psql -U postgres -h postgres -c "\l" | grep bugsin
|
||||
ssh root@projectium.com "cd /opt/bugsink && bugsink-manage check"
|
||||
```
|
||||
|
||||
### CSRF Verification Failed
|
||||
|
||||
**Symptoms:** "CSRF verification failed. Request aborted." error when performing actions in Bugsink UI (resolving issues, changing settings, etc.)
|
||||
|
||||
**Root Cause:**
|
||||
|
||||
Django 4.0+ requires `CSRF_TRUSTED_ORIGINS` to be explicitly configured for HTTPS POST requests. The error occurs because:
|
||||
|
||||
1. Bugsink is accessed via `https://localhost:8443` (nginx HTTPS proxy)
|
||||
2. Django's CSRF protection validates the `Origin` header against `CSRF_TRUSTED_ORIGINS`
|
||||
3. Without explicit configuration, Django rejects POST requests from HTTPS origins
|
||||
|
||||
**Why localhost vs 127.0.0.1 Matters:**
|
||||
|
||||
- `localhost` and `127.0.0.1` are treated as DIFFERENT origins by browsers
|
||||
- If you access Bugsink via `https://localhost:8443`, Django must trust `https://localhost:8443`
|
||||
- If you access via `https://127.0.0.1:8443`, Django must trust `https://127.0.0.1:8443`
|
||||
- The fix includes BOTH to allow either access pattern
|
||||
|
||||
**Configuration (Already Applied):**
|
||||
|
||||
The Bugsink Django configuration in `Dockerfile.dev` includes:
|
||||
|
||||
```python
|
||||
# CSRF Trusted Origins (Django 4.0+ requires full origin for HTTPS POST requests)
|
||||
CSRF_TRUSTED_ORIGINS = [
|
||||
"https://localhost:8443",
|
||||
"https://127.0.0.1:8443",
|
||||
"http://localhost:8000",
|
||||
"http://127.0.0.1:8000",
|
||||
]
|
||||
|
||||
# HTTPS proxy support (nginx reverse proxy on port 8443)
|
||||
SECURE_PROXY_SSL_HEADER = ("HTTP_X_FORWARDED_PROTO", "https")
|
||||
```
|
||||
|
||||
**Verification:**
|
||||
|
||||
```bash
|
||||
# Verify CSRF_TRUSTED_ORIGINS is configured
|
||||
podman exec flyer-crawler-dev sh -c 'cat /opt/bugsink/conf/bugsink_conf.py | grep -A 6 CSRF_TRUSTED'
|
||||
|
||||
# Expected output:
|
||||
# CSRF_TRUSTED_ORIGINS = [
|
||||
# "https://localhost:8443",
|
||||
# "https://127.0.0.1:8443",
|
||||
# "http://localhost:8000",
|
||||
# "http://127.0.0.1:8000",
|
||||
# ]
|
||||
```
|
||||
|
||||
**If Issue Persists After Fix:**
|
||||
|
||||
1. **Rebuild the container image** (configuration is baked into the image):
|
||||
|
||||
```bash
|
||||
podman-compose -f compose.dev.yml down
|
||||
podman build -f Dockerfile.dev -t localhost/flyer-crawler-dev:latest .
|
||||
podman-compose -f compose.dev.yml up -d
|
||||
```
|
||||
|
||||
2. **Clear browser cookies** for localhost:8443
|
||||
|
||||
3. **Check nginx X-Forwarded-Proto header** - the nginx config must set this header for Django to recognize HTTPS:
|
||||
|
||||
```bash
|
||||
podman exec flyer-crawler-dev cat /etc/nginx/sites-available/bugsink | grep X-Forwarded-Proto
|
||||
# Should show: proxy_set_header X-Forwarded-Proto $scheme;
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Related Documentation
|
||||
|
||||
4
package-lock.json
generated
4
package-lock.json
generated
@@ -1,12 +1,12 @@
|
||||
{
|
||||
"name": "flyer-crawler",
|
||||
"version": "0.12.8",
|
||||
"version": "0.12.9",
|
||||
"lockfileVersion": 3,
|
||||
"requires": true,
|
||||
"packages": {
|
||||
"": {
|
||||
"name": "flyer-crawler",
|
||||
"version": "0.12.8",
|
||||
"version": "0.12.9",
|
||||
"dependencies": {
|
||||
"@bull-board/api": "^6.14.2",
|
||||
"@bull-board/express": "^6.14.2",
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
{
|
||||
"name": "flyer-crawler",
|
||||
"private": true,
|
||||
"version": "0.12.8",
|
||||
"version": "0.12.9",
|
||||
"type": "module",
|
||||
"scripts": {
|
||||
"dev": "concurrently \"npm:start:dev\" \"vite\"",
|
||||
|
||||
Reference in New Issue
Block a user