flyer-crawler.projectium.com/docs/adr/ADR-029-error-tracking-with-bugsink.md

ADR-029: Error Tracking and Observability with Bugsink

Date: 2026-02-10

Status: Accepted

Source: Imported from flyer-crawler project (ADR-015)

Related: ADR-027, ADR-028, ADR-030, ADR-032

Context

While ADR-027 established structured logging with Pino, the application lacks a high-level, aggregated view of its health and errors. It is difficult to spot trends, identify recurring issues, or be proactively notified of new types of errors.

Key requirements:

1. Self-hosted: No external SaaS dependencies for error tracking
2. Sentry SDK compatible: Leverage mature, well-documented SDKs
3. Lightweight: Minimal resource overhead in the dev container
4. Production-ready: Same architecture works on bare-metal production servers
5. AI-accessible: MCP server integration for Claude Code and other AI tools

Note: Application Performance Monitoring (APM) and distributed tracing are covered separately in ADR-032.

Decision

We implement a self-hosted error tracking stack using Bugsink as the Sentry-compatible backend, with the following components:

1. Error Tracking Backend: Bugsink

Bugsink is a lightweight, self-hosted Sentry alternative that:

• Runs as a single process (no Kafka, Redis, ClickHouse required)
• Is fully compatible with Sentry SDKs
• Supports ARM64 and AMD64 architectures
• Can use SQLite (dev) or PostgreSQL (production)

Deployment:

• Dev container: Installed as a systemd service inside the container
• Production: Runs as a systemd service on bare-metal, listening on localhost only
• Database: Uses PostgreSQL with a dedicated bugsink user and bugsink database (same PostgreSQL instance as the main application)

2. Backend Integration: @sentry/node

The Express backend integrates @sentry/node SDK to:

• Capture unhandled exceptions before PM2/process manager restarts
• Report errors with full stack traces and context
• Integrate with Pino logger for breadcrumbs
• Filter errors by severity (only 5xx errors sent by default)

3. Frontend Integration: @sentry/react

The React frontend integrates @sentry/react SDK to:

• Wrap the app in an Error Boundary for graceful error handling
• Capture unhandled JavaScript errors
• Report errors with component stack traces
• Filter out browser extension errors
• Frontend Error Correlation: The global API client intercepts 4xx/5xx responses and can attach the x-request-id header to Sentry scope for correlation with backend logs

4. Log Aggregation: Logstash

Logstash parses application and infrastructure logs, forwarding error patterns to Bugsink:

• Installation: Installed inside the dev container (and on bare-metal prod servers)
• Inputs:
  • Pino JSON logs from the Node.js application (PM2 managed)
  • Redis logs (connection errors, memory warnings, slow commands)
  • PostgreSQL function logs (via fn_log() - see ADR-030)
  • NGINX access/error logs
• Filter: Identifies error-level logs (5xx responses, unhandled exceptions, Redis errors)
• Output: Sends to Bugsink via Sentry-compatible HTTP API

This provides a secondary error capture path for:

• Errors that occur before Sentry SDK initialization
• Log-based errors that do not throw exceptions
• Redis connection/performance issues
• Database function errors and slow queries
• Historical error analysis from log files

5. MCP Server Integration: bugsink-mcp

For AI tool integration (Claude Code, Cursor, etc.), we use the open-source bugsink-mcp server:

• No code changes required: Configurable via environment variables
• Capabilities: List projects, get issues, view events, get stacktraces, manage releases
• Configuration:
  • BUGSINK_URL: Points to Bugsink instance (http://localhost:8000 for dev, https://bugsink.example.com for prod)
  • BUGSINK_API_TOKEN: API token from Bugsink (created via Django management command)
  • BUGSINK_ORG_SLUG: Organization identifier (usually "sentry")

Architecture

+---------------------------------------------------------------------------+
|                      Dev Container / Production Server                     |
+---------------------------------------------------------------------------+
|                                                                            |
|  +------------------+        +------------------+                          |
|  |    Frontend      |        |     Backend      |                          |
|  |    (React)       |        |    (Express)     |                          |
|  |  @sentry/react   |        |   @sentry/node   |                          |
|  +--------+---------+        +--------+---------+                          |
|           |                           |                                    |
|           |    Sentry SDK Protocol    |                                    |
|           +-----------+---------------+                                    |
|                       |                                                    |
|                       v                                                    |
|           +----------------------+                                         |
|           |      Bugsink         |                                         |
|           |   (localhost:8000)   |<------------------+                     |
|           |                      |                   |                     |
|           |  PostgreSQL backend  |                   |                     |
|           +----------------------+                   |                     |
|                                                      |                     |
|           +----------------------+                   |                     |
|           |      Logstash        |-------------------+                     |
|           |   (Log Aggregator)   |   Sentry Output                         |
|           |                      |                                         |
|           |  Inputs:             |                                         |
|           |  - PM2/Pino logs     |                                         |
|           |  - Redis logs        |                                         |
|           |  - PostgreSQL logs   |                                         |
|           |  - NGINX logs        |                                         |
|           +----------------------+                                         |
|                   ^   ^   ^   ^                                            |
|                   |   |   |   |                                            |
|       +-----------+   |   |   +-----------+                                |
|       |               |   |               |                                |
|  +----+-----+   +-----+----+   +-----+----+   +-----+----+                 |
|  |  PM2     |   |  Redis   |   | PostgreSQL|   |  NGINX  |                 |
|  |  Logs    |   |  Logs    |   |   Logs    |   |  Logs   |                 |
|  +----------+   +----------+   +-----------+   +---------+                 |
|                                                                            |
|           +----------------------+                                         |
|           |     PostgreSQL       |                                         |
|           |  +----------------+  |                                         |
|           |  | app_database   |  |  (main app database)                    |
|           |  +----------------+  |                                         |
|           |  |   bugsink      |  |  (error tracking database)              |
|           |  +----------------+  |                                         |
|           +----------------------+                                         |
|                                                                            |
+---------------------------------------------------------------------------+

External (Developer Machine):
+--------------------------------------+
|  Claude Code / Cursor / VS Code      |
|  +--------------------------------+  |
|  |  bugsink-mcp                   |  |
|  |  (MCP Server)                  |  |
|  |                                |  |
|  |  BUGSINK_URL=http://localhost:8000
|  |  BUGSINK_API_TOKEN=...         |  |
|  |  BUGSINK_ORG_SLUG=...          |  |
|  +--------------------------------+  |
+--------------------------------------+

Implementation Details

Environment Variables

Variable	Description	Default (Dev)
SENTRY_DSN	Sentry-compatible DSN (backend)	Set after project creation
VITE_SENTRY_DSN	Sentry-compatible DSN (frontend)	Set after project creation
SENTRY_ENVIRONMENT	Environment name	development
SENTRY_DEBUG	Enable debug logging	false
SENTRY_ENABLED	Enable/disable error reporting	true

PostgreSQL Setup

-- Create dedicated Bugsink database and user
CREATE USER bugsink WITH PASSWORD 'bugsink_dev_password';
CREATE DATABASE bugsink OWNER bugsink;
GRANT ALL PRIVILEGES ON DATABASE bugsink TO bugsink;

Bugsink Configuration

# Environment variables for Bugsink service
SECRET_KEY=<random-50-char-string>
DATABASE_URL=postgresql://bugsink:bugsink_dev_password@localhost:5432/bugsink
BASE_URL=http://localhost:8000
PORT=8000

Backend Sentry Integration

Located in src/services/sentry.server.ts:

import * as Sentry from '@sentry/node';
import { config } from '../config/env';

export function initSentry() {
  if (!config.sentry.enabled || !config.sentry.dsn) {
    return;
  }

  Sentry.init({
    dsn: config.sentry.dsn,
    environment: config.sentry.environment || config.server.nodeEnv,
    debug: config.sentry.debug,

    // Performance monitoring - disabled by default (see ADR-032)
    tracesSampleRate: 0,

    // Filter out 4xx errors - only report server errors
    beforeSend(event) {
      const statusCode = event.contexts?.response?.status_code;
      if (statusCode && statusCode >= 400 && statusCode < 500) {
        return null;
      }
      return event;
    },
  });
}

// Set user context after authentication
export function setUserContext(user: { id: string; email: string; name?: string }) {
  Sentry.setUser({
    id: user.id,
    email: user.email,
    username: user.name,
  });
}

// Clear user context on logout
export function clearUserContext() {
  Sentry.setUser(null);
}

Frontend Sentry Integration

Located in src/services/sentry.client.ts:

import * as Sentry from '@sentry/react';
import { config } from '../config';

export function initSentry() {
  if (!config.sentry.enabled || !config.sentry.dsn) {
    return;
  }

  Sentry.init({
    dsn: config.sentry.dsn,
    environment: config.sentry.environment,

    // Performance monitoring - disabled by default (see ADR-032)
    tracesSampleRate: 0,

    // Filter out browser extension errors
    beforeSend(event) {
      // Ignore errors from browser extensions
      if (
        event.exception?.values?.[0]?.stacktrace?.frames?.some((frame) =>
          frame.filename?.includes('extension://'),
        )
      ) {
        return null;
      }
      return event;
    },
  });
}

// Set user context after login
export function setUserContext(user: { id: string; email: string; name?: string }) {
  Sentry.setUser({
    id: user.id,
    email: user.email,
    username: user.name,
  });
}

// Clear user context on logout
export function clearUserContext() {
  Sentry.setUser(null);
}

Error Boundary Component

Located in src/components/ErrorBoundary.tsx:

import * as Sentry from '@sentry/react';
import { Component, ErrorInfo, ReactNode } from 'react';

interface Props {
  children: ReactNode;
  fallback?: ReactNode;
}

interface State {
  hasError: boolean;
}

export class ErrorBoundary extends Component<Props, State> {
  constructor(props: Props) {
    super(props);
    this.state = { hasError: false };
  }

  static getDerivedStateFromError(): State {
    return { hasError: true };
  }

  componentDidCatch(error: Error, errorInfo: ErrorInfo) {
    Sentry.withScope((scope) => {
      scope.setExtras({ componentStack: errorInfo.componentStack });
      Sentry.captureException(error);
    });
  }

  render() {
    if (this.state.hasError) {
      return this.props.fallback || (
        <div className="error-boundary">
          <h1>Something went wrong</h1>
          <p>Please refresh the page or contact support.</p>
        </div>
      );
    }

    return this.props.children;
  }
}

Logstash Pipeline Configuration

Key routing for log sources:

Source	Bugsink Project
Backend (Pino)	Backend API
Worker (Pino)	Backend API
PostgreSQL logs	Backend API
Vite logs	Infrastructure
Redis logs	Infrastructure
NGINX logs	Infrastructure
Frontend errors	Frontend

Consequences

Positive

• Full observability: Aggregated view of errors and trends
• Self-hosted: No external SaaS dependencies or subscription costs
• SDK compatibility: Leverages mature Sentry SDKs with excellent documentation
• AI integration: MCP server enables Claude Code to query and analyze errors
• Unified architecture: Same setup works in dev container and production
• Lightweight: Bugsink runs in a single process, unlike full Sentry (16GB+ RAM)
• Error correlation: Request IDs allow correlation between frontend errors and backend logs

Negative

• Additional services: Bugsink and Logstash add complexity to the container
• PostgreSQL overhead: Additional database for error tracking
• Initial setup: Requires configuration of multiple components
• Logstash learning curve: Pipeline configuration requires Logstash knowledge

Alternatives Considered

1. Full Sentry self-hosted: Rejected due to complexity (Kafka, Redis, ClickHouse, 16GB+ RAM minimum)
2. GlitchTip: Considered, but Bugsink is lighter weight and easier to deploy
3. Sentry SaaS: Rejected due to self-hosted requirement
4. Custom error aggregation: Rejected in favor of proven Sentry SDK ecosystem

References

• Bugsink Documentation
• Bugsink Docker Install
• @sentry/node Documentation
• @sentry/react Documentation
• bugsink-mcp
• Logstash Reference
• ADR-030: PostgreSQL Function Observability
• ADR-032: Application Performance Monitoring