flyer-crawler.projectium.com/docs/tools/BUGSINK-SETUP.md

Bugsink Error Tracking Setup and Usage Guide

This document covers the complete setup and usage of Bugsink for error tracking in the Flyer Crawler application.

Table of Contents

• What is Bugsink
• Environments
• Token Creation
• MCP Integration
• Application Integration
• Logstash Integration
• Using Bugsink
• Common Workflows
• Troubleshooting

---

What is Bugsink

Bugsink is a lightweight, self-hosted error tracking platform that is fully compatible with the Sentry SDK ecosystem. We use Bugsink instead of Sentry SaaS or self-hosted Sentry for several reasons:

Aspect	Bugsink	Self-Hosted Sentry
Resource Usage	Single process, ~256MB RAM	16GB+ RAM, Kafka, ClickHouse
Deployment	Simple pip/binary install	Docker Compose with 20+ services
SDK Compatibility	Full Sentry SDK support	Full Sentry SDK support
Database	PostgreSQL or SQLite	PostgreSQL + ClickHouse
Cost	Free, self-hosted	Free, self-hosted
Maintenance	Minimal	Significant

Key Benefits:

1. Sentry SDK Compatibility: Uses the same @sentry/node and @sentry/react SDKs as Sentry
2. Self-Hosted: All error data stays on our infrastructure
3. Lightweight: Runs alongside the application without significant overhead
4. MCP Integration: AI tools (Claude Code) can query errors via the bugsink-mcp server

Architecture Decision: See ADR-015: Application Performance Monitoring and Error Tracking for the full rationale.

---

Environments

Dev Container (Local Development)

Item	Value
Web UI	https://localhost:8443 (nginx proxy)
Internal URL	http://localhost:8000 (direct)
Credentials	admin@localhost / admin
Backend Project	Project ID 1 - flyer-crawler-dev-backend
Frontend Project	Project ID 2 - flyer-crawler-dev-frontend
Backend DSN	http://<key>@localhost:8000/1
Frontend DSN	http://<key>@localhost:8000/2
Database	postgresql://bugsink:bugsink_dev_password@postgres:5432/bugsink

Configuration Files:

File	Purpose
compose.dev.yml	Initial DSNs using 127.0.0.1:8000 (container startup)
.env.local	OVERRIDES compose.dev.yml with localhost:8000 (app runtime)

Note: .env.local takes precedence over compose.dev.yml environment variables.

Production

Item	Value
Web UI	https://bugsink.projectium.com
Credentials	Managed separately (not shared in docs)
Backend Project	flyer-crawler-backend
Frontend Project	flyer-crawler-frontend
Infra Project	flyer-crawler-infrastructure

Bugsink Projects:

Project Slug	Type	Environment
flyer-crawler-backend	Backend	Production
flyer-crawler-backend-test	Backend	Test
flyer-crawler-frontend	Frontend	Production
flyer-crawler-frontend-test	Frontend	Test
flyer-crawler-infrastructure	Infra	Production
flyer-crawler-test-infrastructure	Infra	Test

---

Token Creation

Bugsink 2.0.11 does NOT have a "Settings > API Keys" menu in the UI. API tokens must be created via Django management command.

Dev Container Token

Run this command from the Windows host (Git Bash or PowerShell):

MSYS_NO_PATHCONV=1 podman exec -e DATABASE_URL=postgresql://bugsink:bugsink_dev_password@postgres:5432/bugsink -e SECRET_KEY=dev-bugsink-secret-key-minimum-50-characters-for-security flyer-crawler-dev sh -c 'cd /opt/bugsink/conf && DJANGO_SETTINGS_MODULE=bugsink_conf PYTHONPATH=/opt/bugsink/conf:/opt/bugsink/lib/python3.10/site-packages /opt/bugsink/bin/python -m django create_auth_token'

Output: A 40-character lowercase hex token (e.g., a609c2886daa4e1e05f1517074d7779a5fb49056)

Production Token

SSH into the production server:

ssh root@projectium.com "cd /opt/bugsink && bugsink-manage create_auth_token"

Output: Same format - 40-character hex token.

Token Storage

Environment	Storage Location	Notes
Dev	.mcp.json (project-level)	Not committed to git
Production	Gitea secrets + settings.json	BUGSINK_TOKEN secret

---

MCP Integration

The bugsink-mcp server allows Claude Code and other AI tools to query Bugsink for error information.

Installation

# Clone the MCP server
cd d:\gitea
git clone https://github.com/j-shelfwood/bugsink-mcp.git
cd bugsink-mcp
npm install
npm run build

Configuration

IMPORTANT: Localhost MCP servers must use project-level .mcp.json due to a known Claude Code loader issue. See BUGSINK-MCP-TROUBLESHOOTING.md for details.

Production (Global settings.json)

Location: ~/.claude/settings.json (or C:\Users\<username>\.claude\settings.json)

{
  "mcpServers": {
    "bugsink": {
      "command": "node",
      "args": ["d:\\gitea\\bugsink-mcp\\dist\\index.js"],
      "env": {
        "BUGSINK_URL": "https://bugsink.projectium.com",
        "BUGSINK_TOKEN": "<40-char-hex-token>"
      }
    }
  }
}

Dev Container (Project-level .mcp.json)

Location: Project root .mcp.json

{
  "mcpServers": {
    "localerrors": {
      "command": "node",
      "args": ["d:\\gitea\\bugsink-mcp\\dist\\index.js"],
      "env": {
        "BUGSINK_URL": "http://127.0.0.1:8000",
        "BUGSINK_TOKEN": "<40-char-hex-token>"
      }
    }
  }
}

Environment Variables

The bugsink-mcp package requires exactly TWO environment variables:

Variable	Description	Required
BUGSINK_URL	Bugsink instance URL	Yes
BUGSINK_TOKEN	API token (40-char hex)	Yes

Common Mistakes:

• Using BUGSINK_API_TOKEN (wrong - use BUGSINK_TOKEN)
• Including BUGSINK_ORG_SLUG (not used by the package)

Available MCP Tools

Tool	Purpose
test_connection	Verify MCP server can reach Bugsink
list_projects	List all projects in the instance
get_project	Get project details including DSN
list_issues	List issues for a project
get_issue	Get detailed issue information
list_events	List individual error occurrences
get_event	Get full event details with context
get_stacktrace	Get pre-rendered Markdown stacktrace
list_releases	List releases for a project
create_release	Create a new release

Tool Prefixes:

• Production: mcp__bugsink__*
• Dev Container: mcp__localerrors__*

Verifying MCP Connection

After configuration, restart Claude Code and test:

// Production
mcp__bugsink__test_connection();
// Expected: "Connection successful: Connected successfully. Found N project(s)."

// Dev Container
mcp__localerrors__test_connection();
// Expected: "Connection successful: Connected successfully. Found N project(s)."

---

Application Integration

Backend (Express/Node.js)

File: src/services/sentry.server.ts

The backend uses @sentry/node SDK v8+ to capture errors:

import * as Sentry from '@sentry/node';
import { config, isSentryConfigured, isProduction, isTest } from '../config/env';

export function initSentry(): void {
  if (!isSentryConfigured || isTest) return;

  Sentry.init({
    dsn: config.sentry.dsn,
    environment: config.sentry.environment || config.server.nodeEnv,
    debug: config.sentry.debug,
    tracesSampleRate: 0, // Performance monitoring disabled
    beforeSend(event, hint) {
      // Custom filtering logic
      return event;
    },
  });
}

Key Functions:

Function	Purpose
initSentry()	Initialize SDK at application startup
captureException()	Manually capture caught errors
captureMessage()	Log non-exception events
setUser()	Set user context after authentication
addBreadcrumb()	Add navigation/action breadcrumbs
getSentryMiddleware()	Get Express middleware for automatic capture

Integration in server.ts:

// At the very top of server.ts, before other imports
import { initSentry, getSentryMiddleware } from './services/sentry.server';
initSentry();

// After Express app creation
const { requestHandler, errorHandler } = getSentryMiddleware();
app.use(requestHandler);

// ... routes ...

// Before final error handler
app.use(errorHandler);

Frontend (React)

File: src/services/sentry.client.ts

The frontend uses @sentry/react SDK:

import * as Sentry from '@sentry/react';
import config from '../config';

export function initSentry(): void {
  if (!config.sentry.dsn || !config.sentry.enabled) return;

  Sentry.init({
    dsn: config.sentry.dsn,
    environment: config.sentry.environment,
    debug: config.sentry.debug,
    tracesSampleRate: 0,
    integrations: [
      Sentry.breadcrumbsIntegration({
        console: true,
        dom: true,
        fetch: true,
        history: true,
        xhr: true,
      }),
    ],
    beforeSend(event) {
      // Filter browser extension errors
      if (
        event.exception?.values?.[0]?.stacktrace?.frames?.some((frame) =>
          frame.filename?.includes('extension://'),
        )
      ) {
        return null;
      }
      return event;
    },
  });
}

Client Configuration (src/config.ts):

const config = {
  sentry: {
    dsn: import.meta.env.VITE_SENTRY_DSN,
    environment: import.meta.env.VITE_SENTRY_ENVIRONMENT || import.meta.env.MODE,
    debug: import.meta.env.VITE_SENTRY_DEBUG === 'true',
    enabled: import.meta.env.VITE_SENTRY_ENABLED !== 'false',
  },
};

Environment Variables

Backend (src/config/env.ts):

Variable	Description	Default
SENTRY_DSN	Sentry-compatible DSN	(optional)
SENTRY_ENABLED	Enable/disable error reporting	true
SENTRY_ENVIRONMENT	Environment tag	NODE_ENV
SENTRY_DEBUG	Enable SDK debug logging	false

Frontend (Vite):

Variable	Description
VITE_SENTRY_DSN	Frontend DSN (separate project)
VITE_SENTRY_ENVIRONMENT	Environment tag
VITE_SENTRY_DEBUG	Enable SDK debug logging
VITE_SENTRY_ENABLED	Enable/disable error reporting

---

Logstash Integration

Logstash aggregates logs from multiple sources and forwards error patterns to Bugsink.

Note: See ADR-015 for the full architecture.

Log Sources

Source	Log Path	Error Detection
Pino (app)	/app/logs/*.log	level >= 50 (error/fatal)
Redis	/var/log/redis/*.log	WARNING/ERROR log levels
PostgreSQL	(future)	ERROR/FATAL log levels

Pipeline Configuration

Location: /etc/logstash/conf.d/bugsink.conf

# === INPUTS ===
input {
  file {
    path => "/app/logs/*.log"
    codec => json
    type => "pino"
    tags => ["app"]
  }

  file {
    path => "/var/log/redis/*.log"
    type => "redis"
    tags => ["redis"]
  }
}

# === FILTERS ===
filter {
  if [type] == "pino" and [level] >= 50 {
    mutate { add_tag => ["error"] }
  }

  if [type] == "redis" {
    grok {
      match => { "message" => "%{POSINT:pid}:%{WORD:role} %{MONTHDAY} %{MONTH} %{TIME} %{WORD:loglevel} %{GREEDYDATA:redis_message}" }
    }
    if [loglevel] in ["WARNING", "ERROR"] {
      mutate { add_tag => ["error"] }
    }
  }
}

# === OUTPUT ===
output {
  if "error" in [tags] {
    http {
      url => "http://localhost:8000/api/store/"
      http_method => "post"
      format => "json"
    }
  }
}

Benefits

1. Secondary Capture Path: Catches errors before SDK initialization
2. Log-Based Errors: Captures errors that don't throw exceptions
3. Infrastructure Monitoring: Redis connection issues, slow commands
4. Historical Analysis: Process existing log files

---

Using Bugsink

Accessing the Web UI

Dev Container:

1. Open https://localhost:8443 in your browser
2. Accept the self-signed certificate warning
3. Login with admin@localhost / admin

Production:

1. Open https://bugsink.projectium.com
2. Login with your credentials

Projects and Teams

Bugsink organizes errors into projects:

Concept	Description
Team	Group of projects (e.g., "Flyer Crawler")
Project	Single application/service
DSN	Data Source Name - unique key for each project

To view projects:

1. Click the project dropdown in the top navigation
2. Or use MCP: mcp__bugsink__list_projects()

Viewing Issues

Issues represent grouped error occurrences. Multiple identical errors are deduplicated into a single issue.

Issue List View:

• Navigate to a project
• Issues are sorted by last occurrence
• Each issue shows: title, count, first/last seen

Issue Detail View:

• Click an issue to see full details
• View aggregated statistics
• See list of individual events
• Access full stacktrace

Viewing Events

Events are individual error occurrences.

Event Information:

• Full stacktrace
• Request context (URL, method, headers)
• User context (if set)
• Breadcrumbs (actions leading to error)
• Tags and extra data

Via MCP:

// List events for an issue
mcp__bugsink__list_events({ issue_id: 'uuid-here' });

// Get full event details
mcp__bugsink__get_event({ event_id: 'uuid-here' });

// Get readable stacktrace
mcp__bugsink__get_stacktrace({ event_id: 'uuid-here' });

Stacktraces and Context

Stacktraces show the call stack at the time of error:

Via Web UI:

• Open an event
• Expand the "Exception" section
• Click frames to see source code context

Via MCP:

• get_stacktrace returns pre-rendered Markdown
• Includes file paths, line numbers, function names

Filtering and Searching

Web UI Filters:

• By status: unresolved, resolved, muted
• By date range
• By release version
• By environment

MCP Filtering:

// Filter by status
mcp__bugsink__list_issues({
  project_id: 1,
  status: 'unresolved',
  limit: 25,
});

// Sort options
mcp__bugsink__list_issues({
  project_id: 1,
  sort: 'last_seen', // or "digest_order"
  order: 'desc', // or "asc"
});

Release Tracking

Releases help identify which version introduced or fixed issues.

Creating Releases:

mcp__bugsink__create_release({
  project_id: 1,
  version: '1.2.3',
});

Viewing Releases:

mcp__bugsink__list_releases({ project_id: 1 });

---

Common Workflows

Investigating Production Errors

1. Check for new errors (via MCP):

   mcp__bugsink__list_issues({
     project_id: 1,
     status: 'unresolved',
     sort: 'last_seen',
     limit: 10,
   });
   

2. Get issue details:

   mcp__bugsink__get_issue({ issue_id: 'uuid' });
   

3. View stacktrace:

   mcp__bugsink__list_events({ issue_id: 'uuid', limit: 1 });
   mcp__bugsink__get_stacktrace({ event_id: 'event-uuid' });
   

4. Examine the code: Use the file path and line numbers from the stacktrace to locate the issue in the codebase.

Tracking Down Bugs

1. Identify error patterns:

   • Group similar errors by message or location
   • Check occurrence counts and frequency

2. Examine request context:

   mcp__bugsink__get_event({ event_id: 'uuid' });
   

   Look for: URL, HTTP method, request body, user info

3. Review breadcrumbs: Understand the sequence of actions leading to the error.

4. Correlate with logs: Use the request ID from the event to search application logs.

Monitoring Error Rates

1. Check issue counts: Compare event counts over time
2. Watch for regressions: Resolved issues that reopen
3. Track new issues: Filter by "first seen" date

Dev Container Debugging

1. Access local Bugsink: https://localhost:8443

2. Trigger a test error:

   curl -X POST http://localhost:3001/api/test/error
   

3. View in Bugsink: Check the dev project for the captured error

4. Query via MCP:

   mcp__localerrors__list_issues({ project_id: 1 });
   

---

Troubleshooting

MCP Server Not Available

Symptoms:

• mcp__localerrors__* tools return "No such tool available"
• mcp__bugsink__* works but mcp__localerrors__* fails

Solutions:

1. Check configuration location: Localhost servers must use project-level .mcp.json, not global settings.json

2. Verify token variable name: Use BUGSINK_TOKEN, not BUGSINK_API_TOKEN

3. Test manually:

   cd d:\gitea\bugsink-mcp
   set BUGSINK_URL=http://localhost:8000
   set BUGSINK_TOKEN=<your-token>
   node dist/index.js
   

   Expected: Bugsink MCP server started

4. Full restart: Close VS Code completely, restart

See BUGSINK-MCP-TROUBLESHOOTING.md for detailed troubleshooting.

Connection Refused to localhost:8000

Cause: Dev container Bugsink service not running

Solutions:

1. Check container status:

   podman exec flyer-crawler-dev systemctl status bugsink
   

2. Start the service:

   podman exec flyer-crawler-dev systemctl start bugsink
   

3. Check logs:

   podman exec flyer-crawler-dev journalctl -u bugsink -n 50
   

Errors Not Appearing in Bugsink

Backend:

1. Check DSN: Verify SENTRY_DSN environment variable is set
2. Check enabled flag: SENTRY_ENABLED should be true
3. Check test environment: Sentry is disabled in NODE_ENV=test

Frontend:

1. Check Vite env: VITE_SENTRY_DSN must be set
2. Verify initialization: Check browser console for Sentry init message
3. Check filtering: beforeSend may be filtering the error

HTTPS Certificate Warnings

Dev Container: Self-signed certificates are expected. Accept the warning.

Production: Should use valid certificates. If warnings appear, check certificate expiration.

Token Invalid or Expired

Symptoms: MCP returns authentication errors

Solutions:

1. Regenerate token: Use Django management command (see Token Creation)
2. Update configuration: Put new token in .mcp.json or settings.json
3. Restart Claude Code: Required after config changes

Bugsink Database Issues

Symptoms: 500 errors in Bugsink UI, connection refused

Dev Container:

# Check PostgreSQL
podman exec flyer-crawler-dev pg_isready -U bugsink -d bugsink -h postgres

# Check database exists
podman exec flyer-crawler-dev psql -U postgres -h postgres -c "\l" | grep bugsink

Production:

ssh root@projectium.com "cd /opt/bugsink && bugsink-manage check"

---

Related Documentation

• ADR-015: Application Performance Monitoring and Error Tracking
• BUGSINK-MCP-TROUBLESHOOTING.md
• DEV-CONTAINER-BUGSINK.md
• BUGSINK-SYNC.md - Bugsink to Gitea issue synchronization
• bugsink-mcp Repository
• Bugsink Documentation
• @sentry/node Documentation
• @sentry/react Documentation