# ADR-054: Bugsink to Gitea Issue Synchronization

**Date**: 2026-01-17

**Status**: Proposed

## Context

The application uses Bugsink (Sentry-compatible self-hosted error tracking) to capture runtime errors across 6 projects:

| Project                           | Type           | Environment  |
| --------------------------------- | -------------- | ------------ |
| flyer-crawler-backend             | Backend        | Production   |
| flyer-crawler-backend-test        | Backend        | Test/Staging |
| flyer-crawler-frontend            | Frontend       | Production   |
| flyer-crawler-frontend-test       | Frontend       | Test/Staging |
| flyer-crawler-infrastructure      | Infrastructure | Production   |
| flyer-crawler-test-infrastructure | Infrastructure | Test/Staging |

Currently, errors remain in Bugsink until manually reviewed. There is no automated workflow to:

1. Create trackable tickets for errors
2. Assign errors to developers
3. Track resolution progress
4. Prevent errors from being forgotten

## Decision

Implement an automated background worker that synchronizes unresolved Bugsink issues to Gitea as trackable tickets. The sync worker will:

1. **Run only on the test/staging server** (not production, not dev container)
2. **Poll all 6 Bugsink projects** for unresolved issues
3. **Create Gitea issues** with full error context
4. **Mark synced issues as resolved** in Bugsink (to prevent re-polling)
5. **Track sync state in Redis** to ensure idempotency

### Why Test/Staging Only?

- The sync worker is a background service that needs API tokens for both Bugsink and Gitea
- Running on test/staging provides a single sync point without duplicating infrastructure
- All 6 Bugsink projects (including production) are synced from this one worker
- Production server stays focused on serving users, not running sync jobs

## Architecture

### Component Overview

```
┌─────────────────────────────────────────────────────────────────────┐
│                     TEST/STAGING SERVER                              │
│                                                                      │
│  ┌──────────────────┐    ┌──────────────────┐    ┌───────────────┐ │
│  │  BullMQ Queue    │───▶│  Sync Worker     │───▶│  Redis DB 15  │ │
│  │  bugsink-sync    │    │  (15min repeat)  │    │  Sync State   │ │
│  └──────────────────┘    └────────┬─────────┘    └───────────────┘ │
│                                   │                                  │
└───────────────────────────────────┼──────────────────────────────────┘
                                    │
                    ┌───────────────┴───────────────┐
                    ▼                               ▼
           ┌──────────────┐                ┌──────────────┐
           │   Bugsink    │                │    Gitea     │
           │ (6 projects) │                │  (1 repo)    │
           └──────────────┘                └──────────────┘
```

### Queue Configuration

| Setting         | Value                  | Rationale                                    |
| --------------- | ---------------------- | -------------------------------------------- |
| Queue Name      | `bugsink-sync`         | Follows existing naming pattern              |
| Repeat Interval | 15 minutes             | Balances responsiveness with API rate limits |
| Retry Attempts  | 3                      | Standard retry policy                        |
| Backoff         | Exponential (30s base) | Handles temporary API failures               |
| Concurrency     | 1                      | Serial processing prevents race conditions   |

### Redis Database Allocation

| Database | Usage               | Owner           |
| -------- | ------------------- | --------------- |
| 0        | BullMQ (Production) | Existing queues |
| 1        | BullMQ (Test)       | Existing queues |
| 2-14     | Reserved            | Future use      |
| 15       | Bugsink Sync State  | This feature    |

### Redis Key Schema

```
bugsink:synced:{bugsink_issue_id}
  └─ Value: JSON {
       gitea_issue_number: number,
       synced_at: ISO timestamp,
       project: string,
       title: string
     }
```

### Gitea Labels

The following labels have been created in `torbo/flyer-crawler.projectium.com`:

| Label                | ID  | Color              | Purpose                            |
| -------------------- | --- | ------------------ | ---------------------------------- |
| `bug:frontend`       | 8   | #e11d48 (Red)      | Frontend JavaScript/React errors   |
| `bug:backend`        | 9   | #ea580c (Orange)   | Backend Node.js/API errors         |
| `bug:infrastructure` | 10  | #7c3aed (Purple)   | Infrastructure errors (Redis, PM2) |
| `env:production`     | 11  | #dc2626 (Dark Red) | Production environment             |
| `env:test`           | 12  | #2563eb (Blue)     | Test/staging environment           |
| `env:development`    | 13  | #6b7280 (Gray)     | Development environment            |
| `source:bugsink`     | 14  | #10b981 (Green)    | Auto-synced from Bugsink           |

### Label Mapping

| Bugsink Project                   | Bug Label          | Env Label      |
| --------------------------------- | ------------------ | -------------- |
| flyer-crawler-backend             | bug:backend        | env:production |
| flyer-crawler-backend-test        | bug:backend        | env:test       |
| flyer-crawler-frontend            | bug:frontend       | env:production |
| flyer-crawler-frontend-test       | bug:frontend       | env:test       |
| flyer-crawler-infrastructure      | bug:infrastructure | env:production |
| flyer-crawler-test-infrastructure | bug:infrastructure | env:test       |

All synced issues also receive the `source:bugsink` label.

## Implementation Details

### New Files

| File                                   | Purpose                                     |
| -------------------------------------- | ------------------------------------------- |
| `src/services/bugsinkSync.server.ts`   | Core synchronization logic                  |
| `src/services/bugsinkClient.server.ts` | HTTP client for Bugsink API                 |
| `src/services/giteaClient.server.ts`   | HTTP client for Gitea API                   |
| `src/types/bugsink.ts`                 | TypeScript interfaces for Bugsink responses |
| `src/routes/admin/bugsink-sync.ts`     | Admin endpoints for manual trigger          |

### Modified Files

| File                                  | Changes                               |
| ------------------------------------- | ------------------------------------- |
| `src/services/queues.server.ts`       | Add `bugsinkSyncQueue` definition     |
| `src/services/workers.server.ts`      | Add sync worker implementation        |
| `src/config/env.ts`                   | Add bugsink sync configuration schema |
| `.env.example`                        | Document new environment variables    |
| `.gitea/workflows/deploy-to-test.yml` | Pass sync-related secrets             |

### Environment Variables

```bash
# Bugsink Configuration
BUGSINK_URL=https://bugsink.projectium.com
BUGSINK_API_TOKEN=77deaa5e...  # Created via Django management command (see BUGSINK-SYNC.md)

# Gitea Configuration
GITEA_URL=https://gitea.projectium.com
GITEA_API_TOKEN=...           # Personal access token with repo scope
GITEA_OWNER=torbo
GITEA_REPO=flyer-crawler.projectium.com

# Sync Control
BUGSINK_SYNC_ENABLED=false    # Set true only in test environment
BUGSINK_SYNC_INTERVAL=15      # Minutes between sync runs
```

### Gitea Issue Template

```markdown
## Error Details

| Field        | Value           |
| ------------ | --------------- |
| **Type**     | {error_type}    |
| **Message**  | {error_message} |
| **Platform** | {platform}      |
| **Level**    | {level}         |

## Occurrence Statistics

- **First Seen**: {first_seen}
- **Last Seen**: {last_seen}
- **Total Occurrences**: {count}

## Request Context

- **URL**: {request_url}
- **Additional Context**: {context}

## Stacktrace

<details>
<summary>Click to expand</summary>

{stacktrace}

</details>

---

**Bugsink Issue**: {bugsink_url}
**Project**: {project_slug}
**Trace ID**: {trace_id}
```

### Sync Workflow

```
1. Worker triggered (every 15 min or manual)
2. For each of 6 Bugsink projects:
   a. List issues with status='unresolved'
   b. For each issue:
      i.   Check Redis for existing sync record
      ii.  If already synced → skip
      iii. Fetch issue details + stacktrace
      iv.  Create Gitea issue with labels
      v.   Store sync record in Redis
      vi.  Mark issue as 'resolved' in Bugsink
3. Log summary (synced: N, skipped: N, failed: N)
```

### Idempotency Guarantees

1. **Redis check before creation**: Prevents duplicate Gitea issues
2. **Atomic Redis write after Gitea create**: Ensures state consistency
3. **Query only unresolved issues**: Resolved issues won't appear in polls
4. **No TTL on Redis keys**: Permanent sync history

## Consequences

### Positive

1. **Visibility**: All application errors become trackable tickets
2. **Accountability**: Errors can be assigned to developers
3. **History**: Complete audit trail of when errors were discovered and resolved
4. **Integration**: Errors appear alongside feature work in Gitea
5. **Automation**: No manual error triage required

### Negative

1. **API Dependencies**: Requires both Bugsink and Gitea APIs to be available
2. **Token Management**: Additional secrets to manage in CI/CD
3. **Potential Noise**: High-frequency errors could create many tickets (mitigated by Bugsink's issue grouping)
4. **Single Point**: Sync only runs on test server (if test server is down, no sync occurs)

### Risks & Mitigations

| Risk                    | Mitigation                                        |
| ----------------------- | ------------------------------------------------- |
| Bugsink API rate limits | 15-minute polling interval                        |
| Gitea API rate limits   | Sequential processing with delays                 |
| Redis connection issues | Reuse existing connection patterns                |
| Duplicate issues        | Redis tracking + idempotent checks                |
| Missing stacktrace      | Graceful degradation (create issue without trace) |

## Admin Interface

### Manual Sync Endpoint

```
POST /api/admin/bugsink/sync
Authorization: Bearer {admin_jwt}

Response:
{
  "success": true,
  "data": {
    "synced": 3,
    "skipped": 12,
    "failed": 0,
    "duration_ms": 2340
  }
}
```

### Sync Status Endpoint

```
GET /api/admin/bugsink/sync/status
Authorization: Bearer {admin_jwt}

Response:
{
  "success": true,
  "data": {
    "enabled": true,
    "last_run": "2026-01-17T10:30:00Z",
    "next_run": "2026-01-17T10:45:00Z",
    "total_synced": 47,
    "projects": [
      { "slug": "flyer-crawler-backend", "synced_count": 12 },
      ...
    ]
  }
}
```

## Implementation Phases

### Phase 1: Core Infrastructure

- Add environment variables to `env.ts` schema
- Create `BugsinkClient` service (HTTP client)
- Create `GiteaClient` service (HTTP client)
- Add Redis db 15 connection for sync tracking

### Phase 2: Sync Logic

- Create `BugsinkSyncService` with sync logic
- Add `bugsink-sync` queue to `queues.server.ts`
- Add sync worker to `workers.server.ts`
- Create TypeScript types for API responses

### Phase 3: Integration

- Add admin endpoints for manual sync trigger
- Update `deploy-to-test.yml` with new secrets
- Add secrets to Gitea repository settings
- Test end-to-end in staging environment

### Phase 4: Documentation

- Update CLAUDE.md with sync information
- Create operational runbook for sync issues

## Future Enhancements

1. **Bi-directional sync**: Update Bugsink when Gitea issue is closed
2. **Smart deduplication**: Detect similar errors across projects
3. **Priority mapping**: High occurrence count → high priority label
4. **Slack/Discord notifications**: Alert on new critical errors
5. **Metrics dashboard**: Track error trends over time

## References

- [ADR-006: Background Job Processing](./0006-background-job-processing-and-task-queues.md)
- [ADR-015: Error Tracking and Observability](./0015-error-tracking-and-observability.md)
- [Bugsink API Documentation](https://bugsink.com/docs/api/)
- [Gitea API Documentation](https://docs.gitea.io/en-us/api-usage/)