flyer-crawler.projectium.com/docs/subagents/INTEGRATIONS-GUIDE.md

Integrations Subagent Guide

The integrations-specialist subagent handles third-party services, webhooks, and external API integrations in the Flyer Crawler project.

Quick Reference

Aspect	Details
Primary Use	External APIs, webhooks, OAuth, third-party services
Key Files	src/services/external/, src/routes/webhooks.routes.ts
Key ADRs	ADR-041 (AI Integration), ADR-016 (API Security), ADR-048 (Auth)
MCP Tools	mcp__gitea-projectium__*, mcp__bugsink__*
Security	API key storage, webhook signatures, OAuth state param
Delegate To	coder (implementation), security-engineer (review), ai-usage (Gemini)

When to Use

Use the integrations-specialist subagent when you need to:

• Integrate with external APIs (OAuth, REST, GraphQL)
• Implement webhook handlers
• Configure third-party services
• Debug external service connectivity
• Handle API authentication flows
• Manage external service rate limits

What integrations-specialist Knows

The integrations-specialist subagent understands:

• OAuth 2.0 flows (authorization code, client credentials)
• REST API integration patterns
• Webhook security (signature verification)
• External service error handling
• Rate limiting and retry strategies
• API key management

Current Integrations

Service	Purpose	Integration Type	Key Files
Google Gemini	AI flyer extraction	REST API	src/services/aiService.server.ts
Bugsink	Error tracking	REST API	MCP: mcp__bugsink__*
Gitea	Repository and CI/CD	REST API	MCP: mcp__gitea-projectium__*
Redis	Caching and job queues	Native client	src/services/redis.server.ts
PostgreSQL	Primary database	Native client	src/services/db/pool.db.ts

Example Requests

Adding External API Integration

"Use integrations-specialist to integrate with the Store API
to automatically fetch store location data. Include proper
error handling, rate limiting, and caching."

OAuth Implementation

"Use integrations-specialist to implement Google OAuth for
user authentication. Include token refresh handling and
session management."

Webhook Handler

"Use integrations-specialist to create a webhook handler for
receiving store inventory updates. Include signature verification
and idempotency handling."

Debugging External Service Issues

"Use integrations-specialist to debug why the Gemini API calls
are intermittently failing with timeout errors. Check connection
pooling, retry logic, and error handling."

Integration Patterns

REST API Client Pattern

// src/services/external/storeApi.server.ts
import { env } from '@/config/env';
import { log } from '@/services/logger.server';

interface StoreApiConfig {
  baseUrl: string;
  apiKey: string;
  timeout: number;
}

class StoreApiClient {
  private config: StoreApiConfig;

  constructor(config: StoreApiConfig) {
    this.config = config;
  }

  async getStoreLocations(storeId: string): Promise<StoreLocation[]> {
    const url = `${this.config.baseUrl}/stores/${storeId}/locations`;

    try {
      const response = await fetch(url, {
        headers: {
          Authorization: `Bearer ${this.config.apiKey}`,
          'Content-Type': 'application/json',
        },
        signal: AbortSignal.timeout(this.config.timeout),
      });

      if (!response.ok) {
        throw new ExternalApiError(`Store API error: ${response.status}`, response.status);
      }

      return response.json();
    } catch (error) {
      log.error({ error, storeId }, 'Failed to fetch store locations');
      throw error;
    }
  }
}

export const storeApiClient = new StoreApiClient({
  baseUrl: env.STORE_API_BASE_URL,
  apiKey: env.STORE_API_KEY,
  timeout: 10000,
});

Webhook Handler Pattern

// src/routes/webhooks.routes.ts
import { Router } from 'express';
import crypto from 'crypto';
import { env } from '@/config/env';

const router = Router();

function verifyWebhookSignature(payload: string, signature: string, secret: string): boolean {
  const expected = crypto.createHmac('sha256', secret).update(payload).digest('hex');
  return crypto.timingSafeEqual(Buffer.from(signature), Buffer.from(`sha256=${expected}`));
}

router.post('/store-updates', async (req, res, next) => {
  try {
    const signature = req.headers['x-webhook-signature'] as string;
    const payload = JSON.stringify(req.body);

    if (!verifyWebhookSignature(payload, signature, env.WEBHOOK_SECRET)) {
      return res.status(401).json({ error: 'Invalid signature' });
    }

    // Process webhook with idempotency check
    const eventId = req.headers['x-event-id'] as string;
    const alreadyProcessed = await checkIdempotencyKey(eventId);

    if (alreadyProcessed) {
      return res.status(200).json({ status: 'already_processed' });
    }

    await processStoreUpdate(req.body);
    await markEventProcessed(eventId);

    res.status(200).json({ status: 'processed' });
  } catch (error) {
    next(error);
  }
});

OAuth Flow Pattern

// src/services/oauth/googleOAuth.server.ts
import { OAuth2Client } from 'google-auth-library';
import { env } from '@/config/env';

const oauth2Client = new OAuth2Client(
  env.GOOGLE_CLIENT_ID,
  env.GOOGLE_CLIENT_SECRET,
  env.GOOGLE_REDIRECT_URI,
);

export function getAuthorizationUrl(): string {
  return oauth2Client.generateAuthUrl({
    access_type: 'offline',
    scope: ['email', 'profile'],
    prompt: 'consent',
  });
}

export async function exchangeCodeForTokens(code: string) {
  const { tokens } = await oauth2Client.getToken(code);
  return tokens;
}

export async function refreshAccessToken(refreshToken: string) {
  oauth2Client.setCredentials({ refresh_token: refreshToken });
  const { credentials } = await oauth2Client.refreshAccessToken();
  return credentials;
}

Error Handling for External Services

Custom Error Classes

// src/services/external/errors.ts
export class ExternalApiError extends Error {
  constructor(
    message: string,
    public statusCode: number,
    public retryable: boolean = false,
  ) {
    super(message);
    this.name = 'ExternalApiError';
  }
}

export class RateLimitError extends ExternalApiError {
  constructor(
    message: string,
    public retryAfter: number,
  ) {
    super(message, 429, true);
    this.name = 'RateLimitError';
  }
}

Retry with Exponential Backoff

async function fetchWithRetry<T>(
  fn: () => Promise<T>,
  options: { maxRetries: number; baseDelay: number },
): Promise<T> {
  let lastError: Error;

  for (let attempt = 0; attempt <= options.maxRetries; attempt++) {
    try {
      return await fn();
    } catch (error) {
      lastError = error as Error;

      if (error instanceof ExternalApiError && !error.retryable) {
        throw error;
      }

      if (attempt < options.maxRetries) {
        const delay = options.baseDelay * Math.pow(2, attempt);
        await new Promise((resolve) => setTimeout(resolve, delay));
      }
    }
  }

  throw lastError!;
}

Rate Limiting Strategies

Token Bucket Pattern

class RateLimiter {
  private tokens: number;
  private lastRefill: number;
  private readonly maxTokens: number;
  private readonly refillRate: number; // tokens per second

  constructor(maxTokens: number, refillRate: number) {
    this.maxTokens = maxTokens;
    this.tokens = maxTokens;
    this.refillRate = refillRate;
    this.lastRefill = Date.now();
  }

  async acquire(): Promise<void> {
    this.refill();

    if (this.tokens < 1) {
      const waitTime = ((1 - this.tokens) / this.refillRate) * 1000;
      await new Promise((resolve) => setTimeout(resolve, waitTime));
      this.refill();
    }

    this.tokens -= 1;
  }

  private refill(): void {
    const now = Date.now();
    const elapsed = (now - this.lastRefill) / 1000;
    this.tokens = Math.min(this.maxTokens, this.tokens + elapsed * this.refillRate);
    this.lastRefill = now;
  }
}

Testing Integrations

Mocking External Services

// src/tests/mocks/storeApi.mock.ts
import { vi } from 'vitest';

export const mockStoreApiClient = {
  getStoreLocations: vi.fn(),
};

vi.mock('@/services/external/storeApi.server', () => ({
  storeApiClient: mockStoreApiClient,
}));

Integration Test with Real Service

// src/tests/integration/storeApi.integration.test.ts
describe('Store API Integration', () => {
  it.skipIf(!env.STORE_API_KEY)('fetches real store locations', async () => {
    const locations = await storeApiClient.getStoreLocations('test-store');
    expect(locations).toBeInstanceOf(Array);
  });
});

MCP Tools for Integrations

Gitea Integration

// List repositories
mcp__gitea-projectium__list_my_repos()

// Create issue
mcp__gitea-projectium__create_issue({
  owner: "projectium",
  repo: "flyer-crawler",
  title: "Issue title",
  body: "Issue description"
})

Bugsink Integration

// List projects
mcp__bugsink__list_projects()

// Get issue details
mcp__bugsink__get_issue({ issue_id: "..." })

// Get stacktrace
mcp__bugsink__get_stacktrace({ event_id: "..." })

Security Considerations

API Key Storage

• Never commit API keys to version control
• Use environment variables via src/config/env.ts
• Rotate keys periodically
• Use separate keys for dev/test/prod

Webhook Security

• Always verify webhook signatures
• Use HTTPS for webhook endpoints
• Implement idempotency
• Log webhook events for audit

OAuth Security

• Use state parameter to prevent CSRF
• Store tokens securely (encrypted at rest)
• Implement token refresh before expiration
• Validate token scopes

Related Documentation

• OVERVIEW.md - Subagent system overview
• SECURITY-DEBUG-GUIDE.md - Security patterns
• AI-USAGE-GUIDE.md - Gemini API integration
• ../adr/0041-ai-gemini-integration-architecture.md - AI integration ADR
• ../adr/0016-api-security-hardening.md - API security
• ../adr/0048-authentication-strategy.md - Authentication