flyer-crawler.projectium.com/docs/subagents/INTEGRATIONS-GUIDE.md

# Integrations Subagent Guide

The **integrations-specialist** subagent handles third-party services, webhooks, and external API integrations in the Flyer Crawler project.

## Quick Reference

| Aspect          | Details                                                                     |
| --------------- | --------------------------------------------------------------------------- |
| **Primary Use** | External APIs, webhooks, OAuth, third-party services                        |
| **Key Files**   | `src/services/external/`, `src/routes/webhooks.routes.ts`                   |
| **Key ADRs**    | ADR-041 (AI Integration), ADR-016 (API Security), ADR-048 (Auth)            |
| **MCP Tools**   | `mcp__gitea-projectium__*`, `mcp__bugsink__*`                               |
| **Security**    | API key storage, webhook signatures, OAuth state param                      |
| **Delegate To** | `coder` (implementation), `security-engineer` (review), `ai-usage` (Gemini) |

## When to Use

Use the **integrations-specialist** subagent when you need to:

- Integrate with external APIs (OAuth, REST, GraphQL)
- Implement webhook handlers
- Configure third-party services
- Debug external service connectivity
- Handle API authentication flows
- Manage external service rate limits

## What integrations-specialist Knows

The integrations-specialist subagent understands:

- OAuth 2.0 flows (authorization code, client credentials)
- REST API integration patterns
- Webhook security (signature verification)
- External service error handling
- Rate limiting and retry strategies
- API key management

## Current Integrations

| Service       | Purpose                | Integration Type | Key Files                          |
| ------------- | ---------------------- | ---------------- | ---------------------------------- |
| Google Gemini | AI flyer extraction    | REST API         | `src/services/aiService.server.ts` |
| Bugsink       | Error tracking         | REST API         | MCP: `mcp__bugsink__*`             |
| Gitea         | Repository and CI/CD   | REST API         | MCP: `mcp__gitea-projectium__*`    |
| Redis         | Caching and job queues | Native client    | `src/services/redis.server.ts`     |
| PostgreSQL    | Primary database       | Native client    | `src/services/db/pool.db.ts`       |

## Example Requests

### Adding External API Integration

```
"Use integrations-specialist to integrate with the Store API
to automatically fetch store location data. Include proper
error handling, rate limiting, and caching."
```

### OAuth Implementation

```
"Use integrations-specialist to implement Google OAuth for
user authentication. Include token refresh handling and
session management."
```

### Webhook Handler

```
"Use integrations-specialist to create a webhook handler for
receiving store inventory updates. Include signature verification
and idempotency handling."
```

### Debugging External Service Issues

```
"Use integrations-specialist to debug why the Gemini API calls
are intermittently failing with timeout errors. Check connection
pooling, retry logic, and error handling."
```

## Integration Patterns

### REST API Client Pattern

```typescript
// src/services/external/storeApi.server.ts
import { env } from '@/config/env';
import { log } from '@/services/logger.server';

interface StoreApiConfig {
  baseUrl: string;
  apiKey: string;
  timeout: number;
}

class StoreApiClient {
  private config: StoreApiConfig;

  constructor(config: StoreApiConfig) {
    this.config = config;
  }

  async getStoreLocations(storeId: string): Promise<StoreLocation[]> {
    const url = `${this.config.baseUrl}/stores/${storeId}/locations`;

    try {
      const response = await fetch(url, {
        headers: {
          Authorization: `Bearer ${this.config.apiKey}`,
          'Content-Type': 'application/json',
        },
        signal: AbortSignal.timeout(this.config.timeout),
      });

      if (!response.ok) {
        throw new ExternalApiError(`Store API error: ${response.status}`, response.status);
      }

      return response.json();
    } catch (error) {
      log.error({ error, storeId }, 'Failed to fetch store locations');
      throw error;
    }
  }
}

export const storeApiClient = new StoreApiClient({
  baseUrl: env.STORE_API_BASE_URL,
  apiKey: env.STORE_API_KEY,
  timeout: 10000,
});
```

### Webhook Handler Pattern

```typescript
// src/routes/webhooks.routes.ts
import { Router } from 'express';
import crypto from 'crypto';
import { env } from '@/config/env';

const router = Router();

function verifyWebhookSignature(payload: string, signature: string, secret: string): boolean {
  const expected = crypto.createHmac('sha256', secret).update(payload).digest('hex');
  return crypto.timingSafeEqual(Buffer.from(signature), Buffer.from(`sha256=${expected}`));
}

router.post('/store-updates', async (req, res, next) => {
  try {
    const signature = req.headers['x-webhook-signature'] as string;
    const payload = JSON.stringify(req.body);

    if (!verifyWebhookSignature(payload, signature, env.WEBHOOK_SECRET)) {
      return res.status(401).json({ error: 'Invalid signature' });
    }

    // Process webhook with idempotency check
    const eventId = req.headers['x-event-id'] as string;
    const alreadyProcessed = await checkIdempotencyKey(eventId);

    if (alreadyProcessed) {
      return res.status(200).json({ status: 'already_processed' });
    }

    await processStoreUpdate(req.body);
    await markEventProcessed(eventId);

    res.status(200).json({ status: 'processed' });
  } catch (error) {
    next(error);
  }
});
```

### OAuth Flow Pattern

```typescript
// src/services/oauth/googleOAuth.server.ts
import { OAuth2Client } from 'google-auth-library';
import { env } from '@/config/env';

const oauth2Client = new OAuth2Client(
  env.GOOGLE_CLIENT_ID,
  env.GOOGLE_CLIENT_SECRET,
  env.GOOGLE_REDIRECT_URI,
);

export function getAuthorizationUrl(): string {
  return oauth2Client.generateAuthUrl({
    access_type: 'offline',
    scope: ['email', 'profile'],
    prompt: 'consent',
  });
}

export async function exchangeCodeForTokens(code: string) {
  const { tokens } = await oauth2Client.getToken(code);
  return tokens;
}

export async function refreshAccessToken(refreshToken: string) {
  oauth2Client.setCredentials({ refresh_token: refreshToken });
  const { credentials } = await oauth2Client.refreshAccessToken();
  return credentials;
}
```

## Error Handling for External Services

### Custom Error Classes

```typescript
// src/services/external/errors.ts
export class ExternalApiError extends Error {
  constructor(
    message: string,
    public statusCode: number,
    public retryable: boolean = false,
  ) {
    super(message);
    this.name = 'ExternalApiError';
  }
}

export class RateLimitError extends ExternalApiError {
  constructor(
    message: string,
    public retryAfter: number,
  ) {
    super(message, 429, true);
    this.name = 'RateLimitError';
  }
}
```

### Retry with Exponential Backoff

```typescript
async function fetchWithRetry<T>(
  fn: () => Promise<T>,
  options: { maxRetries: number; baseDelay: number },
): Promise<T> {
  let lastError: Error;

  for (let attempt = 0; attempt <= options.maxRetries; attempt++) {
    try {
      return await fn();
    } catch (error) {
      lastError = error as Error;

      if (error instanceof ExternalApiError && !error.retryable) {
        throw error;
      }

      if (attempt < options.maxRetries) {
        const delay = options.baseDelay * Math.pow(2, attempt);
        await new Promise((resolve) => setTimeout(resolve, delay));
      }
    }
  }

  throw lastError!;
}
```

## Rate Limiting Strategies

### Token Bucket Pattern

```typescript
class RateLimiter {
  private tokens: number;
  private lastRefill: number;
  private readonly maxTokens: number;
  private readonly refillRate: number; // tokens per second

  constructor(maxTokens: number, refillRate: number) {
    this.maxTokens = maxTokens;
    this.tokens = maxTokens;
    this.refillRate = refillRate;
    this.lastRefill = Date.now();
  }

  async acquire(): Promise<void> {
    this.refill();

    if (this.tokens < 1) {
      const waitTime = ((1 - this.tokens) / this.refillRate) * 1000;
      await new Promise((resolve) => setTimeout(resolve, waitTime));
      this.refill();
    }

    this.tokens -= 1;
  }

  private refill(): void {
    const now = Date.now();
    const elapsed = (now - this.lastRefill) / 1000;
    this.tokens = Math.min(this.maxTokens, this.tokens + elapsed * this.refillRate);
    this.lastRefill = now;
  }
}
```

## Testing Integrations

### Mocking External Services

```typescript
// src/tests/mocks/storeApi.mock.ts
import { vi } from 'vitest';

export const mockStoreApiClient = {
  getStoreLocations: vi.fn(),
};

vi.mock('@/services/external/storeApi.server', () => ({
  storeApiClient: mockStoreApiClient,
}));
```

### Integration Test with Real Service

```typescript
// src/tests/integration/storeApi.integration.test.ts
describe('Store API Integration', () => {
  it.skipIf(!env.STORE_API_KEY)('fetches real store locations', async () => {
    const locations = await storeApiClient.getStoreLocations('test-store');
    expect(locations).toBeInstanceOf(Array);
  });
});
```

## MCP Tools for Integrations

### Gitea Integration

```
// List repositories
mcp__gitea-projectium__list_my_repos()

// Create issue
mcp__gitea-projectium__create_issue({
  owner: "projectium",
  repo: "flyer-crawler",
  title: "Issue title",
  body: "Issue description"
})
```

### Bugsink Integration

```
// List projects
mcp__bugsink__list_projects()

// Get issue details
mcp__bugsink__get_issue({ issue_id: "..." })

// Get stacktrace
mcp__bugsink__get_stacktrace({ event_id: "..." })
```

## Security Considerations

### API Key Storage

- Never commit API keys to version control
- Use environment variables via `src/config/env.ts`
- Rotate keys periodically
- Use separate keys for dev/test/prod

### Webhook Security

- Always verify webhook signatures
- Use HTTPS for webhook endpoints
- Implement idempotency
- Log webhook events for audit

### OAuth Security

- Use state parameter to prevent CSRF
- Store tokens securely (encrypted at rest)
- Implement token refresh before expiration
- Validate token scopes

## Related Documentation

- [OVERVIEW.md](./OVERVIEW.md) - Subagent system overview
- [SECURITY-DEBUG-GUIDE.md](./SECURITY-DEBUG-GUIDE.md) - Security patterns
- [AI-USAGE-GUIDE.md](./AI-USAGE-GUIDE.md) - Gemini API integration
- [../adr/0041-ai-gemini-integration-architecture.md](../adr/0041-ai-gemini-integration-architecture.md) - AI integration ADR
- [../adr/0016-api-security-hardening.md](../adr/0016-api-security-hardening.md) - API security
- [../adr/0048-authentication-strategy.md](../adr/0048-authentication-strategy.md) - Authentication