flyer-crawler.projectium.com/docs/adr/0063-pm2-namespace-implementation.md

ADR-063: PM2 Namespace Implementation

Status

Accepted

Context

Problem

The PM2 process isolation safeguards implemented in ADR-061 successfully prevented cross-application process deletion but introduced operational complexity. Every PM2 command in deployment workflows required:

1. Process name filtering logic (JavaScript inline scripts)
2. Safety abort checks (process count validation)
3. Pre/post verification logging

Additionally, simultaneous test and production deployments created a race condition with pm2 save:

• Test deployment: pm2 save writes test processes to dump file
• Prod deployment: pm2 save writes prod processes to dump file (overwrites test state)
• PM2 daemon restart: Restores incomplete process list

This race condition could cause process loss on PM2 daemon restart.

Requirements

1. Complete isolation between test/prod/dev PM2 processes
2. Eliminate pm2 save race condition
3. Simplify workflow commands
4. Maintain backward compatibility during migration

Decision

Implement PM2 namespaces with separate dump files per environment:

Namespace	Config File	Use Case
flyer-crawler-prod	ecosystem.config.cjs	Production deployment
flyer-crawler-test	ecosystem-test.config.cjs	Test/staging deployment
flyer-crawler-dev	ecosystem.dev.config.cjs	Local development

Implementation

Ecosystem Config Changes

Each config file declares its namespace at the module level:

// ecosystem.config.cjs (production)
module.exports = {
  namespace: 'flyer-crawler-prod',
  apps: [
    /* ... */
  ],
};

// ecosystem-test.config.cjs (test)
module.exports = {
  namespace: 'flyer-crawler-test',
  apps: [
    /* ... */
  ],
};

// ecosystem.dev.config.cjs (development)
module.exports = {
  namespace: 'flyer-crawler-dev',
  apps: [
    /* ... */
  ],
};

Workflow Command Pattern

All PM2 commands require --namespace flag:

# Start/reload
pm2 startOrReload ecosystem.config.cjs --update-env --namespace flyer-crawler-prod

# Process management
pm2 stop flyer-crawler-api --namespace flyer-crawler-prod
pm2 restart flyer-crawler-api flyer-crawler-worker --namespace flyer-crawler-prod
pm2 delete flyer-crawler-api --namespace flyer-crawler-prod

# Status
pm2 list --namespace flyer-crawler-prod
pm2 jlist --namespace flyer-crawler-prod
pm2 logs flyer-crawler-api --namespace flyer-crawler-prod
pm2 describe flyer-crawler-api --namespace flyer-crawler-prod

# Save (namespace-isolated dump file)
pm2 save --namespace flyer-crawler-prod

Migration Script

Zero-downtime migration from unnamed processes to namespaced processes:

#!/bin/bash
# migrate-to-pm2-namespaces.sh

# 1. Stop old processes (by name)
pm2 stop flyer-crawler-api flyer-crawler-worker flyer-crawler-analytics-worker || true
pm2 stop flyer-crawler-api-test flyer-crawler-worker-test flyer-crawler-analytics-worker-test || true

# 2. Delete old processes
pm2 delete flyer-crawler-api flyer-crawler-worker flyer-crawler-analytics-worker || true
pm2 delete flyer-crawler-api-test flyer-crawler-worker-test flyer-crawler-analytics-worker-test || true

# 3. Save to clear old dump file
pm2 save --force

# 4. Start with namespaces
cd /var/www/flyer-crawler.projectium.com
pm2 start ecosystem.config.cjs --namespace flyer-crawler-prod
pm2 save --namespace flyer-crawler-prod

cd /var/www/flyer-crawler-test.projectium.com
pm2 start ecosystem-test.config.cjs --namespace flyer-crawler-test
pm2 save --namespace flyer-crawler-test

Consequences

Positive

1. Complete Process Isolation: Namespaces create logical boundaries preventing cross-environment process operations
2. No Save Race Condition: Each namespace maintains separate dump file at ~/.pm2/dump-<namespace>.pm2
3. Simplified Commands: No inline JavaScript filtering; use explicit namespace flag
4. Clear Organization: pm2 list --namespace <name> shows only relevant processes
5. Retained Safeguards: Defense-in-depth from ADR-061 remains as additional protection layer

Negative

1. Command Verbosity: All PM2 commands require --namespace flag
2. Migration Required: One-time migration to move existing processes into namespaces
3. Learning Curve: Team must remember to include namespace flag

Trade-offs

Without Namespace	With Namespace
pm2 list	pm2 list --namespace flyer-crawler-prod
pm2 logs app	pm2 logs app --namespace flyer-crawler-prod
pm2 restart app	pm2 restart app --namespace flyer-crawler-prod
Filter logic in workflows	Explicit namespace declaration
Single dump file (race condition)	Per-namespace dump files

Files Modified

File	Changes
ecosystem.config.cjs	Added namespace: 'flyer-crawler-prod'
ecosystem-test.config.cjs	Added namespace: 'flyer-crawler-test'
ecosystem.dev.config.cjs	Added namespace: 'flyer-crawler-dev'
.gitea/workflows/deploy-to-prod.yml	Added --namespace flyer-crawler-prod to all PM2 commands
.gitea/workflows/deploy-to-test.yml	Added --namespace flyer-crawler-test to all PM2 commands
.gitea/workflows/restart-pm2.yml	Added --namespace flag for both environments
.gitea/workflows/manual-deploy-major.yml	Added --namespace flyer-crawler-prod to PM2 commands

Verification

After migration, verify namespace isolation:

# Should show only production processes
pm2 list --namespace flyer-crawler-prod

# Should show only test processes
pm2 list --namespace flyer-crawler-test

# Should show only dev processes (if running)
pm2 list --namespace flyer-crawler-dev

# Verify separate dump files exist
ls -la ~/.pm2/dump-flyer-crawler-*.pm2

Related Documentation

• ADR-061: PM2 Process Isolation Safeguards - Prior safeguards (still active)
• ADR-014: Containerization and Deployment Strategy - Overall deployment architecture
• PM2 Namespace Documentation

References

• PM2 Ecosystem File: https://pm2.keymetrics.io/docs/usage/application-declaration/
• PM2 Namespaces: https://pm2.keymetrics.io/docs/usage/process-management/#namespaces