comprehensive documentation review + test fixes
Some checks failed
Deploy to Test Environment / deploy-to-test (push) Failing after 2m15s

This commit is contained in:
2026-01-28 16:35:38 -08:00
parent b6c3ca9abe
commit 45ac4fccf5
38 changed files with 2602 additions and 601 deletions

View File

@@ -2,10 +2,47 @@
Aggregates logs from PostgreSQL, PM2, Redis, NGINX; forwards errors to Bugsink.
**Last verified**: 2026-01-28
**Related documentation**:
- [ADR-050: PostgreSQL Function Observability](../adr/0050-postgresql-function-observability.md)
- [ADR-015: Error Tracking and Observability](../adr/0015-error-tracking-and-observability.md)
- [Monitoring Guide](MONITORING.md)
- [Logstash Troubleshooting Runbook](LOGSTASH-TROUBLESHOOTING.md)
---
## Quick Reference
### Bugsink Project Routing
| Source Type | Environment | Bugsink Project | Project ID |
| -------------- | ----------- | -------------------- | ---------- |
| PM2 API/Worker | Dev | Backend API (Dev) | 1 |
| PostgreSQL | Dev | Backend API (Dev) | 1 |
| Frontend JS | Dev | Frontend (Dev) | 2 |
| Redis/NGINX | Dev | Infrastructure (Dev) | 4 |
| PM2 API/Worker | Production | Backend API (Prod) | 1 |
| PostgreSQL | Production | Backend API (Prod) | 1 |
| PM2 API/Worker | Test | Backend API (Test) | 3 |
### Key DSN Keys (Dev Container)
| Project | DSN Key |
| -------------------- | ---------------------------------- |
| Backend API (Dev) | `cea01396c56246adb5878fa5ee6b1d22` |
| Frontend (Dev) | `d92663cb73cf4145b677b84029e4b762` |
| Infrastructure (Dev) | `14e8791da3d347fa98073261b596cab9` |
---
## Configuration
**Primary config**: `/etc/logstash/conf.d/bugsink.conf`
**Dev container config**: `docker/logstash/bugsink.conf`
### Related Files
| Path | Purpose |
@@ -89,6 +126,34 @@ MSYS_NO_PATHCONV=1 podman exec flyer-crawler-dev ls -la /var/log/redis/
## Troubleshooting
### Decision Tree: Logs Not Appearing in Bugsink
```text
Errors not showing in Bugsink?
|
+-- Logstash running?
| |
| +-- No --> systemctl start logstash
| +-- Yes --> Check pipeline stats
| |
| +-- Events in = 0?
| | |
| | +-- Log files exist? --> ls /var/log/pm2/*.log
| | +-- Permissions OK? --> groups logstash
| |
| +-- Events filtered = high?
| | |
| | +-- Grok failures --> Check log format matches pattern
| |
| +-- Events out but no Bugsink?
| |
| +-- 403 error --> Wrong DSN key
| +-- 500 error --> Invalid event format (check sentry_level)
| +-- Connection refused --> Bugsink not running
```
### Common Issues Table
| Issue | Check | Solution |
| --------------------- | ---------------- | ---------------------------------------------------------------------------------------------- |
| No Bugsink errors | Logstash running | `systemctl status logstash` |
@@ -103,6 +168,25 @@ MSYS_NO_PATHCONV=1 podman exec flyer-crawler-dev ls -la /var/log/redis/
| High disk usage | Log rotation | Verify `/etc/logrotate.d/logstash` configured |
| varchar(7) error | Level validation | Add Ruby filter to validate/normalize `sentry_level` before output |
### Expected Output Examples
**Successful Logstash pipeline stats**:
```json
{
"in": 1523,
"out": 1520,
"filtered": 1520,
"queue_push_duration_in_millis": 45
}
```
**Healthy Bugsink HTTP response**:
```json
{ "id": "a1b2c3d4e5f6..." }
```
## Related Documentation
- **Dev Container Guide**: [DEV-CONTAINER.md](../development/DEV-CONTAINER.md) - PM2 and log aggregation in dev