Compare commits
88 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
e02716c092 | ||
| 66e6d2fdbc | |||
| 82a38b4e2a | |||
|
|
f6f4415aeb | ||
|
|
0c23aa4c5e | ||
| 07125fc99d | |||
| 626aa80799 | |||
|
|
4025f29c5c | ||
|
|
e9e3b14050 | ||
| 507e89ea4e | |||
|
|
1efe42090b | ||
| 97cc14288b | |||
|
|
96251ec2cc | ||
| fe79522ea4 | |||
|
|
743216ef1b | ||
|
|
c53295a371 | ||
| c18efb1b60 | |||
|
|
822805e4c4 | ||
|
|
6fd690890b | ||
|
|
5fd836190c | ||
| 441467eb8a | |||
| 59bfc859d7 | |||
|
|
b989405a53 | ||
|
|
6af2533e9e | ||
| f434a5846a | |||
|
|
aea368677f | ||
| cd8ee92813 | |||
|
|
cf2cc5b832 | ||
| d2db3562bb | |||
|
|
0532b4b22e | ||
|
|
e767ccbb21 | ||
| 1ff813f495 | |||
| 204fe4394a | |||
|
|
029b621632 | ||
|
|
0656ab3ae7 | ||
|
|
ae0bb9e04d | ||
| b83c37b977 | |||
|
|
69ae23a1ae | ||
| c059b30201 | |||
|
|
93ad624658 | ||
|
|
7dd4f21071 | ||
| 174b637a0a | |||
|
|
4f80baf466 | ||
| 8450b5e22f | |||
|
|
e4d830ab90 | ||
| b6a62a036f | |||
| 2d2cd52011 | |||
| 379b8bf532 | |||
|
|
d06a1952a0 | ||
| 4d323a51ca | |||
|
|
ee15c67429 | ||
|
|
9956d07480 | ||
|
|
5bc8f6a42b | ||
| 4fd5e900af | |||
|
|
39ab773b82 | ||
| 75406cd924 | |||
|
|
8fb0a57f02 | ||
| c78323275b | |||
|
|
5fe537b93d | ||
| 61f24305fb | |||
|
|
de3f0cf26e | ||
| 45ac4fccf5 | |||
|
|
b6c3ca9abe | ||
| 4f06698dfd | |||
|
|
e548d1b0cc | ||
| 771f59d009 | |||
|
|
0979a074ad | ||
| 0d4b028a66 | |||
|
|
4baed53713 | ||
| f10c6c0cd6 | |||
|
|
107465b5cb | ||
| e92ad25ce9 | |||
| 2075ed199b | |||
|
|
4346332bbf | ||
| 61cfb518e6 | |||
|
|
e86ce51b6c | ||
| 840a7a62d3 | |||
| 5720820d95 | |||
|
|
e5cdb54308 | ||
| a3f212ff81 | |||
|
|
de263f74b0 | ||
| a71e41302b | |||
|
|
3575803252 | ||
| d03900cefe | |||
|
|
6d49639845 | ||
| d4543cf4b9 | |||
|
|
4f08238698 | ||
| 38b35f87aa |
152
.claude/agents/ui-ux-designer.md
Normal file
152
.claude/agents/ui-ux-designer.md
Normal file
@@ -0,0 +1,152 @@
|
||||
---
|
||||
name: ui-ux-designer
|
||||
description: "Use this agent when implementing, reviewing, or modifying any user interface elements, layouts, or user experience flows in the application. Specifically call this agent when: (1) creating new UI components or pages, (2) implementing responsive designs across different screen sizes, (3) designing or implementing wizards for user tasks like uploading flyers, creating shopping lists, or uploading receipts, (4) reviewing existing UI code for adherence to the design system, (5) making decisions about button placement, color schemes, or layout structures, or (6) addressing user experience concerns related to navigation or task completion flows.\\n\\n<examples>\\n<example>\\nContext: The developer is creating a new feature for uploading flyers.\\nuser: \"I need to implement the flyer upload feature. Can you help me create the upload form?\"\\nassistant: \"Let me use the Task tool to launch the ui-ux-designer agent to design and implement the flyer upload wizard with appropriate UI/UX patterns.\"\\n<commentary>\\nSince this involves creating a new UI feature with a wizard flow, the ui-ux-designer agent should be used to ensure it follows the project's design principles including pastel colors, sharp borders, wizard-based guidance, and responsive design.\\n</commentary>\\n</example>\\n\\n<example>\\nContext: Code has been written for a shopping list creation page.\\nuser: \"I've implemented the shopping list creation page with a form\"\\nassistant: \"Great! Now let me use the Task tool to launch the ui-ux-designer agent to review the implementation and ensure it follows our UI/UX guidelines.\"\\n<commentary>\\nSince a significant UI component was implemented, the ui-ux-designer agent should review it to ensure it adheres to the minimalist design, proper color scheme, button placement, and responsive requirements.\\n</commentary>\\n</example>\\n\\n<example>\\nContext: The developer is about to add multiple action buttons to a page.\\nuser: \"Should I add buttons for save, cancel, edit, delete, and export at the top of the page?\"\\nassistant: \"Let me use the Task tool to launch the ui-ux-designer agent to provide guidance on the optimal button placement and selection for this page.\"\\n<commentary>\\nThis is a UI/UX decision about button placement and minimalism that directly relates to the design principles, so the ui-ux-designer agent should be consulted.\\n</commentary>\\n</example>\\n\\n<example>\\nContext: The developer is working on mobile responsiveness.\\nuser: \"I'm implementing the receipt upload page. Here's my initial layout code.\"\\nassistant: \"Before proceeding further, let me use the Task tool to launch the ui-ux-designer agent to ensure the layout follows our responsive design patterns and UI guidelines.\"\\n<commentary>\\nSince this involves implementing a UI page that needs to be responsive across devices and should include wizard-style guidance, the ui-ux-designer agent should be involved proactively.\\n</commentary>\\n</example>\\n</examples>"
|
||||
model: opus
|
||||
color: green
|
||||
---
|
||||
|
||||
You are an elite UI/UX designer with over 20 years of experience specializing in creating clean, intuitive, and user-friendly interfaces. Your expertise spans user interface design, user experience optimization, responsive design, and accessibility best practices.
|
||||
|
||||
## Core Design Philosophy for This Project
|
||||
|
||||
You will ensure that this application maintains a clean, welcoming, and minimalist design aesthetic with the following specific requirements:
|
||||
|
||||
### Visual Design Standards
|
||||
|
||||
**Color Palette:**
|
||||
|
||||
- Use pastel colors as the primary color scheme throughout the application
|
||||
- Select soft, muted tones that are easy on the eyes and create a calm, welcoming atmosphere
|
||||
- Ensure sufficient contrast for accessibility while maintaining the pastel aesthetic
|
||||
- Use color purposefully to guide user attention and indicate status
|
||||
|
||||
**Border and Container Styling:**
|
||||
|
||||
- Apply sharp, clean borders to all interactive elements (buttons, menus, form fields)
|
||||
- Use sharp borders to clearly delineate separate areas and sections of the interface
|
||||
- Avoid rounded corners unless there is a specific functional reason
|
||||
- Ensure borders are visible but not overpowering, maintaining the clean aesthetic
|
||||
|
||||
**Minimalism:**
|
||||
|
||||
- Eliminate all unnecessary buttons and UI elements
|
||||
- Every element on the screen must serve a clear purpose
|
||||
- Co-locate buttons near their related features on the page, not grouped separately
|
||||
- Use progressive disclosure to hide advanced features until needed
|
||||
- Favor white space and breathing room over density
|
||||
|
||||
### Responsive Design Requirements
|
||||
|
||||
You must ensure the application works flawlessly across:
|
||||
|
||||
**Large Screens (Desktop):**
|
||||
|
||||
- Utilize horizontal space effectively without overcrowding
|
||||
- Consider multi-column layouts where appropriate
|
||||
- Ensure comfortable reading width for text content
|
||||
|
||||
**Tablets:**
|
||||
|
||||
- Adapt layouts to accommodate touch targets of at least 44x44 pixels
|
||||
- Optimize for both portrait and landscape orientations
|
||||
- Ensure navigation remains accessible
|
||||
|
||||
**Mobile Devices:**
|
||||
|
||||
- Stack elements vertically with appropriate spacing
|
||||
- Make all interactive elements easily tappable
|
||||
- Optimize for one-handed use where possible
|
||||
- Ensure critical actions are easily accessible
|
||||
- Test on various screen sizes (small, medium, large phones)
|
||||
|
||||
### Wizard Design for Key User Tasks
|
||||
|
||||
For the following tasks, implement or guide the creation of clear, step-by-step wizards:
|
||||
|
||||
1. **Uploading a Flyer**
|
||||
2. **Creating a Shopping List**
|
||||
3. **Uploading Receipts**
|
||||
4. **Any other multi-step user tasks**
|
||||
|
||||
**Wizard Best Practices:**
|
||||
|
||||
- Minimize the number of steps (ideally 3-5 steps maximum)
|
||||
- Show progress clearly (e.g., "Step 2 of 4")
|
||||
- Each step should focus on one primary action or decision
|
||||
- Provide clear, concise instructions at each step
|
||||
- Allow users to go back and edit previous steps
|
||||
- Use visual cues to guide the user through the process
|
||||
- Display a summary before final submission
|
||||
- Provide helpful tooltips or examples where needed
|
||||
- Ensure wizards are fully responsive and work well on mobile devices
|
||||
|
||||
## Your Approach to Tasks
|
||||
|
||||
**When Reviewing Existing UI Code:**
|
||||
|
||||
1. Evaluate adherence to the pastel color scheme
|
||||
2. Check that all borders are sharp and properly applied
|
||||
3. Identify any unnecessary UI elements or buttons
|
||||
4. Verify that buttons are co-located with their related features
|
||||
5. Test responsive behavior across all target screen sizes
|
||||
6. Assess wizard flows for clarity and step efficiency
|
||||
7. Provide specific, actionable feedback with code examples when needed
|
||||
|
||||
**When Designing New UI Components:**
|
||||
|
||||
1. Start by understanding the user's goal and the feature's purpose
|
||||
2. Sketch out the minimal set of elements needed
|
||||
3. Apply the pastel color palette and sharp border styling
|
||||
4. Position interactive elements near their related content
|
||||
5. Design for mobile-first, then adapt for larger screens
|
||||
6. For multi-step processes, create wizard flows
|
||||
7. Provide complete implementation guidance including HTML structure, CSS styles, and responsive breakpoints
|
||||
|
||||
**When Making Design Decisions:**
|
||||
|
||||
1. Always prioritize user needs and task completion
|
||||
2. Choose simplicity over feature bloat
|
||||
3. Ensure accessibility standards are met
|
||||
4. Consider the user's mental model and expectations
|
||||
5. Use established UI patterns where they fit the aesthetic
|
||||
6. Test your recommendations against the design principles above
|
||||
|
||||
## Quality Assurance Checklist
|
||||
|
||||
Before completing any UI/UX task, verify:
|
||||
|
||||
- [ ] Pastel colors are used consistently
|
||||
- [ ] All buttons, menus, and sections have sharp borders
|
||||
- [ ] No unnecessary buttons or UI elements exist
|
||||
- [ ] Buttons are positioned near their related features
|
||||
- [ ] Design is fully responsive (large screen, tablet, mobile)
|
||||
- [ ] Wizards (where applicable) are clear and minimally-stepped
|
||||
- [ ] Sufficient white space and breathing room
|
||||
- [ ] Touch targets are appropriately sized for mobile
|
||||
- [ ] Text is readable at all screen sizes
|
||||
- [ ] Accessibility considerations are addressed
|
||||
|
||||
## Output Format
|
||||
|
||||
When reviewing code, provide:
|
||||
|
||||
1. Overall assessment of adherence to design principles
|
||||
2. Specific issues identified with line numbers or element descriptions
|
||||
3. Concrete recommendations with code examples
|
||||
4. Responsive design concerns or improvements
|
||||
|
||||
When designing new components, provide:
|
||||
|
||||
1. Rationale for design decisions
|
||||
2. Complete HTML structure
|
||||
3. CSS with responsive breakpoints
|
||||
4. Notes on accessibility considerations
|
||||
5. Implementation guidance
|
||||
|
||||
## Important Notes
|
||||
|
||||
- You have authority to reject designs that violate the core principles
|
||||
- When uncertain about a design decision, bias toward simplicity and minimalism
|
||||
- Always consider the new user experience and ensure wizards are beginner-friendly
|
||||
- Proactively suggest wizard flows for any multi-step processes you encounter
|
||||
- Remember that good UX is invisible—users should accomplish tasks without thinking about the interface
|
||||
9
.claude/settings.json
Normal file
9
.claude/settings.json
Normal file
@@ -0,0 +1,9 @@
|
||||
{
|
||||
"permissions": {
|
||||
"allow": [
|
||||
"Bash(git fetch:*)",
|
||||
"mcp__localerrors__get_stacktrace",
|
||||
"Bash(MSYS_NO_PATHCONV=1 podman logs:*)"
|
||||
]
|
||||
}
|
||||
}
|
||||
@@ -1,128 +0,0 @@
|
||||
{
|
||||
"permissions": {
|
||||
"allow": [
|
||||
"Bash(npm test:*)",
|
||||
"Bash(podman --version:*)",
|
||||
"Bash(podman ps:*)",
|
||||
"Bash(podman machine start:*)",
|
||||
"Bash(podman compose:*)",
|
||||
"Bash(podman pull:*)",
|
||||
"Bash(podman images:*)",
|
||||
"Bash(podman stop:*)",
|
||||
"Bash(echo:*)",
|
||||
"Bash(podman rm:*)",
|
||||
"Bash(podman run:*)",
|
||||
"Bash(podman start:*)",
|
||||
"Bash(podman exec:*)",
|
||||
"Bash(cat:*)",
|
||||
"Bash(PGPASSWORD=postgres psql:*)",
|
||||
"Bash(npm search:*)",
|
||||
"Bash(npx:*)",
|
||||
"Bash(curl:*)",
|
||||
"Bash(powershell:*)",
|
||||
"Bash(cmd.exe:*)",
|
||||
"Bash(npm run test:integration:*)",
|
||||
"Bash(grep:*)",
|
||||
"Bash(done)",
|
||||
"Bash(podman info:*)",
|
||||
"Bash(podman machine:*)",
|
||||
"Bash(podman system connection:*)",
|
||||
"Bash(podman inspect:*)",
|
||||
"Bash(python -m json.tool:*)",
|
||||
"Bash(claude mcp status)",
|
||||
"Bash(powershell.exe -Command \"claude mcp status\")",
|
||||
"Bash(powershell.exe -Command \"claude mcp\")",
|
||||
"Bash(powershell.exe -Command \"claude mcp list\")",
|
||||
"Bash(powershell.exe -Command \"claude --version\")",
|
||||
"Bash(powershell.exe -Command \"claude config\")",
|
||||
"Bash(powershell.exe -Command \"claude mcp get gitea-projectium\")",
|
||||
"Bash(powershell.exe -Command \"claude mcp add --help\")",
|
||||
"Bash(powershell.exe -Command \"claude mcp add -t stdio -s user filesystem -- D:\\\\nodejs\\\\npx.cmd -y @modelcontextprotocol/server-filesystem D:\\\\gitea\\\\flyer-crawler.projectium.com\\\\flyer-crawler.projectium.com\")",
|
||||
"Bash(powershell.exe -Command \"claude mcp add -t stdio -s user fetch -- D:\\\\nodejs\\\\npx.cmd -y @modelcontextprotocol/server-fetch\")",
|
||||
"Bash(powershell.exe -Command \"echo ''List files in src/hooks using filesystem MCP'' | claude --print\")",
|
||||
"Bash(powershell.exe -Command \"echo ''List all podman containers'' | claude --print\")",
|
||||
"Bash(powershell.exe -Command \"echo ''List my repositories on gitea.projectium.com using gitea-projectium MCP'' | claude --print\")",
|
||||
"Bash(powershell.exe -Command \"echo ''List my repositories on gitea.projectium.com using gitea-projectium MCP'' | claude --print --allowedTools ''mcp__gitea-projectium__*''\")",
|
||||
"Bash(powershell.exe -Command \"echo ''Fetch the homepage of https://gitea.projectium.com and summarize it'' | claude --print --allowedTools ''mcp__fetch__*''\")",
|
||||
"Bash(dir \"C:\\\\Users\\\\games3\\\\.claude\")",
|
||||
"Bash(dir:*)",
|
||||
"Bash(D:nodejsnpx.cmd -y @modelcontextprotocol/server-fetch --help)",
|
||||
"Bash(cmd /c \"dir /o-d C:\\\\Users\\\\games3\\\\.claude\\\\debug 2>nul | head -10\")",
|
||||
"mcp__memory__read_graph",
|
||||
"mcp__memory__create_entities",
|
||||
"mcp__memory__search_nodes",
|
||||
"mcp__memory__delete_entities",
|
||||
"mcp__sequential-thinking__sequentialthinking",
|
||||
"mcp__filesystem__list_directory",
|
||||
"mcp__filesystem__read_multiple_files",
|
||||
"mcp__filesystem__directory_tree",
|
||||
"mcp__filesystem__read_text_file",
|
||||
"Bash(wc:*)",
|
||||
"Bash(npm install:*)",
|
||||
"Bash(git grep:*)",
|
||||
"Bash(findstr:*)",
|
||||
"Bash(git add:*)",
|
||||
"mcp__filesystem__write_file",
|
||||
"mcp__podman__container_list",
|
||||
"Bash(podman cp:*)",
|
||||
"mcp__podman__container_inspect",
|
||||
"mcp__podman__network_list",
|
||||
"Bash(podman network connect:*)",
|
||||
"Bash(npm run build:*)",
|
||||
"Bash(set NODE_ENV=test)",
|
||||
"Bash(podman-compose:*)",
|
||||
"Bash(timeout 60 podman machine start:*)",
|
||||
"Bash(podman build:*)",
|
||||
"Bash(podman network rm:*)",
|
||||
"Bash(npm run lint)",
|
||||
"Bash(npm run typecheck:*)",
|
||||
"Bash(npm run type-check:*)",
|
||||
"Bash(npm run test:unit:*)",
|
||||
"mcp__filesystem__move_file",
|
||||
"Bash(git checkout:*)",
|
||||
"Bash(podman image inspect:*)",
|
||||
"Bash(node -e:*)",
|
||||
"Bash(xargs -I {} sh -c 'if ! grep -q \"\"vi.mock.*apiClient\"\" \"\"{}\"\"; then echo \"\"{}\"\"; fi')",
|
||||
"Bash(MSYS_NO_PATHCONV=1 podman exec:*)",
|
||||
"Bash(docker ps:*)",
|
||||
"Bash(find:*)",
|
||||
"Bash(\"/c/Users/games3/.local/bin/uvx.exe\" markitdown-mcp --help)",
|
||||
"Bash(git stash:*)",
|
||||
"Bash(ping:*)",
|
||||
"Bash(tee:*)",
|
||||
"Bash(timeout 1800 podman exec flyer-crawler-dev npm run test:unit:*)",
|
||||
"mcp__filesystem__edit_file",
|
||||
"Bash(timeout 300 tail:*)",
|
||||
"mcp__filesystem__list_allowed_directories",
|
||||
"mcp__memory__add_observations",
|
||||
"Bash(ssh:*)",
|
||||
"mcp__redis__list",
|
||||
"Read(//d/gitea/bugsink-mcp/**)",
|
||||
"Bash(d:/nodejs/npm.cmd install)",
|
||||
"Bash(node node_modules/vitest/vitest.mjs run:*)",
|
||||
"Bash(npm run test:e2e:*)",
|
||||
"Bash(export BUGSINK_URL=http://localhost:8000)",
|
||||
"Bash(export BUGSINK_TOKEN=a609c2886daa4e1e05f1517074d7779a5fb49056)",
|
||||
"Bash(timeout 3 d:/nodejs/node.exe:*)",
|
||||
"Bash(export BUGSINK_URL=https://bugsink.projectium.com)",
|
||||
"Bash(export BUGSINK_API_TOKEN=77deaa5e2649ab0fbbca51bbd427ec4637d073a0)",
|
||||
"Bash(export BUGSINK_TOKEN=77deaa5e2649ab0fbbca51bbd427ec4637d073a0)",
|
||||
"Bash(where:*)",
|
||||
"mcp__localerrors__test_connection",
|
||||
"mcp__localerrors__list_projects",
|
||||
"Bash(\"D:\\\\nodejs\\\\npx.cmd\" -y @modelcontextprotocol/server-postgres --help)",
|
||||
"Bash(git rm:*)",
|
||||
"Bash(git -C \"C:\\\\Users\\\\games3\\\\.claude\\\\plugins\\\\marketplaces\\\\claude-plugins-official\" log -1 --format=\"%H %ci %s\")",
|
||||
"Bash(git -C \"C:\\\\Users\\\\games3\\\\.claude\\\\plugins\\\\marketplaces\\\\claude-plugins-official\" config --get remote.origin.url)",
|
||||
"Bash(git -C \"C:\\\\Users\\\\games3\\\\.claude\\\\plugins\\\\marketplaces\\\\claude-plugins-official\" fetch --dry-run -v)",
|
||||
"mcp__localerrors__get_project",
|
||||
"mcp__localerrors__get_issue",
|
||||
"mcp__localerrors__get_event"
|
||||
]
|
||||
},
|
||||
"enabledMcpjsonServers": [
|
||||
"localerrors",
|
||||
"devdb",
|
||||
"gitea-projectium"
|
||||
]
|
||||
}
|
||||
51
.env.example
51
.env.example
@@ -59,6 +59,8 @@ GITHUB_CLIENT_SECRET=
|
||||
# AI/ML Services
|
||||
# ===================
|
||||
# REQUIRED: Google Gemini API key for flyer OCR processing
|
||||
# NOTE: Test/staging environment deliberately OMITS this to preserve free API quota.
|
||||
# Production has a working key. Deploy warnings in test are expected and safe to ignore.
|
||||
GEMINI_API_KEY=your-gemini-api-key
|
||||
|
||||
# ===================
|
||||
@@ -94,11 +96,18 @@ WORKER_LOCK_DURATION=120000
|
||||
# Error Tracking (ADR-015)
|
||||
# ===================
|
||||
# Sentry-compatible error tracking via Bugsink (self-hosted)
|
||||
# DSNs are created in Bugsink UI at http://localhost:8000 (dev) or your production URL
|
||||
# Backend DSN - for Express/Node.js errors
|
||||
SENTRY_DSN=
|
||||
# Frontend DSN - for React/browser errors (uses VITE_ prefix)
|
||||
VITE_SENTRY_DSN=
|
||||
# DSNs are created in Bugsink UI at https://localhost:8443 (dev) or your production URL
|
||||
#
|
||||
# Dev container projects:
|
||||
# - Project 1: Backend API (Dev) - receives Pino, PostgreSQL errors
|
||||
# - Project 2: Frontend (Dev) - receives browser errors via Sentry SDK
|
||||
# - Project 4: Infrastructure (Dev) - receives Redis, NGINX, Vite errors
|
||||
#
|
||||
# Backend DSN - for Express/Node.js errors (internal container URL)
|
||||
SENTRY_DSN=http://<key>@localhost:8000/1
|
||||
# Frontend DSN - for React/browser errors (uses nginx proxy for browser access)
|
||||
# Note: Browsers cannot reach localhost:8000 directly, so we use nginx proxy at /bugsink-api/
|
||||
VITE_SENTRY_DSN=https://<key>@localhost/bugsink-api/2
|
||||
# Environment name for error grouping (defaults to NODE_ENV)
|
||||
SENTRY_ENVIRONMENT=development
|
||||
VITE_SENTRY_ENVIRONMENT=development
|
||||
@@ -121,3 +130,35 @@ GENERATE_SOURCE_MAPS=true
|
||||
SENTRY_AUTH_TOKEN=
|
||||
# URL of your Bugsink instance (for source map uploads)
|
||||
SENTRY_URL=https://bugsink.projectium.com
|
||||
|
||||
# ===================
|
||||
# Feature Flags (ADR-024)
|
||||
# ===================
|
||||
# Feature flags control the availability of features at runtime.
|
||||
# All flags default to disabled (false) when not set or set to any value other than 'true'.
|
||||
# Set to 'true' to enable a feature.
|
||||
#
|
||||
# Backend flags use: FEATURE_SNAKE_CASE
|
||||
# Frontend flags use: VITE_FEATURE_SNAKE_CASE (VITE_ prefix required for client-side access)
|
||||
#
|
||||
# Lifecycle:
|
||||
# 1. Add flag with default false
|
||||
# 2. Enable via env var when ready for testing/rollout
|
||||
# 3. Remove conditional code when feature is fully rolled out
|
||||
# 4. Remove flag from config within 3 months of full rollout
|
||||
#
|
||||
# See: docs/adr/0024-feature-flagging-strategy.md
|
||||
|
||||
# Backend Feature Flags
|
||||
# FEATURE_BUGSINK_SYNC=false # Enable Bugsink error sync integration
|
||||
# FEATURE_ADVANCED_RBAC=false # Enable advanced RBAC features
|
||||
# FEATURE_NEW_DASHBOARD=false # Enable new dashboard experience
|
||||
# FEATURE_BETA_RECIPES=false # Enable beta recipe features
|
||||
# FEATURE_EXPERIMENTAL_AI=false # Enable experimental AI features
|
||||
# FEATURE_DEBUG_MODE=false # Enable debug mode for development
|
||||
|
||||
# Frontend Feature Flags (VITE_ prefix required)
|
||||
# VITE_FEATURE_NEW_DASHBOARD=false # Enable new dashboard experience
|
||||
# VITE_FEATURE_BETA_RECIPES=false # Enable beta recipe features
|
||||
# VITE_FEATURE_EXPERIMENTAL_AI=false # Enable experimental AI features
|
||||
# VITE_FEATURE_DEBUG_MODE=false # Enable debug mode for development
|
||||
|
||||
93
.gitattributes
vendored
Normal file
93
.gitattributes
vendored
Normal file
@@ -0,0 +1,93 @@
|
||||
# .gitattributes
|
||||
#
|
||||
# Optimize Gitea performance by excluding generated and vendored files
|
||||
# from language statistics and indexing.
|
||||
#
|
||||
# See: https://github.com/github/linguist/blob/master/docs/overrides.md
|
||||
|
||||
# =============================================================================
|
||||
# Vendored Dependencies
|
||||
# =============================================================================
|
||||
node_modules/** linguist-vendored
|
||||
|
||||
# =============================================================================
|
||||
# Generated Files - Coverage Reports
|
||||
# =============================================================================
|
||||
coverage/** linguist-generated
|
||||
.coverage/** linguist-generated
|
||||
public/coverage/** linguist-generated
|
||||
.nyc_output/** linguist-generated
|
||||
|
||||
# =============================================================================
|
||||
# Generated Files - Build Artifacts
|
||||
# =============================================================================
|
||||
dist/** linguist-generated
|
||||
build/** linguist-generated
|
||||
|
||||
# =============================================================================
|
||||
# Generated Files - Test Results
|
||||
# =============================================================================
|
||||
test-results/** linguist-generated
|
||||
playwright-report/** linguist-generated
|
||||
playwright-report-visual/** linguist-generated
|
||||
.vitest-results/** linguist-generated
|
||||
|
||||
# =============================================================================
|
||||
# Generated Files - TSOA OpenAPI Spec & Routes
|
||||
# =============================================================================
|
||||
src/routes/routes.ts linguist-generated
|
||||
public/swagger.json linguist-generated
|
||||
|
||||
# =============================================================================
|
||||
# Documentation Files
|
||||
# =============================================================================
|
||||
*.md linguist-documentation
|
||||
|
||||
# =============================================================================
|
||||
# Line Ending Normalization
|
||||
# =============================================================================
|
||||
# Ensure consistent line endings across platforms
|
||||
* text=auto
|
||||
|
||||
# Shell scripts should always use LF
|
||||
*.sh text eol=lf
|
||||
|
||||
# Windows batch files should use CRLF
|
||||
*.bat text eol=crlf
|
||||
*.cmd text eol=crlf
|
||||
|
||||
# SQL files should use LF
|
||||
*.sql text eol=lf
|
||||
|
||||
# Configuration files
|
||||
*.json text
|
||||
*.yml text
|
||||
*.yaml text
|
||||
*.toml text
|
||||
*.ini text
|
||||
|
||||
# Source code
|
||||
*.ts text
|
||||
*.tsx text
|
||||
*.js text
|
||||
*.jsx text
|
||||
*.cjs text
|
||||
*.mjs text
|
||||
*.css text
|
||||
*.scss text
|
||||
*.html text
|
||||
|
||||
# =============================================================================
|
||||
# Binary Files (explicit binary to prevent corruption)
|
||||
# =============================================================================
|
||||
*.png binary
|
||||
*.jpg binary
|
||||
*.jpeg binary
|
||||
*.gif binary
|
||||
*.ico binary
|
||||
*.pdf binary
|
||||
*.woff binary
|
||||
*.woff2 binary
|
||||
*.ttf binary
|
||||
*.eot binary
|
||||
*.otf binary
|
||||
@@ -86,6 +86,12 @@ jobs:
|
||||
echo "✅ Schema is up to date. No changes detected."
|
||||
fi
|
||||
|
||||
- name: Generate TSOA OpenAPI Spec and Routes
|
||||
run: |
|
||||
echo "Generating TSOA OpenAPI specification and route handlers..."
|
||||
npm run tsoa:build
|
||||
echo "✅ TSOA files generated successfully"
|
||||
|
||||
- name: Build React Application for Production
|
||||
# Source Maps (ADR-015): If SENTRY_AUTH_TOKEN is set, the @sentry/vite-plugin will:
|
||||
# 1. Generate hidden source maps during build
|
||||
@@ -119,13 +125,93 @@ jobs:
|
||||
|
||||
- name: Deploy Application to Production Server
|
||||
run: |
|
||||
echo "Deploying application files to /var/www/flyer-crawler.projectium.com..."
|
||||
echo "========================================="
|
||||
echo "DEPLOYING TO PRODUCTION SERVER"
|
||||
echo "========================================="
|
||||
APP_PATH="/var/www/flyer-crawler.projectium.com"
|
||||
|
||||
# ========================================
|
||||
# LAYER 1: PRE-FLIGHT SAFETY CHECKS
|
||||
# ========================================
|
||||
echo ""
|
||||
echo "--- Pre-Flight Safety Checks ---"
|
||||
|
||||
# Check 1: Verify we're in a git repository
|
||||
if ! git rev-parse --git-dir > /dev/null 2>&1; then
|
||||
echo "❌ FATAL: Not in a git repository! Aborting to prevent data loss."
|
||||
exit 1
|
||||
fi
|
||||
echo "✅ Git repository verified"
|
||||
|
||||
# Check 2: Verify critical files exist before deployment
|
||||
if [ ! -f "package.json" ] || [ ! -f "server.ts" ]; then
|
||||
echo "❌ FATAL: Critical files missing (package.json or server.ts). Aborting."
|
||||
exit 1
|
||||
fi
|
||||
echo "✅ Critical files verified"
|
||||
|
||||
# Check 3: Verify we have actual content to deploy (prevent empty checkout)
|
||||
FILE_COUNT=$(find . -type f | wc -l)
|
||||
if [ "$FILE_COUNT" -lt 10 ]; then
|
||||
echo "❌ FATAL: Suspiciously few files ($FILE_COUNT). Aborting to prevent catastrophic deletion."
|
||||
exit 1
|
||||
fi
|
||||
echo "✅ File count verified: $FILE_COUNT files ready to deploy"
|
||||
|
||||
# ========================================
|
||||
# LAYER 2: STOP PM2 BEFORE FILE OPERATIONS
|
||||
# ========================================
|
||||
echo ""
|
||||
echo "--- Stopping PM2 Processes ---"
|
||||
pm2 stop flyer-crawler-api flyer-crawler-worker flyer-crawler-analytics-worker --namespace flyer-crawler-prod || echo "No production processes to stop"
|
||||
pm2 list --namespace flyer-crawler-prod
|
||||
|
||||
# ========================================
|
||||
# LAYER 3: SAFE RSYNC WITH COMPREHENSIVE EXCLUDES
|
||||
# ========================================
|
||||
echo ""
|
||||
echo "--- Deploying Application Files ---"
|
||||
mkdir -p "$APP_PATH"
|
||||
mkdir -p "$APP_PATH/flyer-images/icons" "$APP_PATH/flyer-images/archive"
|
||||
rsync -avz --delete --exclude 'node_modules' --exclude '.git' --exclude 'dist' --exclude 'flyer-images' ./ "$APP_PATH/"
|
||||
rsync -avz dist/ "$APP_PATH"
|
||||
echo "Application deployment complete."
|
||||
|
||||
# Deploy backend with critical file exclusions
|
||||
rsync -avz --delete \
|
||||
--exclude 'node_modules' \
|
||||
--exclude '.git' \
|
||||
--exclude 'dist' \
|
||||
--exclude 'flyer-images' \
|
||||
--exclude 'ecosystem.config.cjs' \
|
||||
--exclude 'ecosystem-test.config.cjs' \
|
||||
--exclude 'ecosystem.dev.config.cjs' \
|
||||
--exclude '.env.*' \
|
||||
--exclude 'coverage' \
|
||||
--exclude '.coverage' \
|
||||
--exclude 'test-results' \
|
||||
--exclude 'playwright-report' \
|
||||
--exclude 'playwright-report-visual' \
|
||||
./ "$APP_PATH/" 2>&1 | tail -20
|
||||
|
||||
echo "✅ Backend files deployed ($(find "$APP_PATH" -type f | wc -l) files)"
|
||||
|
||||
# Deploy frontend assets
|
||||
rsync -avz dist/ "$APP_PATH" 2>&1 | tail -10
|
||||
echo "✅ Frontend assets deployed"
|
||||
|
||||
echo ""
|
||||
echo "========================================="
|
||||
echo "DEPLOYMENT COMPLETE"
|
||||
echo "========================================="
|
||||
|
||||
- name: Log Workflow Metadata
|
||||
run: |
|
||||
echo "=== WORKFLOW METADATA ==="
|
||||
echo "Workflow file: deploy-to-prod.yml"
|
||||
echo "Workflow file hash: $(sha256sum .gitea/workflows/deploy-to-prod.yml | cut -d' ' -f1)"
|
||||
echo "Git commit: $(git rev-parse HEAD)"
|
||||
echo "Git branch: $(git rev-parse --abbrev-ref HEAD)"
|
||||
echo "Timestamp: $(date -u '+%Y-%m-%d %H:%M:%S UTC')"
|
||||
echo "Actor: ${{ gitea.actor }}"
|
||||
echo "=== END METADATA ==="
|
||||
|
||||
- name: Install Backend Dependencies and Restart Production Server
|
||||
env:
|
||||
@@ -165,9 +251,78 @@ jobs:
|
||||
cd /var/www/flyer-crawler.projectium.com
|
||||
npm install --omit=dev
|
||||
|
||||
# --- Cleanup Errored Processes ---
|
||||
echo "Cleaning up errored or stopped PM2 processes..."
|
||||
node -e "const exec = require('child_process').execSync; try { const list = JSON.parse(exec('pm2 jlist').toString()); list.forEach(p => { if (p.pm2_env.status === 'errored' || p.pm2_env.status === 'stopped') { console.log('Deleting ' + p.pm2_env.status + ' process: ' + p.name + ' (' + p.pm2_env.pm_id + ')'); try { exec('pm2 delete ' + p.pm2_env.pm_id); } catch(e) { console.error('Failed to delete ' + p.pm2_env.pm_id); } } }); } catch (e) { console.error('Error cleaning up processes:', e); }"
|
||||
# === PRE-CLEANUP PM2 STATE LOGGING ===
|
||||
echo "=== PRE-CLEANUP PM2 STATE ==="
|
||||
pm2 jlist --namespace flyer-crawler-prod
|
||||
echo "=== END PRE-CLEANUP STATE ==="
|
||||
|
||||
# --- Cleanup Errored Processes with Defense-in-Depth Safeguards ---
|
||||
echo "Cleaning up errored or stopped PRODUCTION PM2 processes..."
|
||||
node -e "
|
||||
const exec = require('child_process').execSync;
|
||||
try {
|
||||
const list = JSON.parse(exec('pm2 jlist --namespace flyer-crawler-prod').toString());
|
||||
const prodProcesses = ['flyer-crawler-api', 'flyer-crawler-worker', 'flyer-crawler-analytics-worker'];
|
||||
|
||||
// Filter for processes that match our criteria
|
||||
const targetProcesses = list.filter(p =>
|
||||
(p.pm2_env.status === 'errored' || p.pm2_env.status === 'stopped') &&
|
||||
prodProcesses.includes(p.name)
|
||||
);
|
||||
|
||||
// SAFEGUARD 1: Process count validation
|
||||
const totalProcesses = list.length;
|
||||
if (targetProcesses.length === totalProcesses && totalProcesses > 3) {
|
||||
console.error('SAFETY ABORT: Filter would delete ALL processes!');
|
||||
console.error('Total processes: ' + totalProcesses + ', Target processes: ' + targetProcesses.length);
|
||||
console.error('This indicates a potential filter bug. Aborting cleanup.');
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
// SAFEGUARD 2: Explicit name verification
|
||||
console.log('Found ' + targetProcesses.length + ' PRODUCTION processes to clean:');
|
||||
targetProcesses.forEach(p => {
|
||||
console.log(' - ' + p.name + ' (status: ' + p.pm2_env.status + ', pm_id: ' + p.pm2_env.pm_id + ')');
|
||||
});
|
||||
|
||||
// Perform the cleanup
|
||||
targetProcesses.forEach(p => {
|
||||
console.log('Deleting ' + p.pm2_env.status + ' production process: ' + p.name + ' (' + p.pm2_env.pm_id + ')');
|
||||
try {
|
||||
exec('pm2 delete ' + p.pm2_env.pm_id + ' --namespace flyer-crawler-prod');
|
||||
} catch(e) {
|
||||
console.error('Failed to delete ' + p.pm2_env.pm_id);
|
||||
}
|
||||
});
|
||||
|
||||
console.log('Production process cleanup complete.');
|
||||
} catch (e) {
|
||||
console.error('Error cleaning up processes:', e);
|
||||
}
|
||||
"
|
||||
|
||||
# Save PM2 process list after cleanup to persist deletions
|
||||
echo "Saving PM2 process list after cleanup..."
|
||||
pm2 save --namespace flyer-crawler-prod
|
||||
|
||||
# === POST-CLEANUP VERIFICATION ===
|
||||
echo "=== POST-CLEANUP VERIFICATION ==="
|
||||
pm2 jlist --namespace flyer-crawler-prod | node -e "
|
||||
try {
|
||||
const list = JSON.parse(require('fs').readFileSync(0, 'utf-8'));
|
||||
const prodProcesses = list.filter(p => p.name && p.name.startsWith('flyer-crawler-') && !p.name.endsWith('-test') && !p.name.endsWith('-dev'));
|
||||
console.log('Production processes after cleanup:');
|
||||
prodProcesses.forEach(p => {
|
||||
console.log(' ' + p.name + ': ' + p.pm2_env.status);
|
||||
});
|
||||
if (prodProcesses.length === 0) {
|
||||
console.log(' (no production processes currently running)');
|
||||
}
|
||||
} catch (e) {
|
||||
console.error('Failed to parse PM2 output:', e.message);
|
||||
}
|
||||
"
|
||||
echo "=== END POST-CLEANUP VERIFICATION ==="
|
||||
|
||||
# --- Version Check Logic ---
|
||||
# Get the version from the newly deployed package.json
|
||||
@@ -176,7 +331,7 @@ jobs:
|
||||
|
||||
# Get the running version from PM2 for the main API process
|
||||
# We use a small node script to parse the JSON output from pm2 jlist
|
||||
RUNNING_VERSION=$(pm2 jlist | node -e "try { const list = JSON.parse(require('fs').readFileSync(0, 'utf-8')); const app = list.find(p => p.name === 'flyer-crawler-api'); console.log(app ? app.pm2_env.version : ''); } catch(e) { console.log(''); }")
|
||||
RUNNING_VERSION=$(pm2 jlist --namespace flyer-crawler-prod | node -e "try { const list = JSON.parse(require('fs').readFileSync(0, 'utf-8')); const app = list.find(p => p.name === 'flyer-crawler-api'); console.log(app ? app.pm2_env.version : ''); } catch(e) { console.log(''); }")
|
||||
echo "Running PM2 Version: $RUNNING_VERSION"
|
||||
|
||||
if [ "${{ gitea.event.inputs.force_reload }}" == "true" ] || [ "$NEW_VERSION" != "$RUNNING_VERSION" ] || [ -z "$RUNNING_VERSION" ]; then
|
||||
@@ -185,7 +340,7 @@ jobs:
|
||||
else
|
||||
echo "Version mismatch (Running: $RUNNING_VERSION -> Deployed: $NEW_VERSION) or app not running. Reloading PM2..."
|
||||
fi
|
||||
pm2 startOrReload ecosystem.config.cjs --update-env && pm2 save
|
||||
pm2 startOrReload ecosystem.config.cjs --update-env --namespace flyer-crawler-prod && pm2 save --namespace flyer-crawler-prod
|
||||
echo "Production backend server reloaded successfully."
|
||||
else
|
||||
echo "Version $NEW_VERSION is already running. Skipping PM2 reload."
|
||||
@@ -215,14 +370,14 @@ jobs:
|
||||
sleep 5 # Wait a few seconds for the app to start and log its output.
|
||||
|
||||
# Resolve the PM2 ID dynamically to ensure we target the correct process
|
||||
PM2_ID=$(pm2 jlist | node -e "try { const list = JSON.parse(require('fs').readFileSync(0, 'utf-8')); const app = list.find(p => p.name === 'flyer-crawler-api'); console.log(app ? app.pm2_env.pm_id : ''); } catch(e) { console.log(''); }")
|
||||
PM2_ID=$(pm2 jlist --namespace flyer-crawler-prod | node -e "try { const list = JSON.parse(require('fs').readFileSync(0, 'utf-8')); const app = list.find(p => p.name === 'flyer-crawler-api'); console.log(app ? app.pm2_env.pm_id : ''); } catch(e) { console.log(''); }")
|
||||
|
||||
if [ -n "$PM2_ID" ]; then
|
||||
echo "Found process ID: $PM2_ID"
|
||||
pm2 describe "$PM2_ID" || echo "Failed to describe process $PM2_ID"
|
||||
pm2 logs "$PM2_ID" --lines 20 --nostream || echo "Failed to get logs for $PM2_ID"
|
||||
pm2 env "$PM2_ID" || echo "Failed to get env for $PM2_ID"
|
||||
pm2 describe "$PM2_ID" --namespace flyer-crawler-prod || echo "Failed to describe process $PM2_ID"
|
||||
pm2 logs "$PM2_ID" --lines 20 --nostream --namespace flyer-crawler-prod || echo "Failed to get logs for $PM2_ID"
|
||||
pm2 env "$PM2_ID" --namespace flyer-crawler-prod || echo "Failed to get env for $PM2_ID"
|
||||
else
|
||||
echo "Could not find process 'flyer-crawler-api' in pm2 list."
|
||||
pm2 list # Fallback to listing everything to help debug
|
||||
pm2 list --namespace flyer-crawler-prod # Fallback to listing everything to help debug
|
||||
fi
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -56,9 +56,10 @@ jobs:
|
||||
|
||||
- name: Step 1 - Stop Application Server
|
||||
run: |
|
||||
echo "Stopping all PM2 processes to release database connections..."
|
||||
pm2 stop all || echo "PM2 processes were not running."
|
||||
echo "✅ Application server stopped."
|
||||
echo "Stopping PRODUCTION PM2 processes to release database connections..."
|
||||
pm2 stop flyer-crawler-api flyer-crawler-worker flyer-crawler-analytics-worker --namespace flyer-crawler-prod || echo "Production PM2 processes were not running."
|
||||
pm2 save --namespace flyer-crawler-prod
|
||||
echo "✅ Production application server stopped and saved."
|
||||
|
||||
- name: Step 2 - Drop and Recreate Database
|
||||
run: |
|
||||
@@ -91,5 +92,5 @@ jobs:
|
||||
run: |
|
||||
echo "Restarting application server..."
|
||||
cd /var/www/flyer-crawler.projectium.com
|
||||
pm2 startOrReload ecosystem.config.cjs --env production && pm2 save
|
||||
pm2 startOrReload ecosystem.config.cjs --env production --namespace flyer-crawler-prod && pm2 save --namespace flyer-crawler-prod
|
||||
echo "✅ Application server restarted."
|
||||
|
||||
@@ -85,6 +85,12 @@ jobs:
|
||||
echo "✅ Schema is up to date. No changes detected."
|
||||
fi
|
||||
|
||||
- name: Generate TSOA OpenAPI Spec and Routes
|
||||
run: |
|
||||
echo "Generating TSOA OpenAPI specification and route handlers..."
|
||||
npm run tsoa:build
|
||||
echo "✅ TSOA files generated successfully"
|
||||
|
||||
- name: Build React Application for Production
|
||||
run: |
|
||||
if [ -z "${{ secrets.VITE_GOOGLE_GENAI_API_KEY }}" ]; then
|
||||
@@ -109,6 +115,17 @@ jobs:
|
||||
rsync -avz dist/ "$APP_PATH"
|
||||
echo "Application deployment complete."
|
||||
|
||||
- name: Log Workflow Metadata
|
||||
run: |
|
||||
echo "=== WORKFLOW METADATA ==="
|
||||
echo "Workflow file: manual-deploy-major.yml"
|
||||
echo "Workflow file hash: $(sha256sum .gitea/workflows/manual-deploy-major.yml | cut -d' ' -f1)"
|
||||
echo "Git commit: $(git rev-parse HEAD)"
|
||||
echo "Git branch: $(git rev-parse --abbrev-ref HEAD)"
|
||||
echo "Timestamp: $(date -u '+%Y-%m-%d %H:%M:%S UTC')"
|
||||
echo "Actor: ${{ gitea.actor }}"
|
||||
echo "=== END METADATA ==="
|
||||
|
||||
- name: Install Backend Dependencies and Restart Production Server
|
||||
env:
|
||||
# --- Production Secrets Injection ---
|
||||
@@ -138,9 +155,78 @@ jobs:
|
||||
cd /var/www/flyer-crawler.projectium.com
|
||||
npm install --omit=dev
|
||||
|
||||
# --- Cleanup Errored Processes ---
|
||||
echo "Cleaning up errored or stopped PM2 processes..."
|
||||
node -e "const exec = require('child_process').execSync; try { const list = JSON.parse(exec('pm2 jlist').toString()); list.forEach(p => { if (p.pm2_env.status === 'errored' || p.pm2_env.status === 'stopped') { console.log('Deleting ' + p.pm2_env.status + ' process: ' + p.name + ' (' + p.pm2_env.pm_id + ')'); try { exec('pm2 delete ' + p.pm2_env.pm_id); } catch(e) { console.error('Failed to delete ' + p.pm2_env.pm_id); } } }); } catch (e) { console.error('Error cleaning up processes:', e); }"
|
||||
# === PRE-CLEANUP PM2 STATE LOGGING ===
|
||||
echo "=== PRE-CLEANUP PM2 STATE ==="
|
||||
pm2 jlist --namespace flyer-crawler-prod
|
||||
echo "=== END PRE-CLEANUP STATE ==="
|
||||
|
||||
# --- Cleanup Errored Processes with Defense-in-Depth Safeguards ---
|
||||
echo "Cleaning up errored or stopped PRODUCTION PM2 processes..."
|
||||
node -e "
|
||||
const exec = require('child_process').execSync;
|
||||
try {
|
||||
const list = JSON.parse(exec('pm2 jlist --namespace flyer-crawler-prod').toString());
|
||||
const prodProcesses = ['flyer-crawler-api', 'flyer-crawler-worker', 'flyer-crawler-analytics-worker'];
|
||||
|
||||
// Filter for processes that match our criteria
|
||||
const targetProcesses = list.filter(p =>
|
||||
(p.pm2_env.status === 'errored' || p.pm2_env.status === 'stopped') &&
|
||||
prodProcesses.includes(p.name)
|
||||
);
|
||||
|
||||
// SAFEGUARD 1: Process count validation
|
||||
const totalProcesses = list.length;
|
||||
if (targetProcesses.length === totalProcesses && totalProcesses > 3) {
|
||||
console.error('SAFETY ABORT: Filter would delete ALL processes!');
|
||||
console.error('Total processes: ' + totalProcesses + ', Target processes: ' + targetProcesses.length);
|
||||
console.error('This indicates a potential filter bug. Aborting cleanup.');
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
// SAFEGUARD 2: Explicit name verification
|
||||
console.log('Found ' + targetProcesses.length + ' PRODUCTION processes to clean:');
|
||||
targetProcesses.forEach(p => {
|
||||
console.log(' - ' + p.name + ' (status: ' + p.pm2_env.status + ', pm_id: ' + p.pm2_env.pm_id + ')');
|
||||
});
|
||||
|
||||
// Perform the cleanup
|
||||
targetProcesses.forEach(p => {
|
||||
console.log('Deleting ' + p.pm2_env.status + ' production process: ' + p.name + ' (' + p.pm2_env.pm_id + ')');
|
||||
try {
|
||||
exec('pm2 delete ' + p.pm2_env.pm_id + ' --namespace flyer-crawler-prod');
|
||||
} catch(e) {
|
||||
console.error('Failed to delete ' + p.pm2_env.pm_id);
|
||||
}
|
||||
});
|
||||
|
||||
console.log('Production process cleanup complete.');
|
||||
} catch (e) {
|
||||
console.error('Error cleaning up processes:', e);
|
||||
}
|
||||
"
|
||||
|
||||
# Save PM2 process list after cleanup to persist deletions
|
||||
echo "Saving PM2 process list after cleanup..."
|
||||
pm2 save --namespace flyer-crawler-prod
|
||||
|
||||
# === POST-CLEANUP VERIFICATION ===
|
||||
echo "=== POST-CLEANUP VERIFICATION ==="
|
||||
pm2 jlist --namespace flyer-crawler-prod | node -e "
|
||||
try {
|
||||
const list = JSON.parse(require('fs').readFileSync(0, 'utf-8'));
|
||||
const prodProcesses = list.filter(p => p.name && p.name.startsWith('flyer-crawler-') && !p.name.endsWith('-test') && !p.name.endsWith('-dev'));
|
||||
console.log('Production processes after cleanup:');
|
||||
prodProcesses.forEach(p => {
|
||||
console.log(' ' + p.name + ': ' + p.pm2_env.status);
|
||||
});
|
||||
if (prodProcesses.length === 0) {
|
||||
console.log(' (no production processes currently running)');
|
||||
}
|
||||
} catch (e) {
|
||||
console.error('Failed to parse PM2 output:', e.message);
|
||||
}
|
||||
"
|
||||
echo "=== END POST-CLEANUP VERIFICATION ==="
|
||||
|
||||
# --- Version Check Logic ---
|
||||
# Get the version from the newly deployed package.json
|
||||
@@ -149,7 +235,7 @@ jobs:
|
||||
|
||||
# Get the running version from PM2 for the main API process
|
||||
# We use a small node script to parse the JSON output from pm2 jlist
|
||||
RUNNING_VERSION=$(pm2 jlist | node -e "try { const list = JSON.parse(require('fs').readFileSync(0, 'utf-8')); const app = list.find(p => p.name === 'flyer-crawler-api'); console.log(app ? app.pm2_env.version : ''); } catch(e) { console.log(''); }")
|
||||
RUNNING_VERSION=$(pm2 jlist --namespace flyer-crawler-prod | node -e "try { const list = JSON.parse(require('fs').readFileSync(0, 'utf-8')); const app = list.find(p => p.name === 'flyer-crawler-api'); console.log(app ? app.pm2_env.version : ''); } catch(e) { console.log(''); }")
|
||||
echo "Running PM2 Version: $RUNNING_VERSION"
|
||||
|
||||
if [ "${{ gitea.event.inputs.force_reload }}" == "true" ] || [ "$NEW_VERSION" != "$RUNNING_VERSION" ] || [ -z "$RUNNING_VERSION" ]; then
|
||||
@@ -158,7 +244,7 @@ jobs:
|
||||
else
|
||||
echo "Version mismatch (Running: $RUNNING_VERSION -> Deployed: $NEW_VERSION) or app not running. Reloading PM2..."
|
||||
fi
|
||||
pm2 startOrReload ecosystem.config.cjs --env production --update-env && pm2 save
|
||||
pm2 startOrReload ecosystem.config.cjs --env production --update-env --namespace flyer-crawler-prod && pm2 save --namespace flyer-crawler-prod
|
||||
echo "Production backend server reloaded successfully."
|
||||
else
|
||||
echo "Version $NEW_VERSION is already running. Skipping PM2 reload."
|
||||
@@ -181,6 +267,6 @@ jobs:
|
||||
run: |
|
||||
echo "--- Displaying recent PM2 logs for flyer-crawler-api ---"
|
||||
sleep 5
|
||||
pm2 describe flyer-crawler-api || echo "Could not find production pm2 process."
|
||||
pm2 logs flyer-crawler-api --lines 20 --nostream || echo "Could not find production pm2 process."
|
||||
pm2 env flyer-crawler-api || echo "Could not find production pm2 process."
|
||||
pm2 describe flyer-crawler-api --namespace flyer-crawler-prod || echo "Could not find production pm2 process."
|
||||
pm2 logs flyer-crawler-api --lines 20 --nostream --namespace flyer-crawler-prod || echo "Could not find production pm2 process."
|
||||
pm2 env flyer-crawler-api --namespace flyer-crawler-prod || echo "Could not find production pm2 process."
|
||||
258
.gitea/workflows/pm2-diagnostics.yml
Normal file
258
.gitea/workflows/pm2-diagnostics.yml
Normal file
@@ -0,0 +1,258 @@
|
||||
# .gitea/workflows/pm2-diagnostics.yml
|
||||
#
|
||||
# Comprehensive PM2 diagnostics to identify crash causes and problematic projects
|
||||
name: PM2 Diagnostics
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
capture_interval:
|
||||
description: 'Seconds between PM2 state captures (default: 5)'
|
||||
required: false
|
||||
default: '5'
|
||||
duration:
|
||||
description: 'Total monitoring duration in seconds (default: 60)'
|
||||
required: false
|
||||
default: '60'
|
||||
|
||||
jobs:
|
||||
pm2-diagnostics:
|
||||
runs-on: projectium.com
|
||||
|
||||
steps:
|
||||
- name: PM2 Current State Snapshot
|
||||
run: |
|
||||
echo "========================================="
|
||||
echo "PM2 CURRENT STATE SNAPSHOT"
|
||||
echo "========================================="
|
||||
echo ""
|
||||
echo "=== Production Namespace (flyer-crawler-prod) ==="
|
||||
echo "--- PM2 List (Human Readable) ---"
|
||||
pm2 list --namespace flyer-crawler-prod
|
||||
echo ""
|
||||
echo "--- PM2 List (JSON) ---"
|
||||
pm2 jlist --namespace flyer-crawler-prod > /tmp/pm2-state-initial-prod.json
|
||||
cat /tmp/pm2-state-initial-prod.json | jq '.'
|
||||
echo ""
|
||||
echo "=== Test Namespace (flyer-crawler-test) ==="
|
||||
echo "--- PM2 List (Human Readable) ---"
|
||||
pm2 list --namespace flyer-crawler-test
|
||||
echo ""
|
||||
echo "--- PM2 List (JSON) ---"
|
||||
pm2 jlist --namespace flyer-crawler-test > /tmp/pm2-state-initial-test.json
|
||||
cat /tmp/pm2-state-initial-test.json | jq '.'
|
||||
echo ""
|
||||
echo "=== All Namespaces Combined ==="
|
||||
echo "--- PM2 List (All) ---"
|
||||
pm2 list
|
||||
echo ""
|
||||
echo "--- PM2 Daemon Info ---"
|
||||
pm2 info pm2-logrotate || echo "pm2-logrotate not found"
|
||||
echo ""
|
||||
echo "--- PM2 Version ---"
|
||||
pm2 --version
|
||||
echo ""
|
||||
echo "--- Node Version ---"
|
||||
node --version
|
||||
|
||||
- name: PM2 Process Working Directories
|
||||
run: |
|
||||
echo "========================================="
|
||||
echo "PROCESS WORKING DIRECTORIES"
|
||||
echo "========================================="
|
||||
echo ""
|
||||
echo "=== Production Namespace (flyer-crawler-prod) ==="
|
||||
pm2 jlist --namespace flyer-crawler-prod | jq -r '.[] | "Process: \(.name) | CWD: \(.pm2_env.pm_cwd) | Exists: \(if .pm2_env.pm_cwd then "checking..." else "N/A" end)"'
|
||||
echo ""
|
||||
echo "--- Checking if CWDs still exist ---"
|
||||
pm2 jlist --namespace flyer-crawler-prod | jq -r '.[].pm2_env.pm_cwd' | while read cwd; do
|
||||
if [ -n "$cwd" ] && [ "$cwd" != "null" ]; then
|
||||
if [ -d "$cwd" ]; then
|
||||
echo "✅ EXISTS: $cwd"
|
||||
else
|
||||
echo "❌ MISSING: $cwd (THIS WILL CAUSE CRASHES!)"
|
||||
fi
|
||||
fi
|
||||
done
|
||||
echo ""
|
||||
echo "=== Test Namespace (flyer-crawler-test) ==="
|
||||
pm2 jlist --namespace flyer-crawler-test | jq -r '.[] | "Process: \(.name) | CWD: \(.pm2_env.pm_cwd) | Exists: \(if .pm2_env.pm_cwd then "checking..." else "N/A" end)"'
|
||||
echo ""
|
||||
echo "--- Checking if CWDs still exist ---"
|
||||
pm2 jlist --namespace flyer-crawler-test | jq -r '.[].pm2_env.pm_cwd' | while read cwd; do
|
||||
if [ -n "$cwd" ] && [ "$cwd" != "null" ]; then
|
||||
if [ -d "$cwd" ]; then
|
||||
echo "✅ EXISTS: $cwd"
|
||||
else
|
||||
echo "❌ MISSING: $cwd (THIS WILL CAUSE CRASHES!)"
|
||||
fi
|
||||
fi
|
||||
done
|
||||
|
||||
- name: PM2 Log Analysis
|
||||
run: |
|
||||
echo "========================================="
|
||||
echo "PM2 LOG ANALYSIS"
|
||||
echo "========================================="
|
||||
echo ""
|
||||
echo "--- PM2 Daemon Log (Last 100 Lines) ---"
|
||||
tail -100 /home/gitea-runner/.pm2/pm2.log
|
||||
echo ""
|
||||
echo "--- Searching for ENOENT errors ---"
|
||||
grep -i "ENOENT\|no such file or directory\|uv_cwd" /home/gitea-runner/.pm2/pm2.log || echo "No ENOENT errors found"
|
||||
echo ""
|
||||
echo "--- Searching for crash patterns ---"
|
||||
grep -i "crash\|error\|exception" /home/gitea-runner/.pm2/pm2.log | tail -50 || echo "No crashes found"
|
||||
|
||||
- name: Identify All PM2-Managed Projects
|
||||
run: |
|
||||
echo "========================================="
|
||||
echo "ALL PM2-MANAGED PROJECTS"
|
||||
echo "========================================="
|
||||
echo ""
|
||||
echo "=== Production Namespace (flyer-crawler-prod) ==="
|
||||
pm2 jlist --namespace flyer-crawler-prod | jq -r '.[] | "[\(.pm_id)] \(.name) - v\(.pm2_env.version // "N/A") - \(.pm2_env.status) - CWD: \(.pm2_env.pm_cwd)"'
|
||||
echo ""
|
||||
echo "--- Projects by CWD ---"
|
||||
pm2 jlist --namespace flyer-crawler-prod | jq -r '.[].pm2_env.pm_cwd' | sort -u
|
||||
echo ""
|
||||
echo "=== Test Namespace (flyer-crawler-test) ==="
|
||||
pm2 jlist --namespace flyer-crawler-test | jq -r '.[] | "[\(.pm_id)] \(.name) - v\(.pm2_env.version // "N/A") - \(.pm2_env.status) - CWD: \(.pm2_env.pm_cwd)"'
|
||||
echo ""
|
||||
echo "--- Projects by CWD ---"
|
||||
pm2 jlist --namespace flyer-crawler-test | jq -r '.[].pm2_env.pm_cwd' | sort -u
|
||||
echo ""
|
||||
echo "=== All Namespaces (for reference) ==="
|
||||
pm2 jlist | jq -r '.[] | "[\(.pm_id)] \(.name) [ns: \(.pm2_env.namespace // "default")] - \(.pm2_env.status)"'
|
||||
echo ""
|
||||
echo "--- Checking which projects might interfere ---"
|
||||
for dir in /var/www/*; do
|
||||
if [ -d "$dir" ]; then
|
||||
echo ""
|
||||
echo "Directory: $dir"
|
||||
ls -la "$dir" | grep -E "ecosystem|package.json|node_modules" || echo " No PM2/Node files"
|
||||
fi
|
||||
done
|
||||
|
||||
- name: Monitor PM2 State Over Time
|
||||
run: |
|
||||
echo "========================================="
|
||||
echo "PM2 STATE MONITORING"
|
||||
echo "========================================="
|
||||
echo "Monitoring PM2 for ${{ gitea.event.inputs.duration }} seconds..."
|
||||
echo "Capturing state every ${{ gitea.event.inputs.capture_interval }} seconds"
|
||||
echo ""
|
||||
|
||||
INTERVAL=${{ gitea.event.inputs.capture_interval }}
|
||||
DURATION=${{ gitea.event.inputs.duration }}
|
||||
COUNT=$((DURATION / INTERVAL))
|
||||
|
||||
for i in $(seq 1 $COUNT); do
|
||||
echo "--- Capture $i at $(date) ---"
|
||||
echo ""
|
||||
echo "=== Production Namespace (flyer-crawler-prod) ==="
|
||||
pm2 jlist --namespace flyer-crawler-prod | jq -r '.[] | "\(.name): \(.pm2_env.status) (restarts: \(.pm2_env.restart_time))"'
|
||||
|
||||
# Check for crashes in production
|
||||
CRASHED_PROD=$(pm2 jlist --namespace flyer-crawler-prod | jq '[.[] | select(.pm2_env.status == "errored" or .pm2_env.status == "stopped")] | length')
|
||||
if [ "$CRASHED_PROD" -gt 0 ]; then
|
||||
echo "⚠️ WARNING: $CRASHED_PROD PRODUCTION process(es) in crashed state!"
|
||||
pm2 jlist --namespace flyer-crawler-prod | jq -r '.[] | select(.pm2_env.status == "errored" or .pm2_env.status == "stopped") | " - \(.name): \(.pm2_env.status)"'
|
||||
fi
|
||||
|
||||
echo ""
|
||||
echo "=== Test Namespace (flyer-crawler-test) ==="
|
||||
pm2 jlist --namespace flyer-crawler-test | jq -r '.[] | "\(.name): \(.pm2_env.status) (restarts: \(.pm2_env.restart_time))"'
|
||||
|
||||
# Check for crashes in test
|
||||
CRASHED_TEST=$(pm2 jlist --namespace flyer-crawler-test | jq '[.[] | select(.pm2_env.status == "errored" or .pm2_env.status == "stopped")] | length')
|
||||
if [ "$CRASHED_TEST" -gt 0 ]; then
|
||||
echo "⚠️ WARNING: $CRASHED_TEST TEST process(es) in crashed state!"
|
||||
pm2 jlist --namespace flyer-crawler-test | jq -r '.[] | select(.pm2_env.status == "errored" or .pm2_env.status == "stopped") | " - \(.name): \(.pm2_env.status)"'
|
||||
fi
|
||||
|
||||
echo ""
|
||||
sleep $INTERVAL
|
||||
done
|
||||
|
||||
- name: PM2 Dump File Analysis
|
||||
run: |
|
||||
echo "========================================="
|
||||
echo "PM2 DUMP FILE ANALYSIS"
|
||||
echo "========================================="
|
||||
echo "--- Dump file location ---"
|
||||
ls -lh /home/gitea-runner/.pm2/dump.pm2
|
||||
echo ""
|
||||
echo "--- Dump file contents ---"
|
||||
cat /home/gitea-runner/.pm2/dump.pm2 | jq '.'
|
||||
echo ""
|
||||
echo "--- Processes in dump ---"
|
||||
cat /home/gitea-runner/.pm2/dump.pm2 | jq -r '.apps[] | "\(.name) at \(.pm_cwd)"'
|
||||
|
||||
- name: Check for Rogue Deployment Scripts
|
||||
run: |
|
||||
echo "========================================="
|
||||
echo "DEPLOYMENT SCRIPT ANALYSIS"
|
||||
echo "========================================="
|
||||
echo "Checking for scripts that might delete directories..."
|
||||
echo ""
|
||||
for project in flyer-crawler stock-alert; do
|
||||
for env in "" "-test"; do
|
||||
DIR="/var/www/$project$env.projectium.com"
|
||||
if [ -d "$DIR" ]; then
|
||||
echo "--- Project: $project$env ---"
|
||||
echo "Location: $DIR"
|
||||
if [ -f "$DIR/.gitea/workflows/deploy-to-test.yml" ]; then
|
||||
echo "Has deploy-to-test workflow"
|
||||
grep -n "rsync.*--delete\|rm -rf" "$DIR/.gitea/workflows/deploy-to-test.yml" | head -5 || echo "No dangerous commands found"
|
||||
fi
|
||||
if [ -f "$DIR/.gitea/workflows/deploy-to-prod.yml" ]; then
|
||||
echo "Has deploy-to-prod workflow"
|
||||
grep -n "rsync.*--delete\|rm -rf" "$DIR/.gitea/workflows/deploy-to-prod.yml" | head -5 || echo "No dangerous commands found"
|
||||
fi
|
||||
echo ""
|
||||
fi
|
||||
done
|
||||
done
|
||||
|
||||
- name: Generate Diagnostic Report
|
||||
run: |
|
||||
echo "========================================="
|
||||
echo "DIAGNOSTIC SUMMARY"
|
||||
echo "========================================="
|
||||
echo ""
|
||||
echo "=== Production Namespace (flyer-crawler-prod) ==="
|
||||
echo "Total processes: $(pm2 jlist --namespace flyer-crawler-prod | jq 'length')"
|
||||
echo "Online: $(pm2 jlist --namespace flyer-crawler-prod | jq '[.[] | select(.pm2_env.status == "online")] | length')"
|
||||
echo "Stopped: $(pm2 jlist --namespace flyer-crawler-prod | jq '[.[] | select(.pm2_env.status == "stopped")] | length')"
|
||||
echo "Errored: $(pm2 jlist --namespace flyer-crawler-prod | jq '[.[] | select(.pm2_env.status == "errored")] | length')"
|
||||
echo ""
|
||||
echo "Flyer-crawler PROD processes:"
|
||||
pm2 jlist --namespace flyer-crawler-prod | jq -r '.[] | select(.name | contains("flyer-crawler")) | " \(.name): \(.pm2_env.status)"'
|
||||
echo ""
|
||||
echo "=== Test Namespace (flyer-crawler-test) ==="
|
||||
echo "Total processes: $(pm2 jlist --namespace flyer-crawler-test | jq 'length')"
|
||||
echo "Online: $(pm2 jlist --namespace flyer-crawler-test | jq '[.[] | select(.pm2_env.status == "online")] | length')"
|
||||
echo "Stopped: $(pm2 jlist --namespace flyer-crawler-test | jq '[.[] | select(.pm2_env.status == "stopped")] | length')"
|
||||
echo "Errored: $(pm2 jlist --namespace flyer-crawler-test | jq '[.[] | select(.pm2_env.status == "errored")] | length')"
|
||||
echo ""
|
||||
echo "Flyer-crawler TEST processes:"
|
||||
pm2 jlist --namespace flyer-crawler-test | jq -r '.[] | select(.name | contains("flyer-crawler")) | " \(.name): \(.pm2_env.status)"'
|
||||
echo ""
|
||||
echo "=== All Namespaces Summary ==="
|
||||
echo "Total PM2 processes (all): $(pm2 jlist | jq 'length')"
|
||||
echo ""
|
||||
echo "Stock-alert processes (separate project):"
|
||||
pm2 jlist | jq -r '.[] | select(.name | contains("stock-alert")) | " \(.name): \(.pm2_env.status) [ns: \(.pm2_env.namespace // "default")]"'
|
||||
echo ""
|
||||
echo "Other processes:"
|
||||
pm2 jlist | jq -r '.[] | select(.name | contains("flyer-crawler") | not) | select(.name | contains("stock-alert") | not) | " \(.name): \(.pm2_env.status) [ns: \(.pm2_env.namespace // "default")]"'
|
||||
echo ""
|
||||
echo "========================================="
|
||||
echo "RECOMMENDATIONS"
|
||||
echo "========================================="
|
||||
echo "1. Check for missing CWDs (marked with ❌ above)"
|
||||
echo "2. Review PM2 daemon log for ENOENT errors"
|
||||
echo "3. Verify no deployments are running rsync --delete while processes are online"
|
||||
echo "4. Use namespace-specific commands: pm2 list --namespace flyer-crawler-prod"
|
||||
echo "5. Avoid pm2 restart all - use namespace targeting instead"
|
||||
107
.gitea/workflows/restart-pm2.yml
Normal file
107
.gitea/workflows/restart-pm2.yml
Normal file
@@ -0,0 +1,107 @@
|
||||
# .gitea/workflows/restart-pm2.yml
|
||||
#
|
||||
# Manual workflow to restart PM2 processes and verify their status.
|
||||
# Useful for recovering from PM2 daemon crashes or process issues.
|
||||
name: Restart PM2 Processes
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
environment:
|
||||
description: 'Environment to restart (test, production, or both)'
|
||||
required: true
|
||||
default: 'test'
|
||||
type: choice
|
||||
options:
|
||||
- test
|
||||
- production
|
||||
- both
|
||||
|
||||
jobs:
|
||||
restart-pm2:
|
||||
runs-on: projectium.com
|
||||
|
||||
steps:
|
||||
- name: Validate Environment Input
|
||||
run: |
|
||||
echo "Restarting PM2 processes for environment: ${{ gitea.event.inputs.environment }}"
|
||||
|
||||
- name: Restart Test Environment
|
||||
if: gitea.event.inputs.environment == 'test' || gitea.event.inputs.environment == 'both'
|
||||
run: |
|
||||
echo "=== RESTARTING TEST ENVIRONMENT ==="
|
||||
cd /var/www/flyer-crawler-test.projectium.com
|
||||
|
||||
echo "--- Current PM2 State (Before Restart) ---"
|
||||
pm2 list --namespace flyer-crawler-test
|
||||
|
||||
echo "--- Restarting Test Processes ---"
|
||||
pm2 restart flyer-crawler-api-test flyer-crawler-worker-test flyer-crawler-analytics-worker-test --namespace flyer-crawler-test || {
|
||||
echo "Restart failed, attempting to start processes..."
|
||||
pm2 start ecosystem-test.config.cjs --namespace flyer-crawler-test
|
||||
}
|
||||
|
||||
echo "--- Saving PM2 Process List ---"
|
||||
pm2 save --namespace flyer-crawler-test
|
||||
|
||||
echo "--- Waiting 3 seconds for processes to stabilize ---"
|
||||
sleep 3
|
||||
|
||||
echo "=== TEST ENVIRONMENT STATUS ==="
|
||||
pm2 ps --namespace flyer-crawler-test
|
||||
|
||||
- name: Restart Production Environment
|
||||
if: gitea.event.inputs.environment == 'production' || gitea.event.inputs.environment == 'both'
|
||||
run: |
|
||||
echo "=== RESTARTING PRODUCTION ENVIRONMENT ==="
|
||||
cd /var/www/flyer-crawler.projectium.com
|
||||
|
||||
echo "--- Current PM2 State (Before Restart) ---"
|
||||
pm2 list --namespace flyer-crawler-prod
|
||||
|
||||
echo "--- Restarting Production Processes ---"
|
||||
pm2 restart flyer-crawler-api flyer-crawler-worker flyer-crawler-analytics-worker --namespace flyer-crawler-prod || {
|
||||
echo "Restart failed, attempting to start processes..."
|
||||
pm2 start ecosystem.config.cjs --namespace flyer-crawler-prod
|
||||
}
|
||||
|
||||
echo "--- Saving PM2 Process List ---"
|
||||
pm2 save --namespace flyer-crawler-prod
|
||||
|
||||
echo "--- Waiting 3 seconds for processes to stabilize ---"
|
||||
sleep 3
|
||||
|
||||
echo "=== PRODUCTION ENVIRONMENT STATUS ==="
|
||||
pm2 ps --namespace flyer-crawler-prod
|
||||
|
||||
- name: Final PM2 Status (All Processes)
|
||||
run: |
|
||||
echo "========================================="
|
||||
echo "FINAL PM2 STATUS - ALL PROCESSES"
|
||||
echo "========================================="
|
||||
|
||||
if [ "${{ gitea.event.inputs.environment }}" = "test" ]; then
|
||||
echo "--- Test Namespace ---"
|
||||
pm2 ps --namespace flyer-crawler-test
|
||||
echo ""
|
||||
echo "--- PM2 Logs (Last 20 Lines) ---"
|
||||
pm2 logs --namespace flyer-crawler-test --lines 20 --nostream || echo "No logs available"
|
||||
elif [ "${{ gitea.event.inputs.environment }}" = "production" ]; then
|
||||
echo "--- Production Namespace ---"
|
||||
pm2 ps --namespace flyer-crawler-prod
|
||||
echo ""
|
||||
echo "--- PM2 Logs (Last 20 Lines) ---"
|
||||
pm2 logs --namespace flyer-crawler-prod --lines 20 --nostream || echo "No logs available"
|
||||
else
|
||||
echo "--- Test Namespace ---"
|
||||
pm2 ps --namespace flyer-crawler-test
|
||||
echo ""
|
||||
echo "--- Production Namespace ---"
|
||||
pm2 ps --namespace flyer-crawler-prod
|
||||
echo ""
|
||||
echo "--- PM2 Logs - Test (Last 10 Lines) ---"
|
||||
pm2 logs --namespace flyer-crawler-test --lines 10 --nostream || echo "No logs available"
|
||||
echo ""
|
||||
echo "--- PM2 Logs - Production (Last 10 Lines) ---"
|
||||
pm2 logs --namespace flyer-crawler-prod --lines 10 --nostream || echo "No logs available"
|
||||
fi
|
||||
100
.gitea/workflows/sync-test-version.yml
Normal file
100
.gitea/workflows/sync-test-version.yml
Normal file
@@ -0,0 +1,100 @@
|
||||
# .gitea/workflows/sync-test-version.yml
|
||||
#
|
||||
# Lightweight workflow to sync version numbers from production to test environment.
|
||||
# This runs after successful production deployments to update test PM2 metadata
|
||||
# without re-running the full test suite, build, and deployment pipeline.
|
||||
#
|
||||
# Duration: ~30 seconds (vs 5+ minutes for full test deployment)
|
||||
name: Sync Test Version
|
||||
|
||||
on:
|
||||
workflow_run:
|
||||
workflows: ["Deploy to Production"]
|
||||
types:
|
||||
- completed
|
||||
branches:
|
||||
- main
|
||||
|
||||
jobs:
|
||||
sync-version:
|
||||
runs-on: projectium.com
|
||||
# Only run if the production deployment succeeded
|
||||
if: ${{ gitea.event.workflow_run.conclusion == 'success' }}
|
||||
|
||||
steps:
|
||||
- name: Checkout Latest Code
|
||||
uses: actions/checkout@v3
|
||||
with:
|
||||
fetch-depth: 1 # Shallow clone, we only need latest commit
|
||||
|
||||
- name: Update Test Package Version
|
||||
run: |
|
||||
echo "========================================="
|
||||
echo "SYNCING VERSION TO TEST ENVIRONMENT"
|
||||
echo "========================================="
|
||||
|
||||
APP_PATH="/var/www/flyer-crawler-test.projectium.com"
|
||||
|
||||
# Get version from this repo's package.json
|
||||
NEW_VERSION=$(node -p "require('./package.json').version")
|
||||
echo "Production version: $NEW_VERSION"
|
||||
|
||||
# Get current test version
|
||||
if [ -f "$APP_PATH/package.json" ]; then
|
||||
CURRENT_VERSION=$(node -p "require('$APP_PATH/package.json').version")
|
||||
echo "Current test version: $CURRENT_VERSION"
|
||||
else
|
||||
CURRENT_VERSION="unknown"
|
||||
echo "Test package.json not found"
|
||||
fi
|
||||
|
||||
# Only update if versions differ
|
||||
if [ "$NEW_VERSION" != "$CURRENT_VERSION" ]; then
|
||||
echo "Updating test package.json to version $NEW_VERSION..."
|
||||
|
||||
# Update just the version field in test's package.json
|
||||
cd "$APP_PATH"
|
||||
npm version "$NEW_VERSION" --no-git-tag-version --allow-same-version
|
||||
|
||||
echo "✅ Test package.json updated to $NEW_VERSION"
|
||||
else
|
||||
echo "ℹ️ Versions already match, no update needed"
|
||||
fi
|
||||
|
||||
- name: Restart Test PM2 Processes
|
||||
run: |
|
||||
echo "Restarting test PM2 processes to refresh version metadata..."
|
||||
|
||||
# Restart with --update-env to pick up new package.json version
|
||||
pm2 --namespace flyer-crawler-test restart flyer-crawler-api-test flyer-crawler-worker-test flyer-crawler-analytics-worker-test --update-env && pm2 --namespace flyer-crawler-test save
|
||||
|
||||
echo "✅ Test PM2 processes restarted and saved"
|
||||
|
||||
# Show current state
|
||||
echo ""
|
||||
echo "--- Current PM2 State ---"
|
||||
pm2 --namespace flyer-crawler-test list
|
||||
|
||||
# Verify version in PM2 metadata
|
||||
echo ""
|
||||
echo "--- Verifying Version in PM2 ---"
|
||||
pm2 --namespace flyer-crawler-test jlist | node -e "
|
||||
try {
|
||||
const list = JSON.parse(require('fs').readFileSync(0, 'utf-8'));
|
||||
const testProcesses = list.filter(p => p.name && p.name.endsWith('-test'));
|
||||
testProcesses.forEach(p => {
|
||||
console.log(p.name + ': v' + (p.pm2_env.version || 'unknown') + ' (' + p.pm2_env.status + ')');
|
||||
});
|
||||
} catch(e) {
|
||||
console.error('Failed to parse PM2 output');
|
||||
}
|
||||
"
|
||||
|
||||
- name: Summary
|
||||
run: |
|
||||
echo "========================================="
|
||||
echo "VERSION SYNC COMPLETE"
|
||||
echo "========================================="
|
||||
echo "Test environment version updated to match production"
|
||||
echo "No tests run, no builds performed"
|
||||
echo "Duration: ~30 seconds"
|
||||
10
.gitignore
vendored
10
.gitignore
vendored
@@ -14,6 +14,10 @@ dist-ssr
|
||||
.env
|
||||
*.tsbuildinfo
|
||||
|
||||
# tsoa generated files (regenerated on build)
|
||||
src/routes/tsoa-generated.ts
|
||||
src/config/tsoa-spec.json
|
||||
|
||||
# Test coverage
|
||||
coverage
|
||||
.nyc_output
|
||||
@@ -35,6 +39,10 @@ test-output.txt
|
||||
*.sln
|
||||
*.sw?
|
||||
Thumbs.db
|
||||
.claude
|
||||
.claude/settings.local.json
|
||||
nul
|
||||
tmpclaude*
|
||||
|
||||
|
||||
|
||||
test.tmp
|
||||
328
CLAUDE.md
328
CLAUDE.md
@@ -27,6 +27,147 @@ podman exec -it flyer-crawler-dev npm run type-check
|
||||
|
||||
Out-of-sync = test failures.
|
||||
|
||||
### Server Access: READ-ONLY (Production/Test Servers)
|
||||
|
||||
**CRITICAL**: The `claude-win10` user has **READ-ONLY** access to production and test servers.
|
||||
|
||||
| Capability | Status |
|
||||
| ---------------------- | ---------------------- |
|
||||
| Root/sudo access | NO |
|
||||
| Write permissions | NO |
|
||||
| PM2 restart, systemctl | NO - User must execute |
|
||||
|
||||
**Server Operations Workflow**: Diagnose → User executes → Analyze → Fix (1-3 commands) → User executes → Verify
|
||||
|
||||
**Rules**:
|
||||
|
||||
- Provide diagnostic commands first, wait for user to report results
|
||||
- Maximum 3 fix commands at a time (errors may cascade)
|
||||
- Always verify after fixes complete
|
||||
|
||||
### PM2 Namespace Isolation (Production/Test Servers)
|
||||
|
||||
**CRITICAL**: Production and test environments share the same PM2 daemon on the server.
|
||||
|
||||
Flyer-crawler uses PM2 namespaces to isolate test and production processes:
|
||||
|
||||
| Namespace | Purpose | Config File |
|
||||
| --------------------- | ------------------------- | --------------------------- |
|
||||
| `flyer-crawler-prod` | Production environment | `ecosystem.config.cjs` |
|
||||
| `flyer-crawler-test` | Test environment | `ecosystem-test.config.cjs` |
|
||||
| `flyer-crawler-dev` | Development container | `ecosystem.dev.config.cjs` |
|
||||
|
||||
This prevents `pm2 save` race conditions during simultaneous deployments. See [ADR-063](docs/adr/0063-pm2-namespace-implementation.md) for details.
|
||||
|
||||
**See also**: [PM2 Process Isolation Incidents](#pm2-process-isolation-incidents) for past incidents and response procedures.
|
||||
|
||||
| Environment | Processes | Namespace |
|
||||
| ----------- | -------------------------------------------------------------------------------------------- | --------------------- |
|
||||
| Production | `flyer-crawler-api`, `flyer-crawler-worker`, `flyer-crawler-analytics-worker` | `flyer-crawler-prod` |
|
||||
| Test | `flyer-crawler-api-test`, `flyer-crawler-worker-test`, `flyer-crawler-analytics-worker-test` | `flyer-crawler-test` |
|
||||
| Development | `flyer-crawler-api-dev`, `flyer-crawler-worker-dev`, `flyer-crawler-vite-dev` | `flyer-crawler-dev` |
|
||||
|
||||
**Deployment Scripts MUST:**
|
||||
|
||||
- ✅ Use `--namespace` flag for all PM2 commands to scope to correct environment
|
||||
- ✅ Filter PM2 commands by exact process names or name patterns (e.g., `endsWith('-test')`)
|
||||
- ❌ NEVER use `pm2 stop all`, `pm2 delete all`, or `pm2 restart all` without namespace
|
||||
- ❌ NEVER delete/stop processes based solely on status without name filtering
|
||||
- ✅ Always verify process names match the target environment before any operation
|
||||
|
||||
**Examples:**
|
||||
|
||||
```bash
|
||||
# ✅ CORRECT - Production commands with namespace
|
||||
pm2 start ecosystem.config.cjs --namespace flyer-crawler-prod
|
||||
pm2 stop flyer-crawler-api flyer-crawler-worker --namespace flyer-crawler-prod
|
||||
pm2 restart all --namespace flyer-crawler-prod && pm2 save --namespace flyer-crawler-prod
|
||||
pm2 logs --namespace flyer-crawler-prod
|
||||
|
||||
# ✅ CORRECT - Test commands with namespace
|
||||
pm2 start ecosystem-test.config.cjs --namespace flyer-crawler-test
|
||||
pm2 status --namespace flyer-crawler-test
|
||||
pm2 delete all --namespace flyer-crawler-test && pm2 save --namespace flyer-crawler-test
|
||||
|
||||
# ✅ CORRECT - Dev container commands with namespace
|
||||
pm2 start ecosystem.dev.config.cjs --namespace flyer-crawler-dev
|
||||
pm2 logs --namespace flyer-crawler-dev
|
||||
|
||||
# ✅ CORRECT - Test cleanup (filter by namespace + name pattern)
|
||||
# Only delete test processes that are errored/stopped
|
||||
list.forEach(p => {
|
||||
if ((p.pm2_env.status === 'errored' || p.pm2_env.status === 'stopped') &&
|
||||
p.name && p.name.endsWith('-test') &&
|
||||
p.pm2_env.namespace === 'flyer-crawler-test') {
|
||||
exec('pm2 delete ' + p.pm2_env.pm_id + ' --namespace flyer-crawler-test');
|
||||
}
|
||||
});
|
||||
exec('pm2 save --namespace flyer-crawler-test');
|
||||
|
||||
# ❌ WRONG - Missing namespace (affects all environments)
|
||||
pm2 stop all
|
||||
pm2 delete all
|
||||
pm2 restart all
|
||||
|
||||
# ❌ WRONG - No name/namespace filtering (could delete test processes during prod deploy)
|
||||
if (p.pm2_env.status === 'errored') {
|
||||
exec('pm2 delete ' + p.pm2_env.pm_id);
|
||||
}
|
||||
```
|
||||
|
||||
### PM2 Save Requirement (CRITICAL)
|
||||
|
||||
**CRITICAL**: Every `pm2 start`, `pm2 restart`, `pm2 stop`, or `pm2 delete` command MUST be immediately followed by `pm2 save` with the same namespace.
|
||||
|
||||
Without `pm2 save`, processes become ephemeral and will disappear on:
|
||||
|
||||
- PM2 daemon restarts
|
||||
- Server reboots
|
||||
- Internal PM2 reconciliation events
|
||||
|
||||
**Pattern:**
|
||||
|
||||
```bash
|
||||
# ✅ CORRECT - Save after every state change (with namespace)
|
||||
pm2 start ecosystem.config.cjs --namespace flyer-crawler-prod && pm2 save --namespace flyer-crawler-prod
|
||||
pm2 restart my-app --namespace flyer-crawler-prod && pm2 save --namespace flyer-crawler-prod
|
||||
pm2 stop my-app --namespace flyer-crawler-test && pm2 save --namespace flyer-crawler-test
|
||||
pm2 delete my-app --namespace flyer-crawler-test && pm2 save --namespace flyer-crawler-test
|
||||
|
||||
# ❌ WRONG - Missing save (processes become ephemeral)
|
||||
pm2 start ecosystem.config.cjs --namespace flyer-crawler-prod
|
||||
pm2 restart my-app --namespace flyer-crawler-prod
|
||||
|
||||
# ❌ WRONG - Missing namespace (affects wrong environment)
|
||||
pm2 start ecosystem.config.cjs && pm2 save
|
||||
```
|
||||
|
||||
**In Cleanup Scripts:**
|
||||
|
||||
```javascript
|
||||
// ✅ CORRECT - Save after cleanup loop completes (with namespace)
|
||||
const namespace = 'flyer-crawler-test';
|
||||
targetProcesses.forEach((p) => {
|
||||
exec(`pm2 delete ${p.pm2_env.pm_id} --namespace ${namespace}`);
|
||||
});
|
||||
exec(`pm2 save --namespace ${namespace}`); // Persist all deletions
|
||||
|
||||
// ❌ WRONG - Missing save and namespace
|
||||
targetProcesses.forEach((p) => {
|
||||
exec('pm2 delete ' + p.pm2_env.pm_id);
|
||||
});
|
||||
```
|
||||
|
||||
**Why This Matters:**
|
||||
|
||||
PM2 maintains an in-memory process list. The `pm2 save` command writes this list to `~/.pm2/dump.pm2`, which PM2 uses to resurrect processes after daemon restarts. Without it, your carefully managed process state is lost. Using namespaces ensures that `pm2 save` in one environment does not affect another.
|
||||
|
||||
**See Also:**
|
||||
|
||||
- [ADR-014: Containerization and Deployment Strategy](docs/adr/0014-containerization-and-deployment-strategy.md)
|
||||
- [ADR-061: PM2 Process Isolation Safeguards](docs/adr/0061-pm2-process-isolation-safeguards.md)
|
||||
- [ADR-063: PM2 Namespace Implementation](docs/adr/0063-pm2-namespace-implementation.md)
|
||||
|
||||
### Communication Style
|
||||
|
||||
Ask before assuming. Never assume:
|
||||
@@ -40,7 +181,7 @@ Ask before assuming. Never assume:
|
||||
1. **Memory**: `mcp__memory__read_graph` - Recall project context, credentials, known issues
|
||||
2. **Git**: `git log --oneline -10` - Recent changes
|
||||
3. **Containers**: `mcp__podman__container_list` - Running state
|
||||
4. **PM2 Status**: `podman exec flyer-crawler-dev pm2 status` - Process health (API, Worker, Vite)
|
||||
4. **PM2 Status**: `podman exec flyer-crawler-dev pm2 status --namespace flyer-crawler-dev` - Process health (API, Worker, Vite)
|
||||
|
||||
---
|
||||
|
||||
@@ -48,37 +189,41 @@ Ask before assuming. Never assume:
|
||||
|
||||
### Essential Commands
|
||||
|
||||
| Command | Description |
|
||||
| ------------------------------------------------------------ | --------------------- |
|
||||
| `podman exec -it flyer-crawler-dev npm test` | Run all tests |
|
||||
| `podman exec -it flyer-crawler-dev npm run test:unit` | Unit tests only |
|
||||
| `podman exec -it flyer-crawler-dev npm run type-check` | TypeScript check |
|
||||
| `podman exec -it flyer-crawler-dev npm run test:integration` | Integration tests |
|
||||
| `podman exec -it flyer-crawler-dev pm2 status` | PM2 process status |
|
||||
| `podman exec -it flyer-crawler-dev pm2 logs` | View all PM2 logs |
|
||||
| `podman exec -it flyer-crawler-dev pm2 restart all` | Restart all processes |
|
||||
| Command | Description |
|
||||
| ------------------------------------------------------------------------------------ | ------------------------ |
|
||||
| `podman exec -it flyer-crawler-dev npm test` | Run all tests |
|
||||
| `podman exec -it flyer-crawler-dev npm run test:unit` | Unit tests only |
|
||||
| `podman exec -it flyer-crawler-dev npm run type-check` | TypeScript check |
|
||||
| `podman exec -it flyer-crawler-dev npm run test:integration` | Integration tests |
|
||||
| `podman exec -it flyer-crawler-dev pm2 status --namespace flyer-crawler-dev` | PM2 process status (dev) |
|
||||
| `podman exec -it flyer-crawler-dev pm2 logs --namespace flyer-crawler-dev` | View PM2 logs (dev) |
|
||||
| `podman exec -it flyer-crawler-dev pm2 restart all --namespace flyer-crawler-dev` | Restart all (dev) |
|
||||
| `pm2 status --namespace flyer-crawler-prod` | PM2 status (production) |
|
||||
| `pm2 status --namespace flyer-crawler-test` | PM2 status (test) |
|
||||
|
||||
### Key Patterns (with file locations)
|
||||
|
||||
| Pattern | ADR | Implementation | File |
|
||||
| ------------------ | ------- | ------------------------------------------------- | ----------------------------------- |
|
||||
| Error Handling | ADR-001 | `handleDbError()`, throw `NotFoundError` | `src/services/db/errors.db.ts` |
|
||||
| Repository Methods | ADR-034 | `get*` (throws), `find*` (null), `list*` (array) | `src/services/db/*.db.ts` |
|
||||
| API Responses | ADR-028 | `sendSuccess()`, `sendPaginated()`, `sendError()` | `src/utils/apiResponse.ts` |
|
||||
| Transactions | ADR-002 | `withTransaction(async (client) => {...})` | `src/services/db/transaction.db.ts` |
|
||||
| Pattern | ADR | Implementation | File |
|
||||
| ------------------ | ------- | ------------------------------------------------- | ------------------------------------- |
|
||||
| Error Handling | ADR-001 | `handleDbError()`, throw `NotFoundError` | `src/services/db/errors.db.ts` |
|
||||
| Repository Methods | ADR-034 | `get*` (throws), `find*` (null), `list*` (array) | `src/services/db/*.db.ts` |
|
||||
| API Responses | ADR-028 | `sendSuccess()`, `sendPaginated()`, `sendError()` | `src/utils/apiResponse.ts` |
|
||||
| Transactions | ADR-002 | `withTransaction(async (client) => {...})` | `src/services/db/connection.db.ts` |
|
||||
| Feature Flags | ADR-024 | `isFeatureEnabled()`, `useFeatureFlag()` | `src/services/featureFlags.server.ts` |
|
||||
|
||||
### Key Files Quick Access
|
||||
|
||||
| Purpose | File |
|
||||
| ----------------- | -------------------------------- |
|
||||
| Express app | `server.ts` |
|
||||
| Environment | `src/config/env.ts` |
|
||||
| Routes | `src/routes/*.routes.ts` |
|
||||
| Repositories | `src/services/db/*.db.ts` |
|
||||
| Workers | `src/services/workers.server.ts` |
|
||||
| Queues | `src/services/queues.server.ts` |
|
||||
| PM2 Config (Dev) | `ecosystem.dev.config.cjs` |
|
||||
| PM2 Config (Prod) | `ecosystem.config.cjs` |
|
||||
| Purpose | File |
|
||||
| ----------------- | ------------------------------------- |
|
||||
| Express app | `server.ts` |
|
||||
| Environment | `src/config/env.ts` |
|
||||
| Routes | `src/routes/*.routes.ts` |
|
||||
| Repositories | `src/services/db/*.db.ts` |
|
||||
| Workers | `src/services/workers.server.ts` |
|
||||
| Queues | `src/services/queues.server.ts` |
|
||||
| Feature Flags | `src/services/featureFlags.server.ts` |
|
||||
| PM2 Config (Dev) | `ecosystem.dev.config.cjs` |
|
||||
| PM2 Config (Prod) | `ecosystem.config.cjs` |
|
||||
|
||||
---
|
||||
|
||||
@@ -121,25 +266,32 @@ The dev container now matches production by using PM2 for process management.
|
||||
- `flyer-crawler-worker-dev` - Background job worker
|
||||
- `flyer-crawler-vite-dev` - Vite frontend dev server (port 5173)
|
||||
|
||||
### Log Aggregation (ADR-050)
|
||||
### Log Aggregation (ADR-015)
|
||||
|
||||
All logs flow to Bugsink via Logstash:
|
||||
All logs flow to Bugsink via Logstash with 3-project routing:
|
||||
|
||||
| Source | Log Location | Status |
|
||||
| ----------------- | --------------------------------- | ------ |
|
||||
| Backend (Pino) | `/var/log/pm2/api-*.log` | Active |
|
||||
| Worker (Pino) | `/var/log/pm2/worker-*.log` | Active |
|
||||
| Vite | `/var/log/pm2/vite-*.log` | Active |
|
||||
| PostgreSQL | `/var/log/postgresql/*.log` | Active |
|
||||
| Redis | `/var/log/redis/redis-server.log` | Active |
|
||||
| NGINX | `/var/log/nginx/*.log` | Active |
|
||||
| Frontend (Sentry) | Browser -> Bugsink SDK | Active |
|
||||
| Source | Log Location | Bugsink Project |
|
||||
| ----------------- | --------------------------------- | ------------------ |
|
||||
| Backend (Pino) | `/var/log/pm2/api-*.log` | Backend API (1) |
|
||||
| Worker (Pino) | `/var/log/pm2/worker-*.log` | Backend API (1) |
|
||||
| PostgreSQL | `/var/log/postgresql/*.log` | Backend API (1) |
|
||||
| Vite | `/var/log/pm2/vite-*.log` | Infrastructure (4) |
|
||||
| Redis | `/var/log/redis/redis-server.log` | Infrastructure (4) |
|
||||
| NGINX | `/var/log/nginx/*.log` | Infrastructure (4) |
|
||||
| Frontend (Sentry) | Browser -> nginx proxy | Frontend (2) |
|
||||
|
||||
**Bugsink Projects (Dev Container)**:
|
||||
|
||||
- Project 1: Backend API (Dev) - Application errors
|
||||
- Project 2: Frontend (Dev) - Browser errors via nginx proxy
|
||||
- Project 4: Infrastructure (Dev) - Redis, NGINX, Vite errors
|
||||
|
||||
**Key Files**:
|
||||
|
||||
- `ecosystem.dev.config.cjs` - PM2 development configuration
|
||||
- `scripts/dev-entrypoint.sh` - Container startup script
|
||||
- `docker/logstash/bugsink.conf` - Logstash pipeline configuration
|
||||
- `docker/nginx/dev.conf` - NGINX config with Bugsink API proxy
|
||||
|
||||
**Full Dev Container Guide**: See [docs/development/DEV-CONTAINER.md](docs/development/DEV-CONTAINER.md)
|
||||
|
||||
@@ -197,7 +349,7 @@ All logs flow to Bugsink via Logstash:
|
||||
|
||||
**Launch Pattern**:
|
||||
|
||||
```
|
||||
```text
|
||||
Use Task tool with subagent_type: "coder", "db-dev", "tester", etc.
|
||||
```
|
||||
|
||||
@@ -215,9 +367,43 @@ Common issues with solutions:
|
||||
4. **Filename collisions** - Multer predictable names → Use `${Date.now()}-${Math.round(Math.random() * 1e9)}`
|
||||
5. **Response format mismatches** - API format changes → Log response bodies, update assertions
|
||||
6. **External service failures** - PM2/Redis unavailable → try/catch with graceful degradation
|
||||
7. **TZ environment variable breaks async hooks** - `TZ=America/Los_Angeles` causes `RangeError: Invalid triggerAsyncId value: NaN` → Tests now explicitly set `TZ=` (empty) in package.json scripts
|
||||
|
||||
**Full Details**: See test issues section at end of this document or [docs/development/TESTING.md](docs/development/TESTING.md)
|
||||
|
||||
### PM2 Process Isolation Incidents
|
||||
|
||||
**CRITICAL**: PM2 process cleanup scripts can affect all PM2 processes if not properly filtered.
|
||||
|
||||
**Incident**: 2026-02-17 Production Deployment (v0.15.0)
|
||||
|
||||
- **Impact**: ALL PM2 processes on production server were killed
|
||||
- **Affected**: stock-alert.projectium.com and all other PM2-managed applications
|
||||
- **Root Cause**: Under investigation (see [incident report](docs/operations/INCIDENT-2026-02-17-PM2-PROCESS-KILL.md))
|
||||
- **Status**: Safeguards added to prevent recurrence
|
||||
|
||||
**Prevention Measures** (implemented):
|
||||
|
||||
1. Name-based filtering (exact match or pattern-based)
|
||||
2. Pre-cleanup process list logging
|
||||
3. Process count validation (abort if filtering all processes)
|
||||
4. Explicit name verification in logs
|
||||
5. Post-cleanup verification
|
||||
6. Workflow version hash logging
|
||||
|
||||
**If PM2 Incident Occurs**:
|
||||
|
||||
- **DO NOT** attempt another deployment immediately
|
||||
- Follow the [PM2 Incident Response Runbook](docs/operations/PM2-INCIDENT-RESPONSE.md)
|
||||
- Manually restore affected processes
|
||||
- Investigate workflow execution logs before next deployment
|
||||
|
||||
**Related Documentation**:
|
||||
|
||||
- [PM2 Namespace Isolation](#pm2-namespace-isolation-productiontest-servers) (existing section)
|
||||
- [Incident Report 2026-02-17](docs/operations/INCIDENT-2026-02-17-PM2-PROCESS-KILL.md)
|
||||
- [PM2 Incident Response Runbook](docs/operations/PM2-INCIDENT-RESPONSE.md)
|
||||
|
||||
### Git Bash Path Conversion (Windows)
|
||||
|
||||
Git Bash auto-converts Unix paths, breaking container commands.
|
||||
@@ -277,8 +463,8 @@ podman cp "d:/path/file" container:/tmp/file
|
||||
|
||||
**Quick Access**:
|
||||
|
||||
- **Dev**: https://localhost:8443 (`admin@localhost`/`admin`)
|
||||
- **Prod**: https://bugsink.projectium.com
|
||||
- **Dev**: <https://localhost:8443> (`admin@localhost`/`admin`)
|
||||
- **Prod**: <https://bugsink.projectium.com>
|
||||
|
||||
**Token Creation** (required for MCP):
|
||||
|
||||
@@ -286,15 +472,15 @@ podman cp "d:/path/file" container:/tmp/file
|
||||
# Dev container
|
||||
MSYS_NO_PATHCONV=1 podman exec -e DATABASE_URL=postgresql://bugsink:bugsink_dev_password@postgres:5432/bugsink -e SECRET_KEY=dev-bugsink-secret-key-minimum-50-characters-for-security flyer-crawler-dev sh -c 'cd /opt/bugsink/conf && DJANGO_SETTINGS_MODULE=bugsink_conf PYTHONPATH=/opt/bugsink/conf:/opt/bugsink/lib/python3.10/site-packages /opt/bugsink/bin/python -m django create_auth_token'
|
||||
|
||||
# Production (via SSH)
|
||||
ssh root@projectium.com "cd /opt/bugsink && bugsink-manage create_auth_token"
|
||||
# Production (user executes on server)
|
||||
cd /opt/bugsink && bugsink-manage create_auth_token
|
||||
```
|
||||
|
||||
### Logstash
|
||||
|
||||
**See**: [docs/operations/LOGSTASH-QUICK-REF.md](docs/operations/LOGSTASH-QUICK-REF.md)
|
||||
|
||||
Log aggregation: PostgreSQL + PM2 + Redis + NGINX → Bugsink (ADR-050)
|
||||
Log aggregation: PostgreSQL + PM2 + Redis + NGINX → Bugsink (ADR-015)
|
||||
|
||||
---
|
||||
|
||||
@@ -314,59 +500,3 @@ Log aggregation: PostgreSQL + PM2 + Redis + NGINX → Bugsink (ADR-050)
|
||||
| **Logstash** | [LOGSTASH-QUICK-REF.md](docs/operations/LOGSTASH-QUICK-REF.md) |
|
||||
| **ADRs** | [docs/adr/index.md](docs/adr/index.md) |
|
||||
| **All Docs** | [docs/README.md](docs/README.md) |
|
||||
|
||||
---
|
||||
|
||||
## Appendix: Integration Test Issues (Full Details)
|
||||
|
||||
### 1. Vitest globalSetup Context Isolation
|
||||
|
||||
Vitest's `globalSetup` runs in separate Node.js context. Singletons, spies, mocks do NOT share instances with test files.
|
||||
|
||||
**Affected**: BullMQ worker service mocks (AI/DB failure tests)
|
||||
|
||||
**Solutions**: Mark `.todo()`, create test-only API endpoints, use Redis-based mock flags
|
||||
|
||||
```typescript
|
||||
// DOES NOT WORK - different instances
|
||||
const { flyerProcessingService } = await import('../../services/workers.server');
|
||||
flyerProcessingService._getAiProcessor()._setExtractAndValidateData(mockFn);
|
||||
```
|
||||
|
||||
### 2. Cleanup Queue Deletes Before Verification
|
||||
|
||||
Cleanup worker processes jobs in globalSetup context, ignoring test spies.
|
||||
|
||||
**Solution**: Drain and pause queue:
|
||||
|
||||
```typescript
|
||||
const { cleanupQueue } = await import('../../services/queues.server');
|
||||
await cleanupQueue.drain();
|
||||
await cleanupQueue.pause();
|
||||
// ... test ...
|
||||
await cleanupQueue.resume();
|
||||
```
|
||||
|
||||
### 3. Cache Stale After Direct SQL
|
||||
|
||||
Direct `pool.query()` inserts bypass cache invalidation.
|
||||
|
||||
**Solution**: `await cacheService.invalidateFlyers();` after inserts
|
||||
|
||||
### 4. Test Filename Collisions
|
||||
|
||||
Multer predictable filenames cause race conditions.
|
||||
|
||||
**Solution**: Use unique suffix: `${Date.now()}-${Math.round(Math.random() * 1e9)}`
|
||||
|
||||
### 5. Response Format Mismatches
|
||||
|
||||
API formats change: `data.jobId` vs `data.job.id`, nested vs flat, string vs number IDs.
|
||||
|
||||
**Solution**: Log response bodies, update assertions
|
||||
|
||||
### 6. External Service Availability
|
||||
|
||||
PM2/Redis health checks fail when unavailable.
|
||||
|
||||
**Solution**: try/catch with graceful degradation or mock
|
||||
|
||||
@@ -174,6 +174,21 @@ BUGSINK = {\n\
|
||||
}\n\
|
||||
\n\
|
||||
ALLOWED_HOSTS = deduce_allowed_hosts(BUGSINK["BASE_URL"])\n\
|
||||
# Also allow 127.0.0.1 access (both localhost and 127.0.0.1 should work)\n\
|
||||
if "127.0.0.1" not in ALLOWED_HOSTS:\n\
|
||||
ALLOWED_HOSTS.append("127.0.0.1")\n\
|
||||
if "localhost" not in ALLOWED_HOSTS:\n\
|
||||
ALLOWED_HOSTS.append("localhost")\n\
|
||||
\n\
|
||||
# CSRF Trusted Origins (Django 4.0+ requires full origin for HTTPS POST requests)\n\
|
||||
# This fixes "CSRF verification failed" errors when accessing Bugsink via HTTPS\n\
|
||||
# Both localhost and 127.0.0.1 must be trusted to support different access patterns\n\
|
||||
CSRF_TRUSTED_ORIGINS = [\n\
|
||||
"https://localhost:8443",\n\
|
||||
"https://127.0.0.1:8443",\n\
|
||||
"http://localhost:8000",\n\
|
||||
"http://127.0.0.1:8000",\n\
|
||||
]\n\
|
||||
\n\
|
||||
# Console email backend for dev\n\
|
||||
EMAIL_BACKEND = "bugsink.email_backends.QuietConsoleEmailBackend"\n\
|
||||
|
||||
20
README.md
20
README.md
@@ -49,8 +49,8 @@ npm run dev
|
||||
|
||||
The application will be available at:
|
||||
|
||||
- **Frontend**: http://localhost:5173
|
||||
- **Backend API**: http://localhost:3001
|
||||
- **Frontend**: <http://localhost:5173>
|
||||
- **Backend API**: <http://localhost:3001>
|
||||
|
||||
See [docs/getting-started/INSTALL.md](docs/getting-started/INSTALL.md) for detailed setup instructions including:
|
||||
|
||||
@@ -88,7 +88,7 @@ See [docs/development/TESTING.md](docs/development/TESTING.md) for testing guide
|
||||
| [⚙️ Installation Guide](docs/getting-started/INSTALL.md) | Local development setup with Podman |
|
||||
| [🏗️ Architecture Overview](docs/architecture/DATABASE.md) | System design, database, authentication |
|
||||
| [💻 Development Guide](docs/development/TESTING.md) | Testing, debugging, code patterns |
|
||||
| [🚀 Deployment Guide](docs/operations/DEPLOYMENT.md) | Production setup, NGINX, PM2 |
|
||||
| [🚀 Deployment Guide](docs/operations/DEPLOYMENT.md) | Production setup, NGINX, PM2 namespaces |
|
||||
| [🤖 AI Agent Guides](docs/subagents/OVERVIEW.md) | Working with Claude Code subagents |
|
||||
|
||||
### Quick References
|
||||
@@ -126,13 +126,13 @@ See [INSTALL.md](INSTALL.md) for the complete list.
|
||||
|
||||
## Scripts
|
||||
|
||||
| Command | Description |
|
||||
| -------------------- | -------------------------------- |
|
||||
| `npm run dev` | Start development server |
|
||||
| `npm run build` | Build for production |
|
||||
| `npm run start:prod` | Start production server with PM2 |
|
||||
| `npm run test` | Run test suite |
|
||||
| `npm run seed` | Seed development user accounts |
|
||||
| Command | Description |
|
||||
| -------------------- | ------------------------------------------------------------ |
|
||||
| `npm run dev` | Start development server |
|
||||
| `npm run build` | Build for production |
|
||||
| `npm run start:prod` | Start production server with PM2 (uses namespace isolation) |
|
||||
| `npm run test` | Run test suite |
|
||||
| `npm run seed` | Seed development user accounts |
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -57,6 +57,8 @@ services:
|
||||
- '8000:8000' # Bugsink error tracking HTTP (ADR-015)
|
||||
- '8443:8443' # Bugsink error tracking HTTPS (ADR-015)
|
||||
environment:
|
||||
# Timezone: PST (America/Los_Angeles) for consistent log timestamps
|
||||
- TZ=America/Los_Angeles
|
||||
# Core settings
|
||||
- NODE_ENV=development
|
||||
# Database - use service name for Docker networking
|
||||
@@ -122,6 +124,10 @@ services:
|
||||
ports:
|
||||
- '5432:5432'
|
||||
environment:
|
||||
# Timezone: PST (America/Los_Angeles) for consistent log timestamps
|
||||
TZ: America/Los_Angeles
|
||||
# PostgreSQL timezone setting (used by log_timezone and timezone parameters)
|
||||
PGTZ: America/Los_Angeles
|
||||
POSTGRES_USER: postgres
|
||||
POSTGRES_PASSWORD: postgres
|
||||
POSTGRES_DB: flyer_crawler_dev
|
||||
@@ -142,6 +148,8 @@ services:
|
||||
postgres
|
||||
-c config_file=/var/lib/postgresql/data/postgresql.conf
|
||||
-c hba_file=/var/lib/postgresql/data/pg_hba.conf
|
||||
-c timezone=America/Los_Angeles
|
||||
-c log_timezone=America/Los_Angeles
|
||||
-c log_min_messages=notice
|
||||
-c client_min_messages=notice
|
||||
-c logging_collector=on
|
||||
@@ -175,6 +183,9 @@ services:
|
||||
user: root
|
||||
ports:
|
||||
- '6379:6379'
|
||||
environment:
|
||||
# Timezone: PST (America/Los_Angeles) for consistent log timestamps
|
||||
TZ: America/Los_Angeles
|
||||
volumes:
|
||||
- redis_data:/data
|
||||
# Create log volume for Logstash access (ADR-050)
|
||||
|
||||
@@ -12,9 +12,18 @@
|
||||
# - NGINX logs (/var/log/nginx/*.log) - Access and error logs
|
||||
# - Redis logs (/var/log/redis/*.log) - Via shared volume (ADR-050)
|
||||
#
|
||||
# Bugsink Projects:
|
||||
# - Project 1: Backend API (Dev) - Pino errors, PostgreSQL errors
|
||||
# Bugsink Projects (3-project architecture):
|
||||
# - Project 1: Backend API (Dev) - Pino/PM2 app errors, PostgreSQL errors
|
||||
# DSN Key: cea01396c56246adb5878fa5ee6b1d22
|
||||
# - Project 2: Frontend (Dev) - Configured via Sentry SDK in browser
|
||||
# DSN Key: d92663cb73cf4145b677b84029e4b762
|
||||
# - Project 4: Infrastructure (Dev) - Redis, NGINX, PM2 operational logs
|
||||
# DSN Key: 14e8791da3d347fa98073261b596cab9
|
||||
#
|
||||
# Routing Logic:
|
||||
# - Backend logs (type: pm2_api, pm2_worker, pino, postgres) -> Project 1
|
||||
# - Infrastructure logs (type: redis, nginx_error, nginx_5xx) -> Project 4
|
||||
# - Vite errors (type: pm2_vite with errors) -> Project 4 (build tooling)
|
||||
#
|
||||
# Related Documentation:
|
||||
# - docs/adr/0050-postgresql-function-observability.md
|
||||
@@ -112,7 +121,8 @@ input {
|
||||
# ============================================================================
|
||||
# Captures PostgreSQL log output including fn_log() structured JSON messages.
|
||||
# PostgreSQL is configured to write logs to /var/log/postgresql/ (shared volume).
|
||||
# Log format: "2026-01-22 00:00:00 UTC [5724] postgres@flyer_crawler_dev LOG: message"
|
||||
# Log format: "2026-01-22 14:30:00 PST [5724] postgres@flyer_crawler_dev LOG: message"
|
||||
# Note: Timestamps are in PST (America/Los_Angeles) timezone as configured in compose.dev.yml
|
||||
file {
|
||||
path => "/var/log/postgresql/*.log"
|
||||
type => "postgres"
|
||||
@@ -217,10 +227,11 @@ filter {
|
||||
# PostgreSQL Log Processing (ADR-050)
|
||||
# ============================================================================
|
||||
# PostgreSQL log format in dev container:
|
||||
# "2026-01-22 00:00:00 UTC [5724] postgres@flyer_crawler_dev LOG: message"
|
||||
# "2026-01-22 07:06:03 UTC [19851] postgres@flyer_crawler_dev ERROR: column "id" does not exist"
|
||||
# "2026-01-22 14:30:00 PST [5724] postgres@flyer_crawler_dev LOG: message"
|
||||
# "2026-01-22 15:06:03 PST [19851] postgres@flyer_crawler_dev ERROR: column "id" does not exist"
|
||||
# Note: Timestamps are in PST (America/Los_Angeles) timezone
|
||||
if [type] == "postgres" {
|
||||
# Parse PostgreSQL log prefix with UTC timezone
|
||||
# Parse PostgreSQL log prefix with timezone (PST in dev, may vary in prod)
|
||||
grok {
|
||||
match => { "message" => "%{YEAR}-%{MONTHNUM}-%{MONTHDAY} %{TIME} %{WORD:pg_timezone} \[%{POSINT:pg_pid}\] %{DATA:pg_user}@%{DATA:pg_database} %{WORD:pg_level}: ?%{GREEDYDATA:pg_message}" }
|
||||
tag_on_failure => ["_postgres_grok_failure"]
|
||||
@@ -344,26 +355,56 @@ filter {
|
||||
}
|
||||
|
||||
# ============================================================================
|
||||
# Generate Sentry Event ID for all errors
|
||||
# Generate Sentry Event ID and Ensure Required Fields for all errors
|
||||
# ============================================================================
|
||||
# CRITICAL: sentry_level MUST be set for all errors before output.
|
||||
# Bugsink's PostgreSQL schema limits level to varchar(7), so valid values are:
|
||||
# fatal, error, warning, info, debug (all <= 7 chars)
|
||||
# If sentry_level is not set, the literal "%{sentry_level}" (16 chars) is sent,
|
||||
# causing PostgreSQL insertion failures.
|
||||
# ============================================================================
|
||||
if "error" in [tags] {
|
||||
# Use Ruby for robust field handling - handles all edge cases
|
||||
ruby {
|
||||
code => '
|
||||
require "securerandom"
|
||||
event.set("sentry_event_id", SecureRandom.hex(16))
|
||||
'
|
||||
}
|
||||
|
||||
# Ensure error_message has a fallback value
|
||||
if ![error_message] {
|
||||
mutate { add_field => { "error_message" => "%{message}" } }
|
||||
# Generate unique event ID for Sentry
|
||||
event.set("sentry_event_id", SecureRandom.hex(16))
|
||||
|
||||
# =====================================================================
|
||||
# CRITICAL: Validate and set sentry_level
|
||||
# =====================================================================
|
||||
# Valid Sentry levels (max 7 chars for Bugsink PostgreSQL schema):
|
||||
# fatal, error, warning, info, debug
|
||||
# Default to "error" if missing, empty, or invalid.
|
||||
# =====================================================================
|
||||
valid_levels = ["fatal", "error", "warning", "info", "debug"]
|
||||
current_level = event.get("sentry_level")
|
||||
|
||||
if current_level.nil? || current_level.to_s.strip.empty? || !valid_levels.include?(current_level.to_s.downcase)
|
||||
event.set("sentry_level", "error")
|
||||
else
|
||||
# Normalize to lowercase
|
||||
event.set("sentry_level", current_level.to_s.downcase)
|
||||
end
|
||||
|
||||
# =====================================================================
|
||||
# Ensure error_message has a fallback value
|
||||
# =====================================================================
|
||||
error_msg = event.get("error_message")
|
||||
if error_msg.nil? || error_msg.to_s.strip.empty?
|
||||
fallback_msg = event.get("message") || event.get("msg") || "Unknown error"
|
||||
event.set("error_message", fallback_msg.to_s)
|
||||
end
|
||||
'
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
output {
|
||||
# ============================================================================
|
||||
# Forward Errors to Bugsink (Backend API Project)
|
||||
# Forward Errors to Bugsink (Project Routing)
|
||||
# ============================================================================
|
||||
# Bugsink uses Sentry-compatible API. Events must include:
|
||||
# - event_id: 32 hex characters (UUID without dashes)
|
||||
@@ -373,9 +414,50 @@ output {
|
||||
# - platform: "node" for backend, "javascript" for frontend
|
||||
#
|
||||
# Authentication via X-Sentry-Auth header with project's public key.
|
||||
# Dev container DSN: http://cea01396c56246adb5878fa5ee6b1d22@localhost:8000/1
|
||||
#
|
||||
# Project Routing:
|
||||
# - Project 1 (Backend): Pino app logs, PostgreSQL errors
|
||||
# - Project 4 (Infrastructure): Redis, NGINX, Vite build errors
|
||||
# ============================================================================
|
||||
if "error" in [tags] {
|
||||
|
||||
# ============================================================================
|
||||
# Infrastructure Errors -> Project 4
|
||||
# ============================================================================
|
||||
# Redis warnings/errors, NGINX errors, and Vite build errors go to
|
||||
# the Infrastructure project for separation from application code errors.
|
||||
if "error" in [tags] and ([type] == "redis" or [type] == "nginx_error" or [type] == "nginx_access" or [type] == "pm2_vite") {
|
||||
http {
|
||||
url => "http://localhost:8000/api/4/store/"
|
||||
http_method => "post"
|
||||
format => "json"
|
||||
headers => {
|
||||
"X-Sentry-Auth" => "Sentry sentry_key=14e8791da3d347fa98073261b596cab9, sentry_version=7"
|
||||
"Content-Type" => "application/json"
|
||||
}
|
||||
mapping => {
|
||||
"event_id" => "%{sentry_event_id}"
|
||||
"timestamp" => "%{@timestamp}"
|
||||
"level" => "%{sentry_level}"
|
||||
"platform" => "other"
|
||||
"logger" => "%{type}"
|
||||
"message" => "%{error_message}"
|
||||
"extra" => {
|
||||
"hostname" => "%{[host][name]}"
|
||||
"source_type" => "%{type}"
|
||||
"tags" => "%{tags}"
|
||||
"original_message" => "%{message}"
|
||||
"project" => "infrastructure"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
# ============================================================================
|
||||
# Backend Application Errors -> Project 1
|
||||
# ============================================================================
|
||||
# Pino application logs (API, Worker), PostgreSQL function errors, and
|
||||
# native PostgreSQL errors go to the Backend API project.
|
||||
else if "error" in [tags] and ([type] in ["pm2_api", "pm2_worker", "pino", "postgres"]) {
|
||||
http {
|
||||
url => "http://localhost:8000/api/1/store/"
|
||||
http_method => "post"
|
||||
@@ -384,7 +466,6 @@ output {
|
||||
"X-Sentry-Auth" => "Sentry sentry_key=cea01396c56246adb5878fa5ee6b1d22, sentry_version=7"
|
||||
"Content-Type" => "application/json"
|
||||
}
|
||||
# Transform event to Sentry format using regular fields (not @metadata)
|
||||
mapping => {
|
||||
"event_id" => "%{sentry_event_id}"
|
||||
"timestamp" => "%{@timestamp}"
|
||||
@@ -397,6 +478,38 @@ output {
|
||||
"source_type" => "%{type}"
|
||||
"tags" => "%{tags}"
|
||||
"original_message" => "%{message}"
|
||||
"project" => "backend"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
# ============================================================================
|
||||
# Fallback: Any other errors -> Project 1
|
||||
# ============================================================================
|
||||
# Catch-all for any errors that don't match specific routing rules.
|
||||
else if "error" in [tags] {
|
||||
http {
|
||||
url => "http://localhost:8000/api/1/store/"
|
||||
http_method => "post"
|
||||
format => "json"
|
||||
headers => {
|
||||
"X-Sentry-Auth" => "Sentry sentry_key=cea01396c56246adb5878fa5ee6b1d22, sentry_version=7"
|
||||
"Content-Type" => "application/json"
|
||||
}
|
||||
mapping => {
|
||||
"event_id" => "%{sentry_event_id}"
|
||||
"timestamp" => "%{@timestamp}"
|
||||
"level" => "%{sentry_level}"
|
||||
"platform" => "node"
|
||||
"logger" => "%{type}"
|
||||
"message" => "%{error_message}"
|
||||
"extra" => {
|
||||
"hostname" => "%{[host][name]}"
|
||||
"source_type" => "%{type}"
|
||||
"tags" => "%{tags}"
|
||||
"original_message" => "%{message}"
|
||||
"project" => "backend-fallback"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user