Compare commits
339 Commits
v0.2.37
...
a14816c8ee
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
a14816c8ee | ||
|
|
08b220e29c | ||
|
|
d41a3f1887 | ||
| 1f6cdc62d7 | |||
|
|
978c63bacd | ||
| 544eb7ae3c | |||
|
|
f6839f6e14 | ||
| 3fac29436a | |||
|
|
56f45c9301 | ||
| 83460abce4 | |||
|
|
1b084b2ba4 | ||
| 0ea034bdc8 | |||
|
|
fc9e27078a | ||
| fb8cbe8007 | |||
| f49f786c23 | |||
|
|
dd31141d4e | ||
| 8073094760 | |||
|
|
33a1e146ab | ||
| 4f8216db77 | |||
|
|
42d605d19f | ||
| 749350df7f | |||
|
|
ac085100fe | ||
| ce4ecd1268 | |||
|
|
a57cfc396b | ||
| 987badbf8d | |||
|
|
d38fcd21c1 | ||
| 6e36cc3b07 | |||
|
|
62a8a8bf4b | ||
| 96038cfcf4 | |||
|
|
981214fdd0 | ||
| 92b0138108 | |||
|
|
27f0255240 | ||
| 4e06dde9e1 | |||
|
|
b9a0e5b82c | ||
| bb7fe8dc2c | |||
|
|
81f1f2250b | ||
| c6c90bb615 | |||
|
|
60489a626b | ||
| 3c63e1ecbb | |||
|
|
acbcb39cbe | ||
| a87a0b6af1 | |||
|
|
abdc3cb6db | ||
| 7a1bd50119 | |||
|
|
87d75d0571 | ||
| faf2900c28 | |||
|
|
5258efc179 | ||
| 2a5cc5bb51 | |||
|
|
8eaee2844f | ||
| 440a19c3a7 | |||
| 4ae6d84240 | |||
|
|
5870e5c614 | ||
| 2e7ebbd9ed | |||
|
|
dc3fa21359 | ||
| 11aeac5edd | |||
|
|
f6c0c082bc | ||
| 4e22213cd1 | |||
|
|
9815eb3686 | ||
| 2bf4a7c1e6 | |||
|
|
5eed3f51f4 | ||
| d250932c05 | |||
|
|
7d1f964574 | ||
| 3b69e58de3 | |||
|
|
5211aadd22 | ||
| a997d1d0b0 | |||
| cf5f77c58e | |||
|
|
cf0f5bb820 | ||
| 503e7084da | |||
|
|
d8aa19ac40 | ||
| dcd9452b8c | |||
|
|
6d468544e2 | ||
| 2913c7aa09 | |||
|
|
77f9cb6081 | ||
| 2f1d73ca12 | |||
|
|
402e2617ca | ||
| e14c19c112 | |||
|
|
ea46f66c7a | ||
| a42ee5a461 | |||
|
|
71710c8316 | ||
| 1480a73ab0 | |||
|
|
b3efa3c756 | ||
| fb8fd57bb6 | |||
|
|
0a90d9d590 | ||
| 6ab473f5f0 | |||
|
|
c46efe1474 | ||
| 25d6b76f6d | |||
|
|
9ffcc9d65d | ||
| 1285702210 | |||
|
|
d38b751b40 | ||
| e122d55ced | |||
|
|
af9992f773 | ||
| 3912139273 | |||
| b5f7f5e4d1 | |||
|
|
5173059621 | ||
| ebceb0e2e3 | |||
| e75054b1ab | |||
|
|
639313485a | ||
| 4a04e478c4 | |||
|
|
1814469eb4 | ||
| b777430ff7 | |||
|
|
23830c0d4e | ||
| ef42fee982 | |||
|
|
65cb54500c | ||
| 664ad291be | |||
|
|
ff912b9055 | ||
| ec32027bd4 | |||
|
|
59f773639b | ||
| dd2be5eecf | |||
|
|
a94bfbd3e9 | ||
| 338bbc9440 | |||
|
|
60aad04642 | ||
| 7f2aff9a24 | |||
|
|
689320e7d2 | ||
| e457bbf046 | |||
| 68cdbb6066 | |||
|
|
cea6be7145 | ||
| 74a5ca6331 | |||
|
|
62470e7661 | ||
| 2b517683fd | |||
|
|
5d06d1ba09 | ||
| 46c1e56b14 | |||
|
|
78a9b80010 | ||
| d356d9dfb6 | |||
|
|
ab63f83f50 | ||
| b546a55eaf | |||
|
|
dfa53a93dd | ||
| f30464cd0e | |||
|
|
2d2fa3c2c8 | ||
| 58cb391f4b | |||
|
|
0ebe2f0806 | ||
| 7867abc5bc | |||
|
|
cc4c8e2839 | ||
| 33ee2eeac9 | |||
|
|
e0b13f26fb | ||
| eee7f36756 | |||
|
|
622c919733 | ||
| c7f6b6369a | |||
|
|
879d956003 | ||
| 27eaac7ea8 | |||
|
|
93618c57e5 | ||
| 7f043ef704 | |||
|
|
62e35deddc | ||
| 59f6f43d03 | |||
|
|
e675c1a73c | ||
| 3c19084a0a | |||
|
|
e2049c6b9f | ||
| a3839c2f0d | |||
|
|
c1df3d7b1b | ||
| 94782f030d | |||
|
|
1c25b79251 | ||
| 0b0fa8294d | |||
|
|
f49f3a75fb | ||
| 8f14044ae6 | |||
|
|
55e1e425f4 | ||
| 68b16ad2e8 | |||
|
|
6a28934692 | ||
| 78c4a5fee6 | |||
|
|
1ce5f481a8 | ||
|
|
e0120d38fd | ||
| 6b2079ef2c | |||
|
|
0478e176d5 | ||
| 47f7f97cd9 | |||
|
|
b0719d1e39 | ||
| 0039ac3752 | |||
|
|
3c8316f4f7 | ||
| 2564df1c64 | |||
|
|
696c547238 | ||
| 38165bdb9a | |||
|
|
6139dca072 | ||
| 68bfaa50e6 | |||
|
|
9c42621f74 | ||
| 1b98282202 | |||
|
|
b6731b220c | ||
| 3507d455e8 | |||
|
|
92b2adf8e8 | ||
| d6c7452256 | |||
|
|
d812b681dd | ||
| b4306a6092 | |||
|
|
57fdd159d5 | ||
| 4a747ca042 | |||
|
|
e0bf96824c | ||
| e86e09703e | |||
|
|
275741c79e | ||
| 3a40249ddb | |||
|
|
4c70905950 | ||
| 0b4884ff2a | |||
|
|
e4acab77c8 | ||
| 4e20b1b430 | |||
|
|
15747ac942 | ||
| e5fa89ef17 | |||
|
|
2c65da31e9 | ||
| eeec6af905 | |||
|
|
e7d03951b9 | ||
| af8816e0af | |||
|
|
64f6427e1a | ||
| c9b7a75429 | |||
|
|
0490f6922e | ||
| 057c4c9174 | |||
|
|
a9e56bc707 | ||
| e5d09c73b7 | |||
|
|
6e1298b825 | ||
| fc8e43437a | |||
|
|
cb453aa949 | ||
| 2651bd16ae | |||
|
|
91e0f0c46f | ||
| e6986d512b | |||
|
|
8f9c21675c | ||
| 7fb22cdd20 | |||
|
|
780291303d | ||
| 4f607f7d2f | |||
|
|
208227b3ed | ||
| bf1c7d4adf | |||
|
|
a7a30cf983 | ||
| 0bc0676b33 | |||
|
|
73484d3eb4 | ||
| b3253d5bbc | |||
|
|
54f3769e90 | ||
| bad6f74ee6 | |||
|
|
bcf16168b6 | ||
| 498fbd9e0e | |||
|
|
007ff8e538 | ||
| 1fc70e3915 | |||
|
|
d891e47e02 | ||
| 08c39afde4 | |||
|
|
c579543b8a | ||
| 0d84137786 | |||
|
|
20ee30c4b4 | ||
| 93612137e3 | |||
|
|
6e70f08e3c | ||
| 459f5f7976 | |||
|
|
a2e6331ddd | ||
| 13cd30bec9 | |||
|
|
baeb9488c6 | ||
| 0cba0f987e | |||
|
|
958a79997d | ||
| 8fb1c96f93 | |||
| 6e6fe80c7f | |||
|
|
d1554050bd | ||
|
|
b1fae270bb | ||
|
|
c852483e18 | ||
| 2e01ad5bc9 | |||
|
|
26763c7183 | ||
| f0c5c2c45b | |||
|
|
034bb60fd5 | ||
| d4b389cb79 | |||
|
|
a71fb81468 | ||
| 9bee0a013b | |||
|
|
8bcb4311b3 | ||
| 9fd15f3a50 | |||
|
|
e3c876c7be | ||
| 32dcf3b89e | |||
| 7066b937f6 | |||
|
|
8553ea8811 | ||
| 19885a50f7 | |||
|
|
ce82034b9d | ||
| 4528da2934 | |||
|
|
146d4c1351 | ||
| 88625706f4 | |||
|
|
e395faed30 | ||
| e8f8399896 | |||
|
|
ac0115af2b | ||
| f24b15f19b | |||
|
|
e64426bd84 | ||
| 0ec4cd68d2 | |||
|
|
840516d2a3 | ||
| 59355c3eef | |||
| d024935fe9 | |||
|
|
5a5470634e | ||
| 392231ad63 | |||
|
|
4b1c896621 | ||
| 720920a51c | |||
|
|
460adb9506 | ||
| 7aa1f756a9 | |||
|
|
c484a8ca9b | ||
| 28d2c9f4ec | |||
|
|
ee253e9449 | ||
| b6c15e53d0 | |||
|
|
722162c2c3 | ||
| 02a76fe996 | |||
|
|
0ebb03a7ab | ||
| 748ac9e049 | |||
|
|
495edd621c | ||
| 4ffca19db6 | |||
|
|
717427c5d7 | ||
| cc438a0e36 | |||
|
|
a32a0b62fc | ||
| 342f72b713 | |||
|
|
91254d18f3 | ||
| 40580dbf15 | |||
| 7f1d74c047 | |||
|
|
ecec686347 | ||
| 86de680080 | |||
|
|
0371947065 | ||
| 296698758c | |||
|
|
18c1161587 | ||
| 0010396780 | |||
|
|
d4557e13fb | ||
| 3e41130c69 | |||
|
|
d9034563d6 | ||
| 5836a75157 | |||
|
|
790008ae0d | ||
|
|
b5b91eb968 | ||
| 38eb810e7a | |||
|
|
458588a6e7 | ||
| 0b4113417f | |||
|
|
b59d2a9533 | ||
| 6740b35f8a | |||
|
|
92ad82a012 | ||
| 672e4ca597 | |||
|
|
e4d70a9b37 | ||
| c30f1c4162 | |||
|
|
44062a9f5b | ||
| 17fac8cf86 | |||
|
|
9fa8553486 | ||
|
|
f5b0b3b543 | ||
| e3ed5c7e63 | |||
|
|
ae0040e092 | ||
| 1f3f99d430 | |||
|
|
7be72f1758 | ||
| 0967c7a33d | |||
| 1f1c0fa6f3 | |||
|
|
728b1a20d3 | ||
| f248f7cbd0 | |||
|
|
0ad9bb16c2 | ||
| 510787bc5b | |||
|
|
9f696e7676 | ||
|
|
a77105316f | ||
| cadacb63f5 | |||
|
|
62592f707e | ||
| 023e48d99a | |||
|
|
99efca0371 | ||
| 1448950b81 | |||
|
|
a811fdac63 | ||
| 1201fe4d3c | |||
|
|
ba9228c9cb | ||
| b392b82c25 | |||
|
|
87825d13d6 | ||
| 21a6a796cf | |||
|
|
ecd0a73bc8 | ||
|
|
39d61dc7ad |
16
.claude/hooks.json
Normal file
16
.claude/hooks.json
Normal file
@@ -0,0 +1,16 @@
|
||||
{
|
||||
"$schema": "https://claude.ai/schemas/hooks.json",
|
||||
"hooks": {
|
||||
"PreToolUse": [
|
||||
{
|
||||
"matcher": "Bash",
|
||||
"hooks": [
|
||||
{
|
||||
"type": "command",
|
||||
"command": "node -e \"const cmd = process.argv[1] || ''; const isTest = /\\b(npm\\s+(run\\s+)?test|vitest|jest)\\b/i.test(cmd); const isWindows = process.platform === 'win32'; const inContainer = process.env.REMOTE_CONTAINERS === 'true' || process.env.DEVCONTAINER === 'true'; if (isTest && isWindows && !inContainer) { console.error('BLOCKED: Tests must run on Linux. Use Dev Container (Reopen in Container) or WSL.'); process.exit(1); }\" -- \"$CLAUDE_TOOL_INPUT\""
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
104
.claude/settings.local.json
Normal file
104
.claude/settings.local.json
Normal file
@@ -0,0 +1,104 @@
|
||||
{
|
||||
"permissions": {
|
||||
"allow": [
|
||||
"Bash(npm test:*)",
|
||||
"Bash(podman --version:*)",
|
||||
"Bash(podman ps:*)",
|
||||
"Bash(podman machine start:*)",
|
||||
"Bash(podman compose:*)",
|
||||
"Bash(podman pull:*)",
|
||||
"Bash(podman images:*)",
|
||||
"Bash(podman stop:*)",
|
||||
"Bash(echo:*)",
|
||||
"Bash(podman rm:*)",
|
||||
"Bash(podman run:*)",
|
||||
"Bash(podman start:*)",
|
||||
"Bash(podman exec:*)",
|
||||
"Bash(cat:*)",
|
||||
"Bash(PGPASSWORD=postgres psql:*)",
|
||||
"Bash(npm search:*)",
|
||||
"Bash(npx:*)",
|
||||
"Bash(curl:*)",
|
||||
"Bash(powershell:*)",
|
||||
"Bash(cmd.exe:*)",
|
||||
"Bash(npm run test:integration:*)",
|
||||
"Bash(grep:*)",
|
||||
"Bash(done)",
|
||||
"Bash(podman info:*)",
|
||||
"Bash(podman machine:*)",
|
||||
"Bash(podman system connection:*)",
|
||||
"Bash(podman inspect:*)",
|
||||
"Bash(python -m json.tool:*)",
|
||||
"Bash(claude mcp status)",
|
||||
"Bash(powershell.exe -Command \"claude mcp status\")",
|
||||
"Bash(powershell.exe -Command \"claude mcp\")",
|
||||
"Bash(powershell.exe -Command \"claude mcp list\")",
|
||||
"Bash(powershell.exe -Command \"claude --version\")",
|
||||
"Bash(powershell.exe -Command \"claude config\")",
|
||||
"Bash(powershell.exe -Command \"claude mcp get gitea-projectium\")",
|
||||
"Bash(powershell.exe -Command \"claude mcp add --help\")",
|
||||
"Bash(powershell.exe -Command \"claude mcp add -t stdio -s user filesystem -- D:\\\\nodejs\\\\npx.cmd -y @modelcontextprotocol/server-filesystem D:\\\\gitea\\\\flyer-crawler.projectium.com\\\\flyer-crawler.projectium.com\")",
|
||||
"Bash(powershell.exe -Command \"claude mcp add -t stdio -s user fetch -- D:\\\\nodejs\\\\npx.cmd -y @modelcontextprotocol/server-fetch\")",
|
||||
"Bash(powershell.exe -Command \"echo ''List files in src/hooks using filesystem MCP'' | claude --print\")",
|
||||
"Bash(powershell.exe -Command \"echo ''List all podman containers'' | claude --print\")",
|
||||
"Bash(powershell.exe -Command \"echo ''List my repositories on gitea.projectium.com using gitea-projectium MCP'' | claude --print\")",
|
||||
"Bash(powershell.exe -Command \"echo ''List my repositories on gitea.projectium.com using gitea-projectium MCP'' | claude --print --allowedTools ''mcp__gitea-projectium__*''\")",
|
||||
"Bash(powershell.exe -Command \"echo ''Fetch the homepage of https://gitea.projectium.com and summarize it'' | claude --print --allowedTools ''mcp__fetch__*''\")",
|
||||
"Bash(dir \"C:\\\\Users\\\\games3\\\\.claude\")",
|
||||
"Bash(dir:*)",
|
||||
"Bash(D:nodejsnpx.cmd -y @modelcontextprotocol/server-fetch --help)",
|
||||
"Bash(cmd /c \"dir /o-d C:\\\\Users\\\\games3\\\\.claude\\\\debug 2>nul | head -10\")",
|
||||
"mcp__memory__read_graph",
|
||||
"mcp__memory__create_entities",
|
||||
"mcp__memory__search_nodes",
|
||||
"mcp__memory__delete_entities",
|
||||
"mcp__sequential-thinking__sequentialthinking",
|
||||
"mcp__filesystem__list_directory",
|
||||
"mcp__filesystem__read_multiple_files",
|
||||
"mcp__filesystem__directory_tree",
|
||||
"mcp__filesystem__read_text_file",
|
||||
"Bash(wc:*)",
|
||||
"Bash(npm install:*)",
|
||||
"Bash(git grep:*)",
|
||||
"Bash(findstr:*)",
|
||||
"Bash(git add:*)",
|
||||
"mcp__filesystem__write_file",
|
||||
"mcp__podman__container_list",
|
||||
"Bash(podman cp:*)",
|
||||
"mcp__podman__container_inspect",
|
||||
"mcp__podman__network_list",
|
||||
"Bash(podman network connect:*)",
|
||||
"Bash(npm run build:*)",
|
||||
"Bash(set NODE_ENV=test)",
|
||||
"Bash(podman-compose:*)",
|
||||
"Bash(timeout 60 podman machine start:*)",
|
||||
"Bash(podman build:*)",
|
||||
"Bash(podman network rm:*)",
|
||||
"Bash(npm run lint)",
|
||||
"Bash(npm run typecheck:*)",
|
||||
"Bash(npm run type-check:*)",
|
||||
"Bash(npm run test:unit:*)",
|
||||
"mcp__filesystem__move_file",
|
||||
"Bash(git checkout:*)",
|
||||
"Bash(podman image inspect:*)",
|
||||
"Bash(node -e:*)",
|
||||
"Bash(xargs -I {} sh -c 'if ! grep -q \"\"vi.mock.*apiClient\"\" \"\"{}\"\"; then echo \"\"{}\"\"; fi')",
|
||||
"Bash(MSYS_NO_PATHCONV=1 podman exec:*)",
|
||||
"Bash(docker ps:*)",
|
||||
"Bash(find:*)",
|
||||
"Bash(\"/c/Users/games3/.local/bin/uvx.exe\" markitdown-mcp --help)",
|
||||
"Bash(git stash:*)",
|
||||
"Bash(ping:*)",
|
||||
"Bash(tee:*)",
|
||||
"Bash(timeout 1800 podman exec flyer-crawler-dev npm run test:unit:*)",
|
||||
"mcp__filesystem__edit_file",
|
||||
"Bash(timeout 300 tail:*)",
|
||||
"mcp__filesystem__list_allowed_directories",
|
||||
"mcp__memory__add_observations",
|
||||
"Bash(ssh:*)",
|
||||
"mcp__redis__list",
|
||||
"Read(//d/gitea/bugsink-mcp/**)",
|
||||
"Bash(d:/nodejs/npm.cmd install)"
|
||||
]
|
||||
}
|
||||
}
|
||||
@@ -1,18 +1,96 @@
|
||||
{
|
||||
// ============================================================================
|
||||
// VS CODE DEV CONTAINER CONFIGURATION
|
||||
// ============================================================================
|
||||
// This file configures VS Code's Dev Containers extension to provide a
|
||||
// consistent, fully-configured development environment.
|
||||
//
|
||||
// Features:
|
||||
// - Automatic PostgreSQL + Redis startup with healthchecks
|
||||
// - Automatic npm install
|
||||
// - Automatic database schema initialization and seeding
|
||||
// - Pre-configured VS Code extensions (ESLint, Prettier)
|
||||
// - Podman support for Windows users
|
||||
//
|
||||
// Usage:
|
||||
// 1. Install the "Dev Containers" extension in VS Code
|
||||
// 2. Open this project folder
|
||||
// 3. Click "Reopen in Container" when prompted (or use Command Palette)
|
||||
// 4. Wait for container build and initialization
|
||||
// 5. Development server starts automatically
|
||||
// ============================================================================
|
||||
|
||||
"name": "Flyer Crawler Dev (Ubuntu 22.04)",
|
||||
|
||||
// Use Docker Compose for multi-container setup
|
||||
"dockerComposeFile": ["../compose.dev.yml"],
|
||||
"service": "app",
|
||||
"workspaceFolder": "/app",
|
||||
|
||||
// VS Code customizations
|
||||
"customizations": {
|
||||
"vscode": {
|
||||
"extensions": ["dbaeumer.vscode-eslint", "esbenp.prettier-vscode"]
|
||||
"extensions": [
|
||||
// Code quality
|
||||
"dbaeumer.vscode-eslint",
|
||||
"esbenp.prettier-vscode",
|
||||
// TypeScript
|
||||
"ms-vscode.vscode-typescript-next",
|
||||
// Database
|
||||
"mtxr.sqltools",
|
||||
"mtxr.sqltools-driver-pg",
|
||||
// Utilities
|
||||
"eamodio.gitlens",
|
||||
"streetsidesoftware.code-spell-checker"
|
||||
],
|
||||
"settings": {
|
||||
"editor.formatOnSave": true,
|
||||
"editor.defaultFormatter": "esbenp.prettier-vscode",
|
||||
"typescript.preferences.importModuleSpecifier": "relative"
|
||||
}
|
||||
}
|
||||
},
|
||||
|
||||
// Run as root (required for npm global installs)
|
||||
"remoteUser": "root",
|
||||
// Automatically install dependencies when the container is created.
|
||||
// This runs inside the container, populating the isolated node_modules volume.
|
||||
"postCreateCommand": "npm install",
|
||||
|
||||
// ============================================================================
|
||||
// Lifecycle Commands
|
||||
// ============================================================================
|
||||
|
||||
// initializeCommand: Runs on the HOST before the container is created.
|
||||
// Starts Podman machine on Windows (no-op if already running or using Docker).
|
||||
"initializeCommand": "powershell -Command \"podman machine start; exit 0\"",
|
||||
|
||||
// postCreateCommand: Runs ONCE when the container is first created.
|
||||
// This is where we do full initialization: npm install + database setup.
|
||||
"postCreateCommand": "chmod +x scripts/docker-init.sh && ./scripts/docker-init.sh",
|
||||
|
||||
// postAttachCommand: Runs EVERY TIME VS Code attaches to the container.
|
||||
// Starts the development server automatically.
|
||||
"postAttachCommand": "npm run dev:container",
|
||||
// Try to start podman machine, but exit with success (0) even if it's already running
|
||||
"initializeCommand": "powershell -Command \"podman machine start; exit 0\""
|
||||
|
||||
// ============================================================================
|
||||
// Port Forwarding
|
||||
// ============================================================================
|
||||
// Automatically forward these ports from the container to the host
|
||||
"forwardPorts": [3000, 3001],
|
||||
|
||||
// Labels for forwarded ports in VS Code's Ports panel
|
||||
"portsAttributes": {
|
||||
"3000": {
|
||||
"label": "Frontend (Vite)",
|
||||
"onAutoForward": "notify"
|
||||
},
|
||||
"3001": {
|
||||
"label": "Backend API",
|
||||
"onAutoForward": "notify"
|
||||
}
|
||||
},
|
||||
|
||||
// ============================================================================
|
||||
// Features
|
||||
// ============================================================================
|
||||
// Additional dev container features (optional)
|
||||
"features": {}
|
||||
}
|
||||
|
||||
104
.env.example
Normal file
104
.env.example
Normal file
@@ -0,0 +1,104 @@
|
||||
# .env.example
|
||||
# ============================================================================
|
||||
# ENVIRONMENT VARIABLES TEMPLATE
|
||||
# ============================================================================
|
||||
# Copy this file to .env and fill in your values.
|
||||
# For local development with Docker/Podman, these defaults should work out of the box.
|
||||
#
|
||||
# IMPORTANT: Never commit .env files with real credentials to version control!
|
||||
# ============================================================================
|
||||
|
||||
# ===================
|
||||
# Database Configuration
|
||||
# ===================
|
||||
# PostgreSQL connection settings
|
||||
# For container development, use the service name "postgres"
|
||||
DB_HOST=postgres
|
||||
DB_PORT=5432
|
||||
DB_USER=postgres
|
||||
DB_PASSWORD=postgres
|
||||
DB_NAME=flyer_crawler_dev
|
||||
|
||||
# ===================
|
||||
# Redis Configuration
|
||||
# ===================
|
||||
# Redis URL for caching and job queues
|
||||
# For container development, use the service name "redis"
|
||||
REDIS_URL=redis://redis:6379
|
||||
# Optional: Redis password (leave empty if not required)
|
||||
REDIS_PASSWORD=
|
||||
|
||||
# ===================
|
||||
# Application Settings
|
||||
# ===================
|
||||
NODE_ENV=development
|
||||
# Frontend URL for CORS and email links
|
||||
FRONTEND_URL=http://localhost:3000
|
||||
|
||||
# ===================
|
||||
# Authentication
|
||||
# ===================
|
||||
# REQUIRED: Secret key for signing JWT tokens (generate a random 64+ character string)
|
||||
JWT_SECRET=your-super-secret-jwt-key-change-this-in-production
|
||||
|
||||
# OAuth Providers (Optional - enable social login)
|
||||
# Google OAuth - https://console.cloud.google.com/apis/credentials
|
||||
GOOGLE_CLIENT_ID=
|
||||
GOOGLE_CLIENT_SECRET=
|
||||
# GitHub OAuth - https://github.com/settings/developers
|
||||
GITHUB_CLIENT_ID=
|
||||
GITHUB_CLIENT_SECRET=
|
||||
|
||||
# ===================
|
||||
# AI/ML Services
|
||||
# ===================
|
||||
# REQUIRED: Google Gemini API key for flyer OCR processing
|
||||
GEMINI_API_KEY=your-gemini-api-key
|
||||
|
||||
# ===================
|
||||
# External APIs
|
||||
# ===================
|
||||
# Optional: Google Maps API key for geocoding store addresses
|
||||
GOOGLE_MAPS_API_KEY=
|
||||
|
||||
# ===================
|
||||
# Email Configuration (Optional)
|
||||
# ===================
|
||||
# SMTP settings for sending emails (deal notifications, password reset)
|
||||
SMTP_HOST=
|
||||
SMTP_PORT=587
|
||||
SMTP_SECURE=false
|
||||
SMTP_USER=
|
||||
SMTP_PASS=
|
||||
SMTP_FROM_EMAIL=noreply@example.com
|
||||
|
||||
# ===================
|
||||
# Worker Configuration (Optional)
|
||||
# ===================
|
||||
# Concurrency settings for background job workers
|
||||
WORKER_CONCURRENCY=1
|
||||
EMAIL_WORKER_CONCURRENCY=10
|
||||
ANALYTICS_WORKER_CONCURRENCY=1
|
||||
CLEANUP_WORKER_CONCURRENCY=10
|
||||
|
||||
# Worker lock duration in milliseconds (default: 2 minutes)
|
||||
WORKER_LOCK_DURATION=120000
|
||||
|
||||
# ===================
|
||||
# Error Tracking (ADR-015)
|
||||
# ===================
|
||||
# Sentry-compatible error tracking via Bugsink (self-hosted)
|
||||
# DSNs are created in Bugsink UI at http://localhost:8000 (dev) or your production URL
|
||||
# Backend DSN - for Express/Node.js errors
|
||||
SENTRY_DSN=
|
||||
# Frontend DSN - for React/browser errors (uses VITE_ prefix)
|
||||
VITE_SENTRY_DSN=
|
||||
# Environment name for error grouping (defaults to NODE_ENV)
|
||||
SENTRY_ENVIRONMENT=development
|
||||
VITE_SENTRY_ENVIRONMENT=development
|
||||
# Enable/disable error tracking (default: true)
|
||||
SENTRY_ENABLED=true
|
||||
VITE_SENTRY_ENABLED=true
|
||||
# Enable debug mode for SDK troubleshooting (default: false)
|
||||
SENTRY_DEBUG=false
|
||||
VITE_SENTRY_DEBUG=false
|
||||
6
.env.test
Normal file
6
.env.test
Normal file
@@ -0,0 +1,6 @@
|
||||
DB_HOST=10.89.0.4
|
||||
DB_USER=flyer
|
||||
DB_PASSWORD=flyer
|
||||
DB_NAME=flyer_crawler_test
|
||||
REDIS_URL=redis://redis:6379
|
||||
NODE_ENV=test
|
||||
@@ -63,8 +63,8 @@ jobs:
|
||||
- name: Check for Production Database Schema Changes
|
||||
env:
|
||||
DB_HOST: ${{ secrets.DB_HOST }}
|
||||
DB_USER: ${{ secrets.DB_USER }}
|
||||
DB_PASSWORD: ${{ secrets.DB_PASSWORD }}
|
||||
DB_USER: ${{ secrets.DB_USER_PROD }}
|
||||
DB_PASSWORD: ${{ secrets.DB_PASSWORD_PROD }}
|
||||
DB_NAME: ${{ secrets.DB_DATABASE_PROD }}
|
||||
run: |
|
||||
if [ -z "$DB_HOST" ] || [ -z "$DB_USER" ] || [ -z "$DB_PASSWORD" ] || [ -z "$DB_NAME" ]; then
|
||||
@@ -98,6 +98,9 @@ jobs:
|
||||
VITE_APP_VERSION="$(date +'%Y%m%d-%H%M'):$(git rev-parse --short HEAD):$PACKAGE_VERSION" \
|
||||
VITE_APP_COMMIT_URL="$GITEA_SERVER_URL/${{ gitea.repository }}/commit/${{ gitea.sha }}" \
|
||||
VITE_APP_COMMIT_MESSAGE="$COMMIT_MESSAGE" \
|
||||
VITE_SENTRY_DSN="${{ secrets.VITE_SENTRY_DSN }}" \
|
||||
VITE_SENTRY_ENVIRONMENT="production" \
|
||||
VITE_SENTRY_ENABLED="true" \
|
||||
VITE_API_BASE_URL=/api VITE_API_KEY=${{ secrets.VITE_GOOGLE_GENAI_API_KEY }} npm run build
|
||||
|
||||
- name: Deploy Application to Production Server
|
||||
@@ -114,10 +117,11 @@ jobs:
|
||||
env:
|
||||
# --- Production Secrets Injection ---
|
||||
DB_HOST: ${{ secrets.DB_HOST }}
|
||||
DB_USER: ${{ secrets.DB_USER }}
|
||||
DB_PASSWORD: ${{ secrets.DB_PASSWORD }}
|
||||
DB_USER: ${{ secrets.DB_USER_PROD }}
|
||||
DB_PASSWORD: ${{ secrets.DB_PASSWORD_PROD }}
|
||||
DB_NAME: ${{ secrets.DB_DATABASE_PROD }}
|
||||
REDIS_URL: 'redis://localhost:6379'
|
||||
# Explicitly use database 0 for production (test uses database 1)
|
||||
REDIS_URL: 'redis://localhost:6379/0'
|
||||
REDIS_PASSWORD: ${{ secrets.REDIS_PASSWORD_PROD }}
|
||||
FRONTEND_URL: 'https://flyer-crawler.projectium.com'
|
||||
JWT_SECRET: ${{ secrets.JWT_SECRET }}
|
||||
@@ -129,6 +133,15 @@ jobs:
|
||||
SMTP_USER: ''
|
||||
SMTP_PASS: ''
|
||||
SMTP_FROM_EMAIL: 'noreply@flyer-crawler.projectium.com'
|
||||
# OAuth Providers
|
||||
GOOGLE_CLIENT_ID: ${{ secrets.GOOGLE_CLIENT_ID }}
|
||||
GOOGLE_CLIENT_SECRET: ${{ secrets.GOOGLE_CLIENT_SECRET }}
|
||||
GITHUB_CLIENT_ID: ${{ secrets.GH_CLIENT_ID }}
|
||||
GITHUB_CLIENT_SECRET: ${{ secrets.GH_CLIENT_SECRET }}
|
||||
# Sentry/Bugsink Error Tracking (ADR-015)
|
||||
SENTRY_DSN: ${{ secrets.SENTRY_DSN }}
|
||||
SENTRY_ENVIRONMENT: 'production'
|
||||
SENTRY_ENABLED: 'true'
|
||||
run: |
|
||||
if [ -z "$DB_HOST" ] || [ -z "$DB_USER" ] || [ -z "$DB_PASSWORD" ] || [ -z "$DB_NAME" ]; then
|
||||
echo "ERROR: One or more production database secrets (DB_HOST, DB_USER, DB_PASSWORD, DB_DATABASE_PROD) are not set."
|
||||
@@ -158,7 +171,7 @@ jobs:
|
||||
else
|
||||
echo "Version mismatch (Running: $RUNNING_VERSION -> Deployed: $NEW_VERSION) or app not running. Reloading PM2..."
|
||||
fi
|
||||
pm2 startOrReload ecosystem.config.cjs --env production --update-env && pm2 save
|
||||
pm2 startOrReload ecosystem.config.cjs --update-env && pm2 save
|
||||
echo "Production backend server reloaded successfully."
|
||||
else
|
||||
echo "Version $NEW_VERSION is already running. Skipping PM2 reload."
|
||||
@@ -185,7 +198,17 @@ jobs:
|
||||
- name: Show PM2 Environment for Production
|
||||
run: |
|
||||
echo "--- Displaying recent PM2 logs for flyer-crawler-api ---"
|
||||
sleep 5
|
||||
pm2 describe flyer-crawler-api || echo "Could not find production pm2 process."
|
||||
pm2 logs flyer-crawler-api --lines 20 --nostream || echo "Could not find production pm2 process."
|
||||
pm2 env flyer-crawler-api || echo "Could not find production pm2 process."
|
||||
sleep 5 # Wait a few seconds for the app to start and log its output.
|
||||
|
||||
# Resolve the PM2 ID dynamically to ensure we target the correct process
|
||||
PM2_ID=$(pm2 jlist | node -e "try { const list = JSON.parse(require('fs').readFileSync(0, 'utf-8')); const app = list.find(p => p.name === 'flyer-crawler-api'); console.log(app ? app.pm2_env.pm_id : ''); } catch(e) { console.log(''); }")
|
||||
|
||||
if [ -n "$PM2_ID" ]; then
|
||||
echo "Found process ID: $PM2_ID"
|
||||
pm2 describe "$PM2_ID" || echo "Failed to describe process $PM2_ID"
|
||||
pm2 logs "$PM2_ID" --lines 20 --nostream || echo "Failed to get logs for $PM2_ID"
|
||||
pm2 env "$PM2_ID" || echo "Failed to get env for $PM2_ID"
|
||||
else
|
||||
echo "Could not find process 'flyer-crawler-api' in pm2 list."
|
||||
pm2 list # Fallback to listing everything to help debug
|
||||
fi
|
||||
|
||||
@@ -96,6 +96,24 @@ jobs:
|
||||
# It prevents the accumulation of duplicate processes from previous test runs.
|
||||
node -e "const exec = require('child_process').execSync; try { const list = JSON.parse(exec('pm2 jlist').toString()); list.forEach(p => { if (p.name && p.name.endsWith('-test')) { console.log('Deleting test process: ' + p.name + ' (' + p.pm2_env.pm_id + ')'); try { exec('pm2 delete ' + p.pm2_env.pm_id); } catch(e) { console.error('Failed to delete ' + p.pm2_env.pm_id, e.message); } } }); console.log('✅ Test process cleanup complete.'); } catch (e) { if (e.stdout.toString().includes('No process found')) { console.log('No PM2 processes running, cleanup not needed.'); } else { console.error('Error cleaning up test processes:', e.message); } }" || true
|
||||
|
||||
- name: Flush Redis Test Database Before Tests
|
||||
# CRITICAL: Clear Redis database 1 (test database) to remove stale BullMQ jobs.
|
||||
# This prevents old jobs with outdated error messages from polluting test results.
|
||||
# NOTE: We use database 1 for tests to isolate from production (database 0).
|
||||
env:
|
||||
REDIS_PASSWORD: ${{ secrets.REDIS_PASSWORD_TEST }}
|
||||
run: |
|
||||
echo "--- Flushing Redis database 1 (test database) to remove stale jobs ---"
|
||||
if [ -z "$REDIS_PASSWORD" ]; then
|
||||
echo "⚠️ REDIS_PASSWORD_TEST not set, attempting flush without password..."
|
||||
redis-cli -n 1 FLUSHDB || echo "Redis flush failed (no password)"
|
||||
else
|
||||
redis-cli -a "$REDIS_PASSWORD" -n 1 FLUSHDB 2>/dev/null && echo "✅ Redis database 1 (test) flushed successfully." || echo "⚠️ Redis flush failed"
|
||||
fi
|
||||
# Verify the flush worked by checking key count on database 1
|
||||
KEY_COUNT=$(redis-cli -a "$REDIS_PASSWORD" -n 1 DBSIZE 2>/dev/null | grep -oE '[0-9]+' || echo "unknown")
|
||||
echo "Redis database 1 key count after flush: $KEY_COUNT"
|
||||
|
||||
- name: Run All Tests and Generate Merged Coverage Report
|
||||
# This single step runs both unit and integration tests, then merges their
|
||||
# coverage data into a single report. It combines the environment variables
|
||||
@@ -103,20 +121,30 @@ jobs:
|
||||
env:
|
||||
# --- Database credentials for the test suite ---
|
||||
# These are injected from Gitea secrets into the runner's environment.
|
||||
# CRITICAL: Use TEST-specific credentials that have CREATE privileges on the public schema.
|
||||
DB_HOST: ${{ secrets.DB_HOST }}
|
||||
DB_USER: ${{ secrets.DB_USER }}
|
||||
DB_PASSWORD: ${{ secrets.DB_PASSWORD }}
|
||||
DB_NAME: 'flyer-crawler-test' # Explicitly set for tests
|
||||
DB_USER: ${{ secrets.DB_USER_TEST }}
|
||||
DB_PASSWORD: ${{ secrets.DB_PASSWORD_TEST }}
|
||||
DB_NAME: ${{ secrets.DB_DATABASE_TEST }}
|
||||
|
||||
# --- Redis credentials for the test suite ---
|
||||
REDIS_URL: 'redis://localhost:6379'
|
||||
# CRITICAL: Use Redis database 1 to isolate tests from production (which uses db 0).
|
||||
# This prevents the production worker from picking up test jobs.
|
||||
REDIS_URL: 'redis://localhost:6379/1'
|
||||
REDIS_PASSWORD: ${{ secrets.REDIS_PASSWORD_TEST }}
|
||||
|
||||
# --- Integration test specific variables ---
|
||||
FRONTEND_URL: 'http://localhost:3000'
|
||||
FRONTEND_URL: 'https://example.com'
|
||||
VITE_API_BASE_URL: 'http://localhost:3001/api'
|
||||
GEMINI_API_KEY: ${{ secrets.VITE_GOOGLE_GENAI_API_KEY }}
|
||||
|
||||
# --- Storage path for flyer images ---
|
||||
# CRITICAL: Use an absolute path in the test runner's working directory for file storage.
|
||||
# This ensures tests can read processed files to verify their contents (e.g., EXIF stripping).
|
||||
# Without this, multer and flyerProcessingService default to /var/www/.../flyer-images.
|
||||
# NOTE: We use ${{ github.workspace }} which resolves to the checkout directory.
|
||||
STORAGE_PATH: '${{ github.workspace }}/flyer-images'
|
||||
|
||||
# --- JWT Secret for Passport authentication in tests ---
|
||||
JWT_SECRET: ${{ secrets.JWT_SECRET }}
|
||||
|
||||
@@ -171,8 +199,8 @@ jobs:
|
||||
--reporter=verbose --includeTaskLocation --testTimeout=10000 --silent=passed-only || true
|
||||
|
||||
echo "--- Running E2E Tests ---"
|
||||
# Run E2E tests using the dedicated E2E config which inherits from integration config.
|
||||
# We still pass --coverage to enable it, but directory and timeout are now in the config.
|
||||
# Run E2E tests using the dedicated E2E config.
|
||||
# E2E uses port 3098, integration uses 3099 to avoid conflicts.
|
||||
npx vitest run --config vitest.config.e2e.ts --coverage \
|
||||
--coverage.exclude='**/*.test.ts' \
|
||||
--coverage.exclude='**/tests/**' \
|
||||
@@ -213,7 +241,19 @@ jobs:
|
||||
# Run c8: read raw files from the temp dir, and output an Istanbul JSON report.
|
||||
# We only generate the 'json' report here because it's all nyc needs for merging.
|
||||
echo "Server coverage report about to be generated..."
|
||||
npx c8 report --exclude='**/*.test.ts' --exclude='**/tests/**' --exclude='**/mocks/**' --reporter=json --temp-directory .coverage/tmp/integration-server --reports-dir .coverage/integration-server
|
||||
npx c8 report \
|
||||
--include='src/**' \
|
||||
--exclude='**/*.test.ts' \
|
||||
--exclude='**/*.test.tsx' \
|
||||
--exclude='**/tests/**' \
|
||||
--exclude='**/mocks/**' \
|
||||
--exclude='hostexecutor/**' \
|
||||
--exclude='scripts/**' \
|
||||
--exclude='*.config.js' \
|
||||
--exclude='*.config.ts' \
|
||||
--reporter=json \
|
||||
--temp-directory .coverage/tmp/integration-server \
|
||||
--reports-dir .coverage/integration-server
|
||||
echo "Server coverage report generated. Verifying existence:"
|
||||
ls -l .coverage/integration-server/coverage-final.json
|
||||
|
||||
@@ -253,12 +293,18 @@ jobs:
|
||||
--reporter=html \
|
||||
--report-dir .coverage/ \
|
||||
--temp-dir "$NYC_SOURCE_DIR" \
|
||||
--include "src/**" \
|
||||
--exclude "**/*.test.ts" \
|
||||
--exclude "**/*.test.tsx" \
|
||||
--exclude "**/tests/**" \
|
||||
--exclude "**/mocks/**" \
|
||||
--exclude "**/index.tsx" \
|
||||
--exclude "**/vite-env.d.ts" \
|
||||
--exclude "**/vitest.setup.ts"
|
||||
--exclude "**/vitest.setup.ts" \
|
||||
--exclude "hostexecutor/**" \
|
||||
--exclude "scripts/**" \
|
||||
--exclude "*.config.js" \
|
||||
--exclude "*.config.ts"
|
||||
|
||||
# Re-enable secret masking for subsequent steps.
|
||||
echo "::secret-masking::"
|
||||
@@ -283,10 +329,11 @@ jobs:
|
||||
- name: Check for Test Database Schema Changes
|
||||
env:
|
||||
# Use test database credentials for this check.
|
||||
# CRITICAL: Use TEST-specific credentials that have CREATE privileges on the public schema.
|
||||
DB_HOST: ${{ secrets.DB_HOST }}
|
||||
DB_USER: ${{ secrets.DB_USER }}
|
||||
DB_PASSWORD: ${{ secrets.DB_PASSWORD }} # This is used by psql
|
||||
DB_NAME: ${{ secrets.DB_DATABASE_TEST }} # This is used by the application
|
||||
DB_USER: ${{ secrets.DB_USER_TEST }}
|
||||
DB_PASSWORD: ${{ secrets.DB_PASSWORD_TEST }}
|
||||
DB_NAME: ${{ secrets.DB_DATABASE_TEST }}
|
||||
run: |
|
||||
# Fail-fast check to ensure secrets are configured in Gitea.
|
||||
if [ -z "$DB_HOST" ] || [ -z "$DB_USER" ] || [ -z "$DB_PASSWORD" ] || [ -z "$DB_NAME" ]; then
|
||||
@@ -335,11 +382,15 @@ jobs:
|
||||
fi
|
||||
|
||||
GITEA_SERVER_URL="https://gitea.projectium.com" # Your Gitea instance URL
|
||||
COMMIT_MESSAGE=$(git log -1 --grep="\[skip ci\]" --invert-grep --pretty=%s)
|
||||
# Sanitize commit message to prevent shell injection or build breaks (removes quotes, backticks, backslashes, $)
|
||||
COMMIT_MESSAGE=$(git log -1 --grep="\[skip ci\]" --invert-grep --pretty=%s | tr -d '"`\\$')
|
||||
PACKAGE_VERSION=$(node -p "require('./package.json').version")
|
||||
VITE_APP_VERSION="$(date +'%Y%m%d-%H%M'):$(git rev-parse --short HEAD):$PACKAGE_VERSION" \
|
||||
VITE_APP_COMMIT_URL="$GITEA_SERVER_URL/${{ gitea.repository }}/commit/${{ gitea.sha }}" \
|
||||
VITE_APP_COMMIT_MESSAGE="$COMMIT_MESSAGE" \
|
||||
VITE_SENTRY_DSN="${{ secrets.VITE_SENTRY_DSN_TEST }}" \
|
||||
VITE_SENTRY_ENVIRONMENT="test" \
|
||||
VITE_SENTRY_ENABLED="true" \
|
||||
VITE_API_BASE_URL="https://flyer-crawler-test.projectium.com/api" VITE_API_KEY=${{ secrets.VITE_GOOGLE_GENAI_API_KEY_TEST }} npm run build
|
||||
|
||||
- name: Deploy Application to Test Server
|
||||
@@ -378,17 +429,18 @@ jobs:
|
||||
# Your Node.js application will read these directly from `process.env`.
|
||||
|
||||
# Database Credentials
|
||||
# CRITICAL: Use TEST-specific credentials that have CREATE privileges on the public schema.
|
||||
DB_HOST: ${{ secrets.DB_HOST }}
|
||||
DB_USER: ${{ secrets.DB_USER }}
|
||||
DB_PASSWORD: ${{ secrets.DB_PASSWORD }}
|
||||
DB_USER: ${{ secrets.DB_USER_TEST }}
|
||||
DB_PASSWORD: ${{ secrets.DB_PASSWORD_TEST }}
|
||||
DB_NAME: ${{ secrets.DB_DATABASE_TEST }}
|
||||
|
||||
# Redis Credentials
|
||||
REDIS_URL: 'redis://localhost:6379'
|
||||
# Redis Credentials (use database 1 to isolate from production)
|
||||
REDIS_URL: 'redis://localhost:6379/1'
|
||||
REDIS_PASSWORD: ${{ secrets.REDIS_PASSWORD_TEST }}
|
||||
|
||||
# Application Secrets
|
||||
FRONTEND_URL: 'https://flyer-crawler-test.projectium.com'
|
||||
FRONTEND_URL: 'https://example.com'
|
||||
JWT_SECRET: ${{ secrets.JWT_SECRET }}
|
||||
GEMINI_API_KEY: ${{ secrets.VITE_GOOGLE_GENAI_API_KEY_TEST }}
|
||||
GOOGLE_MAPS_API_KEY: ${{ secrets.GOOGLE_MAPS_API_KEY }}
|
||||
@@ -400,6 +452,10 @@ jobs:
|
||||
SMTP_USER: '' # Using MailHog, no auth needed
|
||||
SMTP_PASS: '' # Using MailHog, no auth needed
|
||||
SMTP_FROM_EMAIL: 'noreply@flyer-crawler-test.projectium.com'
|
||||
# Sentry/Bugsink Error Tracking (ADR-015)
|
||||
SENTRY_DSN: ${{ secrets.SENTRY_DSN_TEST }}
|
||||
SENTRY_ENVIRONMENT: 'test'
|
||||
SENTRY_ENABLED: 'true'
|
||||
|
||||
run: |
|
||||
# Fail-fast check to ensure secrets are configured in Gitea.
|
||||
@@ -423,10 +479,11 @@ jobs:
|
||||
echo "Cleaning up errored or stopped PM2 processes..."
|
||||
node -e "const exec = require('child_process').execSync; try { const list = JSON.parse(exec('pm2 jlist').toString()); list.forEach(p => { if (p.pm2_env.status === 'errored' || p.pm2_env.status === 'stopped') { console.log('Deleting ' + p.pm2_env.status + ' process: ' + p.name + ' (' + p.pm2_env.pm_id + ')'); try { exec('pm2 delete ' + p.pm2_env.pm_id); } catch(e) { console.error('Failed to delete ' + p.pm2_env.pm_id); } } }); } catch (e) { console.error('Error cleaning up processes:', e); }"
|
||||
|
||||
# Use `startOrReload` with the ecosystem file. This is the standard, idempotent way to deploy.
|
||||
# It will START the process if it's not running, or RELOAD it if it is.
|
||||
# Use `startOrReload` with the TEST ecosystem file. This starts test-specific processes
|
||||
# (flyer-crawler-api-test, flyer-crawler-worker-test, flyer-crawler-analytics-worker-test)
|
||||
# that run separately from production processes.
|
||||
# We also add `&& pm2 save` to persist the process list across server reboots.
|
||||
pm2 startOrReload ecosystem.config.cjs --env test --update-env && pm2 save
|
||||
pm2 startOrReload ecosystem-test.config.cjs --update-env && pm2 save
|
||||
echo "Test backend server reloaded successfully."
|
||||
|
||||
# After a successful deployment, update the schema hash in the database.
|
||||
@@ -461,7 +518,17 @@ jobs:
|
||||
run: |
|
||||
echo "--- Displaying recent PM2 logs for flyer-crawler-api-test ---"
|
||||
# After a reload, the server restarts. We'll show the last 20 lines of the log to see the startup messages.
|
||||
sleep 5 # Wait a few seconds for the app to start and log its output.
|
||||
pm2 describe flyer-crawler-api-test || echo "Could not find test pm2 process."
|
||||
pm2 logs flyer-crawler-api-test --lines 20 --nostream || echo "Could not find test pm2 process."
|
||||
pm2 env flyer-crawler-api-test || echo "Could not find test pm2 process."
|
||||
sleep 5
|
||||
|
||||
# Resolve the PM2 ID dynamically to ensure we target the correct process
|
||||
PM2_ID=$(pm2 jlist | node -e "try { const list = JSON.parse(require('fs').readFileSync(0, 'utf-8')); const app = list.find(p => p.name === 'flyer-crawler-api-test'); console.log(app ? app.pm2_env.pm_id : ''); } catch(e) { console.log(''); }")
|
||||
|
||||
if [ -n "$PM2_ID" ]; then
|
||||
echo "Found process ID: $PM2_ID"
|
||||
pm2 describe "$PM2_ID" || echo "Failed to describe process $PM2_ID"
|
||||
pm2 logs "$PM2_ID" --lines 20 --nostream || echo "Failed to get logs for $PM2_ID"
|
||||
pm2 env "$PM2_ID" || echo "Failed to get env for $PM2_ID"
|
||||
else
|
||||
echo "Could not find process 'flyer-crawler-api-test' in pm2 list."
|
||||
pm2 list # Fallback to listing everything to help debug
|
||||
fi
|
||||
|
||||
@@ -20,9 +20,9 @@ jobs:
|
||||
# Use production database credentials for this entire job.
|
||||
DB_HOST: ${{ secrets.DB_HOST }}
|
||||
DB_PORT: ${{ secrets.DB_PORT }}
|
||||
DB_USER: ${{ secrets.DB_USER }}
|
||||
DB_PASSWORD: ${{ secrets.DB_PASSWORD }}
|
||||
DB_NAME: ${{ secrets.DB_NAME_PROD }}
|
||||
DB_USER: ${{ secrets.DB_USER_PROD }}
|
||||
DB_PASSWORD: ${{ secrets.DB_PASSWORD_PROD }}
|
||||
DB_NAME: ${{ secrets.DB_DATABASE_PROD }}
|
||||
|
||||
steps:
|
||||
- name: Validate Secrets
|
||||
|
||||
@@ -23,9 +23,9 @@ jobs:
|
||||
env:
|
||||
# Use production database credentials for this entire job.
|
||||
DB_HOST: ${{ secrets.DB_HOST }}
|
||||
DB_USER: ${{ secrets.DB_USER }}
|
||||
DB_PASSWORD: ${{ secrets.DB_PASSWORD }} # Used by psql
|
||||
DB_NAME: ${{ secrets.DB_DATABASE_PROD }} # Used by the application
|
||||
DB_USER: ${{ secrets.DB_USER_PROD }}
|
||||
DB_PASSWORD: ${{ secrets.DB_PASSWORD_PROD }}
|
||||
DB_NAME: ${{ secrets.DB_DATABASE_PROD }}
|
||||
|
||||
steps:
|
||||
- name: Checkout Code
|
||||
|
||||
@@ -23,9 +23,9 @@ jobs:
|
||||
env:
|
||||
# Use test database credentials for this entire job.
|
||||
DB_HOST: ${{ secrets.DB_HOST }}
|
||||
DB_USER: ${{ secrets.DB_USER }}
|
||||
DB_PASSWORD: ${{ secrets.DB_PASSWORD }} # Used by psql
|
||||
DB_NAME: ${{ secrets.DB_DATABASE_TEST }} # Used by the application
|
||||
DB_USER: ${{ secrets.DB_USER_TEST }}
|
||||
DB_PASSWORD: ${{ secrets.DB_PASSWORD_TEST }}
|
||||
DB_NAME: ${{ secrets.DB_DATABASE_TEST }}
|
||||
|
||||
steps:
|
||||
- name: Checkout Code
|
||||
|
||||
@@ -22,8 +22,8 @@ jobs:
|
||||
env:
|
||||
# Use production database credentials for this entire job.
|
||||
DB_HOST: ${{ secrets.DB_HOST }}
|
||||
DB_USER: ${{ secrets.DB_USER }}
|
||||
DB_PASSWORD: ${{ secrets.DB_PASSWORD }}
|
||||
DB_USER: ${{ secrets.DB_USER_PROD }}
|
||||
DB_PASSWORD: ${{ secrets.DB_PASSWORD_PROD }}
|
||||
DB_NAME: ${{ secrets.DB_DATABASE_PROD }}
|
||||
BACKUP_DIR: '/var/www/backups' # Define a dedicated directory for backups
|
||||
|
||||
|
||||
@@ -62,8 +62,8 @@ jobs:
|
||||
- name: Check for Production Database Schema Changes
|
||||
env:
|
||||
DB_HOST: ${{ secrets.DB_HOST }}
|
||||
DB_USER: ${{ secrets.DB_USER }}
|
||||
DB_PASSWORD: ${{ secrets.DB_PASSWORD }}
|
||||
DB_USER: ${{ secrets.DB_USER_PROD }}
|
||||
DB_PASSWORD: ${{ secrets.DB_PASSWORD_PROD }}
|
||||
DB_NAME: ${{ secrets.DB_DATABASE_PROD }}
|
||||
run: |
|
||||
if [ -z "$DB_HOST" ] || [ -z "$DB_USER" ] || [ -z "$DB_PASSWORD" ] || [ -z "$DB_NAME" ]; then
|
||||
@@ -113,10 +113,11 @@ jobs:
|
||||
env:
|
||||
# --- Production Secrets Injection ---
|
||||
DB_HOST: ${{ secrets.DB_HOST }}
|
||||
DB_USER: ${{ secrets.DB_USER }}
|
||||
DB_PASSWORD: ${{ secrets.DB_PASSWORD }}
|
||||
DB_USER: ${{ secrets.DB_USER_PROD }}
|
||||
DB_PASSWORD: ${{ secrets.DB_PASSWORD_PROD }}
|
||||
DB_NAME: ${{ secrets.DB_DATABASE_PROD }}
|
||||
REDIS_URL: 'redis://localhost:6379'
|
||||
# Explicitly use database 0 for production (test uses database 1)
|
||||
REDIS_URL: 'redis://localhost:6379/0'
|
||||
REDIS_PASSWORD: ${{ secrets.REDIS_PASSWORD_PROD }}
|
||||
FRONTEND_URL: 'https://flyer-crawler.projectium.com'
|
||||
JWT_SECRET: ${{ secrets.JWT_SECRET }}
|
||||
|
||||
167
.gitea/workflows/manual-redis-flush-prod.yml
Normal file
167
.gitea/workflows/manual-redis-flush-prod.yml
Normal file
@@ -0,0 +1,167 @@
|
||||
# .gitea/workflows/manual-redis-flush-prod.yml
|
||||
#
|
||||
# DANGER: This workflow is DESTRUCTIVE and intended for manual execution only.
|
||||
# It will completely FLUSH the PRODUCTION Redis database (db 0).
|
||||
# This will clear all BullMQ queues, sessions, caches, and any other Redis data.
|
||||
#
|
||||
name: Manual - Flush Production Redis
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
confirmation:
|
||||
description: 'DANGER: This will FLUSH production Redis. Type "flush-production-redis" to confirm.'
|
||||
required: true
|
||||
default: 'do-not-run'
|
||||
flush_type:
|
||||
description: 'What to flush?'
|
||||
required: true
|
||||
type: choice
|
||||
options:
|
||||
- 'queues-only'
|
||||
- 'entire-database'
|
||||
default: 'queues-only'
|
||||
|
||||
jobs:
|
||||
flush-redis:
|
||||
runs-on: projectium.com # This job runs on your self-hosted Gitea runner.
|
||||
|
||||
env:
|
||||
REDIS_PASSWORD: ${{ secrets.REDIS_PASSWORD_PROD }}
|
||||
|
||||
steps:
|
||||
- name: Checkout Code
|
||||
uses: actions/checkout@v3
|
||||
|
||||
- name: Setup Node.js
|
||||
uses: actions/setup-node@v3
|
||||
with:
|
||||
node-version: '20'
|
||||
cache: 'npm'
|
||||
cache-dependency-path: '**/package-lock.json'
|
||||
|
||||
- name: Install Dependencies
|
||||
run: npm ci
|
||||
|
||||
- name: Validate Secrets
|
||||
run: |
|
||||
if [ -z "$REDIS_PASSWORD" ]; then
|
||||
echo "ERROR: REDIS_PASSWORD_PROD secret is not set in Gitea repository settings."
|
||||
exit 1
|
||||
fi
|
||||
echo "✅ Redis password secret is present."
|
||||
|
||||
- name: Verify Confirmation Phrase
|
||||
run: |
|
||||
if [ "${{ gitea.event.inputs.confirmation }}" != "flush-production-redis" ]; then
|
||||
echo "ERROR: Confirmation phrase did not match. Aborting Redis flush."
|
||||
exit 1
|
||||
fi
|
||||
echo "✅ Confirmation accepted. Proceeding with Redis flush."
|
||||
|
||||
- name: Show Current Redis State
|
||||
run: |
|
||||
echo "--- Current Redis Database 0 (Production) State ---"
|
||||
redis-cli -a "$REDIS_PASSWORD" -n 0 INFO keyspace 2>/dev/null || echo "Could not get keyspace info"
|
||||
echo ""
|
||||
echo "--- Key Count ---"
|
||||
KEY_COUNT=$(redis-cli -a "$REDIS_PASSWORD" -n 0 DBSIZE 2>/dev/null | grep -oE '[0-9]+' || echo "unknown")
|
||||
echo "Production Redis (db 0) key count: $KEY_COUNT"
|
||||
echo ""
|
||||
echo "--- BullMQ Queue Keys ---"
|
||||
redis-cli -a "$REDIS_PASSWORD" -n 0 KEYS "bull:*" 2>/dev/null | head -20 || echo "No BullMQ keys found"
|
||||
|
||||
- name: 🚨 FINAL WARNING & PAUSE 🚨
|
||||
run: |
|
||||
echo "*********************************************************************"
|
||||
echo "WARNING: YOU ARE ABOUT TO FLUSH PRODUCTION REDIS DATA."
|
||||
echo "Flush type: ${{ gitea.event.inputs.flush_type }}"
|
||||
echo ""
|
||||
if [ "${{ gitea.event.inputs.flush_type }}" = "entire-database" ]; then
|
||||
echo "This will DELETE ALL Redis data including sessions, caches, and queues!"
|
||||
else
|
||||
echo "This will DELETE ALL BullMQ queue data (pending jobs, failed jobs, etc.)"
|
||||
fi
|
||||
echo ""
|
||||
echo "This action is IRREVERSIBLE. Press Ctrl+C in the runner terminal NOW to cancel."
|
||||
echo "Sleeping for 10 seconds..."
|
||||
echo "*********************************************************************"
|
||||
sleep 10
|
||||
|
||||
- name: Flush BullMQ Queues Only
|
||||
if: ${{ gitea.event.inputs.flush_type == 'queues-only' }}
|
||||
env:
|
||||
REDIS_URL: 'redis://localhost:6379/0'
|
||||
run: |
|
||||
echo "--- Obliterating BullMQ queues using Node.js ---"
|
||||
node -e "
|
||||
const { Queue } = require('bullmq');
|
||||
const IORedis = require('ioredis');
|
||||
|
||||
const connection = new IORedis(process.env.REDIS_URL, {
|
||||
maxRetriesPerRequest: null,
|
||||
password: process.env.REDIS_PASSWORD,
|
||||
});
|
||||
|
||||
const queueNames = [
|
||||
'flyer-processing',
|
||||
'email-sending',
|
||||
'analytics-reporting',
|
||||
'weekly-analytics-reporting',
|
||||
'file-cleanup',
|
||||
'token-cleanup'
|
||||
];
|
||||
|
||||
(async () => {
|
||||
for (const name of queueNames) {
|
||||
try {
|
||||
const queue = new Queue(name, { connection });
|
||||
const counts = await queue.getJobCounts();
|
||||
console.log('Queue \"' + name + '\" before obliterate:', JSON.stringify(counts));
|
||||
await queue.obliterate({ force: true });
|
||||
console.log('✅ Obliterated queue: ' + name);
|
||||
await queue.close();
|
||||
} catch (err) {
|
||||
console.error('⚠️ Failed to obliterate queue ' + name + ':', err.message);
|
||||
}
|
||||
}
|
||||
await connection.quit();
|
||||
console.log('✅ All BullMQ queues obliterated.');
|
||||
})();
|
||||
"
|
||||
|
||||
- name: Flush Entire Redis Database
|
||||
if: ${{ gitea.event.inputs.flush_type == 'entire-database' }}
|
||||
run: |
|
||||
echo "--- Flushing entire Redis database 0 (production) ---"
|
||||
redis-cli -a "$REDIS_PASSWORD" -n 0 FLUSHDB 2>/dev/null && echo "✅ Redis database 0 flushed successfully." || echo "❌ Redis flush failed"
|
||||
|
||||
- name: Verify Flush Results
|
||||
run: |
|
||||
echo "--- Redis Database 0 (Production) State After Flush ---"
|
||||
KEY_COUNT=$(redis-cli -a "$REDIS_PASSWORD" -n 0 DBSIZE 2>/dev/null | grep -oE '[0-9]+' || echo "unknown")
|
||||
echo "Production Redis (db 0) key count after flush: $KEY_COUNT"
|
||||
echo ""
|
||||
echo "--- Remaining BullMQ Queue Keys ---"
|
||||
BULL_KEYS=$(redis-cli -a "$REDIS_PASSWORD" -n 0 KEYS "bull:*" 2>/dev/null | wc -l || echo "0")
|
||||
echo "BullMQ key count: $BULL_KEYS"
|
||||
|
||||
if [ "${{ gitea.event.inputs.flush_type }}" = "queues-only" ] && [ "$BULL_KEYS" -gt 0 ]; then
|
||||
echo "⚠️ Warning: Some BullMQ keys may still exist. This can happen if new jobs were added during the flush."
|
||||
fi
|
||||
|
||||
- name: Summary
|
||||
run: |
|
||||
echo ""
|
||||
echo "=========================================="
|
||||
echo "PRODUCTION REDIS FLUSH COMPLETE"
|
||||
echo "=========================================="
|
||||
echo "Flush type: ${{ gitea.event.inputs.flush_type }}"
|
||||
echo "Timestamp: $(date -u '+%Y-%m-%d %H:%M:%S UTC')"
|
||||
echo ""
|
||||
echo "NOTE: If you flushed queues, any pending jobs (flyer processing,"
|
||||
echo "emails, analytics, etc.) have been permanently deleted."
|
||||
echo ""
|
||||
echo "The production workers will automatically start processing"
|
||||
echo "new jobs as they are added to the queues."
|
||||
echo "=========================================="
|
||||
16
.gitignore
vendored
16
.gitignore
vendored
@@ -11,6 +11,18 @@ node_modules
|
||||
dist
|
||||
dist-ssr
|
||||
*.local
|
||||
.env
|
||||
*.tsbuildinfo
|
||||
|
||||
# Test coverage
|
||||
coverage
|
||||
.nyc_output
|
||||
.coverage
|
||||
|
||||
# Test artifacts - flyer-images/ is a runtime directory
|
||||
# Test fixtures are stored in src/tests/assets/ instead
|
||||
flyer-images/
|
||||
test-output.txt
|
||||
|
||||
# Editor directories and files
|
||||
.vscode/*
|
||||
@@ -22,3 +34,7 @@ dist-ssr
|
||||
*.njsproj
|
||||
*.sln
|
||||
*.sw?
|
||||
Thumbs.db
|
||||
.claude
|
||||
nul
|
||||
tmpclaude*
|
||||
|
||||
1
.husky/pre-commit
Normal file
1
.husky/pre-commit
Normal file
@@ -0,0 +1 @@
|
||||
npx lint-staged
|
||||
4
.lintstagedrc.json
Normal file
4
.lintstagedrc.json
Normal file
@@ -0,0 +1,4 @@
|
||||
{
|
||||
"*.{js,jsx,ts,tsx}": ["eslint --fix", "prettier --write"],
|
||||
"*.{json,md,css,html,yml,yaml}": ["prettier --write"]
|
||||
}
|
||||
5
.nycrc.json
Normal file
5
.nycrc.json
Normal file
@@ -0,0 +1,5 @@
|
||||
{
|
||||
"text": {
|
||||
"maxCols": 200
|
||||
}
|
||||
}
|
||||
41
.prettierignore
Normal file
41
.prettierignore
Normal file
@@ -0,0 +1,41 @@
|
||||
# Dependencies
|
||||
node_modules/
|
||||
|
||||
# Build output
|
||||
dist/
|
||||
build/
|
||||
.cache/
|
||||
|
||||
# Coverage reports
|
||||
coverage/
|
||||
.coverage/
|
||||
|
||||
# IDE and editor configs
|
||||
.idea/
|
||||
.vscode/
|
||||
*.swp
|
||||
*.swo
|
||||
|
||||
# Logs
|
||||
*.log
|
||||
logs/
|
||||
|
||||
# Environment files (may contain secrets)
|
||||
.env*
|
||||
!.env.example
|
||||
|
||||
# Lock files (managed by package managers)
|
||||
package-lock.json
|
||||
pnpm-lock.yaml
|
||||
yarn.lock
|
||||
|
||||
# Generated files
|
||||
*.min.js
|
||||
*.min.css
|
||||
|
||||
# Git directory
|
||||
.git/
|
||||
.gitea/
|
||||
|
||||
# Test artifacts
|
||||
__snapshots__/
|
||||
110
AUTHENTICATION.md
Normal file
110
AUTHENTICATION.md
Normal file
@@ -0,0 +1,110 @@
|
||||
# Authentication Setup
|
||||
|
||||
Flyer Crawler supports OAuth authentication via Google and GitHub. This guide walks through configuring both providers.
|
||||
|
||||
---
|
||||
|
||||
## Google OAuth
|
||||
|
||||
### Step 1: Create OAuth Credentials
|
||||
|
||||
1. Go to the [Google Cloud Console](https://console.cloud.google.com/)
|
||||
2. Create a new project (or select an existing one)
|
||||
3. Navigate to **APIs & Services > Credentials**
|
||||
4. Click **Create Credentials > OAuth client ID**
|
||||
5. Select **Web application** as the application type
|
||||
|
||||
### Step 2: Configure Authorized Redirect URIs
|
||||
|
||||
Add the callback URL where Google will redirect users after authentication:
|
||||
|
||||
| Environment | Redirect URI |
|
||||
| ----------- | -------------------------------------------------- |
|
||||
| Development | `http://localhost:3001/api/auth/google/callback` |
|
||||
| Production | `https://your-domain.com/api/auth/google/callback` |
|
||||
|
||||
### Step 3: Save Credentials
|
||||
|
||||
After clicking **Create**, you'll receive:
|
||||
|
||||
- **Client ID**
|
||||
- **Client Secret**
|
||||
|
||||
Store these securely as environment variables:
|
||||
|
||||
- `GOOGLE_CLIENT_ID`
|
||||
- `GOOGLE_CLIENT_SECRET`
|
||||
|
||||
---
|
||||
|
||||
## GitHub OAuth
|
||||
|
||||
### Step 1: Create OAuth App
|
||||
|
||||
1. Go to your [GitHub Developer Settings](https://github.com/settings/developers)
|
||||
2. Navigate to **OAuth Apps**
|
||||
3. Click **New OAuth App**
|
||||
|
||||
### Step 2: Fill in Application Details
|
||||
|
||||
| Field | Value |
|
||||
| -------------------------- | ---------------------------------------------------- |
|
||||
| Application name | Flyer Crawler (or your preferred name) |
|
||||
| Homepage URL | `http://localhost:5173` (dev) or your production URL |
|
||||
| Authorization callback URL | `http://localhost:3001/api/auth/github/callback` |
|
||||
|
||||
### Step 3: Save GitHub Credentials
|
||||
|
||||
After clicking **Register application**, you'll receive:
|
||||
|
||||
- **Client ID**
|
||||
- **Client Secret**
|
||||
|
||||
Store these securely as environment variables:
|
||||
|
||||
- `GITHUB_CLIENT_ID`
|
||||
- `GITHUB_CLIENT_SECRET`
|
||||
|
||||
---
|
||||
|
||||
## Environment Variables Summary
|
||||
|
||||
| Variable | Description |
|
||||
| ---------------------- | ---------------------------------------- |
|
||||
| `GOOGLE_CLIENT_ID` | Google OAuth client ID |
|
||||
| `GOOGLE_CLIENT_SECRET` | Google OAuth client secret |
|
||||
| `GITHUB_CLIENT_ID` | GitHub OAuth client ID |
|
||||
| `GITHUB_CLIENT_SECRET` | GitHub OAuth client secret |
|
||||
| `JWT_SECRET` | Secret for signing authentication tokens |
|
||||
|
||||
---
|
||||
|
||||
## Production Considerations
|
||||
|
||||
When deploying to production:
|
||||
|
||||
1. **Update redirect URIs** in both Google Cloud Console and GitHub OAuth settings to use your production domain
|
||||
2. **Use HTTPS** for all callback URLs in production
|
||||
3. **Store secrets securely** using your CI/CD platform's secrets management (e.g., Gitea repository secrets)
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### "redirect_uri_mismatch" Error
|
||||
|
||||
The callback URL in your OAuth provider settings doesn't match what the application is sending. Verify:
|
||||
|
||||
- The URL is exactly correct (no trailing slashes, correct port)
|
||||
- You're using the right environment (dev vs production URLs)
|
||||
|
||||
### "invalid_client" Error
|
||||
|
||||
The Client ID or Client Secret is incorrect. Double-check your environment variables.
|
||||
|
||||
---
|
||||
|
||||
## Related Documentation
|
||||
|
||||
- [Installation Guide](INSTALL.md) - Local development setup
|
||||
- [Deployment Guide](DEPLOYMENT.md) - Production deployment
|
||||
378
CLAUDE-MCP.md
Normal file
378
CLAUDE-MCP.md
Normal file
@@ -0,0 +1,378 @@
|
||||
# Claude Code MCP Configuration Guide
|
||||
|
||||
This document explains how to configure MCP (Model Context Protocol) servers for Claude Code, covering both the CLI and VS Code extension.
|
||||
|
||||
## The Two Config Files
|
||||
|
||||
Claude Code uses **two separate configuration files** for MCP servers. They must be kept in sync manually.
|
||||
|
||||
| File | Used By | Notes |
|
||||
| ------------------------- | ----------------------------- | ------------------------------------------- |
|
||||
| `~/.claude.json` | Claude CLI (`claude` command) | Requires `"type": "stdio"` in each server |
|
||||
| `~/.claude/settings.json` | VS Code Extension | Simpler format, supports `"disabled": true` |
|
||||
|
||||
**Important:** Changes to one file do NOT automatically sync to the other!
|
||||
|
||||
## File Locations (Windows)
|
||||
|
||||
```text
|
||||
C:\Users\<username>\.claude.json # CLI config
|
||||
C:\Users\<username>\.claude\settings.json # VS Code extension config
|
||||
```
|
||||
|
||||
## Config Format Differences
|
||||
|
||||
### VS Code Extension Format (`~/.claude/settings.json`)
|
||||
|
||||
```json
|
||||
{
|
||||
"mcpServers": {
|
||||
"server-name": {
|
||||
"command": "path/to/executable",
|
||||
"args": ["arg1", "arg2"],
|
||||
"env": {
|
||||
"ENV_VAR": "value"
|
||||
},
|
||||
"disabled": true // Optional - disable without removing
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### CLI Format (`~/.claude.json`)
|
||||
|
||||
The CLI config is a larger file with many settings. The `mcpServers` section is nested within it:
|
||||
|
||||
```json
|
||||
{
|
||||
"numStartups": 14,
|
||||
"installMethod": "global",
|
||||
// ... other settings ...
|
||||
"mcpServers": {
|
||||
"server-name": {
|
||||
"type": "stdio", // REQUIRED for CLI
|
||||
"command": "path/to/executable",
|
||||
"args": ["arg1", "arg2"],
|
||||
"env": {
|
||||
"ENV_VAR": "value"
|
||||
}
|
||||
}
|
||||
}
|
||||
// ... more settings ...
|
||||
}
|
||||
```
|
||||
|
||||
**Key difference:** CLI format requires `"type": "stdio"` in each server definition.
|
||||
|
||||
## Common MCP Server Examples
|
||||
|
||||
### Memory (Knowledge Graph)
|
||||
|
||||
```json
|
||||
// VS Code format
|
||||
"memory": {
|
||||
"command": "D:\\nodejs\\npx.cmd",
|
||||
"args": ["-y", "@modelcontextprotocol/server-memory"]
|
||||
}
|
||||
|
||||
// CLI format
|
||||
"memory": {
|
||||
"type": "stdio",
|
||||
"command": "D:\\nodejs\\npx.cmd",
|
||||
"args": ["-y", "@modelcontextprotocol/server-memory"],
|
||||
"env": {}
|
||||
}
|
||||
```
|
||||
|
||||
### Filesystem
|
||||
|
||||
```json
|
||||
// VS Code format
|
||||
"filesystem": {
|
||||
"command": "d:\\nodejs\\node.exe",
|
||||
"args": [
|
||||
"c:\\Users\\<user>\\AppData\\Roaming\\npm\\node_modules\\@modelcontextprotocol\\server-filesystem\\dist\\index.js",
|
||||
"d:\\path\\to\\project"
|
||||
]
|
||||
}
|
||||
|
||||
// CLI format
|
||||
"filesystem": {
|
||||
"type": "stdio",
|
||||
"command": "d:\\nodejs\\node.exe",
|
||||
"args": [
|
||||
"c:\\Users\\<user>\\AppData\\Roaming\\npm\\node_modules\\@modelcontextprotocol\\server-filesystem\\dist\\index.js",
|
||||
"d:\\path\\to\\project"
|
||||
],
|
||||
"env": {}
|
||||
}
|
||||
```
|
||||
|
||||
### Podman/Docker
|
||||
|
||||
```json
|
||||
// VS Code format
|
||||
"podman": {
|
||||
"command": "D:\\nodejs\\npx.cmd",
|
||||
"args": ["-y", "podman-mcp-server@latest"],
|
||||
"env": {
|
||||
"DOCKER_HOST": "npipe:////./pipe/podman-machine-default"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Gitea
|
||||
|
||||
```json
|
||||
// VS Code format
|
||||
"gitea-myserver": {
|
||||
"command": "d:\\gitea-mcp\\gitea-mcp.exe",
|
||||
"args": ["run", "-t", "stdio"],
|
||||
"env": {
|
||||
"GITEA_HOST": "https://gitea.example.com",
|
||||
"GITEA_ACCESS_TOKEN": "your-token-here"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Redis
|
||||
|
||||
```json
|
||||
// VS Code format
|
||||
"redis": {
|
||||
"command": "D:\\nodejs\\npx.cmd",
|
||||
"args": ["-y", "@modelcontextprotocol/server-redis", "redis://localhost:6379"]
|
||||
}
|
||||
```
|
||||
|
||||
### Bugsink (Error Tracking)
|
||||
|
||||
**Important:** Bugsink has a different API than Sentry. Use `bugsink-mcp`, NOT `sentry-selfhosted-mcp`.
|
||||
|
||||
**Note:** The `bugsink-mcp` npm package is NOT published. You must clone and build from source:
|
||||
|
||||
```bash
|
||||
# Clone and build bugsink-mcp
|
||||
git clone https://github.com/j-shelfwood/bugsink-mcp.git d:\gitea\bugsink-mcp
|
||||
cd d:\gitea\bugsink-mcp
|
||||
npm install
|
||||
npm run build
|
||||
```
|
||||
|
||||
```json
|
||||
// VS Code format (using locally built version)
|
||||
"bugsink": {
|
||||
"command": "d:\\nodejs\\node.exe",
|
||||
"args": ["d:\\gitea\\bugsink-mcp\\dist\\index.js"],
|
||||
"env": {
|
||||
"BUGSINK_URL": "https://bugsink.example.com",
|
||||
"BUGSINK_TOKEN": "your-api-token"
|
||||
}
|
||||
}
|
||||
|
||||
// CLI format
|
||||
"bugsink": {
|
||||
"type": "stdio",
|
||||
"command": "d:\\nodejs\\node.exe",
|
||||
"args": ["d:\\gitea\\bugsink-mcp\\dist\\index.js"],
|
||||
"env": {
|
||||
"BUGSINK_URL": "https://bugsink.example.com",
|
||||
"BUGSINK_TOKEN": "your-api-token"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
- GitHub: <https://github.com/j-shelfwood/bugsink-mcp>
|
||||
- Get token from Bugsink UI: Settings > API Tokens
|
||||
- **Do NOT use npx** - the package is not on npm
|
||||
|
||||
### Sentry (Cloud or Self-hosted)
|
||||
|
||||
For actual Sentry instances (not Bugsink), use:
|
||||
|
||||
```json
|
||||
"sentry": {
|
||||
"command": "D:\\nodejs\\npx.cmd",
|
||||
"args": ["-y", "@sentry/mcp-server"],
|
||||
"env": {
|
||||
"SENTRY_AUTH_TOKEN": "your-sentry-token"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Server Not Loading
|
||||
|
||||
1. **Check both config files** - Make sure the server is defined in both `~/.claude.json` AND `~/.claude/settings.json`
|
||||
|
||||
2. **Verify server order** - Servers load sequentially. Broken/slow servers can block others. Put important servers first.
|
||||
|
||||
3. **Check for timeout** - Each server has 30 seconds to connect. Slow npx downloads can cause timeouts.
|
||||
|
||||
4. **Fully restart VS Code** - Window reload is not enough. Close all VS Code windows and reopen.
|
||||
|
||||
### Verifying Configuration
|
||||
|
||||
**For CLI:**
|
||||
|
||||
```bash
|
||||
claude mcp list
|
||||
```
|
||||
|
||||
**For VS Code:**
|
||||
|
||||
1. Open VS Code
|
||||
2. View → Output
|
||||
3. Select "Claude" from the dropdown
|
||||
4. Look for MCP server connection logs
|
||||
|
||||
### Common Errors
|
||||
|
||||
| Error | Cause | Solution |
|
||||
| ------------------------------------ | ----------------------------- | --------------------------------------------------------------------------- |
|
||||
| `Connection timed out after 30000ms` | Server took too long to start | Move server earlier in config, or use pre-installed packages instead of npx |
|
||||
| `npm error 404 Not Found` | Package doesn't exist | Check package name spelling |
|
||||
| `The system cannot find the path` | Wrong executable path | Verify the command path exists |
|
||||
| `Connection closed` | Server crashed on startup | Check server logs, verify environment variables |
|
||||
|
||||
### Disabling Problem Servers
|
||||
|
||||
In `~/.claude/settings.json`, add `"disabled": true`:
|
||||
|
||||
```json
|
||||
"problem-server": {
|
||||
"command": "...",
|
||||
"args": ["..."],
|
||||
"disabled": true
|
||||
}
|
||||
```
|
||||
|
||||
**Note:** The CLI config (`~/.claude.json`) does not support the `disabled` flag. You must remove the server entirely from that file.
|
||||
|
||||
## Adding a New MCP Server
|
||||
|
||||
1. **Install/clone the MCP server** (if not using npx)
|
||||
|
||||
2. **Add to VS Code config** (`~/.claude/settings.json`):
|
||||
|
||||
```json
|
||||
"new-server": {
|
||||
"command": "path/to/command",
|
||||
"args": ["arg1", "arg2"],
|
||||
"env": { "VAR": "value" }
|
||||
}
|
||||
```
|
||||
|
||||
3. **Add to CLI config** (`~/.claude.json`) - find the `mcpServers` section:
|
||||
|
||||
```json
|
||||
"new-server": {
|
||||
"type": "stdio",
|
||||
"command": "path/to/command",
|
||||
"args": ["arg1", "arg2"],
|
||||
"env": { "VAR": "value" }
|
||||
}
|
||||
```
|
||||
|
||||
4. **Fully restart VS Code**
|
||||
|
||||
5. **Verify with `claude mcp list`**
|
||||
|
||||
## Quick Reference: Available MCP Servers
|
||||
|
||||
| Server | Package/Repo | Purpose |
|
||||
| ------------------- | -------------------------------------------------- | --------------------------- |
|
||||
| memory | `@modelcontextprotocol/server-memory` | Knowledge graph persistence |
|
||||
| filesystem | `@modelcontextprotocol/server-filesystem` | File system access |
|
||||
| redis | `@modelcontextprotocol/server-redis` | Redis cache inspection |
|
||||
| postgres | `@modelcontextprotocol/server-postgres` | PostgreSQL queries |
|
||||
| sequential-thinking | `@modelcontextprotocol/server-sequential-thinking` | Step-by-step reasoning |
|
||||
| podman | `podman-mcp-server` | Container management |
|
||||
| gitea | `gitea-mcp` (binary) | Gitea API access |
|
||||
| bugsink | `j-shelfwood/bugsink-mcp` (build from source) | Error tracking for Bugsink |
|
||||
| sentry | `@sentry/mcp-server` | Error tracking for Sentry |
|
||||
| playwright | `@anthropics/mcp-server-playwright` | Browser automation |
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **Keep configs in sync** - When you change one file, update the other
|
||||
|
||||
2. **Order servers by importance** - Put essential servers (memory, filesystem) first
|
||||
|
||||
3. **Disable instead of delete** - Use `"disabled": true` in settings.json to troubleshoot
|
||||
|
||||
4. **Use node.exe directly** - For faster startup, install packages globally and use `node.exe` instead of `npx`
|
||||
|
||||
5. **Store sensitive data in memory** - Use the memory MCP to store API tokens and config for future sessions
|
||||
|
||||
---
|
||||
|
||||
## Future: MCP Launchpad
|
||||
|
||||
**Project:** <https://github.com/kenneth-liao/mcp-launchpad>
|
||||
|
||||
MCP Launchpad is a CLI tool that wraps multiple MCP servers into a single interface. Worth revisiting when:
|
||||
|
||||
- [ ] Windows support is stable (currently experimental)
|
||||
- [ ] Available as an MCP server itself (currently Bash-based)
|
||||
|
||||
**Why it's interesting:**
|
||||
|
||||
| Benefit | Description |
|
||||
| ---------------------- | -------------------------------------------------------------- |
|
||||
| Single config file | No more syncing `~/.claude.json` and `~/.claude/settings.json` |
|
||||
| Project-level configs | Drop `mcp.json` in any project for instant MCP setup |
|
||||
| Context window savings | One MCP server in context instead of 10+, reducing token usage |
|
||||
| Persistent daemon | Keeps server connections alive for faster repeated calls |
|
||||
| Tool search | Find tools across all servers with `mcpl search` |
|
||||
|
||||
**Current limitations:**
|
||||
|
||||
- Experimental Windows support
|
||||
- Requires Python 3.13+ and uv
|
||||
- Claude calls tools via Bash instead of native MCP integration
|
||||
- Different mental model (runtime discovery vs startup loading)
|
||||
|
||||
---
|
||||
|
||||
## Future: Graphiti (Advanced Knowledge Graph)
|
||||
|
||||
**Project:** <https://github.com/getzep/graphiti>
|
||||
|
||||
Graphiti provides temporal-aware knowledge graphs - it tracks not just facts, but _when_ they became true/outdated. Much more powerful than simple memory MCP, but requires significant infrastructure.
|
||||
|
||||
**Ideal setup:** Run on a Linux server, connect via HTTP from Windows:
|
||||
|
||||
```json
|
||||
// Windows client config (settings.json)
|
||||
"graphiti": {
|
||||
"type": "sse",
|
||||
"url": "http://linux-server:8000/mcp/"
|
||||
}
|
||||
```
|
||||
|
||||
**Linux server setup:**
|
||||
|
||||
```bash
|
||||
git clone https://github.com/getzep/graphiti.git
|
||||
cd graphiti/mcp_server
|
||||
docker compose up -d # Starts FalkorDB + MCP server on port 8000
|
||||
```
|
||||
|
||||
**Requirements:**
|
||||
|
||||
- Docker on Linux server
|
||||
- OpenAI API key (for embeddings)
|
||||
- Port 8000 open on LAN
|
||||
|
||||
**Benefits of remote deployment:**
|
||||
|
||||
- Heavy lifting (Neo4j/FalkorDB + embeddings) offloaded to Linux
|
||||
- Always-on server, Windows connects/disconnects freely
|
||||
- Multiple machines can share the same knowledge graph
|
||||
- Avoids Windows Docker/WSL2 complexity
|
||||
|
||||
---
|
||||
|
||||
\_Last updated: January 2026
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user