integration test fixes - claude for the win? try 4 - i have a good feeling
Some checks failed
Deploy to Test Environment / deploy-to-test (push) Failing after 16m58s
Some checks failed
Deploy to Test Environment / deploy-to-test (push) Failing after 16m58s
This commit is contained in:
199
docs/adr/0031-data-retention-and-privacy-compliance.md
Normal file
199
docs/adr/0031-data-retention-and-privacy-compliance.md
Normal file
@@ -0,0 +1,199 @@
|
||||
# ADR-031: Data Retention and Privacy Compliance (GDPR/CCPA)
|
||||
|
||||
**Date**: 2026-01-09
|
||||
|
||||
**Status**: Proposed
|
||||
|
||||
## Context
|
||||
|
||||
The application stores various types of user data:
|
||||
|
||||
1. **User Accounts**: Email, password hash, profile information
|
||||
2. **Shopping Lists**: Personal shopping preferences and history
|
||||
3. **Watch Lists**: Tracked items and price alerts
|
||||
4. **Activity Logs**: User actions for analytics and debugging
|
||||
5. **Tracking Data**: Page views, interactions, feature usage
|
||||
|
||||
Current gaps in privacy compliance:
|
||||
|
||||
- **No Data Retention Policies**: Activity logs accumulate indefinitely
|
||||
- **No User Data Export**: Users cannot export their data (GDPR Article 20)
|
||||
- **No User Data Deletion**: No self-service account deletion (GDPR Article 17)
|
||||
- **No Cookie Consent**: Cookie usage not disclosed or consented
|
||||
- **No Privacy Policy Enforcement**: Privacy commitments not enforced in code
|
||||
|
||||
These gaps create legal exposure for users in EU (GDPR) and California (CCPA).
|
||||
|
||||
## Decision
|
||||
|
||||
We will implement comprehensive data retention and privacy compliance features.
|
||||
|
||||
### 1. Data Retention Policies
|
||||
|
||||
| Data Type | Retention Period | Deletion Method |
|
||||
| ------------------------- | ------------------------ | ------------------------ |
|
||||
| **Activity Logs** | 90 days | Automated cleanup job |
|
||||
| **Tracking Events** | 30 days | Automated cleanup job |
|
||||
| **Deleted User Data** | 30 days (soft delete) | Hard delete after period |
|
||||
| **Expired Sessions** | 7 days after expiry | Token cleanup job |
|
||||
| **Failed Login Attempts** | 24 hours | Automated cleanup |
|
||||
| **Flyer Data** | Indefinite (public data) | N/A |
|
||||
| **User Shopping Lists** | Until account deletion | With account |
|
||||
| **User Watch Lists** | Until account deletion | With account |
|
||||
|
||||
### 2. User Data Export (Right to Portability)
|
||||
|
||||
Implement `GET /api/users/me/export` endpoint:
|
||||
|
||||
```typescript
|
||||
interface UserDataExport {
|
||||
exportDate: string;
|
||||
user: {
|
||||
email: string;
|
||||
created_at: string;
|
||||
profile: ProfileData;
|
||||
};
|
||||
shoppingLists: ShoppingList[];
|
||||
watchedItems: WatchedItem[];
|
||||
priceAlerts: PriceAlert[];
|
||||
achievements: Achievement[];
|
||||
// Exclude: password hash, internal IDs, admin flags
|
||||
}
|
||||
```
|
||||
|
||||
Export formats: JSON (primary), CSV (optional)
|
||||
|
||||
### 3. User Data Deletion (Right to Erasure)
|
||||
|
||||
Implement `DELETE /api/users/me` endpoint:
|
||||
|
||||
1. **Soft Delete**: Mark account as deleted, anonymize PII
|
||||
2. **Grace Period**: 30 days to restore account
|
||||
3. **Hard Delete**: Permanently remove all user data after grace period
|
||||
4. **Audit Log**: Record deletion request (anonymized)
|
||||
|
||||
Deletion cascade:
|
||||
|
||||
- User account → Anonymize email/name
|
||||
- Shopping lists → Delete
|
||||
- Watch lists → Delete
|
||||
- Achievements → Delete
|
||||
- Activity logs → Anonymize user_id
|
||||
- Sessions/tokens → Delete immediately
|
||||
|
||||
### 4. Cookie Consent
|
||||
|
||||
Implement cookie consent banner:
|
||||
|
||||
```typescript
|
||||
// Cookie categories
|
||||
enum CookieCategory {
|
||||
ESSENTIAL = 'essential', // Always allowed (auth, CSRF)
|
||||
FUNCTIONAL = 'functional', // Dark mode, preferences
|
||||
ANALYTICS = 'analytics', // Usage tracking
|
||||
}
|
||||
|
||||
// Store consent in localStorage and server-side
|
||||
interface CookieConsent {
|
||||
essential: true; // Cannot be disabled
|
||||
functional: boolean;
|
||||
analytics: boolean;
|
||||
consentDate: string;
|
||||
consentVersion: string;
|
||||
}
|
||||
```
|
||||
|
||||
### 5. Privacy Policy Enforcement
|
||||
|
||||
Enforce privacy commitments in code:
|
||||
|
||||
- Email addresses never logged in plaintext
|
||||
- Passwords never logged (already in pino redact config)
|
||||
- IP addresses anonymized after 7 days
|
||||
- Third-party data sharing requires explicit consent
|
||||
|
||||
## Implementation Approach
|
||||
|
||||
### Phase 1: Data Retention Jobs
|
||||
|
||||
- Create retention cleanup job in background job service
|
||||
- Add activity_log retention (90 days)
|
||||
- Add tracking_events retention (30 days)
|
||||
|
||||
### Phase 2: User Data Export
|
||||
|
||||
- Create export endpoint
|
||||
- Implement data aggregation query
|
||||
- Add rate limiting (1 export per 24h)
|
||||
|
||||
### Phase 3: Account Deletion
|
||||
|
||||
- Implement soft delete with anonymization
|
||||
- Create hard delete cleanup job
|
||||
- Add account recovery endpoint
|
||||
|
||||
### Phase 4: Cookie Consent
|
||||
|
||||
- Create consent banner component
|
||||
- Store consent preferences
|
||||
- Gate analytics based on consent
|
||||
|
||||
## Consequences
|
||||
|
||||
### Positive
|
||||
|
||||
- **Legal Compliance**: Meets GDPR and CCPA requirements
|
||||
- **User Trust**: Demonstrates commitment to privacy
|
||||
- **Data Hygiene**: Automatic cleanup prevents data bloat
|
||||
- **Reduced Liability**: Less data = less risk
|
||||
|
||||
### Negative
|
||||
|
||||
- **Implementation Effort**: Significant feature development
|
||||
- **Operational Complexity**: Deletion jobs need monitoring
|
||||
- **Feature Limitations**: Some features may be limited without consent
|
||||
|
||||
## Implementation Status
|
||||
|
||||
### What's Implemented
|
||||
|
||||
- ✅ Token cleanup job exists (tokenCleanupQueue)
|
||||
- ❌ Activity log retention
|
||||
- ❌ User data export endpoint
|
||||
- ❌ Account deletion endpoint
|
||||
- ❌ Cookie consent banner
|
||||
- ❌ Data anonymization functions
|
||||
|
||||
### What Needs To Be Done
|
||||
|
||||
1. Add activity_log cleanup to background jobs
|
||||
2. Create `/api/users/me/export` endpoint
|
||||
3. Create `/api/users/me` DELETE endpoint with soft delete
|
||||
4. Implement cookie consent UI component
|
||||
5. Document data retention in privacy policy
|
||||
6. Add anonymization utility functions
|
||||
|
||||
## Key Files (To Be Created/Modified)
|
||||
|
||||
- `src/services/backgroundJobService.ts` - Add retention jobs
|
||||
- `src/routes/user.routes.ts` - Add export/delete endpoints
|
||||
- `src/services/privacyService.server.ts` - Data export/deletion logic
|
||||
- `src/components/CookieConsent.tsx` - Consent banner
|
||||
- `src/utils/anonymize.ts` - Data anonymization utilities
|
||||
|
||||
## Compliance Checklist
|
||||
|
||||
### GDPR Requirements
|
||||
|
||||
- [ ] Article 15: Right of Access (data export)
|
||||
- [ ] Article 17: Right to Erasure (account deletion)
|
||||
- [ ] Article 20: Right to Data Portability (JSON export)
|
||||
- [ ] Article 7: Conditions for Consent (cookie consent)
|
||||
- [ ] Article 13: Information to be Provided (privacy policy)
|
||||
|
||||
### CCPA Requirements
|
||||
|
||||
- [ ] Right to Know (data export)
|
||||
- [ ] Right to Delete (account deletion)
|
||||
- [ ] Right to Opt-Out (cookie consent for analytics)
|
||||
- [ ] Non-Discrimination (no feature penalty for privacy choices)
|
||||
Reference in New Issue
Block a user