Flyer Crawler - Grocery AI Analyzer

Flyer Crawler is a web application that uses the Google Gemini AI to extract, analyze, and manage data from grocery store flyers. Users can upload flyer images or PDFs, and the application will automatically identify items, prices, and sale dates, storing the structured data in a PostgreSQL database for historical analysis, price tracking, and personalized deal alerts.

We are working on an app to help people save money, by finding good deals that are only advertized in store flyers/ads. So, the primary purpose of the site is to make uploading flyers as easy as possible and as accurate as possible, and to store peoples needs, so sales can be matched to needs.

Features

• AI-Powered Data Extraction: Upload PNG, JPG, or PDF flyers to automatically extract store names, sale dates, and a detailed list of items with prices and quantities.
• Bulk Import: Process multiple flyers at once with a summary report of successes, skips (duplicates), and errors.
• Database Integration: All extracted data is saved to a PostgreSQL database, enabling long-term persistence and analysis.
• Personalized Watchlist: Authenticated users can create a "watchlist" of specific grocery items they want to track.
• Active Deal Alerts: The app highlights current sales on your watched items from all valid flyers in the database.
• Price History Charts: Visualize the price trends of your watched items over time.
• Shopping List Management: Users can create multiple shopping lists, add items from flyers or their watchlist, and track purchased items.
• User Authentication & Management: Secure user sign-up, login, and profile management, including a secure account deletion process.
• Dynamic UI: A responsive interface with dark mode and a choice between metric/imperial unit systems.

Tech Stack

• Frontend: React, TypeScript, Tailwind CSS
• AI: Google Gemini API (@google/genai)
• Backend: Node.js with Express
• Database: PostgreSQL
• Authentication: Passport.js
• UI Components: Recharts for charts

---

Required Environment Variables & Setup

This project requires several secret keys to function. Create a .env file in the root of the project and see the env.example file for a complete template.

• For the AI Service: VITE_GOOGLE_GENAI_API_KEY. This is your public-facing Google Gemini API key.
• For the Database: You will need to provide connection details for your PostgreSQL database, such as DB_USER, DB_HOST, DB_DATABASE, DB_PASSWORD, and DB_PORT.

Setup and Installation

Step 1: Set Up PostgreSQL Database

1. Set up a PostgreSQL database instance.
2. Run the Database Schema:
   • Connect to your database using a tool like psql or DBeaver.
   • Open sql/schema.sql.txt, copy its entire contents, and execute it against your database.
   • This will create all necessary tables, functions, and relationships.

Step 2: Install Dependencies and Run the Application

1. Install Dependencies:

   npm install
   

2. Run the Application:

   npm run dev
   

Step 3: Seed Development Users (Optional)

To create the initial admin@example.com and user@example.com accounts, you can run the seed script:

npm run seed

After running, you may need to restart your IDE's TypeScript server to pick up the changes.

NGINX mime types issue

sudo nano /etc/nginx/mime.types

change

application/javascript js;

TO

application/javascript js mjs;

RESTART NGINX

sudo nginx -t sudo systemctl reload nginx

actually the proper change was to do this in the /etc/nginx/sites-available/flyer-crawler.projectium.com file

for OAuth

1. Get Google OAuth Credentials This is a crucial step that you must do outside the codebase:

Go to the Google Cloud Console.

Create a new project (or select an existing one).

In the navigation menu, go to APIs & Services > Credentials.

Click Create Credentials > OAuth client ID.

Select Web application as the application type.

Under Authorized redirect URIs, click ADD URI and enter the URL where Google will redirect users back to your server. For local development, this will be: http://localhost:3001/api/auth/google/callback.

Click Create. You will be given a Client ID and a Client Secret.

Add these credentials to your .env file at the project root:

plaintext GOOGLE_CLIENT_ID="your-client-id-from-google" GOOGLE_CLIENT_SECRET="your-client-secret-from-google"

2. Get GitHub OAuth Credentials You'll need to obtain a Client ID and Client Secret from GitHub:

Go to your GitHub profile settings.

Navigate to Developer settings > OAuth Apps.

Click New OAuth App.

Fill in the required fields:

Application name: A descriptive name for your app (e.g., "Flyer Crawler"). Homepage URL: The base URL of your application (e.g., http://localhost:5173 for local development). Authorization callback URL: This is where GitHub will redirect users after they authorize your app. For local development, this will be: http://localhost:3001/api/auth/github/callback. Click Register application.

You will be given a Client ID and a Client Secret.

Add these credentials to your .env file at the project root:

plaintext GITHUB_CLIENT_ID="your-github-client-id" GITHUB_CLIENT_SECRET="your-github-client-secret"