Torben Sorensen f6094df456
Some checks are pending
Deploy to Web Server flyer-crawler.projectium.com / deploy (push) Has started running
database expansion prior to creating on server - also error cleanup, some logging - DONE now for testing hehehe
2025-11-20 20:31:40 -08:00

Flyer Crawler - Grocery AI Analyzer

Flyer Crawler is a web application that uses the Google Gemini AI to extract, analyze, and manage data from grocery store flyers. Users can upload flyer images or PDFs, and the application will automatically identify items, prices, and sale dates, storing the structured data in a PostgreSQL database for historical analysis, price tracking, and personalized deal alerts.

We are working on an app to help people save money, by finding good deals that are only advertized in store flyers/ads. So, the primary purpose of the site is to make uploading flyers as easy as possible and as accurate as possible, and to store peoples needs, so sales can be matched to needs.

Features

  • AI-Powered Data Extraction: Upload PNG, JPG, or PDF flyers to automatically extract store names, sale dates, and a detailed list of items with prices and quantities.
  • Bulk Import: Process multiple flyers at once with a summary report of successes, skips (duplicates), and errors.
  • Database Integration: All extracted data is saved to a PostgreSQL database, enabling long-term persistence and analysis.
  • Personalized Watchlist: Authenticated users can create a "watchlist" of specific grocery items they want to track.
  • Active Deal Alerts: The app highlights current sales on your watched items from all valid flyers in the database.
  • Price History Charts: Visualize the price trends of your watched items over time.
  • Shopping List Management: Users can create multiple shopping lists, add items from flyers or their watchlist, and track purchased items.
  • User Authentication & Management: Secure user sign-up, login, and profile management, including a secure account deletion process.
  • Dynamic UI: A responsive interface with dark mode and a choice between metric/imperial unit systems.

Tech Stack

  • Frontend: React, TypeScript, Tailwind CSS
  • AI: Google Gemini API (@google/genai)
  • Backend: Node.js with Express
  • Database: PostgreSQL
  • Authentication: Passport.js
  • UI Components: Recharts for charts

Required Environment Variables & Setup

This project requires several secret keys to function. Create a .env file in the root of the project and see the env.example file for a complete template.

  • For the AI Service: VITE_GOOGLE_GENAI_API_KEY. This is your public-facing Google Gemini API key.
  • For the Database: You will need to provide connection details for your PostgreSQL database, such as DB_USER, DB_HOST, DB_DATABASE, DB_PASSWORD, and DB_PORT.

Setup and Installation

Step 1: Set Up PostgreSQL Database

  1. Set up a PostgreSQL database instance.
  2. Run the Database Schema:
    • Connect to your database using a tool like psql or DBeaver.
    • Open sql/schema.sql.txt, copy its entire contents, and execute it against your database.
    • This will create all necessary tables, functions, and relationships.

Step 2: Install Dependencies and Run the Application

  1. Install Dependencies:

    npm install
    
  2. Run the Application:

    npm run dev
    

Step 3: Seed Development Users (Optional)

To create the initial admin@example.com and user@example.com accounts, you can run the seed script:

npm run seed

After running, you may need to restart your IDE's TypeScript server to pick up the changes.

NGINX mime types issue

sudo nano /etc/nginx/mime.types

change

application/javascript js;

TO

application/javascript js mjs;

RESTART NGINX

sudo nginx -t sudo systemctl reload nginx

actually the proper change was to do this in the /etc/nginx/sites-available/flyer-crawler.projectium.com file

Description
The Google AI based Flyer Crawler App
Readme 14 MiB
Languages
TypeScript 97.1%
PLpgSQL 1.5%
JavaScript 0.6%
PowerShell 0.4%
Shell 0.3%