From a food photo to likely ingredients, estimated nutrition, and a grocery-planning view.
Try the simulated demo → · Releases · Contribute
Important
PantryPin is an experimental MVP. The public GitHub Pages experience uses canned food scenarios selected by filename and simulated store data—it does not inspect image pixels. Self-hosting with an OpenAI API key enables actual image analysis. Live retailer pricing and inventory are not implemented yet.
Food photos are rich in inspiration but poor in practical detail. PantryPin explores one continuous workflow:
- See it — start with a meal, snack, dessert, fruit, or drink photo.
- Understand it — turn the dish into likely ingredients, a calorie range, macros, and confidence signals.
- Recreate it — compare an illustrative ingredient basket and open relevant searches on established cooking sites.
The aim is not false precision. A single photo cannot reveal exact portions, hidden oils, or every ingredient, so uncertainty is treated as part of the interface—not hidden in fine print.
| Capability | GitHub Pages demo | Self-hosted with API key |
|---|---|---|
| Photo content analyzed | No—canned scenario selected by filename | Yes—vision model analyzes the image |
| Dish, ingredients, and nutrition | Sample output | Model estimates with confidence levels |
| Store prices, locations, and pickup times | Simulated | Simulated; retailer API not connected |
| Recipe resources | Generated search links | Generated search links |
| Photo persistence by PantryPin | None | None; the image is sent to your configured model provider |
Try a file containing cake or pizza in its filename to switch scenarios in the Pages demo; other names show the grain-bowl scenario.
- Drag-and-drop food photo upload with validation and preview
- Structured dish and ingredient breakdowns
- Calorie ranges and macro estimates instead of a misleading single truth
- Per-ingredient confidence labels and uncertainty notes
- Deterministic ZIP-sensitive demo baskets for Walmart, ALDI, and Target
- Responsive comparison views for ingredients, stores, and recipe searches
- Optional OpenAI Responses API integration using image input and a strict JSON schema
- Static GitHub Pages deployment plus a separable server API
- Accessible keyboard controls and reduced-motion support
Requires Node.js 20.9 or newer.
git clone https://github.com/nahin333/where-and-how-much.git
cd where-and-how-much
npm install
cp .env.example .env.local
npm run devOpen http://localhost:3000.
Without an API key, the local server returns labeled demo data. To enable real image analysis:
# .env.local
OPENAI_API_KEY=your_key_here
OPENAI_VISION_MODEL=gpt-4o-miniNever prefix the secret with NEXT_PUBLIC_ or commit .env.local.
| Variable | Where | Purpose |
|---|---|---|
OPENAI_API_KEY |
Server only | Enables vision analysis |
OPENAI_VISION_MODEL |
Server only | Chooses the vision-capable model |
ALLOWED_ORIGIN |
External API server | Restricts cross-origin frontend requests |
NEXT_PUBLIC_API_BASE_URL |
Frontend build | Points Pages to a separately hosted API |
NEXT_PUBLIC_STATIC_DEMO |
Frontend build | Enables browser-only scenario mode |
NEXT_PUBLIC_BASE_PATH |
Pages build | Sets the repository subpath |
flowchart LR
Photo[Food photo] --> UI[Next.js interface]
UI -->|Pages demo| Scenarios[Canned scenarios]
UI -->|Self-hosted| API[POST /api/analyze]
API --> Vision[Vision model]
Vision --> Structured[Structured food estimate]
Scenarios --> View[Ingredients · nutrition · basket]
Structured --> View
Pricing[Pricing provider boundary] -. simulated today .-> View
View --> Searches[Cooking-site searches]
The main integration boundaries are:
lib/analysis.ts— image analysis and schema validationlib/pricing.ts— replaceable pricing-provider boundaryapp/api/analyze/route.ts— server endpoint and CORS boundaryapp/page.tsx— upload and results experience
The included Pages workflow tests, builds, and publishes the static demo after every push to main.
- Go to Settings → Pages.
- Set Source to GitHub Actions.
- Push to
mainor run the workflow manually.
Deploy the Next.js application to a Node-compatible host, configure OPENAI_API_KEY, and add authentication and rate limits before sharing the AI endpoint publicly.
To keep the frontend on Pages, deploy only the API elsewhere and configure:
# API host
ALLOWED_ORIGIN=https://nahin333.github.io
# Pages workflow
NEXT_PUBLIC_API_BASE_URL=https://your-api.example.com
NEXT_PUBLIC_STATIC_DEMO=false
- Ingredient and nutrition results are estimates, not medical or dietary advice.
- In AI mode, uploaded images pass through your backend to the configured model provider. Review that provider's data controls for your deployment.
- Demo store names, prices, distances, and pickup windows are simulated. PantryPin does not query, scrape, represent, or receive sponsorship from Walmart, ALDI, Target, or any other retailer.
- Recipe cards open generated searches; PantryPin does not reproduce, select, or verify individual publisher recipes.
- Third-party retailer, publisher, and model-provider names and marks belong to their respective owners; no affiliation or endorsement is implied.
- Confirm ingredients directly when allergies or food safety are involved.
- Licensed live grocery search with normalized package pricing
- Image resizing and signed uploads for production-scale photos
- Portion calibration and editable ingredient quantities
- Dietary and allergen signals with explicit confidence
- Saved analyses and shareable basket comparisons
- Authentication, rate limiting, moderation, and observability
- Evaluation set for dish, ingredient, and calorie-estimation quality
Have a different priority? Open a feature request.
Ideas, bug reports, design improvements, pricing-provider adapters, and evaluation work are welcome. Read CONTRIBUTING.md, choose an issue, and open a focused pull request.
A strong first contribution: add another deterministic demo scenario with tests and an honest confidence note.
If this direction is useful, star PantryPin to follow the project—or fork it and take the experiment somewhere new.
See CHANGELOG.md and GitHub Releases.
PantryPin's original source code, documentation, and project artwork are released under the MIT License. Third-party packages, services, names, and marks remain subject to their respective licenses and terms.