E-commerce Scraper API
Automated competitor price monitoring across 50+ sites with intelligent deduplication.
Industry
E-commerce Analytics (demo)
Timeline
2 weeks
Stack
Python, FastAPI, Playwright, Docker, Redis, PostgreSQL
Role
Backend Development + DevOps + API Design
Problem
Growth team needed competitor price monitoring across 50+ e-commerce sites with deduplication, price-change alerts, and structured data export. Manual monitoring was taking 20+ hours weekly and missing critical price drops that affected competitive positioning.
Approach
Built rotating proxy pool with Playwright for JavaScript-heavy sites. Implemented smart rate limiting, duplicate detection via content hashing, and real-time price-change webhooks. Containerized entire system with Redis caching for performance and horizontal scaling capability.
Results
Key Features
Target site configuration dashboard with success rate monitoring
Real-time scraping status with proxy rotation visualization
Price history charts with automated alert trigger configuration
API documentation with structured output examples and rate limits
Export scheduler with multiple format options and webhook integrations