Open Scouts is an open-source AI monitoring tool that lets users create automated ‘scouts’ to track web content—like new restaurants, AI news, or product updates—and receive email alerts when relevant results are found. It’s designed for researchers, journalists, and professionals who need to stay updated on dynamic online information without manual searching.
Built with Next.js 15, React 19, and TypeScript, the platform leverages Supabase for authentication, real-time data, and edge functions; pgvector for semantic search on AI-generated summaries; Firecrawl for web scraping; OpenAI for intelligent query generation; and Resend for email delivery. The architecture supports scalable, isolated scout execution via pg_cron and pg_net, enabling thousands of concurrent scouts with user-level isolation and secure credential storage.
What You Get
- AI-Powered Scout Creation - Users type natural language queries like ‘Scout for any recent Indian restaurants near me’ and the system automatically generates optimized search parameters using OpenAI GPT-4 with function calling.
- Scheduled Web Monitoring - Scouts run on customizable intervals (hourly, every 3 days, weekly) using pg_cron to trigger automated web searches via Firecrawl without manual intervention.
- Semantic Search on Execution Summaries - Each scout execution generates a one-sentence AI summary stored with pgvector embeddings, enabling semantic search and retrieval of past results by meaning, not just keywords.
- User-Managed Firecrawl API Keys - Each user can input their own Firecrawl API key in-app, ensuring usage is tracked to their account and avoiding shared key limitations.
- Email Alerts via Resend - Automated HTML email notifications are sent when scouts find results, with support for test emails and domain verification for bulk delivery on Resend’s free tier.
- Scalable Dispatcher Architecture - A pg_cron + pg_net + edge function pipeline runs each scout in an isolated Deno function (256MB memory, 400s timeout), enabling thousands of concurrent scouts with resource isolation.
- Supabase Auth with Google OAuth - Secure user authentication via email/password and optional Google OAuth, with Row Level Security ensuring users only access their own scouts and data.
- Real-Time Result Dashboard - Users view all scout results in real time via Supabase Realtime subscriptions, with AI-generated summaries and direct links to source content.
Common Use Cases
- Tracking local business openings - A food blogger uses Open Scouts to monitor for new Indian restaurants in their city and receives email alerts with links to their websites and menus.
- Monitoring AI industry news - A startup founder sets up a scout to track mentions of ‘LLM regulation’ or ‘open-source AI models’ across news sites and blogs to stay ahead of trends.
- Competitor product tracking - A SaaS product manager creates a scout to detect new features announced on competitor websites and gets notified via email when changes occur.
- Academic research alerts - A PhD student configures a scout to find new papers on ‘transformers in healthcare’ from academic blogs and arXiv, receiving summaries and links daily.
Under The Hood
Architecture
- Clear separation between client and server layers, with API routes managing authentication and data flow via Supabase while client components handle UI state and interactions
- Modular component architecture using Radix UI and Tailwind CSS, promoting reuse and consistent theming across the application
- Dependency injection through custom server utilities that encapsulate Supabase client initialization, ensuring uniform auth and database access
- Service layer abstraction for external API interactions, decoupling business logic from UI and data persistence layers
- State management coordinated between Supabase auth, user preferences, and client-side hooks for real-time updates
- Strong type safety enforced via TypeScript and Zod schemas across API routes and data models
Tech Stack
- Full-stack React/Next.js application leveraging App Router and RSC with TypeScript for type-safe server and client rendering
- Tailwind CSS with utility-first styling and component libraries like shadcn/ui for consistent, themable UI components
- Supabase serving as the backbone for authentication, real-time data persistence, and server-side integration
- AI/ML tooling integrated for LLM-driven content extraction and agent workflows
- PostHog embedded at the server level for precise user behavior tracking without client-side overhead
- Comprehensive dev tooling including ESLint, Prettier, and TypeScript presets to maintain code consistency
Code Quality
- Error handling is generic and lacks structured recovery mechanisms, limiting debuggability and user feedback
- Absence of testing across all layers leaves critical paths like authentication and rate limiting unverified
- Code organization is component-driven with clear layer separation, but shared utilities and server logic show inconsistent encapsulation
- Naming conventions are mostly consistent but occasionally lack domain-specific clarity
- Type safety is well-implemented in props and APIs, but runtime validation is missing in data-fetching paths
- Linting and formatting standards are implied but not formally enforced or documented
What Makes It Unique
- Seamless integration of Firecrawl’s web scraping API with Supabase Edge Functions to enable serverless, rate-limited data collection without background workers
- Animated ASCII explosions and dynamic curvy connectors transform standard error boundaries into immersive, brand-aligned experiences
- Custom CapsuleButton with motion-based press animations and blur transitions delivers a tactile, high-fidelity interaction layer rare in SaaS interfaces
- Real-time AI chat interface tightly coupled with scout execution history enables natural language refinement of scraping goals
- Server-side geolocation reverse-engineering via Nominatim eliminates reliance on third-party location services
- PostHog analytics implemented with server-side flush optimizations to minimize client-side latency and bloat