Komşu — the Agent Upstairs Neighbor
An AI agent that turns any ordinary CCTV camera into a watchful neighbor. It sees the scene, reasons about what matters, and acts on its own to keep you informed — in Turkish, over WhatsApp, Telegram and the web.
- 🔗 Live demo: komsu.aymam.ai
- 💻 Source: github.com/mfatihdinc61/komsu
- 🏷️ A product concept by aymam.ai
Why it's an agent, not just a camera app
Most "smart camera" tools are passive — they record, maybe run a fixed detector. Komşu closes the full perceive → reason → act loop and holds standing goals on your behalf:
- Perceive — streams RTSP, detects motion locally (grid diff, no cloud cost), and sends only meaningful frames to a vision model.
- Reason — Google Gemini describes the scene, answers free-form questions, and judges whether user-defined conditions are met, all via structured JSON output.
- Act — proactively messages you: danger alerts and your own natural-language requests, delivered on the channel you used.
- Hold intent — you say "minibüs gelirse bugün haber ver" and it parses that into a persistent goal, watches for it across frames, and notifies you when it happens.
The watch feature is the clearest expression of agency: the operator hands over a goal in plain language, and the system autonomously pursues it until it's met or expires — no rules to configure, no dashboards to watch.
What it does
- Live view + AI chat — ask "şu an ne oluyor?", "son bir saatte ne oldu?", "kaç araba var?" on the web, WhatsApp or Telegram. Present-tense questions analyze the live frame; detail questions re-analyze saved frames (colour, clothing, count).
- Activity log — motion triggers a short, timestamped scene description, kept in history for time-window questions ("dün akşam ne oldu?", "geçen hafta?").
- Proactive danger alerts — smoking, fire, fall, fight, weapon, medical, intrusion → an image + caption to preset numbers, using an approved WhatsApp template so alerts land even outside the 24-hour service window.
- Standing "watch" requests (the agentic core) — set conditional, recurring notifications in natural language and manage them ("uyarılarım neler?", "1. uyarıyı iptal et"). Evaluated inside the existing scene-description call, so it adds ~0% to API usage.
- "Show me the frame" — send a photo of a moment (now, a time, or a described event) with zero extra model calls.
Architecture
One Gemini call per scene does triple duty — description + danger detection + watch evaluation — through a single structured-output schema, keeping free-tier quota under control while the agent stays responsive.
``` RTSP camera ─ffmpeg─▶ MJPEG ─▶ motion (grid diff) ─▶ [cooldown] ─▶ one Gemini call │ ┌─────────────────────┬──────────────────┴────────────────┐ ▼ ▼ ▼ scene description danger alerts watch matches (SQLite event log) (WhatsApp template) (notify the requester)
web (live MJPEG + chat) · WhatsApp Cloud API (webhook) · Telegram (long-poll) └─ shared chat(): Q&A, image requests, watch commands ```
Engineering highlights
- Resilience for unattended operation: multi-key round-robin, per-model quota fallback (a model whose daily free-tier quota is spent rolls to the next), transient-retry with backoff, and self-healing WhatsApp webhooks that re-assert the correct callback on every boot.
- Cost-aware design: local motion gating + a describe-cooldown keep API usage bounded; the watch feature piggybacks on the existing call.
- Multimodal + structured output: JSON-schema responses make the model's scene reads, alerts, and watch matches machine-reliable.
- Real deployment: AWS EC2 (Ubuntu, systemd) behind Caddy with automatic Let's Encrypt TLS; DNS on AWS Lightsail.
Privacy by design
The assistant is instructed to never identify individuals, describe faces, or read plates — referring to people only in general terms ("two people", "a person with a bicycle"). This keeps it aligned with KVKK/GDPR expectations for camera data; footage in the demo is used with the owner's permission.
Tech stack
Node.js · Express · Google Gemini (vision) · ffmpeg / RTSP · SQLite · WhatsApp Cloud API · Telegram Bot API · AWS EC2 · Caddy / Let's Encrypt
