Search your local images and videos with natural language, OCR text, or visual similarity — no cloud, no telemetry, no accounts. Free & open source (Global Edition).
FrameScout is a fully offline, AI-powered visual search engine for your local images and videos.
Type "sunset beach with friends", paste a reference image, or search by text inside images — and get results in milliseconds. Everything runs on your machine. Nothing leaves your computer.
⬇️ Download the latest release · 📖 Quick Start · 🤝 Contributing
🆕 v3.2.0 is here! A sweeping polish pass across the whole product: normalized hybrid scoring (semantic search no longer drowned out by filename matches), scene-detection video frame extraction, FTS5 fault-tolerant text search, lightbox preview with OCR hit red-boxes, right-click open/reveal/copy, drag-to-search, scan pause/cancel with ETA, multi-select + CSV/Markdown/JSON export, database backup, and a full data-consistency pass. See changelog.
🌐 Multilingual interface: 9 UI languages — English, 简体中文, 繁體中文, 日本語, Русский, 한국어, Deutsch, Français, Español — English by default. Switch anytime in Settings → Language.
🔤 Multi-language OCR: recognition for 4 built-in languages (English, 简体中文, 繁體中文, 日本語) plus 5 on-demand downloadable models (Русский, 한국어, Deutsch, Français, Español), with more coming soon. See Localization & OCR Languages.
Just want to try it? Download the latest Windows executable from the releases page — no installation required. Unzip and run. The prebuilt app includes a free trial of 100 indexed frames (videos count per extracted frame); the source is fully open under Apache 2.0 — the trial limit lives in the source too, so you're free to modify or remove it and build your own version.
You have hundreds of gigabytes of screenshots, photos, and video clips scattered across your drives.
Existing solutions force you to choose between two bad options:
☁️ Upload everything to the cloud — and lose your privacy forever.
📁 Stick with local file explorers — and spend hours hunting through folders.
FrameScout offers a third way: AI-powered search that runs entirely on your machine. No uploads. No subscriptions. No compromises.
We believe in transparency. Here's what FrameScout doesn't do yet — and what we plan to do about it:
| Limitation | Workaround / Future Plan |
|---|---|
| Windows 10/11 only (no macOS/Linux yet) | Cross-platform builds planned |
| FlatVector search is O(N·d) | Will migrate to Faiss/Annoy when N > 50K |
| No filesystem real-time monitoring | Manual re-scan; inotify/watchdog planned |
| No LLM-generated captions | Deliberate — would require network access |
| OCR supports 4 built-in + 5 downloadable languages (not the full 15 listed yet) | 6 more languages planned (RapidOCR rec-model swap) |
| OCR on-demand is per-file only | Batch OCR for multiple files planned |
| PDF/DOCX ingestion not yet supported | Convert to .txt/.md first; text extraction planned |
| Database backup is one-way (no in-app restore) | Restore by copying the .db back; in-app restore planned |
| Smart folders save a single text query + channel toggles (no boolean/time/type/seed conditions yet) | Full condition editor planned |
| FTS5 full-text search covers imported text entries only (not OCR / filenames / notes) | Expanding coverage planned |
| No partition / batch / date-range search filter | Planned |
| Relevance thresholds & video frame-extraction limits are fixed, not user-adjustable | Settings planned |
| Score breakdown bars are approximate for literal channels | Exact per-channel scores planned |
| Feature | Description |
|---|---|
| 🔒 100% Offline | No internet required. Models are bundled locally. Your data never leaves your computer. |
| 💡 Semantic Search (ONNX + SigLIP 2) | Type words and find matching images by meaning, not just filenames. |
| 🖼️ Image-to-Image Search | Drop a reference image to find visually similar ones in your library. |
| 🔍 OCR Text Search | Extracts and indexes text from images (English, 简体中文, 繁體中文, 日本語 built-in; more downloadable). Search "receipt from March" and find it instantly. Supports on-demand OCR on selected files. |
| 📋 Show All Mode | Load every indexed file. New files are sorted by intake time and always appear at the top. |
| 🎬 Video Frame Indexing | Automatically extracts key frames from videos (scene detection) and indexes them alongside static images. |
| 🧩 Visual Clustering | Discover groups of similar images (duplicates, near-duplicates, burst shots) with one click. |
| 📁 Smart Folders | Save a text search as a named, persistent folder that re-runs against your library whenever you open it. |
| 📝 Personal Notes | Attach markdown notes to any image. Notes are searchable. Next time you'll find it more effortlessly. |
FrameScout doesn't just find videos — it pinpoints the exact timestamp:
- Semantic Matching: Search using natural language (e.g., "sunset under the pier" or "code snippet on screen").
- One-Click Instant Seek: Click any search result to jump directly to that exact second (
01:23:45) in your video.
FrameScout ships with a 9-language interface and a three-tier OCR recognition system, configured in Settings:
Interface language (9) — switch instantly in Settings → Language: English, 简体中文, 繁體中文, 日本語, Русский, 한국어, Deutsch, Français, Español (English by default).
OCR recognition languages (three tiers) — pick any combination in the OCR recognition languages list:
| Tier | Languages | Notes |
|---|---|---|
| 🟢 Built-in recognition | English, 简体中文, 繁體中文, 日本語 | Ship with the app; 繁體中文 & 日本語 models can be deleted |
| 🔵 Downloadable recognition | Русский, 한국어, Deutsch, Français, Español | Models download on demand with progress, resume, SHA-256 verification & disk-space check; removable anytime |
| ⚪ Coming soon | Português, Italiano, Polski, Türkçe, Tiếng Việt, Bahasa Indonesia | Planned — shown in the list but not yet available |
OCR recognition models are obtained from the official RapidAI/RapidOCR model library (verified against official SHA-256 hashes) and stored under models/ocr/. Language metadata lives in a single source of truth — src/inference-worker/ocr_manifest.json.
FrameScout/
├── src/
│ ├── FrameScout-UI/ # Vue 3 + Tauri desktop app (Rust core)
│ ├── inference-worker/ # Python AI inference engine (ONNX + SigLIP 2 + RapidOCR)
│ │ └── models/ # Downloaded by scripts/download_models.py (git-ignored)
│ │ ├── siglip2-base/ # SigLIP 2 ONNX models (vision + text)
│ │ └── bge-m3/ # BGE-M3 text embeddings (optional)
│ ├── proto/ # ZeroMQ communication schema (Protobuf)
├── scripts/ # Utility scripts (model download, etc.)
├── README.md
├── CONTRIBUTING.md
└── LICENSE
OCR (RapidOCR / PP-OCRv4) weights ship inside the
rapidocr_onnxruntimewheel for English & Simplified Chinese. Traditional Chinese & Japanese recognition models are bundled; the rest are downloadable on demand intomodels/ocr/. See Localization & OCR Languages.
Full directory tree available in TREE.md.
FrameScout uses a three-process architecture for maximum performance, safety, and modularity:
┌─────────────────────────────────────────────────────────┐
│ Vue 3 Frontend │
│ (Search Console, Result Grid, Clusters) │
└──────────────────────┬──────────────────────────────────┘
│ Tauri IPC
▼
┌─────────────────────────────────────────────────────────┐
│ Rust/Tauri Core │
│ ┌──────────┐ ┌──────────────┐ ┌────────────────┐ │
│ │ File I/O │ │ SQLite + │ │ FlatVector │ │
│ │ WalkDir │ │ Rusqlite │ │ Matrix (768D) │ │
│ └──────────┘ └──────────────┘ └────────────────┘ │
└──────────────────────┬──────────────────────────────────┘
│ ZeroMQ + Protobuf (REQ/REP)
▼
┌─────────────────────────────────────────────────────────┐
│ Python Inference Worker │
│ ┌──────────────────────────────────────────────────┐ │
│ │ ONNX + SigLIP 2 (Text + Image Embeddings, 768D) │ │
│ └──────────────────────────────────────────────────┘ │
│ ┌──────────────────────────────────────────────────┐ │
│ │ RapidOCR (Text Extraction, offline) │ │
│ └──────────────────────────────────────────────────┘ │
│ ┌──────────────────────────────────────────────────┐ │
│ │ OpenCV (Video Frame Extraction, scene-detection) │ │
│ └──────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────┘
| Decision | Reason |
|---|---|
| Rust + Tauri (not Electron) | Native webview, binary is ~10x smaller, memory footprint is a fraction of Electron |
| Python worker (not all-Rust) | SigLIP 2 and RapidOCR have mature Python ecosystems; no need to reinvent the wheel |
| ONNX Runtime (not PyTorch) | Universal GPU acceleration (NVIDIA CUDA, AMD DirectML, Intel, CPU fallback); smaller footprint |
| ZeroMQ + Protobuf (not HTTP) | Microsecond latency, type-safe schema, works fully offline |
| FlatVector matrix (not Faiss) | For N < 50,000, O(N·d) brute-force is fast enough (~40ms) and has zero dependencies |
| SQLite + WAL mode | Battle-tested, embedded, supports concurrent reads during writes |
Indexing a folder: Rust walks the selected directory, collects file paths, and sends them in batches to the Python worker via ZMQ. Python generates SigLIP 2 image embeddings (768D) and OCR text for each image or video frame, then returns the results. Rust stores everything in SQLite and the in-memory FlatVector matrix, while the frontend shows real-time progress.
Searching by text: The frontend sends a query to Rust, which forwards it to Python for SigLIP 2 text embedding. Rust searches the FlatVector matrix by dot-product similarity and returns raw per-channel scores (semantic, OCR, note, filename) to the frontend, which normalizes and fuses them into a final ranked result.
Searching by image: The same flow as text search, except Python generates a SigLIP 2 image embedding from the reference image instead of a text embedding.
FrameScout combines four signals into a unified relevance score. Since v3.2.0 the backend returns raw, independently computed signals — the SigLIP cosine similarity and per-channel literal hit flags — and the frontend normalizes each one before fusing, so semantic matches are no longer drowned out by filename matches:
fused_score = sigmoid(cosine) × semantic_weight (semantic meaning)
+ ocr_hit × ocr_weight (text in image)
+ note_hit × note_weight (user annotations)
+ filename_hit × filename_weight (path/name match)
The normalization and fusion live in the frontend (utils/score.ts), and each channel can be toggled independently in the UI. Results are scored, ranked (with a stable tie-breaker), and paginated in real time. Note: the fusion weights and relevance thresholds are currently fixed constants — see Known Limitations.
All 768‑dimensional SigLIP 2 embeddings are stored in a single Vec<f32> for cache-friendly sequential access:
pub struct FlatVectorMatrix {
pub dim: usize, // 768
pub flat_vectors: Vec<f32>, // [v0_0..v0_767, v1_0..v1_767, ...]
pub metadata: Vec<ImageMeta>,
}Search is a simple dot-product loop over contiguous memory — no malloc, no indirection, no external library. At 50,000 images, a full scan completes in ~40ms.
Tested on a desktop with AMD Ryzen 7 5800X + 32GB RAM + NVIDIA RTX 3070 (ONNX with CUDA).
| Collection Size | Text Query Latency | Image Query Latency | Index Throughput |
|---|---|---|---|
| 1,000 images | 0.8 ms | 1.1 ms | ~1,200 img/min |
| 10,000 images | 7.2 ms | 9.5 ms | ~800 img/min |
| 50,000 images | 38.0 ms | 45.0 ms | ~600 img/min |
Disclaimer: These are benchmark results from our development environment. Actual performance depends on your hardware and dataset characteristics. Reproduce on your machine with
python src/inference-worker/scripts/benchmark.py --library <path>(writesbenchmarks/YYYY-MM-DD.json).
- Rust 1.75+ (2021 edition)
- Python 3.10+
- Node.js 18+ & npm
- SigLIP 2 ONNX model files (see below)
# Clone the repository
git clone https://github.com/bobgsning/FrameScout.git
cd FrameScout
# 1. Install frontend dependencies
cd src/FrameScout-UI
npm install
# 2. Set up the Python inference worker
cd ../inference-worker
python -m venv .venv
source .venv/bin/activate # Windows: .venv\Scripts\activate
pip install -r requirements.txt
# > ⚠️ This step requires internet access. After completion, FrameScout works fully offline.
# 3. Download SigLIP 2 + BGE-M3 models (one-time, ~1.5GB total) (from src/inference-worker)
python ../../scripts/download_models.py
# 4. Build and run (Global Edition)
# Return to frontend directory for building
cd ../FrameScout-UI
npm run tauri dev # Development mode
# npm run tauri build # Production buildThe Python inference worker must be compiled into a standalone executable for Tauri to launch it. The preferred packager is Nuitka (no temp-extraction delay, ~30–50% smaller, millisecond startup); PyInstaller is a legacy fallback.
# From the repository root
cd src/inference-worker
# Activate your virtual environment first, then:
.\build_nuitka.ps1 # release (standalone + onefile)
# .\build_nuitka.ps1 -Onedir # debug (standalone only, faster to build)The script handles the DirectML capi DLLs, the wheel-bundled RapidOCR weights, the external
models\ resources, and deployment to ..\FrameScout-UI\src-tauri\bin\ai_worker\ automatically.
See src/inference-worker/BUILD.md §4 for details.
Note: The packaged worker looks for models in
./models/relative to its own location. The Nuitka script copiesmodels\next to the exe for you.
- The splash screen appears: "FRAME SCOUT NEURAL LINK ESTABLISHING..."
- SigLIP 2 + RapidOCR + BGE-M3 models load into memory (varies depending on hardware)
- You'll see the main interface with the search console
- Click Browse → select a folder containing images
- Click Start Indexing → watch the real-time extraction bus
- Type a description in the search bar → get millisecond results
A note on honesty: FrameScout is actively developed by a small team. The items below are implemented and functional, but a few are still in a "first version" state — the detailed gaps are listed up front in Known Limitations. Everything marked "planned" is genuinely not built yet.
- Core search (text + image + OCR + notes)
- SigLIP 2 migration (768D embeddings, ONNX Runtime)
- Normalized hybrid scoring (semantic no longer drowned by filename matches)
- Visual clustering (medoid representative, sampled for large libraries)
- Smart folders (persistent, renamable, real counts, delete confirmation) — single-query scope
- "Show All" mode (flat listing without pagination)
- Index-time sorting (new files always appear first)
- RwLock optimization (non-blocking search during scan)
- Query preprocessing (trim + NFKC + tokenization + negative terms)
- Scene-detection video frame extraction + adjacent-frame dedup
- FTS5 trigram fault-tolerant text search — text entries only
- Lightbox preview (zoom / rotate / fullscreen / OCR hit red-boxes)
- Right-click open / reveal-in-explorer / copy path
- Drag-and-drop image-to-search
- Scan pause/cancel + ETA + failed-file retry
- Multi-select + candidate set + CSV/Markdown/JSON export
- Global Toast notifications + keyboard shortcuts (Esc /
// arrows / Space) - Search history + de-jargoned search modes
- Frame-accurate video notes + Markdown note rendering
- Database backup / integrity check — one-way (no in-app restore)
- Data timeline + "on this day last year"
- RRF cross-channel fusion (
search_unified) - RPC timeout convergence (no more infinite hangs)
- Unified search wired to the frontend + per-channel score-source bars
- Settings page + preferences persistence (path / page-size remembered across restarts)
- Scan stop button + cancelled batches distinguished in reports
- Disk-offline ghost protection (offline drives greyed out & purge blocked)
- Batch note editing (apply one note to many selected files)
- Faiss integration for N > 50K
- Automatic port fallback (worker retries the next port if binding fails)
- Smart-folder condition editor (boolean / time / file-type / seed-image conditions)
- FTS5 coverage for OCR / filenames / notes
- Partition / batch / date-range search
- In-app database restore
- Multi-modal fusion search (text + image simultaneously)
- Binary PDF/DOCX text extraction
- User-adjustable relevance thresholds & video extraction limits
- macOS build (Apple Silicon native)
- Linux build (AppImage + Flatpak)
- Folder tree sidebar view
- Per-image delete (not just bulk ghost purge)
- Virtual scrolling for very large libraries
FrameScout is open source under Apache 2.0. The core architecture, algorithms, and communication protocols are fully available for study and modification.
See CONTRIBUTING.md for guidelines.
| Component | License | Notes |
|---|---|---|
Rust/Tauri core (src/FrameScout-UI/src-tauri/) |
Apache 2.0 | Full source available |
Vue 3 frontend (src/FrameScout-UI/) |
Apache 2.0 | Full source available |
Protobuf definitions (src/proto/) |
Apache 2.0 | Full source available |
| Python inference worker | Apache 2.0 | Included in this repository |
FrameScout core is licensed under Apache License 2.0. See LICENSE for details. Third-party components are listed in THIRD_PARTY_NOTICES.md.
© 2026 AetherFlow Labs
"FrameScout", "FrameScout — Offline AI Search", and the FrameScout logo are trademarks of Bob G. S. Ning.
All other trademarks are the property of their respective owners.
- Google SigLIP 2 — for the open-source contrastive language-image pre-training model (used via ONNX)
- RapidOCR — for the lightweight, multi-language OCR engine
- ONNX Runtime — for cross-platform, hardware-accelerated inference
- Tauri — for the secure, lightweight desktop app framework
- ZeroMQ — for reliable, high-performance inter-process communication
- Prost — for idiomatic Protobuf support in Rust
- The Rust Community — for building a language where safety and performance coexist
- Issues & Bug Reports: GitHub Issues
- General Questions: bobgsning@outlook.com
- Security Concerns: Please email directly (PGP key available on request)
🤝 We need your help! FrameScout is a small project right now. If you're passionate about privacy-first AI tools, we'd love your contribution — whether it's code, documentation, bug reports, or just testing on your machine. We're currently preparing our first good first issues. In the meantime, feel free to open a discussion or browse the codebase.
FrameScout — Your images. Your privacy. Your search.
🔒 100% Offline · ⚡ Millisecond Search · 🖥️ Local AI Inference