Skip to content

Enable real classifiers in local dev by default - #952

Open
mihow wants to merge 5 commits into
mainfrom
feat/default-processing-service
Open

mihow wants to merge 5 commits into
mainfrom
feat/default-processing-service

Conversation

@mihow

@mihow mihow commented Sep 17, 2025 •

Copy link
Copy Markdown
Collaborator

Summary

Local development now runs real models: the main Antenna docker compose stack starts the example processing service as ml_backend instead of the minimal stub, so jobs in a local stack produce real detections and classifications. Once #992 lands, that includes the global moth species classifier. The models download on the first job and are kept in a volume, so rebuilding the container does not download them again. CI is unchanged and keeps the fast, model-free minimal service.

List of Changes

Change (effect) How Notes
Local jobs run real models Root docker-compose.yml: ml_backend builds and mounts processing_services/example docker-compose.ci.yml keeps minimal
Models download once, not on every rebuild New ml_backend_model_cache volume on /root/.cache (Hugging Face and torch caches) Addresses the review suggestion about cache volumes
Developers know what to expect README: which service each stack runs, the first-download size, how to start minimal next to the main stack

Testing

On a local stack built from this branch, with only ml_backend started: the first request to the zero-shot detector pipeline took 60 s and downloaded 3.6 GB of models into the cache volume; after recreating the container, the same request took 13.7 s with no download. Both returned 6 detections.

Still open (from the original description)

To try it: docker compose build ml_backend && docker compose up -d, then use Register Pipelines on the project's processing services page.

Related: #992 (global moths in example), #1252 (the minimal service as a pull-mode worker).

🤖 Generated with Claude Code

https://claude.ai/code/session_01TNNRg9NDr6opFZgK3nxx5C

@netlify

netlify Bot commented Sep 17, 2025 •

Copy link
Copy Markdown

✅ Deploy Preview for antenna-preview canceled.

Name Link
🔨 Latest commit 6806786
🔍 Latest deploy log https://app.netlify.com/projects/antenna-preview/deploys/6aceb1225a9599c61c84eb1d

@coderabbitai

coderabbitai Bot commented Nov 18, 2025 •

Copy link
Copy Markdown
Contributor
📝 Walkthrough

The docker-compose.yml file was updated to change the ml_backend service's build context and volume mount paths from ./processing_services/minimal to ./processing_services/example.

Changes

Cohort / File(s) Summary
Docker Compose Configuration
docker-compose.yml
Updated ml_backend service build context and mounted volume path from minimal to example subdirectory

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~2 minutes

  • Straightforward path substitution with no logic changes
  • Single file affected with repetitive pattern (same update applied twice)
Pre-merge checks: Passed 3
Passed Passed checks: 3 passed
Check name Status Explanation
Title check Passed The title clearly and specifically describes the main change: enabling real classifiers in the local development environment by default, which aligns with the docker-compose.yml changes updating the M…
Docstring Coverage Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Description check Passed The description is complete enough for review. It explains the purpose, lists the file and behavior changes, documents testing results and commands, identifies related issues, notes open work, and sta…
✨ Finishing touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch feat/default-processing-service
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@mihow
mihow marked this pull request as ready for review November 18, 2025 22:19
Copilot AI review requested due to automatic review settings November 18, 2025 22:19

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR switches the default ML backend for local development from the minimal processing service to the example processing service, enabling real ML classifiers (transformers, PyTorch) instead of the basic mock implementations.

Key changes:

  • Updates the ml_backend service in docker-compose.yml to use the example processing service with real ML models
  • The CI environment continues to use the minimal service for faster testing

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread docker-compose.yml
Comment on lines 161 to +162
volumes:
- ./processing_services/minimal/:/app
- ./processing_services/example/:/app

Copilot AI Nov 18, 2025

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The example service uses ML models (transformers, PyTorch) that download models on first use. Consider adding cache volumes to avoid re-downloading models on container restarts:

volumes:
  - ./processing_services/example/:/app
  - ./processing_services/example/huggingface_cache:/root/.cache/huggingface
  - ./processing_services/example/pytorch_cache:/root/.cache/torch

This is similar to the configuration in processing_services/example/docker-compose.yml and would improve startup performance after the initial model download.

Copilot uses AI. Check for mistakes.
mihow and others added 3 commits October 13, 2026 15:03
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TNNRg9NDr6opFZgK3nxx5C
The local stack now runs the example service as ml_backend, which downloads its models
on first use. Without a cache volume every rebuild downloaded them again. Also describes
which service each stack runs.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TNNRg9NDr6opFZgK3nxx5C
… in local dev

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TNNRg9NDr6opFZgK3nxx5C
@netlify

netlify Bot commented Oct 13, 2026 •

Copy link
Copy Markdown

✅ Deploy Preview for antenna-ssec canceled.

Name Link
🔨 Latest commit 6806786
🔍 Latest deploy log https://app.netlify.com/projects/antenna-ssec/deploys/6aceb122254a0a9d13359c3c

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants