Repository navigation
Evaluate and Improve Example Workflows #219
Description
Problem
Currently, we have no systematic way to:
- Measure how well these workflows perform their intended tasks
- Test prompt improvements before deploying them
- Compare different prompt variations or model configurations
- Validate that changes don't regress quality
- Provide quality benchmarks for the community
Solution
Use the Gemini CLI evaluation framework to systematically test and improve the effectiveness of prompts and configurations used in our example workflows. This will enable data-driven optimization of our provided workflows and give the community tools to evaluate their own Gemini CLI automations.
Dependencies
- Gemini CLI evaluation framework: Gemini CLI Evaluation Framework for Specific Use Cases google-gemini/gemini-cli#6757
- Prompt Reusability: Reusability of Prompts #76
Activity
15 remaining items
feat: establish robust evaluation framework for workflow benchmarks
feat: establish robust evaluation framework for workflow benchmarks (#…
feat: expand eval dataset with edge and complex cases and refine prom…