Repository navigation
Guardrails: Documentation and Testing #1309
Description
Is your feature request related to a problem?
We currently lack a structured standalone evaluation dataset for all validators in guardrails. This absence makes it difficult to assess validator performance effectively.
Describe the solution you'd like
- Create an evaluation sheet to be run periodically with validators.
- Ensure the dataset is representative of different input types for each validator.
- Report the findings in a structured document.
Original issue
Describe the current behavior
Currently we lack structured standalone eval dataset for all validators in guardrails.
Describe the enhancement you'd like
Create an eval sheet to be run periodically with validators. The dataset should be representative of different input types for individual validator. Report the findings in a structured document.
Why is this enhancement needed?
Explain the benefits (e.g., performance, usability, maintainability, scalability).
Additional context
Add any other context, metrics, screenshots, or examples about the enhancement here.
Metadata
Metadata
Assignees
Labels
Type
Projects
- Status
Activity
[-]Guardrails: Eval set for standalone guardrails[/-][+]Evaluation: Structured dataset for validators[/+][-]Evaluation: Structured dataset for validators[/-][+]Guardrails: Documentation and Testing[/+]