[Prajna1999]

Summary

Adds a standalone evaluation set for all 12 guardrail validators. Building and running it turned up three validator bugs, which this PR also fixes.

• Eval set: backend/Benchmarking datasets/guardrails_validator_eval_set.xlsx
  • 3,766 cases.
  • 4 benchmark sets compiled: content safety, PII, gender bias, and topic relevance (education + healthcare).
  • 8 synthetic sets generated: slur, profanity, NSFW, LlamaGuard, topic relevance (agriculture), ban list, answer relevance, LLM critic.
  • Built by build_eval_workbook.py from the CSVs next to it.
• Fix: BanList left the first letter of each banned word ("a sonography" → "a s").
• Fix: slur list entries with nukta letters never matched (12 Hindi entries such as बत्तमीज़ and भड़वा).
• Fix: gender_assumption_bias rewrites (identity rows, possessive her, Devanagari substring matches, lowercased output). F1 went from 0.81 to 0.93 and false positives from 80 to 0.
• First eval run: run_eval_workbook.py scored 11 of 12 validators. The report is in runs/2026-10-27/RUN_REPORT.md; LlamaGuard is blocked because the Guardrails Hub was shut down.
• Full notes and findings: backend/Benchmarking datasets/FINDINGS.md.

Eval workbook

Every validator sheet shares one schema:

case_id, validator, input_text, expected_result (FAIL = should trigger), label, expected_output, category, difficulty, language (en / hi / hi-Latn), domain, config_ref, source, notes

Validator-specific columns follow, then yellow run columns (actual_result, actual_output, match).

Supporting sheets:

• Index: coverage counts per validator, plus live TP/FP/FN/TN, accuracy, precision, recall and F1 formulas that fill in once actual_result is entered.
• Configs: the exact parameters behind each config_ref.
• Run_Log: one row per validator per periodic run.
• README: column guide and run steps.

Validator	Origin	Rows (FAIL / PASS)
content_safety	Benchmark	1,801 (1,139 / 662)
pii_remover	Benchmark	419 (256 / 163)
gender_assumption_bias	Benchmark	600 (300 / 300)
topic_relevance_llm	Benchmark	355 (115 / 240)
uli_slur_match	Synthetic	79 (41 / 38)
profanity_free	Synthetic	79 (43 / 36)
nsfw_text	Synthetic	74 (38 / 36)
llamaguard_7b	Synthetic	66 (37 / 29)
topic_relevance	Synthetic	70 (38 / 32)
ban_list	Synthetic	79 (47 / 32)
answer_relevance_custom_llm	Synthetic	72 (42 / 30)
llm_critic	Synthetic	72 (43 / 29)

Bug fixes

BanList span off by one. The hub BanList maps spaceless indices to 1-based positions, so fix_value kept the first letter of each match and error_spans started one character late.

• Added app/core/validators/ban_list.py, a subclass with 0-based spans.
• BanListSafetyValidatorConfig and app/evaluation/ban_list/run.py now use it.
• Exact match on the synthetic ban-list FAIL rows went from 0/37 to 34/37.
• Four integration tests had the buggy output ("this contains b") in their expected values. They now expect the banned word removed completely.

Slur list not NFKC-normalised. Input is NFKC-normalised, which decomposes ज़ and ड़, but the list labels were only lowercased. So 12 Devanagari entries could never match.

• load_slur_list now applies NFKC.
• content_safety reuses LexicalSlur, so it is fixed too.

gender_assumption_bias rewrites were often wrong (exact match 0.31 in the eval run).

• Rows that map a word to itself (woh, attendant) are skipped. They caused all 80 false positives.
• her becomes their when a noun follows and them otherwise.
• Word boundaries include Devanagari vowel signs, virama and nukta. Before, छात्र matched inside छात्रा, and गया। never matched.
• Replacements keep the matched word's casing.
• hers → theirs in the word list.
• Result: precision went from 0.76 to 1.00, F1 from 0.81 to 0.93, and fix exact match from 0.31 to 0.62.

Findings worth a look (not fixed here)

• uli_slur_match:
  • Generic words in the list cause false positives (mad, stupid, kutta, pataka, chakki (severity H), halwa, …).
  • Devanagari matches inside longer words (मूत्र is flagged as मूत).
• profanity_free:
  • Misses all Devanagari and most Hinglish profanity, plus obfuscations (sh1t, f u c k).
  • Flags "The cock crows" and clinical words.
• ban_list:
  • Fuzzy matching hits across word boundaries ("gender checklist", "Sonographer", "cake certificate").
  • Misses Hindi transliterations (सोनोग्राफी).
• answer_relevance_custom_llm: the strict default prompt is expected to reject concise-but-correct answers and correct safe redirects.
• Data caveats:
  • The gender benchmark has only 15 unique sentences.
  • About 99% of the content-safety lexical rows contain a slur-list term, so report per source_dataset.
  • Synthetic labels are LLM-written and spot-reviewed.

Test plan

• pytest app/tests: 394 passed, 14 skipped.
• Gender tests updated for case-preserving output. New tests cover identity rows, object vs possessive her, and Devanagari boundaries; 9 of them fail without the fix.
• New app/tests/validators/test_ban_list.py: full-word removal, start of text, multi-word phrase, span positions.
• New nukta regression test in test_lexical_slurs.py. It fails without the fix.
• Workbook formulas evaluated with the formulas Python engine: 0 errors. Index metrics match a Python recompute on simulated results.
• Re-run uli_slur_match / content_safety evals to measure the nukta fix.
• Open the workbook in Excel or Google Sheets and confirm the Index formulas render.

🤖 Generated with Claude Code

[coderabbitai]

Important

Review skipped

Auto reviews are limited based on label configuration.

🏷️ Required labels (at least one): 1

• ready-for-review

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

Run configuration

• Source mode: Auto
• Repository: ProjectTech4DevAI/kaapi-guardrails
• Branch: feat/eval-dataset
• File: .coderabbit.yaml
• Commit: d13c0cd
• Layers: Repository YAML
• Review profile: CHILL
• Plan: Advanced
• Run ID: 25971bb2-d4ea-4391-9b52-7b9e507ab219

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}