INNER CODE UNIT · Python

evaluate_response

valqore/valqore · blog/01-fine-tuning/eval_example.py:40

def evaluate_response(response: str, checks: list[tuple[str, str]]) -> tuple[int, int]:
    """Check if response contains expected substrings."""
    passed = sum(1 for _, substr in checks if substr.lower() in response.lower())
    return passed, len(checks)


# Usage:
# response = ollama.generate(model="valqore", prompt=test.prompt)
# passed, total = evaluate_response(response, test.checks)
# print(f"{test.id}: {passed}/{total} checks passed ({passed/total:.0%})")

View source record →