MentorEvaluator class

Prompt / regression evaluation harness (roadmap #10). Runs each EvalCase through the configured ChatClient, reading the provider's real per-case token usage, applies a built-in non-empty check and an optional lightweight LLM-as-judge, and returns a MentorEvalReport that fails when a run exceeds a token budget or drops below a quality threshold — so you can gate CI on it. When the developer supplies native checks/evaluators (MentorEvalOptions.Checks / MentorEvalOptions.Evaluators, e.g. EvalChecks, FoundryEvals, MEAI quality/safety), the Agent Framework's native evaluation runs on top and its verdicts also gate the report. Uses only already-referenced packages (Microsoft.Agents.AI). Point it at your mentor's ChatClient and pass your system instructions to catch prompt regressions; for full-pipeline token checks, drive the orchestrator instead and assert on IMentorMetrics.GetSnapshot().

Namespace: Novior.MentorAgent.Evaluation. Package: Novior.MentorAgent.

public sealed class MentorEvaluator

dotnet add package Novior.MentorAgent

public MentorEvaluator(IOptions<MentorOptions> options, ILogger<MentorEvaluator> logger)

public Task<MentorEvalReport> RunAsync(IEnumerable<EvalCase> cases, MentorEvalOptions? evalOptions = null, CancellationToken ct = default)