MentorEvalOptions class
The MentorEvalOptions class of Novior.MentorAgent.Evaluation: Knobs for an evaluation run.
On this page
Knobs for an evaluation run.
Definition
Section titled “Definition”Namespace: Novior.MentorAgent.Evaluation. Package: Novior.MentorAgent.
public sealed class MentorEvalOptions
dotnet add package Novior.MentorAgentConstructors
Section titled “Constructors”public MentorEvalOptions()
Properties
Section titled “Properties”Checks
Section titled “Checks”public IReadOnlyList<EvalCheck>? Checks { get; set; }
Extra native Agent Framework deterministic checks applied to every case — e.g. EvalChecks.KeywordCheck("weather"), EvalChecks.ToolCalledCheck("get_weather"), or FunctionEvaluator.Create(...). Added on top of the built-in non-empty check.
Evaluators
Section titled “Evaluators”public IReadOnlyList<IAgentEvaluator>? Evaluators { get; set; }
Additional native Agent Framework evaluators run alongside the local checks — e.g. FoundryEvals (cloud quality & safety) or MEAI quality/safety evaluators wrapped for the agent. Each one's pass/fail also gates the report. Requires the corresponding package (Microsoft.Agents.AI.Foundry, Microsoft.Extensions.AI.Evaluation.Quality/Safety).
public bool Judge { get; set; }
Run the built-in lightweight LLM-as-judge for cases that have an EvalCase.Expectation.
JudgeClient
Section titled “JudgeClient”public IChatClient? JudgeClient { get; set; }
Client used for the built-in judge; defaults to the mentor's configured ChatClient.
MaxTotalTokens
Section titled “MaxTotalTokens”public long? MaxTotalTokens { get; set; }
Global token budget for the whole run — the report fails if exceeded (token regression gate).
MinQuality
Section titled “MinQuality”public double? MinQuality { get; set; }
Minimum average quality (0–1). Requires MentorEvalOptions.Judge = true and case expectations.
SystemInstructions
Section titled “SystemInstructions”public string? SystemInstructions { get; set; }
System prompt applied to every case (e.g. your mentor instructions under test).