MentorEvalOptions class

Knobs for an evaluation run.

Namespace: Novior.MentorAgent.Evaluation. Package: Novior.MentorAgent.

public sealed class MentorEvalOptions

dotnet add package Novior.MentorAgent

public MentorEvalOptions()

public IReadOnlyList<EvalCheck>? Checks { get; set; }

Extra native Agent Framework deterministic checks applied to every case — e.g. EvalChecks.KeywordCheck("weather"), EvalChecks.ToolCalledCheck("get_weather"), or FunctionEvaluator.Create(...). Added on top of the built-in non-empty check.

public IReadOnlyList<IAgentEvaluator>? Evaluators { get; set; }

Additional native Agent Framework evaluators run alongside the local checks — e.g. FoundryEvals (cloud quality & safety) or MEAI quality/safety evaluators wrapped for the agent. Each one's pass/fail also gates the report. Requires the corresponding package (Microsoft.Agents.AI.Foundry, Microsoft.Extensions.AI.Evaluation.Quality/Safety).

public bool Judge { get; set; }

Run the built-in lightweight LLM-as-judge for cases that have an EvalCase.Expectation.

public IChatClient? JudgeClient { get; set; }

Client used for the built-in judge; defaults to the mentor's configured ChatClient.

public long? MaxTotalTokens { get; set; }

Global token budget for the whole run — the report fails if exceeded (token regression gate).

public double? MinQuality { get; set; }

Minimum average quality (0–1). Requires MentorEvalOptions.Judge = true and case expectations.

public string? SystemInstructions { get; set; }

System prompt applied to every case (e.g. your mentor instructions under test).