EvalTurn record
The EvalTurn record of Novior.MentorAgent.Evaluation: One user turn of an EvalConversation, with what it must do. Every check is optional; a turn with none…
On this page
One user turn of an EvalConversation, with what it must do. Every check is optional; a turn with none passes when it answers with some text and without an error.
Definition
Section titled “Definition”Namespace: Novior.MentorAgent.Evaluation. Package: Novior.MentorAgent.
public sealed record EvalTurn : IEquatable<EvalTurn>
dotnet add package Novior.MentorAgentConstructors
Section titled “Constructors”public EvalTurn(string Input)
One user turn of an EvalConversation, with what it must do. Every check is optional; a turn with none passes when it answers with some text and without an error.
Parameters
Input: The user message of this turn.
Properties
Section titled “Properties”ExpectRefusal
Section titled “ExpectRefusal”public EvalRefusal? ExpectRefusal { get; init; }
The refusal this turn must get: the mentor turns the message down instead of answering, and the user sees an error. With it, that refusal passes the turn and the conversation goes on; a turn that was answered, refused the other way, or failed, fails, and its notes say which. null, the default, expects none: a refusal fails the turn as any error does.
A refusal is one of the two judgements on the message: EvalRefusal.OutOfScope and EvalRefusal.Guardrail. A safety check that could not run, a message too long, the rate limit and a turn without a tenant are not refusals: they stay errors, expected or not, so a case that must be refused does not pass because the guardrail is down.
ExpectTools
Section titled “ExpectTools”public IReadOnlyList<string>? ExpectTools { get; init; }
Tools this turn must call, by technical name (for example get_order): the turn fails if any of them is not in MentorEvalResult.ToolsCalled. This is how a case says "at turn 2 it must call X".
Expectation
Section titled “Expectation”public string? Expectation { get; init; }
The expected behaviour, for the built-in judge, as EvalCase.Expectation.
ForbidTools
Section titled “ForbidTools”public IReadOnlyList<string>? ForbidTools { get; init; }
Tools this turn must not call, by technical name: the turn fails if any of them was called.
public string Input { get; init; }
The user message of this turn.
MaxTimeToFirstToken
Section titled “MaxTimeToFirstToken”public TimeSpan? MaxTimeToFirstToken { get; init; }
The longest acceptable wait for the first streamed text of the answer; the turn fails if the first text came later, or never came.
MaxTokens
Section titled “MaxTokens”public long? MaxTokens { get; init; }
Token budget of this turn — every chat-model call it made; the turn fails if exceeded.
