MentorAgent Docs
Menu

Questa pagina non è ancora tradotta in italiano: qui la trovi in inglese.

ReferencePackage: Novior.MentorAgent

EvalTurn record

The EvalTurn record of Novior.MentorAgent.Evaluation: One user turn of an EvalConversation, with what it must do. Every check is optional; a turn with none…

On this page

One user turn of an EvalConversation, with what it must do. Every check is optional; a turn with none passes when it answers with some text and without an error.

Namespace: Novior.MentorAgent.Evaluation. Package: Novior.MentorAgent.

public sealed record EvalTurn : IEquatable<EvalTurn>

Terminal
dotnet add package Novior.MentorAgent

public EvalTurn(string Input)

One user turn of an EvalConversation, with what it must do. Every check is optional; a turn with none passes when it answers with some text and without an error.

Parameters

  • Input: The user message of this turn.

public EvalRefusal? ExpectRefusal { get; init; }

The refusal this turn must get: the mentor turns the message down instead of answering, and the user sees an error. With it, that refusal passes the turn and the conversation goes on; a turn that was answered, refused the other way, or failed, fails, and its notes say which. null, the default, expects none: a refusal fails the turn as any error does.

A refusal is one of the two judgements on the message: EvalRefusal.OutOfScope and EvalRefusal.Guardrail. A safety check that could not run, a message too long, the rate limit and a turn without a tenant are not refusals: they stay errors, expected or not, so a case that must be refused does not pass because the guardrail is down.

public IReadOnlyList<string>? ExpectTools { get; init; }

Tools this turn must call, by technical name (for example get_order): the turn fails if any of them is not in MentorEvalResult.ToolsCalled. This is how a case says "at turn 2 it must call X".

public string? Expectation { get; init; }

The expected behaviour, for the built-in judge, as EvalCase.Expectation.

public IReadOnlyList<string>? ForbidTools { get; init; }

Tools this turn must not call, by technical name: the turn fails if any of them was called.

public string Input { get; init; }

The user message of this turn.

public TimeSpan? MaxTimeToFirstToken { get; init; }

The longest acceptable wait for the first streamed text of the answer; the turn fails if the first text came later, or never came.

public long? MaxTokens { get; init; }

Token budget of this turn — every chat-model call it made; the turn fails if exceeded.