# Test without a model

> Test your actions, roles and confirmation boxes in automated tests with a fake IChatClient, develop offline with a local model, and catch regressions.

- URL: https://docs.mentoragent.net/configure/test-without-model
- Verified with: MentorAgent 1.2.0
- Last updated: 2026-11-07
- Packages: Novior.MentorAgent

Use this page to test your actions, their roles and their confirmation boxes in an automated test, without calling a model. A fake model plays the part of the real one: it asks MentorAgent to call one of your actions, and MentorAgent runs that call as in production. The test needs no key, no network and no money, and it gives the same result every time.

## When to choose it

Choose it for the rules that your code decides, which a test must check on every build:

- **Everything but the model is real.** MentorAgent finds your actions, offers the model only those that the user may run, checks the roles, shows the confirmation box, and runs your method. Your [action policies](https://docs.mentoragent.net/reference/action-policies) and [observers](https://docs.mentoragent.net/reference/action-policies#imentoractionobserver) run too.
- **The model's choices are not tested.** The fake model always asks for the tool that you give it. Whether a real model picks the right action for a message is a question for a test with the real model: see [Catch regressions with the real model](https://docs.mentoragent.net/configure/test-without-model#catch-regressions-with-the-real-model).

> **Note:**
> MentorAgent does not ship a fake model. The one on this page is about 50 lines: copy it into your test project.

## Write a fake model

MentorAgent talks to the model through an `IChatClient` of `Microsoft.Extensions.AI`, so a fake model is a class that implements it. This one asks for one tool on the first call, with the arguments that you give it. On the next call it gets the result of the tool, and answers "Done.":

```csharp title="ScriptedChatClient.cs"
using System.Runtime.CompilerServices;
using Microsoft.Extensions.AI;

// A model that asks for one tool, then answers "Done." No network, no key.
public sealed class ScriptedChatClient(
    string tool, Dictionary<string, object?> arguments) : IChatClient
{
    const string CallId = "call-1";

    // The tools that MentorAgent offered, and what the tool call returned
    public List<string> OfferedTools { get; } = [];
    public string? ToolResult { get; private set; }

    public Task<ChatResponse> GetResponseAsync(
        IEnumerable<ChatMessage> messages, ChatOptions? options = null,
        CancellationToken cancellationToken = default)
        => Task.FromResult(new ChatResponse(Next(messages, options)));

    public async IAsyncEnumerable<ChatResponseUpdate> GetStreamingResponseAsync(
        IEnumerable<ChatMessage> messages, ChatOptions? options = null,
        [EnumeratorCancellation] CancellationToken cancellationToken = default)
    {
        await Task.Yield();
        var reply = Next(messages, options);
        yield return new ChatResponseUpdate(reply.Role, reply.Contents)
        {
            MessageId = Guid.NewGuid().ToString("N"),
        };
    }

    ChatMessage Next(IEnumerable<ChatMessage> messages, ChatOptions? options)
    {
        var result = messages.SelectMany(m => m.Contents)
            .OfType<FunctionResultContent>()
            .FirstOrDefault(r => r.CallId == CallId);
        if (result is not null)
        {
            // Second call: the tool ran, or was refused. Keep what it returned
            ToolResult = result.Result?.ToString();
            return new ChatMessage(ChatRole.Assistant, "Done.");
        }
        if (options?.Tools is not { Count: > 0 } tools)
            return new ChatMessage(ChatRole.Assistant, "Done.");

        // First call: ask for the tool even if it is not offered,
        // like a model that makes up a tool name
        OfferedTools.AddRange(tools.Select(t => t.Name));
        return new ChatMessage(ChatRole.Assistant,
            [new FunctionCallContent(CallId, tool, arguments)]);
    }

    public object? GetService(Type serviceType, object? serviceKey = null) => null;

    public void Dispose() { }
}
```

- **It asks for the tool even when the tool is not offered**, like a model that makes up a tool name. That is how a test proves that a call made anyway is refused.
- **`OfferedTools`** lists the tools that MentorAgent offered on the first call. An action that the user may not run is not in the list.
- **`ToolResult`** is what the tool call returned to the model: the value of your method, or the reason why the call was refused.
- **A call without tools gets "Done."** at once. That covers the calls that are not a turn of the chat, such as an [evaluation](https://docs.mentoragent.net/configure/test-without-model#catch-regressions-with-the-real-model).

## Build the app under test

The test builds the same services as your app, with the fake model in `ChatClient`. These are the actions of the example: a write that needs the `Supervisors` role and a confirmation, and a read.

```csharp title="CaseMergeActions.cs"
public class CaseMergeActions
{
    // What the actions did: in your app, your own service and its store
    public List<string> Merged { get; } = [];

    [MentorAction(
        Description = "Merges a duplicate support case into another case",
        RequiredRoles = ["Supervisors"],
        RequiresConfirmation = true)]
    public string MergeCases(string duplicate, string into)
    {
        Merged.Add($"{duplicate} into {into}");
        return $"Case {duplicate} merged into {into}.";
    }

    [MentorAction(
        Description = "Gets the owner of a support case",
        ReadOnly = true)]
    public string GetCaseOwner(string caseNumber) => "Alex";
}
```

In the example the class records what it did, so that the test can read it. In your app, register your real service with a fake store, or with a test database.

The test host registers the actions, signs a user in and adds MentorAgent with the fake model. Then it sends one message, as the widget does, and answers every confirmation box:

```csharp title="MentorTestHost.cs"
using System.Security.Claims;
using Microsoft.AspNetCore.Components.Authorization;
using Microsoft.Extensions.AI;
using Novior.MentorAgent.Abstractions.Interfaces;   // IMentorOrchestrator
using Novior.MentorAgent.Extensions;                // AddMentorAgent

public static class MentorTestHost
{
    // Your services, MentorAgent with the fake model, and a signed-in user
    public static WebApplication Build(IChatClient model, ClaimsPrincipal user)
    {
        var builder = WebApplication.CreateBuilder();
        builder.Services.AddSingleton<CaseMergeActions>();
        builder.Services.AddScoped<AuthenticationStateProvider>(
            _ => new SignedIn(user));
        builder.Services.AddMentorAgent(options =>
        {
            options.AppName = "Case management";
            options.ChatClient = model;
            options.ScanAssemblies = [typeof(CaseMergeActions).Assembly];
        });
        return builder.Build();
    }

    // One turn: the message, and the user's answer to every confirmation
    // box. Returns how many boxes the user saw.
    public static async Task<int> SendAsync(
        WebApplication app, string message, bool approve)
    {
        await using var scope = app.Services.CreateAsyncScope();
        var services = scope.ServiceProvider;
        var state = services.GetRequiredService<IMentorStateService>();
        var boxes = 0;
        state.OnConfirmationRequired += request =>
        {
            Interlocked.Increment(ref boxes);
            if (approve) _ = state.ConfirmAsync(request.ActionId);
            else state.Cancel(request.ActionId);
        };
        await services.GetRequiredService<IMentorOrchestrator>()
            .SendMessageAsync(message);
        return boxes;
    }

    sealed class SignedIn(ClaimsPrincipal user) : AuthenticationStateProvider
    {
        public override Task<AuthenticationState> GetAuthenticationStateAsync()
            => Task.FromResult(new AuthenticationState(user));
    }
}
```

- **The user** comes from the `AuthenticationStateProvider`, as in a Blazor app. MentorAgent reads the roles from the user's role claims, so the test gives each user the roles to test.
- **The confirmation box** reaches the test through `IMentorStateService.OnConfirmationRequired`. The test answers it with `ConfirmAsync` or `Cancel`, as the user would with the buttons of the box.
- **No web server starts.** The test builds the app and calls `IMentorOrchestrator`, the service behind the widget, from a scope of its own.

The test project references the ASP.NET Core shared framework, because the host is a `WebApplication`. In a project with the `Microsoft.NET.Sdk` SDK, add it, with the namespaces that an ASP.NET Core project imports by itself:

```xml
<ItemGroup>
  <FrameworkReference Include="Microsoft.AspNetCore.App" />
  <Using Include="Microsoft.AspNetCore.Builder" />
  <Using Include="Microsoft.Extensions.DependencyInjection" />
</ItemGroup>
```

## Test actions, roles and confirmation

Each test picks a user and the tool that the fake model asks for, sends one message, and checks what happened:

```csharp title="CaseMergeTests.cs"
using System.Security.Claims;

public static class CaseMergeTests
{
    static readonly ClaimsPrincipal Supervisor = SignedInUser("Supervisors");
    static readonly ClaimsPrincipal Agent = SignedInUser("Agents");
    const string Ask = "Merge C-2 into C-1";

    // A supervisor confirms: one box, then the cases are merged
    public static async Task SupervisorMergesAfterConfirming()
    {
        var model = new ScriptedChatClient("merge_cases",
            new() { ["duplicate"] = "C-2", ["into"] = "C-1" });
        await using var app = MentorTestHost.Build(model, Supervisor);

        var boxes = await MentorTestHost.SendAsync(app, Ask, approve: true);

        var cases = app.Services.GetRequiredService<CaseMergeActions>();
        Check(boxes == 1, "one confirmation box");
        Check(cases.Merged.SequenceEqual(["C-2 into C-1"]), "the cases merged");
    }

    // The supervisor cancels: the box is shown, and nothing changes
    public static async Task NothingChangesWhenTheUserCancels()
    {
        var model = new ScriptedChatClient("merge_cases",
            new() { ["duplicate"] = "C-2", ["into"] = "C-1" });
        await using var app = MentorTestHost.Build(model, Supervisor);

        var boxes = await MentorTestHost.SendAsync(app, Ask, approve: false);

        var cases = app.Services.GetRequiredService<CaseMergeActions>();
        Check(boxes == 1, "one confirmation box");
        Check(cases.Merged.Count == 0, "nothing merged");
    }

    // An agent lacks the role: not offered, no box, refused if called anyway
    public static async Task AnAgentCannotMerge()
    {
        var model = new ScriptedChatClient("merge_cases",
            new() { ["duplicate"] = "C-2", ["into"] = "C-1" });
        await using var app = MentorTestHost.Build(model, Agent);

        var boxes = await MentorTestHost.SendAsync(app, Ask, approve: true);

        var cases = app.Services.GetRequiredService<CaseMergeActions>();
        Check(!model.OfferedTools.Contains("merge_cases"), "not offered");
        Check(model.ToolResult?.Contains("Access denied") == true, "refused");
        Check(boxes == 0, "no confirmation box");
        Check(cases.Merged.Count == 0, "nothing merged");
    }

    static ClaimsPrincipal SignedInUser(string role) => new(new ClaimsIdentity(
        [new Claim(ClaimTypes.Name, "alex"), new Claim(ClaimTypes.Role, role)],
        authenticationType: "Cookies"));

    // In your tests, use the assertions of your framework, such as Assert.True
    static void Check(bool condition, string expected)
    {
        if (!condition)
            throw new InvalidOperationException($"Expected: {expected}");
    }
}
```

- **A supervisor confirms.** One box is shown, and the method runs after the answer.
- **The supervisor cancels.** The box is shown, and the method never runs.
- **An agent lacks the role.** `merge_cases` is not offered to the model. The call that the fake model makes anyway is refused before any box appears, and the model gets the refusal as the result of the tool. See [Roles, confirmation and governance](https://docs.mentoragent.net/concepts/governance).

The examples check their results with a small `Check` method, so they compile without a test framework. In your tests, use the assertions of your framework, and mark each method as a test, for example with `[Fact]` in xUnit.

To test an [action policy](https://docs.mentoragent.net/reference/action-policies), register it in the test host as in your app. The call of the fake model goes through it. When the policy stops the call, `ToolResult` holds what the model reads: that a rule stopped the call, followed by the reason of your policy.

## Develop offline with a local model

To try the chat by hand without a key or a network, run a model on your machine. [Ollama](https://docs.mentoragent.net/configure/model-providers#ollama-and-other-local-models) exposes an endpoint that the OpenAI client can call, so only the `ChatClient` line changes. Pick a model that supports tool calling: a model without it answers, but never runs your actions.

A local model is slower than a hosted one, and it can choose differently. Use it to develop, and keep the tests on the fake model, which gives the same result every time.

## Catch regressions with the real model

A change to your instructions or to your model can make the answers worse or longer without breaking any test. `MentorEvaluator` sends a list of messages to your `ChatClient`, measures the tokens of each answer, and fails when an answer is empty or the tokens go over the budget:

```csharp title="EvaluationTests.cs"
using Novior.MentorAgent.Evaluation;   // MentorEvaluator, EvalCase

var evaluator = services.GetRequiredService<MentorEvaluator>();
var report = await evaluator.RunAsync(
    [
        new EvalCase("Who owns case C-1?", MaxTokens: 1500),
        new EvalCase("What can you do for me?", MaxTokens: 1500),
    ],
    new MentorEvalOptions
    {
        SystemInstructions = "You are the assistant of a case desk.",
        MaxTotalTokens = 3000,   // fails when a change makes answers longer
    });
report.ThrowIfFailed();   // fails the test on a regression
```

- **It calls the real model**, so it costs money and needs the key. Run it on its own schedule, not on every build.
- **It tests the answers, not your actions.** Each message goes straight to the `ChatClient`, with the instructions in `SystemInstructions`: no tools, no roles and no confirmation. To test your actions, use the fake model above.
- **With `Judge = true` and an expectation in each `EvalCase`**, a model scores how well each answer meets the expectation, and `MinQuality` sets the lowest score that passes.

## Common errors

- **The test waits and never ends.** A confirmation box is waiting for an answer. Subscribe to `OnConfirmationRequired` before you send the message, in the same scope.
- **No confirmation box, and the method never runs.** The user does not hold the role of the action. Check the claims of the test user: MentorAgent reads them with `IsInRole`.
- **The fake model asks for the tool, but nothing runs.** The tool name is not the one of your action. A tool name is the method name in snake case, with a trailing `Async` dropped first: `MergeCasesAsync` is `merge_cases`.
- **The test throws `InvalidOperationException`: `Configure ChatClient or Agent`.** The fake model is not in `ChatClient`. MentorAgent checks its options the first time it reads them, here at the first message. Set the fake model in the `AddMentorAgent` callback, as in the test host.

---

- Previous: [Roles from your sign-in](https://docs.mentoragent.net/configure/roles-from-sign-in.md)
- Next: [Cost and latency](https://docs.mentoragent.net/configure/cost-and-latency.md)