Evaluations

Rate responses, run arena-style head-to-heads, and build a personalised leaderboard so your organisation standardises on the right models.

Evaluations turn everyday chat feedback into evidence about which models work best for your organisation — not on generic public benchmarks, but on your real tasks, your language, your quality bar.

Why it matters

Public benchmarks don't reflect your workflows, your tone requirements, or your domain. IntraLLM builds a personalised leaderboard from ratings collected during normal work. Teams use it to decide which model to standardise on, and administrators use it to validate deployments (see Evaluations (Admin)).

Two ways to contribute

1. Rate during normal chat

  1. Chat as usual with a selected model.
  2. Give a thumbs up or thumbs down on responses (with an optional comment) via the response actions.
  3. For the rating to affect rankings, make a comparable sibling response available: either regenerate the reply or switch models and ask the same question again.

2. Arena mode (head-to-head)

  1. In the model selector choose Arena Model — it randomly draws from the available pool, removing brand bias.
  2. Add the second model with +.
  3. Ask the same question; compare both answers.
  4. Rate them. In head-to-heads an upvote on one side effectively downvotes the other — upvote only the clearly better answer.

Also useful: the scored Compare workspace (1–10 grades with comments) for deliberate evaluation sessions.

Topic tags

For organisations spanning many domains (support, engineering, creative, finance), ratings can be grouped by topic:

  • Automatic tagging attempts to label chats by topic; accuracy varies.
  • Manual tagging — while rating, add or correct the topic tag. Better tags yield better per-topic rankings, so teams can pick the best model per use case, not overall.

The leaderboard

  • Ratings feed an Elo-style rating — like any competitive ranking, positions stabilise as more comparisons accumulate. One rating proves little; a week of team usage is meaningful.
  • Rankings live in Admin Settings → Evaluations, where administrators review performance per model and per topic.
  • Results reflect your environment only.

Privacy

Evaluation data stays on your instance. Nothing leaves your environment unless your organisation explicitly opts into an external workflow. See Why IntraLLM on data sovereignty.