Skip to content

LLMs

The LLMs view allows administrators to configure connections to large language model providers. An LLM configuration defines the connection details and model selection that an assistant uses to generate responses. AI Support supports four LLM provider types: OpenAI, Anthropic, Gemini, and Ollama.

LLMs List View

LLMs List View

LLMs Grid

The grid displays the following columns:

  • Id: Unique numeric identifier of the LLM configuration.
  • Name: The display name of the LLM configuration.
  • Type: The provider, shown with its logo and name: OpenAI, Ollama, Anthropic, or Gemini.
  • Status: A badge indicating whether the LLM is enabled or disabled. Disabled LLMs are not available for selection when configuring assistants.
  • System: A badge marking the LLM used for the appjar's internal tasks. See The System LLM.
  • Actions: A per-row actions menu.

Filters

The view provides the following filter fields:

  • Id: Filters by the LLM's numeric identifier.
  • Name: Filters LLMs whose name contains the entered text.
  • Type: A dropdown to filter by provider type (OpenAI, Ollama, Anthropic, or Gemini).
  • Status: A dropdown to filter by enabled or disabled status.

Once any filter field has a value, the Filter and Clear buttons become active. Pressing Enter in any filter field also applies the filters. When the filters do not fit in two rows, they collapse into a Filters button; see Filtering the Lists.

At the top right of the filter area, the + LLM button opens the dialog for creating a new LLM configuration.

Row Actions

Click the actions menu on any row to access the following options:

  • Edit: Opens the LLM creation/edition dialog with the current configuration pre-filled.
  • Enable / Disable: Toggles the LLM's enabled status.
  • Set as system: Makes this the system LLM. It appears only on models that are not already the system LLM, and asks for confirmation, naming the model that is losing the flag.
  • Delete: Removes the LLM configuration. If the LLM is assigned to one or more assistants, a replacement dialog asks for the model to reassign them to, and states how many assistants the action will modify.

The model marked as system cannot be deleted. Doing so reports that the model is marked as system and that the flag must be moved first.

The System LLM

At most one LLM is marked as the system LLM. It handles the work that happens around a conversation rather than inside it — generating the short session summaries shown on session cards, classifying messages when profanity checking is enabled, and transcribing audio, such as voice notes, for assistants whose model cannot hear it.

The flag adds this responsibility rather than reserving the model for it. A system LLM can also be assigned to assistants and used to answer users like any other model, which is the usual arrangement when a single model serves the whole application.

Marking one is optional. When no LLM is marked as the system LLM — or when the one marked is disabled — summaries and message classification fall back to the language model of the assistant handling the conversation, so nothing stops working. Different conversations may then use different models for the same internal task.

Transcription is the exception: only the system LLM transcribes audio, and only when its provider offers speech-to-text. OpenAI and Gemini do; Anthropic and Ollama do not. Without a system LLM that can transcribe, audio sent to a model that cannot hear it reaches the model as a note naming the file, and users can record voice notes only for assistants whose own model accepts audio.

The flag is therefore worth setting when the internal tasks should be handled by a specific model regardless of the assistant in use: a cheaper or faster model for summarising, for example, while the assistants answer with a stronger one. It is also required for voice notes to be transcribed.

Creating and Editing an LLM

Clicking the + LLM button or the Edit action opens a dialog with the following fields:

LLM Creation Dialog

LLM Creation/Edition Dialog
  • Name (required): The display name for this LLM configuration. Must be unique across all LLMs.
  • System: A toggle that marks this model as the system LLM, described in the dialog as being used for internal tasks, with only one allowed. When it is on and the selected provider has no speech-to-text, the dialog warns Voice notes will not be transcribed: this provider has no speech-to-text.
  • Enabled: A toggle that determines whether the LLM is available for use in assistant configurations. Enabled by default.
  • Type (required): A dropdown to select the provider. Choosing a provider changes the additional configuration fields displayed in the dialog.
  • Accepted attachments: The kinds of files the model can read besides text, described in Accepted Attachments.

The dialog opens with an About LLMs section explaining what a language model is in this context. Its footer holds the Test button, described in Testing the Configuration, next to Cancel and Save.

Note

If saving reports that the LLM could not be saved: the database rejected it, the application's database was created by an earlier version of AI Support and still restricts the provider types or the size of an LLM's configuration. The developers of the application must apply the upgrade notes before the LLM can be saved.

OpenAI Configuration

When the type is set to OpenAI, the following fields are displayed:

  • Url (required): The API endpoint URL. Defaults to https://api.openai.com/v1. This can be changed to point to a compatible API endpoint, such as an Azure OpenAI deployment or another OpenAI-compatible service.
  • Key (required): The API key used for authentication with the OpenAI service. The field is masked: the value is shown as dots, and the eye button at its right reveals it while it is being checked or corrected.
  • Model (required): The model name to use (e.g., gpt-4.1, gpt-5). The available models depend on the API key's permissions and the provider's offerings.

OpenAI LLM dialog with the masked Key field

OpenAI configuration, with the API key concealed

Anthropic Configuration

When the type is set to Anthropic, the dialog shows the same fields as for OpenAI:

  • Url (required): The API endpoint URL. Defaults to https://api.anthropic.com/v1/.
  • Key (required): The Anthropic API key. The field is masked, with an eye button that reveals it.
  • Model (required): The model name to use (e.g., claude-sonnet-4-5, claude-haiku-4-5).

Gemini Configuration

When the type is set to Gemini, the dialog shows the same fields:

  • Url (required): The API endpoint URL. Defaults to https://generativelanguage.googleapis.com/v1beta.
  • Key (required): The Gemini API key from Google. The field is masked, with an eye button that reveals it.
  • Model (required): The model name to use (e.g., gemini-2.5-flash, gemini-2.5-pro).

Ollama Configuration

When the type is set to Ollama, the following fields are displayed:

  • Host (required): The scheme and hostname of the Ollama server. Defaults to http://localhost. The value must include a scheme and a hostname; the dialog reports which of the two is missing.
  • Port (required): The port number of the Ollama service. Defaults to 11434, and must be between 1 and 65535.
  • Model (required): The model name to use (e.g., llama3.2:3b, mistral). The available models depend on what has been pulled to the Ollama server.

The resolved endpoint is shown beneath the fields as the host and port are edited, so the address can be checked before saving.

Accepted Attachments

Models differ in what they can read besides text. The Accepted attachments checkboxes record which kinds of files this model accepts, and they decide what users can attach in conversations with the assistants that use it. Only the kinds the provider can carry are offered:

  • OpenAI and Gemini: Image, Video, Audio, and PDF.
  • Anthropic: Image and PDF.
  • Ollama: Image.

The helper text reads Prefilled from the model. Text is always accepted. When an LLM is created, and whenever its type or model name is changed, the checkboxes are filled in from what is known about the model: for Ollama, from what the Ollama server reports about it; for the other providers, from a catalog of known models that ships with AI Support. A model that is not in the catalog starts with text only. The selection stays editable, so it can be corrected for a model the catalog does not know. An LLM saved by an earlier version uses the catalog's selection for its model until it is edited.

A file the model does not accept is not lost: a PDF file reaches the model as its text, audio reaches it as its transcript when the system LLM can transcribe it, and any other file reaches it as a note naming the file. The user who attached it is told. Even with Audio checked, a model only hears audio in the formats its provider reads: MP3 and WAV for OpenAI, and also OGG, WebM, M4A, AAC, FLAC, and AIFF for Gemini.

Testing the Configuration

The Test button, at the left of the dialog footer, sends the model a short message with the configuration currently in the dialog, without saving it. Its tooltip reads Send the model a short message with this configuration. The fields are validated first, as they are when saving.

When the model answers, the button shows a check mark for a few seconds and a notification reports that the model answered. When it does not, the button shows a cross for a few seconds and the The model did not answer dialog shows the reason the provider gave, with any credentials removed from it. Testing before saving is the quickest way to catch a wrong key, endpoint, or model name before an assistant fails in front of users.

Choosing Between Providers

OpenAI is suitable for organizations that prefer a cloud-hosted solution with access to the latest commercial models. It requires an API key and incurs per-token costs based on usage. The same type also serves any other provider with an OpenAI-compatible API.

Anthropic and Gemini are cloud-hosted alternatives, also reached with an API key and billed per token. Neither offers a separate moderation service, so when one of them screens messages for inappropriate language, it does so through the model itself, as Ollama does. A Gemini system LLM can transcribe audio; an Anthropic one cannot. The LLM Inspector reports their token counts as Unknown, since AI Support has no tokenizer for them.

Ollama is suitable for organizations that need to run models locally, either for data privacy reasons, to reduce latency, or to avoid per-token costs. It requires a self-hosted Ollama server with the desired models pre-installed.

All provider types can be configured simultaneously, allowing administrators to create different assistants powered by different models depending on the use case.