Engines and your hardware

The engine is what runs your offline models. The standard one works on any graphics card; optional engines get more out of specific hardware. Engine changes apply now, not at the next launch: installing or removing an engine reloads the model you have open onto the engine that now applies.

Standard engine

Ships with the app. Runs on any graphics card - NVIDIA, AMD, Intel, Apple - via Vulkan (Linux, Windows) or Metal (macOS), and falls back to your processor when needed. If the GPU path misbehaves, the app steps down gracefully and tells you; Try GPU again reloads the open model onto the graphics card at once. A reboot, or a run that never reached the card, is not counted as a crash.

Speed is one number with one meaning on every engine - completion tokens over the time they streamed - and the tokens panel also shows the reading speed when the engine reports it.

CUDA engine (NVIDIA)

A one-click ~850 MB add-on built for NVIDIA cards, offered on the home page when your machine qualifies and managed under Settings → Engines. It reads long prompts and documents much faster; generation speed varies by card generation. Needs an NVIDIA driver from 2023 or newer.

MLX engine (Apple Silicon, preview)

Optional, per model, chats only, no speed promise. Get a model's MLX version and chats for that model run on MLX - its row says so and keeps separate measured speeds for MLX and the standard engine. Remove the MLX version, or the engine, and the standard engine's fit applies again.

Your own server

Connect any OpenAI-compatible server you run - another machine with llama.cpp, a vLLM box, a Mac cluster. Settings → Engines → Your own server: paste the URL, the app health-checks it, measures its real speed, and its models join your picker. Your conversations go only to it.

With a server connected, a third auto mode appears - Auto - My Hardware - that treats your connected machines as routing candidates alongside this device, never the online relay.

Fine-tune this computer

At the bottom of the Engines page: worker threads, and the machine-wide generation settings every AI inherits unless it sets its own. See Fine-tune.

Utility work stays small

Memory, routing verdicts and document reading run on the optional helper - two small models on their own servers on your processor, downloaded only when you ask and removable in Settings → Storage. They never compete with your chat model for graphics memory.