Documentation / Overview
Fermion runtime documentation
Run Neutrino and Phonon models locally: chat, transcribe and dictate, expose an OpenAI-compatible API, and call tools.
fermion 0.2.10Python 3.10 to 3.14macOS, Linux, and WindowsApache-2.0
Quickstart
Start a local chat
Python 3.10 or newer. The first run downloads and verifies the model.
shell
pip install fermion-research
fermion chat neutrino-8bSpeech
Transcribe and dictate
Same install. The first run downloads and verifies Phonon-2.
shell
fermion transcribe phonon-2 recording.wav
fermion listen phonon-2Start building
Choose the path closest to what you are trying to do.
Install and runInstall the package and start a local Neutrino chat.Transcribe and dictateTurn audio into text, dictate live in the terminal, or stream over WebSocket.Serve an APIRun an OpenAI-compatible endpoint with streaming and tools.Call toolsGive the model OpenAI function schemas and handle its calls.Use speculative decodingPair the 8B model with the 0.6B draft and verify identity.Read the CLI referenceFind commands, shared flags, defaults, and exit behavior.
Guides and reference
Getting startedModels, downloads, and storageKnow what the CLI downloads, where it stores the files, and how much free space the first run needs.Getting startedChoose and verify a backendFermion selects an optimized native CPU runtime where available and a PyTorch reference path elsewhere.GuidesChat and generateUse the interactive session for conversation and the one-shot command for deterministic scripts.GuidesTransformers, GGUF, and MLXUse the same Neutrino release through the library path, the Fermion llama.cpp fork, or Apple GPU code.ReferenceServer API referenceRoutes, request fields, streaming frames, audio routes, errors, context clamps, and operational health data.ReferenceEnvironment variablesConfigure caches, transport behavior, native execution, sampling, sessions, and Hugging Face progress.