A language model is a stack of matrices. Neutrino-1 8B is 36 decoder layers of them, attention projections and feed-forward blocks, roughly eight billion learned values. For a decade the field has stored those values one way, as floating point numbers of 16 or more bits each, on an assumption that is rarely said out loud: that intelligence needs precision, and that shrinking the numbers shrinks the mind.
Neutrino models store almost every weight as a choice among three states. Measured against Qwen3-8B, the full-precision base it was built from, on identical public harnesses, Neutrino-1 8B holds about 96 percent of general knowledge at 4.2 times fewer bytes. Most of what these systems know never lived in the low-order digits.
1. The descent in stored bits
Every halving of stored width halves the bytes a decoder has to move to emit one token, which is why the industry has spent a decade walking down the ladder. Eight-bit inference became standard practice in the late 2010s. Four-bit weights went mainstream with the open 7B models of 2023. Ternary weights were proposed for small networks a decade ago, and by 2024 a transformer language model had been trained to live at that depth. Each of those steps traded arithmetic for capability at a rate the field accepted. Neutrino-1 8B holds 72.1 on the full 57-subject knowledge battery while storing its projections three states wide, which is the step the ladder was missing.
2. One decoder layer
The unit that repeats 36 times is a decoder layer of seven weight matrices. Four of them are attention: a query projection of 4096 by 4096, key and value projections a quarter that width because 32 query heads share 8 key and value heads at head dimension 128, and an output projection back to 4096. Three of them are feed-forward: a gate and an up projection that lift the residual stream from 4096 to 12288, and a down projection that returns it. Those seven matrices are 192,937,984 ternary weights, identical in every layer, and the feed-forward group is 78.3 percent of them.
The 21.7 to 78.3 split is the reason depth-wise measurements of this model are dominated by what the feed-forward matrices do. It is also why the container is perfectly regular: no layer carries a different rank, a different width, or a different budget, so the weight lane costs exactly 72,351,744 bytes per layer, 36 times over.
3. The representation
A ternary weight is a three-way choice: minus, zero, or plus. The choice carries no magnitude of its own, so the format attaches magnitude at a coarser granularity. Weights are grouped into blocks, fixed runs of consecutive weights along the input dimension of each linear layer, and every block shares one higher-precision scale. The scale sets how loud every choice in its block is; the states set the direction of each contribution, or remove it. A matrix that spent 16 bits on every weight becomes a stream of three-state decisions with a thin lane of block scales riding beside it, and the stored weight is always the product of the two: state times scale.
Zero is the state that changes the statistics. A floating point network can only make a connection small; a ternary network can delete it. Trained natively, the zero state becomes learned connectivity: the format records not only how the surviving connections vote but which connections exist at all. The trained weight distribution shows the difference. In place of the dense bell of a float layer there are three mass points, with most of the mass on zero: the format sets how much of each tensor falls silent, and the learned weights decide which connections go.
Not every tensor rides the ternary lane, and the exceptions are structural. A linear layer’s output feature sums hundreds of three-state contributions, so individual state errors cancel inside the sum; that summation is what makes the format survivable. An embedding lookup has no such summation: it returns one row verbatim, and any error in that row is the token’s entire representation. The output head faces the opposite edge of the same problem: it separates the next token from a vocabulary of 151,936 alternatives by small logit margins, precisely the differences a three-state grid cannot hold. The normalization gains are single per-channel multipliers, 308,224 values across the whole model that each scale an entire activation stream; there is no block of neighbors to average them against. Embeddings, output head, and norms therefore stay at higher precision, and that decision is what keeps every lookup and every logit exact.
4. Native training, not rounding
The obvious route into a coarse format is to train in floating point and round afterward. At ternary depth that route collapses. Published aggressive post-training rounding of strong 8B-class models lands near chance on four-choice knowledge batteries, around 25 on a 0 to 100 scale, where a model of the same class trained natively in the format holds 72.1.
The collapse has a mechanism, and it lives in the sum. One output feature of one linear layer is the accumulation of several hundred weight-times-activation terms, and a trained network stores its content in how those terms cancel against each other. Rounding projects each weight independently onto the nearest of a handful of levels. Every individual error stays inside the same small bound, and the errors are uncorrelated, so they accumulate along the row as a random walk instead of cancelling. The feature that leaves the layer is not a noisier version of the right answer; it is a different number, and 36 layers compound the difference. The failure is not noise. It is amnesia, and its signature is the cliff: scores do not degrade toward chance, they arrive there.
The published record separates on exactly that line. Two independent one-shot roundings of an 8B to two bits score 24.2 and 24.7 on five-shot MMLU, against 25.0 for answering at random, so the remaining signal is inside measurement noise of nothing. Three models whose training ran inside the coarse format instead land at 47.24, 65.75 and 72.1 at the same artifact size. The rounding route does not cost a few points at these depths; it costs the model.
Native training inverts the order of constraint and knowledge. Every forward pass the model runs during training is a ternary forward pass: the loss is computed against the format, and the gradient passes through the discretization to a higher-precision carrier that exists only inside the optimizer and never ships. Descent under that constraint never learns a solution the format cannot store. It selects sign patterns, zero placements, and block scales that work as a set, trading precision it does not have for structure it can afford. There is no rounding step at the end because there is nothing left to round. The format is the medium the model learns in, not a post-process applied to it. The training methods that hold this stable at frontier quality are unpublished.
5. The information price of a ternary weight
A choice among three equally likely states carries 1.58 bits of information in one stored weight. Storage hardware deals in whole bits, so a naive container spends two bits per state and wastes about a fifth of itself on the difference. Trained models widen the gap further: the three states are not equally common in practice, and a whole-bit container charges every state the same regardless. Both overspends are recoverable, and the release transport recovers them.
The weight stream ships coded near its information content and expands on the receiving machine. The 8B download is 2.56 GB and expands to the 3.88 GB artifact the engine serves, byte for byte. The same transport takes the 0.6B model from 328 MB on disk to a 238 MB download, within 0.007 percent of the order-0 coding bound for its stream. Nothing on the return path is approximated. The small file is not a compressed copy of the model; it is the model, written without wasted bits.
6. The shipped weights, layer by layer
The shipped container can be read directly, so we read it. Of Neutrino-1 8B’s 8.19 billion parameters, 6,945,767,424 are ternary states, every attention and feed-forward projection across the 36 layers, and just under 63 percent of them are exactly zero. The level and the shape of that silence have different owners. The format sets how much of the network falls silent; training decides where the silence falls. Reading all 252 tensors of the container gives the map below, one cell per projection per layer.
Two structures come out of the grid. The four attention projections hold a band 1.67 points wide across all 36 layers, from 61.84 percent at the layer 1 key projection to 63.51 percent at the layer 25 value projection; depth does not change how connected attention is. The feed-forward matrices deviate in exactly one place: layers 2 through 4, where the down projection reaches 72.47 percent zeros and the gate 70.48, roughly ten points above the body of the network, before both settle by layer 5. The single densest tensor in the model sits one layer earlier, the layer 1 down projection at 60.59 percent, so the model goes from its most connected feed-forward matrix to its most silent one in a single step.
The deviation is a property of the learned weights themselves. The early feed-forward rows carry heavier tails than the rest of the network, and nothing in the format or the training objective asked for it. It is the clearest depth-wise structure the container shows, and it appears only in the matrices that write back into the residual stream.
The surviving weights come out balanced. Across 6,945,767,424 ternary weights the model holds 18.677 percent plus against 18.694 percent minus, a gap of 0.017 of a point, and the balance holds layer by layer including through the layers 2 to 4 surge where both wings notch together. Nothing in the format, the container, or the training objective imposes that symmetry.
7. Where the bytes are
Parameters and bytes are two different accounts, and the format separates them. Of the 8B’s 8,190,735,360 parameters, 6,945,767,424 are ternary states, 1,244,659,712 are int8 embedding parameters across two untied vocabulary tables, and 308,224 are fp32 norm gains. Weighed in bytes, the same model is a 2.60 GB ternary lane, a 1.24 GB vocabulary, and 26 MB of scales, norms and row metadata. The ternary states are 84.8 percent of the parameters and 67.2 percent of the file, and the distance between those two figures is the whole point of the format.
At 0.6B the accounting tilts the other way. A vocabulary of 151,936 tokens costs what it costs whatever the model around it weighs, so one tied int8 table of 155,582,464 parameters is 47.5 percent of that file against a weight lane of 50.4 percent. The smaller the model, the larger the share of the download that is vocabulary, and the 8B’s 32.1 percent is the comfortable end of that curve.
Generation lives where the weights fit. The 3.88 GB artifact runs on an base-M5 MacBook at 33.7tokens per second, and on an NVIDIA L4 it decodes at 30.7 tokens per second inside 4.68 GiB of VRAM at 4k context, which puts it on 8 GB cards. Single-stream decode is bound by memory movement: every generated token reads every live weight once, so artifact size divides directly into token rate. The format’s eightfold reduction sets the ceiling, and the shipped artifact moves 4.2 times fewer bytes per token than Qwen3-8B at bf16 on the same memory system. Small is not a compromise against fast. Small is why fast happens.
8. Coded transport
The occupancy statistics return as wire cost. Layer by layer the ternary lane codes for download to between 0.516 and 0.569 of its raw bytes, and which layer lands where is not arbitrary: the zero share of a layer predicts its coded size at a correlation of minus 0.92 across the 36 layers. Layers 2 through 4, the silent ones, code smallest. The ordering holds by projection type as well, with the down projection tightest at 0.543 and the key projection loosest at 0.567.
The embedding lanes behave the opposite way. An int8 vocabulary table is close to incompressible and holds 0.883 of its raw size, so the download arithmetic is a tight weight lane blended with a dense vocabulary: 3.88 GB ships as 2.56 GB, 0.661 of raw, and the blend ratio is set by the composition in the previous section.
9. What scale changes and what it does not
Running the same walk over the 0.6B container separates the properties of the format from the properties of a particular model. Zero, plus and minus share land within four tenths of a point of the 8B across a fourteenfold difference in parameter count: 62.26 percent zero against 62.63, 18.87 plus against 18.68, 18.86 minus against 18.69. The state statistics are a constant of the format, not a feature of one training run.
Everything downstream of the state statistics moves. The 0.6B is 28 layers of hidden width 1024 with a feed-forward width of 3072, so attention is 40 percent of a layer against 21.7 percent at 8B, the ternary states are 73.9 percent of its parameters against 84.8, and the vocabulary crosses above the weight lane in the byte account. The format holds its shape; the accounting around it is a function of scale.
10. Retention, axis by axis
Retention resolves differently on every axis, and averaging it into one number would erase the science. Against Qwen3-8B at full precision, at 4.2 times fewer bytes and on identical public harnesses, Neutrino-1 8B holds 96 percent of general knowledge, 87 percent of knowledge on the re-annotated set, 87 percent of strict instruction following (every constraint in a prompt must pass), and 79 percent of tool calling. Divide each of those by the bytes it took to store, and the same four axes read 4.03, 3.65, 3.65 and 3.32 times the capability the reference carries per byte.
The ordering is the finding. Knowledge behaves like a bulk property of the weights: stored redundantly across many of them, protected by the same summation that protects a linear layer from per-state error, it survives the descent nearly intact. Behavior is a chain property, scored per episode: a strict instruction run or a tool-calling episode is credited only when every step lands, so an episode metric multiplies per-step behavior instead of averaging it. Knowledge above behavior is an ordering that has held across every model we have trained, and it tells you which axes respond to weights and which respond to verification.
One axis comes back ahead of the reference. The model holds its required answer format more reliably than the reference does, ending cleanly where the reference drifts. A model can leave floating point and come back better behaved. Discipline, unlike knowledge, is not a bulk property of the weights; it is a behavior training installs, and the format does not stop it.
11. The conservation of behavior
Behavioral axes are coupled, and the coupling is a property of post-training rather than of any one run. Stage after stage, pressure applied to one behavioral axis resolved onto one specific other axis instead of diffusing across the model. Pressure on tool calling resolved onto strict instruction following. Pressure on instruction following spared tool calling and resolved onto termination discipline. The exchange was axis-specific, repeatable and directional, as though the behaviors drew on one budget. Which axis receives the exchange under which pressure is measured stage by stage.
Multi-checker co-training moves the exchange out of the axes under pressure. When one stage verifies two behaviors simultaneously, each judged by its own criterion, both rise together: the first simultaneous two-axis gain in the program. The conservation itself holds, and it resolves onto whatever the verification set does not watch, which makes the width of the verification set the design variable.
The implication cuts in both directions. No behavioral gain is established until every axis on the model is measured under it, because the exchange resolves wherever nothing is watching. And an axis rises when it is verified, not when it is trained harder, so the lever is the width of simultaneous verification. That reframing, from training harder to verifying wider, now sets the design of every post-training stage we run.
12. The open floor
General knowledge survives at 96 percent of Qwen3-8B at full precision, one behavioral axis returns ahead of it, and the state statistics of the format hold constant from 0.6B to 8B across a fourteenfold difference in parameter count. Every measurement in this post is an instrument for one question: where the floor of arithmetic sits against the floor of intelligence, axis by axis. That question is the research program, and the format is the place it is now run.
