The road your request travels
The big picture — five places
- Your computer
- The
- Remote machines
The program on my computer
A , a , an app. It takes your request and decides what to send.
The network
The road across the internet. Nothing but text and files travels on it.
The distant calculating machine
A in a takes the request and hands it to a that already has the loaded, so the model is not opened again for every question. A very large model may be spread over several chips or machines.
The calculating model
A neural learns numerical from examples. At generation time, those weights and the available determine scores for candidate . The does not simply look up a stored internet sentence or count only the word immediately before it.
Back on my computer
The shows the answer, and — only if you allow it — does the work.
Zones 1 and 5 are the same computer: yours
Opening up place three
-
Large files on hold the learned numbers and information used to load the .
-
A reads it and puts it on a chip built for calculating, then keeps it there.
-
A doorkeeper checks the key and the request waits its turn.
-
The calculating runs against the numbers that are already up there.
Step by step
Eight steps, in order. Step eight is where the road forks.
You send it
Keyboard and mouse go through the , which hands the signal to the you are looking at. The microphone is different: only if you allow it, the program takes the sound and turns it into letters with — or sends the sound to a that can listen directly.
The program packs a bag
Your question, the recent conversation and the files or images you attached go into the bag, along with the instructions and tools the app has switched on. It does not scoop up your whole — only what you allowed goes along. The bag has a size limit, so the app trims, shortens or leaves out older parts. That is why an can seem to forget what you said earlier.
It crosses the internet
Text usually travels by , the same locked road a web page uses. Things that must keep talking both ways, like live voice, keep a connection open instead.
The machine receives it
The — the that holds the 's huge block of numbers in and does the calculating — cuts the text it received into , the small pieces a model reads. If the model is not in memory yet, it has to be loaded first, which is why the first answer is slow.
It draws the next token
The gives every possible next a score and picks one. It adds that token to the end and comes back to this same step, again and again, until a token meaning "finished" appears or a length limit is reached. How it picks is a setting: a low takes the top score more often, so the answers vary less — but a steadier answer is not a truer one.
back to this step, token by token
It sends back as it goes
The answer does not wait until it is finished. Text arrives a few pieces at a time; some sound, pictures and video are streamed too, while others arrive as a finished file.
The program puts it on screen
That is why letters appear as if someone were typing. Nothing is being slowly thought out in front of you — the pieces are simply arriving, and how big they are and how fast they come is the service's choice.
Here the road forks
If the answer is only an answer, you are done. If it contains "please do this", follow the second road.
Two roads, not oneTwo roads, not one
Drawing these as a single road is where the frightening misunderstanding comes from. They are not the same.
Talking only
-
Text on screen
-
Ends here
Read more
Letters flow onto the screen and that is the end of it. Nothing on your computer is touched. Lyen, the study helper on this site, works this way.
Borrowing hands
-
Spotting a tool request
-
Asking permission
-
Actually doing it
-
Packing the result
The result goes into the bag at step two and out again at step three
Read more
What comes back is still only text. When that text is a tool request — "edit this file" — the on your computer checks it and does it instead. Then it packs the result into the bag at step two and sends it off again at step three, and the conversation continues. That loop is what people call an .
A well-built program asks you before it deletes a file or runs a command — and it is people who decide to switch that question off. That question is the safety catch: the program enforces it, not a polite instruction to the model.
If we must compare it to something
-
One body swaps its calculating part out. The same can be wired to a model from a different company tomorrow.
-
One calculating part serves crowds of people who do not know each other, all at the same time. Nothing alive could do that.
Read more
The model is a part that only calculates; the body is the program on your computer, or the service around it. A calculating part has no hands — it cannot create a file or open an app by itself. It can only ask, in text. Whatever actually moves is done by the program, within what it is allowed to do.
Two more things
The machine is not always far away
This very site is that picture
| 1 · Your app | 2 · The | 3 · Remote machine | 4 · The calculating | 5 · Your screen | |
|---|---|---|---|---|---|
| Inside my own | Yes | No | No | Yes | Yes |
| By way of our | Yes | Yes | Yes | Yes | Yes |
| On a distant company’s machine | Yes | Yes | Yes | Yes | Yes |
On the middle road (through our ), the real calculating still happens on the third road, on a company's machine far away. Our server only checks the request and carries it.
Read more
Small models can run inside your own computer. The voice on this site works that way: once the is downloaded, turning text into speech happens on your device. Chatting is different — those messages still go through SONUB to the service.
Open the study helper and ask it something. Your sends it to our server, and our server hands it to a distant calculating machine. The letters coming back in pieces is step six; the letters growing on your screen is step seven. The helper stays on the talking-only road: it never touches a file on your computer.
The study helper. Our server passes the request along; it does not run a model.
This is where the calculating happens. Which runtime they use in there is not published.
Notes for grown-ups
Slightly more exact wording, for anyone who wants it.
Token
A model does not read words. A cuts the text into — sometimes a whole word, sometimes a few letters, sometimes a single — and each one becomes a number. Different models cut the same sentence differently, so the same text can count as a different number of tokens.
Streaming
Partial output is pushed out as it is produced, usually over . How big each piece is differs by product; some send a sentence at a time rather than a token.
Kinds of model
(large language model), (small language model), (reads pictures as well), and for images and video. LLMs and SLMs differ mainly in size, VLMs also read pictures, and diffusion models produce pictures and video rather than text.
The pick
How the next token is chosen is a setting. Turn the down and the model takes the highest score more often, so the same input gives more similar answers; turn it up and they vary more. Even at zero, real services do not promise exactly the same answer every time, and a steadier answer is not a more correct one.
RAG, MCP, skills
Names for ways of putting more into the bag. searches documents and attaches what it finds; is a common way to connect tools; skills are prepared sets of instructions. Which of them are available is decided by the app's settings and — never by the model alone.
Programs that run models
What this site actually uses is (for speech) and (for dictation). For large language models on a server, programs with the same job — , , and others — are used instead. Which company runs which is not published.
Weights, parameter count and file size
Weights, parameter count and file size
are numbers learned during . include weights and other learned values, such as biases. A 70B has about 70 billion parameters; reduces the used to store them, not their count. Here, “weight size” means the size of all these stored model parameters.
Weight size in bytes ≈ parameter count × bits per parameter ÷ 8. Divide by 8 because 1 byte is 8 bits. For 70B at 4 bits: 70,000,000,000 × 4 ÷ 8 = 35,000,000,000 bytes, or 35 in decimal units. This excludes quantization and working .
Q4, Q6 and Q8: the same numbers in less space
Q4, Q6 and Q8: the same numbers in less space
, Q6 and Q8 usually mean roughly 4, 6 and 8 per . For the same 70B , the theoretical sizes are 35 , 52.5 GB and 70 GB. Against a 16-bit baseline of 140 GB, these are 25%, 37.5% and 50%.
These are estimates, not exact download sizes. Formats such as mix precisions and add scales and . Quantization approximates values more coarsely and can reduce answer quality; it does not remove .
Does an unquantized original always mean 16 bits?
Does an unquantized original always mean 16 bits?
No. Many released store their original in or BF16, both 16-bit formats. They use the same storage per value but differ in range and precision. This comparison uses a 16-bit original as an example.
“Original” is not a bit width: a checkpoint may instead use (32 ), mixed precision, or already weights. Check the and the actual . Calling every original “16-bit” would be incorrect.
Does a smaller file use less memory and GPU?
Does a smaller file use less memory and GPU?
Saving usually makes the files smaller. If the keeps them packed in , weight memory also shrinks. Quantizing only while loading can leave the original files unchanged; expanding weights while loading can increase memory again.
memory ≈ weights + + temporary buffers + . Weight quantization does not automatically quantize the KV cache. Longer conversations and more simultaneous requests can need more . / utilization and speed do not scale directly with bit width: support, unpacking and transfer matter.
Is a model one file containing numbers and a program?
Is a model one file containing numbers and a program?
The largest part of a typical download is learned numbers arranged in . A model can occupy one file or many shards, with configuration and files alongside. Settings describe the architecture; the tokenizer maps text pieces to .
can bundle tensors, architecture and tokenizer information; Safetensors stores tensors and metadata, with architecture settings often separate. can also describe a . Describing a structure is different from including the engine that executes it. such as or are separate that read this information and run calculations on /.
Think of a recipe, its measured ingredients and a cook: the model describes what to calculate and the learned values to use; the runner performs the steps. It is misleading to say that an ordinary model file contains the entire execution program.
Numbers, precision, memory — and the runtime that puts them to work
-
Model file Sheet music
-
Runtime Conductor
-
CPU · GPU Performers
| Best at | |
|---|---|
| Many kinds of on many devices | |
| Running on your own machine () | |
| Huge models on personal PCs |
Read more
An is a learned arrangement of vast numbers and computation. A is the engine that loads that arrangement into and executes it on a , , or another .
Think of the model file as a score, the CPU and GPU as musicians, and the runtime as the conductor that reads the score and coordinates the performance.
— A general-purpose, cross-platform engine that can run models for images, speech, language, and more through hardware-specific .
— An engine focused on running and with minimal setup across and , commonly using model files.
— A runtime built to run very large models (models that pick only some of their many experts for each step) on a personal computer. It ties GPU, CPU, main memory (RAM) and the links between them together as if they were one machine.
What is a parameter?
- Matrix
- = 1
- Matrix
[0.2, −0.7, 1.3]
1 vector · 3 scalar numbers · 3 parameters
In , B means billion—not . An 8B model has about eight billion parameters, each a single number (a scalar), not eight billion .
Parameters and weights
- Weights
- Biases
- Embeddings
- Normalization scales
y = w₁x₁ + w₂x₂ + bEvery is a parameter, but not every parameter is a weight.
Read more
A parameter is one scalar number learned or adjusted by the model. A vector, matrix, or tensor is a ; every scalar number inside it counts as one parameter.
A weight is a major kind of parameter: it controls how strongly an input affects a result. Biases, , and may also be parameters.
A parameter is not a box holding a whole fact. The tensor or matrix is the cabinet, and each parameter is one numbered slot inside it. Bit width says how precisely that number is recorded.
How many bits does one parameter use?
Read more
A parameter is one number, but it can be stored at . Fewer bits usually reduce memory use, while and implementation details add some .
Why is an 8B model about 8 GB? (at 8-bit)
weight size ≈ parameter count × bits per parameter ÷ 8These are simplified decimal estimates for weights only. Real quantized files can be larger because they also store scales, metadata, and other structures.
Read more
At 8-bit precision, each parameter takes about one byte. Eight billion parameters therefore need roughly 8 for weights alone. The same parameter count takes a different amount at a different precision.
MoE — a model where only a few experts work
MoE (Mixture of Experts) means "a combination of many experts". The model holds many specialised parts (experts) and, depending on the input, computes with only some of them.
- Experts picked for this token
- Experts resting this time (still in memory)
Reading the name — Qwen3-Coder 30B-A3B
30BThe whole model has about 30 billion parameters.A3BAbout 3 billion parameters take part in computing each (A = Active).Effect — it needs less computation than a model that computes all 30 billion parameters every time.
Memory — the experts that might be picked must be ready, so you still need enough RAM to hold the whole model.
That makes MoE a good candidate for a CPU without a GPU (for example 12 vCPU and 48 GB RAM): the big model fits in memory while each step computes much less. Real speed still depends on the CPU and memory performance.
Read more
The experts in MoE do not each own a human-style subject. They are small whose roles split up by themselves during . The router is trained too: for every token it scores the experts and picks the best few, so different tokens in the same sentence can use different experts.
Numbers from the official Qwen model card (checked 2026-10-23)
Running the model needs more memory
- Weights
- KV cache
- Compute buffers
- Runtime overhead
The grows with longer
working memory = weights + KV cache + compute buffers + runtime overheadRead more
The weight file is only the starting point. Inference also needs a KV cache for earlier tokens, temporary , and runtime overhead. A theoretically 4 GB model may therefore need 5–8 GB or more; longer context usually grows the KV cache.
How can small numbers express so much knowledge?
Read more
Eight bits can represent 256 levels, but no single parameter contains a sentence such as “Seoul is the capital of South Korea.” Knowledge is distributed across patterns formed by billions of parameters interacting through many layers.
One is only a color value. Arrange millions of pixels and a face or landscape appears. Model knowledge emerges from distributed numerical patterns in a similar way.
The model is a vast pattern of scalar parameters grouped into vectors, matrices, and tensors. Precision determines the theoretical weight size; the runtime loads those weights and uses additional working memory to execute them on the available hardware.
The same road, step by step
- 1 From my hands to the program
- 2 Packing what gets sent
- 3 Crossing the internet
- 4 At the server door
- 5 Inside the server — the machine and the runtime
- 6 The model calculates
- 7 One token at a time, over and over
- 8 The way back
- 9 The program takes it in
- 10 Borrowing hands, over and over
1 From my hands to the program
-
A key goes down
A tiny computer inside the keyboard notices which key moved, turns it into a number, and sends it down the cable.
-
The operating system catches it
A driver in the turns that number into an and it for whichever window is in front.
-
The program pulls it off the queue
Every runs a loop that takes one event at a time and deals with it. The letter appearing in the box is this step.
-
The microphone takes another route
Sound arrives as tens of thousands of measurements a second. A model turns the spoken part into letters before the text model sees it — though some newer models can listen to the sound directly.
2 Packing what gets sent
-
The system prompt
Rules written in advance by whoever built the program go in first — what it is, and how it should answer.
-
The conversation so far
Earlier turns go in, in order. There is a limit, so the app drops, shortens or summarizes the oldest ones first — and whatever is left out is simply not seen.
-
Whatever you attached
An image or file is attached in a form the service accepts: raw bytes, a file , a link, or a long line of letters called . From a the service may pull out the text, take pictures of the pages, or read them with .
-
Searching your documents (RAG)
This is . Your question is turned into numbers (or keywords), and the pieces of your documents that match it best are found and packed in too. The model itself is not retrained — and the pieces it finds still need checking.
-
The list of tools
The name and input shape of every tool the app allows are written out like a menu. The app can list its own tools, or get them from an server.
-
Skills and commands
Bundles of instructions someone wrote earlier — skills — get slotted in when the task needs them. A can also carry scripts and reference files, so read it before you trust it.
-
Making it fit
The whole bag is counted in tokens. If it goes over the limit, the app has to cut, shorten or leave something out — usually the oldest material first.
3 Crossing the internet
-
Finding the address
The server’s name is looked up and turned into a numeric address ().
-
Opening the road and locking it
Before anything is sent, your computer and the server open a road and agree on a , so nobody in between can read what passes ( and ; the newer does both with ).
-
Saying who you are
A key or a token is attached to the head of the request. This is what decides whose account gets charged.
-
Sending the request
Everything packed so far becomes one request, usually written as , and goes out. When both sides need to keep talking — live voice, for example — a connection is held open instead.
4 At the server door
-
The doorkeeper
The key is checked, the call rate is checked, and the request is handed to a machine that is free.
-
The queue
Requests are not calculated right away; they wait briefly in line. When their turn comes, the runtime works through several waiting questions at once against the one set of numbers it already holds.
-
Checking what came in
The request is looked over against the rules before anything is calculated.
5 Inside the server — the machine and the runtime
-
A file of numbers lies asleep
A model that has finished training is a large file on a . By itself it calculates nothing. The on this site is four files, about 398 MB together.
-
A runtime loads it, once
When the server starts (or the app gets ready), the runtime reads that file, prepares it to run and keeps it in memory, so it is not opened again for every question. If memory runs short or the server restarts, it has to be loaded again.
-
An accelerator takes the multiplying
The same multiplication has to happen an enormous number of times, so it is handed to a chip built for exactly that, with fast memory beside it where the numbers sit. An ordinary CPU can do it too — it is only slower.
The runtime does not live inside the accelerator. It is a separate program that reads the file and puts the numbers into the memory next to that chip.
Grouping several requests is step 17, and keeping the finished work is step 27 — this same program does both.
6 The model calculates
-
Cutting into tokens
The text is cut into the pieces this model's knows, and each piece gets a fixed number (an ID). Another model may cut the same text into different pieces.
-
Numbers into lists of numbers
Each ID picks out a long list of numbers (an embedding vector) learned during training. The ID number itself means nothing; it is these lists that can end up alike for pieces that are used in alike ways.
-
Marking the order
Where each token sits is given as numbers too — without it the model could not tell "dog bites man" from "man bites dog". Models do this in different ways; (RoPE) is one common way.
-
Looking at each other (attention)
: for every token the model works out how much to look at each earlier token, and mixes their information together accordingly. It only looks backwards — never at tokens that have not been written yet.
-
Layer after layer
The numbers pass through many layers — dozens in large models — each doing attention plus a round of multiplying and adding. Every layer shifts the numbers a little; how many layers there are depends on the model.
-
Scoring everything
The last layer gives a score to every token it knows — tens or hundreds of thousands of scores at once, depending on the model.
-
Picking one
The final scores are turned into chances (). makes the chances sharper or flatter, keep only the likeliest few, and then one token is drawn. Always taking the highest score is called .
7 One token at a time, over and over
-
Adding it to the end
-
Keeping the work already done
Numbers already worked out for earlier tokens are kept (the KV cache) instead of being redone. That saves time but uses memory, and the overall speed still depends on how long the text is, how busy the server is and the .
-
Back to step 22
The text, now one token longer, goes through the model once more — the parts already worked out are reused, thanks to step 27. Round and round, until a token meaning "finished" appears or the length limit is reached.
This is the loop that writes the answer — one token at a time, back to step 22.
8 The way back
-
Turning token numbers back into readable text
-
Letting it flow as it comes
Rather than waiting for the whole answer, small chunks are pushed out as they are produced ().
-
Checking what goes out
The outgoing text is looked over against the rules as well.
-
Counting what was used
Tokens in and tokens out are counted. This is where the bill comes from.
9 The program takes it in
-
Stitching the arriving chunks in order
-
Drawing it on screen
is read and turned into headings, lists and boxes. This is the part that looks like typing.
-
Spotting a tool request
The program watches for a in the agreed format sitting inside the answer.
-
Checking the arguments
Whether that tool exists, and whether the arguments have the right shape, is checked before anything runs.
10 Borrowing hands, over and over
-
Asking permission
For anything risky — deleting a file, running a command — a well-built app asks you first. That check lives in the app, not in the model, and switching it off is a decision a person makes.
-
Actually doing it
The app does what the model asked for: a file is edited, a command is run, or another program (an MCP server) is called — only within the it was given.
-
Packing the result
What came out — or why it failed — is added to the conversation from step 6 as a new line.
-
Back to step 15
The refilled bag, now holding that result, is sent again. This repeats until the model says it is finished, or the app stops it because of a limit, a cancel or an error.
This is the loop people call an — back to step 15, again and again.
Read next
Loading…
Choose a pending booking to cancel. Each cancellation requires confirmation.
Your :