OkayStudy
EN English 한국어 日本語 中文 हिन्दी বাংলা اردو Español Português Français Deutsch Русский العربية Bahasa Indonesia Tiếng Việt ไทย Türkçe

How to enable notifications

Notifications are unavailable in this browser or on this address. Try a supported browser on the secure SONUB site.

If notifications are blocked, allow them in your browser settings.

  1. Open the site controls beside the address bar, or your browser’s site settings. Set Notifications for this site to Allow.
  2. In your device settings, allow notifications for your browser and check Do Not Disturb or Focus.
  3. Return to SONUB, reload the page, and use the bell to enable notifications.

On iPhone or iPad, add SONUB to your Home Screen, open it there, then allow notifications.
Internals

The road your request travels

An office where a robot receptionist sends a question card to just three of many expert desks, and only those three lit-up experts work on it
Inside the model file

Numbers, precision, memory — and the runtime that puts them to work

  1. Model file Sheet music
  2. Runtime Conductor
  3. CPU · GPU Performers
Three well-known runtimes
Best at
Many kinds of on many devices
Running on your own machine ()
Huge models on personal PCs
Read more

An is a learned arrangement of vast numbers and computation. A is the engine that loads that arrangement into and executes it on a , , or another .

Think of the model file as a score, the CPU and GPU as musicians, and the runtime as the conductor that reads the score and coordinates the performance.

— A general-purpose, cross-platform engine that can run models for images, speech, language, and more through hardware-specific .

— An engine focused on running and with minimal setup across and , commonly using model files.

— A runtime built to run very large models (models that pick only some of their many experts for each step) on a personal computer. It ties GPU, CPU, main memory (RAM) and the links between them together as if they were one machine.

What is a parameter?

How parameters sit inside tensors
    • Matrix
        • = 1
[0.2, −0.7, 1.3] 1 vector · 3 scalar numbers · 3 parameters
0.10.2−0.30.7−0.40.9 This is one 2 × 3 , but it contains six numbers—so it contributes six .

In , B means billion—not . An 8B model has about eight billion parameters, each a single number (a scalar), not eight billion .

Parameters and weights

Parameters
  • Weights
  • Biases
  • Embeddings
  • Normalization scales
y = w₁x₁ + w₂x₂ + b

Every is a parameter, but not every parameter is a weight.

Read more

A parameter is one scalar number learned or adjusted by the model. A vector, matrix, or tensor is a ; every scalar number inside it counts as one parameter.

A weight is a major kind of parameter: it controls how strongly an input affects a result. Biases, , and may also be parameters.

A parameter is not a box holding a whole fact. The tensor or matrix is the cabinet, and each parameter is one numbered slot inside it. Bit width says how precisely that number is recorded.

How many bits does one parameter use?

Bits per parameter
  • 32 bit · 4 B
  • / BF16 16 bit · 2 B
  • 8 bit · 1 B
  • 4-bit 4 bit · ≈ 0.5 B
Read more

A parameter is one number, but it can be stored at . Fewer bits usually reduce memory use, while and implementation details add some .

Why is an 8B model about 8 GB? (at 8-bit)

weight size ≈ parameter count × bits per parameter ÷ 8
8B
  • FP16 ≈ 16 GB
  • 8-bit ≈ 8 GB
  • 4-bit ≈ 4 GB
32B
  • FP16 ≈ 64 GB
  • 8-bit ≈ 32 GB
  • 4-bit ≈ 16 GB
70B
  • FP16 ≈ 140 GB
  • 8-bit ≈ 70 GB
  • 4-bit ≈ 35 GB

These are simplified decimal estimates for weights only. Real quantized files can be larger because they also store scales, metadata, and other structures.

Read more

At 8-bit precision, each parameter takes about one byte. Eight billion parameters therefore need roughly 8 for weights alone. The same parameter count takes a different amount at a different precision.

MoE — a model where only a few experts work

MoE (Mixture of Experts) means "a combination of many experts". The model holds many specialised parts (experts) and, depending on the input, computes with only some of them.

An office where a robot receptionist sends a question card to just three of many expert desks, and only those three lit-up experts work on it
It is like a company with 30 experts where only the 3 needed for each question do the work. It does not mean there are exactly 30 or 3 experts.
Qwen3-Coder 30B-A3B — for each token, only 8 of its 128 experts compute
One token
Router — picks experts
On to the next step
  • Experts picked for this token
  • Experts resting this time (still in memory)

Reading the name — Qwen3-Coder 30B-A3B

30BThe whole model has about 30 billion parameters.
A3BAbout 3 billion parameters take part in computing each (A = Active).
Parameters (B = billion)
  • Whole model — what memory must hold ≈ 30.5B
  • What is computed per token ≈ 3.3B

Effect — it needs less computation than a model that computes all 30 billion parameters every time.

Memory — the experts that might be picked must be ready, so you still need enough RAM to hold the whole model.

That makes MoE a good candidate for a CPU without a GPU (for example 12 vCPU and 48 GB RAM): the big model fits in memory while each step computes much less. Real speed still depends on the CPU and memory performance.

Read more

The experts in MoE do not each own a human-style subject. They are small whose roles split up by themselves during . The router is trained too: for every token it scores the experts and picks the best few, so different tokens in the same sentence can use different experts.

Numbers from the official Qwen model card (checked 2026-10-23)

Running the model needs more memory

How memory is split while the model runs Example
  • Weights
  • KV cache
  • Compute buffers
  • Runtime overhead

The grows with longer

working memory = weights + KV cache + compute buffers + runtime overhead
Read more

The weight file is only the starting point. Inference also needs a KV cache for earlier tokens, temporary , and runtime overhead. A theoretically 4 GB model may therefore need 5–8 GB or more; longer context usually grows the KV cache.

How can small numbers express so much knowledge?

One dot is just one color, but many dots together make a picture
Read more

Eight bits can represent 256 levels, but no single parameter contains a sentence such as “Seoul is the capital of South Korea.” Knowledge is distributed across patterns formed by billions of parameters interacting through many layers.

One is only a color value. Arrange millions of pixels and a face or landscape appears. Model knowledge emerges from distributed numerical patterns in a similar way.

The model is a vast pattern of scalar parameters grouped into vectors, matrices, and tensors. Precision determines the theoretical weight size; the runtime loads those weights and uses additional working memory to execute them on the available hardware.

The same road, step by step

  1. 1 From my hands to the program
  2. 2 Packing what gets sent
  3. 3 Crossing the internet
  4. 4 At the server door
  5. 5 Inside the server — the machine and the runtime
  6. 6 The model calculates
  7. 7 One token at a time, over and over
  8. 8 The way back
  9. 9 The program takes it in
  10. 10 Borrowing hands, over and over
1 From my hands to the program place 1 4 steps

  1. A key goes down

    A tiny computer inside the keyboard notices which key moved, turns it into a number, and sends it down the cable.

  2. The operating system catches it

    A driver in the turns that number into an and it for whichever window is in front.

  3. The program pulls it off the queue

    Every runs a loop that takes one event at a time and deals with it. The letter appearing in the box is this step.

  4. The microphone takes another route

    Sound arrives as tens of thousands of measurements a second. A model turns the spoken part into letters before the text model sees it — though some newer models can listen to the sound directly.

2 Packing what gets sent place 1 7 steps

  1. The system prompt

    Rules written in advance by whoever built the program go in first — what it is, and how it should answer.

  2. The conversation so far

    Earlier turns go in, in order. There is a limit, so the app drops, shortens or summarizes the oldest ones first — and whatever is left out is simply not seen.

  3. Whatever you attached

    An image or file is attached in a form the service accepts: raw bytes, a file , a link, or a long line of letters called . From a the service may pull out the text, take pictures of the pages, or read them with .

  4. Searching your documents (RAG)

    This is . Your question is turned into numbers (or keywords), and the pieces of your documents that match it best are found and packed in too. The model itself is not retrained — and the pieces it finds still need checking.

  5. The list of tools

    The name and input shape of every tool the app allows are written out like a menu. The app can list its own tools, or get them from an server.

  6. Skills and commands

    Bundles of instructions someone wrote earlier — skills — get slotted in when the task needs them. A can also carry scripts and reference files, so read it before you trust it.

  7. Making it fit

    The whole bag is counted in tokens. If it goes over the limit, the app has to cut, shorten or leave something out — usually the oldest material first.

3 Crossing the internet place 2 4 steps

  1. Finding the address

    The server’s name is looked up and turned into a numeric address ().

  2. Opening the road and locking it

    Before anything is sent, your computer and the server open a road and agree on a , so nobody in between can read what passes ( and ; the newer does both with ).

  3. Saying who you are

    A key or a token is attached to the head of the request. This is what decides whose account gets charged.

  4. Sending the request

    Everything packed so far becomes one request, usually written as , and goes out. When both sides need to keep talking — live voice, for example — a connection is held open instead.

4 At the server door place 3 3 steps

  1. The doorkeeper

    The key is checked, the call rate is checked, and the request is handed to a machine that is free.

  2. The queue

    Requests are not calculated right away; they wait briefly in line. When their turn comes, the runtime works through several waiting questions at once against the one set of numbers it already holds.

  3. Checking what came in

    The request is looked over against the rules before anything is calculated.

5 Inside the server — the machine and the runtime place 3 3 steps

  1. A file of numbers lies asleep

    A model that has finished training is a large file on a . By itself it calculates nothing. The on this site is four files, about 398 MB together.

  2. A runtime loads it, once

    When the server starts (or the app gets ready), the runtime reads that file, prepares it to run and keeps it in memory, so it is not opened again for every question. If memory runs short or the server restarts, it has to be loaded again.

  3. An accelerator takes the multiplying

    The same multiplication has to happen an enormous number of times, so it is handed to a chip built for exactly that, with fast memory beside it where the numbers sit. An ordinary CPU can do it too — it is only slower.

How one server is laid out
One server computer
Ordinary processor and memory
Weight file on the disk
Model runtime — loads it and looks after it
Accelerator and the fast memory beside it

The runtime does not live inside the accelerator. It is a separate program that reads the file and puts the numbers into the memory next to that chip.

Grouping several requests is step 17, and keeping the finished work is step 27 — this same program does both.

6 The model calculates place 4 7 steps

  1. Cutting into tokens

    The text is cut into the pieces this model's knows, and each piece gets a fixed number (an ID). Another model may cut the same text into different pieces.

  2. Numbers into lists of numbers

    Each ID picks out a long list of numbers (an embedding vector) learned during training. The ID number itself means nothing; it is these lists that can end up alike for pieces that are used in alike ways.

  3. Marking the order

    Where each token sits is given as numbers too — without it the model could not tell "dog bites man" from "man bites dog". Models do this in different ways; (RoPE) is one common way.

  4. Looking at each other (attention)

    : for every token the model works out how much to look at each earlier token, and mixes their information together accordingly. It only looks backwards — never at tokens that have not been written yet.

  5. Layer after layer

    The numbers pass through many layers — dozens in large models — each doing attention plus a round of multiplying and adding. Every layer shifts the numbers a little; how many layers there are depends on the model.

  6. Scoring everything

    The last layer gives a score to every token it knows — tens or hundreds of thousands of scores at once, depending on the model.

  7. Picking one

    The final scores are turned into chances (). makes the chances sharper or flatter, keep only the likeliest few, and then one token is drawn. Always taking the highest score is called .

7 One token at a time, over and over place 4 3 steps

  1. Adding it to the end

  2. Keeping the work already done

    Numbers already worked out for earlier tokens are kept (the KV cache) instead of being redone. That saves time but uses memory, and the overall speed still depends on how long the text is, how busy the server is and the .

  3. Back to step 22

    The text, now one token longer, goes through the model once more — the parts already worked out are reused, thanks to step 27. Round and round, until a token meaning "finished" appears or the length limit is reached.

This is the loop that writes the answer — one token at a time, back to step 22.

8 The way back place 2 4 steps

  1. Turning token numbers back into readable text

  2. Letting it flow as it comes

    Rather than waiting for the whole answer, small chunks are pushed out as they are produced ().

  3. Checking what goes out

    The outgoing text is looked over against the rules as well.

  4. Counting what was used

    Tokens in and tokens out are counted. This is where the bill comes from.

9 The program takes it in place 5 4 steps

  1. Stitching the arriving chunks in order

  2. Drawing it on screen

    is read and turned into headings, lists and boxes. This is the part that looks like typing.

  3. Spotting a tool request

    The program watches for a in the agreed format sitting inside the answer.

  4. Checking the arguments

    Whether that tool exists, and whether the arguments have the right shape, is checked before anything runs.

10 Borrowing hands, over and over place 5 4 steps

  1. Asking permission

    For anything risky — deleting a file, running a command — a well-built app asks you first. That check lives in the app, not in the model, and switching it off is a decision a person makes.

  2. Actually doing it

    The app does what the model asked for: a file is edited, a command is run, or another program (an MCP server) is called — only within the it was given.

  3. Packing the result

    What came out — or why it failed — is added to the conversation from step 6 as a new line.

  4. Back to step 15

    The refilled bag, now holding that result, is sent again. This repeats until the model says it is finished, or the app stops it because of a limit, a cancel or an error.

This is the loop people call an — back to step 15, again and again.

Read next

Keep going

Choose a pending booking to cancel. Each cancellation requires confirmation.

Your :

Close