LLM Sampling Parameters Cheat Sheet
temperature, top_p, penalties — what each one actually changes
Every sampling parameter reshapes the same probability distribution. Knowing which part each one touches is the difference between tuning and guessing.
The parameters
| Parameter | Effect | Typical range |
|---|---|---|
temperature
|
Flattens or sharpens the whole distribution | 0 – 1.0 (deterministic → creative) |
top_p
|
Keeps only the smallest set of tokens summing to p | 0.9 – 1.0 |
top_k
|
Keeps only the k most likely tokens | 20 – 100 |
presence_penalty
|
Penalises a token for having appeared at all | -2.0 – 2.0 |
frequency_penalty
|
Penalises in proportion to how often it appeared | -2.0 – 2.0 |
max_tokens
|
Hard ceiling on the reply, not a target | Model dependent |
stop
|
Sequences that end generation immediately | Up to 4 strings |
seed
|
Best-effort reproducibility; never guaranteed | Any integer |
Settings by job
| Job | Settings | Note |
|---|---|---|
Extraction / JSON
|
temperature 0 | Anything above 0 will eventually break your schema |
Classification
|
temperature 0, max_tokens tiny | Cap the reply to the label length |
Summarisation
|
temperature 0.2–0.3 | A little variety, no invention |
Chat assistant
|
temperature 0.7 | The common default for a reason |
Brainstorming
|
temperature 0.9–1.0 | Sample several and pick |
Code generation
|
temperature 0–0.2 | Correctness has one shape |
Do not tune both
temperature and top_p both control randomness by different means. Every major provider recommends changing ONE and leaving the other at its default. Moving both makes the effect of either impossible to reason about, and is the usual cause of "it went strange and I cannot say why".
Frequently asked questions
Should I change temperature or top_p?
One, never both. They control the same randomness by different means, and moving both makes the effect of either impossible to reason about. Every major provider says the same thing.
Does temperature 0 make output deterministic?
Close, but not guaranteed. Batching, hardware and model updates can all still change the result. Treat it as "as repeatable as this system offers", not as a promise.
Was this cheat sheet useful?
Comments
No comments yet — be the first.
Keep going
Browse all
More cheat sheets
Need a different cheat sheet?
Tell us what you would like to see and we will build it — free.
Request a cheat sheet
