temperature, top_p, penalties — what each one actually changes
Every sampling parameter reshapes the same probability distribution. Knowing which part each one touches is the difference between tuning and guessing.
| Parameter | Effect | Typical range |
|---|---|---|
temperature |
Flattens or sharpens the whole distribution | 0 – 1.0 (deterministic → creative) |
top_p |
Keeps only the smallest set of tokens summing to p | 0.9 – 1.0 |
top_k |
Keeps only the k most likely tokens | 20 – 100 |
presence_penalty |
Penalises a token for having appeared at all | -2.0 – 2.0 |
frequency_penalty |
Penalises in proportion to how often it appeared | -2.0 – 2.0 |
max_tokens |
Hard ceiling on the reply, not a target | Model dependent |
stop |
Sequences that end generation immediately | Up to 4 strings |
seed |
Best-effort reproducibility; never guaranteed | Any integer |
| Job | Settings | Note |
|---|---|---|
Extraction / JSON |
temperature 0 | Anything above 0 will eventually break your schema |
Classification |
temperature 0, max_tokens tiny | Cap the reply to the label length |
Summarisation |
temperature 0.2–0.3 | A little variety, no invention |
Chat assistant |
temperature 0.7 | The common default for a reason |
Brainstorming |
temperature 0.9–1.0 | Sample several and pick |
Code generation |
temperature 0–0.2 | Correctness has one shape |
One, never both. They control the same randomness by different means, and moving both makes the effect of either impossible to reason about. Every major provider says the same thing.
Close, but not guaranteed. Batching, hardware and model updates can all still change the result. Treat it as "as repeatable as this system offers", not as a promise.