Temperature in LLMs Explained: What It Actually Controls

Temperature is one of the most confusing settings in an LLM, and I think a big part of the confusion comes from the way people describe it.

One of the most frustrating descriptions I hear is that temperature is a “creativity control.” I’ve even seen people treat it like some kind of IQ booster—like turning it up will make the model think harder or come up with better ideas.

That’s not really what’s going on.

A better way to think about temperature is: it lets the model be a little more random at certain points. That randomness can sometimes stumble into a path you wouldn’t have gotten otherwise—not because it’s “more creative,” but because it’s exploring alternatives.

Prompt:
Give me 5 taglines for a note-taking app for researchers.

Temperature 0.1 (deterministic):
- Capture ideas. Keep your evidence.
- Notes that stay linked to sources.
- Research notes, organized for recall.

Temperature 0.8 (more variation):
- Footnotes for your future self.
- From messy insight to citable memory.
- Turn scattered reading into usable knowledge.

Here’s the analogy I like: imagine you’re trying to find a new restaurant. You normally go to the right, because that’s the direction you always go. But one day you decide, for no particular reason, to go to the left instead. You’re still trying to find a restaurant. You still have the same goal. You just changed your path.

That’s temperature.

It basically says: you don’t have to be so strict about picking the single most likely next step every time. You can loosen up and try a different turn once in a while. But you still eventually need to find the pattern that makes sense.

This can be helpful up to a point. For example, if you’re seeing repetitive phrasing or predictable outputs, a little randomness can help the model avoid getting stuck in the same grooves.

But there’s a tradeoff: higher temperature often doesn’t make things “creative,” it makes them random. More like rolling dice and seeing what happens.

Practical Defaults

Use lower temperature (0.0-0.3):
- extraction
- factual QA
- deterministic transforms

Use medium/high temperature (0.6-0.9):
- brainstorming
- copy variants
- divergent ideation

And if you crank temperature really high—especially with earlier models—you could get outputs that were absolutely crazy. It was also a good way to see how readily these models would hallucinate when they started drifting outside their training distribution.

These days, you’ll notice we almost never adjust temperature. That’s not because it doesn’t do anything—it’s because modern models are so finely trained for the kinds of tasks we usually do that temperature tuning often isn’t that helpful. The main reason to touch it now is when you intentionally want to search more random paths.