Temperature & Sampling
"Temperature" controls how random the model is.
Low Temp = Predictable, boring. High Temp = Creative, chaotic.
BALANCED
Context Input: "The cat sat on the..."
Next Token Probability
mat
87.0%
rug
9.7%
chair
2.4%
sofa
0.6%
floor
0.2%
moon
0.0%
pizza
0.0%
*Notice how increasing temperature flattens the curve, giving unlikely words (like "moon") a chance to be picked.