Learn AI

AI Concepts Workshop

© 2026 Cloudy Software Ltd

Understanding Tokens

The basic building blocks of Large Language Models.

Type any sentence above. Models don't read words like we do—they read chunks of characters called Tokens.

Count: 10
Artificial·Intelligence·is·fascinating!

Notice how words might be split, and spaces are included. A rough rule of thumb: 1000 tokens ≈ 750 words.

Why Tokens Matter?

Cost

API usage is often billed by token count (input + output).

Memory

Models have a 'Context Window' limit (e.g. 128k tokens) which determines how much they can remember.

Computation

More tokens mean more processing power and time required to generate a response.