What Is a Token? The Unit AI Reads and Bills In

In plain English

Models do not read words. Text is split into tokens, and everything is measured in them: the context window, the pricing, the speed of generation.

A useful rough figure for English is that a token is about three-quarters of a word. Code, names and other languages tokenise less efficiently.

What to know

Sub-word units

Common words are one token; unusual ones split into several.

The billing unit

Input and output tokens are charged, usually at different rates.

The context unit

Window limits are expressed in tokens, not characters or words.

Language-dependent

English is the most efficient; many other languages cost more per word.

Why it matters

Tokens are how AI costs are actually calculated, which makes them the unit to think in when designing anything that runs at volume. A workflow that pastes a whole document per request rather than the relevant section can cost ten times more for the same result.

Common mistakes

×Estimating cost in words or characters rather than tokens.

×Sending far more context than the task requires.

×Ignoring that output tokens are usually the more expensive side.

×Assuming token counts transfer between languages.

FAQs

How many tokens is a page of text?

Roughly 500 to 750 for a typical page of English prose.

Why does the same prompt cost more in another language?

Less efficient tokenisation means more tokens for the same meaning.

WRITTEN BY TARIQ SALLAM

Marketing Consultant. Entrepreneur. Content Creator.

I'm a marketing consultant, entrepreneur and content creator. I help businesses grow through practical marketing, websites, SEO, content and AI.

More About Tariq →