Tokens — AI isn't priced by the character

The unit on AI price lists is neither 'characters' nor 'words' but 'tokens'. Because of this difference, the same content costs more in Korean than in English, and your actual bill can rise even when the price list stays the same. Here is the rundown, with real price lists as of August 2026.

---

Background — why you need to understand tokens now

Until a few years ago, AI pricing was just a 'monthly subscription'. Not anymore. Beyond developers using the API directly, subscription users started wondering what they consumed and how much once they ran into the message 'you have reached your limit'.

On top of that, the spread of agentic AI has sent consumption itself surging. It used to be one question, one answer; now the AI searches, reads files, calls tools, and reads the results again. All of that is metered in tokens.

The problem is that this unit, the 'token', runs against intuition. It is not a character count, a word count, or a page count.

---

What is a token?

A token is the smallest unit into which AI breaks text when processing it. The key point is that the more frequently a chunk of characters appears, the more likely it is bundled into a single token.

According to Anthropic's official documentation, in English roughly 1 token ≈ 4 characters ≈ 0.75 words(Anthropic Pricing Docs, 2026). The English 'hello' is a common combination and fits in one token, but the Korean '안녕하세요' does not get the same treatment.

Why? The tokenizer (the dictionary that splits sentences into tokens) gives slots to combinations it saw often in the training data. Since most of the training data is English, English fragments like 'the', 'ing', and 'tion' are compressed efficiently, while Korean is allotted fewer slots.

Korean's own structure compounds this. Korean is an agglutinative language, so particles and endings keep attaching to the stem. '먹다 / 먹었다 / 먹었었다' has a far wider range of variation than English 'eat / ate / eaten', and the tokenizer has to handle each variant.

---

Key data — actual price lists as of August 2026

These are the API prices of major models. The unit is USD per million tokens, with input (what I send) and output (what the AI writes) priced separately.

Model	Input	Output	Output/input multiple
Claude Haiku 4.5	$1	$5	5x
Claude Sonnet 5	$2	$10	5x
Claude Opus 5	$5	$25	5x
GPT-5.6 Luna	$0.20	$1.20	6x
GPT-5.6 Terra	$2	$12	6x
GPT-5.6 Sol	$5	$30	6x
Gemini 3.1 Pro	$2	$12	6x

(Sources: Anthropic official Pricing docs, checked August 2026 / OpenAI GPT-5.6 prices reflect the July 30, 2026 cut / Google Gemini 3.1 Pro is for the tier up to 200K tokens)

? Interpretation: The most important thing in the table is the rightmost column. Output costs 5–6 times more than input. Getting answers 'short and in a fixed format' saves more than trimming your prompt. The habit of asking for answers in table form or JSON genuinely saves money.

Three discount and surcharge mechanisms attach to this.

• Prompt caching: If you put long documents or system prompts you send repeatedly into the cache, subsequent reads are billed at only **10%** of the base input price (per Anthropic). In exchange, the first write to the cache costs 1.25x (5 minutes) to 2x (1 hour).
• Batch processing: Bundle jobs that don't need real-time responses and you get 50% off both input and output.
• Charges outside tokens: Server tools such as web search are billed separately. Per Anthropic, web search is $10 per 1,000 searches, and the content the search pulls in is counted again as input tokens.

---

In-depth analysis

Is Korean really several times more expensive?

Be careful with the numbers here. The figure 'Korean is 3–5x English' is widely quoted in Korean communities and some articles, but results vary greatly depending on when they were measured and which tokenizer generation was used.

• In large-scale measurements on the old-generation tokenizer (cl100k_base, the GPT-3.5/4 family), Korean was reported at about 2.36xEnglish.
• In a case measuring 285 actual blog posts on the new-generation tokenizer (o200k_base, GPT-4o onward), the Korean/English token ratio came down to about 1.4x .
• At a Korean event in 2024, Microsoft said the GPT-4 tokenizer improvements had cut the Korean burden 'from 2x to 1.1x'. Note that this is a vendor claim, not an independently verified figure.

⚠️ In short, the direction is clear (Korean is at a disadvantage), but the multiple swings from 1.4x to over 2x depending on model generation and the nature of the text. Results also differ between text mixed with code and pure prose. If cost matters for a task, don't use someone else's average; the accurate approach is to run that actual text through the tokenizer yourself.

When the price list stays the same but the bill goes up

This is the most counterintuitive point of token billing. Even if the unit price stays the same, the real cost rises when the tokenizer changes.

Anthropic states in its official documentation that models from Claude 4.7 onward use a new tokenizer that generates about 30% more tokens for the same text(a design traded for better performance). The price per million tokens is identical, but feeding in the same document produces more tokens, so the bill goes up.

So when looking at AI costs, there are two numbers to watch: unit price (the price list) and consumption (the tokenizer). Most pricing comparison articles cover only the former because the latter is tedious to measure.

Text isn't the only thing that is tokens

Images, PDFs, and web pages are all converted into tokens and added to the input charge. The rough benchmarks given in Anthropic's documentation are as follows.

• A typical web page (10 kB) → about 2,500 tokens
• A large technical document (100 kB) → about 25,000 tokens
• A paper PDF (500 kB) → about 125,000 tokens

This is why the bill jumps when 'all I did was add one PDF'.

---

How much is it really? — working it out

Let's assume we are writing one blog post: feed in 20,000 tokens of reference material and get back a 3,000-token draft.

Model, input cost, output cost, total, in won (assuming 1,400 won)

Model	Input	cost	Output	cost, total
Claude Haiku 4.5	$0.020	$0.015	$0.035	about 49 won
Claude Sonnet 5	$0.040	$0.030	$0.070	about 98 won
Claude Opus 5	$0.100	$0.075	$0.175	about 245 won

The same task differs 5x purely by model choice. Add Korean source material and input tokens grow by around 1.4x; use batching or caching and it drops to less than half.

The key point is that model choice is the biggest cost lever. Using a top-tier model for simple summarization, classification, or tagging is like chartering a cargo plane to send a parcel.

---

Implications for Korea

First, subscription users are not exempt. The usage limits of major AI services are mostly token-based. For the same fee, users working in Korean hit the limit sooner. This is why the strategy 'prompt in English, answer in Korean' circulates in practice.

Second, it gives the sovereign AI debate a concrete basis. Training a separate tokenizer on Korean data and combining it improves Korean processing efficiency. Indeed, one Korean model technical report describes adding about 19,000 Korean-specific tokens to o200k_base and improving processing efficiency per Korean byte by 12.6%(Motif 2.6B technical report). The improvement is not dramatic, but accumulated at the national scale it becomes an infrastructure cost difference that is hard to ignore.

Third, the estimate trap in enterprise adoption. If you budget by converting overseas vendors' English-based case studies as is, the actual bill will diverge. A step for measuring tokens on your own real documents at the PoC stage is needed.

---

Outlook and variables to watch

• Positive variable: The trend is that the burden on non-English languages shrinks as tokenizer generations advance. Measurements show the new-generation tokenizer has substantially reduced the CJK penalty.
• Risk variable: The opposite direction exists too. If designs that use more tokens for the sake of performance are adopted, perceived cost rises. The Claude 4.7-onward case above is one such example.
• Checkpoints: ① When switching models, check the change in token consumption, not just the unit price ② whether long-context surcharge tiers apply ③ the share of workloads to which caching and batching can be applied.

---

Conclusion

1. The unit of AI pricing is tokens, not characters. And tokens are not fair across languages.
2. Output costs 5–6 times more than input. Controlling answer length and format has more effect than trimming the prompt.
3. Unit price and consumption are separate matters. Even if the price list stays the same, the bill changes when the tokenizer changes.

The claim 'Korean is 3–5x more expensive' has the right direction but stale numbers. The only accurate answer is to measure your own documents.

---

[References]

• Anthropic, Pricing (official docs, checked August 2026): https://platform.claude.com/docs/en/about-claude/pricing
• OpenAI, GPT-5.6 announcement: https://openai.com/index/gpt-5-6/
• Google Cloud, Generative AI pricing: https://cloud.google.com/gemini-enterprise-agent-platform/generative-ai/pricing
• Measured Korean token ratio (285-post corpus): https://jangwook.net/en/blog/en/multilingual-llm-token-tax-experiment/
• CJK token efficiency measurement: https://masonailab.com/en/insights/token-efficiency/
• CJK text processing guide (the 2.36x figure): https://tonybaloney.github.io/posts/cjk-chinese-japanese-korean-llm-ai-best-practices.html
• Coverage of Microsoft's token efficiency remarks (ZDNet Korea, April 2024): https://zdnet.co.kr/view/?no=20240430131643
• Motif 2.6B technical report (12.6% Korean tokenizer improvement): https://arxiv.org/pdf/2508.09148
• Korean coverage of the Korean token burden: https://kitpa.org/news/1601