Large Model API Token Fees

When we input text into a large model API, the model does not directly "read" the words. Instead, it first uses a tokenizer to break the text down into the smallest processing units—tokens. These units are neither individual Chinese characters nor letters, nor do they entirely correspond to words in natural language. Instead, they form "semantic fragments" based on semantic relevance and frequency of occurrence.

Take OpenAI's GPT series as an example: the English sentence "ChatGPT is smart." is split into 6 tokens: Chat, G, PT, is, smart, .); the Chinese sentence "你好，世界！"(Hello World) is split into four tokens: "你好", "," "世界", "！". This segmentation logic stems from word frequency statistics during model training. For instance, '苹果' (apple) is grouped as a single token due to high co-occurrence frequency, while "鸭蛋" (duck egg) might be split into two tokens because of its lower frequency.

Technically speaking, tokens serve as the bridge connecting human language to machine computation. Tokenizers use algorithms like Byte-Pair Encoding (BPE) to map text into sequences of numerical codes. The model then computes relationships between these codes to achieve understanding and generation, with each token corresponding to a basic computational operation.

The choice of token-based pricing for large model APIs stems fundamentally from its strong correlation with computational costs. This model is more rational than traditional per-call pricing for three key reasons:

1. Direct Quantification of Computing Power Consumption

The process of large models processing text essentially involves complex matrix operations on token sequences. Longer input texts (more tokens) demand greater GPU memory allocation, increased computational operations, and extended processing times. For example, processing a 1,000-token document can consume many times more computational resources than a 100-token document. Token-based pricing is, in effect, "pay-per-computation," accurately reflecting the resources used.

2. Balancing the Dual Costs of Input and Output

A single API call has two cost components: input Prompt Tokens and model-generated Completion Tokens. These are typically priced differently. For OpenAI's GPT-4 Turbo, input tokens cost $0.01 per 1,000, while output tokens cost $0.03 per 1,000. Output is usually more expensive because generation involves complex reasoning and creative organization, which is more computationally intensive than simply understanding an input.

For instance, if a user question consumes 50 tokens and the response consumes 150 tokens:

Input Cost: (50 / 1000) * $0.01 = $0.0005

Output Cost: (150 / 1000) * $0.03 = $0.0045

Total Cost: $0.0050

If input and output were uniformly priced, billing inequities would arise: in "short question, long answer" scenarios, service providers would bear additional costs; conversely, in "long question, short answer" scenarios, users would pay unnecessary premiums for the input portion.

3. Encouraging Efficient Resource Utilization

By-the-token pricing incentivizes users to be efficient: streamline prompts, remove redundant information, and use parameters like max_tokens to cap output length. This prevents wasteful, overly long responses. In contrast, a flat per-call fee cannot distinguish between the computational demand of "What's the weather?" and "Write a ten-thousand-word report," leading to potential resource abuse or unfair pricing.

Just as restaurant prep cooks vary in their knife skills, different large models' word segmenters employ distinct splitting logics, directly impacting token count results. These differences primarily stem from variations in training data, vocabulary design, and segmentation algorithms:

GPT Series (OpenAI): Primarily employs the BPE algorithm or its variants (GPT-4 uses tiktoken, which still follows the BPE approach at its core). Excels at handling subword segmentation in English contexts. For example, "playing" is split into 'play' and "ing," preserving semantic integrity while improving compression efficiency.

DeepSeek: Features unique logic for handling high-frequency Chinese compound words. Both "HaHa" and " HaHaHa" are treated as single tokens, while "HaHaHaHaHa" splits into 'Haha' + "HaHaHa" tokens, reflecting adaptation to everyday expression patterns.

Qwen (Tongyi Qianwen): Offers more flexible splitting for individual Chinese characters. Most common characters remain single tokens , whereas low-frequency characters in DeepSeek may split into two or more tokens.

Claude Series: Utilizes a variant of BPE with a large vocabulary, supporting longer context windows (up to 100,000 tokens). Its segmentation is coarser, treating common phrases as single tokens, resulting in relatively fewer total tokens when processing long texts.

This variation means that the same text segment may yield different token counts across different models. For example, "laptopcomputer"—a high-frequency compound term in technology—is typically merged into a single token across most models. However, "tabletphone" (a less common term) may be split into two tokens: 'tablet' and "phone." Some earlier-trained models might even fragment it into smaller subword units.

For users, pre-estimating token counts is key to cost control. The following four methods cover needs ranging from quick estimates to precise calculations:

1. Empirical Formula for Quick Calculation

English: 1 Token ≈ 0.75 words; 100 words approximately equals 130 Tokens;

Chinese: General colloquial text (e.g., daily conversations, news reports) 1000 characters equates to 1200-1400 tokens; Technical/specialized texts (e.g., academic papers, technical documentation) 1000 characters equates to 1500-1800 tokens. This formula is for preliminary budgeting only. Precise estimation requires online tools (e.g., tiktokenizer) or code tools (e.g., tiktoken library).

2. Precise Calculation with Online Tools

Tiktokenizer: Supports mainstream models like GPT-4o and DeepSeek. Input text to display token segmentation results and identifiers. For example, entering "by and large" reveals its segmentation into 3 tokens across different models.

OpenAI Token Calculator: Optimized for OpenAI models, it simultaneously estimates costs. Entering "Write a 500-word essay" predicts required tokens and fees.

3. Batch Calculation with Code Tools

Automated estimation can be achieved using Python's tiktoken library. Sample code is as follows:

enc = tiktoken.encoding_for_model("gpt-4")

# Encode text and compute token count

tokens = enc.encode("Hello, world!")

# Outputs 4

print(len(tokens))

This method is suitable for developers integrating token counting functionality into projects to monitor consumption in real time.

4. API Parameter Control

Setting the max_tokens parameter when calling the API (e.g., "max_tokens": 200) enforces a limit on the number of output tokens, preventing unexpected overages. For tasks like mathematical computations or data queries, replacing lengthy text descriptions with function calls can further reduce token consumption. For example, when querying "weather in a city over the past 7 days," a long text description would require detailed specification of the query requirements (approximately 50 tokens). In contrast, calling the function `get_weather(city="Beijing", days=7)` requires only about 20 tokens.

Tokens serve not only as the fundamental units for large models to process language but also as the "measurement standard" for computing resource consumption. The token-based billing model not only reflects the logical soundness of the technology but also provides users with opportunities for cost optimization.

Understanding the differences in tokenization rules and mastering token estimation methods are essential for achieving efficient resource utilization while enjoying the benefits of large models. As model technology evolves, token segmentation strategies and billing models may continue to advance, but the core principle of "paying for computational resource consumption" will remain applicable for the foreseeable future.