How to calculate tokens in PDFs, DOCX, code, images, and media

A production LLM request is rarely one text box. It can contain a confidential instruction, extracted report pages, proprietary repository fragments, structured data, and one or more images. Measuring only the pasted prompt systematically understates both context occupancy and input cost; uploading the rest to a separate counter can create an unnecessary enterprise disclosure path.

The multimodal measurement pipeline

1. Acquire locally. The browser receives the selected files; 100% of submitted prompt, document, code, data, and image content remains in the active browser session.
2. Classify by modality. Plain text and source formats are decoded directly, rich documents enter an extraction path, and images enter a dimension/formula path.
3. Normalize text. Extracted strings are combined with pasted input without pretending that document bytes equal model tokens.
4. Apply the model method. OpenAI-compatible text uses the configured BPE vocabulary; other families use a disclosed deterministic projection. Images use supported provider formulas.
5. Aggregate in a Web Worker. The calculation runs off the main interface thread without a token-counting API request.
6. Project context and cost. Input tokens are compared with the model context limit and multiplied by the selected model's input price.

PDF and DOCX are extraction problems first

A PDF stores positioned drawing instructions, fonts, and sometimes raster scans—not necessarily a clean reading-order string. A DOCX is a ZIP-based package of XML parts. TokenCalculator.dev extracts readable PDF text and DOCX paragraph content locally, then tokenizes the resulting text. A scanned PDF without a text layer needs OCR and is deliberately reported as unsupported rather than assigned a misleading count.

Extraction order can differ from the order a provider's own file-ingestion system reconstructs. Headers, footers, tables, hidden runs, OCR, and provider-added document framing are the main reconciliation points.

Extracted text is also not the whole bill. Every provider that accepts a PDF sends each page to the model as an image as well, and charges for it. Gemini publishes the rate—560 tokens a page on Gemini 3 by default, 258 on Gemini 2.x—so a 40-page report carries 22,400 tokens of page images before a word of its text is counted. OpenAI, Anthropic, and xAI accept PDFs without publishing a per-page figure.

Source code and data files preserve expensive syntax

JSON keys, escaped strings, indentation, punctuation, UUIDs, minified bundles, and base64-like sequences can tokenize very differently from natural language. The local pipeline decodes supported source and data formats as text; it does not summarize them before counting. That makes the result useful for repository review, log analysis, RAG chunk planning, and batch prompts.

Images are geometry, detail, and model generation

Image file size is not a safe token proxy. Providers resize, tile, patch, or cap images according to model-specific rules. The browser reads dimensions locally and applies the documented formula supported by the selected model. See the provider image-token analysis and image reference.

Audio, video, and pages are duration and counts

These inputs do not go through a tokenizer at all. They are billed from length, which means a browser can measure them exactly: read duration from the file's metadata, or the page count from the PDF, and multiply by the published rate.

Two numbers are worth memorising. A one-hour recording is 90,000 input tokens on Gemini 3. Ten minutes of video with sound is 57,000. Both are large enough to change which model a workload can afford, and neither is visible in a price-per-million table.

Outside Google, the picture is mostly blank. No current OpenAI or Claude text model accepts audio or video at all. Qwen, Kimi, and GLM accept video, and Meta's Muse Spark accepts audio, but none of those vendors publishes a token rate. A calculator has two honest options there: name the gap, or invent a number. This one names the gap, with the model named in the message. The rates and the gaps are listed in the audio, video, and PDF reference.

Validate the result before budgeting production

Use the local count as a preflight measurement. For material spend, compare it with the provider's token-count endpoint or response usage metadata using the same model, message structure, tool definitions, image detail, and files. The difference is useful telemetry: it reveals provider framing and preprocessing that no browser-only calculator can observe.