How to Calculate Local LLM Energy per Token

The faster of two matched gemma4:26b settings was not the most efficient. One run gained 1.3% output speed and used 23.3% more energy per output token.

The short answer: Divide average GPU watts by output tokens per second. The result is joules per output token. In my RTX 5090 test, num_batch=256 used about 0.870 joules per output token. The 512 setting used about 1.072. Speed alone hid that difference.

Canonical URL: https://bmdpat.com/blog/local-llm-energy-per-token-2026

How do watts become joules per token?

NIST defines a watt as one joule per second. A model rate is output tokens per second. Divide the first rate by the second rate, and seconds cancel.

joules per output token = average GPU watts / output tokens per second

This number answers a narrow question. It estimates GPU energy during the measured generation period. It does not include the rest of the computer, model load time, prompt work, cooling, or idle power.

What did the two RTX 5090 rows show?

The August 20, 2026 5090 Report includes two gemma4:26b workshop rows. Both used Ollama, a 2,048-token context, 91 input tokens, and 482 output tokens. Both passed the fixed tasks.

The arithmetic is direct. 187.6 / 215.7 = 0.8697. 234.3 / 218.5 = 1.0723. I round each result to three decimal places in the table.

The 512 row generated 1.3% faster. Average GPU power rose 24.9%. Energy per output token rose 23.3%.

Why is energy per token better than watts alone?

Watts measure a rate of energy use. They do not say how much work finished during that time. A 200-watt run can use less total energy than a 150-watt run if it finishes much sooner.

Tokens per second have the opposite gap. They say how fast output arrived, but not how much power the GPU drew to produce it. The fastest row can also be the least efficient row.

Joules per token join those two fields. The unit still does not say whether the answer was correct. I only compare rows after the task check passes. My earlier failed-row note shows why a fast row with a failed task stays out of production.

Can I turn this into electricity cost?

Yes, but I label the estimate. One million output tokens at 0.870 joules each equals 870,000 joules. Divide by 3.6 million joules per kilowatt-hour. The generation estimate is about 0.242 kWh.

The 512 row reaches about 0.298 kWh per million output tokens by the same method. That is a difference of about 0.056 kWh for the measured generation phase.

I do not call either number a full local inference cost. A full receipt also needs wall power, prompt work, load time, idle time, hardware cost, and the electricity rate. My local AI workbench note uses the same boundary when it converts one measured run into a rough electricity estimate.

When should I use this metric?

I use it after two rows pass the same task check. I keep the model, quant, runtime, workload, context, prompt, and output cap fixed. I also record the token counts so a later reader can see whether the runs did the same work.

Then I ask three questions. Did the task pass? Did the faster setting remove useful wait time? Did energy per output token improve or get worse?

For this workload, num_batch=256 is the better measured setting. It produced 98.7% of the 512 row's output rate while using less average GPU power and less energy per output token. That is a result for this RTX 5090 test, not a default for every model or GPU.

What should the benchmark receipt keep?

Keep the raw fields before the derived metric. Save the model, quant, runtime version, GPU, workload, context, output cap, input tokens, output tokens, output rate, average GPU power, and task result.

Calculate joules per output token from those saved values. Do not save only the rounded answer. The raw row lets another builder check the math and change the boundary.

Also name what the power sensor measured. GPU power is not wall power. A generation average is not a whole-request average. A clear label stops a small metric from becoming a large claim.

Accompanying prompt

What the prompt does: It calculates energy per output token from local LLM rows and rejects comparisons that lack a task check.

Copy/paste this prompt:

Copy-ready prompt

Paste the exact block into your coding agent.

No article chrome, no footnotes, no formatting drift.

Copy promptSelect text

Role: You are a local LLM benchmark reviewer. Context: I will give you benchmark rows with model, quant, runtime, GPU, workload, context, token counts, output rate, average GPU power, and task result. Task: 1. Keep only rows that passed the same task check. 2. Calculate joules per output token as watts divided by output tok/s. 3. Compare speed, average GPU power, and energy per output token. 4. State which measurement boundaries remain outside the result. Output: - A table with the raw fields and calculated joules per token. - One setting choice for this measured workload. - A list of missing fields needed for a full cost estimate. Constraints: - Do not treat GPU power as wall power. - Do not include failed or unmatched task rows in the recommendation. - Keep full precision for the calculation and round only the shown result. - Say unknown when average power or output rate is missing.

24 lines/901 chars

Ready

This prompt and every other one we publish live in the free prompt library.

Copy the block above.

Get the next measured local AI build note from The 5090 Reports.