Start with the workload
Efficiency depends on model architecture, precision, input length, output length, batching and concurrency. A result for a long-context coding agent cannot be assumed to describe short customer-service messages or image generation.
Ask whether the benchmark preserves the distribution and dependencies of production traffic. NVIDIA's new result uses recorded agentic coding trajectories, which is stronger evidence for that use case than one fixed sequence length but still not a universal workload.
Read the service-level constraint
Throughput can rise by batching more requests, while individual users wait longer. A fair comparison holds an interactivity or latency target and measures how much work the system completes inside it.
Inspect the full performance curve where available. An 'up to' ratio often describes one point, and another platform may compare differently at a stricter time-to-first-token or per-user generation rate.
Define the power boundary
Chip power is not facility power. Production energy includes CPUs, memory, networking, storage, cooling and idle capacity. Verify whether the reported megawatt figure measures a processor, a rack or the data-center input.
Utilization also matters. A highly efficient system at full load may waste energy when traffic is bursty. Capacity planning should model real arrival patterns, redundancy and the power consumed while hardware waits.
Connect tokens to useful work
Tokens are a convenient infrastructure unit but not a product outcome. A model that generates unnecessary reasoning, retries failed tool calls or produces work rejected by a reviewer can look efficient at the server and expensive in the workflow.
Pair energy and token metrics with correct completed tasks. Publish the model, software version, hardware configuration, measurement method and limitations so later results can be reproduced rather than repeated as marketing shorthand.
Sources & further reading
Social-media activity is treated as a signal of attention, not proof. Product claims are attributed to the linked publisher or announcement.