RAG Energy Measurement Framework
RAG Energy Measurement Framework
Retrieval-Augmented Generation systems are deployed as multi-node pipelines, yet their energy footprint remains opaque—existing benchmarks report only end-to-end totals, hiding where energy is actually spent. This framework instruments a 4-node RAG deployment (CITI Knowledge Management System) to attribute energy at three granularities: per module, per node, and per processing stage. Applied to a 2³ factorial experiment (1,200 queries × 8 configurations = 9,600 sessions), it reveals that architectural choices—not query volume—dominate energy consumption.
Architecture
The measurement infrastructure wraps each pipeline node with hardware-counter-based energy sensors (RAPL for CPU, NVML for GPU), synchronized through a central orchestrator. Each query execution produces a structured energy trace decomposed by processing stage.
Node-level energy instrumentation architecture: hardware counters capture per-stage energy across the distributed RAG pipeline
The target system deploys four nodes running distinct pipeline stages: embedding, optional HyDE generation, retrieval with optional reranking, and LLM generation. Three binary module toggles (HyDE, Reranking, Ultra-think) produce 8 distinct configurations evaluated in a full factorial design.
4-node CITI KMS deployment topology used for energy attribution
Full 2³ factorial design: 8 configurations from three binary module toggles
Results
Energy by Configuration
The 8 configurations split into two distinct energy clusters driven entirely by a single toggle—HyDE.
Per-request energy across all 8 configurations: HyDE configurations (1,666–1,786J) vs non-HyDE configurations (<435J)
| Cluster | Configurations | Energy Range | Defining Feature |
|---|---|---|---|
| High | C5–C8 | 1,666–1,786 J | HyDE enabled |
| Low | C1–C4 | 318–435 J | HyDE disabled |
Always-on activation (all modules enabled) costs 5.6× more energy than the baseline configuration (1,786J vs 318J per request)—yet quality improvement remains below 0.06 points on either evaluation metric.
Stage-Level Attribution
The stage-level breakdown reveals that HyDE’s hypothetical document generation dominates the energy budget, consuming 1,355J per request—roughly 4× the entire baseline request.
Energy attribution by processing stage: HyDE generation dwarfs all other stages combined
| Stage | Energy (J) | Share |
|---|---|---|
| HyDE generation | 1,355 | 76% (when enabled) |
| LLM generation | 250–310 | 14–73% |
| Retrieval + reranking | 45–85 | 3–20% |
| Embedding | 8–15 | <3% |
Key Findings
- HyDE dominates: 1,355J marginal cost per request, producing a clear bimodal energy distribution across configurations
- Diminishing returns: Activating all optional modules yields <0.06 quality improvement for a 5.6× energy penalty
- Attribution enables optimization: Stage-level decomposition identifies HyDE as the singular target for energy-aware design
- Measurement scales: The framework handles 9,600 sessions with per-stage granularity, providing the empirical foundation for routing decisions
Outcomes
The framework provides the first stage-attributed energy dataset for a production-grade modular RAG system. Its primary contribution is not the measurement tooling itself, but the empirical insight it enables: module activation decisions have dramatically asymmetric energy-quality trade-offs. This dataset directly informs the companion project on energy-aware query routing, where per-configuration energy profiles become the optimization target.