Skip to main content

# Ecological impact

If you want to track the ecological impact of your LLM Usage, you can enable it in the Otoroshi LLM Extension (it should be enabled by default).

### Configuration​

    impacts {
      enabled = true
      enabled = ${?CLOUD_APIM_EXTENSIONS_LLM_EXTENSION_IMPACTS_ENABLED}
      embed-description-in-json = true
      embed-description-in-json = ${?CLOUD_APIM_EXTENSIONS_LLM_EXTENSION_IMPACTS_EMBED_DESCRIPTION_IN_JSON}
      electricity-mix = "WOR"
      electricity-mix = ${?CLOUD_APIM_EXTENSIONS_LLM_EXTENSION_IMPACTS_ELECTRICITY_MIX}
      embed-impacts-in-responses = false
      embed-impacts-in-responses = ${?CLOUD_APIM_EXTENSIONS_LLM_EXTENSION_IMPACTS_EMBED_IMPACTS_IN_RESPONSES}
      custom-models = "{\"aliases\": [],\"models\": []}"
      custom-models = ${?CLOUD_APIM_EXTENSIONS_LLM_EXTENSION_IMPACTS_CUSTOM_MODELS}
      custom-electricity-mix = "name,adpe,pe,gwp\n"
      custom-electricity-mix = ${?CLOUD_APIM_EXTENSIONS_LLM_EXTENSION_IMPACTS_CUSTOM_ELECTRICITY_MIX}
    }

Parameter | Type | Default | Description
--- | --- | --- | ---
`enabled` | boolean | `true` | Enable or disable ecological impact tracking
`embed-description-in-json` | boolean | `true` | Include human-readable descriptions in the impact JSON output
`electricity-mix` | string | `"WOR"` | Default electricity mix zone code (see available zones below)
`embed-impacts-in-responses` | boolean | `false` | Always embed impacts in LLM responses
`custom-models` | string (JSON) | `"{}"` | Custom model architectures for impact computation (see below)
`custom-electricity-mix` | string (CSV) | `""` | Custom electricity mix definitions (see below)

Once it's enabled, audit events of kind `LLMUsageAudit` will have an `impacts` field. You can also embed the `impacts` value in your LLM responses using the `impacts.embed-impacts-in-responses` config or by adding `?embed_impacts=true` to your query. Please remember that the `impacts` value is just an estimate of the real ecological impact of your LLM usage.

### Impact metrics​

The ecological impact output contains the following metrics:

Metric | Unit | Description
--- | --- | ---
`energy` | kWh | Energy: related to the final electricity consumption
`gwp` | kgCO2eq | Global Warming Potential (GWP): related to climate change, commonly known as GHG emissions
`adpe` | kgSbeq | Abiotic Depletion Potential for Elements (ADPe): related to the depletion of minerals and metals
`pe` | MJ | Primary Energy (PE): related to the energy consumed from primary sources like oil, gas or coal

Each metric contains a `value` with `min`, `max`, and `avg` estimates. The output is split into:

- Top-level metrics: combined usage + embodied impacts
- `usage`: impacts of the energy consumption during model execution
- `embodied`: impacts related to resource extraction, manufacturing, and transportation of the hardware

### Supported providers​

The ecological impact computation is based on project EcoLogits and only supports right now the following providers:

- OpenAI
- Anthropic
- Mistral
- Gemini
- Cohere
- HuggingFace

For unsupported providers, you can set metadata on your provider entities to map them to a supported provider/model:

- `eco-impacts-provider`: the provider used for eco impacts computation
- `eco-impacts-model`: the model used for eco impacts computation
- `eco-impacts-electricity-mix-zone`: the electricity mix zone to use for this provider

### Available electricity mix zones​

The following ISO country codes are available as electricity mix zones:

Code | Country | Code | Country | Code | Country
--- | --- | --- | --- | --- | ---
`ARG` | Argentina | `GBR` | United Kingdom | `NLD` | Netherlands
`AUS` | Australia | `GRC` | Greece | `NOR` | Norway
`AUT` | Austria | `HUN` | Hungary | `NZL` | New Zealand
`BEL` | Belgium | `IDN` | Indonesia | `POL` | Poland
`BGR` | Bulgaria | `IND` | India | `PRT` | Portugal
`BRA` | Brazil | `IRL` | Ireland | `ROU` | Romania
`CAN` | Canada | `ISL` | Iceland | `RUS` | Russia
`CHE` | Switzerland | `ITA` | Italy | `SVK` | Slovakia
`CHL` | Chile | `JPN` | Japan | `SVN` | Slovenia
`CHN` | China | `KOR` | South Korea | `SWE` | Sweden
`CYP` | Cyprus | `LTU` | Lithuania | `THA` | Thailand
`CZE` | Czech Republic | `LUX` | Luxembourg | `TUR` | Turkey
`DEU` | Germany | `LVA` | Latvia | `TWN` | Taiwan
`DNK` | Denmark | `MEX` | Mexico | `UKR` | Ukraine
`ESP` | Spain | `MLT` | Malta | `USA` | United States
`EST` | Estonia | `MYS` | Malaysia | `WOR` | World (default)
`FIN` | Finland |  |  |  |
`FRA` | France |  |  |  |

### Custom models​

You can define custom model architectures for impact computation using the `custom-models` config. The format is:

    {
      "aliases": [],
      "models": [
        {
          "type": "model",
          "provider": "openai",
          "name": "my-custom-model",
          "architecture": {
            "type": "dense",
            "parameters": 1760000000000
          },
          "warnings": [],
          "sources": []
        }
      ]
    }

The `architecture.type` can be `"dense"` or `"moe"` (Mixture of Experts). For MoE models, use:

    {
      "architecture": {
        "type": "moe",
        "total": 1760000000000,
        "active": 440000000000
      }
    }

### Custom electricity mix​

You can define custom electricity mix zones using the `custom-electricity-mix` config. The format is CSV with columns: `name,adpe,pe,gwp`:

    name,adpe,pe,gwp
    MYZONE,0.00000004858,9.3135,0.04418

- `adpe`: Abiotic Depletion Potential for Elements (kgSbeq/kWh)
- `pe`: Primary Energy (MJ/kWh)
- `gwp`: Global Warming Potential (kgCO2eq/kWh)

## Example of ecological impact embed in responses​

NOTE: you can embed ecological impact informations by using `embed_impacts=true` query param

    $ curl --request POST \
      --url 'http://test.oto.tools:8080/v1/chat/completions?embed_impacts=true' \
      --header 'content-type: application/json' \
      --data '{
      "messages": [
        {
          "role": "user",
          "content": "tell me a joke"
        }
      ]
    }'

    {
      "id": "chatcmpl-POPo2920QC8UdFT6MEF8MnnMBOGXDNdg",
      "object": "chat.completion",
      "created": 1743161221,
      "model": "gpt-4o-mini",
      "system_fingerprint": "fp-8hSBjQkQnl3InFEJ88ckzfm3JUtNC2bH",
      "choices": [
        {
          "index": 0,
          "message": {
            "role": "assistant",
            "content": "Why did the scarecrow win an award?\n\nBecause he was outstanding in his field!"
          },
          "logprobs": null,
          "finish_reason": "stop"
        }
      ],
      "usage": {
        "prompt_tokens": 11,
        "completion_tokens": 18,
        "total_tokens": 29,
        "completion_tokens_details": {
          "reasoning_tokens": 0
        }
      },
      "impacts": {
        "energy": {
          "value": {
            "min": 0.000045859386,
            "max": 0.000140325354,
            "avg": 0.00009309237000000001
          },
          "unit": "kWh"
        },
        "gwp": {
          "value": {
            "min": 0.00002877619380504681,
            "max": 0.00008550637626962295,
            "avg": 0.00005714128503733488
          },
          "unit": "kgCO2eq"
        },
        "adpe": {
          "value": {
            "min": 1.183887945516329e-10,
            "max": 1.8973743554267144e-10,
            "avg": 1.5406311504715217e-10
          },
          "unit": "kgSbeq"
        },
        "pe": {
          "value": {
            "min": 0.00047959646949585386,
            "max": 0.001435187811341041,
            "avg": 0.0009573921404184475
          },
          "unit": "MJ"
        },
        "usage": {
          "energy": {
            "value": {
              "min": 0.000045859386,
              "max": 0.000140325354,
              "avg": 0.00009309237000000001
            },
            "unit": "kWh"
          },
          "gwp": {
            "value": {
              "min": 0.000027078958526507997,
              "max": 0.00008285903437921199,
              "avg": 0.000054968996452859996
            },
            "unit": "kgCO2eq"
          },
          "adpe": {
            "value": {
              "min": 3.3830835927287995e-12,
              "max": 1.0351913624863199e-11,
              "avg": 6.8674986087959995e-12
            },
            "unit": "kgSbeq"
          },
          "pe": {
            "value": {
              "min": 0.000458043547368,
              "max": 0.001401569635752,
              "avg": 0.00092980659156
            },
            "unit": "MJ"
          }
        },
        "embodied": {
          "gwp": {
            "value": {
              "min": 0.0000016972352785388126,
              "max": 0.000002647341890410959,
              "avg": 0.0000021722885844748857
            },
            "unit": "kgCO2eq"
          },
          "adpe": {
            "value": {
              "min": 1.150057109589041e-10,
              "max": 1.7938552191780823e-10,
              "avg": 1.4719561643835617e-10
            },
            "unit": "kgSbeq"
          },
          "pe": {
            "value": {
              "min": 0.00002155292212785388,
              "max": 0.0000336181755890411,
              "avg": 0.00002758554885844749
            },
            "unit": "MJ"
          }
        },
        "warnings": null
      }
    }

## Example of LLMUsageAudit event with ecological impact​

    {
      "@id" : "1905575568334651655",
      "@timestamp" : 1743159594030,
      "@type" : "AuditEvent",
      "@product" : "otoroshi",
      "@serviceId" : "",
      "@service" : "Otoroshi",
      "@env" : "dev",
      "audit" : "LLMUsageAudit",
      "provider_kind" : "openai",
      "provider" : "provider_10bbc76d-7cd8-4cb7-b760-61e749a1b691",
      "duration" : 497,
      "model" : "gpt-4o-mini",
      "rate_limit" : {
        "requests_limit" : 10000,
        "requests_remaining" : 9998,
        "tokens_limit" : 200000,
        "tokens_remaining" : 199993
      },
      "usage" : {
        "prompt_tokens" : 11,
        "generation_tokens" : 18,
        "reasoning_tokens" : 0
      },
      "error" : null,
      "consumed_using" : "chat/completion/blocking",
      "user" : null,
      "apikey" : null,
      "route" : {
        "id" : "route_e4a9d6cb3-d859-4203-a860-8d1dd6d09557",
        "name" : "test",
        "..."
      },
      "input_prompt" : [ {
        "role" : "user",
        "content" : "tell me a joke"
      } ],
      "output" : {
        "generations" : [ {
          "message" : {
            "role" : "assistant",
            "content" : "Why did the scarecrow win an award?\n\nBecause he was outstanding in his field!"
          }
        } ],
        "metadata" : {
          "rate_limit" : { "..." },
          "usage" : { "..." }
        }
      },
      "provider_details" : {
        "id" : "provider_10bbc76d-7cd8-4cb7-b760-61e749a1b691",
        "name" : "OpenAI",
        "provider" : "openai",
        "..."
      },
      "impacts" : {
        "energy" : {
          "value" : {
            "min" : 0.000045859386,
            "max" : 0.000140325354,
            "avg" : 0.00009309237000000001
          },
          "unit" : "kWh",
          "description" : "Energy: related to the final electricity consumption in kWh"
        },
        "gwp" : {
          "value" : {
            "min" : 0.00002877619380504681,
            "max" : 0.00008550637626962295,
            "avg" : 0.00005714128503733488
          },
          "unit" : "kgCO2eq",
          "description" : "Global Warming Potential (GWP): related to climate change, commonly known as GHG emissions in kgCO2eq"
        },
        "adpe" : {
          "value" : {
            "min" : 1.183887945516329E-10,
            "max" : 1.8973743554267144E-10,
            "avg" : 1.5406311504715217E-10
          },
          "unit" : "kgSbeq",
          "description" : "Abiotic Depletion Potential for Elements (ADPe): related to the depletion of minerals and metals in kgSbeq"
        },
        "pe" : {
          "value" : {
            "min" : 0.00047959646949585386,
            "max" : 0.001435187811341041,
            "avg" : 0.0009573921404184475
          },
          "unit" : "MJ",
          "description" : "Primary Energy (PE): related to the energy consumed from primary sources like oil, gas or coal in MJ"
        },
        "usage" : {
          "energy" : { "..." },
          "gwp" : { "..." },
          "adpe" : { "..." },
          "pe" : { "..." },
          "description" : "related to the impacts of the energy consumption during model execution"
        },
        "embodied" : {
          "gwp" : { "..." },
          "adpe" : { "..." },
          "pe" : { "..." },
          "description" : "related to resource extraction, manufacturing and transportation of the hardware"
        },
        "warnings" : null
      },
      "costs" : null,
      "budgets" : null,
      "consumer_rate_limit" : null
    }

## Dashboard example​