# pllm.metrics Python API

Public objects, practical usage, and research context for pllm.metrics.

[View canonical HTML](https://pllm.run/sdk/reference/python/pllm/metrics/)

Document ID: `pllm.docs.reference.python.pllm.metrics`  
Release: `0.1.0`

Metric definitions attach units and optimization direction to measurements so evidence and Pareto search do not guess whether larger or smaller is better. Presence documents API availability, not complete model, security, or deployment coverage; check [implementation status](/sdk/reference/status/) before relying on a path.

## Research context

This API is project infrastructure rather than an implementation of one specific paper. Relevant method pages link their primary sources separately.

## Python SDK example

The example performs a bounded offline task and checks an observable result. It does not download a checkpoint or contact a provider.

```python
import pllm.metrics as metrics

latency = metrics.Latency(statistic="p95", phase="online")
assert latency.to_spec()["params"] == {"phase": "online", "statistic": "p95"}
```

API: [`pllm.metrics`](/sdk/reference/python/pllm/metrics/#objects-and-signatures)

## Objects and signatures

### `Accuracy`

Selects component `pllm/accuracy` during `benchmark`; it provides `ratio`, `higher-is-better`.

- Canonical object: `pllm.metrics.Accuracy`
- Kind: `class`
- Signature/type: `__init__(self, *, dataset: str, measure: str='task_score') -> None`
- Public exports: `pllm.metrics.Accuracy`
- Public members: `descriptor: ComponentDescriptor`; `__init__(self, *, dataset: str, measure: str='task_score') -> None`; `classmethod describe(cls) -> ComponentDescriptor`

### `Communication`

Selects component `pllm/communication` during `benchmark`; it provides `bytes`, `lower-is-better`.

- Canonical object: `pllm.metrics.Communication`
- Kind: `class`
- Signature/type: `__init__(self, *, direction: str='total', phase: str='online') -> None`
- Public exports: `pllm.metrics.Communication`
- Public members: `descriptor: ComponentDescriptor`; `__init__(self, *, direction: str='total', phase: str='online') -> None`; `classmethod describe(cls) -> ComponentDescriptor`

### `Cost`

Selects component `pllm/cost` during `benchmark`; it provides `currency`, `lower-is-better`.

- Canonical object: `pllm.metrics.Cost`
- Kind: `class`
- Signature/type: `__init__(self, *, currency: str='USD', basis: str='request') -> None`
- Public exports: `pllm.metrics.Cost`
- Public members: `descriptor: ComponentDescriptor`; `__init__(self, *, currency: str='USD', basis: str='request') -> None`; `classmethod describe(cls) -> ComponentDescriptor`

### `Energy`

Selects component `pllm/energy` during `benchmark`; it provides `joules`, `lower-is-better`.

- Canonical object: `pllm.metrics.Energy`
- Kind: `class`
- Signature/type: `__init__(self, *, source: str='not_available') -> None`
- Public exports: `pllm.metrics.Energy`
- Public members: `descriptor: ComponentDescriptor`; `__init__(self, *, source: str='not_available') -> None`; `classmethod describe(cls) -> ComponentDescriptor`

### `Latency`

Selects component `pllm/latency` during `benchmark`; it provides `seconds`, `lower-is-better`.

- Canonical object: `pllm.metrics.Latency`
- Kind: `class`
- Signature/type: `__init__(self, *, statistic: str='median', phase: str='online') -> None`
- Public exports: `pllm.metrics.Latency`
- Public members: `descriptor: ComponentDescriptor`; `__init__(self, *, statistic: str='median', phase: str='online') -> None`; `classmethod describe(cls) -> ComponentDescriptor`

### `Memory`

Selects component `pllm/memory` during `benchmark`; it provides `bytes`, `lower-is-better`.

- Canonical object: `pllm.metrics.Memory`
- Kind: `class`
- Signature/type: `__init__(self, *, kind: str='peak_rss') -> None`
- Public exports: `pllm.metrics.Memory`
- Public members: `descriptor: ComponentDescriptor`; `__init__(self, *, kind: str='peak_rss') -> None`; `classmethod describe(cls) -> ComponentDescriptor`

### `Metric`

Immutable `Metric` record carrying `component`, `params` for the workflow where metric definitions attach units and optimization direction to measurements so evidence and pareto search do not guess whether larger or smaller is better.

- Canonical object: `pllm.metrics.Metric`
- Kind: `class`
- Signature/type: `constructor not exposed`
- Public exports: `pllm.metrics.Metric`

### `Perplexity`

Selects component `pllm/perplexity` during `benchmark`; it provides `ratio`, `lower-is-better`.

- Canonical object: `pllm.metrics.Perplexity`
- Kind: `class`
- Signature/type: `__init__(self, *, dataset: str) -> None`
- Public exports: `pllm.metrics.Perplexity`
- Public members: `descriptor: ComponentDescriptor`; `__init__(self, *, dataset: str) -> None`; `classmethod describe(cls) -> ComponentDescriptor`

### `Throughput`

Selects component `pllm/throughput` during `benchmark`; it provides `per-second`, `higher-is-better`.

- Canonical object: `pllm.metrics.Throughput`
- Kind: `class`
- Signature/type: `__init__(self, *, basis: str='tokens') -> None`
- Public exports: `pllm.metrics.Throughput`
- Public members: `descriptor: ComponentDescriptor`; `__init__(self, *, basis: str='tokens') -> None`; `classmethod describe(cls) -> ComponentDescriptor`