Inspect a private inference plan

Define a reproducible experiment, lower a semantic model plan, and check its execution coverage.

Define the experiment

from pllm import Experiment, MaskedLinearCpu, Model
from pllm.deployment import Deployment
from pllm.kernels import Cpu
from pllm.runtime import ExecutionBudget

experiment = Experiment(
    name="qwen-local",
    pipeline=MaskedLinearCpu(
        Model("Qwen/Qwen2.5-0.5B-Instruct"),
        kernels=Cpu(threads=4),
    ),
    deployment=Deployment.local(root=".pllm/qwen-local"),
    budget=ExecutionBudget(requests=1, max_input_tokens=128, max_new_tokens=32),
)

resolved = experiment.resolve()
assert resolved.configuration_digest == experiment.configuration_digest()
print(experiment.configuration_digest())
print(experiment.to_spec())

The checked-in examples/first_request.py executes this same resolution path in the test suite.

The SDK configuration model keeps this declaration inspectable without starting a service or loading a model. You can inspect or export an equivalent checked-in JSON or YAML file:

pllm config show examples/pllm.yaml --format json
pllm config export examples/pllm.yaml --output experiment.resolved.json

Python configuration targets execute local code. In noninteractive use, you must explicitly allow them with --trust-python.

Lower and compose

The lower_model model-planning path accepts public model configuration and workload limits. It does not accept a model repository name or load weights:

import pllm
from pllm.passes import KvCacheEviction
from pllm.profiles import MaskedLinearCpu

config = {
    "model_type": "qwen2", "hidden_size": 896, "intermediate_size": 4864,
    "num_hidden_layers": 24, "num_attention_heads": 14,
    "num_key_value_heads": 2, "vocab_size": 151936,
    "max_position_embeddings": 32768, "hidden_act": "silu",
    "rms_norm_eps": 1e-6, "rope_theta": 1000000,
    "tie_word_embeddings": True,
}
plan = pllm.lower_model(config, batch=1, max_input_tokens=128, max_new_tokens=32)
composed = plan.apply(KvCacheEviction())
pipeline = MaskedLinearCpu(pllm.Model("Qwen/Qwen2.5-0.5B-Instruct"))
coverage = composed.coverage(pipeline)
assert composed.digest != plan.digest
assert coverage.complete is False
print(composed.digest, coverage.complete, coverage.to_dict())

This plan describes work that could precede an inference request. The MPCache structural pass is functional for dense Qwen2/Qwen3 plans, but the reported compiler coverage is currently incomplete, so the example does not perform private text generation. Semantic planning, protected runtime execution, model quality, benchmark results, privacy evidence, and deployment readiness are separate status checks.

Inspect component metadata with pllm components list. See runtime, protocol interfaces, and current support. To send an executable request from a compatible client, continue to the integration guides.

Inspect a private inference plan

Define the experiment

Lower and compose

On this page