Run private inference

Applications connect to the trusted loopback gateway. They do not send ordinary Responses API or Chat Completions API requests to an inference provider.

Local development

pllm gateway --local --model Qwen/Qwen2.5-0.5B-Instruct --local-files-only --api-key local

This command starts the gateway plus co-located inference and preparation roles. It exercises the private protocol, but one machine does not demonstrate operator non-collusion.

Configured roles

pllm gateway --config client.toml

The client configuration identifies independently operated HTTPS role endpoints and credentials. The gateway remains inside the client boundary and exposes /v1/responses and /v1/chat/completions on loopback.

Use the exact gateway command reference for flags. Then choose a tested consumer integration or use the native client boundary directly.