Provider connections

Without --local, the gateway stays inside the client boundary and connects to provider-role URLs. It does not start those services and it does not make their endpoints OpenAI compatible.

Credentials

Set PLLM_INFERENCE_API_KEY and PLLM_PREPARATION_API_KEY in the gateway environment. Keep those role-specific credentials distinct and out of argv, shell history, and checked-in configuration. Public prepared inference needs both URLs; a profile with no preparation role must omit the preparation URL.

Examples

Connect to separately operated providers

Set PLLM_INFERENCE_API_KEY and PLLM_PREPARATION_API_KEY in the environment; credentials should not appear in shell arguments.

pllm gateway --inference-url https://inference.example.com --preparation-url https://preparation.example.com

Start provider roles with pllm serve inference and pllm serve preparation. Both can resolve the same Python, JSON, or YAML experiment target used for local gateway execution and benchmarking.