Model inference that belongs to your environment.

Quickstart

Create once, inject on deploy.

Create a Model API Resource for an explicit environment. Confirm production intent before running the command; model create does not use --yes.

anyhost model create -e dev
anyhost model status -e dev
anyhost deploy -e dev
anyhost model test -e dev --model openai/gpt-5-mini

A ready Resource injects these server-side runtime values:

ANYHOST_MODEL_API_KEY
ANYHOST_MODEL_API_BASE_URL
ANYHOST_MODEL_API_RESOURCE_ID

The base URL already includes /v1. The Runtime Key is platform-managed: it is never printed by the CLI and must not be placed in source, logs, browser bundles, or provider-branded variables.

anyhost model test runs a fixed, small, metered canary. It accepts a Catalog Model ID but no arbitrary prompt and returns only Generation and settlement status.

Rotation uses a bounded two-phase overlap: run anyhost model rotate-key -e dev, deploy and verify a real model request, then run the command again with --finalize. Finalization requires a successful post-rotation deployment and observed successor use before revoking the predecessor.

Catalog

Choose a released Catalog Model ID.

The authenticated Catalog is the authority for model identity, lifecycle, protocols, capabilities, context length, output limits, and customer reference price ceilings. Model IDs and display names preserve the published developer and model identity so people can find the model they intend to use. Every customer_prices entry includes an exact decimal price, currency, unit, and kind: ceiling. New active models become available to every ready Resource without editing each Resource.

curl -sS "$ANYHOST_MODEL_API_BASE_URL/models" \
  -H "Authorization: Bearer $ANYHOST_MODEL_API_KEY"

Send an exact returned data[].id; do not derive an ID from the display name. Suspended models remain visible but are temporarily unavailable; deprecated models include replacement and sunset information. The Catalog does not expose the serving Provider, endpoint, route, private account cost, or aggregator implementation.

Inference

Use the protocol your application needs.

GET  /models
POST /chat/completions
POST /responses
POST /messages
GET  /generation/{generation_id}

All calls use Authorization: Bearer <ANYHOST_MODEL_API_KEY>. Messages uses the same Bearer contract and does not require anthropic-version.

curl -sS "$ANYHOST_MODEL_API_BASE_URL/chat/completions" \
  -H "Authorization: Bearer $ANYHOST_MODEL_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "'"$MODEL_ID"'",
    "messages": [{"role":"user","content":"Hello"}]
  }'

Supported content, tools, structured output, reasoning, modalities, and parameters follow the selected model's published capabilities. Caller-selected routing, fallback, alternate-model, debug, plugin, private-observability, background, and stored-response controls are not accepted. If a client sends store, it must be false.

Streaming

Consume native SSE without waiting for billing.

Set stream: true and process protocol events in order. Anyhost does not buffer the full output for settlement. Every admitted response includes x-anyhost-generation-id; the same public ID replaces private response IDs.

Usage and settlement

One hundred Model Credits represent one USD.

A final usage object reports the customer charge as decimal strings:

{
  "cost": "0.0015",
  "credits": "0.15",
  "settlement_status": "settled"
}

If settlement is still pending, cost and credits are null. Query GET /generation/{generation_id} with the same Resource key. Do not estimate a final customer charge from token counts in application code.

A Resource budget is independent from the Workspace balance:

anyhost model budget -e dev --credits 500 --period monthly
anyhost model budget -e dev --clear

Errors and retries

The caller owns retry decisions.

Chat Completions and Responses use an OpenAI-shaped error object; Messages uses a Messages-shaped error object. Branch on HTTP status and stable error.code, not message text.

Anyhost never automatically reissues inference. A caller retry creates a new Generation and may create a different output and another charge. Preserve safe Retry-After evidence when present and make product retries explicit.

Privacy boundary

Generation records contain usage, not content.

Anyhost does not persist model inputs or outputs. Generation records contain Resource attribution, lifecycle, content-free token and cache usage, charge, timestamps, and safe errors. The service does not claim Zero Data Retention; eligible serving routes do not train on customer content, while temporary service-side retention may still occur.