AI Inference

AI Inference is Metis’s user app for deploying and operating models as serverless endpoints. Launch endpoints with a ready-to-use inference engine (vLLM) or your own container image (Docker), build inference infrastructure with workloads, templates, and volumes, and evaluate performance with Benchmark.

Every screen is shown for the cluster and project selected at the top of the sidebar.

AI Inference and AI Inference Admin

• AI Inference — the user app for creating and operating endpoints and infrastructure in projects you belong to.
• AI Inference Admin — the admin app for viewing endpoints and workloads across all projects and managing Kueue scheduling, monitoring, and token usage (launched from Admin Center). → AI Inference Admin

Menus

Group	Menu	What it does
—	Dashboard	View resource usage and recent activity for the current cluster and project.
Serverless	Docker	Operate serverless endpoints that run your own container image.
vLLM	GPU · CPU · Rebellions	Choose a model and operate vLLM serverless endpoints, split by compute type.
Infrastructure	Templates · Workloads · Volumes	Manage workload templates, workloads (pods), and volumes.
Evaluation	Benchmark	Evaluate endpoint performance with benchmarks.
Management	Projects	Create and manage projects.

Dashboard

See cluster and project info, resource usage, and recent activity on one screen.

Open Dashboard

Serverless (Docker)

Create and operate serverless endpoints from container images.

Browse endpoints·Create an endpoint

vLLM

Choose a model to create a vLLM endpoint, then test, edit, or delete it.

Browse endpoints·Create an endpoint·Manage endpoints