Tool tree
Agent-native LLM inference cluster orchestrator built with WebMCP (document.modelContext.registerTool).
Implementation
gradeNot graded
gradeNot graded
get_cluster_topologyAnswer
Get current LLM cluster nodes, replica counts, latencies and VRAM usage.
View tool JSON
{
"name": "get_cluster_topology",
"kind": "answer",
"impl": "imperative",
"description": "Get current LLM cluster nodes, replica counts, latencies and VRAM usage.",
"inputSchema": {
"type": "object",
"properties": {
"filterType": {
"type": "string",
"description": "Filter by node type: Inference, Gateway, Cache, Database"
}
}
}
}scale_service_replicasAction
Scale the replica count of a designated node to absorb traffic spikes.
View tool JSON
{
"name": "scale_service_replicas",
"kind": "act",
"impl": "imperative",
"description": "Scale the replica count of a designated node to absorb traffic spikes.",
"inputSchema": {
"type": "object",
"properties": {
"nodeId": {
"type": "string",
"description": "Node ID (e.g. llm-deepseek-r1, llm-qwen-7b)"
},
"replicas": {
"type": "number",
"description": "Target replica count (1 to 16)"
}
},
"required": [
"nodeId",
"replicas"
]
}
}optimize_gpu_memoryAction
Evict stale prefix KV caches and defragment GPU memory.
View tool JSON
{
"name": "optimize_gpu_memory",
"kind": "act",
"impl": "imperative",
"description": "Evict stale prefix KV caches and defragment GPU memory.",
"inputSchema": {
"type": "object",
"properties": {
"targetNodeId": {
"type": "string",
"description": "Node ID or \"all\" to defragment entire cluster"
}
}
}
}run_inference_benchmarkAction
Trigger an automated throughput and latency stress test across the cluster.
View tool JSON
{
"name": "run_inference_benchmark",
"kind": "act",
"impl": "imperative",
"description": "Trigger an automated throughput and latency stress test across the cluster.",
"inputSchema": {
"type": "object",
"properties": {
"concurrency": {
"type": "number",
"description": "Simulated request concurrency"
}
},
"required": [
"concurrency"
]
}
}