A cloud-native, decoupled architecture of "Cloud Brain + Distributed Edge Executors", built for secure enterprise AI Agent inference at the edge.
The cloud brain handles orchestration, routing policy, tenant management and billing, while edge executors run model inference and tool execution close to the data source — data never needs to leave your premises.
1 · Agent requests arrive via the OpenAI-compatible gateway at the nearest edge station.
2 · The smart routing engine scores complexity and routes to local edge inference or the cloud brain.
3 · The security gateway keeps data, RAG and tool calls inside a Zero-Trust sandbox.
4 · The token factory packs, caches and schedules requests for maximum GPU throughput.
5 · Usage is metered per tenant and billed in real time; nodes are managed via one-click OTA.
Route every agent request to the best execution point. Location-aware policies keep data in-country, complexity scoring sends hard tasks to bigger GPUs, and load-based overflow spills to the cloud brain with automatic desensitisation.
A Zero-Trust enforcement point at every edge station: enterprise data stays local for RAG retrieval, tool execution runs in isolated sandboxes (Micro-VM / container / network namespace), and sensitive fields are desensitised before any data leaves the node.
Squeeze maximum tokens-per-second out of every GPU: prefix caching reuses shared context, dynamic batching packs concurrent requests, and multi-LoRA serving hosts many tenant-specific adapters on one model — powered by vLLM / SGLang.
Developers integrate in minutes with OpenAI-compatible APIs. ReAct-style tool-call traces, per-step observability and SDKs make agent behaviour transparent and debuggable.
Serve many customers on shared edge hardware without compromising isolation: vGPU/MIG slicing partitions the GPU, token usage is metered per tenant in real time, and over-quota traffic is billed transparently including cloud overflow.
Operate fleets of edge stations from one console: ZTP boots a new node into service with zero manual configuration, OTA pushes platform and model updates in grey batches, and scheduled health inspections raise alerts before users notice.
| Metric | Target |
|---|---|
| Platform availability | ≥ 99.5% |
| Edge inference TTFT (time to first token) | < 10 ms |
| Concurrent agent sessions per edge node | ≥ 50 |
| API compatibility | OpenAI /v1/chat/completions |
| Inference backends | vLLM · SGLang |
| Isolation levels | Network NS · Container · Micro-VM |
| Data residency | On-premise by default; cloud overflow desensitised |
| Ops model | ZTP · OTA · one-click node ops |
Log in to the AISP Portal console or request a guided demo from our team.