v1.0 · PBDR architecture
OQOAI PBDR
Policy-Based Decentralized Routing for enterprise AI inference — turn idle GPUs into a private, fault-tolerant cluster.
10+ cost dimensions
<30s deployment
Zero single point of failure
1000+ nodes
Key features
Intelligent routing
- 10‑dim cost vector (latency, GPU/VRAM, temp, queue…)
- Adaptive exploration / exploitation
- Hard constraints: model, context, VRAM
Flexible policies
- Presets: balanced latency throughput
- Custom weights via API / web UI
- Real‑time policy swap
Admin dashboard
- Real‑time GPU metrics (temp, util, VRAM)
- Automatic node discovery
- Bulk config & policy management
Supported backends
- Ollama API
- OpenAI API (vLLM, TGI, llama.cpp)
- Automatic API type detection
Fault tolerance
- Versioned protocol (ETag/If‑Version)
- Automatic recovery & state caching
- No single point of failure
Enterprise ready
- MIT licensed · transparent audit
- On‑premise, HIPAA / GDPR compliant
- Works across cloud & local networks
Comparison
| Feature | PBDR | Kubernetes | NGINX LB | Custom |
|---|
| GPU‑aware routing | ✅ | ❌ | ❌ | ⚠️ |
| Policy‑based routing | ✅ | ❌ | ⚠️ | ⚠️ |
| Zero configuration | ✅ | ❌ | ❌ | ❌ |
| Decentralized architecture | ✅ | ❌ | ❌ | ⚠️ |
| <30s deployment | ✅ | ❌ | ❌ | ❌ |
| Real‑time GPU metrics | ✅ | ⚠️ | ❌ | ⚠️ |
| Automatic node discovery | ✅ | ⚠️ | ❌ | ❌ |
| Multi‑model support | ✅ | ❌ | ❌ | ⚠️ |
| Enterprise‑grade security | ✅ | ✅ | ✅ | ⚠️ |
| Open source (MIT) | ✅ | ✅ | ✅ | ❌ |
Use cases
Enterprise RAG
Deploy on 100 workstations, serve 100 clients with 90% cost reduction — keep data inside the corporate network.
R&D labs
Utilize idle GPUs after hours, dynamic model loading for multiple research teams.
Healthcare (HIPAA)
On‑premise deployment, audit‑ready logging, no data leaves the network.
Architecture
Components
- PBDR Server — worker node with GPU (port 8080 / 11434)
- PBDR Client — intelligent router (port 8080)
- PBDR Admin — web dashboard (port 8081)
Decentralized · client‑side decisions · stateless nodes
Scientific publication: doi.org/10.17513/doi.26
Performance
<5ms routing decision
1s discovery interval
<100ms policy switch
>99.9% MTBF
1000+ nodes
<2% CPU overhead / 100 nodes
Turn idle corporate GPUs into a private AI cluster — no dedicated infrastructure, full data sovereignty, and policy‑driven routing.
Quick start
# Clone & run server (Linux)
git clone https://github.com/oqo-ai/OQOAI-PBDR.git
cd pbdr
chmod +x ./pbdr_server_OAA4_en
./pbdr_server_OAA4_en pbdr_server_config_test.json
# Start client (router) on another node
./pbdr_client_OAA5_en pbdr_client_config_test.json
# Launch admin dashboard
./pbdr_admin3_en pbdr_admin_config_test.json
# → open http://<admin-ip>:8081