AI Backend & Architecture

Building PrismBot: A Multi-Tenant, Multi-Agent Chat Architecture

A multi-tenant AI chatbot backend built with NestJS and TypeScript that utilizes LangGraph for a supervisor-specialist agent routing system, delivering real-time responses and autonomous CRM updates via WebSockets.

Platform

SaaS Backend

Duration

Ongoing Development

3-StageRAG pipeline with parallel query expansion

100%Tenant isolation across vectors, KB, and CRM

2Specialist agents (RAG Expert & CRM Sales)

The Challenge

Single-agent LLM systems struggle with context bloat and lack the ability to simultaneously query complex knowledge bases while also updating customer relationship databases autonomously. Additionally, serving multiple enterprise tenants requires strict data isolation to prevent data leakage.

Static AI pipelines cannot easily update CRM records based on conversation context.
Monolithic LLMs are vulnerable to jailbreaks and script injections.
Data leakage risks in shared vector databases and chat histories.
Tightly coupled WebSocket logic makes it hard to scale AI agent tools.

Project Goals

Build a strict multi-tenant architecture scoping requests to specific tenants from the first step.

Implement a Supervisor LLM that never answers questions, but strictly routes to specialist agents.

Create a comprehensive RAG pipeline with query expansion and MMR reranking.

Automate CRM profile updates using conversational context without manual intervention.

Decouple Socket.io infrastructure from agent logic to allow safe admin interventions.

How We Built It

Strict Multi-Tenancy Architecture

Every request is scoped to a tenant by resolving the domain during the WebSocket handshake. Vector embeddings are isolated via namespace columns, and OpenAI API keys are decrypted per tenant.

Key DecisionInjecting a custom YAML tenant context block into every LLM call.

ResultAbsolute data isolation and highly customized bot personas per client.

LangGraph Supervisor Routing

Deployed a Supervisor LLM that only evaluates the flow state (Guardrails → Flow 2 → Flow 3 → Flow 1). It checks for greetings or CRM replies before routing genuine queries to specialized agents.

Key DecisionRestricting the supervisor from answering questions directly.

ResultEfficient, modular execution where specialized agents handle focused tasks.

Three-Stage RAG Pipeline

Built an advanced RAG Expert agent that runs a 3-stage pipeline: Query Expansion (generating 3 variants via OpenAI), parallel embedding and PGVector search, and MMR reranking (λ=0.7) to balance relevance and diversity

Key DecisionExpanding fetched chunks to include ±1 neighboring chunks for coherence.

ResultHighly accurate, context-aware answers deduplicated by chunk ID

Autonomous CRM Sales Agent

Created a mandatory CRM agent that runs after the RAG pipeline. It reads the MongoDB user record, infers new signals from the chat, and updates the database without overwriting existing maps.

Key DecisionUtilizing dot-notation $set in MongoDB to cleanly merge new user data.

ResultAutomated collection of user information for warmer human handoffs.

Decoupled WebSocket Delivery Bridge

Abstracted WebSocket delivery by creating GatewayCallbacks closures bound to the socket. The agents only know they are calling a shared send_response tool.

Key DecisionUtilizing a single send_response tool for all message delivery across agents.

ResultAgents remain agnostic to Socket.io internals, enabling clean architecture.

The Outcome

The PrismBot architecture proves that moving away from monolithic LLM prompts toward a multi-agent, supervisor-driven model drastically improves reliability and feature depth. By strictly isolating tenant context and decoupling the WebSocket layer from the LLM execution, we created a highly scalable platform.

Key Takeaways

Parallel query expansion significantly improves the recall of relevant vector chunks.

Abstracting WebSockets via closures allows LLM agents to safely trigger UI updates without knowing the transport layer.

Running a mandatory CRM agent post-query ensures user profiles stay updated dynamically.

Pre-pipeline guardrails are essential to prevent costly and dangerous prompt injections.

Conclusion & Impact

"The PrismBot architecture proves that moving away from monolithic LLM prompts toward a multi-agent, supervisor-driven model drastically improves reliability and feature depth. By strictly isolating tenant context and decoupling the WebSocket layer from the LLM execution, we created a highly scalable platform."

Technology Stack

NestJSTypeScriptLangGraphSocket.IOMongoDBPostgreSQL + PGVectorOpenAI

Ready to achieve similar results?

Let our dedicated engineers bring your vision to life with cutting-edge technology and unparalleled expertise.

Start a project