# Fiber Copilot, a multi-tenant AI platform orchestrated by Temporal > How we designed Fiber Copilot to stand up chatbots and copilots on your own data, in your own cloud, with the parser, vector store and model you choose. Published: 2024-11-07 Author: Preetham Reddy Tags: AI/ML, Temporal, Databricks Canonical: https://www.techfabric.com/blog/how-we-designed-a-multi-tenant-multi-cloud-and-multi-model-ai-platform----fiber-copilot --- Putting a chatbot or copilot on your company's data takes a document parser, a vector database, a model and somewhere to run them. The right choice for each depends on what your data looks like, whether that's PDFs and handwritten notes or a data platform like Databricks or Snowflake, and on which cloud you already use. Built by hand, every one of those projects means months of setup and integration. It isn't cost-effective, and at the pace better components keep arriving it goes out of date quickly. What you'd want instead is a chatbot on your own data in hours, provisioned in your own cloud subscription, say Elastic Search and Azure OpenAI on Azure, or Qdrant and Bedrock on AWS, with control over your data and over who can use each assistant. That's what we designed Fiber Copilot to do. With the recent advancements in the Generative AI space, we've had many conversations with our clients on how they can take advantage of Large Language Models and introduce Generative AI experiences in their applications that can democratize access to data, simplify their current workflows, and help their employees be more productive. Since every client is different and their needs are different, we often had to build custom solutions tailored to their needs and found ourselves doing the same thing repeatedly. Since there are many components and moving pieces in a Generative AI system, there's no way to build a one-size-fits-all system. We choose the components based on cost, speed, and features that are appropriate for our clients. A typical Generative AI system will have these building blocks and depending on the company and their use case, we often have to choose different components, spreading across native cloud provider components to third-party and custom-built components.‍ 1. **Vector Database**(Azure AI Search, Quadrant, Cloudflare Vector Store, Elastic Search, Vertex AI Search, etc) 2. **Document Parsing Engine** (Azure Document Intelligence, Llama Parse etc.) 3. **AI Model Serving Platform (**Azure OpenAI, AWS Bedrock etc.) 4. **Knowledge Graphs** 5. **Intelligent Data Platform**(Databricks, Snowflake, Azure Fabric, etc.) 6. **File Storage** (Azure Blobs, AWS S3) 7. **Retrieval Augmented Generation (RAG) techniques** 8. **Workflow Orchestration for continuous training (**[**Temporal**](https://temporal.io/)**)** 9. **Identity Access Management**(Azure Entra ID, Ping, Okta etc.) and many more… As you can see, there are a few core components that are needed for every meaningful Generative AI platform. But not every enterprise uses the same cloud platform, has the same set of requirements, or even tries to solve the same problem. Some of them have tons of documents in unstructured format (PDFs, spreadsheets, word docs, handwritten notes) accumulated over decades. Some of them have intelligent data platforms with advanced capabilities. Some of them have semi-structured data and traditional relational databases. Each scenario requires using a different kind of parsing engine that's more suited for our customer's needs, a different vector database, a different foundational AI model that's fine-tuned for their use case, etc. For this reason, we often find ourselves trying to build a custom platform for each of our clients with manual setup and integration. This approach isn't future-proof, isn't cost-effective for our customers, and can quickly go out of date, considering the pace at which innovation is happening in this space. There's got to be a better way to rapidly provision infrastructure and swap out components as newer, better components arrive. We also want to have a platform that can host multiple tenants and provide an amazing user experience (UX) to ingest, process, train, and deploy AI chatbots and copilots on the fly. ‍ ## **Introducing The Fiber Copilot Platform** ![Fiber Copilot landing page headed 'Enterprise AI Copilots Trained On Your Data', with a chat window answering a question about aerospace compliance standards such as NIST SP 800-171 and DoD CMMC.](/blog-media/73e393ce-6706a721d0ab8cdef1948cda_66b54781efba504506d13de9_6629637759f4239cabb34cdd_Fiber-CoPilo) _Fiber.inc AI Copilot Platform_ That's why we decided to build Fiber, a new multi-tenant, multi-cloud, and multi-model AIOps platform, that can be used to quickly create and deploy AI chatbots and copilots trained on enterprise data. Users will have full flexibility to choose various options for components in the system and provision underlying infrastructure at the click of a button, reducing months' worth of work to a few hours. For example, they'll have the option to choose Elastic Search for vector search, Azure Document Intelligence for parsing engine, and Azure OpenAI for model serving and deploy it to Azure. Or they could choose Qdrant for vector search, LlamaParse for parsing engine, and AWS Bedrock for model serving and deploy it to AWS. They'll have many more options to choose from, from native cloud provider options to third-party and open-source options. It's their choice. What platform they want to deploy to and what components best fit their use case is for up to them to choose. Fiber provides them total control over their data privacy and secure access to cloud infrastructure. Fiber will enable the companies to quickly create chatbots and copilots trained on their proprietary data and deploy it across their Enterprise through granularly controlled permissions and security policies.‍ ‍ ## **Multi-Tenant Architecture** Here's the high-level architecture: ```mermaid %% caption: Tenant A, Tenant B and Tenant C each keep their customers and Azure OpenAI Service in a subscription they own, so the application in the provider subscription reaches each tenant only through the access that tenant grants. flowchart TD subgraph TC["Tenant C, owned subscription"] C1["Customers"] C2["Azure OpenAI Service"] end subgraph TB["Tenant B, owned subscription"] B1["Customers"] B2["Azure OpenAI Service"] end subgraph TA["Tenant A, owned subscription"] A1["Customers"] A2["Azure OpenAI Service"] end subgraph P["Provider subscription"] APP["Application"] end A1 --> APP B1 --> APP C1 --> APP APP --> A2 APP --> B2 APP --> C2 class APP accent ``` We've decided to build a multi-tenant system that can be easily provisioned while fully preserving our customers' data privacy and control over their data. Each tenant will have all their resources provisioned in their cloud subscription. All their data, vector stores, AI models and parsing engines will be provisioned in their cloud platform of choice. Fiber applications and APIs will be granted granular access to resources based on the permissions granted in the Identity and Access Management System authenticated with OAuth 2.0. The entire infrastructure provisioning process is automated and [orchestrated with Temporal workflows](/blog/ai-at-scale-managing-cloud-multi-tenant-ai-infrastructure-with-temporal) and its durable execution capabilities. ```mermaid %% caption: Each tenant's resources live in their own cloud subscription, and Fiber applications and APIs reach them only through granular permissions. flowchart TD A["Temporal workflows"] -->|"orchestrated"| B["Tenant cloud subscription"] B --> C["Data, vector stores, AI models, parsing engines"] D["Identity and Access Management System"] -->|"permissions"| E["Fiber applications and APIs"] E -->|"granular access"| B class B accent ``` ![Authentication on Amazon EKS: Tenant A and B users go through a load balancer to an ingress controller on *.example.com; each tenant signs in via its own OIDC proxy and identity provider and reaches its own chatbot and vector database.](/blog-media/f1f1da0a-6706a721d0ab8cdef1948cd4_66b54781efba504506d13df4_662987e0dd18dfff90a88f3e_Authenticati) _Authentication Architecture_ ## **Tying it all together with Temporal** Since we have so many dependencies on external cloud providers and their APIs, we needed to have a durable and deterministic execution of processes in our system. [Temporal](https://temporal.io/) durable execution framework provides a guarantee that no matter what happens, process crashes, network or storage outages, its orchestrator helps the failed processes recover by rehydrating them in a different server and continuing the flow of execution. The [Temporal](https://temporal.io/) framework is the backbone for all our underlying workflows. We execute workflows while onboarding new clients, and users, provisioning infrastructure, training AI models, creating vector embeddings etc., The beauty of the Temporal framework is that it works with our pre-existing choices for runtime, test framework, CI/CD environments, and any web framework. We don't need to choose a specific language or server technology to make use of Temporal workflows. We have workloads written in Python, Typescript, C#, .NET Core, Bicep DSL, Terraform, etc., and Temporal just works with every one of those technologies. There's no other workflow orchestration engine that's as flexible as Temporal and supports all the major programming languages. It's truly a joy to work with it and we couldn't be more excited. This flexible architecture of Fiber allows us to quickly provision a Generative AI platform for our clients so they can jump right into creating AI chatbots but is also future-proof. If newer models arrive, or there's a better vector search database, or a newer parsing engine works better, it's only a matter of a few clicks to swap these components, and Temporal workflow will do the rest to retrain the AI model and populate the vector store! Check out Fiber Copilot and [reach out](/contact) if you'd like to try it out! ‍ , [**Preetham Reddy**](/blog-authors/preetham-reddy) Founder, [TechFabric](/) and [Fiber](https://fiber.inc/) ## Frequently asked questions ### Can I swap the vector database or the model after the assistant is live? Fiber is designed so you pick the new component and Temporal runs the change. The same workflows that built the first setup retrain the AI model and populate the new vector store, so the swap is a few clicks and not a rebuild. Since the data stays in your own cloud subscription, the new component is provisioned there too, and it's worth checking that your identity setup grants Fiber access to it. ### Does provisioning start over if a cloud API fails or a process crashes halfway? The workflow doesn't start over from scratch. Temporal rehydrates the failed process on a different server and carries on from where it stopped. That matters here because provisioning leans on many external cloud providers and their APIs, so a single outage shouldn't leave a tenant half built. ### How does Fiber get access to the resources in my cloud? Fiber's applications and APIs reach your resources only with the permissions granted in your Identity and Access Management system (Azure Entra ID, Ping, Okta), authenticated with OAuth 2.0. Assistants are rolled out across the company through granularly controlled permissions and security policies.