In Part 2 of this series, I wrote about the ways that AI orchestration supports building and running AI applications and agents at scale. Notably, I called out the importance of the prompt as the interface between these applications and agents and the intelligence that AI inference provides. Today, I want to talk about the ways that Kamiwaza uniquely achieves the vision of a common interface for enterprise intelligence orchestration.
Many of us got our start with LLMs through a public chat interface from companies like ChatGPT, Anthropic, or Google. Most of our interactions relied on what those models already “knew” from training on billions of sources across the Internet.
Most enterprise software is not asking questions of the Internet, however. The vast majority of mission critical software is interacting with the state of the business itself. They are creating, retrieving, or asking questions about the data that represents current or past state.
There is no business intelligence, artificial or otherwise, without business data.
So, the first element of delivering intelligence to the enterprise is incorporating data in the questions and in the responses. A simple example is asking a question about an order, like “what is the status of order 12345?” The question needs to incorporate the order number, and the response will need to include the status of that order, as retrieved from whatever is the system of record for that status.
Kamiwaza is unique in that it builds a complete “AI friendly” picture of an enterprise's data landscape. Not only what is recorded in data stores, file systems, and even websites, but where that data resides and who can access it under what circumstances. You point Kamiwaza to the data sources you want to use in inference, and Kamiwaza automatically generates and maintains three key representations of the data in those sources: metadata like filenames, network locations, schemas, and so on; a semantic vector map of the data via an embedding model; and an ontological picture of how data entities are related to each other.
The resulting combination of metadata, semantic vector representations, and relationship graphing is critical to everything that Kamiwaza does with a prompt, either in its preparation or its processing. Kamiwaza uses that information to make fulfilling prompt requests as efficient as possible.
Let’s start with the way Kamiwaza gives you access to key data when you submit a prompt, and routes inference to where that data resides.
One of the most challenging things about any distributed systems environment—especially one in an enterprise with many revenue streams and organizational units—is data gravity. Because most data has more value early in its existence than later, application and systems architects tend to place workloads as close to where data is generated and/or stored as possible. This is in order to minimize latency introduced by the network when storing or retrieving that data.
So, if you use a variety of data sources in your AI processing, you have a conundrum. You can either choose to move the inference to where the data lives, or move the data to where the inference is taking place.
Because other applications using the same data are probably already deployed in order to optimize data gravity, you can’t move the data. So, the latter—moving data to inference—usually means copying that data to a data lake or other unified data store. This, of course, introduces a few risks:
On the other hand, running inference where the data is means that every prompt can use the system of record for a given data element. There is a small cost in terms of integrating multiple responses into a single response, but AI is highly capable of such processing.
This is the philosophy on which Kamiwaza is built. Our technology goes beyond simple data management and prompt routing to coordinate RAG-supported inference with the aforementioned understanding of what data is available, where that data resides, and who can access that data in what situations.
In addition to being smart with inference, it is critical that intelligence orchestration is open to the technologies that IT organizations want to use at any given time. The AI technology landscape is evolving rapidly, and I contend that we can’t be sure what “best practices” and architectures will look like five years from now. So, a common infrastructure for AI usage has to take into account an ever changing landscape for hardware, models, and data sources.
Kamiwaza does this in part by curating key models in the Huggingface catalog, understanding their compatibility with key processor technologies and applicability to key use cases. When choosing an open source model to deploy, Kamiwaza won’t suggest one that your environment can’t handle.
(Kamiwaza is hardware and model agnostic; you are free to select from any model available in Huggingface and attempt to deploy it. However, if you follow Kamiwaza recommendations, you know the model will be compatible with your specific hardware.)
And this is active curation. As new models are introduced and new hardware hits the market, our suggestions will update to reflect the new landscape.
One last thing that we do uniquely is apply access control to every action that we take, whether it is model management, data access, or inference. Kamiwaza is built to make sure that both users and agents acting on behalf of users are permitted to take those actions. We even insure that a given user is permitted to deploy or activate a given agent.
What is unique about this is our use of Relationship-Based Access Control (ReBAC). ReBAC allows access rules to be applied to the relationships between various entities in your AI environment using the graph captured in the Distributed Data Engine. This enables companies to define detailed requirements, such as “only allow Joe to access order data related to the customers he is assigned to manage,” or “revoke all access for users involved in a project if that project is cancelled or closed.”
Combined with our location awareness, in which we know where data and models reside, and can route data requests and inference thusly, ReBAC gives organizations complete control over how, where, and when data is accessed. In fact, data never has to leave a location to be useful for prompts from any other location.
The result of all this is a platform that serves as “intelligence delivery” for your AI applications, agents, and chat environments. One consistent way to make sure data is readily available, consumable, and secure regardless of the agent development technology, AI models, or physical infrastructure used to deliver that intelligence.
Kamiwaza makes available the OpenAI API for interaction with those components, which means it will work with just about any platform that supports calling models through that API. It truly is the most open, flexible, and secure way to scale private AI usage in agentic and general AI applications.
If you have questions or comments about Kamiwaza and its intelligence delivery capabilities, please feel free to comment below, or to contact a Kamiwaza representative directly. We look forward to working with you to achieve your most important AI outcomes.