Video: Building & securing AI agents: A tech leader crash course | Duration: 3740s | Summary: Building & securing AI agents: A tech leader crash course

Transcript for "Building & securing AI agents: A tech leader crash course": Hello, everybody, and welcome to the latest session between Pangea and AIAI. It's great to have you all joining. Gonna wait just a few seconds while folks are coming into the room, but thank you for tuning in wherever you are, whether it's morning, afternoon, evening. It's great to have you joining us. We're gonna be joined by Saurabh Satish today, who is the cofounder and CTO at Pangea. Today's session is gonna be around building and securing AI agents, a tech leader crash course. Super session today from Sourabh, which is gonna cover demos of MPC configuration, an overview of existing security guardrails from open source projects and cloud service provider offerings all the way through to commercial tools and DIY approaches, carry on with a comparison of pros and cons across various guardrail solutions, and then finally, actionable best practices for implementing those guardrails that will secure AI agents without slowing down your innovation. It's gonna be awesome to have Saurabh leading the presentation today. If you guys have any questions whilst you're watching, drop them in the chat on the right hand side. And I will be delivering a q and a session in about an hour's time. So drop those questions in there, and we will get them we'll get to them at the end of the session. Thanks for tuning in. We're gonna hand straight over, and here we go. We'll join you folks in an hour for q and a. Hello, everyone. My name is Saurabh. I'm CTO and cofounder for Tangier, and I'm here to talk about building and securing AI agents. Hopefully, this is gonna be a fun session with some, hands on, look at code of writing an AI agent. So in terms of agenda, I'll just briefly cover some generative AI concepts, especially in as it relates to enterprise AI architectures, that is relevant to this, discussion. We look at some code of building a basic AI agent and how MCP changes the implementation of an AI agent. And then we look at basic, agent security issues, especially in in in relationship to, how MCPs are being used by the agents. And then we'll talk about the security solutions, guardrails, open source, commercial, engineering and security best practices. And then we'll end the session with a brief look at NGS guardrails, that are available. Alright. So, hopefully, both of these concepts are nothing new, but, nonetheless, probably important to recap in the context of this discussion. As you all know, LLMs are basically a mathematical representation of all information on on that it has been training. But LLMs are really trained on the data at a per till a particular point in time. That means that LMS really don't have any access to real time information, things like stock price, current news, weather, etcetera. And and when we talk about LMS, we are really referring to condition models, which are, really, large models trained on a vast amount of data. Whereas, small language models or domain language models are models which are smaller and really trained on a subset of data pertaining to a special topic on which we really want the LLM to excel and not get conflated or confused about information from all types of sources of information. Context window is really the working memory of an LLM, which allows the LLM to analyze more, take into account more and more data as it tries to predict the next set of tokens or answer a user's question. The bigger the memory, the more it can look back and respond, and this is what really enables the error based application to have a conversational experience. Prompts are really, constructed to communicate various forms of input to the LMM, to achieve the desired outcome. And in order to do so, there are, you know, ways in which the data is communicated to the LMM. They are really communicated on behalf of certain roles. So a system role really is is meant to facilitate communicating from an application developer admin's perspective as to what the application should be doing, how it should be doing, and so on and so forth. And then the user role really captures the input that it is given to by the user to the application that is then passed on to the LMM. Context is, is really the data that is associated with, along with the user's question, in order to make the LMM, you know, take that information into account in order to come up with the answers. And then the assistant role really represents the answers that are given back by the LLM. And then, tools is yet another role that has been recently added in order for, the AI applications to communicate, a set of capabilities that it has that makes it, capable of interacting with, other source of information, that can be, you know or or things it can do in order to, you know, generate more, relevant data that can be passed back to the other end, for it to respond, to the user question. AI apps and agents basically use LMs to process the input data, and get the relevant output. LMs by themselves cannot really execute any tools and actions. You might often hear LMs calling tools. That's really a short way of communicating the fact that, LMs can guide the agents to call the tools. So, again, very important point. LMs by themselves have no capability of interacting with the external environment or calling tools. They they really are used to guide an agent or an application to do whatever is needed in order to extract the information that can be incorporated into input to the LLM for it to do its task. Now in case of enterprises, the the important thing is, that in order to answer the user's question, relevant data from enterprise applications and, data stores have to be passed into the LLM for a particular enterprise use case. You know, in case of, enterprises, this data could be locked in enterprise applications or data source like Google Drive, etcetera. And most of, you know, in in in the most common architecture for, in production AI applications and enterprises is all about retrieving this data and storing it into a data store, in a format that makes it and that is very efficient for, for retrieval so that the relevant piece of information can be passed as context to the, to the LLM for it to then come back with a response. That is this is, so retrieval augmented generation, is really the term given to this kind of capability where, where the data is retrieved from a retrieved from other data sources and augmented to the user question as an input, to the LLM to generate an answer. And very often, you know, it's stored in a vector DB, which is nothing but a specialized database, for efficient information retrievals. Agentic RAG is kind of an evolution of a RAG architecture which uses VectorDB, wherein an agent can be used to interact with this environment to retrieve the the piece of information for the user's question, rather than just storing retrieving it from our from our vector DB. Now, yeah, agents can, interact with the external environment, and services and applications, but they can also interact with a vector DB, which can, again, be this source of information. So it's not necessary that the agents are connecting with other applications. They could simply be connecting to a database to retrieve the information. Nonetheless, you know, agentic RAG, just are more real time and are able to retrieve information from, many other, you know, applications that the agent could be connected to. Agentic workflow really is the whole process of orchestration, collaboration, with, the ability, the whole process of orchestration, collaboration to, you know, answer a user's question. In a typical AI workflow, a user's question is really given to an AI for it to then generate the response on the basis of the information that is provided to the, into the AI by virtue of the input, or the information that AI model itself has. Whereas in case of an agentic workflow, you know, there are there is a a much more well orchestrated, way of answering a user's question where where a user's question is, you know, is augmented with, you know, a way to solve the problem, and the capabilities that the agent can, have such that the LLM can break down the user's question into a plan. And as part of, multiple steps of a plan, the LLM can guide the AI agent on what actions and tools to execute in order to retrieve, the answer for the step, and then incorporate that information into its plan and decide what is the next course of action, and and repeat this loop till it has, you know, either retrieved the right answer or has no more actions to execute, and and then that result is given back to the user. Now this interaction with the environment that I've been saying, is done on behalf of tools, could also be done by collaborating with other AI agents. So it's not just about executing tools on behalf of an of an AI agent. It could be also about collaborating with other AI agents which have been purpose built for a particular, use case. Now, as you might have realized, AI agents are nothing but software programs, which basically have an ability to interact with the environment. So in terms of confidence, AI agents really have four components. The first one is its ability to interact with the other end where it does things like reasoning and planning and, breaking down the problem into subtasks and, and and, you know, figuring out how to come up with the right answer by invoking the right, you know, or executing the right steps. Prompts are enable you know, prompts are a component where an AI agent guides the LLM on how to solve the problem, exposes what kind of capabilities it has so that the LLM can, invoke the right or guide the agent, invoke the right tools for the for the steps. Then the agents, can optionally also have short term and long term memory, where it can cache or store historical information, other piece of information, you know, from, from the current conversation, past conversations, etcetera, so that it can more, efficiently respond, to certain pieces of the question rather than invoking the tools all the time. So, it kind of has an ability to cache the results of the tool calling or have more data that it can use, in order to provide the right kind of context to the user's question, so that the answers can be more efficiently responded to. And lastly, the agents really are the the piece of code that implement, or call the tools, not the LLMs. So the tool implementation, tool calling is kind of part of the agent functionality. So let's look at really how, a basic agent works. You know, when when a user's input comes in, the agent really communicates, you know, to the agent system prompt, which basically guides the agent on how to solve the problem. And in order to solve the problem, the agent exposes the set of capabilities it has in order to assist with that problem solving. We are exposing, you know, these capabilities as tools to the, to the LLM and then the user problem. Now what is really interesting here is that, the tools have kind of, have a description, have, you know, what are the input parameters for the format of the input, and how it is gonna produce the answer. And so the the real key aspect here is the tool description. The tool description has to be detailed enough, complete enough so that the LLM can recognize that this is the right tool to call for this particular step in the plan for it to, you know, answer the user question. And so when the user question along with the system content tool is given to the LLM, the LLM can identify the plan, and for the steps of the plan, identify which tool, what input to the tools needs to be, sent over to the agent. The agent then executes the tool, identifies the results of the tools, gives it back to the LLM along with all the history, and then the LLM can decide what more tools to call as part of its plan or whether this is the end of, you know, executing all the tools and that it has the right final answer, and then it packages the right final answer back to the user. And the agent, ends up responding to the user or the caller of the agent. So this is, you know, this is really the the basics of an AI agent. And and, in order to make it all real, let's really look at code. So I've, I have a sample, agent that I've written in Python. It uses no frameworks because I really didn't want to, I really wanted to understand how it works at your score and not be, able to see what's going on by virtue of using some libraries or frameworks. Again, the most important part is the system prompt, which I'm showing here on the right. And as you can see here, the agent kind of declares to the LM that, hey. This is my purpose. These are my capabilities. So here I have a calculator capability. I have a response time capability, and the description tells what these capabilities are. And then I guide the LLM, on, how to kind of, answer a user's question. So given a user question, it needs to be able to, you know, properly, break down the problem into sub steps, and be able to use any of the available capabilities, that the agent has in order to extract the input, which can then be, in order to, perform the action, whose results are then gonna be sent back to the LMM as an observation. The LMM should take into account the, user's question, the multiple steps that they're asked to execute, the result of individual steps in order to decide what to do next, which could be about executing more actions or maybe, you know, compiling the results of the call to actions as results and giving the final answer back to the user. So this is the system prompt, and and the more important piece is that, the ability for it to interact with the environment is declared, to the LLM as a set of tools. So let's look at code. So I'm gonna go into, my, agent code. And and this is really the more most interesting piece of the entire agent code. I have, an agent dot pi five, which pretty much the entirety of the implementation of the agent. And the tools that, the agent can declare to the LLM are implemented in tools dot py file. I've just picked, the response time tool which returns a random number, and the calculator just basically calls an eval on the input. And so from, from the main interesting, functionality of the agent, here's what is going on. When the user in gives an input, it it, along with the system prompt, is sent as an input to the LLM. The LLM responds back with a plan and the action to execute on behalf of the first step of the plan. If the results of the LLM doesn't have any action, it means that the LLM is done, and there's nothing more to be done. But if there is an action and an action input, it really is about the agent then invoking the action, extracting the results, and passing the results back, as a subsequent, you know, input to the LLM. The LLM can then can then take the results into account in order to decide what the next set of action is or, you know, it's done executing all the actions and, here's the final answer. So, you know, this is how simple the, agent is. And now just let's, run the agent and see it in action. So here, I'm gonna just run the run the agent. So you can see that. Let's so here it says, I'm your helpful AI agent. How can I help you? I'm just gonna ask her the question, what is the response time of abc.com. So let's see what the agent does in, you know, in in the whole, code as it executes. So the first loop, it takes a user input and comes up with a thought that in order to respond to this question, I really need to invoke the response time tool. Now it's able to find find that response time is the tool to call because it kind of is able to interpret on the base of the tool description that it is the right tool for the steps that need to be executed for answering the user's question. So it responds back with, hey. This is the action to call, and it forms the input as expected, by the action, by virtue of the, you know, tool description, and gives it back to the agent. Agent then identifies that, hey. This is the tool that I need to call. ESF is one of the tools that I have. Here's the input. It it works to retrieve the results and passes it back to the, LLM as an observation. The LLM then takes that observation into account, the entire history of the conversation, which is a system prompt, the post question, the plan, the steps, etcetera, and then says that, hey. You know, the response time for the website abc.com is fifty one milliseconds. There's nothing more to be done because that was the question that was asked, so I have no more actions to call upon, and this should be really the answer that is returned to the user. So now let's look at a little more, you know, interesting question that, we can ask. So what is the sum of response times of, abc.com and cde.com? So, here, you know, hopefully, the LMM recognizes that there are multiple steps for answering these these questions, which is invoking the response time tool and invoking the calculator tool in order to finally respond. So let's see, how the application does or the agent does. So here, in order to answer this question, it really, looks at the question, the history of, question and answer that have occurred in the past, and it comes up with a plan, and says that, hey. You know, I already see the response time for a bc.com in my memory, so I really don't need to invoke any actions for retrieving response time for a bc.com. But I don't know anything about cd.com. So let me, tell the agent to, invoke the response time pool to retrieve, the response time of cd.com, which the agent gladly, you know, responds with, which then is taken into account by the agent as an observation. And now the agent says, well, now I have the response time of cd.com, and all I need to do is sum that up by invoking the calculator tool. So it passes on the right, in you know, calls upon the agent to invoke the calculator tool with the input of, the two response times. The agent then gladly, answers, executes the calculator tool, gives the result back as an observation to the LLM. And the LLM is able to identify, that, hey. You know, I have the final answer. There is no more actions to be executed. So here's the, the final answer that needs to be given back to the user. So you can see how the, LMM was, was guided by the system prompt on how to solve the user's problem, was able to identify the capabilities that the agent exposes to the LMM, guide the agent to execute the right set of tools, incorporated its results, and was able to finally come up with the answer that the user was looking for. So this is really, you know, how the AI agents work in the most simplest form. So let's get back into the slides and continue talking about, you know, what this, AI agent architecture was, what are the pros and cons, and what how does MCB changes anything. So what you might have observed is that, what you saw was a very coupled architecture. The the tool, the system prompt, all was kind of in the agent fold. It also required me be an expert on all of these external services that I'm interfacing with, and be able to call the right API, be able to take the results, you know, package them, as appropriate results that can be sent back to the LN and so on and so forth. And, the tool implementation, because it was literally as code within the agent, the only way for me to share this tool implementation with other agent authors is by code, which is really not a great way to share, you know, share implementations. So MCP, which really is more about the way context is packaged, to be sent to the LM. For me as an application developer, yes, the protocol and the structured, you know, way of sending data back and forth between the LM is important. But more importantly, it helps me think about the AI agent in more modular way, where now the application can have, not have to hard code the tool implementation within the agent, but can be moved out into an external component that can be shared by multiple agents. It can be maintained independent of the agent code, and makes my life as an AI agent or developer really simple. I've dedicated the, the implementation of the MCB server and the tools to the expert on that particular service. They can maintain it, and I can just simply consume their capabilities as part of my agent code. So an MCB based, application, is really made up of three components. There's a host application, which is housing the client code that can talk to the MCP server, retrieve the tools that it can pass on to the m c to the, LLM as the set of capabilities that it has. And then, when the m c when the LLM tells which tool to call, the client can basically interface with the server to call the tool on the server, and the server can give back the results. So the entire tool description, tool implementation, all that is kind of on the MCB server, and the agent is simply interfacing with the MCP server to, execute the tools and call the tools. MCP server already has three pieces. It has a tool implementation. It also has an ability to expose resources back to the agent or facilitate, you know, any kind of persistent store that might be needed across multiple tooling locations. And it also, can have prompt templates which really exposes how to really invoke the right set of tools, and pass in, and and retrieve the results, out of the tool invocations because, the expert on the of the particular service knows how to take the input and output out of their their services, you know, as it is interfacing with the LMS. So what really changes in the basic AI architecture agent architecture that we have looked? Here you can see that all that has happened is instead of having the tools hard coded within the agent, its implementation has moved out of the agent into the MCB server. So now, when the user prompt comes in, the agent can connect to one or more MCB servers, retrieve its tools, and then expose those tools as capabilities to the LLM, along with the user question so that the LLM can, you know again, the role of the LLM doesn't really change. It, again, comes back to the plan, the tool, the tool input parameters, and so on and so forth to the agent. And now the agent, instead of invoking the tools locally, it can then call upon the MCP server to invoke the tool, get back the results, package the results back into the input to the LLM, and let the LLM do its job of executing the plan till it has the final answer. So from, from the interaction of the agent with the LLM, nothing much has changed. Just the agent has, has become simpler in terms of not having to implement the tools, but just having to interface with the MCP server to retrieve the tools and call the tools. So here, you know, from a core perspective, instead of passing and hard porting the tools within the system prompt, now the AI agent can dynamically create the system prompt by connecting to MCP servers, enumerating their capabilities, incorporating that into the system prompt, and sending it to the LLM. That's all that has changed. Instead of hard coding, you are dynamically building the system prompt. And the beauty of this integration is, because MCB server is external, because the agent connect to the MCB server, if the MCB server evolves, they might introduce more tools, change the implementation, because the, interfaces are changing of the dependent service and so on and so forth. All of that is on the MCB server side. The AI agent can automatically evolve by virtue of new tools that might be exposed by the MCB server that it connects to, and become, and and can evolve a set of capabilities that it declares to the LLM. So, the exciting piece is that if the service introduces new functionality that is exposed to the new tool, nothing changes on the AI agent. It dynamically can expose this new capability to the LMM for it to be able to solve more interesting problems than it could in the past. The other interesting nuance over here is because elements are so good at, you know, processing on structured data, producing, data in the required format. You know, MCB servers, are expecting the action input to come in JSON format because they're really implementing this as JSON RPC. I can tell the LLM to structure the input to the action as a JSON object, and, LLM will gladly comply and give me back the action input in the expected format. And that I can seamlessly pass to the, MCP server when, requesting you to invoke a tool, and so that the interaction can continue on. That's so those are the kind of, things that have, that change in an AI agent architecture, in in in light of MCP. So now let's look at, the security risks of an AI agent. As you might have realized, AI agents are really meant to serve the needs of diverse, kind of users within an enterprise. So, you know and different users within enterprise could belong to different roles, can have different kinds of privileges, authorizations on various systems, etcetera, whereas the agents are serving the needs of many users. So by definition, the agents, have, more capability than, than a variety of users within enterprise that are that are calling upon the agent. So, when a user is interacting with an agent, you know, users may be able to trick the LLM or guide the, sorry, trick the agent or guide the agent to perform actions that they would not otherwise authorized, unless the agent is really taking the user, you know, authorization into account when invoking the tools. So unless the AI agents have been designed, correctly, there is a possibility that the user may be able to tell the agent or guide the agent to perform elevated, actions, privileged actions that they themselves did not have access to. And then because the agents are processing input, whether it is user input or data that has been, collected on behalf of the action results from external systems, etcetera. You know, as long as there is some data involved, there is room for, malicious or power play where, the input data can be, can be constructed in malicious ways to cause the agent to misbehave. We'll we'll dig deeper into what this misbehavior really is, but, unless the input data is analyzed properly, cleansed properly, there is risk of, causing the agent to misbehave by, what you're making it do unexpected things. Now because the agents are interacting with external systems, in order to interact, you know, with, protected, systems and services, There are access tokens and credentials involved. If the agents are not properly managing these credentials and and and, in services like Vault, there is room for, accidental leakage of credentials by virtue of source code, you know, and and so on and so forth on behalf of the agent. Then, you know, as as we talked about, a capability called, memory that is part of an agent design, There is risk of, you know, that memory being, exploited to retrieve information on privilege, you know, on privileged questions, or entities that the user did not have access to. So what I mean by that is, let's say, you know, the response time was being questioned about two sites, abc.com and cd.com. And maybe the user did not have access to, the response time of abc.com. But if the agent ends up recording the response time of abc.com, you know, and and when a user who did not have access to query the response time of cd.com abc.com asked a question, and if the agent just leverages the memory instead of trying to execute the tool, it could potentially answer the question about abc.com, which the user did not have, privilege to, privilege to. And thereby, the memory can, serve, answer to to, your questions about privileged resources, that the user could not have access in the past. So when when designing, the memory of an agent, what to record, what to cache, what to use as context to the LLM, I want us to keep into account what kind of isolation mechanisms have to be put in place when it comes to multi user scenarios, interfacing with, with other external systems, which have different levels of privileges for different kinds of users. And lastly, because, agents are invoking these tools, which are invoking functionality on top of other, excellent services, because this is machine code, if the agents go, have a bug or run away loops, etcetera, they can really, stress the external systems by, more than necessary tool invocation or API invocation on top of these services, thereby causing these services to become unresponsive or overloaded, causing denial of service not just for agent, but on, these services that had other purpose within the enterprise. And not to mention resource abuse where, you know, it can, cause over usage of tokens on top of LMs or resources on these services, you know, making them inaccessible for valid use cases. MCP by themselves, you know, expose an add on other risks. And as I've mentioned, MCP servers are think of them as simply as wrapping the endpoints on these external services as tools exposed back to the LLMs. MCP servers really can be used in two ways. One is you can literally run the MCP server locally, as a mechanism to invoke the endpoint, that you were going to otherwise invoke directly within the agent code. But MCP is just giving you a level of abstraction to these endpoints. All these MCP servers can be, hosted as a web service shared, that can be shared across multiple agents. Now when these are, exposed as a shared service, you have to make sure that they are properly authenticated, proper authorization checks are performed so that different users of this MCP server can be properly authenticated and authorized to invoke the tools that are exposed by these MCP servers. So authentication authorization becomes the first level of security risk on top of an an MCP server. And then, you know, these, MCP servers, depending on who's, writing them, where you're getting them from, These could be, things that your team has locally developed or you have a source of from various vendors. You have to, you know, there are security risks therein where, things like full descriptions can have, interesting malicious tokens that goes to the LLM that influence, how the LLM responds, what tools it calls, what parameters it creates for these tools, and that can lead to all kinds of interesting attack scenarios, leveraging the agent functionality. So I'm just gonna walk through, like, three, easy to comprehend examples. The first one is called tool poising where, you know, where an MCP server, you know, that is created by a malicious, actor or a bad actor, really, you know, makes the tool perform other than intended actions. So here, me as a malicious, as an attacker, giving, you know, that that publishes an MCP server can really, make a benign action like adding to numbers, make it do other things. So here, the description of the, add tool guides the l m to execute other capabilities on behalf of the agent to create, to fill in for other parameters of this tool that needs to be executed. So here, the add tool exposes three parameters, the two numbers that have to be added and the side note. Now the side note has to be populated by the, by the agent. The description of the tool guides the agents saying that the side note really is to be populated by reading a privileged file on the agent and passing that content of the file as an input to the, to the tool and location. So me as an, publisher of this MCP server can get access to, confidential and privileged information that the agent had access to, because that content will be passing as a side note, input parameter. So here, I, as a publisher of a malicious MCB server, can make a very benign tool, do other risky activities on behalf of the agent, and get access to information, that the agent has access to. So this is a tool poisoning attack, and it just causes the agent to do more actions, that you did not expect, to be done, when using that MCB server than it's true. Rug pull is a classic supply chain attack. So you as a developer, when you're, you know, when you're you know, when you want to incorporate interesting tools as part of your agent functionality, it could simply be, you know, looking for, MCB servers that have been already implemented, and using them as local or remote servers. And and the way you would go about deciding on which MCB server to use is by looking at what kind of capabilities it has. Maybe you'll do some due diligence to look at the port if it is available to see what it does, and and you would just then trust it and incorporate that as part of the agent functionality, not realizing that, the capabilities of the MCP server can evolve and change over a period of time, whether as code or as an external server. And, certainly, unknown to you, this MCP server could be doing other malicious accuracy or objectionable actions that you did not want to be executed, and you would simply be unaware of these changes over a period of time unless you're kind of, you know, constantly keeping on top of what what is the evolving set of capabilities of the NCP server that you have, a dependency on. So this is kind of a classic supply chain attack. You know, you can think of it as the old school library dependency where the, capabilities of the library can change over time if you're not really fixed on a particular version and, and to check some of the, you know, a specific information of the library because the library information can change over a period of time, and can do more than expected things. Tool shadowing is the third, kind of a tactically where, really, one tool description, is guiding the agent such that when it executes other benign tools, it can, it can, influence the behavior of those other benign tools. What I mean by that here is that the publisher of the act tool on a particular MCP server, is counting on the fact that if the agent ends up having has an ability to send an email, that when creating the input parameter for the send email tool, also add this one more parameter to send a copy of the email to the attacker's own email address. So it's not about, you know, having new capabilities in in the ad tool to do malicious activity. Act tool by itself is doing is very benign, does not do anything malicious by itself. But the description of the act tool tells the agent to change the parameters, or append to the set of parameters to send emails so that it can send an email to not just the intended recipient but also to the attacker. So it is kind of abusing an authorized benign tool by virtue of, you know, introducing malicious parameters, that needs to be passed on to the invocation of this benign tool. So, hopefully, you know, that kind of shed some light on kind of lists that MCP servers can introduce within your environment. Let's look at some of the engineering best practices. So when you think about implementing an agent, there are many choices available all the way from building these AI agents ground up. You know, there's really not too much complicated, code in building an agent. Or you can use frameworks like Landchain, Llama index, and so on support that can really and and there are all kinds of tools available like AI Studio, that can really, help you build agent in a no code, local way, where it really hides all the interaction with the LMs, processing of the user input, back and forth between the LN, and really helps you focus on on the business logic. So but by virtue depending on this framework, you're really, introducing a dependency on this opaque layer that attracts to the LMM. And I would, you know, the kind of person I am, I kind of, get very anxious when I'm depending on a black box that does things for me, and I have no control over what it's gonna do and how I can improve, make it more efficient, and so on and so forth. Nonetheless, it's a very legitimate and valid choice available to you. So, yes, it's a pretty important decision whether you wanna introduce a dependency in a framework or you're gonna write it from ground up. The pros and cons, you know, make the decision wisely. The second important thing in, building an agent is prompt changing and context changing. So prompt changing is all about really designing the system prompt thoroughly, which can guide the agent to, you know, do things like breaking down the task properly, generating the, action and location, input properly, how to, you know, how to use the results of the action properly, and so on and so forth. And a a good system prompt can also allow your agent to deal with errors so that it can, respond with alternate tools, different parameters, etcetera, so that your agent can become more resilient. And as agents have memory, the, information can be, you know, extracted very, efficiently and cleverly from the memory to create the context, which is extremely relevant to the user's question and not just the entirety of the history that we might have. Because the entirety of history might have lots of irrelevant piece of information pertaining to the user question. So the more relevant context you can give to the LLM, the more effective it can be about which tools to invoke, and how to answer the user's question. The next is about LLM selection and various APIs and parameters of invoking the, LLM functionality. Not every LLM is the same. Some are good at, you know, tasks like summarization. Some are good at creating tasks like reasoning and planning. You can't just simply blindly change an LMM underneath an agent and expect it to continue exhibiting exactly the same functionality because, a summarization LMM, might not be that great at, you know, planning and reasoning and breaking down the tasks and, you know, and so on and so forth. So be aware of that. And the other important thing is, most of the LLMs or published models now have some built in guardrails. And you might be benefiting from these guardrails unknowingly. So things like, you know, LLM psychotropic will not, answer questions about, violence or cell phone. And your application might be benefiting, by not having access to that kind of information. But if you switch from entropic to some other LLM that does not have these guardrails, now suddenly you are susceptible to this content that can be generated by the LLM that you did not want, you know, to be part of your application. And then, in terms of the system prompt, which really allows the LLM to strategize around, problem solving, there are many different techniques. You can you should really choose what technique is most relevant to your use case, and communicate that, technique to the LLM as part of your system form. I've already touched base on, you know, the necessity of resiliency and error handling, but keep in mind that LLMs are kind of indeterministic. You know, at the end of the day, they are passing out such a data depending on what data they're looking at. You could get things like that. Inputs to the action calls, you could get, you know, the actions could themselves fail and could generate results that the LLM was not expecting, and so on and so forth. So keep all of those kind of issues in mind when you're designing the agent and interaction with the LLM and interaction with the tools, and design proper loop exits and those kind of things so that you're not having infinite loop kind of issues, within your agent. And lastly, have a testing strategy, which kind of checks the agent for some very predictable behaviors. And as you're changing things like planning strategies or LLMs, you can test your agent for, continued consistent capability that you wanted out of your agent. Now in terms of data security best practices, everything up here is about data. Be very careful about where you're sourcing the data from. They need to be trusted sources. And as you're pulling the data on these sources, you need to make sure that there are no malicious tokens, no sensitive information before you send it to the LLM or before you return the answers from the LLM to the tools, to the user, etcetera. The agent the AI applications should follow the least for this principle. They should only be privileged to do things that are really the intent of the application. They should not have access to the local file system and publish files unless it is really necessary for the functionality of the agent, so that you can reduce the attack surface. You know, prompt injuring, again, you you can really express good constraints about the application, the agent behavior in the system prompt, thereby letting the LLM remain true to the intent and not, you know, digress into, areas that are really not the scope of the AI application of the agent. So, designing a proper system prompt with constraints is very important for your application to remain on topic and secure and so on and so forth. Of course, there are prompt ingestion attacks which can evade those constants expressed as part of the system prompt, and, hence, you need to have other guardrails, which do not minimize the effect of, you know, properly designed system problem, but they really help you to keep the application more secure from arbitrary input. So things like prompt injection guardrails, guardrails that can identify sensitive information, help you properly redact and manage the sensitive information, do things like malicious IOC checks, making sure that the input parameters or tools or information being exposed back to the user is not malicious, or known malicious entity. So these are the kind of, you know, best factors from a security point of view that you need to keep into account take into account when building, AI applications and agents. Now in terms of, guardrails that you should take into account, there are really, various choices available to you, all the way from guardrails that are part of the LLM or part of the platform that's post CLM, open source. You can build your own or you can use a commercial vendor. Every choice has pros and cons. You know, the LLM guardrails are usually very generic that, you know, that really addresses broad use cases and not focused on enterprise specific use cases. So it'll be good at you know, various alarms can be good at things like violence and self harm, but they're really not gonna be able to address things like, you know, if, if you have a financial application that should not engage in, in legal advice, that legal advice is a legal risk. It's a liability risk for the application. There's nothing wrong with legal advice, but that might be a topic that you might want to avoid, on a application. And the generic guardrails available in ILM are not really gonna be able to help you with that kind of use case. Now open source, is a great option, but it really relies on your team having really good expertise to use that open source tool, maintain the open source tool, scale and perform, etcetera, in order to get the best outcomes. You can build everything ground up on your own. That's definitely an option. But, again, you need to wisely decide if that is an investment you want to make. That could be a distraction to to your core business objective. You know, you are a financial, you know, portfolio management company. Do you really want to build, prompt injection detection capabilities as a core skill of your engine team, or do you want to really leverage open source, commercial, and other offerings available and focus on the business use case? So that comes comes to the option of commercial endpoint vendors like Pangea. Commercial vendors are gonna be really focused on solving the problem to the best of their ability. That's their whole focus. So they are constantly on top of, how the attack surface is evolving, keeping the protection and guardrails up to speed, on defending the applications against these new techniques. And and they can really, you know, be very relevant to the evolving attack surface attack, you know, landscape. And they they also would be, focused on, making these capabilities easy to use, easy to integrate. And and, you know, from a cost perspective, I would argue that commercial offerings could be cheaper, you know, because build your own and open source require spend in different ways, like people, maintenance, infrastructure, and so on and so forth. So you could I could argue that commercial offerings could be cheaper compared to build your own on open source offerings that you could otherwise take advantage of. I'll quickly touch base on comparing NEEMO within commercial offering because NEEMO is one of the most popular. I mean, the takeaway from on this slide is, in order to effectively use NEEMO, you really have to become an expert on NEEMO. NEEMO exposes, a different way of, expressing the guardrails that is you need to build expertise on colang. You need to understand how to invoke the right guardrails, what are the triggers to invoke those guardrails, and so on and so forth. So you have to, really understand Nexmo's capabilities, how to invoke the right guardrails at the right point in time, maintain, and scale the Nexmo infrastructure, to be useful for your application. But as a commercial offering like Pangaea can really make it easy. You you know, in our case, it's a single API call. You you declare, what kind of, guardrails need to be enforced and what use case. You invoke the API with the right parameters, and the API does the magic and gives you back the results. So, the least overhead in terms of maintaining, and using the capabilities, are, you know, really the strong point of, Pangaea AI guardrails. And then Nexmo like Infosys could be dependent on other LLM vendors for more advanced security capabilities that is to offer AKA LLM as judge. Whereas in case of, Pangaea, we really don't have any external dependency, although we can offer exactly the same kind of capabilities AKA LLM as judge by incorporating these elements as part of our service and not having to introduce a dependency on third party element. Now, I can briefly talk about, our guardrails. Now in terms of Pangaea's guardrails, we look at the entire security, landscape as in in five pillars. There's prompt injection, detection, which is fulfilled via the prompt guard capability, which really excels at detecting various kinds of prompt injection attacks, direct and direct, jailbreaking, multi turn attacks, and so on and so forth. Content and data moderation really focuses on identifying sensitive information on how you can, manage sensitive information as it traverses your application, including topics that you don't want, to facilitate on your AI application, thereby reducing risk, reducing liability concerns, and so on and so forth. Lack of visibility is a big problem. What is going on? Which user is asking? What kind of questions? What kind of answers they're getting? Security teams have a dire need to get visibility on top of AI usage. And so Telstra's auto log service provides this auditing. The AI Guard service, that you can invoke via single API call inherently does the auto logging unless you explicitly ask not to log, thereby giving the security teams and the application developers full visibility on the usage of AI by virtue of calling this API. Authorization is, you know, as we've talked about, a big problem for AI applications where they can perform privileged actions, tool, and locations, etcetera, or even access data that, users were not authorized to. Pangea auth these service along with an open source library that we published called MultiPASS really allows you to enforce proper authorization on top of, you know, architecture side lag application where we can enforce document and chunk level authorization in real time to make sure that users are only getting access to content that they are really authorized to and not getting access to information that they were not authorized to. All of these capabilities are and more, are then offered for, an agent a agent tech AI or AI agents. You know, we, we provide authentication authorization capability that you can implement as part of the MCP server when you're launching these MCP servers as a shared service within the enterprise or externally for you for external users. It we also have an MCP proxy that allows your clients to always, go through an, a security guardrail, ten year AI guardrails when inter interfacing with other servers. So every tool call, input and output is processed by this proxy, without requiring any port changes on either the agent or the MCB server. And by invocation of the guardrail, we can make sure that the any tool input or output does not have malicious tokens, prompt injection tokens, sensitive information, and so on and so forth. And, again, this proxy can then also be doing logging to give you full visibility into what the, agent has been doing. Now in terms of the guardrail, service that we have AI guardrail service, it has a notion of recipes that really is all about setting up what kind of guardrail needs to be invoked, for what kind of use case within your AI application. And you can just preconfigure the service with these recipes and just invoke the API with the right recipe for the right use case. So if you want to call the API for ingestion use case where you want to make sure no sensitive information is leaking into the context data or vector DB, you can design a recipe to just focus on sensitive information leakage and call it an ingestion recipe and invoke it in ingestion use case. If you want to make sure that there is no malicious tokens coming in as part of the user input or the input going into the other end, you can design the recipes to do the respective things and invoke the right recipe at the right point in time in your application. Now, of course, this API call can be integrated with the application, but we also provide these APIs, where you don't have to really modify any application code because we integrate with things like API gateways, with, you know, frameworks like, Langchain. We provide, the ability to involve these APIs as, you know, a browser plug in when the user is interacting with AI applications. So these APIs can be involved, you know, any in any place or any way, or anywhere where, AI interactions are happening, not necessarily as code within, within your AI application. So the APIs really make it flexible and give you various, choices where you want to invoke these APIs. So that was a quick, summary of, Pangea's AI capabilities. Thank you for, taking the time, and, you know, I hope this session, gave you some better insights, some new information, in into AI agents, their functionality, and their capabilities, risks, what are the various choices available to you. I I hope the session was useful. Thanks. Amazing. Sourabh, thank you so much for the fantastic session. I'm gonna invite you back onto the stage, if you're able to rejoin. But, yeah, I really I mean, going by the the amount of activity in the chat, that was wonderful to see. So thanks for so much detail, and maybe you have a couple of minutes. I'm I'm conscious the webinar has two and a half minutes left on it. So we'll do a couple of questions, and then I've dropped your LinkedIn in the chat as well, and folks can reach out. But if you're if you're happy to, sir, let's start with Carolina Johnson's, question. She asked fairly early on, so, maybe you can re give some context. But she says, so is this what ChatGPT does for me when I ask for something outside its knowledge? So it goes into public databases to retrieve the right in the right answer for her. That's API action she asks. Yeah. I mean, chat g p d, web interface or thick application if you're interacting with is actually just an application using LLM in the way I've described. So when you ask a question and it, it has the ability to invoke things like web searches and leaking out the other source of information to pull in the right kind of data that needs to be then incorporated for, sending it to the LLM as a context to come back with the right answer. So it feels like Chargebee has access to real time information, which as an application it has, but the LLM backing it does not. And it's able to answer that question about real time things. Awesome. Brian. Thanks for joining, Brian. Great to to hear from you. He asked, for a Genentech AI governance layer, do you have an Genentech risk scoring and agent onboarding assessment code flow? No. Not really. I mean, we are not, providing any kind of scoring for how secure your agents are. That's not something that we do. And then we've got maybe time for one more quick one from Scott Garson. He asked, how does the LN know what tools are available to it? Yeah. I think we made it if it wasn't clear, I'll repeat that. You know, the the agent is really declaring to the LLM as part of its, system prompt of the data that is giving to the LLM along with the user's question that, hey. This is the set of capabilities that I have. So all of the information about what the agent can do is being exposed to the LLM by virtue of it declaring its capabilities. And it's the key thing to note here is that the description of the tools is really what makes the LLM recognize the fact which right tool to invoke or what kind of problem that the LLM is trying to solve. So the more details you have in your description of the tool, the more likelihood that the, LNM be able to identify which tool to involve or that problem to be solved. So the declaration of the tools is happening via the system prompt or even, you know, the protocol has kind of like, with MCP, it's really a tools array that goes along with the system prompt, and the LLM is able to identify what kind of capability the agent has. Awesome. I know we're overrunning slightly, but I thought Jim Dushami had a awesome question. So let's, quickly go through his. It was, if an LLM is aware of many more agents, for example, if there are multiple agents that claim to do response time, would it ask all applicable agents to pick one and ask until it gets an answer? Yeah. I mean, it it yes. I mean, there is a there is the tool redefinition problem where, you know, it's not just about multiple agents, but you could be connecting to multiple MCP servers. And multiple MCP servers could be exposing exactly the same tool, in which case, the LLM can really arbitrarily choose, one of the many tools. When I say arbitrary, it's not that arbitrary. You you know, the more detailed description the tool has, the more clear parameters that it says, what it needs, number of parameters that its tools have, all of these things are taken into account or influence the selection of the right tool by the NLM, when it tells the agent to invoke the tool. So, yes, it is a problem, a problem that can be managed. And, you know, you need to, monitor the fact that there are redundant tools being exposed to the LLM and, hence, you know, any of the available tools can be picked as one of the tools to be involved on behalf of the problem solving step. Fantastic. I think, Sourabh, let's let's leave it there because I know we're overrunning and, don't wanna take up any more of your time. Do you have any kind of closing remarks you wanna leave people with, or maybe you're happy for them to to reach out directly? Yeah. Folks, feel free to reach out to me directly. More than happy to interact. You know, my LinkedIn profile was posted by Tim. But thanks for, taking the time to attend this session, and I hope, you learned something and, you know, happy building secure AI agents. Awesome. Wonderful. I'll now leave it on. We'll be back with the Pangea in a few weeks' time with another webinar session. But for today, Sourabh, thank you so much for leading the session. Thank you everyone for joining us, and we'll see you soon. Cheers. Thank you. Thank you, Tim.