Name: The power couple: AI + humans for smarter customer service
Uploaded: 2025-05-20T16:06:03.876Z
Duration: 44 min 8 s
Description: The power couple: AI + humans for smarter customer service

Transcript for "The power couple: AI + humans for smarter customer service": Hi, everybody. Thank you for coming to our webinar today. We are going to be joined by Brian, the director of solutions consulting at Kustomer, and Aditya, the product leader at Kustomer. Today, they're going to be sharing how AI and humans form the perfect power couple for smarter Kustomer service. So I'm very excited to introduce both of them, and I hope that you both that you will enjoy today. Thank you. Hi, everyone. Let me share my screen. Alright. So, appreciate everyone's time today. I'm joined today with, Aditya, our AI product lead, a Kustomer, and and, I lead up our solution consulting team. We really wanna come to you today and talk about how AI and humans, equal a a smarter Kustomer service experience. There was early, early in this presentation, we wanted to cover a couple of great stats around, an article that BCG wrote, really addressing what there's a huge addressable market to, help companies evolve with their AI journey today. 74% of those companies are struggling to achieve AI and and get the value out of that perceived investment, and that's what we really wanna help focus on and and, you know, share information around how we've helped companies, through that journey and and some of our methodology around it. Now the inverse of 74% is really 26% of those companies that are, really figuring out that strategy between aim, humans and AI are seeing a lot of benefit in gains. Upwards of up, one and a half times revenue growth and 1.6 times shareholder returns. That's really impactful. And me as a solution consulting, leader coming into to situations and companies and executives, it's really helpful to to get a lot of metrics that play into those numbers and really what does that mean, for individual companies. And that could be hundreds of thousands, if not millions of dollars of input that that goes back into the business. And gonna talk a little bit more about this, but it ultimately turns cost center. Hey, Brian. So See you. Hey, Brian. Sorry to sorry to interject. I think you're showing the, incorrect screen. I think I I just realized it as well. It's on the slideshow mode. It is not. Was it was still showing the, non slideshow version. That's no fun. Is that working? No. Is this working? Yep. There we go. Awesome. So, let me go back to oh, this one. So, really, it it's what's holding us back as companies. There's a few things that play a part into this. A lot of it is just disconnection. I can speak from a few examples that that have been brought up in the nearest, conversation as as yesterday. Companies come and say, I wanna test your AI, and that's not necessarily the right way to look at it. It should be coming to us as I have a use case that I'd like you to evaluate and how AI can influence it. And that starts into this specific slide of, looking at how CX leaders want to influence AI and the the experience that users are coming into their their sites, whether it's a specific type of channel, whether it's something that's gonna help an agent be better at their jobs, in essence, creating them superhumans, whether it's a, you know, password reset or I have a return or just simple basic questions around their their, their experience. That's really where CX leaders seem to be thinking about how they want AI to influence their business. There's the customer facing portion of AI. There's the internal portion of AI, and there's a lot of in between. We can get as granular as talking about what models are right for what type of interaction, etcetera. But But it's not just turn AI to a chatbot and expect to do magical magic and and and all the work for you. It's really geared around how do you want that to influence people's daily jobs, how do you want it to supplement what's existing, How do you make them better? And ultimately change that that end user experience, which in turn does things like changes CSAT and and response times and and close times and and all of those sorts of things. So a lot of that when you're going into company, or sorry, when you're coming to us or coming to other AI vendors to evaluate how AI is gonna influence your job or your company or your customer's experiences, you gotta have some of those things in mind. Or, again, we're happy to to help pave that path and and share with with that crawl, walk, run approach, if you will. We found that, really, AI for CX aligns in three major categories, operational efficiency, CSAT loyalty, and revenue growth. Operational efficiency really dives into basic metrics like average handle time, faster to keep track of agents? How do you cut down on support costs when you can offer twenty four seven support where you don't have to have a physical human standing by and and being there to answer those questions? And the last part of this is agent efficiency and productivity. You're gonna hear us talk about a few different things when we when we get to, more of the product. But we have AIR, which is AI for reps, and that's really that internal agent coaching. We have AIC, which is that customer facing, you know, omnichannel approach of of how do you handle different types of inbound requests, various types of of actions at different types of stages, all influence operational efficiency. And as you start peeling back an onion and and how to incorporate AI into those different parts of the business, you are really able to fight off pieces that make sense. You don't have to get overwhelmed by the amount of information that that's trying to be, handled or or dealt with, but really gets into the to the you know, how do you make your your cost center much cheaper and and more efficient? And then that, in turn, turns into a better CSAT. I'm a happy customer of a lot of different brands that leverage AI, and when I can reach out and get the information that I want when I want it, I tend to be a happier customer. If you need to be, you know, transferred to an agent or something like that, you're able to do that in a very seamless fashion. And most importantly, you don't have to retell your story or or, share any of that information that that makes it kind of frustrating as a customer to to get into that. So a lot of these things go into that CSAT loyalty aspect, creating personalization. So if you have a, you know, a hoodie that you bought that's not the right size, being able to authenticate that you are that person, being able to go get historical data, making that, you know, correct purchase or right size or fix that issue with that hand, being able to do that in the fastest possible where that customer is is another indicator of of providing a a happy experience and turn a a more loyal customer. So there's a lot of other things that we wanna go into and help companies track around CSAT loyalty, feedback loops, how do you collect feedback on an ongoing basis, how do you surface that back to the user, how do you continue to delight them at, wherever they are in their journey with with you as the, as the the the end user experience holder, if you will. But, again, we wanna keep that across channels and and make that, seamless no matter where you are. And then the last area, we've seen a lot of of time and time again, companies that have CX organizations that are really cost centers, turning them into revenue growth engines. How do you proactively reach out and provide upsell opportunities? How do you delight them with maybe coupon codes to, on their next purchase if they had a negative experience or something like that? But but mostly around those agent efficiencies, you're able to get companies to facilitate, better experiences with lower cost of ownership, faster average handle times, and in some cases, you're able to really directly correlate that to, renewals, upsells, and and turning that cost center into a into a growth monster. How do we do that, you might ask? That's a great question, and I'm gonna pass that over to Aditya, to go over a few other areas of how we do that. Yeah. Thanks, Brian. So Brian talked a lot about how the market kind of perceives AI and some of the goals that they wanna achieve using AI within their organization. And in in order to achieve that, in order to think about the how, the first and biggest step is probably thinking about the two types of parties that are involved in this interaction or in these transactions. One is the AI agent, and the other one is human the human agent. We always like to tell our customers to think about them as partners in this journey. Think of them as collaborators. Think of, how you can achieve the outcomes by allowing AI and humans to collaborate effectively with each other, not necessarily replacing one another in order to achieve those goals. And so the way we like to think about it is, see the previous slide, right. Right. Is thinking about using AI for speed, consistency, automation, whereas we use humans for more nuanced or emotional intelligence based interactions. So, you know, use humans for some of the more complicated ones where maybe there's an escalation, maybe there's an upset customer, etcetera. But then leverage AI to take care of all of the routine tasks, the low effort, high volume work. Think of it as, like, the WISMO use cases or, you know, where is my, order use case? But then the human being pulled in for maybe, an escalation for a VIP customer that is particularly upset with, with their particular with the products that they have purchased. Next slide, please. How do we then think about deploying it? Now that you have determined, okay, which are the types of use cases that AI needs to, work on whereas where does humans need where do humans need to be brought in? You also need to think about how you deploy this within your organization in a step by step manner. The way we like to think about this is you always wanna think big. How what is your big picture in the sky that you wanna achieve? But then at the same time, start with incremental small steps. So step one is always thinking about building trust. So positioning AI as an assistant, thinking about AI as something that is enabling humans to do better at their roles, whether it is a human agent or supervisor, or also potentially helping our customers with some early tasks. You start small. Start with maybe one specific use case that might be the highest volume that you see that is, like, low effort but high volume so you can start to see some immediate gains within the organization. Focus on some of the repetitive tasks where things are more transactional in nature and speed matters the most. And then make sure you also have clear success criteria for each and every gate as you think about expanding. And probably the biggest thing that we have seen a lot of our customers work on as well as our own internal practices is around testing and optimizing. We don't think of AI at Kustomer as like a one and done approach. It's always around you start you build one or two use cases. You continuously test it out, optimize that experience, and then continue to expand those use cases from, say, the first two or three to maybe the next six to eight that you wanna, offload onto the AI, engine. Next slide, please. And really where where to start. The the next couple of slides are sort of talking about customer's philosophy to adopting AI within the organization. The first one, you have human in the loop, where you've we've started to talk about AI agents for reps. Reps over here is really human agents where you might say, you know what? I'm starting off with adopting my AI in a fairly early journey where I want AI to quickly summarize information for my human agents to work on. It's relatively low low risk. I'm not as familiar with AI, but let me start off taking that on so that humans don't have to look through long conversation some long conversation histories, search for customer information. It's all being presented to them in a summarized way, leveraging AI. The second step in that journey is starting to leverage AI to figure out, okay, what should I respond to the customer with next? How should I better tailor my message so that it's more empathetic, that that there are no spelling mistakes, grammatical issues, etcetera? So that's like second step where AI is sort of helping you craft messages. The third step in that journey that we think about is how do we also now introduce maybe a copilot where the AI can now and, oh, sorry. The human agent can now interact with AI to maybe get questions answered. Maybe it's because maybe the AI is connected to knowledge bases. Maybe the AI is also connected to, third party data sources, maybe like your, you know, auto management system or, calendaring solution, etcetera. So the Copilot can now respond to question that the rep is asking or the human agent is asking without the human agent actually going to different systems and trying to search for that information. The next part of that is now starting to allow AI to also start suggesting actions based on all of the context it has. It already has access to your conversations, to customer information, to your historical, you know, KB article or knowledge based articles, etcetera. And using all of that, AI can now intelligently suggest actions to the human agent, and the human agent is still in control of all of these steps. So the key thing to keep in mind with the first three steps of this journey is that AI is still largely assisting humans in order for the human to be able to do their job much better. So it's still that the human is in control, almost thinking of the human as a supervisor in this entire mix before you then go to the next step of your journey where you're starting to put in AI directly in front of customers. And even then, you may start off with smaller steps. Maybe you start off with allowing the AI to respond to basic questions that are otherwise present on your FAQs. Maybe the AI can pull certain information from the customer's history to, give it access to maybe order information or travel information, etcetera. So it's still largely q and a. And then finally, as part of that, it's like as soon as the AI sees a question coming in that requires an escalation to a human agent, it's able to clearly hand that off to a human agent to take additional steps. So, again, over here, it's still you know, AI is starting to take on more actions and automate that without a human being able to without a human having to be involved in it. But, eventually, you know, at this step, the human still gets involved for some of the more nuanced cases. Maybe it's like an auto return or a trip needs to be canceled and money needs to be refunded, and you do not wanna use AI to perform some of those more, you know, critical steps. And the final one is when you're really starting to move into this autonomous journey where AI is now starting to drive a lot of the conversations and is able to take on more and more complex, steps where maybe you can start to initiate returns. Maybe you can start to update information on the customer's profile without a human being necessarily overseeing every single step of the way. And maybe it also can go to the external making product recommendations. Going back to the original point around CX being the revenue driver, you can start to see AI becoming a little bit more autonomous and starting to recommend related products for the user based on maybe their purchase is free, based on their preferences, etcetera. And finally, you may still need to hand over, you know, you may still see situations where a human is working on a ticket or a conversation, and that actually says, you know what? The remaining of this conversation can actually be handed back to, an AI agent. Because after this, I've already taken care of the escalation. I've taken care of the upset Kustomer, but then if the Kustomer has any additional steps and the AI is able to handle it, I can now hand back to the AI agent. And so what you're seeing over here is sort of what we have seen as a as, like, an optimal deployment path or deployment strategy for how as a as an admin or a CX leader, you can start to take on or onboard AI onto your organization. You start off with relatively lower risk where, you know, if you're very new to the AI journey, you start off with AI just summarizing all of the information that's there on your CRM. And then over time, you can start to deploy AI in a much more staggered fashion up to the point where AI is a lot more autonomous, taking on, you know, 70% of the interactions, and then leveraging humans only for the last 20% of the, of the world that's needed. Next slide, Brian, if you want. And so the way we think about this in Kustomer is really a five pronged approach. One where, you know, when we we call AI agents to be designed as smart agents, they're specialized, they're multichannel, they're they have advanced reasoning, they're responsive, and finally, and the most important one is it's a it's a team. So I'll go through each of these in a second. So with specialized, what that really means is you can create AI agents to be very specialized in specific actions that they're expected to. The the way I think about the analogy over here is, you know, similar to any CX organization where you may have teams of agents where one team might be focused on l one support, another one might be focused on l two support, so on and so forth. Each of those teams have very specialized functions. They're allowed to take care of certain actions, certain steps. And if they need to escalate, they will find another team member to escalate that to. So, normally, that's sort of how we think about AI agents within Kustomer as well where when they're deploying AI agents, you're thinking about them as specialized, workforce, folks that will focus on specific actions and do those actions really well. They're also multichannel, which means they can operate across several channels, be able to adapt itself to different channels, whether it's on chat or email or voice. It's there's a lot of reasoning that goes into play. We have invested heavily into our AI reasoning, engine that allows the agent to actually take through go through multiple steps in order to complete a user's request. So it's not just a one shot approach where response comes in and it's sorry. The request comes in and the AI is responding. It's actually processing the request, looking at all of the information it has access to, the knowledge base, any custom information that it's got access to, etcetera, to determine what's the best next response and action to take in order to, in order to, ensure that the Kustomer, issue is taken care of. The first one or fourth one, I'm sorry, is being response how does it continuously tailor its responses to the customer's, tone, their maybe based on whether they are VIP customer or not, based on whether they are a high value customer or not, etcetera. How can it continuously be personalizing its responses depending on who the customer is? And then finally, the way we also think about, AI agents are customer going back to the first analogy that I put in where we always think of AI agents as a team of agents that are working not just amongst each other, but also with their human counterparts in order to resolve the, resolve the customer's issue. So you have this team of specialized agents. An agent that is focused on maybe order status, another one that is focused on order returns, a third one focused on, order purchases or recommendations, working in concert with each other, but also along with the human counterpart if they need to escalate based on policies. Maybe a refund is over a certain amount. Therefore, I need to off, you know, send it over to a human agent to take care of the request or maybe ask for an approval before it actually completes the request. And so that sort of brings me to the last one, which is our notion of human in the loop. Right? Next slide, please, Brian. Where AI and humans are really working together where, you know, AI might start off the conversation saying, alright. I see that you need help with changing the date of your flight. It said, look. You know what? I can't change the date of the flight, but it requires a change fee or it may require, like, a escalation to a human because it requires some additional steps. It requires some approvals, maybe from a manager to maybe waive certain fees of the Kustomer, the VIP, for example. Hands it back to a human. The human looks at the request, says, yep. This has been approved for, for a change or maybe there's some extraneous policy that the human is able to apply saying, yep. This is a one off use case, so that's fine. I'll approve any waiver in fees, and then hands it back to the AI agent to actually complete the request. So throughout this process where the the AI agent is also driving the entire conversation with the Kustomer, but then leveraging the human on just these exception use cases so that a human can still, one, oversee the AI and make sure that, like, those exception use cases are not, you know, being mishandled or incorrectly handled. So it's sort of so the human being a supervisor in that entire step, but then leveraging AI to still take care of all of the road tasks, the communication with the Kustomer, the empathy with the Kustomer, etcetera. And then that's really how we think about human in the loop within Kustomer where it's not just one single handoff from an AI to human, but also the human being able to now hand it back to an AI to complete the request. Right. Next slide. And so yeah. So if you please conclude my section of the slides, it's really around how do we think about the right agent at the right time. And keep in mind over here, the right agent at the right time is not just a human. It's also not just an AI agent. It's AI agent. Yep. You decide when to pull in an AI agent where it can partner with a human agent. It can it's not necessarily like a replacement at every single point in time. It's a team of specialists. So even amongst your AI agents and human agents, one of the specialties that you're including within your CX organization, whether it is, an AI agent that is focused on where is my auto step on what is my auto status to the point where you may have a human to escalate certain exception handling to. It's always powered by context. So whether it's an AI agent or a human agent, you're thinking about context awareness for not just the for not just for the human where it has access to all of the information on the CRM, but also the AI. How do you make sure it has access to all of the real time customer data so that it can take the next best action at the right time? And then finally, making sure that these agents are always available across all of the available channels from chat to email to voice so they can provide tailored and personalized experiences on each of them. Right? And where we are today, a lot of companies think we're in the yellow. But in reality, we're actually AI assisting humans. And that's one of those fundamental shifts that we need to figure out how we, either help companies or companies want to fit. But, ultimately, AI is helping humans today, and that's things like, you know, chat GBT. We always ask it to, you know, summarize notes or, make this better or fix the spelling here and things like that. When in reality, a well oiled deployed group of AI teams, is really just gonna, we're gonna help them. Right? We're configuring them of when to engage, how to engage. We're gonna tell them when to to do certain things, but because that's the best for our business and best for our company and best for our customers' end user experience to make sure that they're delighted. And, really, that journey is is what we wanna be a part of. So in summary, we really want three things to be taken away. AI and humans can and will and should exist. We wanna help companies with the crawl, walk, run phase. There's a lot of companies out there that want nothing to do with AI. There's a lot of companies out there that want everything to do with AI, from beta programs to, newest, greatest thing, And we wanna be a part of that entire journey, and we really wanna help the the humans in the loop, take advantage of technology and and delighting those customers and be involved when that makes sense, for different companies at different times. And the good news is not every company is the same. Everybody has their own, use cases, needs, budgets, all of that that goes into to how we can we can help and how AI plays a part in that journey. So I'm gonna stop sharing my screen and want to come over to questions. There looks to be, quite a few. So let me start. Somebody's asking if they're gonna be, forwarded a recording after this. I would assume so. I will have to check-in with, our host, but I would assume that this is recorded. So, yes. But there's also there's also during the q and a section, by the way. How do you place boundaries on what AI AI agents can talk about, and there's longer, hold on, with customers? Do you wanna take that, Aditya? Yes. So there's a few ways we do that, and I I'll also club this with another question from Tina around, you know, how do we how do we train AI agents on corporate policies to ensure that sponsor actor? So those two sort of go hand in hand, in terms of the response itself. So the way we think about AI agents over here is you always wanna give it you always wanna ground it with real world information about corporate policies, about, your FAQs that your internal agents might be using, human agents that is, as well as, the tone and the brand of your of your, of your organization. And the fourth part is also around making sure you have given given guardrails around information it should not be talking about, so specific specifically around things like competitors. And so using all of those pieces, you can essentially give AI agents instructions or contextual data on what it can or cannot do. So it's a lot of business to do with, you know, giving it, English oh, sorry. English or, you know, just basic instructions that you would typically just type out with without having to necessarily code anything into the AI agents or prompt engineering, being able to give the AI agents what it should be doing, what it can't talk about, what it cannot talk about. And at the same time, when should it also, escalate to a human in case, you know, it's if that's something that is outside of its, admin. So that's sort of how you would give it boundaries where you're spending a lot of time talking about prompt engineering instructions, etcetera. At Kustomer, we also have a lot of back end prompt engineering that we have done to make sure that the AI agent does not hallucinate, as they call it. But at the same time, we also give the ability for our customers to add on top of that to make sure that they can customize it to their brand, to their voice, etcetera. What AI agent tool are you using, and how did you evaluate vendors to find the right tool? Yeah. So we, we use OpenAI today, especially some of the more later models. And the way we have the way we do that is by ascertaining running a lot of evals or evaluations. This is, again, like a technical term in the AI world where you can essentially run through eval score and code, to determine is this model is this, vendor applicable or or effective for the use cases that my AI agent or agents are expected to do? We certainly expect that this will expand even within Kustomer where we will be able to offer multiple models that customers themselves can pick up, whether it's, you know, OpenAI or Anthropic in the future, etcetera, and then give customers the ability to also pick and choose. Okay. I want maybe I wanna use OpenAI for certain activities because I believe that, their models are better suited for x y z use cases, whereas Anthropic might be better for some others. And so over time, we'll certainly be making it such that multiple, models could be used. But today, we use OpenAI and how we evaluated that. There's a follow-up question. I can take a first pass at it because I've I've needed to learn this. But do you recommend training our human agents on prompt AI? Being in an AI company myself and having a team of solution consultants as well as PS and CS folks, understanding prompt engineering, I think, is is table stakes. It sounds intimidating. A lot of people when they first come in, they think, wow. I'm not an engineer. I don't know how to do that. But when you start peeling back what an AI agent is and how it works, it's really English. Right? The more specific, succinct instructions you tell it, the better it's gonna be, and it is this. It's it's a conversation of, you know, hi. I'm an AI agent in the customer support industry, and I deal with incoming requests from our customers, etcetera. I think the most technical side of it is better understanding markdown, and leveraging certain phrases in certain ways. But outside of that, I think, a prompt engineering one zero one course or online, there's so much information out there to just read up on what it is. At least it gives you a a baseline of how AI agents are trained and and what's going on. There's of course, you could read till you're blue in the face, and and there's so much information out there. But I think the simple answer to your question is, I don't know about training, but I would at least make it a an education one on one of this is how this system works and how AI works with other systems and what it does and things like that is a is an easy way to, you know, allow people dip their toe in the water and see how interesting it is. And, Aditya, you might have another comment as well. No. I think it's exactly what you said. I mean, it's even though the word it it has the word engineering, and that's really just around, leveraging natural language to help the your context of the AI agent. Some of the things that we have also learned from our internal testing and, you know, creating and seeing implementations is that the more specific you are with the AI agents, the better it's gonna perform in the long run. Anytime you leave any scope for ambiguity or, you know, yeah, anytime you leave scope for ambiguity, that's kind of when you see, things like hallucinations take place. The other one that I'll also call out is this is also where the team of specialized agents comes into play where the more specific you are with individual agents, the better they will perform as a team in the long run. So to Brian's point, yeah, I think prompt engineering is certainly gonna become table stakes very soon, if not already, and something that we will certainly encourage more folks to have the knowledge of. One thing you you brought up, that I wanted to cover, I was asked a question two weeks ago by one of the new, sales reps that came on is, why can't you just have one AI agent that does everything? And the response that I gave, and I think Aditya and I were were chatting about this, but the more people understand prompt engineering and understanding what AI agents are capable of doing is very similar to that slide that Aditya talked about. You wouldn't have one person do everything. You wouldn't have one person that's an engineer, that's a finance person, that's a a professional services person, that's a sales rep, that's in marketing, that's in HR. It just doesn't really work. Everybody would be doing, you know, mediocre work and not really great at anything, and the same thing is with AI agents. Right? You wanna have that specialization, and the more specialized agents you can have and learn from, the better that experience is gonna be. And that leads into one of the questions that someone asked. Tina, how do you ensure that AI agents and human agents don't duplicate work? One of the areas that we focus on different than other vendors in the space, we focus around the conversation. A lot of other vendors are a ticket, and a ticket has a lot of tasks and things that go along with that ticket. Customer puts all the emphasis around the user and the conversation so that if Aditya emails in, an issue, but he also picks up the phone and calls and he also sends a text with Kustomer, that's all coming into one place because Aditya is is a human being. He he has different interactions, different places, different you know, he wants to engage with us at different times. And Kustomer, we capture all that in one place. So you wouldn't have the AI agent responding that a human agent wouldn't see or multiple, responses in a certain thread because of the fundamental nature of how we consider a conversation, not a not a ticket. Aditya, I don't know if you wanted to have any insights into that specific question, but how AI agents and humans don't duplicate work. Yeah. No. I think you hit the nail on the head. The only thing other one that I'll also add is you also wanna think about them working together. I think there was another question over here around insurance will hand off between AI and human agents. Part of that also goes into when the hand off is happening from the AI to the human agent, giving context to the human agent as to what was the work that was already done by the AI agent before it was handed over to an AI agent. Oh, sorry. To the human agent. So that way the human agent knows, okay. These three things were already done. I need to do four and five without having to worry about one, two, three because the AI has already taken care of it. So that's another thing that we have in our product that allows the as the transition is happening, the human agent immediately has clear indication as to what work has already been done by the AI agent before they need to actually pick up the ticket or the conversation. And so that way, that's another way you ensure that human agents do not duplicate the work that's already been done. There's a question from, Aditya. What KPIs or metrics do you use to evaluate the success of AI enhanced customer service efforts? We have let me start over with we have a lot of data. Again, one of the the benefits of customers of platform is that we're a sec CX platform. We're also an AI company. It's not we're not looking for the different, pieces of information to key AI and where to interact and how to interject. We have all of that internal to us. We have pre built dashboards that help look at, everything from average handle time to involvement rate to responses that are AI inclusive and without AI. So you get a good understanding of the level of involvement that AI plays in that Kustomer journey, whether it be by channel, by product, by specific pieces of data that you wanna keep on the Kustomer company, those types of, you know, the data models that and part of this, we help you, with when you become a Kustomer of of Kustomer. We help you gather that data that that makes that delighted end user experience. And in doing so, we're also building those reports or, in essence, turning them on because we do a lot of it for you, on how to measure the success rates of of those, the AI agents being involved, 100% close rates versus AI inclusive versus human by itself. So there's a lot of those different metrics that we play into, to answer that. I don't think it's the same for every company at all. Again, a lot of the, you know, proof of concepts and demos that I do, every company has different KPIs that matter to them, and it's on us to figure out how we apply those different metrics and and tweak them with, you know, the dials and and knobs of sorts, if you will, to make sure that it is in the language that you need it as a as a company to make sure that those KPIs are being met or, in some cases, not being met. Right? So you know the areas to focus. Aditya, I don't know if you have anything you'd like to to add around I think I think you covered all the all the parts. Cool. Let's see. How do you train your AI agents on your corporate policies to ensure their responses are accurate? I think we covered this earlier, but I'll just re we've seen it. So the way we think about training or onboard or, setting up the AI agents is and I'll probably club this with the, with the most recent question from Banner around, are you, you know, how how do you use OpenAI, and do we use custom GPT or integrating the agent into the CRM? So both of them are sort of related. The way we use OpenAI is mostly for the LLM capabilities or the generative capabilities. We also built our our own reasoning engine on top of it, that can essentially call the LLM to say, okay. What should the you know, based on all of this context, based on this prompt, based on this training, etcetera, what's the next best thing to do? And the way that, happens so in terms of training or grounding the AI agent, which is, probably the, better word to use over here is the notion of once we have so when we're when we're configuring the AI agent or the AI agent team, you're also giving it access to your company knowledge base. You may say, do not mention x y z about competitors or do not, you know, recommend things that are outside of these policies. You will also you also have the ability to give it instructions to say, this AI agent is specialized for, you know, x use case. This agent y is specialized for, y use case, so on and so forth. And so you give all of those instructions or grounding instructions in natural language, which is then used as part of our reasoning engine in order to then, communicate with OpenAI and then say, okay. Here's all of the context that I have about the customer, the historical message, etcetera. Based on, you know, your general capabilities, OpenAI, please tell me what should the next action be or what should the next response be. And so that's sort of how we integrate with OpenAI. It's not necessarily like a custom GPT. It's almost like a a hazing engine that we have built on our end that interacts with OpenAI to take advantage of its LLM, or generative capabilities. How do you guys handle GDPR compliance around AI, especially in Europe? So we are so we are GDPR, compliant in terms of, all the requirements that GDPR itself, you know, has. We've we've been compliant from a platform side of the house as well as from an AI side of the house, and we try to take a fairly, you know, conservative approach, especially when we deal with AI in Europe and other parts of the even parts of The US where rules and rules and regulations are changing pretty dynamically at this point. And so we try to take a, you know, longer term view around, okay, this is sort of where we think the, think the legal side is moving towards, all the governments are moving towards. I try to take a more, you know, like, we try to take a more conservative stance so that we can be far more compliant than maybe, you know, but, yeah, than we need to potentially. And so, yeah, we still constantly we have a team in in house that make sure that we are staying of staying, in line with all of the policies. Anytime we build anything new, we go through what we call a trust review in, in house that goes through all of the regulations, make sure that any feature or any product that we're building goes through all of those checks and balances before we, release anything. And so that's just baked into a product development cycle to make sure that we're not just compliant with what we have right now, but we continue to stay compliant longer term. This one next one was, how do you track the volume of different inquiry types? Traditional ticket flows require users to click on present, preset inquiry categories before submitting their inquiry, whereas chatbots are more open ended or multiple types of inquiries can be asked within one chat. So a lot of this has to do with the way we capture data. Again, a lot of companies historically, focus around a ticket and the information around a ticket. We focus on the conversation. Now various parts of a conversation all come from a user or a company that has several users as an example. That data is stored not only at the company level, at the user level, also at the conversation level. So if somebody calls in or chats in or emails in and there's multiple issues in the same type of conversation, we're able to attribute all of that, at various points. Some of it automatic, some of it, you know, based off of a of a, you know, a fetch. For instance, Shopify is an integration where we're bringing all of that Shopify data into Kustomer so that when a company or Kustomer reaches out and inquires about something, we already have all that information. So if they selected, say, broken item, wrong size, wrong address, all of these different things that could play into, let's say, a retail use case, we're able to capture or better yet already have that information associated to that conversation. So it allows for a much more seamless, efficient response to that end user. And in AI, because we have our own AI looking at our data models, you're able to leverage that data in that outbound and inbound communication very seamlessly. How can AI accurately detect or can it detect and respond to a specific tone such as sarcasm, passive aggressiveness, tense politeness to distinguish between genuine and critical intent in text conversations? You wanna take that one, Aditya? Yeah. No. That's a that's a really good question. And this is sort of where the LLM comes into play, where all of these models from OpenAI, anthropic, etcetera, trained on massive massive datasets to, be able to detect exactly that. And so yeah. So that's sort of where going back to my earlier question earlier response around how we leverage our respond our AI engine along both side OpenAI. It's relying on OpenAI for these types of things where it's like saying, here's the instruction that I need the LLM to respond to. Here's the message. Here's the context, etcetera. And then the LLM leverages its own training and modeling, etcetera, to then detect, okay, if the instruction from, Kustomer of the case says, if the customers, or if the client is unhappy, then do x. Then the LLM will sort of detect that if at all the message was, was showing some sort of unhappiness, and then we'll give the, appropriate response back. And so it's all that's kinda where we where we do rely on the LLM to make the make the call. And a lot of these LLMs are pretty good about detecting that, especially as, again, as long as you're giving it all of the element context. And so this is sort of where I go back to saying, making sure that the AI has the context is super critical. It's not just about giving the AI the most recent message that the Kustomer sent in, but it's also the historical, historical thread maybe of the conversation to say, okay. Here's the entire history of the thread so far, and here's the most recent message. Now go ahead and tell me what I should be what what the what the, next action is in order to resolve the customization. So giving that context will certainly help the AI be able to detect it better and better. I think that was the last question. Alright. Well, thank you all again for your time today. We're around if you ever wanted to follow-up after this session. Both Aditya and I work in Kustomer. We're very happy to speak with anyone at any time, but thank you for spending your Tuesday morning with us. Thanks a lot. Thank you, Oona. Thank you both for joining and leading this webinar today. I'm also going to share, their LinkedIns in the chat just quickly with everybody in case you wanna connect with them there. I'm sure they'd be happy for any follow-up questions. But, yeah, we really appreciate your time and hope to see you again soon. Bye.