Name: No more data bottlenecks: How GenAI changes product analytics
Uploaded: 2025-10-15T16:31:57.466Z
Duration: 37 min 36 s
Description: No more data bottlenecks: How GenAI changes product analytics

Transcript for "No more data bottlenecks: How GenAI changes product analytics": Hi. Good to meet you all. I'm Ronnie, cofounder and CEO of ClarityQ. At ClarityQ, we've built a GenAI platform for data analytics, including product analytics. It serves as a copilot for data analysts and at the same time allows product and business users to ask questions in natural language, get accurate answers with key insights and compelling visualizations. In today's webinar, we're going to explore how GenAI is changing the way companies access and use data. We'll cover what every strong solution must include, what to expect from GenAI solutions as they evolve, and the central role analysts play in this revolution. And, of course, we'll spice it up with real world examples. So let's dive in. So this is me. I started out in consulting at BearingPoint, which used to be KPMG consulting before joining Benny Landa, the father of digital printing, where I led business development for six and a half years. The day I left Landa was the day I met my cofounder Orly, and we started our first startup, SafetyK. Typical startup journey, ups, downs, fundraising, building teams, sleepless nights, but it ended up well and we sold the company to Palo Alto based, Applovin. After that, Orly and I stated Applovin as co GMs of Applovin Israel building new products. Three and a half years later, the entrepreneurial bug hit again. You can't stop yourself when it's in your nature. And we built some products along the way, and then we realized we knew what we wanted to fix. That's when we started ClarityQ with our third cofounder at the who is our CTO. So why GenAI for analytics? We've been building products for years at Applovin and beyond. And we've always been dependent on data, whether it was understanding what worked better, analyzing user journey, or preparing slides for the board. We always needed our analysts. They were amazing, but always busy. So we constantly had to prioritize. Do you want this slide or should they keep on working on dashboards? That led to huge delays in decisions. Or worse, decisions were not based on data, but on our gut feelings. That's not the right way to work, but we didn't have a real alternative. So we asked ourselves, is it just us? So we talked to others and realized the pain was universal. We surveyed over 60 companies from many verticals, to b, b to c, and a 100% of the answers pointed out data access and hidden insights as an issue at least to some extent. So what exactly are the issues GenAI solutions are addressing? Every product company faces endless data questions on a daily or even an hourly basis. Questions such as show ARR by segment and deals and even offer a way to increase low segments ARR. Or how many users used the new feature we just spent six months developing? Analyze this AB test for me, analyze that cohort for me, and so on and so on. These questions are essential for smart decision making, but they always rely on data and analytics teams for answers. And here lies the problem. Analysts are the ones that know the date how the data is built. The meaning of in app analytics events and their parameters, they know how to query the data using SQL, and they are probably the only ones that know the bits and bytes. The small details and bugs every company had has in their data. The problem is that the analytics teams are often super busy and become bottlenecks, not because they aren't good. In fact, most of the companies we work with have great analysts, but because they're overloaded. Growth, marketing, product, and finance teams flood with questions. Sometimes to make a simple product decision, like determining how the near term roadmap should look like or which bugs to prioritize, you have to wait days or even weeks or make decisions without any supporting data. On the other hand, analysts are expected to handle dozens of ad hoc questions every day. And even building reports for repetitive questions, they still need to answer specific questions for individual stakeholders or build new products new dashboards, sorry, for new features. And if you think about it, in the past couple of years, GenAI has transformed how developers work. Every engineer today has, some kind of Copilot, such as Copilot or Cursor or Cloud Code to speed up coding, automate repetitive tasks, and boost productivity. But when it comes to data and product analytics, the story is very different. Analysts are still facing the same bottlenecks, endless ad hoc requests, repetitive queries, and the constant pressure to translate data into business insight through sometimes long complex SQL queries without a copilot to help them. And then King Jin AI with a huge promise to help solve these problems. First, democratize access to data. Data is no longer just for analysts, but also for product managers, marketing, growth teams, management, and other business users. It offers empowerment to analysts in the shape of a copilot, which help them write the long SQL queries asking the right questions and building dashboards. And it accelerates decision making by automatically providing real time insights and actionable recommendations. Now one of the first questions we always get, and I'm sure you're thinking that as well, is why can't I just use ChatGPT or CLO to generate SQL and answer my data questions? The shorter answer is general purpose models don't have the context. The longer answer, they don't understand your tables, your analytics events, your business metrics. Without that, you'll quickly run into wrong answers. And wrong answers in data mean wrong business decisions. You could try to teach a general model your context, but it's a never ending job. Every time your schema changes, every new event, every metric tweak, you'll need to redo that work, and it wouldn't be usable by non analysts since only data experts could step in and fix issues. Second, data is a world of its own. Analytics requires a data catered interface, One that supports validation, feedback loops, SQL editing, and collaboration. ChatGPT simply isn't built for those workflows. Third, accuracy. With data, hallucinations aren't just funny, and we know that hallucination is an issue. They can actually drive teams towards the wrong decisions. And finally, both your data and the Gen AI models themselves are constantly changing. Which model is best for your use case? How do you ensure alignment with your BI tools and maintain a single source of truth? If you teach how to calculate churn, but you change this definition in your BI tool a week later, you're already broken consistency, and you're in trouble. That's why specialized Gen AI solutions for data analytics are needed. They bring context, workflow, accuracy, and trust into this equation. So how does it look like? This is ClarityQ, our GenAI for analytics solution. Let's take a quick tour of what GenAI for analytics is about. Let's say you're a game publisher with millions of users and tons of product data. You've got analysts, game designer, product managers, all trying to make decisions fast. Instead of writing complex sequels or building dashboards for every single question, they just ask in clarity queue. Let's look at an example. Which countries with a thousand plus installs have high usage and low revenue per user? We added a quick time filter. All filters are total totally customizable. And ClarityQ doesn't just spit out an answer. It walks you through its entire thinking process. You see exactly how the agent plans the steps. It's looking at 1,000 plus installs, high usage and low revenue per user. And it understands that it needs to analyze install count, usage metrics and revenue metrics by country. It will go to the semantic layer for relevant models and metrics. Now, it searches your semantic catalog for the right metrics and features. It writes and sometimes rewrites the SQL and self corrects if anything goes off track, fully transparent to you. We see it now running the SQL. It's quite a long SQL, actually. And the table that we got from running it, you'll see, analysts can always tweak the underlying SQL. They can edit it, fine tune it, and rerun it. Full control, zero black boxes. All of this in real time and fully visible to you. The result, a rich visual answer. Charts, graphs, tables, and key insights. All shareable, zoomable, downloadable. Here we see the list of countries with high usage but low revenue. We see combined score of this and each one of its own with India being the best opportunity. By the way, you can even pause the process in the middle and tell it, hey, actually include only Android users and it will adjust on the fly. From here, keep the conversation going and feel free to ask something more exploratory such as anything in the data that can indicate why there is low revenue in India. Clarity queue will analyze trends, surface calm, common patterns, and even suggest actions and follow-up questions, turning one query into an insight journey. We'll get back to this question in a minute. Now let's look under the hood. We provide a table and event catalog. They're your automated, always up to date source of truth. Table catalog, hold your structured data, event catalog organizes your in app events, parameters, values. And from all that ClarityQ build your semantic catalog with your metrics, features, segments and entities. It's like giving the AI a crash course in your company's brain. Now let's go back to our session and see if our agents found out why the low revenue in India. Yes. It did. We see the eCPM gap, the in app purchase conversion. We won't dive into it, but the possibilities are endless. And this is a wrap of our demo of ClarityQ. As you could see, many of those capabilities connect back to one key ingredient, context. We've already touched on context, but it's worth emphasizing. Business context is a foundation of every successful GenAI solution for data analysts. Raw data on its own isn't enough. The model needs to understand the meaning behind it and how things are calculated in each company. For example, the term churn can mean very different things depending on the business. And I'm sure that you all feel that daily active users might be calculated differently even within the company. The same goes for revenue. It can be net. It could be something else. And it in many places that depends on customer custom business logic. Without this context, the numbers can be misleading. And when we talk about context, we're thinking about much more than just knowing the data schema. It's knowing what each table represents, what every column and parameter means, what each event stands for, and most importantly, how all these pieces connect. Together, they define entities, features, metrics, segments, and dimensions that actually matter for your company. Without the semantic layer, you'll get surface level answers. With it, you get accurate, trustworthy insights that truly reflect your business reality. To to reach a high level of context and understanding, ClarityQ relies on two fundamental building blocks, a table catalog and an event catalog that we just saw briefly in the demo. The table catalog defines the meaning of each table and column, what they represent, and how they should be used. The event catalog service serves a similar role for analytics events, describing what each event tracks, the parameters it carries, and its business need. In ClarityQ, these catalogs are generated automatically by gathering signals from multiple sources from across the company. But automation alone isn't enough. Human expertise is still essential. Analysts who know the database need to review and verify these catalogs to make sure the definitions are accurate and stay consistent over time. And we provide the tools to ensure the easiest and fastest verification process. Beyond the table and event catalog, there are several other inputs that strengthen context understanding in GenAI for analytics. A key source is the company's existing BI tools like Tableau, Looker, Power BI. These already capture metrics, definitions, and dashboards, and they're critical for maintaining a single source of truth. Another valuable source is the query history and certified sequels. They reflect how analysts have solved problems before, and they provide a practical knowledge base for the model. Context also depends on vertical. The way the gaming company structures its data is very different from SaaS platforms or ecommerce businesses, and the model needs to reflect those differences. Product documentation can also play an important role. It often includes definitions and descriptions that help enrich the semantic layer. And finally, humans are still essential. The data team has to stay involved and review, validate, and guide the model. Later on, we'll dive deeper into the human role in this loop. When we bring together the table and event catalog, along with the additional sources we just discussed, the output is a rich semantic catalog. You can think of it as the brain behind the agent. It's what allows the system to truly understand the business context. The semantic catalog holds the definitions of entities, features, metrics, segments, and dimensions, and the relationship between all of them. This is what ensures that when you ask a question, the model doesn't just look at the data, but it interprets it interprets it correctly in the context of your business. So we now know the business context is the key to making AI solutions for data analysis accurate. But this is important. Validating that context and keeping it up to date still requires the involvement of the data team. AI can help exact extract context and map it into business metrics, but every company's data is unique with its own quirks and specifics. Analysts are the ones who hold the map. They know how the data is actually built, the SQL underneath the details of the tables and events, and they also see they know the business meaning behind metrics and the company's unique definition. And this is why AI based data solutions rely on the data team for as the entry point. Analysts use the solution as their copilot, helping them write complex SQLs, build dashboards, and surface insights and accrumentations. And as they do that, they're naturally also checking accuracy, giving feedback, and fist fixing mistakes. In many ways, they become the managers of the AI agent. Think of an AI tool for legal teams. The the lawyers are lucky. The very people using the system are also the ones qualified to just whether an answer is correct or not. In data analytics, it's not like that. Business users don't know the data or the sequels well enough to validate answers. So we can't just hand the agent over to them. We need to we need the analyst to use it first, guide it, correct it, and only once they feel comfortable, give it the stamp of approval so it can be widely used by product and business users. And the same process repeats every time the data changes. A new feature, a new Tiva analyst test and validate as part of their ongoing daily work with a a the agent as their copilot. They make adjustments, fix on the goes, so it doesn't become a burden, but rather a natural part of their workflow before the solution runs out rolls out more widely. That's how the agent improves and how it scales responsibly. And analysts in the old world, much of their time was spent on ad hoc queries and routine dashboard building. In the new world with AI handling many of those repetitive tasks and business users can act access the data independently, analysts can focus on higher value work, deep research, AB testing, and the emergent emerging role of supervising AI. And across all these areas, AI makes them faster, sharper, and more efficient. So how does a strong AI solution support this new analyst role? First, it should build and update the semantics as automatically as possible and its scale. Ideally, analysts only need to step in to verify the AI's understanding. Second, it gives analysts simple, intuitive tools to guide, update, and verify the agent when needed. And third, it should make ongoing updates as seamless and automated as possible, so analysts can quickly adjust context and logic with whenever the business evolves. In short, AI takes care of the heavy lifting that analysts remain firmly in the driver's seat. After working closely with product companies over the years, we've learned some important lessons, and these have really shaped both our road map and our domain understanding. Firstly, users will ask anything from obvious questions to very specific requests like changing a chart color to match the company's logo. Second, there is no forgiveness for inaccuracy. Unlike ChargeePT that has here and there wrong answers, there is no, forgiveness for inaccuracy with data. It immediately breaks trust. The burden of proof is on the AI solution. Third, we just discussed the data team has to be on board and use the solution themselves. Without their involvement, it simply doesn't work. Fourth, no one likes onboarding. Literally, every single deal starts with the question, how complicated is ClarityQ onboarding? And finally, explainability and transparency are nonnegotiable. People need to see how the answers were reached and to even be able to start a conversation about it. Without that, they won't rely on the answers and the overall solution. The natural net question is how can a good AI solution address these takeaways? We said that users will ask anything. The solution needs to go beyond just SQL generation. It has to support a wide variety of age of questions. Some might be really unexplained unexpected. Sorry. Because there's no forgiveness for inaccuracies, the the system must break down the complex questions into sub questions, always examining the integrity of the data handed. And, of course, as we discussed before, the solution should rely on semantic understanding of the data to deliver correct answer. Third, as already mentioned, we know the data team has to be on board. So the solution should provide dedicated tools and features that help analysts guide, verify, and refine the AI. And, yes, nobody likes onboarding. This means that semantic catalog needs to be built as automatically as possible, minimizing setup work. And finally, explainability and transparency are a must. The solution should present the reasoning process and the logic step by step, allowing people to refine and adjust it on the way. In short, every one of these lessons translates directly to design principles for strong Gen AI solutions. So where do we actually see users getting the most value from Gen AI analytics? These are the common areas, revenue optimization, return retention and churn, feature adoption, user acquisition, and audience analysis, cohort analysis, funnel analysis, AB testing, and analyzing user journey. But it's important to say these are just examples. The real value of GenAI is that it's flexible. Teams can apply it on a wide variety of questions across product, marketing, revenue, and growth. Let's see another example in action. Here's a k gaming company that uses ClarityQ to analyzes to analyze their AB test. In the AB test, they have something called the story event that represents a special high impact gameplay moment. And they wanted to find the optimal interval between these events. Is it every one and a half minute, every five minutes, every fifteen minutes? And the hypothesis was that longer interval will push users for in app purchases. So the the primary, metric that they were following was the average revenue per user, ARPU. So they were asking, can you show me the average revenue per user, the ad ARPU, and the in app purchase per test group? Now we see the overall ARPU is higher in both test groups, but more so in group b that that's fifteen minutes apart between intervals. So they wanted to check a secondary metric, first time deposits. And we see that in group b, first time deposits also performed better. The percent I mean, of first time deposits and the period per cohort. And they asked, what about the average runs per user for these cohorts in the period? K. They wanted to make sure that no other metric got hurt. So the end of purchase went higher, the ad revenue per user went higher, and the average could show the d one and d three retention per cohort. They wanted to know that they didn't their retention wasn't hurt based on that. And we see actually that the retention is also overall better in both test groups. So the conclusion was that group b, the fifteen minutes, is the winner in iOS. So they changed their game to have the star event every fifteen minutes, and the user will therefore spend more money on in app purchases. And this is only one of many analysis they do on a daily basis using our solution. So, sorry, so after everything we've discussed, the question becomes, what should you really pay attention when choosing a JNI solution for analytics? There there are a few musts. Accuracy, of course, context understanding, so solutions actually grasp your data and business logic, adaptability to data changes since your data is never static, full transparency and explainability. You need to see why the AI gave a certain answer, tools for data teams to control the semantics, and, of course, privacy and security. Then there are key differentiators, the things that separate one solution from another. Ease for onboarding, aligning with your existing BI tool, the coverage and depth of questions it can handle, rich visualization, a strong feature set like a feedback loop, insights, recommendations, and dashboards, and finally, performance and speed. These are the pillars to look for when evaluating Gen AI solutions. They're what determines whether the tool will really develop deliver value across your organization. To wrap up, GenAI is a real game changer. It puts data directly in the hands of product managers and business users anytime, anywhere. But it only works if it's grounded in evolving data structures, semantics, and logic. Without the foundation, it quickly becomes unreliable. That's why analysts remain so critical in the new AI age. They're not being replaced. They're shaping, supervising, and guiding the AI to make it trustworthy and impactful. Tomorrow's winners will be the companies where analysts and AI truly work together, freeing up time for deeper deeper, more strategic analysis. We we're all living through a fascinating shift in data analytics right now, and it raises an exciting question. What might data analytics look like a year from now or two years from now? I hope you found this session interesting, and we're now, happy to answer any question you might have asked during the session or right now. Thank you. Hi. So sorry. Sorry. Sorry. Hi. So I'm happy to introduce you now to Orly Shoavi, my cofounder, which I mentioned at the beginning. And we're happy to answer any questions you might have. And I see a few of them here already during the session. Looks great. So you don't try to replace analysts. No. Not at all. As Roni mentioned, we're actually empowering analysts to be, like, four or even five times more efficient. We provide them a, help with the, with the generating complex SQL. No more writing SQL manually and much more faster. We provide them a way to get valuable insights really in minutes and also recommendations. And on the other hand, we also allow business users to ask data collection in natural language, which means the analysts don't need to, answer, questions coming from across the companies all the time. And they have more time, to work on more valuable tasks such as deep research and AB testing, predictive analytics, etcetera. So they're actually even more valuable and efficient than before. And also for the parts that's, addresses the business users, as Roni mentioned time of the time in the in the presentation, analytics, they also play also very important role. They are the ones that makes the make sure the AI agent is accurate, is updated. So our solution indeed changes the way the role of analytics, but definitely, they still stay super significant and the role is still very, very central in the in the data process of the company. More questions. How do your solutions like you ensure this and evaluate your, your own, I guess, our own accuracy? That's super, important questions that we deal with it a lot. This is something that our, I think many of our team of our team, this is exactly what they are, working on on a daily basis. For every product or every customer that we have, we generate a a very thorough extensive datasets. We run evaluations to make sure that our agents, answer accurately on each and every question. So we also compare it to the existing BI tool of the of the company. If they use Tableau or Luther or Power BI, they already have that with the earlier metrics, and we can run our agents with the same metrics and make sure that we are, answering exactly the same, we're accurate. We also have a built in feedback group where analysts can, fix, verify, align, refine the the SQL, and our agent actually learns from, its mistakes. So we ensure, that we are never gonna do do the same mistake twice. Also, if you remember then all of our accuracy is actually built on the semantic catalog, which is something that we, invest a lot in. We build it from the the ITUs and the the query history, and, actually the jargon and the internal terminology of the company. And we never let business user use the agent and and the semantic catalog without the approval, of the analytics and the data team. So we always have, we always make sure the semantic catalog is super accurate so the agent won't make any mistakes. Let's see. Thanks for the demo. We have really sensitive data about our users. How do you handle this? Also important questions. We do take it very, very seriously with the third cofounder, you see Roni. He's coming from a cybersecurity, domain, so it's super important for us as well. Of course, we are SOC two. We have GDPR compliance. We always ensure that, that we are with the the most strict standards. And we are, you can as as our customer, you can control actually which data are we accessing. You can provide us only access to the tables that you want Clarit Keith to, actually, answer questions about. And, also, we only require read only access. We never write or store anything or copy anything. It stays on the data warehouse of the customer, and we connect to that to to the to the data warehouse. You know, we don't, like I said before, and answer questions from there. I'll take the next one. Yeah. Sounds good. How much time does it take to onboard ClarityQ? So, we take typically, it takes us between two to three weeks from kickoff to going live. So what we do and Orly just mentioned that we do only we need only read only access to your data warehouse. And during the POC and the POC prep, most of the heavy lifting is on us. We do the work to understand your data structure and create a valid semantic catalog. Anything like we just mentioned, anything, connection to your BI tools and your query history, that's super helpful. But, your team's main job is only to review the catalogs that we create and validate that we've understood things correctly. So and then we set up a thirty minute weekly call just to see that we're on the right track. And once the POC is prep is complete, you get access to the product and can start asking any question you want on your data. This the system is already trained on your specific data model, so it's ready to use from day one. Let's see. Another question here. I think we have another one. Which one? How do you handle non events or changing events? Oh, sure. That is a great question, and we get that quite a lot. We automatically scan for changes in your events on a daily basis. We detect both new events and events that have disappeared. The second part is really important because sometimes and I'm sure you know that if suddenly an event disappeared means that there is some kind of bug in your tracking implementation. It's an early warning sign. So when we find changes, we automatically update your catalog and flag anything that needs review. You'll get notified about what changed so nothing slips through the cracks. The key here is that analysts can approve or adjust these changes directly. There's no more manual redocumentation every time a column changes or new event gets added. You're not stuck maintaining spreadsheets or notion docs that are dated the moment you publish them. I think I have here one more. Can ClarityQ learn my products terms? Absolutely. ClarityQ automatically learns and understand their product's unique language. We talked before about churn and daily active user. So even if it's something like you have a way that you call, you segment your users as good users. We have a customer that has it and it's thirty minutes, on day one. So ClarityQ learns this language and understands that every time somebody in this company says good user, they mean users that on the first day spend at least thirty minutes, in the game. In this way, we can, give you answers the way you'd like it. No need to think about the wording. Just ask in plain language, and we will know the intent of your data labels or SQL structure. Bring it on. Any more questions? Yep. I think they're down here. Yeah. So thank you very much for your time. It was great showing you, how we look at the world of Gen AI for analytics. Our emails are rony@clarityq.aiandorley. @clarityq.ai. If you have any questions, I'm happy to connect. Thank you. To see this stage.