Transcript for "The AI role play that closed $1M": Hi, everyone. My name is Martina, and I'm a community manager here at the Sales Enablement Collective. It's so great to have you here at our webinar today. Feel free to use the chat to introduce yourself and let us know where you are tuning from. Also, feel free to use the q and a function to ask our lovely speakers any questions you might have throughout the webinar. I will now hand it over to Todd, principal value engineer at the great. Over to you, Todd. Thanks very much, Martina, and welcome everybody. I see we have a very global crowd today. That's super cool. I'm really excited today to talk about the amazing success that the TECHS systems team has had, recently with their enablement program. So I'll just jump right in. I lead the value engineering practice here at Degreed. I've been with the company for a long time, bounced around a bunch of different teams, but currently work alongside our solutions engineers. So I live very much in, what I hope is the day to day for many of you. And I'm really excited today to be presenting alongside Chris and Steph from TECsystems. As I mentioned, I've been at Degreed a long time. Chris and Steph were some of our earliest customers. And the thing that I think is most interesting and most relevant here is just, the approach that they take to their work. Chris and and their and Steph and their team have been the first ones to stick their hands up every time something new comes out to try new things, not just with us, but with a lot of their partners. They're really great partners to work with. But more importantly, I think they really embody the spirit of what Degreed is all about. And so if you're not familiar with Degreed, we are a skills transformation engine. We, I think, are different than a lot of other tools out on the market just because what we, what we see as the sort of primary problem is a little bit different. We see and Chris and I think Steph will have a great story to sort of, explain this. The real problem that a lot of companies are dealing with now is the change happens so constantly and the changes are coming so quickly that just one solution, you know, one enablement program, one onboarding program isn't really getting the job done for a lot of companies anymore. Need to be thinking much more systematically about how do we keep people moving forward as they come into the company, moving through their career. And Chris and Steph have been really forward thinking, in the way that they architect their enablement programs. And so I I know you all wanna hear the story, so we're gonna jump right in. But, Chris, Steph, maybe you could just introduce yourselves, and then we'll get on with the the story. Absolutely. Good morning, everyone. Todd, first, thanks for the, the opportunity to sit and chat a little bit more about something that we did at Lens, because we've learned a lot more since that. But, I've been with Allegis Group and Tech Systems for twenty six years, and I primarily focus, in the OD and talent space. And I get to oversee all of TEC Systems development environments and advise the other six brands that we have as as an Allegious family, and I'll get to dig into that a little bit more. But, Steph, why don't you go ahead and and and introduce yourself as well? Yeah. Thank you so much, Todd. And, I'm Stephanie Coon. I'm one of the senior learning leaders within TEC systems. I work directly with Chris Harry, and I help bring all of that to life. So one of the key functions of our team and what we support is how can we help with sales enablement and, scaling seller readiness. So looking forward to talking with you all today about how we did that. Great. So why don't we go ahead and jump in? And and, Todd, you you had shared a slide that's that showed, changes is different thinking differently about change, and and I'll jump in here and give you, some context. But in that thinking about change differently, one of the things for us is that, we have a mindset that we will continue to experiment and change and adapt, continuously. And that is our form of change moving forward. And we are trying to mimic someone's constant use, even, like, similarly with a phone, that it it's going to move and change, and we're gonna do this together. And we'll get down this road in the conversation about Maestro. And this is just one of the pieces, in our tech stack that has started to show, some really, some really strong impact that we're now measuring much more than we were when we were at Lens. But to give you some context about Allegis, Allegis is seven specific operating go to market brands. And across the sales workforce, there are 4,000 global sellers. But for today, we'll talk about tech systems. We have 1,500 specific sellers. 500 of those sellers are from about six to seven months up to about two years. And then we have the group that we'll focus on today, and we call them SDRs. And they come to us, directly out of college and or, about a year or two in sales, but very, very new into a sales career. And they're about 350 of them, and we have a very, very strong focus on them to get them skilled and moved on very, very quickly, prior to their six month change into their first tenured, or I should say titled role as an account manager. So let me give you a, a little bit more context. As we started to talk about Maestro, which at this point is now almost May or June of last year, we had a very specific opportunity that, had been a challenge for us for quite a while. We have about a 100 locations in The US, and we have numerous others in in in 17 different countries. And a sales leader in our organization in Allegiance possibly could have 15 to 20 or more direct reports when you think about a sales group in a in a location. And that presents a a very specific problem for us around when you think about the coaching around sales and what we call live deal coaching. It is very difficult to to get a seller paired with a leader on a very consistent basis, which creates possibly a lack of skill development in the early parts of their career. So when we took that job and looked at that job and we said, well, the highest value activity is the notion of deal coaching. Enter Maestro where you actually then can think about efficiency, repeatability, and consistency of the environment which you are building skill. And then I think the third piece to this is scale success. We have we have goals to shoot 75 to 80% or higher for all those 350 people that come into our company to get successful prior to six months to get them on the right path, get them to quota, and get them moving on because we know that we have a great opportunity if we can do that sooner versus later. A little more context for the conversation is the idea of our sales process. We have a standard pretty much a standard sales process. It's seven steps. The high value activity at the top of the funnel prospect, interest, and qualify is where you can all imagine. This is high reps, high contact with leadership, high coaching. And when we go back to that idea of of high value activity, we had to figure out the recipe for where do we get our leaders' time just before the person has to pick up the phone and make the first phone call. I need as much time for my leadership to get in the weeds, talking business, talking about the play, talking about the value message that they're going to speak directly to that customer about for a particular opportunity or just the prospect in and of itself. So what do we do all prior to that? And that is that repeatable, efficient, very directive environment that we have created in Maestro. And and I think it's important that Steph go through, and we're gonna see what she's gonna do now, but I want her to go ahead and jump in and give you some ideas about the the topics, where we place them, and how they are playing out thus far. And I think the one thing that I would point out to you as you're thinking about this, the notion of intelligent design and placement throughout your process is critical. So, Steph, let me go ahead and turn it over to you. Thank you so much. So what did we do? So we had a cohort of approximately 22 new sellers. Chris, mentioned earlier that they are SDRs, so they're sales development reps. Their onboarding program is approximately twenty weeks. They're led by a dedicated facilitator within OD and they do a variety of things, whether it is group discussions, virtual, instructional led classes. And we were really looking at where can we be intentional with the core skills of a salesperson that we know that if they can really get the opportunity to practice often and be able to demonstrate proficiency, that's gonna help them be successful when they get into the real world post onboarding, and they're actually executing on those specific tasks in their territory. So we started small, and we went with four intentional skills based maestros that we broke them down into very specific, skills that we wanted to measure. So as you see on the screen, the first was the elevator pitch. The ability for our seller to demonstrate who they are, what they do, and what they support is really critical in being able to show up first to the customer, whether that's in person or on the phone. We then looked at measuring how they would overcome objections. So can you handle that first pushback that most buyers would give you, which could be something like, I don't have time to speak today. I'm not interested. I'm already engaged with another, you know, firm that is helping us. How are you going to overcome those objections? The next would be discovery questions. What are you going to ask? How are you going to uncover the key challenges and pain points, the desired future state of our buyers? So that way, you can properly position an effective solution that meets their needs. But in order to be able to do that, you have to put all of these core skills together, and that's where we ended with a full prospecting call. So are they able to demonstrate and explain who they are and what they do? Are they then able to ask those questions? Are they able to be ready to field the core challenges that they typically get from those buyers? And we found that by enabling by enabling the work force to focus on very micro specific skills, it allowed us to hone in on where there might be gaps or where there might be challenges for them to overcome as they go through their learning. And so what were the results? So it allowed us to move business forward. Not only were the reps able to increase their confidence, we got a lot of that anecdotal feedback. Right? Maestro was a great tool to use. It allowed me to practice at the point of need when I was learning through my onboarding program. I didn't sort of have that big brother feel where my leader was looking over my shoulder. I could do it as often and as frequent as I wanted. I felt safe in that environment. So we got a lot of really great feedback. But aside from the increase in confidence, which is just as important, as moving the business forward and gathering those performance metrics, Where our organization has been heading in the last few years is we want to be really great at providing services to our buyers. So in order for us to do that, we need our sellers to be able to get that meeting with that buyer so that that they can continue that relationship. And we found 25% of that cohort were able to get a qualified meeting with a c suite level or director plus within the first six months of their job. That is something that we haven't been able to capture, and for them to be able to get that meeting is incredible and I think is a direct reflection, not just on Maestro, but the entire learning experience of a cohort based model in Degreed coupled with the practice and repetition that they get from Maestro. Now here's where we've been able to Chris's point, we've been able to even, assess this further since we, had this presentation earlier this year. We've been monitoring this cohort really closely. And with our services deals, we find that the sooner you can be successful, the more likely you are to, one, obviously have that confidence, but you are also going to be retained, Meaning, we are going to reduce the attrition that we're seeing within our sales force because, obviously, they're becoming successful. We were able to, pull data that showed that two sellers were able to land a deal and close two deals totaling over 1,000,000 plus in revenue. And with our goal of getting to 10, billion as an organization by 2030, we need to have opportunity in our pipeline, and we need to be able to move business through what Chris mentioned earlier is that sales process that we all experienced from the top of the funnel to the bottom of the funnel. But, Chris, can you share and provide a little bit more context on that 1,000,000 and some of the other performance metrics that you've seen, since Degreed and speaking at the LENS conference? Absolutely. I think the one thing that I would point out with this million dollars is it's two specific deals, but the the most important part of the one deal, the larger of the two, about 600,000 of this this revenue is it was a cold account, and it had been cold for years. We made an internal referral to the CIO of that company, and this particular SDR, along with some support in the office back to that live deal coaching, made a direct call into a CIO. And this is a person who's less than six months in our company, and was able to get the meeting. And what what we're looking for now is is that when we look and add more cohorts in, so we had to get to, February of this year to now start adding additional groups in. So we actually have three cohorts now in the measurement model. And and as I was was I'd said to to Steph is that what we're seeing is that in those other two cohorts, more than 60% of each of those cohorts, this is almost another 30 people together, are getting meetings with c and v level individuals and talking about services business. And in the last month, we have 14 deals that are sitting at Qualify in that same two cohorts, which, when we look back over the past three to five years, those time frames are unheard of. And I think to Steph's point, the notion of this cohort based, very focused and very pointed way of practicing along with the live deal coaching has has really changed the game. And I think as we look and design and put more things more things in place, will just amplify that. A couple of other things data wise when we look at these really important, deal velocity type metrics in that process, the amount of outreach is three and four times what we used to see two to three years ago from our SDR cohort. We're averaging almost 80 outreaches a week in some cases, for for our SDRs. The amount of, spread, which is our particular on our talent business, one of the major metrics, 70% of each of those cohorts has put spread on the board for themselves prior to six months. That was taking, in many cases, thirty six to fifty weeks in the past. And what we know about an engaged workforce, at least for us, is that the sooner we can put those things on the board for them, one, the confidence comes. Two, they're actually earning commissions and work past six months that, we've not seen. So we know that we're having a much higher engagement rate, and the the the feedback, the the, summative feedback that we're getting on these processes is something, again, we've not seen before. So we know that we are firing on the right cylinder of the mix of things that we're using, to Todd's point about looking at change differently. So I'd share those things, Steph. We know it's really, working and is a additive tool and benefit to, you know, what we've been doing in our SDR onboarding program, but we don't wanna stop there. And we've, since degree lens earlier this year, we've also launched four additional role plays to our talent delivery team. So these are recruiters who work really hard to find qualified candidates to fill our business, whether that is a one to one where we need to have a talent person sit on-site at a customer base, or maybe it's a team of consultants that they go and they actually fulfill a required service deal that those sellers are working so hard to get. We want to offer the same opportunity, not just to our salespeople, but also to our recruiters because they need to be able to do, core fundamental recruiting, skills and be able to demonstrate proficiency. And so we've launched them, and we're currently monitoring their and their, adoption of that Maestro tool. We don't wanna also forget about our leaders. So we have a leadership academy within our Degreed platform. You have several thousand leaders in our organization, and their ability to be able to effectively coach not just to our sales people, but to delivery and then just general soft skills when it comes to leadership. Think of things like performance feedback conversations. How do you effectively have a performance feedback conversation or a tough conversation with your direct report? And what do you say? What do you do? How do you, adhere to the core values and principles that we'd set forth within our leadership academy? We're looking at expanding and scaling there. We also offer a badging and credentialing program within our sales and delivery, programs. And right now, we rely heavily on our leaders' ability to participate in a role play that is done in person where we have rubrics that are built out and they're scored. But imagine a world where we could take Maestro and we can offer that that seller and that recruiter the opportunity to practice before they get into that real lifetime role play with their leader. We're looking at really critical opportunities where we can have that seller and delivery person practice before they actually, you know, get into the game and, try to earn their badge to verify their ability. Our back office is going through a major transformation right now. We're switching over to Oracle Fusion. When you think about even our back office and their ability to have conversations with contracts, to work collaboratively within the internal workforce, we wanna look at all these great opportunities that we feel will just represent our business better and help our people do better in their job. And so the vision is continuous learning, not just onboarding, but where are the high value opportunities where we find gaps, but we also wanna verify people's ability, across our entire organization. We also wanna focus in on the critical roles within the four to, four years to eighteen months. So hyperskilling. Not only onboarding, but how do you effectively sell and market our products? What do you do when they change? So when you think about your organizations, how can you create Maestros that allow your sellers, your, business development manage managers to go to market, effectively showcase what it is that you do? And then, eventually, we'd like to scale it across to some of our sister companies, as we are one Allegis, and Degreed is a platform that multiple operating companies within our organization use today. We wanna scale that across to them as well. So lessons learned. Right? We didn't come out of the gate perfect, and I hope to share some of these challenges that we face so that, you know, if your organization's currently using Maestro or you, are really engaged in using Maestro in the future, we found that starting small was really critical. In the beginning, we tried to layer a variety of skills within a Maestro, and it's not Maestro's fault. It wasn't the tool and the technology. It just made it really difficult to keep the conversation going. And so I found that starting small, being really intentional with the core skills or the core concepts that you're looking to put together, one skill per experience is really gonna be able to help you. And we also found that this wasn't necessarily going to replace our human connection and the leader's responsibility to engage with their team member. So this was a complimentary tool, meaning I want you to also engage in Maestro, but I also want you to have the conversation with your leader because that human interaction matters just as as much as that tool does. Building governance early. So our team, small and mighty, but we established governance. And that governance was who was going to be able to design the Maestro, how is it going to be designed, and creating standard operating procedures. So that way there's consistency within our Maestros. We're a pretty big team within OD. It's about 60 people, variety of levels. We're all not instructional designers. So having that governance was really important to ensure that people are adhering to those guidelines, and you're creating really well thought out maestros. And to Chris' point earlier, we definitely did adjust, and we test, and we repeated that process across the entire, pilot duration. And so not only were we just launching to those sellers, but we also launched internally to say, mister and missus Smee, how can you help us iterate on this? Are we going down the right path? And we did a lot of trial and error, and Degreed was a really, you know, phenomenal partner in helping us test out those features and functionalities. So, Steph, I this, this last quote, if anybody has read the book Power of Moments by the Heath brothers, I I I have read that book once a year for probably the last good seven or eight years, and I think there's some really interesting kind of thoughts in there. But but when you think about any of our employees for all of us on this call today is, you know, this idea of we don't have to leave defining moments, in their development, their engagement, the culture, whatever those are. We don't have to leave them to chance. And I think that's what, we've with Degreed's partnership have really, really pushed, the envelope on these richer experiences. And I would say that, you know, trying to be almost a frontier partner is is a really kind of interesting concept, and, we couldn't be happier where we are this far, and and we look to kind of push it on. So so, Todd, I'll I'll give it back to you at this point, but, we appreciate, again, appreciate the opportunity to share this stuff. Thank you both. We're gonna get to some questions in just a second. Martina, we've got a poll for the audience. But while Martina's pulling that up, one of the questions that popped up in the, chat while you guys were talking was from a a Cody in the audience, and he was asking specifically about, how this all fits together in your heads with, all of the other tools that sellers have at their fingertips these days. Right? And that's a combination of the sales tools like Salesforce and Seismic and Gong, but also the newer AI tools like Cloud or Glean or Gemini that that a lot of companies are rolling out. How do you see putting all of these things together? Right? I I think it ties back to the the idea that Steph talked about around continuous learning. But, Chris, you had some really, sort of clear ideas here that I think would be useful for people to see. How are you putting these pieces together? Yeah. Okay. Yeah. I I got a couple thoughts here, and I'm gonna use two terms for for me, and and they are not industry terms by any mean, but the the the notion of embedded and active. When we think about the AI technologies and the stack in and of itself, there is the notion of there is all sorts of of new enhancements and things that are being placed in agents and and bots and things around the stack. Salesforce, Gong, Seismic, you mentioned those. We have those. And I can give you kind of an idea of a of a quick design of one. So sales leader so prior to a sales leader getting engaged on a prospect or an opportunity, the seller has been working in in Maestro extensively. There comes a time when they sit down with their leader to do deal coaching. The leader will have an agent that will deal with pipeline insights through Salesforce. That will kind of give the structure to the conversation, will give the data necessary to start looking at the play that the seller is trying to run, the messaging that's going on, and medic, is our process within there. They'll they'll go through all of that. At that point, there is another tool that is built on Gemini that actually helps the seller and the leader pick the exact playbook for the prospect and the opportunity in Seismic. Seismic sits on top and embedded in Salesforce. That will structure a process that we are moving into Seismic with on another AI technology that will role play specifically the playbook against the opportunity. So now a different layer of specificity. Once they go through the first call with the customer, which is recorded in Gong, so we have Gong, the Gong recording, we actually will have set up with a rubric to know, did they cover everything in the value message? Did we hit all the points? And where did we fall short? Then that kind of creates this three sixty view of the process in the top of the funnel for the leader. And that we're trying to get very well oiled, like just a a consistent machine that that gets us into repetition because we know it's high rep and we know that it has to be exact to be really well understood by the customer. When you talk about the notion of active, active is hands on board with other types of technologies, copilot. We have an internal leadership coach that we're using. We are actually right now in the process of kicking off, some new projects that will define pure task, skill, continuity across different roles with by just title. That's all that we're gonna do. We're gonna look at task and skill continuity, and where might we shape some different work, whether it be in sales, delivery, ops, corporate, whatever. We're trying to figure those groups out right now to start small again, and pick out the roles, how will we shape, and then what do we go back and do with the human so we have an augmented human, that is using all of these different technologies. But they are mindfully placed and minimalistic in their nature because we also know that we have more technology now than we've ever had, and we can outkick the human's ability to deal with it first and foremost. And the other thing that we're really concerned about is skill regression Is if we constantly serve up everything to the person and they have nothing to think about, nothing to use inference, decision making, and they're just meant to essentially read off a script, kick back a card to a customer, we lose every day. There's no chance that the sale is too nuanced. And I think that we have to really think about some of those things as we continually add, and we have great partners in sales ops. We have great partners in sales leadership that are right on board with us that we are gonna go a very particular speed, implement on both the embedded and active side, and be very conscious of our people and their experience, internally to the external environment. I think it says a lot that the comment that you made, Chris, that got the most, hand clap icons was, a second ago when you talked about the skill regression. The I think that's something that a lot of people are sort of running into right now, in. this movement. We've dumped a lot of the these tools on people, and now we're seeing, sort of some of the the downsides. So we'll come back to that in a minute in the q and a. For the audience, if you have questions, you can drop them into the the q and a. We'll get to those in just a minute. I've got a few questions to get us going. But before we get there, Martina, if you could bring up the audience poll. Just gonna do one quick question. So the question for you all is just Chris and Steph talked about, specifically an onboarding use case. But what is your CRO's biggest talent priority? Is it that faster ramp for onboarding? Is it driving higher performance for people who are already ramped? Is it driving career mobility and leadership development for people to give them that career path? Or is it some retention? Right? So we already have a few votes coming in. We'll give it another sixty seconds or so for a few more. It looks like higher performance. Not a big shock if I listen to our CRO as well. But I think, you know, as as the audience is is filling this in, I, you know, I asked that question just a minute ago, because I think your answer, Chris, really illustrates this systematic approach, to continuous learning along the entire employee life cycle of a seller, not just at one specific moment here, another moment, some other time. So I thought that was really interesting, and I think the fact that you guys put these moments together, is just a a a really novel and and and different approach to what we typically see in in a lot of our clients. So looks like we're not getting any more answers. Performance wins, not a big shocker there, but, we'll move on to the the q and a. So my first question, I think, is for for Steph. You mentioned that your team is small and mighty. I remember at Lens, you talked about, the ability to sort of create some of these things with, without a lot of engineering resources, which wouldn't have been the case a couple of years ago if you wanted to do something like this. So, I think a lot of people in the audience have small and mighty teams for enablement. What does the workload look like to deliver something like this for you guys, and what does that you know, how is that any different than the way things used to work for you guys? What's the the transformation that's happened for you guys? Yeah. So our team, I have a core group of learning experience designers who are strategically aligned to our sales and our delivery onboarding programs. So not only do they know, you know, adult learning, but they also know the programs pretty well. And they work hand in hand with our directors and program managers, to ensure that what we're building is aligned and supporting the core behaviors and skills. I will say, we are not role play prompting experts by any means. This has been completely self taught with the partnership of Degreed. Todd, your team has been phenomenal, and I'm not just sort of tooting their horn here. Like, the ability for them to walk us through what the role play capabilities are, how to strategically set them up. We intuitively then took that information, applied it to what our design principles are, and we were able to just iterate and practice. I will say the platform is extremely intuitive. It is user friendly, and we found through trial and error and working with Degreed, you know, what's the best way to set up the role plays so that the conversation continues to flow, freely. And we ultimately get the learner through the role play so that way we can score and we can look at where they did well and where there's still opportunity. And so it was about four of us. And I would say the workload in the beginning was just getting to know the tool, exploring, practicing, spending a couple hours a week doing that. And we went with things that we thought and felt that we knew the best. Right? We didn't try to tackle the most complex, the most difficult skill. And that's why I always say start small because you have one time to make a really good first impression, and we wanted to go with something that we felt really great with but also was very monumental in the seller's onboarding program to be able to capitalize on. And so we found that, hey. Let's go with the core critical skills. What do our facilitators see in the classroom as well as during the onboarding program that their sellers are struggling with? And then what does that do from a, organization perspective in what do we need to align to from the highest level of our organization to align to the strategic goals. And that was really the capability of building up the top of the funnel skills, and that starts in onboarding and it starts at prospecting. So it was a little bit of a no brainer. But what I would say is the other designers that are in our department, it's about 12 to 15 of them. By having someone take the lead on what to do, what this is, and what it's not, and building out the standard operating procedures coupled with, Todd, your team, and their ability with Brad to help us onboard, that's been monumental in helping us get upskilled. And I would say it's pretty simple. I think it's just what's your use case, what problem are you trying to solve, and that's where you gotta start, and all you need to do is start with one. Great. Thanks thanks very much. Chris, my next question is for you. You know, Steph just described kind of how her team is operating with a new set of tools. But I think underneath that is the bigger picture of what we're reading about in the news every day, which is that work in general is being redesigned. And when you read that in the newspapers, it's typically, you know, in a really sort of generic way. But how specifically is the work of your team in OD and learning enablement, talent development? How are the roles changing? How is some of the the work changing? And, specifically, for for for the way that you guys operate. Yeah. I I would give you a a very clear example of some change is is our five we have five major, director level roles across the organization. One is this field sales leader that we've been talking about in this deal coaching connection. But then we have MSP and enterprise and national and services directors. And what I would say is is with all of this, injection of new technology and what is being automated, what is being served up, new dashboards, work that's already started to become a bit more autonomous and served up, We have actually gone back to the to the drawing board on all five of those roles. We actually just finished a major design session, a true OD work design session with some tools in mind and actual outcomes that the chief revenue officer is looking at very differently moving forward. And then how do you cascade those accountabilities and responsibilities? And where does the the the the, the fabric or the net get tossed across the entire organization. So we don't have duplicate. We don't have, sometimes, in opposing view of work. That is getting very, very shaped in a much different way now. So we now have five new role descriptions for those jobs, very specific tasks, very specific accountabilities, and a third section that is, I need this from my partner of this job. So each of these role things now has a new success profile. We call them successor profile. At the very bottom, there is a section that has the other four director jobs listed and the very specific accountability to what that role needs from the other four. We never had that before, and you will find things from a technology for technology perspective in there. You will find very specific tasks around things like market and territory planning that was not there before. So I think we're becoming as I had said earlier, this idea of task and skill continuums across environments, is becoming you know, that's just a word for myself. But it's really this interconnectedness of the matrix is becoming so important. And then what sources of truth are we using to know that the business is moving is now becoming a really big focus around whether they be agents that are serving this into one set of dashboards, and I always always kid them about it. It's like, you know, you need your cockpit of your jet fighter, and you have all your instruments. And whether it's cloudy or not, you can still fly because you have all of your data in front of you. And I think that that is becoming a really interesting new frontier. But as I mentioned earlier, we are now at the Allegiance level getting ready to kick off some very, very big work with Copilot and some other internal tools around entire parts of the workforce that will really change, I do believe, some of the core tasks that can be, like I said, either automated or shifted to use technology to augment the human. And I want to be very clear. When I say augment, I truly mean augment. I do not mean replace and I do not mean loss of job or anything like that. We were certainly looking at an augmented future, and we will have a balance of things that are autonomous that we can go do that way, and that's actually the right business model for that. So, that's kicking off very shortly. And we're excited about that. But that that, again, is another new frontier that we will get into this iterative testing and and cycling, of of of a bit of, you know, spending to to to get some things done. So So what's the advice that you guys are giving to your teams about sort of how they stay relevant as the job market for enablement people evolves? Right? How do you, make sure you're on the augmented side of that equation in two or three years when when things are very different? Steph, you wanna take that? Yeah. Try it. Right? So one of the, you know, one of the things that, I think is really great under Chris's leadership is, the the willingness to try new things. And when you look at sales enablement, you know, we, are a seismic sales enablement company. We are always first to raise our hands to try and pilot new ideas and concepts as long as we feel that that tool may or may not be able to, you know, help us achieve the gaps and the the goals that we're trying to achieve. And so if you can try it out and, you have a really great intentional use case and challenge that you're trying to solve, that's been super helpful. And that also applies within our l and d team. Right? So you mentioned the changing in roles, and then you mentioned how to stay relevant. We're constantly encouraging our team members to try new things, whether it is AI enabled tools, or whether it's new and different ways to deliver a learning experience to a learner. That's really given us the freedom to evolve and has helped us iterate and do things differently and I think stay on the forefront. And that has definitely impacted the way in which your traditional instructional designer has shown up. They're now sort of these design strategists. Right? They're the ones who are helping inform our strategic leaders of, hey. Did you know that this exists? And here's where I think it could help you. We're becoming a trusted partner and a consultant to our leaders, and we're not necessarily an afterthought. We're actually being considered and have a seat at the table so that way we can help them make informed decisions on how to bring certain things to life. Great. We're gonna jump into some of the audience questions too. The first one that I see here at the top of the sort of upvote list is, how did you guys measure this? Chris, you talked a lot about data a minute ago. There's obviously a very systematic approach here, but, also, you started off with the story. You know, this is really connected to these four specific moments in the sales, process. So what's the mechanism and the methodology for for measuring this? So I'll start, Steph, and jump in. So so, ultimately, we have now, we have probably two or three different dashboards. And one of them is specifically laid out against the sales process, And it has all of the inputs against stage and exit criteria, and they have numeric values to those things. And it really gets down to deal velocity. How quickly are things moving through the pipeline? And then how are they getting to, the qualify stage to start solutioning? Because we know the deal velocity metrics, if we get to qualify depending on the type of business in a particular time frame, we're 60% more time more likely to win the business. So that is a series of dashboards. All of the SDRs have rubrics. They have a badging process. They have very specific learning, knowledge, and outcomes that they have to get to, that gets them the process of shifting from an SDR to an account manager, from an account manager to an account executive. Those are all measured along the way. So there's inputs, outputs, and badges that create that environment. And we have a partner, Credly, that that that does that work with us. And then you have a number of things now that are at the CRO level that come down from the top. So you think about the bottom coming up to the from the from the bottom and the top coming down and meeting at the middle at the director level on whatever the the field the the senior VP of field operations is is measuring. There's four things to the VPs. There's four things to the director, and then it goes down to the individual seller and leads, that are managing all of those particular things through the pipeline. So it is very, very focused on the pipeline, and that is transitioning talent business to services business, which changes the life cycle and which changes the the the time to deal. So there those are all things that are measured now. But I would say we're trying to keep it very focused so we can see if it's moving or not. A 100 you know, a thousand metrics is not necessarily as good as a 100 very focused ones. So so we have three dashboards. Steph, would you add anything to that? Yeah. So the only thing that I would add, and I'll get really specific here, is, within our sellers, one of the core skills and and things they need to be able to do is effective business development and outreach. And that means picking up the phone. And if you recall one of the role plays that we did was the elevator pitch. We actually focused really heavily the past couple years to ensure that every everybody knew the words to happy birthday which was our elevator pitch. We rebranded, we added some things, provided clarity. Right? And so the ability for them to effectively execute the elevator pitch, the way that we've been able to measure that is because the the buyer's picking up the phone. And so one of the key indicators that Chris is speaking to is what is the conversion rate from prospect to interest? Well, the requirement to get from prospect to interest is to book a meeting. And so when you look back at the, the, metric that we shared where that seller was able to get a services qualified meeting within six months, that might sound small, but it's actually monumental in the fact that that has not been happening. And so their outreach has been more intentional. It's been more focused. They're following the script, so to speak, and we're able to measure it from that performance perspective. But here's what I would also say. How do I know that it's working out and people are engaging with it? Well, we did the old fashioned measurement, right, which was surveys, collecting feedback, conducting focus groups. Now the benefit to us is is that we had a small controlled population that was intentional. We wanted to be able to measure the value because, again, we've never launched anything like this before. We had no idea how it would land no matter how great our intentions were. And so being able to have our designers conduct the survey, Yep. collect that sort of anecdotal feedback that helped build confidence, but then also strategically placing those maestros at the right point in time of their onboarding. And then we had phenomenal facilitators who were also able to speak on our behalf to really collect that feedback from their cohorts to really see if it's working. We found that collectively, over the duration of the four maestros, they're spending an average of about five to seven minutes per role play, which is actually the sweet spot because if you think about it, fifteen minutes for a role play can be a really long time. So five to seven minutes was great, and they're using it multiple times over, spending about an hour in total for all four role plays, multiple times per role play to be able to get the true fulfillment that they need and to be able to achieve what Chris mentioned, which would be success in their ability to outreach to prospects. And related question, you mentioned during the, story, the first cohort, I think it was 25% of the new sellers hit the the mark. In the second cohort, it was 60. Were there any sort of design lessons learned along the way? Were any tweaks made, or was that just a function of people starting to get more comfortable and and, and getting better at this? Yeah. It's the latter. We didn't make any design adjustments. There was not any major structural feedback or changes to the Maestros. So they've been running as they've been for the past, you know, three quarters. That was really just the opportunity. As Chris mentioned, we now have more cohorts going through our program. That's based on the hiring that we've done, just scheduling those cohorts. And so when we were at Degreed Lens, we really only had one cohort's feedback and one cohort opportunity, to collect that feedback. Now we have more. Chris, Yeah. would you add anything? No. That's that's a 100% correct. Paul and a few other people, I think you mentioned this, but they, are asking about the metric of 25% of the pilot sellers who landed their first meetings. What was that baseline? What was that before? So that's that's that's the same we got that same question at Lens. And the interesting. thing, because it's a services deal, it would be very different. We we would have very specific metrics, talent to talent business, which is our core staffing model. But when you talk about services, it was taking our sellers almost six months to a year to get the first meeting in the old model. So we don't really have a baseline of saying, well, it was taking them, you know, forty weeks to get it because that just was unheard of. But when we started to measure because what what we had to get to was we have a twenty week program that these folks are going through as Steph just talked about. Then you have to get past about four to six months' worth of time in the role, which is their their AM time, to get the metrics. So so so you're you're really kind of looking at a longer time frame. But at the end of the day, when we went back, we looked at two years' worth of time prior to the first cohort. We did not have a single GS win in the few two years prior, and we've now had two within six months. And as I was saying, the second two cohorts, which is almost 30 people total, and we're rolling a new cohort in every other week now because we've got about a 180 hires this summer, to the measurement process. The second two, which was, again, 30 people, 14 specific GS deals were in qualified between those two three those two groups at the same time period coming before twenty weeks in the role as an SDR. Again, so what we're seeing is this repetitive ability to get these meetings, get them into interest, and get the right people around the services deal because an SDR cannot sell a services deal by themselves. It's it's too complex of a sale, so you need partners to come in. So they get the meeting, then your partners come in, and they carry it together through Qualify into solution. And we know that if we can get if like, when we look at that 14 number, that is it that is that is that is firing on all cylinders for us because that's a high degree of all 30 getting something, almost 50%, because it wasn't a multiple per for one person. It was almost one deal to one person of about half the group. So that's a really big success factor for us looking forward. Yeah. One of the viewers, Dan, is asking, if it's not too commercially sensitive, could it be possible to share the role play prompts to recreate in his own tools? Obviously, it's not a sort of conducive format for doing that. But if you could maybe talk a little bit about just what's the the thinking process that goes into designing the prompts for each of those things. And the other thing is my understanding is it's not just about a prompt. It's also with the context that you're feeding it in the system that you're building around it. Right? So what are the other and one of the other, viewers was just asking. I think they missed the description of the solution itself. But I think if like, what were the components of the solution? Right? What are some of the thinking that goes into the the way that you executed that stuff? Yeah. Absolutely. So there it's not necessarily a prompt per se, like, how you would engage with ChatGPT or Copilot. On the back end, the ability to be able to design the role play, was really simple because there's fields that you get to fill out that help you create and preserve the structure and the integrity of the role play. And so Chris mentioned we have success profiles. So we looked at what does an account manager need to be able to do and demonstrate from a behavior and a skill perspective that's actually going to prove out that they're good and they're fluent in those skills and behaviors. And so we tied that a lot into how we built out the role play. So for example, if I need someone to build or if I need someone to create an elevator pitch role play, I'm going to load into the platform a document that has the elevate, the elevator pitch role play script. That is sort of the source of truth. And then from there, we were able to phase the conversation into three different phases. One was to have the role play, through audio say, okay, Todd. Tell me a little bit about your role, who you support, and what you're trying to do. Todd would engage through audio with the role play, explain what it is that they're trying to overcome, which would be to demonstrate proficiency in an elevator pitch, and then they would then be able to practice. The role play on the back end was structured into phase two where we'd be looking for keywords, thoroughness, confidence, tone, and it was educating itself on the script that we uploaded. So that way, that role play on the back end knew exactly what to look for. And once the learner got through the conversation to a certain point, the role play then asks, okay. Maybe I'm going to ask an open ended question, or I'm going to provide some, pushback, at which point we instructed the role play to stop because I didn't want it to go into the overcoming objections portion of the conversation. Again, I'm only focusing on the ability of an elevator pitch. At that point in time, the one thing I really like about Maestro is is that it engages with immediate feedback. It tells you what you did well, where you could maybe improve in some opportunity areas, and then it offers you an opportunity to either stop or you can retry it again. So we try to ensure or we did. We ensured that what we're asking the seller to be able to do and demonstrate in their role and what we're asking them to prove and verify in their learning, that that role play complements that, and it all connects together. And we followed that structure for all the subsequent role plays, so it was really familiar to them, and they know exactly what to expect. Todd, on the, on the design part of this, one of the things that that I think is really well done and it is evolving, obviously, is that it almost interviews you on the back end. It's almost like an interview when you're when the designer is working with in the in the environment. You know, the series of questions, the ability to implement as much information you want in there and be get as as broad or as detailed as you wish is the information that you place in there, and it truly becomes an iterative thing. We we actually had Steph had a team of people come from one of our sales offices and come in and live test with the designer sitting on the other side back and forth. They're going through this process. They're talking to the they're talking to the the to the role play, and they stop, and they go, okay. So how did that go? Is that realistic? Is it not what is it missing? What what count can we better it? At the at that moment, the designer is in the other side of the tool typing in and adjusting things, and they're doing it again. So it's this really kind of interesting build test case type model that, you know, is like I said, it's almost an interview, which is really well you know, I. think it's intentionally designed that way for ease of use, which is pretty cool. Yeah. That is cool. Another, semi related design question from the audience. Right now, Maestro Degree doesn't use avatars in these role plays. We have them coming soon. But the question here is, assuming it was a specific choice, well, I guess it wasn't a specific choice. We're we have them coming. But does the voice only interface did you see any advantages or disadvantages with sellers in that? Did you get any feedback on the voice only? Yes. And so the the voice the AI feedback that we received for that is it doesn't sound too robotic. There's minimal delays. So I think one of the telling signs of using any sort of AI bot, you always have a little bit of a delay which can be frustrating. Maestro's set up really well to reduce that delay significantly, so the conversation flowed really fluid. There is not only a audio component, but there's also a text feature that you can do. So if you just want to chat but here's what I would say is there is behavioral science around whether or not to use an audio role play versus a text role play, and there's going to be pros and cons to doing that. I think when you look at text role play, some of the key, use cases that I would say would just be like, hey, help me craft maybe a performance feedback email or message to my direct hire and here's what I wanna say to him or her and how does this sound and could I be more direct? Do I need to be more, pragmatic about it? Whatever that might be. But when you think of salespeople, salespeople don't spend or shouldn't spend a lot of time on email or text. We want them to be in front of the customer or on the phone and so that requires them to verbally regurgitate and articulate what it is that we need them to do and to be able to demonstrate proficiency, and audio is one of the better ways to do that. Great. I think we got time for couple more questions. I only see one more, in the the chat here. But, you mentioned so the question from, Nigel is how much time do you all allot for the reps to, to do the role plays and learning activities? I know you mentioned, Steph, I think it was five to seven minutes on average, per interaction, multiple interactions over the course of the, the program. Any, so, I mean, this is one very specific, situation, but I'm curious if you have, how that might be different for different personas or, if you have any other sort of guidance for, how do you drive adoption. The other thing that's related to this that I think was, interesting is you started off the story mentioning that part of the appeal of using the AI tools was to take some of the coaching burden off of the managers. Chris described this, really compelling idea of an adaptive feedback loop system that, you know, magically serves up the right stuff to the right people at the right time. But the reality today is that you still need people to, to drive engagement and adoption. So I'm curious, you know, in all of that, what's the manager's role? How are the managers, being used to, amplify or or or or drive these results? Yes. So the first question regarding, like, timing allocation and suggested practice. So we prescribed and recommended that each seller engage with each role play a minimum of three times, And we were budgeting or allocating roughly fifteen minutes per role play is what we were trying to help set expectations for for the seller so they knew how to budget their time accordingly. And then on average, through that single cohort, we found, again, it was about five to seven minutes. They were still doing the reps of at least a minimum of three. And the reason why we picked three was because the first time you're probably a little shaky with it. It might not pan out exactly how you thought. The second time you're adjusting. And then the third time, you're probably in a really good rhythm and a good flow, and you know what to expect. And you can really fine tune and sharpen the skills that we're looking you to practice. So we went that direction, and that was all prescribed and laid out specifically in their participant guide as well as through their online degree platform and program. So that way, they know exactly what they needed to do. Chris, I'll have the second half of the question turned over to you. Give me give me the second half again. I'm listening I'm listening specifically. Oh, so you started off by saying that a big part of the value of the AI was to take some of the burden off of the managers, but how are you how are the managers. being used to drive the results? Yeah. So so I would say we are shifting, as much time to the deal code the live deal coaching now. That that's that that was the whole reason from the beginning with the span of control issue. And I think we're seeing there's a there's a interesting connection that we didn't talk about, but all of our directors in the field must teach in our sales program. They must come to corporate and teach. So they come, and they're either a table coach or a a lead facilitator with some of our people in the room that are our professional facilitators. So that's the connection of understanding the core development structures and journey that we have for all of our sellers. So they are understanding the shift in the use of technology. There was a rollout plan that we had very in the beginning to those directors of how this was going to shift and what the expectation was of saying, okay. So you're not doing this. We want more of this. And then, ultimately, what we've also done is when we rolled out the sales process two years ago, almost two years ago, is the idea of how we have developed them along the way with new technology and where they know the exit criteria for someone to leave, whether it'd be prospect interest and qualify and the connection to what the seller has done to the connection of their deal coaching. It all sounds like a lot. It's not that complicated, but it works really well. When you have the person who understands what's happening, they teach in class, they come and then they do this deal coaching. It it kind of all seamlessly goes together. But we are also constantly upscaling those directors. They've gotten two two two different events this year already around sales process and coaching. So, I would say that that's that's the biggest mix of how we're doing things now. Great. Well, I'm seeing the big flashing red light, so we're at the top of the hour. I appreciate very much, not just your partnership, but you're sharing the story today. Super. valuable, and I continue to be impressed by the, you. know, the the systems thinking that I see from you guys. So thanks very much. Thank Thank you. Have a great day. thanks to the audience for spending your time with us. Yeah. Thank you.