Innovation in Diversity Training – Using Microlearning to Engage Employees to Drive Behavioral Change
Research shows that organizations spend $8 billion dollars a year on diversity and inclusion training programs. Yet, diversity and inclusion teams often struggle to measure performance outcomes and behavior change after formal training is complete. One reason for this is the lack of reinforcement of core training concepts. Research has shown that if not reinforced, up to 79% of content is lost within a month.
In reality, D&I training is never “complete”. With workforces and workplaces that are ever changing, this vast practice area requires continuous reinforcement and assessment to identify knowledge gaps. By helping employees learn how to embrace diversity, organizations will drive innovation and deliver long term, tangible benefits to customers, employees and shareholders alike.
Download this complimentary webinar where Kari Heistad, CEO of Culture Coach International, and Meredith Odgers, Director of Marketing at Qstream, discussed innovation in diversity and inclusion through the lens of microlearning, engagement, reinforcement, and measurement.
By watching the recording, you’ll take away:
- How to deliver engaging, continuous reinforcement of D&I concepts and behaviors
- Innovative methods to surface employees’ understanding of D&I in practice across diverse and multicultural teams
- New ways to measure impact of diversity and inclusion programs
Culture Coach International
Diversity has been the hallmark of Kari Heistad’s work and her experiences have brought her into contact with world leaders, inner-city youth, non-profit organizations and business people. She specializes in making the issues of cross cultural awareness, diversity and inclusion and organizational development engaging and insightful to her clients. Her programs have been called “insightful, unique, exciting” and “eye-opening”. Her powerful approach makes these issues accessible and the skills gained become powerful tools in the workplace.
Kari’s work as CEO of Culture Coach International (CCI) focuses on building respect across different groups of people. By focusing on building respect, CCI is able to help organizations put culture to work for them instead of against them. Intent upon helping organizations to understand the bottom line impact of cultural and diversity issues, the work of CCI is both practical and strategic in nature. Changing demographics are forcing organizations of all kinds, both nationally and internationally, to recognize the changing face and needs of not only their workforces but also their customers. As institutions seek to capitalize on untapped markets, cultural competency has moved from a luxury to a necessity in today’s marketplace.
Director of Global Marketing
Meredith and her team bring the Qstream brand to life for customers, partners and Qstreamers alike. Qstream’s mobile microlearning solution is used across a wide variety of business critical learning programs to directly impact performance, including to support reinforcement of diversity and inclusion training. Having spent 20+ years as a marketing leader in Australia, Singapore, Hong Kong, and the UK with responsibility for Asia, Europe and North America, Meredith is a firm believer that a diverse and inclusive corporate culture is key to building and sustaining high performance businesses.
Meredith: Welcome to today’s webcast presented by Qstream and Culture Coach. Today’s webcast is on Innovations in Diversity Training. Thanks for joining us from wherever you are in the world. I’m actually in Dublin right now, but based in London. We’ve got Kari and David over there in Boston and I’m sure many of you are dialed in from all over, so welcome. I am Meredith Odgers of Qstream and I’ll be your host for today’s webcast.
So today, if you like to join the conversation online we will be live tweeting and socializing everything out using #workplacediversity, #behaviorchange and just one more to add in there #trainingreinforcement. Just a few housekeeping matters before we get into the main event. The slides and the recording will be made available from us tomorrow or later today, so everyone who’s registered and attending will get those.
If you have questions for us throughout whether it’s logistics or questions for us as speakers today, please do use the Q&A part of your console so that we can move through those either throughout or we’ll certainly reserve some time at the end for those. We do encourage you to use the chat to comment on what you’re hearing, get a discussion going amongst yourselves but don’t be shy and certainly get it going.
I’ve already introduced myself and I would like to introduce Kari Heistad, founder and CEO of Culture Coach International. Welcome, Kari.
Kari: Thank you, Meredith. I appreciate the invitation to join you and welcome to everybody who’s joining us from wherever you are.
Meredith: Great. Thanks. So, I am going to tell you just a little bit about Kari and myself, as we get started but before I do this is where I’d really like to hear from you. So, using the chat if you can share why you signed up for today’s webcast, what really caught your eye and just to get us going I’ll actually go first. So, yes I’m on here with Kari and I’m at Qstream but for me culture in diversity is not really necessarily my own professional specialty area but that’s certainly why we have Kari here with us today. I will say, though, that it is a personal interest area. I’ve been lucky enough to work all over the world, in regional and global roles and outside of Australia, Singapore, Hong Kong and now London. We are headquartered at Qstream in the U.S. We also worked across customers, teams, suppliers located across all continents with the exception of South America, I must say. I have had the opportunity and I really had to embrace diversity and really make the extra effort to truly understand buyers, sellers, business behaviors, all the things that are not really natively my own, I suppose. Through that experience I actually felt like the odd one out many times, whether it’s age, culture, local custom, gender or even tenure and that made me feel different in myself but it also meant that I perceived I was being treated differently at times. Now, that was necessarily a negative experience or with negative intent but there has been times where probably unconscious bias was certainly at play and it really did have an effect on productivity until trust and connection could actually be rebuilt. So, that’s my interest in this topic today, and I have to say that I’ve been learning a lot through this process with Kari.
The one thing I have learned, also just to take it off piece a little bit, the one thing that always broke down any barriers and help develop and understanding of everyone that I was working with was a large team feast, especially in Asia and so sharing a meal would usually get everyone to shed their layers.
Okay, so I can see here that we’ve got a few people dialing in designing inclusive leadership training, so leadership in particular. We’ve got somebody from a mining company, integrate microlearning for the D&I programs, gain a bit of understanding of how to leverage and promote behaviors, to promote inclusion, so there’s a lot things there. We can come back and probably we’ll hope address those as we go through. Thanks everyone for sharing.
With such, I’ll just do a little background on Kari to set the scene. Unlike me, diversity has been the whole map of Kari’s career and her experiences have really bought here into contact with world leaders, non-profit organizations and business. A little fun fact, I hope you don’t mind Kari, but this is something I thought that was quite interesting from your background that your passion for the international exploration really did start at quite a young age, so third grade you wrote an NSA titled, “Who I Am?” and you wrote “I also enjoy travelling. I find it interesting to compare different kinds of geography, different styles of living and to meet new and different people.” Well, I’m guessing you certainly have through your career.
Today, Kari has really harnessed that spirit into an impressive career and now into her business with Culture Coach. We’ll hear a little bit on Culture Coach in a moment but they very much specialize in bringing some of the issues around cross-culture awareness to the floor and ultimately making these issues and the skills gained from this knowledge become quite powerful tools in the workplace.
Okay, so a little bit more now on what we’ll cover today. Thanks for everyone who is really getting involved in the chat. I’m trying to set a keeper, a one eye on that while we’re moving through. I can come back to some of those and don’t forget if you do have any questions throughout put them into the Q&A or we will monitor the chat as well.
Today’s, webcast, looking at the registration list D&I experts, corporate culture specialist, so today we’re not necessarily going to focus on D&I topics per se, instead we are going to look at innovations in a way that learning is designed, delivered and reinforced with durable behavior change. We’ll look at how you can measure those efforts and you’ll have some homework at the end to get a little hands on with the experience. So, we’ll talk through that right at the end there.
Just to put a bit of context, I won’t spend long on this but, but I do know that many of you probably hadn’t had contact with Qstream, maybe even Culture Coach before so I’ll just tell a little bit about Qstream and why we’re here today. Qstream we’re all about making people better at what they do and we do that through technology enabled microlearning programs. We work with quite large organizations all across the globe, so actually our customer base is quite diverse but with customers they are supporting and enabling a very diverse group of learners within country, across geographies, across different business units, et cetera, and we do that through engaging learning in the process and there are a number of ways that we do that, reinforcing critical job knowledge really specific to the job so learners can apply and recall, or to recall and apply that mastery when they’re actually out in the field or doing their job and also providing learning analytics on that content to show that each program is outcome driven and measurable.
That’s a little about us. I’m going to hand it over to Kari now just to introduce Culture Coach and then we’ll get into the main event.
Kari: Well, thank you, Meredith. I think from the Culture Coach side we ended up with a lot of experience in the content development field partly because we are designing large-scale training program and then that led into an interest to say, “How do we actually help our clients to follow up on the content that we are delivering in training program,” and so we got into this kind of micro-education space in the analog or the paper versions by developing pocket guides and things like that which our clients have used really well and they reinforce those key ideas that we have presented in training programs and so when we met up with Qstream and realize what Qstream was doing in terms of the reinforcement in micro-education on the digital side it was a really easy partnership for us to be able to form. So, we’re excited to be able to bring this tool into the diversity and inclusion space and to be able to help people to understand how that technology can help them drive some deeper behavior change.
Meredith: Right, thanks, Kari. Okay, so we’re going to get you all involved now in our first poll.
We talked a fair bit about background and us and actually we want to know a little bit more about you. You’ve already started sharing on the chat and thank you for that so keep discussing and sharing there, but we actually want to get your thoughts on what you think the definition or your definition of micro-education is. It could be called microlearning in the context here. Kari and I have debated this, we’d use micro-education, but I think you get the gist. We’re going to take it to our poll now. What you’ll actually see on the screen, when I launched this poll, are some options that you can go through.
So, I want to give you a scenario. So, you’ve rolled out a D&I program and that’s included a D&I video series that you’ve launched through new e-learning modules, say through your LMS or another technology platform and then your CDO has heard that micro-education could be an effective tool to support this program. How would you explain micro-education? So, I’m going to launch the poll now and you’ve got four responses here to choose from. Here we go. Just make sure I get the technology right.
Okay, so first option is micro-education or microlearning is chunking detailed information into a shorter format. Its information delivered just in time triggered by situational triggers, two triggers there but you get the idea. It’s a series of short presentations and videos usually on some online platform or a guided learning experience that repetitively pushes job-specific, job-relevant scenarios over time. I can see we’re getting quite a few responses in there. I’ll just leave this open for just a moment. Everyone is getting active. There’s a lot of change going on. I’ll see it when it settles down. Okay, we’re slowing down and the response is coming in but I think we’ve got a good range there. All right, I’m going to close the poll in three, two, one.
Okay, I’m going to share the results. We have got quite a range over here, so a lot of people, about a quarter, think it’s chunking detailed information into a shorter format. We do have a heavier balance towards the guided-learning style approach, the push, and then a lot of short series of presentations and videos.
The reality is with this is they’re probably all right to a certain degree. There is actually a lot of definitions out there around what microlearning is, how it’s perceived, etcetera, but today we’re actually going to take it through a lens of a more guided-learning approach. This is really critical, either information, skills, behaviors that you want to reinforce in your business, things that you can’t really afford your workforce not to have or do. So, we’re really taking it from that lens today.
Okay, so with that in mind Kari you might want to make a few comments on that, over to you.
Kari: Sure. I think one of the things that we look at for either diversity professionals or training professionals in general is really how do we take the work that we’re doing and make sure that it’s relevant to senior leaders and to the organization overall, and so I think part of that then becomes the question in what are the challenges that we face in building a better culture and how do we make sure that we are aligning that and being aware of what keeps our senior leaders up at night and so we see organizational strategy side list on the left there, with that much more globalism, need upscale and rescale and when we did interviews with CEOs they were really looking at how to attract and retain top talent and develop new leader. On the other side, the people strategy, this is the side that we typically work with quite a bit which is how do you actually drive behavior change, how do you really engage people and how do you make sure that the work that you’re doing is aligned to organizational goal. This is what it takes to really begin to look at it, so in the context of where we’re at right now and what would be driving us to be having some of the conversation that we’re having today is about looking at these challenges and really saying, how do we actually help to drive behavior change because that’s going to help us to address the challenges that we see here on this particular slide.
Then, we’ll seen on the next slide as we drive towards… why do we really want to do this in the bottom and what are the business benefits to this in terms of the outcomes that we’re driving towards and it’s really about how do we then create an inclusive engaged workplace that allows us to drive current activity, innovation and ultimately sales because we are all trying to market or sell something in terms of what we’re doing. So, it’s really about how do we understand behavior change and then how are we driving it to really create this more engaged and innovative workplace.
As we think about that in terms of the macropicture we know that there is a whole variety of ways that we can provide corporate culture education around that and there are many departments involved in it, but I think it breaks down into three main content areas. The workforce, what are individual skills that our employees need to have to be successful. The workplace, which is much more about what are processes that we need to have in place to be successful and then basically how do we train our employees to understand those processes and to do them, and the last one being the marketplace, how do we anticipate market change and what skills our employees need to have to be effective in the market place. So, when we look at them we’re going to talk through the ideas, the workforce, the workplace and the marketplace and understand that in all those are trying to drive behavior change, we do training and work with employees.
Meredith: Okay, so we’re going to get you hands-on again. Thanks, Kari, for that. I think before we go into the next part we should set the scene and there’s many ways to deliver training per se. Trying to just get a feel for how you deliver training, so what kind of modes do you use. This is multiple choice so I think you can select more than one if I’m right. I’ll launch this poll in just a moment. Here we’re actually asking you what modes or the primary modes of deliver that you use for D&I training today? So, instructor led so that’s typically class room, one to one; manager-led training so very much team base, team meetings, group share, that kind of things; employee-led training, now this is training that is typically self-service online, Netflix style, going to a library and self-select which training modules you like to take; technology-enabled training and we’ve actually distinguished this from technology-led training. Technology-enabled is through webinars, global workforce is distributed teams probably trying to cover people in multiple locations but technology-led training is really along that guided approach, so really pushing out the training program and content via technology putting it right into the hands of your learners and workforce. Some people might be on here but actually doing any D&I training right now, so this an education exercise and discovery exercise for you.
This is multiple choice and we’re getting quite a few people looking like they do use multiple channels or modes, which is not surprising, so usually a blended approach is typically used depending on the individual objectives. We’re slow down on the poll, so I’m going to end this now and then share the results.
Kari, I can see here that instructor-led training is primary, followed not too far behind by employee-led training. How do you see these results? Is this surprising? Is it in line with what you’re seeing in organizations today?
Kari: It is in line. I think the instructor-led training is the traditional way that we’re doing it and I think particularly around diversity and inclusion when we’re doing instructor-led training it’s a chance for us to engage employees in conversation and discussion and create awareness and that kind of interactivity is part of what is that learning process for them, so it’s a great way to actually drive some learning for them and get them to take it on from their own, but I am seeing increasingly that people are using a multi-pronged approach to D&I training, the instructor-led training plus some additional follow-up, some other pieces and as workforce has become more dispersed and as people become more spread out and work different work schedules, I know that the customers were working with are increasingly coming up with a challenge of, I can’t do instructor led for a variety of reasons. What are other options? I think that’s where people are beginning to expand those options that they have and how to use that and we’ll talk in a minute about different modes of training with that, but I think that’s where we begin to say, “How do you actually pull in that multidimensional approach but this is mirroring a lot of what I’m seeing in the market place currently, yes.
Meredith: Okay, good. We just had a question come in by a caller on the chat there: So would instructor-led training apply if the facilitators are employees within the organization? My thought would be yes. I mean, it doesn’t necessarily need to be.
I guess it’s personal or one-to-one or classroom or in-person training led by an instructor or specialist in that area which could be a lot of you on the call today.
Okay, thanks Carla for that question. If people have questions just a little ad break here, send them either in through that chat or you can see on your console there is a Q&A. So, pop them in there as well and we’ll get to those. I will stop showing this and we’ll move on.
All right, this is really interesting. I always like this slide. There’s many ways to train. There’s a lot of investment being made into to corporate training as a whole. We got this particular step, I think it was from a Josh Burson study or quote, but the interesting thing is while we’re investing a lot into corporate learning and training most of that delivery is not always conducive to the way our brain actually learns. At the end of the day, we are human and we’re generally wired to forget unless we do something with that information after the initial delivery or the initial learning takes place. This is quite a conundrum, I guess, for Ellen Dealy there is a tremendous amount of investment, learning technology, instructional design, teeny costs if we got to travel somewhere, consultancy and bringing people in externally or specialists subject matters and whilst its best intention and actually some of that learning is effective there is a typically quick decay of those learning. I guest therefore one could argue a poor ROI of that training investment now all is not necessarily lost but a lot is.
Kari and I have been searching for market sizing of the proportion of that that may be spend on culture and diversity training, so actually if anyone does have a good source do share it. We’re all keen on finding that out. But even if there was 1% of the purported 200 billion still 20 million being spent potentially on D&I training in just a year, so that’s a significant investment and I guess the question is, “Wouldn’t you want every single cent of that to count and have a real lasting impact on building a diverse and productive culture.
That’s what we’re going to unpack today a little bit further, how to overcome that forgetting curve phenomenon and how to develop our D&I programs in a way that will lead to a more durable and lasting behavior change and really become part of a culture, I suppose.
With that, we’re actually going to move onto a couple of different areas and the first one that we’re going to address in is engagement. It’s probably obvious to say that if a learner is not engaged then they’re not going to learn and no learning then no retention of concepts and no application of those concepts on the job and no behavior change. So, without that as almost a starting point it could be deemed that the effort is wasted. So, we’re going to explore this a little bit more and Kari and I’m going to hand it over to you now to share some thoughts on this.
Kari: Sure. I think one of the things if we think about it for training is what are some of the end results that we’re trying to get out of the training that we’re actually doing? We’re investing a lot of time, money, resources into training and so when we think about the best way to actually train there is a variety of ways that we could do it. We’ve created this chart. I’m not going to talk you through it all. We’ll send you the link afterwards, you can read it in more detail, but basically what we want to think about is the mode that we use to choose for training is important because it’s going to be dependent upon the number of people we need to train, the content, what the type of content is, our budget, our timeline and all those types of things, and so if we look at the different options of D&I trainers or learning and development professionals to say which is the best ones and then how do you create a learning stream so that you can present content and then be able to reinforce it for people to get them to think through this in a variety of different ways.
I know when I was in grad school they said you have to present the context to someone at least five to seven times before they all absorb everything. So, I think it goes back to that idea in the way that the brain works in terms of needing to reinforce things and so as you think about designing your training programs in your organizations I’d encourage you to think about how do you mix and match what these are to develop a learning stream that reinforces the key concepts and ideas that you’re trying to reinforce because that reinforcement of those key ideas is what’s going to really allow you to build behavior change and be able to make the change that you actually want to have happen.
Meredith: Excellent. One of the questions I just put out to the audience there on the chat is, of these which have you found effective for D&I training and reinforcement, actually which ever works for you. I’m curious to hear and see your chat on that.
Kari: I think the piece we can think about is that if we take into account as we design training program how the brain actually works we can actually begin to achieve much higher levels of employee in engagement with our program, knowledge retention and proficiency. I think for those of us in the training industry and D&I in particular it’s been a bit like how do we drive that behavior change and I think some of it is, for us to… we’re really good people, people and I think it’s time for us to use technology to our benefit and understand that how our brain works and use the tech to be able to help us to drive towards some of the results that we really want to be able to get in the end.
When we think about what’s the link to technology and how we can use that, there is a variety of different ways we can use technology as a tool to help us. If we are looking at instructor-led or manager-led training program we know technology there could be used in terms of videos, creating word clouds or poll. Things like that allow to engage people and to get a lot more information from the learner as you go.
If you’re using online training, right now a lot of us has hosted tech platforms posting content there that people have been going and walking them through for themselves versus tech enabled, which is much more about using technology to push content toward learners in bite-sized pieces. Then the last piece microlearning versus micro-education, Meredith and I had been differentiating the two in this particular webinar because what I’m seeing is that increasingly microlearning has come to learn like online videos and short videos there, micro-education being much more around scenario-based learning. So, they are both being presented in short two to three minute increments but there is beginning to be some parsing of the details in terms of the defining of those two terms, but throughout all of these different modes of training we could see that technology can really help us and as we start to think about it from a D&I perspective is how do we use technology to help us drive that change that we’re looking for.
Meredith: Interesting. Actually, Kari, we got a couple of questions that are coming in through the Q&A.
Kari: Of course.
Meredith: Two of them are actually on the measurement piece, so I’m going to just let the people that did ask those that we will actually be getting to that as part of the webcast today, so just hold on tight and we’ll come back to you, but very interesting that the audience is going to the measurement piece.
Keeping all that you’ve said in mind, Kari, there’s many ways to deliver that D&I training and this is one methodology to reinforce behaviors in the daily workflow. For example, you could use technology such as an app like Qstream in a way to deliver a scenario to the learner in their hand, mobile or to their desktop a precision scenario that is actually really related, particularly to their workplace or their job role and not just generic, I suppose, that might not be so related and therefore that helps with the engagement as well. When they actually answer those challenges and get engaged in that learning process, and this is going back to that engagement part, responding immediately, getting immediate feedback, “Did I answer correctly or incorrectly?” But then explaining what the correct or incorrect answer is and that’s another opportunity for that reinforcement of the concepts, of the D&I concepts and then it will make it and a little bit fun and competitive across teams and cohorts, et cetera if you choose to do so.
This is one approach and it seems very simple but actually that’s taking in a lot of the theory that Kari talked about in terms of repetition, spacing, mobile delivery, not just shrinking the content but actually forming the content for the delivery device in a way that’s engaging and fun and keeps the learners interested over time. With that we’re going to wrap up the engagement part of this and then look at content.
The question that were trying to answer here in this actually is, “Can the shape of the content really change behavior,” and we spent a little bit of time talking about this already in the first couple of concepts but let’s face it, we all know that developing content effectively can be quite a challenge to “get it right,” you require subject matter expertise and sometimes client’s compliance approval and there needs to be an acceptable threshold of what you’d call desirable difficulty to challenge the learner get cognitive wheels turning and that can be a quite a time-consuming process but sometimes less is more and not only that I guess really being very thoughtful into delivery, what is the main concept that we want our learners to take away from this experience. So, we’re going to dive into that a little bit more and Kari is going to talk a little bit about the content development piece of things.
Kari: Great. So, when we think about scenario-based learning, I think this is one of the ideas is that we’re trying to create content that helps the learner to step into somebody else’s shoes and to see that content from that perspective and what we know is that in doing that the brain is able to orientate itself more, get more of a virtual kind of feeling on, “What would I do in that type of situation.” So, the scenario-based learning is really about how do we help the learner to understand the situation, a real life one, in the case of scenario-based learning it’s very short because it’s only one or two minutes long. Think of it as a mini case study for those of you who design content and do them in a training program, we have the luxury of 20 or 30 minutes to explore a case study but it’s really much more around how do I create that sure way that I’m going to think about behavior and ask them to explore what that begin to look like. In a moment we’re going to give you a chance to do that. I will say that do text limits that the next scenario of questions we’ve done here is a little bit short but it will at least give you a sense of what that begins to look like for our next poll in terms of what does scenario-based learning look and feel like and how can you help use that to drive behavior.
Meredith: Great. Kari, you ready for me to start launching that poll now?
Kari: Yeah that would be terrific. So, as we look at it here we’ve got a scenario based here with Leigh is a recruiter and he’s working with a hiring manager and they’re looking for ways to avoid bias in the hiring process as they seek to fill several open positions. How can bias come in to the hiring process from the company’s internal perspective and you’ve got four choices here: Telling the interview panel that the next candidate is a diversity candidate, scheduling extra time for the candidate that Leigh feels he connects with the most easily, telling a candidate that they are a diverse candidate and the last one is selecting more candidates that went to the same school as the CEO. We’ll give you a moment to choose your choices there, again it’s multiple choice so if you think there are multiple ways that bias can come into this then you can choose more than one answer.
Meredith: Interesting, Kari, here some of the chat that’s been going while you’ve been working through is what’s being working or maybe what hasn’t in terms of the modalities for D&I training. A lot of people have talked about instructor-led and classroom and the odd one has mentioned technology-led guided through technology, but I think the general feeling is that a blended approach and actually then implementing online learning more as reinforcement potentially to some of that classroom training is really helpful. So, yeah, we can cover that a little bit later in the questions as well.
Okay, so the poll is looking like it’s slowing down. I’m going to close it in a couple of seconds so we’ve got majority of people are in, so I’m going to end the poll in three, two, one and here I’ll share the results.
Kari: It looks like we’ve had a mixture of answers for what people have and as we think about what options are we can think about it just in case from the internal perspective so by telling the interview panel the next candidate is a diversity candidate we’re setting up bias in their minds because that’s automatically telling them there is something that the diversity person is seen as different, so that’s setting up the bias in terms of like, “Oh, here is something different.” That could put into their minds the fact that, “Are we really sure we want a different kind of candidate?” So that could be coming in there.
Scheduling extra interview time. I mean part of the challenge is not fair in the process but part of it is also setting up the bias that Leigh got his bias there about who he thinks would be the best candidate who may or may not be the best candidate according to the hiring manager. So, with understanding there Leigh is putting his personal bias versus saying, I’m going to give everybody the same amount of time, and be able to let the hiring manager to make that choice from there.
I’m going to skip number three for a second and the fourth one there is selecting more candidates from the CEO’s school. That is of course limiting your talent pool by saying, we’re going to prefer folks that come from this school. I often get questions about this when we do some training programs, because they would say, “Well, you know, this hiring manager,” or, “This person prefers candidates who went to these particular school,” and I often joke that do you want the person who graduated last in their class from that school or do you want someone from a different school who graduated top of their class? That’s a bias about where you think top candidates could come from when in reality they could come from anywhere.
The third one in the middle there around telling a candidate they are diverse candidate is creating bias but an external bias for the candidate about the company. Part of it is understanding and saying bias can get introduced from both an internal perspective as well as from an external perspective and obviously when you’re trying to do recruiting you’re trying to eliminate those opportunities for bias so you can have the most effective way of recruiting people and getting the best candidates in the door.
Meredith: Excellent. Okay, well, thanks Kari. Now you’ve all had real-life experience, so I’m going to stop sharing that and move onto the next concept.
Here is the final concepts and this is coming back to measurement part. The question that we’re trying to answer here is do you have the Intel to know what’s working and what’s not? In reality there are a number of different metrics that are used for measuring and analyzing broad LMD programs, D&I program, et cetera, but Kari is going to take us through a little bit more of that discussion. So, over to you.
Kari: I think one of the challenges that diversity and inclusion professionals run up against all the time is, how do I show the value of my work? That’s a general topic for another webinar perhaps but for today we’re just going to be focusing in on training and what those begin to look like.
On the left-hand side of the slide you see some of our more standard metrics. We can look at attendance and evaluation and things like that that’s been traditionally what we’ve tracked. The power though is that when we add technology to what we’re doing, we’re able to drive and understand metrics in a very different way. Once we’re using text-enabled metrics we can start to say, what questions do people answer the most quickly? Which ones are they getting right the first time versus needing a second or third try in order to be able to get it right? Which team is struggling more than somebody else? As you start to look at the data and the metrics you can then parse out team A maybe struggling with this type of content but team B is struggling with a different type of content and so when you have limited time, energy and money you don’t have to do a one size fits all, you can go to team A and say, “Let’s reinforce the content you’re struggling,” or you can go to team B and say, “Let’s reinforce the content you’re struggling with.” They don’t have to repeat the content that they already have and they understand and so you can begin to drill this down from global perspective, team perspective even to an individual perspective and that allows you to provide a lot more individual coaching and it will allow you to focus the use of the resources that you have in terms of time and dollars and getting the biggest impact for what you are actually doing with that.
We can see then on the next slide too is that we start to say, what do we look at then for lagging and leading indicators” This is where tech can also help us to say the lagging ones are much more activity based than completion numbers, the leading numbers are much more around how do we look at engagement level? With technology enabled training how do we get an engagement level that’s very high and we know that people are engaged for it and we know then we can begin to look at their awareness and concepts and kind of identify some hot spots. This is, I think, an area which for me as a diversity professional because I’ve been doing it for a very long time, it’s very exciting to me, to be able to look at the tech and go, “Oh my goodness. Can you take a look at that?” because it’s giving me numbers and it’s giving my clients numbers that we haven’t really had before and the ability to use before. So, it’s a fascinating way to be able to use technology to leverage it and that in turn allows us to go to senior leaders and say, here is the power of what we’re doing.
The next slide you’ll see there is a visual representation of some of those metrics. It’s called a heat map. If you haven’t seen one before, on the left you see red is lower proficiency, green is a higher proficiency so if you look at it and you bucket your questions and the content you’re presenting and measure it correctly you can start to say, “Oh look, here are areas where they’re doing well.” Here are areas that they need better help,” and you can turn on the right and say, well, this is our current proficiency versus where we started off from. This is a fantastic way to very quickly and easily show your senior leaders, this is the power of the investments that you made into training. We went from the one on the left to the one on the right and it’s very quick and easy visual way to give a metric about that. Obviously, with heat maps there’s data behind it and you can do dig in and poke around and find out all kinds of things for yourself, but however you’re create a heat map of some kind or something like this using whatever technology tool you might have, I have found it to be a very powerful way to give a visual representation of the impact of the training program that you’re doing.
Meredith: Kari, this is really interesting because actually I think we can almost like just pause to take a couple of the questions here because it seems like the right point in time.
Most of the questions coming in on the Q&A are around the whole metric piece, so we’ve got questions like these. So, isn’t the most valid measurement a change in performance that drives students or learners closer to meeting their business objectives? The other question is in a similar vein, are the metrics tied to behavior change or performance outcomes? So, one way to look at this for the people that asked those are interested in this example that Kari has taken you through on the… down the left-hand side are the D&I topic areas that the Qstream challenge has addressed and these are bucketed and grouped together. Across the top in this example you’ve actually got by region but you could easily tag your learner groups with different things. For example, the most obvious one could be in terms of linking performance and cross tabulate it with another metric such as how they perform in some of these diversity training topics could be in sale. So, you could then tag meeting X percent of quarter attainment or meeting certain sales goals, et cetera across the top and then still have your topics down the left-hand side. Then, you can start to dive into which cohorts, whether it’s by geography, team, business line performing well and also mastering some of these areas. Are there any correlations that can be made between how they’re actually performing in the Qstream on these topics and their performance and you may or may not find that but there are ways to actually overlay the data and mastery of understanding D&I topic with things like other business or performance metrics whatever they may be. So, I just thought I’d jump in there to talk about that. But Kari, do you have anything to add on that at this point?
Kari: Yeah. I think the biggest is obviously we always want to drive towards behavior change so whatever metrics you are using around that I think if you’re beginning to look at something like unconscious bias are you starting to have conversations with hiring managers about how bias may be coming in to the hiring process are you using ways and tools and things like that to check their bias in that process of hiring people. So there’s a variety of ways you can look at behavior change that you can start to measure and I think that’s ultimately what we’re trying to go towards. So, for anything of what we’re trying to do in the training space is like the end goal is, how do we drive awareness which in turn drives behavior change and so that would be ultimately the best metrics you could have.
Meredith: Yeah and that’s really interesting. I mean, you mentioned certain cohort like hiring managers. Again, these programs, the end audience, the end learner could be employees, frontline or staff. It could be team managers like frontline managers. It could be hiring managers. So, these programs can be ran across whatever you’re trying to achieve in your business objectives and therefore the learner groups that are associated with those, the audience, for those. So, I guess don’t just think of this in terms of the end employee it can be the manager and leadership levels as well.
Some real side note, I did read a really interesting article here in the U.K. about how one of the big bangs are focused on doing diversity inclusion training particularly on the unconscious bias area amongst their top 200 executives and leadership team, so they could be the audience in that case.
Kari: There was a question that came up, somebody had asked about initial versus current in the titles there in the heat map. Initial being the first time somebody sees the content and then current being if they’ve gone through, the first one if they’ve got repetition of the content like they would through Qstream, what they’ve gone through the second or third time and where their proficiency level has increased, so that’s the difference in terms of if you’re giving content more than ones you can test initial and then you can test where you are later.
Meredith: Yeah. The thing to add to that, because I can see what are you measuring, you’re actually measuring the proficiency of how that learner answered the scenario-based questions. Did they get it wrong or right, etcetera? That all then ties results in a proficiency score, which is really what underlies these heat maps. That’s their initial proficiency in that area and understanding and as Kari said their most current or if you finish the Qstream after a number of repetitions how they actually ended that. So, you can actually see their progression. Okay, some good questions there.
Bringing this all together we have covered quite a lot of ground here, we’ve talked about engagement, reinforcement, the role of content and technology in the design and the delivery, and we have also had a few hands on experiences. Just bring that all together now in a summary for some of the concepts to take away and they will go onto your homework not long after that. You’re not really getting homework but it is voluntary. Bringing that together, the guiding principles here for delivering learning in a way that leads to durable behavior change, firstly, make it easy, take large or complex subjects and break them into bite-sized scenarios that can be digested easily in the learning experience. Making it stick, really think about the delivery mode and methodology to deliver that learning in the way the brain actually learns and that’s often through retention, retrieval practice, recall and ultimately that then altogether drives ongoing behavior change.
Today, you’ve got distributed workforces, D&I programs that typically run across organizations that have multiple people in multiple locations, whether it’s in country, even in city or across regions and continents, so having something that is able to be easily deployed via mobile is really key for facilitating quite a set of standardized way of delivery but there may, of course, need to be some local or regional variations. We have a lot of customers that need to take core content and slightly tailor it towards their country or region throughout the cultural sensitivity or it could be local rules and regulation or compliance practice that they need to adapt that content to.
Make it engaging. This is really key and we talked about this quite a lot. So, using techniques, game mechanics, fun competition, peer socialization and really taking it at a quite a personalized approach to even coaching once the frontline managers have that data and finally making it measurable. We talked about that just in the most recent slide, but really using engagement and proficiency and changes, the actual change in those metrics across the learning program to see where you’re moving the dial but also to really surface gaps, those gaps and having an understanding of where your learners are lacking some understanding of certain concepts or even mastering them means that you can really hone your future training initiatives. Do I need to run a deep dive training in particular area? Is it just a gentle coaching action that a manager can be made aware of and then reinforce through one-to-one conversation? There’re many different parts that can be taken from there once you actually have that Intel.
Kari, I’m going to hand it over to you for the last few slides and a few takeaways.
Kari: Yeah. I think I was trying to make sure we’ve got some key takeaways for people to think about, so I think as we wrap up we want to leave you with a couple key ideas that you can take back to your organization and more specifically the big question being, how can you use technology to really help you achieve your organization goals and using them to then increase engagement, create inclusive workplaces and really drive behavior change? So, in terms of the workforce, what do you need to upscale or rescale employees to plan for the future and how can you use technology to help you do that; the workplace, what are issues keeping your senior leaders up at night and how can you help to solve those issues; and in the marketplace, how can you use D&I programs to build your organization’s brand in the marketplace and be able to help your organization thrive.
I know, for myself, I’m increasingly paying attention to what an organization’s reputation is around D&I and making buying decisions based on from that and I’ve had some fascinating conversations with a lot of people who are doing that as well. So, we know that diversion inclusion although mainly is a people focused is also linked to that incredible bottom line as well.
We’ve got one final other option for you here is we’re going to invite you to come take a spine on Qstream and do it on us. So, we’re going to invite you to take a mini course on us. You can design any content you want on Qstream. What we’ve done though is to create some D&I content around it so that people can poke around, explore it, see how these courses might work. If you wanted to use the courses that we have, you are welcome to do or we can customize them for you or help you create your own. The Qstream we’ve got here for the topics are up there right now and you can ask to find out for any of them or if there’s a couple of them that are of interest to you we can set them up for you so you could have a couple of them going and you get a chance to enjoy the touch and feel of Qstream and see what that interactive technology-enabled training begins to feel like for yourself.
Meredith: Great, so there’s your homework as I say not mandatory but it does give you a good way to get involved and try this out for yourself.
We do have a few questions here. We don’t have a lot of time left but I think we have answered some of them already. We’ll just go straight to these. If you’ve got questions do send them in. If we don’t have time to finish them all today, we will certainly take note of those and respond to you, so please do let us know who you are and we can answer you offline.
The first one is just in relation to the scenario and content. Do the scenario questions change in that learning program?
Kari: Yeah. You can create scenarios that you can use for a training program, say for a wide variety of people depending on what you’re trying to do but then you can also turn around and create scenarios that are very specific to a micro group of people. I think that’s the piece that depends on what you’re trying to drive for a training outcome, you’re trying to help great teamwork in general or you’re trying to help an engineering team to understand how to work with people from India, for example. In that case the scenarios you would create for both of those would be different. You can use it for a broader piece but you can also bring it right down into a very micro group of people to focus on.
Meredith: Excellent. Okay, just trying to work through these. Some of the ones that, is there a specific password to enter, get involved in the content? Yes, to those people we will share the recorded presentation.
Imelda here has asked a question around what kind of tools are being used currently to train D&I online? I think we can maybe address that from the earlier slide.
Kari: Yeah. I think from online you’ve got a variety of things from online training. It’s a static training people are moving through within some of that. You’ve obviously got some other technology platforms in terms of Qstream and things that are helping to drive it online and then I think webinars would be the piece as well that I’m seeing people use for online tool.
Meredith: Excellent. Imelda has also asked D&I training projected for the next ten years. I’m not sure exactly if that means in terms of investment or the context. If anyone does have that number, please do let Kari and I know. So, Imelda we might need a little more information to answer that one offline. Where are we?
Kari: I would also say too, though on that is that I’m seeing a bigger push now probably more than in the last 25 years I’ve been involved in the industry around diversion inclusion training and really working to say how does it align to the bottom line? What can we do with diversity inclusion training to help solve an issue that’s keeping our CEO up at night? I think increasingly the more diversity professionals we can align the work that we’re doing towards solving real-life business issues, the more we can get buy in from the rest of the organization for the work that we’re doing and the more that we can really help to drive that organizational change that we’re trying to drive.
I think the diversion inclusion training is going to get more and I think it’s getting more practical at the same time and I think it’s really getting more focused in terms of helping to achieve those critical organizational goals.
Meredith: Okay. That pretty much brings us to time. I think we’ve answered most of the questions there. If you have any further questions please do contact us and ask us. Kari and my details are here at the end. For those who did ask about exploring the Qstream that Kari mentioned earlier and if there’s a specific password, please do contact Kari and she will guide you through that. Otherwise, any final words from you today, Kari?
Kari: I want to thank everybody for coming. It’s an incredibly fun passionate topic to be able to look at how we can use technology to help us in the diversity inclusion space and I hope the ideas that we presented will be able to give you some new tools and concepts and ideas about how you can actually use tech in a very people centric field like diversity and inclusion to help you improve the bottom line.
Meredith: Fantastic. Well, thanks everyone for joining today. We’ve really enjoyed having you all online. Thanks for everyone who did contribute on the chat. I really enjoyed that too. Thanks, Kari, and we’ll close it off there. Goodbye everyone and have a good rest of your day.
Kari: Thank you.
END OF TRANSCRIPT