Hallo, und Bott in scholarshipakis von Münich und willkommen zur Einhandlung Vести uns heuteあ sizes about the communication impact index a ai powered analysis of an organization's communication internal communication and the engagement impact it creates it's about a thorough communication analysis that enables organizations to evaluate measure and improve knowingly my name is elmar and i've been with implement for almost 10 years now and ever since mostly dealing with change communication projects that goes from large it implementations over strategy rollouts reorganizations onto speech writing and communication training always with the ambition to change the way we communicate in organizations away from disengaging corporate language towards more human language which is simple edgy relatable honest simply engaging but more on that in a little while i'm super happy that i'm not alone here today but my good colleague with me um christopher yes uh thank you elmer and good morning everyone um my name is christopher milling um i also work with change communication at implement uh i've been around implement for for two years now um and i come from a background also in in external communication in the communication agency and what struck me uh also is uh after i started implement is that even though i meet a lot of really ambitious skilled uh communication professionals in the organizations we work with um then there's still this tendency to you know organizations externally put a lot of care into the communication super engaging but when it comes to the internal that seems to be trapped somehow and um and yeah that was kind of our observations and experiences that led us to to the work that we're going to present uh for you all now um um um as i said just in my little introduction um we have done a lot of change communication projects uh in the past uh 10 years and what we see you know we have done that with a lot of different organizations with a lot of diverse group of organizations but what we see is sometimes quite monotonous uh in meaning that we often see that communication is steered by you know driven by conservative thinking um um of course of course an organization has to take into account a numerous amount of constituencies stakeholder groups that need to be kind of satisfied and keep on their side kept on their side so employees managers departments uh what have you uh so when something is broadcasted um and the amendment process and approval process is in effect often a test you know will this communication offend anybody you know and with that mindset in mind of course the messages are honed to be as safe and sterile as possible um which of course is unlikely to offend but also uh equally unlikely to mobilize and engage but you know that's actually we think pretty sad because what communication actually really could do is actually rock your organization really to unleash its full potential by engaging the employees and make the organization that unstoppable force it could be so like music you know so like music you know communication can bring us together you know fill us with energy move towards to a shared purpose um but that's actually not the case we think um um you can see the numbers here this is from our own survey um which maybe some of you have attended a webinar in in the past uh days um our our x-ray survey where you can see that only you know 30 percent uh of employees find internal communication engaging and only 50 percent find senior leadership communication engaging so these numbers come from our latest survey and they kind of confirm our our observations so the question is kind of you know why has internal communication been set on mute and maybe more important what can we do to change that yeah one thing is uh what we also can see is a big thing of the why is that uh communication is uh look Scoots christopher will talk a little bit more about that in a little while but what you know really doesn't get measured also really doesn't matter um otherwise you know in organizations we measure everything kpis all over the place but with communication we still feel it's a bit different and that's why it's always or sometimes really hard to to pinpoint you know where can we really improve what's really the issue here with our communication so to to fix the internal communication we really think we need to measure evaluate and improve and the communication impact index which we're going to introduce to you today will do just that yeah it is it is it quantifies communication impact it is a tool based and and and thus also scalable being able to to you know process large amounts of communication and it gives clear and actionable output right and then let's start to dive more into what that is all about um just a quick overview of what the communication impact index comprises of so it's a holistic analysis uh where we employ six complementary methods to evaluate the communication uh in our organization um a cornerstone here is uh the ai powered content analysis which we will be talking much more about and also uh demonstrating a new tool we've developed um that can help process large amounts of communication content and get a structured overview of how an organization communicates um but much more about that in a second we also do uh qualitative text statistics combining licks numbers and sentence lengths with something like sentiment scoring and um and topic analysis so there's a possibility of getting you know in addition to the to the ai content analysis a good input into what does our communication look like what does our communication look like on a structural level those are sort of the the the digital tools and the digital analysis parts then we also look at it from a from a crash perspective from an expertise perspective and because of course there are things where you need the human eye you always need the human eye to complement the ai um and we do uh case studies include that in this this approach here why because communication of course plays a role all the time in an organization but especially in those defining moments like big transformation we have a crisis or a crisis so um so we look at specific cases uh to get an idea of how does the communication work and engage and then finally of course we need the audience perspective to to measure and understand how is the impact and the effect of the communication so we we do interviews surveys with the audiences just as you would do to understand an external target group and figuring out what are their needs and their experiences for the communication and and finally also looking at the channels you have just as you would do in typical classic uh communication measurement figuring out when we put something out on our internal channels how are they uh how is the interaction the clicks the openings and readings so on and this is really our way of getting a comprehensive picture of the communication in the organization um the output looks something like this and and we'll get back to it but imagine you stand uh an organization stands with like a full data overview of the communication um from a content perspective from a channel perspective from an audience perspective and connecting that to the impact so this is really about seeing the the big picture uh on the data level but also breaking it down and understanding the different formats and contents we're working on so um that was just a brief overview now we'll dive much more into the the method and the thought behind it um and and maybe starting by by by asking how do we make communication measurement count for something um um um because what what we also observe uh is that we rarely or or maybe we dare almost to say never um measure internal communication um we've at least rarely experienced it and and the question is also why uh one thing it is that it can be difficult right especially if you want to really understand the connection all the way down from the input to the the communication you make up to the impact level how does it affect our organization this is a tricky and and um resource demanding process um resource demanding process um or it can be at least so that's one reason but but another underlying reason is really also we think that often when we try to do it when we see it attempt to it doesn't really create the value needed for this measurement to to overturn what what elma was talking about uh the politics and and the taste it's it's not always strong enough unfortunately and why is that well often uh the measurement focuses too much on the channels maybe includes the audience if we're lucky um but but getting the full picture of how do we create impact uh is is rarely done and difficult to do so that's that kind of the challenge we the the gauntlet we picked up here uh and and with the communication impact index are trying to to give an answer to and um there's like two two pillars in in how we do that one is that uh we started with a what we will say is a strong theory of impact revolving around employee engagement um and the second is that we we measure quality um quality defined as what works works and yeah um often when we say that we get a little bit the reaction quality isn't that in the eye of the beholder isn't that a subjective thing and we know that to a certain degree yes there is a matter of taste but this is exactly also what we want to challenge here and and help organizations challenge there is something that works there are known factors that affect that where communication can affect the engagement and and this is what we want to look into so and so so that's the the position we're taking this so let's start with the theory of impact um that's revolving around engagement so um when we when we started designing this we we asked ourselves the question where where should we focus um and an employee engagement was was an obvious candidate for several reasons first of all because there are quite well established links to to business performance financial performance we we know from many studies that um employee engagement positively affects uh how the organization is doing on on on something like productivity innovation and also profit profitability so so there we know there is a good link what we also know is that there is a pain in many organizations so depending on which study you look at something between a fourth and a third of of employees worldwide are really engaged in their work right and so there's a lot of potential to make that better and we believe that community um communication also given the numbers that elmer showed um that can really help in that pursuit so so can we understand how can we be better at understanding how communication can help towards that goal and what we then did is that we kind of backtracked and calculated backwards and said okay where what does come what constitutes employee engagement and where does communication have to play a role in that obviously there are things of of my engagement as an employee that is not dependent on communication um but communication definitely also has a role to play so we defined sort of six engagement drivers um where we said their communication is super important it could be my sense of purpose of being in a organization that has a clear direction and i've i can connect my own work to an overall meaningful uh purpose it could be trust and transparency um not just in my nearest nearest peers than my my my boss but also in the organization overall and the direction we're taking it could be a sense of care that i believe that top management uh is actually uh is actually caring about my perspective uh and what's going on on the people level and not just pursuing the numbers so these kind of things we know are connected to engagement and and that's also why we know communication can have a role to play and then we went back further and uh and said how do we then look at communication internal communication and and measure that and evaluate that on an input and an output level right so input is what should we do um and here the new thing at least uh we as we see is that we dive into this quality perspective and understand how we write how we communicate how does that affect on the higher levels um we also look at the activity levels so what how much and what is put out there and then of course the output asking the audience what are their perceptions how do they react to it um and also looking at the channels and the key question we're really trying to answer here in this theory of impact is is the communication engaging is it helping um driving engagement among employees then we went further and said okay um if this is our theory of impact what do we mean by this quality aspect um and and here you see the the framework that we've developed which is based on our experience with engaging communication and um what we see in research what works that's really the question what makes communication engage people and we've defined these 12 qualities um of engaging communication which are those that we assess in the content analysis and furthermore each of these 12 qualities uh are broken down into several subparameters so that we can really get close to and observe specifics in uh the communication and all of this is powered by ai and that we will show in a little while how that's done so first of all just the overall flow of of how such an analysis can be conducted we do it in um so first of all just the three steps the first is really starting with the strategic framing like looking at the organization that's important we need to take this general theory of impact and then develop hypothesis about how does that look in our organization um how is our engagement what do we know about that how is how is that connected to our performance our strategic challenges and and opportunities um and then start going backwards and looking at the communication and that's what we analyze in the second step so we employ the six methods that i showed in the beginning um getting that comprehensive data overview and understanding what's going on and then finally servicing back up in this third step here anchoring it strategically um looking at our hypothesis and also helping develop some recommendations how can we data driven turn communication into the engaging force it could be so yeah the first step was about the strategic framing um taking the generic theory of impact and asking a lot of questions about how does that look in this context um it's really about tailoring it a little bit to the organization which also has an importance in the next phase in analysis because of course the scope can be very different right in some in some organizations you might want to look specifically at a specific business unit a specific challenge or maybe you uh investigating leadership communication so there's a bit of scoping going on there in the first part of the analysis based on our theory of impact and what we know is uh is uh is a challenge or what we think is interesting to understand better then we collect the data we run it through our methods and then we go back into sense making and here it's really about looking across those six methods um because one data analysis uh one analytical approach doesn't give you the full picture here we need to to make sense of it become comprehensive picture and then look start looking at our impact case and understanding links between the communication level the engagement level and the impact level MR.잖아. already dive a little bit into point number two the the categorization of data of course when we have defined the scope of the data and that could be communication regarding a certain transformation it could be the total communication of the organization in the past five years it could be something very small like you know the intranet in the last week or something you know maybe that's not too small in some organizations but then when we have collected that data we we categorize it in in three categories um which is operational communication uh and then strategic communication maybe with a more long-term view on maybe more high-level perspective sharing the vision whereas operational communication we consider more being being on the day-to-day giving clear instructions you know steps one to five this is what you need to do a little bit more informative in that sense and then we have uh cultural communication which is a communication in terms of you know building the culture you know communicating the values uh making sure you know to to to make people rally around uh the same purpose and so on right and um that's what we do and why why is that important because of course we have we are not having the same requirements to the communication uh um you know no matter what kind of right probably operational communication does not need to be as aspirational and edgy or even dialogical than then maybe uh cultural communication needs to be or strategic communication needs to be so this is truly important that that of course we also can have quite plain simple straight to the point communication in the organization if it serves the purpose it needs to serve and then that's fine you know um so that's super important so this is also why we are weighing these 12 dimensions christopher has has introduced to you uh uh to to the point in the uh uh the the um and then that's the um this is important that you can see the um this is a pretty good idea um and then we go to to to compare comparison and sense making also aggregating all the data into into in the end in one index number but not only that of course um maybe a little bit you know how does that we go a little bit into how how does that analysis work what is actually happening in the ai tool and without going too much into detail there's at least one thing i would like to share with you here It is a quite sophisticated prompt-based analysis, which runs twofold, actually. So one is that the tool analyzes the communication, a certain dimension like vividness or simplicity, along four subcategories, which Christopher also quickly talked about earlier in this webinar. And looking at these four categories and then comprising out of how this communication does, out in these four categories, making an overall score for the whole dimension for this piece of communication. On the other hand, we also allow the AI to look a little bit broader and give it more space and freedom to look also beyond these four categories and not imposing that kind of rigidness on the AI tool. So we have kind of a both-and approach. Why do we do that? Because when we are only free-floating and let AI evaluate and assess, then we have maybe a little bit of a harder time in the strategic dialogue with the client afterwards to really pinpoint what is it exactly that makes it less simple or simpler. On the other hand, we do not want to constrain AI and take away the beauty of AI to also give it more space and explore more freely. And as one of the results of the output is not only numbers, but the tool also gives a very, very thorough overview over where in the communication we can see that a certain score is achieved and what maybe needs to be done in order to achieve an even higher score. So it gives a lot of information, right? But we are running this both-and approach to really have a more rigid one so that we can, you know, backwards engineer a little bit better, but also giving it the space and freedom it deserves and also unleashing the full power of this tool. Then when we have both the scores, it will be combined for each quality. And then in the end, we have a very rich output, which are the 12 dimensions, but for each of the 12 dimensions also, four subcategories, which is then in the end, 60 data points out of this analysis, plus then a lot of also qualitative outputs where it really gives you a lot of examples for each of the categories, where it does good, where it doesn't do good. So we can really have a lot of things to talk about afterwards and how to improve. Yeah. So I've been talking enough. I think we've been talking a lot now, 23 minutes, I can see. So let's finally see how it works. We start with a little piece of communication. It is an internal email that Elon Musk sent out to the Twitter employees when he got into his position as the new owner and CEO of Twitter. So let's see a short video. Do you remember when Elon Musk took over Twitter? As communicators of change, we do. In particular, we remember the fuzz that Elon Musk's first email to all Twitter employees created. Now, we thought it might be interesting to see what we could learn using the Communication Impact Index AI tool on this piece of internal communication. The email was rather short and to the point, and it was his first appearance as the new owner and CEO of Twitter. Let's jump to the Communication Impact Index tool. In the tool, we have an analysis tab, a data insights tab, another for prompt setting, and then a place to upload data. Let's upload Elon Musk's email. The data appears here, and we can see that it identifies the content as an email. All that is left for us is to select which type of internal communication this is. Operational, which refers to day-to-day communication. Strategic, meaning forward-looking communication. or cultural, or cultural, in the sense of transmitting corporate values. Let's select these and run the analysis. Now we have the option to explore the findings in Excel or go to the Data Insight tab. At the top, we have the communication quantified in the form of readability scores and number of documents. Below, we have the scores on each of the 12 parameters. We can see that honesty is high and vividness and dialogical is low. Both of these low scores we might expect due to the nature of one-way CEO communication. However, we also see low scores on appreciation, empathy, enthusiasm, and aspirations. These are questionable and might explain the negativity his email generated. If we prefer, we can also see the result in a radar chart here. Now, imagine what kind of insight this could generate in your company. So, now you got a glimpse of how the tool is working with a very prominent example, I would say. This has been an example for one piece of communication. So, let's look at another case where we feed the system with a little bit more communication. It is easy to have an opinion about a single piece of internal communication, but it is difficult to understand the impact of the masses. This is where the Communication Impact Index is powerful. It is scalable, meaning we can measure the quality of large amounts of content and get consistent results. Let's look at how. In the tool, we can upload vast amount of communication from various sources and then evaluate these along the three dimensions, operational, strategic, and cultural. When ready, we hit analyze and wait for the results. We can now download the results or view them via the data insights tab. At the top, we have the three categories. Below that, the readability scores and the file count. Further down, we have the scores along the 12 dimensions. We can now isolate each of the dimensions to see how the communication performs. The power of the tool now becomes visible. What we see is a communicative blueprint of an organization. With this insight, we can have open conversations about topics like, are we engaging our employees the right way? And, what areas of our communication could we improve? To not get lost in the averages, we can also examine the outliers via the box plot chart. And for a comparative view, the radar chart provides great profiling insight. Yes. So, another case with a little bit more communication. And of course, this is thinkable for even a lot more communication. So, yeah. Nothing happened here. Only our screen went out. Thank you. okay sorry for that uh little hiccup here the technical one um so the tool works the techniques here for the webinar maybe doesn't do so much i hope you're all with us still um as i just said you know um the case for for using that tool is also thinkable with a lot of more a lot more communication and what you have seen is of course only the first overview of the results downloading the report in excel gives a lot more data and a lot more rich overview of of the um the the communication we have two more examples with you just a few more examples here and and just follow up on that you said like for communication data nerd as for instance me i think that's like paradise to sit with those 60 data points across a lot of of communication content start breaking it down into different channels formats time periods you can do so much to understand what it looks like um but just just to um to show some examples of the scoring um here's an example we did uh where we because we've done a lot of testing with this tool to to make sure that the parameters the qualities are evaluated as we would like them to be that the tool is accurate um according to to our definitions and so we've we've played around and we tested and one thing we did was also trying out on different formats from for instance here a vacuum cleaning manual which is not your typical type of internal communication but nonetheless it's fun to see how it scores pretty much low on everything like we would rarely imagine that a manual for a vacuum cleaner is engaging and true it scores great on concreteness uh it's okay simple but but not super simple and then everything from aspirational to enthusiastic they are very very low but also here that could be okay you know if such a communication is needed in some circumstances but christopher what's with that honesty part here why is that manual honest yeah the the honesty uh so we look into what the the the the ai explained and uh and it actually fits with the definitions we've we've we've made of of honesty uh for instance the ai also says well it gets the score because it is uh fact based it's focusing on facts it's comprehensive it's really trying to get the full give the full picture not hide anything um and then last but not least it's um very very uh sort of explicit about potential problems or issues that could arise or updates that needs to come so actually the ai is is is is doing it right um um but this also shows that you really need to put the human eye on understand the context to get the right image out of these scores in itself it's not a good or bad picture we see here it's about does it serve the purpose does it work so when we not want to know if communication in an organization works in the sense of being engaging then we need to apply that look at the scores we get um and just to show another example uh a different output is here jim cook speech from his uh commencement speech at at stanford and there you see a higher scores on on aspiration and enthusiasm and all of that so it really makes a difference what kind of communication content we have and and these are of course single pieces but the really interesting stuff is that when we start looking across the newsletters from the last two years the ceo emails the strategic presentations and understanding how are we doing on that so um so um there was a demo of of the tool a deep dive into what it looks like and and the output that the ai analysis gives but as we've covered earlier this is just one part of the analysis and um the full output of of the communication impact in this um there's more to it one thing is that we um we develop uh a benchmark sorry an index score which is first based on a base score where we we take the evaluation of the ai tool compared to the interview scores so we so we do kpi scoring with the employees because their experience experience of communication is of course super important to understand what's going on and and if it's actually working and then finally we adjust that looking using the other methods and develop for civil level communication mainly try to develop an overall impact score um that is an indication of how are you doing compared to the potential you're pursuing um but i think um but i think for for many communication professionals um what really gets interesting um sorry my slides are lacking a bit um there we go is also the the the full overview that you can get out of this again the breakdowns you can do and look at different formats different channels different functions or categories of of communication to understand how have we done how are we developing and what should our targets be to create the engagement we want to do and when we have that overview um we can surface back up as we mentioned uh in the beginning because that's where it also uh that's the main point of of what we're doing here really and one thing is understanding our communication input the quality and the also the output level what is the reception but i really understanding how do we create engagement and how are we um affecting oral business impact and that's what we then do and then we take the data we take the analysis we take the conclusions and insights and try uh and try uh to to go back to the hypothesis about the theory of impact uh look at how how do we see that is it happening where are the links missing are we driving enough sense of purpose are we driving a sense of belonging could we do better on any of these things and that of course feeds potentials recommendations and so on at the end um so really with the communication impact index what we want to to enable organizations to do here at the end is we want to enable organizations to do here at the end is we want to make internal communication um data driven um data driven and also reclaim what works because we know there are a lot of passionate ambitious and skilled communication professionals out there that actually want and are able to do all this uh to do the engaging communication but often it just gets trapped in these politics and this tastes um and a little bit in the communication complacency so so we want to arm the organizations to be stronger and data driven in communication next um um we want to help position communication like as emma said just as we measure any other aspect of what we do in the organization i have kpis we like on our supply chain our sales organization anything communication needs to have that to to show the value that's created the potential and that there's a strategic approach to it um and here we think that the data image here can help support that we know there are always a strategic approach um but the data can make it much more sound and visible to also to a c-level then it's also about as i mentioned before challenging the complacency that can happen in our organization how we've always communicated sometimes we experience this little bit fear of bothering employees with too much communication now that's too much let's keep it simple and low but actually when we when we ask in our service employees misengaging communication as we saw earlier so the data can help us show that and also show exactly how can we be better and bolder how exactly and then finally all that we think that is really important and we think that is really meant to lead uh to one single thing which is about um turning the music back on in organizations um using communication as the music uh it could be helping you rock out there uh help us all all of us rock um and we think the communication impact is is a really important way to or a good way to to get that going um and challenge that complacency um so that's our presentation for now um there's time for a few questions despite technical glimpse um can we have a free discover and play version uh you can definitely uh have a demo um where we where we can play around a little bit and test it with a certain amount of communication so to to look at it a little bit deeper uh so demos are definitely possible totally yeah um there's a question of can the ai only uh can we have a really analyze text or also videos and images um that's a really good question so right now it's mainly text based but the the newest versions are actually uh of that's released from from chatty and so on are slowly allowing us also to evaluate other formats and so we think the tool will like right now we can for instance analyze video scripts and then also have to uh uh look apply the human eye but we believe very soon the the tool will be able to also look at videos and so on which is of course super important for communication analysis yes yes free demo please uh ranges don't seem too extreme one score is a 2.53 and another is two what does this gap tell us if the range of data is not wide yeah that's a super good question um and something we've been working a lot on so with averages we will tend to see small gaps uh unless of course it's very extreme because there is often a consistency in how we communicate so where it gets interesting is course for instance we saw the video that we have the box plot so we can see the spread across the content that we are analyzing and that helps us understand okay is this really an average that's uh covering up a big spread and what does that then tell us or is it very very precise um and of course if if all our scores are around two two and a half then it doesn't tell us much so it's about breaking it down and working with the analysis what we would say is also that if we see like very average scores similar scores then probably communication is not really doing what it should right the way we have to do so the way we have to do so the way we have to do so the way we have to design this framework is to to see how can we push and be be sharp and edgy in how we communicate and if we always score around two then maybe we're not edgy enough maybe we're not aspirational enough um so there's so the short answer is you need to like further into what the data underneath the averages are to really understand it can the ai tool integrated for example in the internet based on sharepoint and does everything have to be or does everything have to be uploaded in a word document so i think for now um the integration we are we are working on things like that and constantly develop the tool right now we need to upload the communication but it can read several files doesn't need to be a word it can be of course acrobat uh it can be powerpoint it can be presentations um so all kinds of files um but of course we're continuously working on that too um to to also have the video analysis but also also integration methods and ways of doing um um will you share the slide deck that's uh quickly yeah we will share a selection of the slides after um so yeah and then is the tool google suit compatible um that we have to to to ask our programmer for um so sorry to to have put a pin in that one um not sure good good and sorry for needing to look down here because our big stream that kind of uh kind of failed us failed us yeah but we manage i think um um there are not any more questions for um there are not any more questions for now so um what we really can offer is to have a a conversation and and and demos and and you know test test it out with your organization's communication uh um you know feel free to contact us get in touch um if you like um we're really happy to to help and support um and looking forward to yeah to good conversations around um around your organization's communication and how the tool could support you in improving on that thanks a lot from outside and uh thanks for for chipping in and then visiting this webinar and have a great day uh all the best uh from here from munich thank you bye-bye