KPMG Ignition IBC Knowledge Pathway
Developed with University of Illinois Urbana-Champaign Illinois Business Consulting
Glossary
Further considerations
Final results
Analysis and compilation
Primary research
Secondary research
Methodology and research
Background
Home
Let's begin
Knowledge is power. At KPMG Ignition, we work with clients every day. Our experiences give us a trusted, front-row seat to a client’s issues, challenges, aspirations, and needs. We aspired to create a better way to generate valuable insights for our clients in real time, allowing them (and us) to see horizontally across organizations, within verticals and across CXO peer groups. This required us to build a bridge between the traditional, tactile design thinking methods and tools leveraged in our experiences—such as sticky notes—and the outcomes—synthesized themes, actionable insights and roadmaps. “Can we build it?” was the easiest question to answer. We leveraged artificial intelligence (AI), machine learning, natural language processing, and OCR to build our proof of concept, “SCRIBE,” and create a scalable approach. The other dimensions of uncertainly around this issue required true experimentation. Enter, University of Illinois – Urbana-Champaign Illinois Business Consulting (IBC). IBC helps students gain real-world, project-based consulting experience while providing us with the fresh thinking and outside perspectives we needed for this experiment.
Next
Back
How can we effectively construct a list of keywords related to the constructs of Innovation, Aspiration and Concern— while considering the nuances of human psychology and binary nature of Data Science- that can be utilize to understand the mindset of KPMG’s clients while using the advantages of SCRIBE?
Hypothesis
IBC’s team was introduced to KPMG Ignition’s key methodologies and objective, and ultimately “SCRIBE”, our tool. They were provided with a hypothesis (see right panel). If sticky notes are a crucial input into design thinking, our hypothesis was that sentiment analysis could provide valuable clues that translate sticky note word choice into a client’s mindset. We relied on cognitive and data science in order to understand how humans perceive emotions, concepts, and thought processes—how they are replicated using natural language processing (NLP), optical character recognition (OCR), and machine learning in order to build a logic and an ontology.
IBC utilized a three-phase structure for this experiment.
IBC conducted secondary research that involved examining KPMG Ignition history of Ignition client experiences, surveys, and analyses. The team explored areas of cognitive and data science to understand how humans perceive emotions, concepts, and thought processes, and how they are replicated using natural language processing (NLP), optical character recognition (OCR), and machine learning. The team discovered the processes used in the realm of ontology, which provided them with the knowledge to efficiently define three words of interest into constructive networks of definitional understanding. Research results and definitions are provided on the right.
Measures client doubtfulness and anxiety
Concern:
Alignment between proposals and client expectation
Aspiration:
Measurement of the novelty/uniqueness of ideas
Innovation:
Concern
Aspiration
Innovation
Client experience surveys Ignition processes and methods Industry analysis Scholarly articles and literature review Procurement of initial data sets
• • • • •
Process of secondary research
1.
To better understand how the three key concepts relate to Ignition’s mission, methodologies and clients, IBC conducted primary research. It consulted experts in various fields and learned that analyzing language and sentiment is complex and requires a human touch that AI can't replicate. It also discovered that word frequency alone isn't enough to measure importance, and context is crucial. Based on this information, the team created a shortlist of potential keywords that accurately represent their concepts into defined networks of understanding. This practice— referred to as Nomological Integration— is utilized within psychometric analysis and ensures that scientists can clearly place and expand their definitions into networks of what their words are describing and what they are not for precise and accurate measurement during their research and methodology.
• Useful • Idea • Creative • Special
• Quality • Spur • New • Neat
• See • Try • Think • Wish
• Aim • Believe • Hope • Want
• Wish • More • Slow • Feel
• Doubt • Boring • Worry • Fear
Review of ignition processes and methods SME/Professional interviews Survey conduction Research of NLP, psychology and cognitive and data science
Process of primary research
2.
• • • •
Amid the multiple factors at play when determining sentiment, perception, and intention of word choice, IBC began keyword curation based on the following:
The prevalence of faint praises can be reduced, but not controlled for.
Interview feedback:
Words with direct connotative meaning are better at measuring sentiment.
Confidence rating:
The context of keywords in their statement can change their sentiment.
Contextual differences:
Keywords that are business specific but not considered “lingo” are stronger predictors of sentiment.
Industry knowledge:
There is no correlation between the frequency of word use and the strength of word sentiment.
Frequency versus strength:
Anonymity of feedback can be helpful in providing confidence in the sentiment of keywords.
Anonymous understandings:
Conclusion statements of research Findings Word Frequency and Strength Analysis IBC Group Voting Curation of Final List of Keywords
Process of analysis and compilation
3.
To refine its list of keywords, the team analyzed word frequency and connotative strength. It found that words related to aspiration needed further analysis, so inverse word frequency was explored to validate their effectiveness. The final list of keywords was selected based on “true” word frequency and other factors outlined from its previous research. To ensure the words were meaningful, the team used a strength analysis inspired by the practice of TF-IDF, which is commonly used in data science. After this analysis, the team selected the top five words for each construct with a final vote to ensure their alignment. See how the sorting of selected words compare.
Sorting selected words based on frequency
The final recommendation included the three constructs, and their definitions and a list of dynamic and static key words that are to be used by KPMG Ignition to understand client sentiment during sessions.
Understanding the reasoning behind key words and their underlying meaning
• Clients are driven to use innovation when they want to set themselves apart in the market. • Capitalizing innovation brings in novelty & enhances business processes. • Apart from product invention, innovations span across HR, financial adaptability and leadership.
Key takeaways
What does innovation mean—a term without a clear definition? • Factors that influence a client to use innovation relative advantage, compatibility, risk relative. • Studies show that innovation creates opportunities for sustainable growth with new market spaces. • In a survey, the words most associated with innovation were growth, creativity and new thinking. • 50% of people think innovation is problem-solving.
• Clients strive for competence and relatedness when setting goals for their company allowing for growth. • Clients who focus solely on profitability development may cause risk and harm and future evolution should be considered. • Extrinsic company aspirations does not inherently provide positive mental health to workers.
The path taken: consequences of attaining intrinsic and extrinsic aspirations in post-college life. • Psychological need for autonomy, competence, and relatedness drive intrinsic/extrinsic goals. • Focusing on intrinsic development generally leads to less risky actions as it focuses on internal expansion. • Attainment of extrinsic goals has no connection with psychological satisfaction.
• Could be correlated to dissatisfaction, worry, and lack of control. • Clients’ concerns tend to aggregate when they face unclear goals and high expectation. • Concern, emotion and passive attitude could be detected by NLP from clients' feedback.
People feel concern when they are uncertain/not confident with the future. • Usually arises with slight stress, fear and doubt. Could happen when clients perceive challenges and hinderance. • Concern is generated when clients perceive gaps between expectation and reality. • It could be detected by NLP when negative attitude is displayed in certain topics.
The students have developed a framework that connects the fields of cognitive science and data science. To bring this framework to life, we now begin the work of applying this framework to an actual conceptual model to validate the understandings from our initial recommendations and hypothesis. This will involve collecting a large amount of data, starting with an external sentiment lexicon (a data set of words and their attributed feelings). We will also feed as many sticky notes as possible from our client sessions into the model, then use the sentiment lexicon to understand the sentiment behind the words that the students have chosen. Based on this model’s output, we will be able to identify the attributed sentiment of the words used by our clients during our sessions. From here, we will also create a survey to validate this attributed sentiment relative to how the words are used in sentences.
Example: On a scale of 1-5 based on how innovative, aspirational, or concerning are the following sentences: • "I'm unsure about my data." • "This is really insightful." and • "We are in the market to be disruptive." By comparing the results of the survey to the analysis from the model, we can refine our hypothesis from the beginning of our project to understanding how our constructs do (or don’t) represent how people use the words on the left, relative to the word and their definitions on the right. Once we have completed these steps, we can combine the word lists and constructs into a visualization to better understand how the framework that the students have created can be used to bridge the gap between cognitive and data science.
More
Construct: An idea that is created, defined, and agreed upon which aims to bring meaning to observable objects or beings. Ontology: A school of thought that aims to classify and explain the nature and understanding of concepts, entities, thoughts, and matter. Nomological Network: A map of a set of concepts in an area of focus with their traceable and observed relationships between them. NLP: Natural Language Processing; A extension of Al that can intake text (optical character recognition (OCR) or direct input) and put it into understandings that a computer can process and transform for further input to a model and by extension, is output from the same model.
TF-IDF: Term frequency-inverse document frequency (TF-IDF) is a numerical measurement that reflects how important a specific word is to a body of work in relation to a collection of works it consists. This statistic is computed by taking term frequency (how many times the word appears in a singular work) and multiplying it by inverse document frequency (how many times a word appears in a collection of works).The TF-IDF is used by data scientists in many applications but is an integral part of sentiment analysis of texts.
Example: We have a large set of sticky notes that were compiled from an I like, I wish, I wonder design thinking exercise that occurred after clients were shown the intelligent forecasting story. We want to rank the individual sticky notes by how relevant the words in the phrase "Innovation is disruptive" is to them, based on the TF-IDF. We randomly pick one of our 5000 stickies
and see that— out of the 15 words on the sticky— the word “innovation” occurs once, “is” three times and “disruptive” does not occur at all. But out of the 5000 total sticky notes, innovation is seen on 135 stickies, is on 4865, and disruptive 0. The TF-IDF for those three words would be .24 for innovation, .0054 for is and 0 for disruptive. The word innovation would be the strongest word. with innovation next, and disruptive being weak as it does not have a statistic. Ideally, stickies that use the word is would be removed due to not having a direct connotative relationship, so innovation would be the most important word.
