WBB's Thought Leadership

Mixed Methods Research for Quality Improvement

February 20

Research data in healthcare operations is often less structured and clean than desirable, and access to healthcare workers for surveys, focus groups, and interviews is often constrained due to work environments, staff workload, and other restrictions.

WBB Take:  Computer Assisted Qualitative Data Analysis Software (CAQDAS) systems used in mixed methods approaches with Natural Language Processing (NLP) tools can give valid and useful insights despite constraints. While Quality Improvement (QI) practitioners and researchers should always strive to acquire data that are valid, reliable, and complete from all value chain stakeholders, in practice this may not always be possible.
Rather than abandon projects in which access to stakeholder is limited or other data constraints exist, the agile practitioner can used mixed methods approaches with qualitative and quantitative tools to make the best out of the situation, and deliver usable insights for process improvement.
Cited by Matthew Loxton

Excerpt: “Most training in research methods tells us how best to sample, process, and analyze data. However, there are times when a researcher is faced with data-collection and analysis options that are far from ideal. In practice, there may be restrictions on how data may be collected and what data are allowed to be collected. In these situations, innovative use of tools can help to produce valuable analysis even when conditions are difficult.”
“In this article, a team of healthcare analysts and process improvement subject matter experts from the consulting firm Whitney, Bradley, & Brown (WBB) describe an approach to a constrained data environment. In this project, the WBB team used software tools to perform both qualitative and quantitative analysis while carrying out measurement and evaluation (M&E) of the deployment of a software application across a large healthcare system. For qualitative analysis, the team used MAXQDA, and for quantitative analysis, they used R.”

“Step 1: What does the sponsor wish to know?
We held several sessions with the sponsor to document objectives and to develop a list of high-level questions. The sponsor had a small number of specific questions in mind, but for the most part, had only general thoughts and concerns. Using the text from sponsor sessions (In place of audio recording, the researchers used stenographical means to capture direct quotes from the sponsor and user sessions), we generated topics and questions and used MAXQDA to sort and classify question fragments and concerns into distinct topic categories and questions.”

“Step 2: Research Plan
It was already known that surveys were not allowed, that interviews could take no longer than 30 minutes, and that participants would not tolerate a set of formal questions. Accordingly, the research plan was to use MAXQDA to associate text chunks from interview notes to codes already developed for each question.”

“Step 3: Interviews
During the 30-minute interviews, the researchers remained aware of the codes as they related to the interview content and allocated codes where participant stories aligned with a question code (In other projects in which recording, MAXApp could be effectively used in the field to record and pre-code segments in this way).”

“Step 4: Qualitative Analysis
The WBB team analyzed interview notes and direct quotations using MAXQDA to associate existing codes drawn from our measurement framework corpus of metrics to text segments. The measurement framework has been developed over many years of performing assessments of healthcare policy implementations, technology deployments, and workflow optimization projects.”

“Step 5: Sentiment Analysis in R
Once the team had classified text chunks in the partial transcriptions and interviewer notes from the semi-structured interviews, we used the R SentimentAnalysis Natural Language Processing (NLP) statistical sentiment analysis tool to quantify the degree to which the associated coded segments were positive or negative in sentiment. The R package provided a scale from -1 (strongly negative sentiment) to +1 (strongly positive sentiment) by evaluating words and phrases in the text against a dictionary of terms with given sentiment valence.”

“Step 6: Combinational Analysis
The final step was to construct a report that merged the qualitative analysis developed from the coded segments with the statistical analysis of the same segments. This enabled us to describe the meaning behind the text, give an interpretation of the participant’s voice, and provide a statistical inference of the strength and direction of their sentiment. The report described what the users were thinking with regard to the deployed software, whether this was positive or negative, and how strongly they felt about it as a group.”

Using MAXQDA to develop categories and match user stories to a sponsor’s research questions was effective in providing insight into how users perceived the deployment of a critical technology for patient management. R SentimentAnalysis provided a low-impact way to quantify user sentiment. The two together enabled a sponsor to easily identify the specific areas that needed attention, a means to prioritize them by how strongly users were likely to feel about them, and to note successes that were also strongly felt.”
Source: MAXQDA Blog



Quality Comes Standard 

We are known for our quality services, strong leadership and long-standing customer relationships. Read more >


Learn More



Our Country's Toughest Challenges 

We're looking for professionals in the fields of concept development to sustained engineering. Read more >


Learn More

Thought   Leadership 


Leading discussions by WBB's industry professionals. 

Browse our thought leadership pieces. Read more >


Learn More