Research methods in innovation.

Types of research – a deep dive into case studies.

Quantitative

Quantitative research uses a statistical interpretation of, among others, observational data using predetermined methods. It is the type of research that is most published by medical journals but is by no means the only one.

Qualitative

Qualitative research is more used in the social sciences but is also relevant in medicine, especially to innovative projects. With qualitative methods, the questions are more open-ended, and data arrives in a less structured format. Interview data, documents, and audio-visual data are all used to bring out themes, patterns, and interpretations. As a clinician, you can immediately see how qualitative research is relevant to medical projects. Patients are more than a list of blood test parameters or diagnoses. This type of data represents not patients, but people with fluctuating disease, positions in society and their families, and a reality that we know is fundamental to understand for any therapeutic intervention to succeed. When innovative projects fail, it is rarely to do with the technology itself, but because these “non-clinical” factors are not taken into account.

One of the research methods used in qualitative research which you may have come across is case studies. In 2006 Flyvbjerg published an interesting paper about case studies and popular misconceptions or even negative attitudes towards them.^1. I will go over these misunderstandings because you may find you also had these misconceptions without realising them.

1. Theoretical knowledge is more valuable than practical knowledge. This refers to how people learn. Yet experts become so by working on many case studies. A fact recognised by Harvard since the 1980s when case study based learning was introduced. Kuhn argues that “a scientific discipline without a large number of thoroughly executed case studies is a discipline without systemic production of exemplars…and a discipline without exemplars is an ineffective one”. Thus, Kuhn concludes that social sciences need more good case studies. I would argue that this also extends to innovation in digital health and medicine generally.
2. One can not generalise from a single case, the single case study cannot contribute to scientific development. To refute this argument, we need go no further than Galileo’s rejection of Aristotle’s law was based on a conceptual experiment and only later, if at all, on a practical one. Even if he did actually carry out the experiment, it is controversial that he used the leaning tower of Pisa to demonstrate that objects fall at the same speed, independently of their size, as Aristotle had previously argued. No RCTs were carried out to prove this. Case studies can be used to generalise the type of test that Popper called ‘falsification’, a clear-cut test he illustrated with the ‘All swans are white’ example. Popper said that if only one swan were to be black, the proposition would be false and needed further investigation. Flyvbjerg uses this to argue against the misunderstanding that case studies can not be used to generalise by identifying case studies as a method of identifying ‘black swans’ because of the in-depth nature of case studies.
3. The case study is most useful for the generation of hypotheses – other methods are more suitable for hypotheses testing and theory building. This misunderstanding is based on the previous premise that you can’t generalise. So, if that is invalidated, this misunderstanding too is invalidated. Flyvberg maintains that although useful for generating hypotheses, case studies can also be used for other parts of the research process. It is also true that atypical or extreme cases often reveal more information.
4. The case study contains a bias towards verification. Understood as a tendency to confirm the researcher’s preconceived notions, the study, therefore, becomes of doubtful scientific value. Bias is something inherent to all researchers, no matter what method they use. Even Charles Darwin developed his own method of recognising this ‘I had . . . during many years followed a golden rule, namely, that whenever a published fact, a new observation or thought came across me, which was opposed to my general results, to make a memorandum of it without fail and at once; for I had found by experience that such facts and thoughts were far more apt to escape from the memory than favorable ones. Owing to this habit, very few objections were raised against my views, which I had not at least noticed and attempted to answer.’ Case studies have their own but different rigour of methods which means researchers using the case study method have been led to change their initial hypotheses. Indeed, in other methods, the subjective bias may be across many cases rather than one without being challenged.
5. Often difficult to summarise specific cases studies. This final misunderstanding which has been used as a criticism or limitation, is for the fans of case studies a sign that the study has uncovered a particularly rich problematic for which a summary is not desirable. Nietzsche explains, ‘Above all,’ he says about doing science, ‘one should not wish to divest existence of its rich ambiguity’ (emphasis in original).

The choice to use a case study is often because it allows multiple and emergent data collection methods.²

Mixed methods – the best of both worlds!

Mixed methods use both predetermine and emerging methods from a pragmatic point of view. Multiple forms and levels of data are used which include both statistical analysis and text analysis. Critical interpretation across databases is obtained, bringing together the different research traditions to benefit the patient. So, if you have an idea for a research project or your innovation throws up a question that doesn’t fit neatly into a box or method, perhaps it doesn’t have to. Mixed methods are, for me, the best way of looking into the reality of patients as people and, therefore, what makes them tick.

Lessons to be learnt from qualitative research methods and the social sciences.

1. The importance of categories.

Although we may be aware of obvious cut off points such as age which has led to many trials excluding a large proportion of the population, there are other aspects to the categories you may choose when researching innovation. Applying a category gives power to the person deciding the categories, especially if it is imposed. Choosing to associate with a category may give back the power to the participant, but you have to be clear that this inherently gives legitimacy to certain viewpoints. And silences. Putting someone in a category is an action of power, but so is the ability to remove a person from a category. If you think about yourself, you might not want to be always classed as an iPhone user, or as an iPhone user to all the people you meet. It is a frivolous yet serious example if you start thinking about other categories you are placed in, such as hypertension, marital status, and weekly alcohol intake.

Age is a tightly coupled category, but some categories may be more transient, such as mental status and have real implications on access to certain services or stigma. When you want to capture or even market to innovation users, taking into account the categories you choose and the people you silence is vital.

2) Logics of inquiry

There are generally three logics of inquiry or ways of thinking about the research you wish to undertake or have been undertaken. Again, being aware of these will make you recognise your own biases and find the line of inquiry best suited to your innovation.

An inductive line of inquiry starts bottom-up with the people’s own subjective interpretations. These lead to questions and hypotheses to investigate, which in turn are confirmed or refuted. The end product of an inductive inquiry is a theory. You may well find yourself taking this approach based on your experience with an innovative product and experience.

A deductive approach is top-down, that is existing empirical work gives a set of statements. The researcher then applies the hypotheses and generates a new hypothesis or question.

Finally, the abductive line of inquiry is driven by a social phenomenon. It tries to explain why something is happening now and in what way. Here the researcher looks to find common processes.

How do you know which study is good?

Most medics are used to doing critical appraisals of scientific journal articles. Everyone has their own method, but for those who aren’t used to doing this, the Oxford-based CASP team have a checklist you can work through. If you are interested in hearing how clinicians debate these papers and the questions, try the Resus room papers podcasts—all links are in the podcast notes.

Suppose you are thinking of presenting research or presenting information. In that case, you can assess your own work before going in front of the lions with the PROMPT criteria in which presentation, relevance, objectivity, method, provenance and timeliness are all looked at. The PROMPT criteria are as relevant for researchers as for marketers and innovators coming up with a new idea. If you know the criteria your audience or buyers are using, you can prepare yourself better.

Finally, when you get involved in a discussion about research, recognise when what you see is “ad hominem”. A critique is directed at the person producing the research rather than the work itself, attacking the messenger, not the message. “Ad hominem” attacks are especially relevant as regards COVID research in areas such as vaccinations and face coverings where emotions run high.

What sort of research is needed in innovation?

Why?

Research is due diligence by another name, especially in the early stages. Further down the line, it can be seen as auditing or market research systematically. You are probably already doing it but perhaps not with the best tools.

Who by?

Yourself. As just mentioned, you are probably involved in some sort of research, whether you realise it or not. You can also outsource to people who have already published on the matter. They may be interested in collaborating, and at the very least, can direct you to others in the field. A quick search in PubMed can lead you to the people you need. Perhaps, you need to think outside the box and go to social scientists rather than traditional medical researchers. Social scientists may have a research approach more suited to the world of innovation in digital health and other spheres where mass RCTS or randomised controlled trials can’t be carried out. You can find these researchers in the sociology department of any university, and many will already be crossing over with psychology and other health sciences. There is a loss of knowledge when people research in geographically different spaces. You only need to look at the canons of Arabic medicine being reinvented and “rediscovered” in the Middle Ages in Europe. It is still going on when those involved in healthcare don’t look at what is happening in spheres that are not scientific as per their understanding of the concept.

When?

So, when do you need to do the research? Early on is the answer. Or at least it is important that you think about the potential research your treatment or innovation can lead to so that you collect all the data you need, following GDPR, of course. A retrospective study, looking at data collected previously, may not be as prestigious as a RCT. However, as we have seen, if you know the limits of each research method, it can still be valuable.

Another option is if you are aware of the ongoing multi-centre trials, you may find something you can jump on. For example, if you have a point of care blood coagulation device, you may want to look at centres that are taking place in the CRASH-4 trial. On their website, they explain ‘the CRASH-4 trial aims to provide reliable evidence about the effects of early intramuscular TXA on intracranial haemorrhage, disability, death, and dementia in older adults with symptomatic mild head injury.’ Prehospital knowledge of blood coagulation may well be incorporated into later treatment algorithms for mild head injury. Talking to those involved in this type of research gives you a foot in the door to a specific clinician population who have a special interest in haemorrhage.

Take home.

Different research methods have their uses, perhaps more than we have previously thought when it comes to case studies.

Learning from the social sciences and using mixed methods can fill the gap in innovation research and implementation.

Remain critical of the research and yourself. You are already a researcher with bias, whether you realise it or not.

Innovative medical devices on their own are not enough. They need to be validated and integrated into the health system and the lives of patients.

Resources.

Five Misunderstandings About Case-Study Research. Flyvbjerg B

CASP Critical appraisal checklists.

Resus room podcasts discussing papers.

PROMPT criteria.

1.        Flyvbjerg, B. Five Misunderstandings About Case-Study Research: http://dx.doi.org/10.1177/1077800405284363 12, 219–245 (2016).

2.        Paparini, S. et al. Evaluating complex interventions in context: systematic, meta-narrative review of case study approaches. BMC Medical Research Methodology 2021 21:1 21, 1–22 (2021).