Hello, and welcome to one of a series of podcasts exploring key issues or areas of interest in impact evaluation today. We hope you enjoy the podcast, and please don’t forget to tweet your thoughts at #impactframeworks. Thank you for listening.
Hi. My name is Chris Carrigan. I am the lead for patient and public involvement for DataCan. DataCan is the health data research hub for cancer, a UK-wide partnership that is unlocking the power of health data to improve cancer. Our aim is simply to improve care and outcomes for people with cancer by making high quality health data available for cancer researchers and health professionals. My role is to lead the patient and public involvement role within that.
Hi. My name is Mark Taylor. I work for the National Institute for Health Research. But I am also a long-term multiple sclerosis sufferer, for over 19 years.
My name is Trish Greenhalgh. I am a GP by training, and I am also a Professor of primary care and health sciences at the University of Oxford.
So, welcome everybody to this podcast. Which is of particular interest to me in the role I play within DataCan, which is to ensure that the patient voice is involved in everything that we do including recruiting, planning, prioritisation, approval, reporting. In fact, every part of the program. An aspect which is becoming much more to the fore as we see more patient involvement in many areas is the role of patients defining the impact of research. From a different angle, how we actually measure the impact of patient involvement in research. So, to help cover some of these things, maybe I will ask Mark, to start with. Mark, how have you seen the role of patient involvement or PPI, as we might keep calling it, in research develop over the years.
Thank you. I think it is something we have been looking at for quite a while. And it is a question that was first posed by Simon Denegri who used to run Involve, about two or three years back, when his group was working on introducing the national standards for public involvement in research. And one of the questions that that posed, or that work posed, was: What is the impact of PPIE so far? The problem with that was we couldn’t really work it out without creating some sort of baseline, without doing a historical dive to what had happened in the past and what people had expected when NIHR was set up and pushed PPIE as such a large part of what they did. So, a couple of years ago I contacted Trish in Oxford and asked her to look at historical review of PPI policy from the NIHR perspective.
And Trish, what did you find when you did that review?
I don’t want to oversell this, but my team – as Mark said – my team were contracted by NIHR to do a sort of deep dive into the history of PPI as the NIHR got involved with it. And I think they did incredibly well. It is largely a very positive story. This was taken seriously right from the outset of the NIHR, and in fact, it doesn’t begin when the NIHR began. And that is about 12 or 13 years ago, I think, maybe a bit more. It began more than that. That even before the NIHR was set up in the UK, the research funders and research organisations in this country were very keen to involve patients and the public in research. And I remember in the days when we were allowed to go abroad to give talks, people would come up to me and say, “You guys are doing PPIE really well in the UK.” So, I would say, although I have published stuff that is critical about the way PPI is done, and there are certainly many, many ways in which we need to improve it, broadly speaking, I think this is a very positive story. So, as we have already heard, in recent years, there has been a framework of standards. But I think way before that, there has been some excellent research on PPI. Sandy Oliver’s work springs to mind – I think that was published in 2004 – where they were looking at the literature interviewing patients and the public about their involvement, and very quickly picking up a number of themes. Such as, for example, tokenism, representativeness, whether or not the processes were truly democratic, or whether someone had sort of picked their favourite patient who was assumed to be trusted not to rock the boat to sit on committees. You know, what do we do about conflict, what do we do about the patient who really believes something very different from what the researchers believe about the illness, for example? Or do we want to just pick patients who share our views on the topic. And so, those sorts of issues around conflict and democracy. [unclear 00:05:30 – Learnt a lot?] right from the beginning. But I think what has happened in the last 10 or 15 years is that there has been a systematic kind of arrangement of those various themes and issues. And there is now quite a good evidence base on it. And I suppose the last thing I would say before I hand it back to you is, I think there are two broad perspectives on this. One is a rather technical approach which says, “Look, here is a list of things you have got to do. And we want you to tick all the boxes to prove you have done it.” And there is, if you like, a more critical approach, which is to say, “We need proper power sharing between patients, service users, members of the public, and researchers.” And that is going to be uncomfortable. And it is not just a question of how many bums on seats you have got when you have a PPI meeting. So, I think that tension between the technobureaucratic approach, and the – what I call the democratic approach – is a fruitful one to explore.
That is interesting. And you talked about the systematic arrangements which are being put in place. Do you think there is a danger that they may become tick box, because it is systematised. There is a series of check sheets you go down. Do you think it could actually increase the danger of PPI becoming too tick boxed?
Well, I think, yes. And I think that is why people sometimes roll their eyes when you talk about PPI. It is a box that everybody has to fill out whenever they write a grant application. And in some ways, it can become a bit mechanical. There is another box with the information governance in it. And it feels as if, “Oh goodness! This is just making work for me.” On the other hand, that slightly bureaucratic, visible dimension of PPI actually does prompt you to think hard about it and do it properly. And I think it also manages to weed out research teams who really haven’t thought about it at all. But yes, I think we do have to have-, you know, we have to reflect on the meaning of the bureaucratic tick boxes. It is not through the tick boxes that you get good PPI, of course, it is through the relationships and it is through actually caring about it. On the other hand, I am not as critical of the tick boxes as, let’s say, I used to be.
I think there is a dynamic tension, actually. I tend to call it managerialism. When something becomes managerialism, that is when you have got the problem. Because people do it without thinking about it. And I think the issue is not just tick box on the side of academia, the applicants for funding, but also on the side of those who manage the funding. There has to be an ethos on both sides of the fence. So, funders who come up with a system which is-, there has got to be a separate path on the application form that will get filled in – but they themselves not having an understanding of what PPI means in practice, as an ethos, I think that, by definition, becomes managerial and backfires as well. PPIE should be an ethos for a funder. It should be something that drives the funding forward. There are many terms like ‘patient empowerment’ which sound like a cliché, but they have real meaning. And if you forget the fact that patients feel empowered, and feel included, and feel part of the process, then no matter what you do, no matter what your best intentions are, it will be a tick box, and it will remain so.
If we are talking about impact measures generally, you can actually measure a tick box. It is something that is easily measurable, which is a potential risk. What are the other measures of impact that you have come across in the NIHR side, Mark?
It is a question I get asked in general anyway, how do you measure impact? And I would give the same answer every time, which is, you don’t measure it, you describe it. And then maybe some numbers to back up that description. But you describe it. When you are looking at the impact of anything, PPI notwithstanding, is you are looking for case studies of people who can describe what the impact has been for them. So, if you are looking for the impact of patient involvement in research, you need to have the patient story. You need to have evidence that patients do feel that research has moved for them. That the process has changed for them. That the outcomes of research are more relevant for them. So, NIHR, like a lot of funders, do an awful lot of patient involvement and engagement work. It is a serious part of our DNA. From having patients involved in peer review, and even before that – research prioritisation. It is part of everything that we do. But I would always caution against trying to measure it in terms of numbers. Because that then adds to the potential of it becoming a tick boxing exercise. It is a tick box to apply, it is a number you put in when you do your monitoring. That is not necessarily showing any form of cultural change or any ethos change.
Yes. It is interesting when you are talking about the patient stories, because the experience I have had with patients in some of our areas is they are very, very keen to tell their stories about how they found taking part in the research for them. And by and large, I think I would agree with you Trisha, it is seen by people to be a broadly positive thing. There is a separate angle which springs to mind though, which is, how do we measure the societal impact of patient involvement in research, rather than just the individual patient involved. Is that something we need to think about?
The majority of patients who are involved in our research are involved not just for themselves and their family member who might have a particular disease, say, but at a more abstract level, they want to bring the patient/lay voice more generally. And I mentioned earlier, this whole question of are they representative? You know, take me. I am a white, middle class person in their early sixties type thing. How can I represent all patients with whatever illness I might have? I can’t at one level. I am not statistically representative on men, for example. On the other hand, in my experience, patients who are on steering groups, who come to patient discussions – which are often online these days – they do try to bring in not just their own experience, but the experience of others with the illness. Because they are often in touch with other patients, they don’t just know their own story, they know other people’s stories. They might have a parent or grandparent with the same condition. They might be involved in a national patient forum, or something like that. So, I think this idea of the individual patient versus the societal element, they are on a continuum. And that is actually one of the things that I love. I am reminded of when I used to sit on NICE. One of the NICE appraisal panels. They used to have patient reps on those. And the patient rep would have-, they would have lived experience of one or more diseases. But they were actually very interested in whatever disease and drug we were talking about, saying “what is the impact of this for the patient?” Do you see what I mean? So, I think patients bring something much richer than sometimes we give them credit for.
I also think, actually, if you take into account that we live in a multimorbid world, that – from my point of view – you are rarely going to be talking to somebody who has just one condition half the time it seems. I have multiple sclerosis, but I have two or three other things that are wrong with me on top of that. I know-, because Trisha is right. If you have condition A, you tend to know a lot of people with condition A. You know, I know a lot of people with MS who themselves have two or three other conditions at the same time. So, most people have a broad experience. I think there needs to be more work in ensuring that patients involved in research are as broad as possible in that sense. But I do think that we need to take into account that individuals are rarely self-siloing, if I can use that terrible term. And by definition, they are constantly reaching out to their tribe because that is the way that everyone lives. Even in today’s pandemic world.
There is another issue here, which something Mark said raised for me. Which is that lived experience of one illness can often transfer to another one, in that we are all working in the same healthcare system. We are all, as patients, trying to access the same system. So, for example, years ago my dear departed father-in-law had been called for a diabetes check and he was told to see the nurse. And he couldn’t believe that having been registered with the GP practice for 20 years they would fob him off with a nurse. So, I did a bit of work and said, “Well actually, in practice, nurses are pretty good at looking after diabetes” and he went along and developed a wonderful relationship with the practice nurse, who guess what? Was at least as good as the GP. The point is, when he got his next illness, and there was a nurse-led clinic for that, he was falling over himself. He couldn’t wait to go see the nurse because he already knew that nurse-led chronic disease management clinics were a good thing, and he would be well cared for and well supported, and he could ask the nurse anything he wanted. So, the lived experience of one chronic condition kind of teaches you things about the system that can transfer across to another chronic condition. And the same, of course, is true if you are a parent and your child is having an acute illness. There will be a similar approach to another acute illness. So, I think we shouldn’t be too disease focused about this. I think the illness journey can be very similar with different diseases.
Clearly, if we are looking across the health system, what we need is impact for all. Do we think our PPI itself is impactful enough on reducing inequalities by being overly exclusive to types of patients who get involved? I.e., are we open enough as a PPIE function to all types of people from all types of backgrounds?
One of the really good things about inequalities in PPI is, even though we haven’t got it right, we sure know that it is the problem. And I think the NIHR is very reflexive about it, and I think researchers are very aware that the low hanging fruit is not the only place we need to go when “doing PPI”. What I would say is that – in my experience of trying to reach seldom heard groups, seldom represented groups, hard to reach groups, if you like, although I know that is not a popular expression – is it is labour intensive. It can involve more conflict, more kinds of conflict. And it is also more expensive because it is labour intensive. It involves a lot of groundwork building relationships, and that sometimes sits awkwardly in economic models of PPI. So, when you are doing the budget for your application, you know, people say, “You have got to get hold of so-and-so many people” and people have an idea of how much that is going to cost. But there is quite a lot of hidden costs in reaching, for example, the homeless, the housebound elderly, people who may be a bit confused because they need a helper, all that kind of thing. And I think one of the things we need to do is resource PPI efforts with seldom heard groups because they cost more.
I think that is an excellent point. I have to say, on behalf of a colleague of mine at the NIHR centre for engagement and dissemination that I was rightly told off by using the term ‘hard to reach’. And was told, “They are not hard to reach, we are just hard to work with.” And I thought that was a fair point. We don’t make it easy for ourselves because of the resourcing issue. And I think a lot more work needs to be done on how we broaden our horizons and make ourselves easier to work with overall. I’m not suggesting we have to beat ourselves up about this, but there are potential techniques that could make it easier. Although, we should at least investigate whether they might help. But even that I don’t think will be perfect. There will be certain groups that will always be-, or will always find us difficult to work with. We just need to be aware of that.
Mark, Trisha mentioned the point about resourcing the PPIE efforts sufficiently, which I would wholeheartedly agree with. Whenever I go and ask for more money for anything, the first question I get asked is, “What are the results going to be?” If we were going to ask for more resource to put into wider, stronger PPI, we are still going to be faced with a fundamental question: what impact is that going to have? You have clearly put some thought into impact measures. How would you begin to answer some of those questions?
Firstly, I would always be hopeful. Because if you look at how things have changed over the past ten or fifteen years, and how the resources have begun to flow or increase, I think there is evidence that over the past few years, money, further resources, have actually gone into the PPI work from an NIHR perspective. And we are not the only funder to put money into this, and resources into this, although I have always been very proud of NIHR for the way it has made PPIE central to its mission in that sense. I think there isn’t an easy answer to saying, “Give us more money” or “What is the impact?” straightaway. As I said, to work out the impact of any particular investment, or any particular activity, takes time to build in those impact measures, to be actually able to tell that story. Which is why one of the reasons we have started to look back, one of the reasons we asked Trisha and her colleague Jewel in the first place to actually do the historical perspective, one of the reasons why we are building on that, working with Oxford and Sheffield to do a further piece of work to interview a set of patients in London, Oxford, and Sheffield about what their historical perspectives as to how PPI has changed their involvement with research, so that we can build up that evidence base from a qualitative basis. But it will take time. But I will say again, pushing, pushing, and pushing over a period of time has increased those resources, and we shouldn’t stop now.
Yes. I suppose, if we are still talking about inequalities, I think we should because I think this is the $64,000 question. One of the approaches that I have found really helpful is to build partnerships at an organisational level, particularly with the third sector. We did a study a few years ago looking at different minority ethnic groups and their reasons for not coming forward for hepatitis-C screening and treatment. It was an NIHR-funded program grant. And one of the things we did was, we worked with different ethnic charities – quite small ones sometimes, locally – and we would go along and say, “Look. We would like to get the views of these people. How can we work together?” And different groups had completely different ideas as to how they might work with us. And what was great was to have a bit of flexibility in the funding so that we could say, “Yes, we can fund,” for example, “one of your members for half a day a week to go around and visit people.” Or “We can fund meetings in your community venue where people come for their lunch clubs”, or whatever. And it was the flexibility in use of resources, so that we could go to groups who were used to dealing with particular groups, and they would be able to say, “This is how we think you should do it,” rather than us go along with a firm plan, and say, “Right, we would like to get 15 people to come along at nine ‘o clock on a Monday morning,” or whatever, which isn’t the way to do it.
I agree. I think that this is long-term relationship building. This is strategic partnerships. This is far more than – as Trish says – just saying let’s get X people together and have a cup of coffee for a meeting. It is much more long-term planning. It is much more about the infrastructure of PPIE as well. And again, without trying to raise the NIHR flag, over the past few years through NIHR funding streams like the biomedical research centre, there has been some resource put in to create a PPIE infrastructure, the Centre for Engagement and Dissemination on top continuing to push the ethos of PPIE, which – from my point of view – is extremely important to stop it slipping back to managerialism. It is a long-term investment that needs to be kept up because, by definition, then, Chris, when you are asking for more money next year or the year after, people understand instinctively, hopefully, why it is important in the first place. Rather than just having to make a basic elevator pitch even just to get someone’s attention. Which I think was the problem if you go back ten or fifteen years ago.
On a slightly different angle in terms of learning from PPIE – because everything we have talked about so far have been thinking about the positive sides of PPIE – have either of you come across any negative aspects of PPIE in the world you have been inhabiting?
I think there have certainly been awkward moments. Without any doubt, there have been awkward moments. I am remembering some particular meetings, particular data sharing sessions where we had invited patients and the public to come, and we would present data. And most invited patients, very interested. One or two seemed bent on wrecking the whole session. But I am not sure that is a problem with PPIE, because that happens in senior management committees as well. There is one individual who is behaving badly. So, I think, let’s face it, folk are very variable, and some people have a bad day. I think there have certainly been efforts to involve and engage the public that haven’t worked particularly well. But I would say almost all of those have provided important learning opportunities. I can give you an example, if you like. I worked for 30 years in London, and quite a lot of my time in the East End of London. We used to work a lot with-, particularly with the East London mosque, but with faith-based groups and organisations in areas of high social deprivation and ethnic diversity. And the East London mosque was amazing. They had relationships with all sorts of health and care organisations, educational organisations. There were lots and lots of things going on. So, you would sort of go along there and say, we would like to, for example, do some eye screening. Or find out people’s views on eye screening. “Oh yes, come along. We’ll just sort of announce it during the Friday service.” Or something like that. So, it was all part of it. Anyway, when I came to Oxford, I thought, “Let’s start building relationships with some of the mosques here.” And it didn’t go down too well. And it was only when I went along and said, “Look, this is what we would like to do. Who can I work with?” that I realised the reason why it worked in East London, but it wasn’t working the same way in the place I went to in Oxford was because there had been 20 or 30 years of relationship building, of particular individuals working very hard to overcome problems of trust, to make sure that the money followed the work, to feed back the results of research studies to those organisations so they didn’t feel that they were just getting people to give an armful of blood, all those kind of problems. And that when you start again in a different town or city, to start building relationships with, for example, faith groups, you have to realise that it is going to take you, perhaps five years, before you get to first base. Now, you could say it was a disaster the first time we tried it, or you could say we learnt an important lesson the first time we tried it, and we now know that we have got to work more slowly. We have got to work extremely diplomatically. And we have also got to understand a little bit more about different faith organisations in a community. Some are keener than others. Some are better resourced and set up than others to interface with academic groups. Does that make sense? I think from failures we can learn more strategic lessons, I suppose, is what I am trying to get across.
Yes. I think it is a really good point. Maybe one thing to follow that up is, if I am a new researcher coming into the start of my career, what I would like to do when I get involved in PPIE is to work from what has worked, and equally learn from what the failures have been. What would your advice be to a young researcher coming in saying, “I am interested in doing PPIE within research. Where do I go to for a start point?” Is that something that the NIHR support directly, Mark?
Yes, I mean there is a lot of information available through the centre for engagement dissemination. Any large university hospital that has a biomedical research centre will have a PPIE infrastructure. So, there is a lot of resource out there. Even if you are an early-stage researcher applying for funds for the first time, NIHR has a system called the research design service, which are regional offices which will help you apply for NIHR funds. And actually, they will also help for the smaller charities as well. And they will have PPIE experts. But on top of that, the simple answer is, find the patient group you wish to work with. So, if you are an MS-, or interested in MS research, talk to the MS society. See who they have that you could talk to directly. I am sure I can speak on behalf of all people with MS. We are a friendly bunch. We love to have a chat. A cup of tea and a chat about research is a fantastic thing for us.
Talking from the research perspective, if I bring my patient hat on and I am approached by a researcher to say would I like to get involved in patient involvement in this research-, maybe I will start with Trish. Convince me it is a good thing to do as a patient.
As a patient? Well it is certainly a good thing for somebody to do. I am trying to remember the name of the study-, actually it is probably better not to name it – but where it came before an ethics committee and they really thought that what they were going to do was a liver biopsy on every patient, every three months for eighteen months. And it actually took a patient to say, “Hang on a minute. Have you ever had a liver biopsy? And is there any other way you could get the data apart from sticking something that looks like an apple core into my fleshy liver every three months?” I mean, it is a slightly comical example, but I think one of the things that patients do is they do a reality check with-, I mean, they talk about things like “How many times am I going to have to go to the hospital?”, “How many times am I going to have to find a car parking space?”, “Are you going to pay my car parking fees?” These very practical things. Because most of us would be very happy to help in research, but actually, the practicalities can be prohibitive. So, those very common sense and practical questions. So, coming back to your question: persuade me to get involved. Look, if no patient gets involved in the design of research studies, they are going to be designed around the researchers’ convenience, and they might be – first of all, rather painful and uncomfortable, as in liver biopsies – but also very practically awkward and time consuming for patients. So, just having patient involvement will make research easier to do. But also, I think there is a more kind of metaphysical thing here, which is to say, it is your illness. This whole thing is about trying to improve either life expectancy or quality of life or something that should be helping patients. If it doesn’t, then there is something really badly wrong. Actually, can I tell another story? Can I tell my favourite story about patient involvement in research? Which I think is relevant. Many, many years ago, there was something called the Clark Review of Research Priorities in Research. And I was asked to cochair the diabetes section of that. And this must be 25 years ago. And we were really forward-thinking in this. We decided we would have some focus groups of patients to help set research priorities. This is way before this was fashionable. So, we didn’t get it quite right. We had meetings of clinicians first, to get their priorities. And then what we did was we had another meeting of patients to feed them what the clinicians had said. So, there was a way to go. Anyway, the clinicians had said that behaviour change would be a really good thing to do research on in diabetes. So, we fed that back to the patient group. And the patient group said, yes, behaviour change would be fantastic. And it was only when we asked them to expand on that that we realised that the clinicians thought that the patients’ behaviour needed to change, but the patients thought that the clinicians’ behaviour needed to change. And we did actually recommend two different lots of behaviour change research. One on patients, and one to change the behaviour of the doctors, frankly, to make them a little bit less paternalistic, I think the issue was. That is just another example of why get involved as a patient, because if not, you are going to end up with clinician centred research.
The types of research and the way we are conducting research, there is a lot more being done now by using – what we call real world data, real world evidence – which is routinely collected healthcare data, rather than data collected in clinical trials. And working with that, it is more difficult to get-, you can’t talk to patients who are particularly involved in that. You are talking about a general use of patients’ data. Does this make it even more important that PPIE is involved, so that we have people who can represent those whose data we are using, even if it is not their own data?
Yes. I would suggest it is, actually. Especially if you are talking about general data sets. And having patient representation to make sure the right parts of those data sets are actually used. The lived experience, to say, “Actually that is representative, and that isn’t.” To actually help shape the research question itself, based on that general data. I think that is extremely important. If you look at the work of the James Lind Alliance on research prioritisation, for example, to have patient viewpoint at the beginning of the process is extremely important. Even if they are not being involved in the research itself, if that makes sense.
Maybe one last question, unless there is anything we have missed, one last question I have got for the both of you: in terms of PPIE and the impact of PPIE, what would you like to see most, looking forward in this area?
From my perspective, I am very, very keen to see how we involve patients in impact assessment itself. So, NIHR and other funders involve patients at all stages of the research process now. As I think I have already said, from the preapplication through prioritisation through application through monitoring. But I am not sure we really have engaged with patients in the impact of research itself. And I am very keen to find ways of doing that. Now, you know, myself and some colleagues have sat down and thought about some models we might want to use, how we might want to engage with patients beyond the simple ‘let’s have patient representation on any impact assessment steering committee’ how we might do this. We do have a paper coming out soon that will discuss the use of online citizens’ juries to be able to gain a broad viewpoint from patients. Even the ones that find us hard to work with. So that we get a better understanding of what they view the impact of the research we undertake is. So, from my point of view, we need to keep on the ethos side of PPI. But we need to expand what we do to better understand how patients can inform and be part of, and be empowered to do, impact assessment of the research that we fund.
Yes. I would strongly support what Mark has said there. I think the other thing I would like to pick up on, you just earlier on talked very briefly about the James Lind Alliance, and I think it is worth giving them a plug. This is something that I was a little bit, sort of semi-sceptical about before I got to know them and got involved with some of the things they are doing. But this is an excellent organisation. The idea is it is about research priority setting. It tends to be disease-focused, but it doesn’t-, not all of their initiatives are disease-focused. So, mostly, it is people who either suffer from a particular disease or have got a family member with it. And there is a call for setting new priorities. And then there is quite a structured process of summarising what the evidence base is already, and what ongoing research studies are doing, and then saying to patients – and to clinicians, actually – “where are the gaps? What new research do we need to do?” And I followed-, there is one going on in my department for heart failure. I think it is probably finished now. There is another one on rare anaemias. You know, that kind of thing. And there is some interesting stuff being done on multimorbidity. Now, this is great because it can completely reframe the priority areas and funders can then look at what they call the top ten lists. These are the top ten things that patients and clinicians say need to be researched. And that might be matched with a particular funder’s interest. So, I think this is really good. It is really upstream patient and public involvement. I would like to see more of that, really. I would like to see a source of funding for running James Lind Alliance partnerships to set research priorities. I actually think that would save money in the long-term.
To be honest, it is not a huge leap of the imagination to see research prioritisation repurposed in some ways to look at impact as well. Rather than ask the question, “What research would you like to prioritise?” to certain patient groups, or certain key patient and public groups, to actually ask people to start to begin to think about what they would hope to see as success of research programs in certain areas. And I think it is not impossible to see groups brought together to help us understand what we should be looking for in five, ten, fifteen years’ time to see if research really has changed. If you get people with condition A together, and say, “if this research is successful, what I would like to see is A, B, and C,” and see whether A, B, and C actually have happened ten years down the line. Rather than the dry impact assessments, which can look at case studies which are meaningful to those in power. That is probably the wrong term. But those who run things, rather than those who are affected by things. And as I said, the research prioritisation model, the James Lind Alliance model, is an excellent one, but it could be re-tuned. We could take it and use it in different ways.
And what is it going to take for that to happen then, Mark?
Well, we go back to the old question about resource. James Lind Alliance doesn’t work for free. There are fantastic people who work within that particular structure who need to be paid. They do what they can with the money they’ve got. The idea of using citizens’ juries, even online ones, there is a cost to that. And actually, for that sort of thing, I think there needs to be a certain amount of research first to see whether it actually works. Whether it is relevant. It is a fine idea but going from fine idea to let’s roll it out, I think, would be a bit of a risk. So, finding resource to actually research the model and see under what context it might work, I think, is the next step there. I don’t know what Trish feels.
I agree. I totally agree. I think it is quite difficult, because you don’t know-, what they call JLAPSPs, the priority setting partnerships, are funded in quite different ways. Some of them are funded – you guys know – some of them are funded, for example, with biomedical research centre money. Some of them are funded from charities. And they have very different histories, and very different path dependencies. I love the fact that they are all rather different and each one plays out differently. On the other hand, I have heard from an awful lot of people who have been involved in them, the worst bit about this was getting the funding. And you are not talking about that much funding. And I think it would be really good to encourage a wide variety of funders to put up some pump-priming money to do the priority setting exercises before they then put several million into a research program that might actually not be a priority.
I think you both made excellent points, which might bring us to a natural conclusion. I think, for me, it is clear that we are in this for the long haul. PPIE takes a long time, as you both described. I think there are some really solid things we could look at in terms of next steps in evaluating the impact-, demonstrating impact of PPIE. Not least following a model such as the James Lind Alliance and others, which could be adapted. I would suggest that anybody who is listening to this podcast, maybe has a look at the James Lind Alliance website. Educate themselves about what it does and think about whether that model could be used to evaluate impact more generally. Unless either of you two have anything extra to say, I am wondering whether that might be a reasonable time to close the podcast.
I would just like to say that if people want to have a look at the Centre for Engagement Dissemination, the NIHR website, they will also find some very useful resources on that.
If people want to tweet about the podcast, Mark, is there a hashtag they could use?
Yes. It is #impactframeworks.
Okay. Thank you to Mark, and thank you to Trish, and thank you to those listening as well.
Thank you for listening to this podcast. It is one of four in a series of exploring different aspects of impact culture. Please return to the website to discover the others. And don’t forget to tweet us your comments and questions to #impactframeworks. Once again, thank you for listening.
