Key words : Feasibility study, Graft perfusion, Monitoring technologies, Patient-public involvement, Qualitative interviews

Introduction

In the United Kingdom, about 3 million people have chronic kidney disease.¹ End-stage renal disease is fatal without treatment, and in the absence of a kidney transplant, around 50% of dialysis patients die within 5 years due to associated impediments.² Kidney transplant is the safest and most dependable form of long-term renal replacement therapy for patients with end-stage renal disease.³

According to the National Health Service (NHS) Blood and Transplant, in the year 2021-2022, an increase of 106% and 17% were recorded in the number of kidney transplants from living donors and deceased donors, respectively, compared with the previous year. Despite this, the number of patients registered on the kidney transplant list increased to 5023 people.⁴ One reason for the disparity between demand and supply is the reduction of transplant activity during the COVID-19 pandemic.⁵

About 7% to 8% of kidney transplant recipients experience the loss of their kidney graft in the first year after implant.⁶ Immediate vascular compli-cations account for 35% of total kidney grafts lost during the first 30 days after implant.⁷ Early graft loss leads to a cascade of complex repercussions for the patients, including physiological stress and morbidity due to additional procedures.⁶ The NHS incurs financial costs for the additional treatment.⁷ The number of patients registered on the kidney transplant list is also increased after reactivation of a patient whose graft has failed.⁸

Early identification of immediate vascular complications is crucial, as a timely intervention may rescue the compromised graft.⁹ Traditional clinical assessment, supplemented by color duplex scan, is considered the standard care postoperative moni-toring technique.¹⁰ Although the color duplex scan technique has limitations, there is no consensus on the benefit of any other blood flow monitoring device.¹⁰

An implantable Doppler (ID) probe is a blood flow monitoring device that has been used successfully in plastic surgery and breast reconstructive surgery.¹¹ Following the same principle, the ID probe may have the potential for use as a blood flow monitoring device in kidney transplant.¹² Our search of the medical literature revealed 3 observational studies that described the role of ID probes in kidney transplant surgery. These were nonexperimental studies that lacked procedural rigor and methodological quality and provided only preliminary information.^9,12,13 To address the lack of evidence in the contemporary literature, we proposed a feasibility study to support the development of, and investigate the uncertainties around, a future pragmatic superiority large-scale trial. It is crucial to account for the views and expectations of health care professionals and patients as a factor in the design of the research protocol for the feasibility study.

Patient and public involvement

Patient and public involvement (PPI) is defined as an active communication between patients, members of the public, and medical decision-makers for the development of collaborations to promote innovations and improve health care.¹⁴ In health care research, PPI refers to “research being carried out with or by patients and members of the public rather than to, about, or for them.”¹⁵

Patient and public involvement consultation responds to the needs and concerns of patients and stakeholders and provides them with a sense of empowerment and value.¹⁵ It provides an alternate perspective from the participants, with the expe-rience of the condition, which may add valuable information to the researchers.¹⁶ Patient and public involvement is emphasized in the development of study protocols for its value to identify barriers to implementation and to promote high quality and meaningful effect of the proposed research.¹⁶ Patient and public involvement consultation is suggested as an evidence-based practice for designing research, as the patient-centered care initiative is increasing in health care systems worldwide.¹⁷ It increases the applicability of the research because stakeholders are more likely to make changes indicated by the study if they are continuously engaged to collaborate in the project and are confident of its internal validity and the reliability of the results.¹⁷

The NHS advocates the inclusion of local voices for the design and improvement of regional health and social care, to create a truly patient-led health service.¹⁸ The National Institute of Health Research (NIHR) emphasizes the multifaceted and conti-nuously evolving role of PPI in formulation of research protocols, development of grant applications, and dissemination of the results.¹⁹ To encourage and support active public participation, the NIHR established a national advisory group known as INVOLVE in 1996, which is the lead for the advancement of PPI in health care research across the United Kingdom.¹⁹ The National Institute of Clinical Excellence recommends incorporation of PPI in the development of guidelines as it explores barriers to implementation and ethical issues related to such barriers.²⁰

Aims and objectives

This PPI consultation aimed to explore the views and expectations of the stakeholders (kidney transplant patients, clinicians, and nurses with direct experience of the monitoring device) on the protocol develo-pment of the feasibility study. Its objective was to reveal areas for improvement in the study protocol, understand the perceptions of stakeholders regarding research in the postoperative graft surveillance, and identify potential confounding factors to the research and the challenges to the implementation of the monitoring device in clinical practice.

Materials and Methods

Participant selection and settings

The ID probe has been used intermittently in the Southwest Transplant Centre (SWTC) for the past 7 years. This allows the opportunity to conduct the proposed feasibility study investigating the role of the ID probe in the unit. Accordingly, participants with experience of the monitoring device were recruited from the same setting. To ensure the selection of information-rich cases and to attain a comprehensive perspective consistent with the aim of the PPI consultation, a purposeful sampling consisting of participants from all groups directly involved with the ID probe was done.²¹

An invitation to participate was sent to the health care staff of the SWTC through NHS E-mail. The E-mail included a brief description of the proposed feasibility study and the purpose of the PPI consultation. There was an overwhelming response from the staff showing their enthusiasm to participate in the research that was related to the postoperative monitoring of kidney transplant patients. The kidney transplant patients were recruited in the surgical ward and consented to the study preoperatively. The participants consisted of 5 clinicians, 3 nurses, and 4 kidney transplant patients. A total of 12 participants (5 male, 7 female) were recruited, keeping in view previous published PPI consultations that achieved theoretical saturation with similar sample sizes.²²

The written informed consent included approval for the interview and further correspondence for member-checking. The PPI consultation was conducted using an exploratory inductive approach from February 2022 to April 2022 (3 months). Patient and public involvement consultation is beneficial in timings and settings that are comfortable for the participants.²³ The surgical ward of the SWTC was selected to enable participants to relax and be open in their routine setup. Privacy of the interviews was maintained to allow participant anonymity. These were conducted during the lunch break to avoid distractions from clinical commitments.

Data collection

We selected semi-structured interviews with open-ended questions, as these represent the most effective way to explore the views of the participants.²⁴ We conducted interviews in accordance with the NIHR instructions, from the PPI in Health and Social Care Research Handbook for Researchers.²⁵ The interview guide comprised key steps such as an introduction, a brief description of the proposed feasibility study, an explanation of the purpose of the PPI consultation, consent, and interview questions (Table 1). The interview questions were aided by probe questions to facilitate further elaboration (Table 2). The results of this PPI were reported in accordance with the Consolidated Criteria for Reporting Qualitative Research (ie, COREQ) checklist, following the recommended reporting guidelines for qualitative studies and PPI consultations.²⁶

All interview proceedings were audio-recorded with consent from each participant; the recordings were then deleted after transcription. Participants were informed that any quotes used in the report would be anonymized. Field notes were taken during all interviews to ensure the richness of data. The duration of the interviews was 30 to 45 minutes.

Ethical considerations

According to the NIHR-established national advisory group INVOLVE, formal ethical approval is not required to conduct PPI consultations aimed to explore opinions on research studies, because the participants are acting as specialist advisors in planning and design of the research protocol.¹⁹ However, it was ensured that data protection was enacted according to the General Data Protection Regulation 2018 and that all necessary actions were undertaken to protect confidentiality and anony-mization of participants in line with Good Clinical Practice.²⁷ This PPI is registered with the Research and Development Department of University Hospitals Plymouth NHS Trust (reference No. 21/SUR/626).

Data analyses

Thematic analysis is regarded as a foundational method for qualitative analysis as it provides a practical and flexible approach for identifying, analyzing, and reporting themes within qualitative data.²⁸ Because the interviewer had a theoretical interest in the ID probe, to reduce the potential for interviewer bias, thematic analysis was conducted by the inductive approach of data-driven identification of themes.²⁹ Because all participants were directly involved with the ID probe, the thematic analysis was performed at the latent level to identify underlying ideas and assumptions.³⁰

Data were analyzed and categorized into themes, according to the 6-phase guide to thematic analysis³¹ (Figure 1). In phase 1, the process of familiarization started from the initial step of data collection and continued by reading the data actively to look for patterns of potential interest and meaning. In phase 2, coding was data-driven such that all data extracts were coded and collated together into meaningful groups. In phase 3, different codes were sorted and collated into the broader level of identified themes. In phase 4, themes were refined according to the validity of individual themes to the data set, internal homogeneity, and external heterogeneity. These collapsed into each other, were broken down into separate themes, or were removed due to lack of supporting data. In phase 5, the “essence” of each theme was identified, which is defined as the attributes that each theme represents from the thematic map of the data. Each theme was refined according to its essence, and a narrative was constructed, keeping in line with the research question. Subthemes were devised to demonstrate the hierarchy of meaning within the data. In phase 6, a concise writeup was produced that comprised an analysis of the fully worked-out themes elaborating the merit and validity of our qualitative research. Thematic analysis was performed with NVivo software (version 12).

Rigor and trustworthiness

Demonstration of rigor and trustworthiness is necessary for qualitative research to substantively improve clinical practice and policy, because these details add credibility to the sample and validity to the results.³² The following methods were adopted in the PPI consultation to ensure rigor and confer respectability to the qualitative research.³³

Purposeful sampling with triangulation

The sample comprised participants from all the groups directly involved with the device to facilitate information collection from every perspective, validating credibility to the sample and richness of the data.³⁴ It was ensured that the sample was representative of the target population, necessary for the external validity of a study. The participants were selected randomly, according to the staff on call on the day of the kidney transplant, according to their respective rotation plans. Recruitment was continued until theoretical saturation was attained.³⁵

Reflexivity

Reflexivity is fundamental to the credibility of qualitative research.³⁶ The interviewer was a transplant surgeon with an interest in the early postoperative monitoring of kidney transplant patients. A previous working relationship existed between the researcher and the participants, who were informed of the purpose of the PPI consultation. To ensure trans-parency with the responses and improve the validity of the results, a reflexive diary was kept for bracketing out any bias from the author’s perspective.²⁹

Respondent validation

The findings were brought back to the members to obtain their opinions on any interpretations drawn from their interviews.³⁷ Member checking was conducted, and the participants were satisfied after reviewing the results of thematic analysis. Peer review of the themes was also performed independently by another clinician.³⁸ There was concordance in the perspective of male and female respondents.

Results

Three key themes emerged. (1) Experience with the ID probe as a monitoring device reflected that it was well received by the patients; however, there was a clinical equipoise among the health care professionals. (2) Recognition of the need for research into early postoperative graft monitoring displayed the stakeholder understanding regarding the role of a blood flow monitoring device to improve surgical outcomes. (3) Recommendations for the smooth conduct of the proposed study included suggestions for improvement in the study protocol, informative sessions for the patients and nurses, and innovative ideas to improve the monitoring device.

Codes, categories, subthemes, and themes are listed in (Table 3). A word cloud for the most frequently used words in the PPI consultation was formulated (Figure 2).

Experiences with the implantable Doppler probe monitoring device

Mixed feedback was received from health care professionals, reflecting a clinical equipoise. However, responses from the patients were largely in favor of the monitoring device.

This theme was derived from 2 subthemes. The first subtheme was positive perceptions of the monitoring device, which included categories such as benefits to the NHS, health care professionals, and kidney transplant patients. The participants felt that the use of the device would have a positive effect on the NHS Trust. The health resources saved by the reduction of complications could be diverted to more severe cases. “We would be able to save money, which the Trust is trying to do by cutting expenses in all fields. All little pockets of saved money would add up.” (Nurse 1) “The overall patient outcome and graft survival of our Trust would improve.” (Clinician 2) “If the Ultrasound registrar is not caught up in the transplant ward than he would be utilizing his time on some other emergency.” (Clinician 3) The participants felt that the device may have the potential to improve patient and graft outcome, by the early identification of vascular complications. It also decreased the surgery-related anxiety of patients. “I heard the sound of my new kidney, and it was the most amazing moment of my life. I developed a psychological relationship with it.” (Patient 3) “I was very anxious before the surgery. It made me feel whatever I had gone through was worth it.” (Patient 2) “The sound was a sense of accomplishment for me and my best friend who had donated her kidney [to me]. We both hugged and cried together. We made our families listen and they were thrilled. It was a very sentimental time for all of us. I will never forget that moment, that gratitude and tears.” (Patient 1)

Some clinicians felt reassured by the device, while others were not convinced due to a lack of evidence in the medical literature. Some nurses felt less confident, as they were not trained to interpret the audio signals. It was suggested that the sound of the device might bother other patients. These conclude the second subtheme of barriers to the implementation. “The first thing which I note [when reviewing the patient in the recovery] is the Doppler signals. I do get reassured when I hear it. Although it’s not an evidence-based practice, I feel safe to postpone scan till morning.” (Clinician 2) “The [Doppler] signals do calm down the staff looking after the patients. I do not exactly understand them as we have not been trained to interpret the results. I also feel sometimes the patients are tired and the sound might disturb their sleep.” (Nurse 3) “The Doppler probe is just a hassle to put in. It is not supported by evidence, and I feel it just takes extra operative time.” (Clinician 4)

Recognition of the need for research in the postoperative graft monitoring

All participants acknowledged the importance of a blood flow monitoring device in kidney transplant surgery that can support medical staff and improve patient outcomes. They expressed a need for a study to obtain evidence that would justify the use of a postoperative vascular monitoring device, particularly in patients with no other measurement parameters. “There is no stress, and we can get a continuous transplant kidney [perfusion] assessment anytime which is reassuring especially in the absence of urine output measurement. (However, there is not much [information] out there to justify its use and research in this regard would be quite useful.)” (Clinician 1) “I do not feel confident using this device in the patients I operate upon because it is not backed by evidence. If the effectiveness of the monitoring device is supported by the evidence borne out by the patient outcomes, I will be happy to use it in my patients.” (Clinician 5)

Recommendations for the smooth conduct of the proposed study

This theme was composed of a subtheme identifying solutions to the potential challenges and practical planning for the proposed study. The participants advocated educational sessions, training workshops, and a study with a mixed methods methodology. Innovative ideas to increase understanding and simplify interpretation of the device were also put forward. “I would be more confident with training sessions for the nursing staff that would educate us how to interpret the signals. It would be easier for us if there would be a screen on the device that would give a visual interpretation of the Doppler signals.” (Nurse 3) “Video information regarding this device should be given to patients in the clinic and a picture of it should be added in the patient information sheet. It would be helpful if the patients are aware of what to expect [after surgery].” (Patient 3) “The connecting wire between the probe and the monitoring device can be accidentally dislodged resulting in false-positive results. This can be avoided by eliminating the need of the wire by developing WiFi/Bluetooth connection between the probe and the monitor.” (Nurse 1) “A qualitative component should be added in the feasibility study as it would help to highlight the real problems faced during the research. The evidence obtained in the feasibility study should be presented nationwide.” (Clinician 2)

Discussion

The main purpose of this PPI consultation was to explore the ideas and expectations of stakeholders regarding development of the research protocol and identify challenges to the conduct of the proposed feasibility randomized controlled trial. Patient and public involvement consultations have proven to be valuable and an integral part of designing health research studies.^20,22

The clinicians revealed clinical equipoise on the usefulness of the ID probe as a monitoring device.³⁹ The main reason given was gaps in evidence, despite recognition of the need for a blood flow sensing technology in clinical practice.⁴⁰ The clinical equipoise indicates the need for research in a field.⁴¹ Earlier studies have recommended clinical equipoise as a necessary ingredient to conduct clinical trials with the therapeutic intent and without bias.⁴²

The clinicians indicated areas for improvement in the study protocol. They suggested primary outcomes that aligned directly with the aim of the study (ie, identification of a compromised graft and the requirement of duplex ultrasonography scans postoperatively). Pocock and Stone have described primary outcome as the most plausible evidence associated with the intervention or exposure of interest.⁴³

The nurses identified the biggest challenge to implementation as their lack of training with the monitoring device. Nurses are the first-line health care professionals taking care of kidney transplant patients postoperatively, particularly out of working hours. This limitation was recognized, as without appropriate training, nurses would not be able to interpret the signals of the monitoring device. This problem was addressed by inclusion of training sessions for nurses who were assigned to posto-perative care. Macurik and colleagues studied 20 medical professionals and described the beneficial effects of training staff in intervention plans.⁴⁴

The health care participants also suggested inclusion of an embedded qualitative study in the proposed research. The challenges encountered during running of the feasibility study (ie, recruitment, retention, and outcomes) along with the perceptions of stakeholders, would best be understood by sharing their experiences. Grant and colleagues conducted an embedded qualitative study by recruiting 40 participants (patients and health care professionals) that revealed 4 key implementation priority areas to their randomized controlled trial.⁴⁵

The patients were keen for recruitment in the study. This may be because the monitoring device does not cause any additional burden on patients, while providing a potential safety net. They liked the idea that they would be able to listen to monitoring signals representing the blood flow to their transplanted kidneys and in a way participate in their postoperative care. The patients recommended educational sessions before kidney transplant surgery. They were willing to be randomized to allocated groups (with and without the monitoring device). They felt reassured that no complication related to this device had been recorded in the past 5 years in the transplant center. As the monitoring device does not cause any hinderance to the postoperative mobilization, the patients did not mind it being attached to their bedside.

Recruiting the required number of participants is crucial to the success of any study. Newington and Metcalfe interviewed 11 clinical research team members, with an aim to explore factors influencing recruitment to research. Altruism, engagement with the participants, and logistic issues were identified as key factors inspiring retention.⁴⁶ This is concordant with the findings of this PPI consultation.

The essence of the PPI consultation is that the patients consider Doppler signals of the monitoring device formed a psychological relationship between them and the implanted kidney. In living kidney donations, they linked the signals as a causative factor for building a deep emotional bond between the donor, recipient, and their families. Professional anxiety was identified as a barrier to the imple-mentation of the monitoring device. These apprehen-sions were due to lack of information about the working of the device and gaps in evidence regarding the role of this device in the medical literature.

This PPI consultation resulted in invaluable suggestions. Nurses proposed organizing training sessions. The patients liked the idea of including awareness videos in their educational sessions and pictures of the monitoring device in the patient information sheet of the proposed study. Innovative ideas were obtained to simplify the understanding of the device, such as the addition of an image display screen to enable visual interpretation of the Doppler signals and incorporating a WiFi/Bluetooth for a wireless monitoring system.

Limitations

The participants were known to the researcher and might have felt inclined to give desirable answers, resulting in the introduction of social desirability bias. However, the viewpoint of the interviewer on the device was not disclosed.⁴⁷ Likewise, the tendency of the interviewer to support any pre-existing assertions was reduced by keeping a reflexive research diary and formulating research questions in a neutral format.⁴⁸

Multiple coding increases the authenticity of results by generating new ideas and alternative interpretations, as it constitutes 2 or more independent researchers from different backgrounds.⁴⁹ This was not undertaken in the PPI consultation. However, peer review of codes and themes was conducted.

Triangulation involves the collection of data by multiple methods and increases the internal validity of the research. Only semi-structured interviews were used in our study, as it was anticipated by the data saturation that different methods would only yield similar findings.⁵⁰

Conclusions

This study highlights the utility of the PPI in the development of a research protocol. The PPI consul-tations lead to improvement in the research design and ensured that future research is patient-centered. The role of PPI consultation in future research projects is recommended.

References:

1. Kidney Care UK. Facts and stats. 2022. https://www.kidneycareuk.org/news-and-campaigns/facts-and-stats/
   CrossRef - PubMed
   
2. Laplante S, Liu FX, Culleton B, Bernardo A, King D, Hudson P. The cost effectiveness of high-dose versus conventional haemodialysis: a systematic review. Appl Health Econ Health Policy. 2016;14(2):185-193. doi:10.1007/s40258-015-0212-3
   CrossRef - PubMed
   
3. Kramer A, Pippias M, Noordzij M, et al. The European Renal Association-European Dialysis and Transplant Association (ERA-EDTA) Registry Annual Report 2015: a summary. Clin Kidney J. 2018;11(1):108-122. doi:10.1093/ckj/sfx149
   CrossRef - PubMed
   
4. NHS Blood and Transplant. Organ and Tissue Donation and Transplantation Activity report 2021/2022. https://www.odt.nhs.uk/statistics-and-reports/annual-activity-report/
   CrossRef - PubMed
   
5. Sharma V, Shaw A, Lowe M, Summers A, van Dellen D, Augustine T. The impact of the COVID-19 pandemic on renal transplantation in the UK. Clin Med (Lond). 2020;20(4):e82-e86. doi:10.7861/clinmed.2020-0183
   CrossRef - PubMed
   
6. de Kok MJ, Schaapherder AF, Mensink JW, et al. A nationwide evaluation of deceased donor kidney transplantation indicates detrimental consequences of early graft loss. Kidney Int. 2020;97(6):1243-1252. doi:10.1016/j.kint.2020.01.043
   CrossRef - PubMed
   
7. Tavakkoli M, Zafarghandi RM, Taghavi R, Ghoreifi A, Zafarghandi MM. Immediate vascular complications after kidney transplant: experience from 2100 recipients. Exp Clin Transplant. 2017;15(5):504-508. doi:10.6002/ect.2016.0057
   CrossRef - PubMed
   
8. Benko T, Halfmann P, Gackler A, et al. Long-term outcome of third, fourth and fifth kidney transplantation: technical aspects and immunological challenges. Clin Kidney J. 2019;12(6):895-900. doi:10.1093/ckj/sfz014
   CrossRef - PubMed
   
9. Hakim DN, Nader MA, Sood A, Kandilis A, Hakim NS. Rescue of transplanted kidney thanks to an implantable Doppler probe: is this the future? Exp Clin Transplant. 2016;14(4):454-455. doi:10.6002/ect.2014.0135
   CrossRef - PubMed
   
10. Kasiske BL, Vazquez MA, Harmon WE, et al. Recommendations for the outpatient surveillance of renal transplant recipients. American Society of Transplantation. J Am Soc Nephrol. 2000;11 Suppl 15:S1-S86.
    CrossRef - PubMed
    
11. Oliver DW, Whitaker IS, Giele H, Critchley P, Cassell O. The Cook-Swartz venous Doppler probe for the post-operative monitoring of free tissue transfers in the United Kingdom: a preliminary report. Br J Plast Surg. 2005;58(3):366-370. doi:10.1016/j.bjps.2004.12.003
    CrossRef - PubMed
    
12. Crane J, Hakim N. The use of an implantable Doppler flow probe in kidney transplantation: first report in the literature. Exp Clin Transplant. 2011;9(2):118-120.
    CrossRef - PubMed
    
13. Malik MS, Akoh J, Aroori S, Latour JM. Implantable Doppler probe as a vascular monitoring device in kidney transplant patients: investigation of use at a single center. Exp Clin Transplant. 2022;20(4):355-361. doi:10.6002/ect.2021.0500
    CrossRef - PubMed
    
14. Koniotou M, Evans BA, Chatters R, et al. Involving older people in a multi-centre randomised trial of a complex intervention in pre-hospital emergency care: implementation of a collaborative model. Trials. 2015;16:298. doi:10.1186/s13063-015-0821-z
    CrossRef - PubMed
    
15. Webster L, Groskreutz D, Grinbergs-Saull A, et al. Development of a core outcome set for disease modification trials in mild to moderate dementia: a systematic review, patient and public consultation and consensus recommendations. Health Technol Assess. 2017;21(26):1-192. doi:10.3310/hta21260
    CrossRef - PubMed
    
16. Skovlund PC, Nielsen BK, Thaysen HV, et al. The impact of patient involvement in research: a case study of the planning, conduct and dissemination of a clinical, controlled trial. Res Involv Engagem. 2020;6:43. doi:10.1186/s40900-020-00214-5
    CrossRef - PubMed
    
17. Gove D, Diaz-Ponce A, Georges J, et al. Alzheimer Europe’s position on involving people with dementia in research through PPI (patient and public involvement). Aging Ment Health. 2018;22(6):723-729. doi:10.1080/13607863.2017.1317334
    CrossRef - PubMed
    
18. Mockford C, Staniszewska S, Griffiths F, Herron-Marx S. The impact of patient and public involvement on UK NHS health care: a systematic review. Int J Qual Health Care. 2012;24(1):28-38. doi:10.1093/intqhc/mzr066
    CrossRef - PubMed
    
19. National Institute for Health and Care Research. Involve patients. 2022. https://www.nihr.ac.uk/health-and-care-professionals/engagement-and-participation-in-research/involve-patients.htm
    CrossRef - PubMed
    
20. Plehhova K, Paquette N, Gould J, Coyle C. Understanding the patient PPI journey: results of a survey on PPI treatment initiation and patient experience. J Prim Care Community Health. 2022;13:21501319221088688. doi:10.1177/21501319221088688
    CrossRef - PubMed
    
21. Valerio MA, Rodriguez N, Winkler P, et al. Comparing two sampling methods to engage hard-to-reach communities in research priority setting. BMC Med Res Methodol. 2016;16(1):146. doi:10.1186/s12874-016-0242-z
    CrossRef - PubMed
22. Lovegrove CJ, Bannigan K, Cheeseman D, Latour JM. The involvement of people with Parkinson’s in designing a study of the lived experience of anxiety. Br J Occup Therap. 2017;80(8):494-501. doi:10.1177/0308022617700654
    CrossRef - PubMed
    
23. Kleme J, Pohjanoksa-Mantyla M, Airaksinen M, et al. Patient perspective in health technology assessment of pharmaceuticals in Finland. Int J Technol Assess Health Care. 2014;30(3):306-311. doi:10.1017/S0266462314000282
    CrossRef - PubMed
    
24. Mejdahl C, Nielsen BK, Hjøllund NHI, Lomborg K. Use of patient-reported outcomes in outpatient settings as a means of patient involvement and self-management support: a qualitative study of the patient perspective. Eur J Pers Cent Healthc. 2016;4(2):359-367. doi:10.5750/ejpch.v4i2.1125
    CrossRef - PubMed
    
25. Hughes M, Duffy C. Public involvement in health and social sciences research: a concept analysis. Health Expect. 2018;21(6):1183-1190. doi:10.1111/hex.12825
    CrossRef - PubMed
    
26. Tong A, Sainsbury P, Craig J. Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. Int J Qual Health Care. 2007;19(6):349-357. doi:10.1093/intqhc/mzm042
    CrossRef - PubMed
    
27. Goddard M. The EU General Data Protection Regulation (GDPR): European regulation that has a global impact. Int J Mark Res. 2017;59(6):703-705. doi:10.2501/IJMR-2017-050
    CrossRef - PubMed
    
28. Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol. 2006;3(2):77-101. doi:10.1191/1478088706qp063oa
    CrossRef - PubMed
    
29. Ortlipp M. Keeping and using reflective journals in the qualitative research process. Qualitative Rep. 2008;13(4):695-705. doi:10.46743/2160-3715/2008.1579
    CrossRef - PubMed
    
30. Braun V, Clarke V. Reflecting on reflexive thematic analysis. Qual Res Sport Exerc Health. 2019;11(4):589-597. doi:10.1080/2159676X.2019.1628806
    CrossRef - PubMed
    
31. Maguire M, Delahunt B. Doing a thematic analysis: a practical, step-by-step guide for learning and teaching scholars. All Ireland J Higher Educ. 2017;9(3).
    CrossRef - PubMed
    
32. Hadi MA, Jose Closs S. Ensuring rigour and trustworthiness of qualitative research in clinical pharmacy. Int J Clin Pharm. 2016;38(3):641-646. doi:10.1007/s11096-015-0237-6
    CrossRef - PubMed
    
33. Mays N, Pope C. Quality in qualitative research. Qual Res Health Care. 2020;211-233. doi:10.1002/9781119410867.ch15
    CrossRef - PubMed
    
34. Metzger MJ, Flanigin AJ, Medders RB. Social and heuristic approaches to credibility evaluation online. J Commun. 2010;60(3):413-439. doi:10.1111/j.1460-2466.2010.01488.x
    CrossRef - PubMed
    
35. Saunders B, Sim J, Kingstone T, et al. Saturation in qualitative research: exploring its conceptualization and operationalization. Qual Quant. 2018;52(4):1893-1907. doi:10.1007/s11135-017-0574-8
    CrossRef - PubMed
    
36. Jootun D, McGhee G, Campus H, Lanarkshire, Marland GR. Reflexivity: promoting rigour in qualitative research. Nurs Stand. 2009;23(23):42-46. doi:10.7748/ns.23.23.42.s50
    CrossRef - PubMed
    
37. Torrance H. Triangulation, respondent validation, and democratic participation in mixed methods research. J Mixed Meth Res. 2012 6(2):111-123. doi.org/10.1177/1558689812437185
    CrossRef - PubMed
    
38. Bradley L, Thouesny S. Students’ collaborative peer reviewing in an online writing environment. Themes Sci Tech Educ. 2017;10(2):69-83.
    CrossRef - PubMed
    
39. Malik MS, Akoh JA, Houlberg K. The use of the implantable Doppler probe as a blood flow monitoring device in clinical settings: a narrative review of the evidence. Exp Clin Transplant. 2023; 21(2):83-92. doi:10.6002/ect.2022.0349.
    CrossRef - PubMed
    
40. Malik MS, Saif I, Akoh J. Are signals from implantable Doppler probes always accurate for prediction of blood flow in kidney transplant? Exp Clin Transplant. 2023;21(1): 52-54. doi:10.6002/ect.2022.0318.
    CrossRef - PubMed
    
41. Hey SP, London AJ, Weijer C, Rid A, Miller F. Is the concept of clinical equipoise still relevant to research? BMJ. 2017;359:j5787. doi:10.1136/bmj.j5787
    CrossRef - PubMed
    
42. Cook C, Sheets C. Clinical equipoise, and personal equipoise: two necessary ingredients for reducing bias in manual therapy trials. J Man Manip Ther. 2011;19(1):55-57. doi:10.1179/106698111X12899036752014
    CrossRef - PubMed
    
43. Pocock SJ, Stone GW. The primary outcome fails: what next? N Engl J Med. 2016;375(9):861-870. doi:10.1056/NEJMra1510064
    CrossRef - PubMed
    
44. Macurik KM, O’Kane N, Malanga P, Reid DH. Video training of support staff in intervention plans for challenging behavior: comparison with live training. Behav Intervent. 2008;23(3):143-163. doi:10.1002/bin.261
    CrossRef - PubMed
    
45. Grant S, Hodgkinson J, Schwartz C, et al. Using mHealth for the management of hypertension in UK primary care: an embedded qualitative study of the TASMINH4 randomised controlled trial. Br J Gen Pract. 2019;69(686):e612-e620. doi:10.3399/bjgp19X704585
    CrossRef - PubMed
    
46. Newington L, Metcalfe A. Factors influencing recruitment to research: qualitative study of the experiences and perceptions of research teams. BMC Med Res Methodol. 2014;14:10. doi:10.1186/1471-2288-14-10
    CrossRef - PubMed
    
47. Krumpal I. Determinants of social desirability bias in sensitive surveys: a literature review. Qual Quant 2013;47:2025-2047. doi:10.1007/s11135-011-9640-9
    CrossRef - PubMed
    
48. Engin M. Research diary: a tool for scaffolding. Int J Qual Methods. 2011;10(3):296-306. doi:10.1177/160940691101000308
    CrossRef - PubMed
    
49. Sweeney A, Greenwood KE, Williams S, Wykes T, Rose DS. Hearing the voices of service user researchers in collaborative qualitative data analysis: the case for multiple coding. Health Expect. 2013;16(4):e89-e99. doi:10.1111/j.1369-7625.2012.00810.x
    CrossRef - PubMed
    
50. Roberts P, Priest H. Reliability, and validity in research. Nurs Stand. 2006;20(44):41-45. doi:10.7748/ns2006.07.20.44.41.c6560
    CrossRef - PubMed