Experiential evidence methods
Literature searching and crowdsourcing processes identified 23 methods for gathering experiential evidence. The methods were mapped to the typology and stratified by mode, context and structure. The table includes methods identified through the literature review and crowdsourcing activity. This is not an exhaustive list of methods.
In person relies on interaction that happens in one place at a certain time, virtual relies on interaction that happens in different places remotely at the same time, digital relies on interaction that happens online in different places and at different times.
Talk-generated methods
Method | Revealing issues | Identifying allegiances and shared views | Generating solutions | Reaching accord | Virtual | In-person | Digital | Individual | Groups | Highly structured | Semi-structured |
---|---|---|---|---|---|---|---|---|---|---|---|
Structured, semi-structured or unstructured conversation with individuals or small groups to gather perspectives, opinions and world views. | Reveals issues | Does not identify alliegances and shared views | Generates solutions | Does not reach accord | Not virtual | Not in-person | Not digital | Individual | Groups | Highly structured | Semi-structured |
Roundtable + Small group discussions with well-informed individuals to exchange perspectives and debate an issue or topic. | Reveals issues | Identifies allegiances and shared views | Generates solutions | Does not reach accord | Virtual | In-person | Not digital | Not individual | Groups | Not highly structured | Semi-structured |
In-depth exploration of an issue or topic with 8 to 12 individuals, under the direction of a facilitator, for up to 2.5 hours. | Reveals issues | Does not identify alliegances and shared views | Generates solutions | Does not reach accord | Virtual | In-person | Not digital | Not individual | Groups | Not highly structured | Semi-structured |
Yarning circles + A conversational process that involves the sharing of stories and the development of knowledge in a culturally prescribed and respectful way. | Reveals issues | Identifies allegiances and shared views | Generates solutions | Does not reach accord | Virtual | In-person | Not digital | Not individual | Groups | Not highly structured | Not semi-structured |
Workshops + An interactive process to discuss among groups of various sizes. | Reveals issues | Does not identify alliegances and shared views | Generates solutions | Does not reach accord | Virtual | In-person | Not digital | Not individual | Groups | Not highly structured | Semi-structured |
Nominal group technique + A structured approach to group brainstorming which combines information gathering, idea generation and consensus building on a specific topic. | Reveals issues | Identifies allegiances and shared views | Generates solutions | Reaches accord | Virtual | In-person | Not digital | Not individual | Groups | Not highly structured | Semi-structured |
User or working groups 2, 8, 9*+ A small group established to process a particular issue or achieve a specific task. | Reveals issues | Does not identify alliegances and shared views | Generates solutions | Does not reach accord | Virtual | In-person | Not digital | Not individual | Groups | Not highly structured | Semi-structured |
Dialogues + Small group sessions, usually over one day, where individuals receive information and deliberate on a topic. A formal vote, ranking exercise or pre- and post- surveys assess the deliberation outcome. | Does not reveal issues | Identifies allegiances and shared views | Generates solutions | Reaches accord | Virtual | In-person | Not digital | Not individual | Groups | Highly structured | Not semi-structured |
Experts or representatives 4, 9, 10* Health professionals and consumer representatives on advisory councils, committees and participating in research or improvement initiatives and projects. | Reveals issues | Does not identify alliegances and shared views | Generates solutions | Does not reach accord | Virtual | In-person | Not digital | Individual | Not groups | Not highly structured | Not semi-structured |
A panel of experts evaluate the scientific literature related to an issue or topic, hear testimonies and then formulate recommendations. | Does not reveal issues | Identifies allegiances and shared views | Generates solutions | Reaches accord | Not virtual | In-person | Not digital | Not individual | Groups | Highly structured | Not semi-structured |
Investigates the perspectives of individuals who represent different views on a topic through ranking and sorting statements. | Does not reveal issues | Identifies allegiances and shared views | Does not generate solutions | Reaches accord | Not virtual | In-person | Digital | Not individual | Groups | Highly structured | Not semi-structured |
Citizen juries or mini publics 8, 11*+ A small representative group of 12 to 24 individuals, deliberate on an issue or topic (generally one clearly framed question), over the period of 2 to 7 days. The groups are presented with evidence-based expert views. | Does not reveal issues | Identifies allegiances and shared views | Generates solutions | Reaches accord | Not virtual | In-person | Not digital | Not individual | Groups | Highly structured | Not semi-structured |
A demographically representative group used to assess public preference or opinion. | Reveals issues | Identifies allegiances and shared views | Does not generate solutions | Reaches accord | Not virtual | In-person | Not digital | Not individual | Groups | Not highly structured | Semi-structured |
Narratives or storytelling 3, 13-15*+ An account of an individual’s experience and includes detail, characters and some plot. Communicated in verbal, print, audio or video formats. | Reveals issues | Does not identify alliegances and shared views | Does not generate solutions | Does not reach accord | Virtual | In-person | Not digital | Individual | Groups | Not highly structured | Semi-structured |
Self-reported verbal, print, audio or video illustrations, including key elements of a scenario, experience or setting. | Reveals issues | Does not identify alliegances and shared views | Generates solutions | Does not reach accord | Virtual | In-person | Digital | Individual | Groups | Highly structured | Semi-structured |
Gathering information by observing people, places and/or processes either directly or indirectly. | Reveals issues | Does not identify alliegances and shared views | Does not generate solutions | Does not reach accord | Not virtual | In-person | Not digital | Individual | Groups | Not highly structured | Semi-structured |
Text-generated methods
Method | Revealing issues | Identifying allegiances and shared views | Generating solutions | Reaching accord | Virtual | In-person | Digital | Individual | Groups | Highly structured | Semi-structured |
---|---|---|---|---|---|---|---|---|---|---|---|
A checklist or rating scale presents a list of events or characteristics on which a person can indicate, by a check mark or rating on a scale, whether the event or characteristic of interest did occur or is present. | Reveals issues | Identifies allegiances and shared views | Does not generate solutions | Does not reach accord | Not virtual | Not in-person | Digital | Individual | Not groups | Highly structured | Not semi-structured |
Measures informed opinion on a particular issue, through aggregated individual opinions, rather than consensus. | Reveals issues | Identifies allegiances and shared views | Does not generate solutions | Does not reach accord | Not virtual | Not in-person | Digital | Not individual | Groups | Highly structured | Not semi-structured |
Delphi + Iterative multi-stage process designed to combine opinion into group consensus, through multiple rounds of opinion sharing, discussion and aggregation of quantitative data. | Does not reveal issues | Identifies allegiances and shared views | Does not generate solutions | Reaches accord | Virtual | In-person | Digital | Not individual | Groups | Highly structured | Not semi-structured |
Written posts, comments and microblogs on social media platforms and networking sites. | Reveals issues | Does not identify alliegances and shared views | Generates solutions | Does not reach accord | Virtual | Not in-person | Digital | Individual | Not groups | Not highly structured | Not semi-structured |
Questionnaire / survey 2, 4, 6, 8, 19*+ Specific set of open* or closed written questions to gather information from a large number of individuals. | Does not reveal issues | Identifies allegiances and shared views | Does not generate solutions | Does not reach accord | Virtual | In-person | Digital | Individual | Not groups | Highly structured | Not semi-structured |
Visual or digital-generated methods
Method | Revealing issues | Identifying allegiances and shared views | Generating solutions | Reaching accord | Virtual | In-person | Digital | Individual | Groups | Highly structured | Semi-structured |
---|---|---|---|---|---|---|---|---|---|---|---|
A process of having a large group, including experts and non-experts, share ideas or solve issues through activities, such as open challenges, contests, hackathons and innovation tournaments. | Reveals issues | Identifies allegiances and shared views | Generates solutions | Reaches accord | Virtual | In-person | Digital | Individual | Groups | Not highly structured | Semi-structured |
Creative arts 3, 5, 7, 23*+ Art genres, such as performance, writing, painting, photography, collage and installation art, as a method or as technical, communication or aesthetic element. | Reveals issues | Does not identify alliegances and shared views | Does not generate solutions | Reaches accord | Virtual | In-person | Digital | Individual | Not groups | Not highly structured | Not semi-structured |
Notes
- * Peer reviewed literature
- + Crowdsourcing
References
- Ottrey E, Jong J, Porter J. Ethnography in nutrition and dietetics research: a systematic review. J Acad Nutr Diet. 2018 Oct;118(10):1903-42.e10. DOI: 10.1016/j.jand.2018.06.002
- Mockford C, Staniszewska S, Griffiths F, Herron-Marx S. The impact of patient and public involvement on UK NHS health care: a systematic review. Int J Qual Health Care. 2012 Feb;24(1):28-38. DOI: 10.1093/intqhc/mzr066.
- Haijes HA, van Thiel GJMW. Participatory methods in pediatric participatory research: a systematic review. Pediatr Res. 2016 May;79(5):676-83. DOI: 10.1038/pr.2015.279
- Jones EL, Williams-Yesson BA, Hackett RC, et al. Quality of reporting on patient and public involvement within surgical research: a systematic review. Ann Surg. 2015 Feb;261(2):243-50. DOI: 10.1097/SLA.0000000000000768
- Tugault-Lafleur CN, Black JL, Barr SI. A systematic review of methods to assess children's diets in the school context. Adv Nutr. 2017 Jan 17;8(1):63-79. DOI: 10.3945/an.116.013144
- Allen EN, Chandler CIR, Mandimika N, et al. Eliciting adverse effects data from participants in clinical trials. Cochrane Database Syst Rev. 2018 Jan 16;1(1):MR000039. DOI: 10.1002/14651858.MR000039.pub2.
- Bradbury-Jones C, Isham L, Taylor J. The complexities and contradictions in participatory research with vulnerable children and young people: A qualitative systematic review. Social Sci Med. 2018 Oct;215:80-91. DOI: 10.1016/j.socscimed.2018.08.038
- Dalton J, Chambers D, Harden M, et al. Service user engagement in health service reconfiguration: a rapid evidence synthesis. J Health Serv Res Policy. 2016 Jul;21(3):195-205. DOI: 10.1177/1355819615623305
- Dawson S, Campbell SM, Giles SJ, et al. Black and minority ethnic group involvement in health and social care research: A systematic review. Health Expect. 2018 Feb;21(1):3-22. DOI: 10.1111/hex.12597
- Concannon TW, Fuster M, Saunders T, et al. A systematic review of stakeholder engagement in comparative effectiveness and patient-centered outcomes research. J Gen Intern Med. 2014 Dec;29(12):1692-701. DOI: 10.1007/s11606-014-2878-x
- Street J, Duszynski K, Krawczyk S, Braunack-Mayer A. The use of citizens' juries in health policy decision-making: a systematic review. Social Sci Med. 2014 May;109:1-9. DOI: 10.1016/j.socscimed.2014.03.005
- Dziopa F, Ahern K. A systematic literature review of the applications of Q-technique and its methodology. Methodology (Gott). 2011;7(2):39-55. DOI: 10.1027/1614-2241/a000021
- O'Brien MR, Clark D. Use of unsolicited first-person written illness narratives in research: Systematic review. J Adv Nurs. 2010 Aug;66(8):1671-82. DOI: 10.1111/j.1365-2648.2010.05349.x
- Miloyan B, McFarlane K, Vasquez-Echeverria A. The adapted autobiographical interview: a systematic review and proposal for conduct and reporting. Behav Brain Res. 2019 Sep 16;370:111881. DOI: 10.1016/j.bbr.2019.03.050
- Fadlallah R, El-Jardali F, Nomier M, et al. Using narratives to impact health policy-making: a systematic review. Health Res Policy Syst. Mar 5;17(1):26. DOI: 10.1186/s12961-019-0423-4
- Bradbury-Jones C, Isham L, Taylor J. The complexities and contradictions in participatory research with vulnerable children and young people: A qualitative systematic review. Soc Sci Med. 2018 Oct;215:80-91. DOI: 10.1016/j.socscimed.2018.08.038
- Koskan A, Klasko L, Davis SN, et al. Use and taxonomy of social media in cancer-related research: a systematic review. Am J Public Health. 2014 Jul;104(7):e20-37. DOI: 10.2105/AJPH.2014.301980
- Mairs K, McNeil H, McLeod J, et al. Online strategies to facilitate health-related knowledge transfer: a systematic search and review. Health Info Libr J. 2013 Dec;30(4):261-77. DOI: 10.1111/hir.12048