Skip to Main Content

Department of Health Library OLD: Research Process

Research Guide

The purpose of this page is to: 

  • Introduce the concept of evidence-informed research practice
  • Outline the steps involved in developing a research strategy
  • Provide tips on how to search

Evidence-Informed Research Practice

"Evidence-informed decision-making (EIDM) is ‘the process of distilling and disseminating the best available evidence from research, practice and experience and using that evidence to inform and improve public health policy and practice. Put simply, it means finding, using and sharing what works in public health"   [Evidence-informed decision-making: Information and tools [web-page]. Canadian Best Practices Portal. Accessed 2016 Aug]   

Useful Resources

What is Evidence-Based Public Policy?

The major goal of evidence-based policy development is to ensure that the experience, expertise and judgment of decision-makers is supported and resourced with the best available objective evidence and systematic research. Policy research is not expected to produce the solutions or decisions. It is meant to provide accurate, reliable and credible information, knowledge and analysis to inform public policy. The knowledge base it produces provides an important ingredient for the policy development process to reduce risk and improve outcomes, but it is not a substitute for the process

(Townsend T, Kunimoto B. Capacity, collaboration and culture: The future of the policy research function in the Government of Canada. [Canada]: Government of Canada. Policy Research Initiative. 2009 Mar. Available from: http://www.horizons.gc.ca/sites/default/files/Publication-alt-format/2009-0010-eng.pdf. Accessed 2016 Aug 11).

SUPPORT Tools for evidence-informed health policymaking (STP)

Complete article series available from: Health Research Policy and Systems. 2009:7(Suppl 1)

Further reading

"Evidence-informed public health involves integrating the best available research evidence into the decision-making process and using that evidence to inform and improve public health practice and policy. Additional factors – community health issues and local context; community and political preferences and actions; and public health resources – create the environment in which that research evidence is interpreted and applied"

A model for evidence-informed decision-making in public health. Fact sheet. 2012. National Collaborating Centre for Methods and Tools. Accessed 2016 Nov 10.

Further Reading

"The starting point for evidence-based management is that management decisions should be based on a combination of critical thinking and the best available evidence. And by ‘evidence’, we mean information, facts or data supporting (or contradicting) a claim, assumption or hypothesis. Evidence may come from scientific research, but internal business information and even professional experience can count as ‘evidence’. In principle, then, all managers base their decisions on ‘evidence’."

Evidence-based management graphic

Center for Evidence Based Management. What is evidence-based management?[Web-page]. Accessed 2016 Aug 15

Further reading

Evidence-based clinical practice involves the best external evidence, your clinical expertise and patient values & expectations

                       The Evidence-Based Medicine Triad

The Stages of the Research Process

Stages in the process:

  • Define
  • Search
  • Appraise
  • Synthesize, Adapt & Implement
  • Evaluate

FROM: National Collaborating Centre for Methods & Tools (Canadian)

 

DEFINE

The aim is to develop a "searchable" question to guide your research. A number of frameworks have been developed to assist in this process.  

The most well known & widely used is PICO - Patient (or Problem)-Intervention-Comparison-Outcome.

Other frameworks include:

  • PICOT- Patient (or Problem)-Intervention-Comparison-Outcome-Time Frame

  • PICOS- Patient (or Problem)-Interventions-Comparators-Outcomes-Context-Study design

  • EPICOT- Evidence-Population-Intervention-Comparison-Outcome-Timestamp [designed to address research recommendations on the effect of treatment]

  • PECODR-Population-Exposure-Comparison-Outcome-Duration-Results        Useful for case control studies and cohort studies. Duration can be used to clarify the length of the follow up period and the Results could be used for Number Needed to Treat or similar

  • SPICE - Setting (context)-Perspective-Intervention-Comparison-Evaluation Useful for qualitative studies that seek to evaluate a service. Perspective relates to users or potential users. Evaluation is how you plan to measure the success of the intervention

  • ECLIPSE - Expectation-Client group-Location-Impact-Professionals involved-Service                                                                                              This framework is useful for questions relating to health policy and management issues

  • PESTLE - Political-Economic-Social-Technological-Environmental-Legal    An analysis tool that can be used by organizations for identifying external factors which may influence their strategic development, marketing strategies, new technologies or organizational change

  • SPIDER - Sample-Phenomenon of Interest-Design (of study)-Evaluation-Research type                                                                                       Useful for qualitative or mixed methods research. Phenomenon of interest includes behaviors and/or experiences e.g. compliance

      FROM: http://www.workforce.southcentral.nhs.uk/pdfLit_search_protocols_2013.pdf.  Accessed Mar 3 2014

Resources to assist you in formulating your searchable question:

SEARCH

Once the question has been clarified, the next task is to begin to identify & locate the best available evidence-based research to answer your question:

 

What type and level of evidence is required?

The type of evidence you require is dependent upon whether your research is primarily qualitative or quantitative in nature, and whether the focus is clinical or non-clinical.

Some useful resources:

 

Where to search 

The published journal literature is not the only source that you may need to consult.  Other avenues to consider include statistical sources and the grey literature.

Some useful resources:

 

Qualitative and Non-Clinical Research

Levels of evidence Research question & Study type

Qualitative level of evidence pyramid

Center for Evidence-Based Management. What are the levels of evidence? [Web page]. Accessed 2016 Aug 15

Qualitative Research Study Types

Center for Evidence-Based Management. Which research design for which question? [Web page].Accessed 2016 Aug 15

 

Quantitative and Clinical Research

Levels of Evidence Research Question & Study Type

Evidence-Based Practice: Study Design.
Medical Center Library and Archives, Duke University
[Web-page] Accessed 2017 Mar 9

Query Type Study Type
Aetiology Cohort study
Diagnosis Prospective, blind comparison
Therapy RCT, cohort, case control, case series
Prognosis Cohort, case control, case series
Harm RCT, cohort, case control, case series
Prevention RCT, prospective

 

APPRAISE

Once the research has been identified the next step is to appraise (assess) the quality of research evidence.  

The appraisal process is designed to address the following three questions:

Is the study valid?

What are the results?

Are the results useful?

A number of tools & checklists have been developed to assist with this process. The type of  tool or critical appraisal checklist you will use is dependent upon what you are researching, including the query type, and what type of studies constitute best evidence to support that type of question.

Resources

A series of articles in the BMJ that explain how to read and interpret different kinds of research papers including diagnostic or screening tests, drug trials, qualitative reviews and economic analyses as well as articles about assessing methodology and interpreting statistics for the non-statistician.

Compendium of Critical Appraisal Tools for Public Health Practice 

An overview of critical appraisal tools that can be used to help incorporate the best research evidence into public health practice and policy. [Ciliska D, Thomas H, Buffett C. An Introduction to Evidence-Informed Public Health and A Compendium of Critical Appraisal Tools for Public Health Practice [Online pdf]. National Methods Centre for Collaborating Tools. 2008. Accessed 2016 Nov 19]

Critical Appraisal Skills Programme (CASP): Making sense of the evidence

CASP checklists 

AMSTAR: assessing methodological quality of systematic reviews

This tool was developed for general use and may be applied to assess the methodological quality of systematic reviews of public health interventions. It should be noted, however, that its use has only been tested on systematic reviews of randomized controlled trial designs. AMSTAR is being used by a number of groups, including the Canadian Agency for Drugs and Technologies in Health and The Cochrane Effective Practice and Organization of Care Group (EPOC)

 

Checklist for reviewing a randomized controlled trial of a social program or project, to assess whether it produced valid evidence

[Coalition for Evidence-Based Policy. Checklist for reviewing a randomized controlled trial of a social program or project, to assess whether it produced valid evidence [Online pdf]. 2010. Accessed 2016 Aug 23]

 

Continuum of evidence of effectiveness [Online pdf]. [US. National Center for Injury Prevention and Control. Division of Violence Prevention. Continuum of Evidence of Effectiveness. Accessed 2016 Aug 23]

Critical appraisal of qualitative research [Online pdf]. [Dr Kritika Samsi, King's College London]

 

AACODS checklist for the evaluation and critical appraisal of grey literature

 

Further Reading

Policy appraisal and health

"In order to make ... best use of ... resources, the costs and benefits of alternative policies, programmes and projects need to be carefully weighed up ...  Special problems can arise in identifying and weighing up the effects on health and health services of policies, programmes and projects. This guide provides specific advice in the health field ..." [United Kingdom. Department of Health. Policy appraisal and health [Online pdf]. Accessed 2016 Aug 23]

SYNTHESIZE AND ADAPT

These two steps are inter-related.  The "synthesize" step focuses on the big picture and how to determine what the evidence says in relation to the issue being investigated.  The "adapt" step focuses on (1) determining the relevance of the evidence and (2) how  best to tailor any policy recommendations or actions for practice to the local context.

SYNTHESIZE

This step is where you  interpret and form clear and actionable recommendations for practice based on the evidence found.

It is designed to answer the question:

  • What does the research evidence tell me about the issue?

 

Resources

From research to practice: A knowledge transfer planning guide

This guide expands on the John Lavis's "formulation of the evidence on knowledge transfer which he expresses as five principles that should guide knowledge transfer to maximize uptake. These principles can be expressed as five basic questions:

  • What (is the message)?
  • To whom (audience)?
  • By whom (messenger)?
  • How (transfer method)?
  • With what expected impact (evaluation)?"

[Reardon R, Lavis J, Gibson J. From research to practice: A knowledge transfer planning guide.Toronto, Ontario: Institute for Work & Health. 2006. pg ii. Accessed 2016 Aug 23]

 

 

 

 

SYNTHESIZE & ADAPT

These two steps are inter-related.  The "synthesize" step focuses on the big picture and how to determine what the evidence says in relation to the issue being investigated.  The "adapt" step focuses on (1) determining the relevance of the evidence and (2) how  best to tailor any policy recommendations or actions for practice to the local context.

 

ADAPT

This step focuses on answering the question

  • Can I use this research with my client, community or population?

This step is crucial to determine if the intervention will be relevant and successful in your community. To adapt evidence to the local context, consider these issues:

  • the public health issue
  • the intervention
  • the local community/intended population
  • the organization and any partners

Model for evidence-informed decision making in public health

(From: Australian Public Service Commission. Challenges of evidence-based policy making. Last update 2009 Jun 19. Available from: http://www.apsc.gov.au/publications-and-media/archive/publications-archive/evidence-based-policy. Accessed 2016 Aug 11.)

IMPLEMENT

This stage is designed to address the question:

  • How will I use the research evidence in my practice?

It focuses on figuring out how to use the adapted evidence in your local setting and involves the development of a detailed implementation plan.

Creating an implementation plan consists of these three critical steps:

  1. Conducting a situational assessment
  2. Planning a program
  3. Disseminating the intervention

 

Resources

Toolkit: Implementation of best practice guidelines. (2nd ed.)[ Registered Nurses' Association of Ontario. Toolkit: Implementation of best practice guidelines. (2nd ed.). [Web-page]. 2012. Accessed: 2016 Aug 23]

 

From research to practice: A knowledge transfer planning guide

This guide expands on the John Lavis's "formulation of the evidence on knowledge transfer which he expresses as five principles that should guide knowledge transfer to maximize uptake. These principles can be expressed as five basic questions:

  • What (is the message)?
  • To whom (audience)?
  • By whom (messenger)?
  • How (transfer method)?
  • With what expected impact (evaluation)?"

[Reardon R, Lavis J, Gibson J. From research to practice: A knowledge transfer planning guide.Toronto, Ontario: Institute for Work & Health. 2006. pg ii. Accessed 2016 Aug 23

 

Further reading

Implementing administrative evidence based practices: Lessons from the field in six local health departments across the United States  [Duggan K, Aisaka K, Tabak RG, Smith C, Erwin P, Brownson RC. Implementing administrative evidence based practices: Lessons from the field in six local health departments across the United States. BMC Health Services Research. 2015;15:221]

EVALUATE

The final step in the process helps you answer these two questions:

  1. Did we do what we planned to do?
  2. Did we achieve what we expected?

There are two kinds of evaluation:

  1. Process evaluation assesses whether the program or implementation plan was delivered as intended
  2. Outcome evaluation determines if the program or implementation achieved the anticipated results or if there were any unanticipated outcomes

Both evaluations involve these steps:

  • Identify outcomes of interest
  • Select indicators to measure these outcomes
  • Collect baseline data about outcomes of interest
  • Measure outcomes post-intervention to determine effectiveness of the program or implementation plan
  • Interpret these results to inform further planning and refine the program or implementation plan

Contact Us

Monday - Friday 8.00am - 4.00pm (excluding public holidays)          (08) 9222 4352          doh.library@health.wa.gov.au