The REal Life EVidence AssessmeNt Tool (RELEVANT): development of a novel quality assurance asset to rate observational comparative effectiveness research studies.

Authors:
Jonathan D Campbell
Jonathan D Campbell
Skaggs School of Pharmacy and Pharmaceutical Sciences
United States
Robert Perry
Robert Perry
University of Kentucky
United States
Nikolaos G Papadopoulos
Nikolaos G Papadopoulos
2nd Pediatric Clinic
Toronto | Canada
Jerry Krishnan
Jerry Krishnan
University of Illinois at Chicago
Chicago | United States
Guy Brusselle
Guy Brusselle
Ghent University Hospital
Belgium
Alison Chisholm
Alison Chisholm
University of Aberdeen
United Kingdom
Leif Bjermer
Leif Bjermer
Lund University
Lund | Sweden
Michael Thomas
Michael Thomas
Wake Forest School of Medicine
United States

Clin Transl Allergy 2019 27;9:21. Epub 2019 Mar 27.

16Hôpital Cochin (APHP), University Paris Descartes (EA2511), Paris, France.

Background: Evidence from observational comparative effectiveness research (CER) is ranked below that from randomized controlled trials in traditional evidence hierarchies. However, asthma observational CER studies represent an important complementary evidence source answering different research questions and are particularly valuable in guiding clinical decision making in real-life patient and practice settings. Tools are required to assist in quality appraisal of observational CER to enable identification of and confidence in high-quality CER evidence to inform guideline development.

Methods: The REal Life EVidence AssessmeNt Tool (RELEVANT) was developed through a step-wise approach. We conducted an iterative refinement of the tool based on Task Force member expertise and feedback from pilot testing the tool until reaching adequate inter-rater agreement percentages. Two distinct pilots were conducted-the first involving six members of the Respiratory Effectiveness Group (REG) and European Academy of Allergy and Clinical Immunology (EAACI) joint Task Force for quality appraisal of observational asthma CER; the second involving 22 members of REG and EAACI membership. The final tool consists of 21 quality sub-items distributed across seven methodology domains: Background, Design, Measures, Analysis, Results, Discussion/Interpretation, and Conflict of Interest. Eleven of these sub-items are considered critical and named "primary sub-items".

Results: Following the second pilot, RELEVANT showed inter-rater agreement ≥ 70% for 94% of all primary and 93% for all secondary sub-items tested across three rater groups. For observational CER to be classified as sufficiently high quality for future guideline consideration, all RELEVANT primary sub-items must be fulfilled. The ten secondary sub-items further qualify the relative strengths and weaknesses of the published CER evidence. RELEVANT could also be applicable to general quality appraisal of observational CER across other medical specialties.

Conclusions: RELEVANT is the first quality checklist to assist in the appraisal of published observational CER developed through iterative feedback derived from pilot implementation and inter-rater agreement evaluation. Developed for a REG-EAACI Task Force quality appraisal of recent asthma CER, RELEVANT also has wider utility to support appraisal of CER literature in general (including pre-publication). It may also assist in manuscript development and in educating relevant stakeholders about key quality markers in observational CER.

Download full-text PDF

Source
http://dx.doi.org/10.1186/s13601-019-0256-9DOI Listing
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC6436213PMC

Still can't find the full text of the article?

We can help you send a request to the authors directly.
March 2019
3 Reads

Publication Analysis

Top Keywords

observational cer
24
quality appraisal
16
task force
12
inter-rater agreement
12
cer
12
appraisal observational
12
quality
9
observational
9
involving members
8
comparative effectiveness
8
cer evidence
8
force quality
8
observational comparative
8
assessment tool
8
evidence assessment
8
life evidence
8
real life
8
relevant
8
asthma cer
8
secondary sub-items
8

References

(Supplied by CrossRef)
Article in BMJ
GH Guyatt et al.
BMJ 2008
Article in Respir Med
K Herland et al.
Respir Med 2005
Article in Breathe.
D Price et al.
Breathe. 2015
Article in Lancet
M Rawlins et al.
Lancet 2008
Article in Am J Respir Crit Care Med
D Price et al.
Am J Respir Crit Care Med 2013
Article in Lancet Respir Med.
N Roche et al.
Lancet Respir Med. 2013
Article in Clin Transl Allergy.
N Roche et al.
Clin Transl Allergy. 2019
Article in J Manag Care Pharm
NA Dreyer et al.
J Manag Care Pharm 2014
Article in J Clin Epidemiol
KE Thorpe et al.
J Clin Epidemiol 2009
Article in Ann Am Thorac Soc
N Roche et al.
Ann Am Thorac Soc 2014
Article in Value Health
M Berger et al.
Value Health 2014

Similar Publications