Pay for Quality

KCE Reports 118 Pay for Quality 7
Searching, application of in- and exclusion criteria and quality appraisal was performed
by two reviewers independently (DDS and PVH). In case of non corresponding results,
consensus was sought, led by the third reviewer (RR).
To describe evidence about the equity dimension of P4Q models a systematic review
process on a selection of publications from the main database was performed by two
other reviewers (SW and PB).
2.1.1.1 Search strategy
The sources consisted in both phases of the following electronic databases: Medline,
Embase, the Cochrane Library, Web of Science, PsycInfo and Econlit.
An iterative procedure was followed. Publications were first judged on the basis of title
review, and excluded if clearly irrelevant. Subsequently, abstract review was performed
for all articles that remained after title selection and for the articles of which eligibility
was unclear. Finally a full text analysis was performed for all articles for which relevance
remained doubtful.
Appendix 1 and 2 give an overview of the search strategy including search string, limits,
and number of retrievals per database, respectively during phase 1 and 2. A combination
of MeSH terms and non MeSH terms was used wherever possible. A time period of
2000-2008 was applied in the search for systematic reviews (see Appendix 1). The
period 2004-2008 or 2005-2008 was used in searching additional primary studies,
depending on the date and the used database in the latest existing relevant high quality
systematic reviews. (see Appendix 2). In chapter 4 the reviews were not discussed.
They only served as a means to set the date in the search for additional primary studies.
Hence, only the primary articles included in the selected reviews, and the additional
primary articles were taken into account for this study.
After completion of the report, an additional search from January 2009 to July 2009 was
performed, to identify more recent relevant studies that could be of value to our
report. The results of this additional search are described in appendix 16C.
In addition, all references of relevant publications were screened for additional material.
Forward citation tracking was applied on all relevant publications. More than sixty
international experts were asked to provide any not yet retrieved relevant publication,
using a standard template. The experts were selected based on their own number of
publications with regard to P4Q, with a minimum of two publications.
As a final check the following journals were hand searched through their online
archives: the New England Journal of Medicine, the Journal of the American Medical
Association, the British Medical Journal, the Lancet, Medical care, Health Affairs, Health
Policy and Health Economics. This last step didn’t yield any unretrieved publication and
therefore confirmed a high level of completeness of the search strategy.
The decision to include peer reviewed literature only is based on various reasons.
During the first phase of the review (search for existing systematic reviews) a broad
comprehensive approach including grey literature was tested through the screening of
specific topic related websites and the use of a Google, Scirus and Sumsearch search
engine. The websites included the Institute of Medicine, the AHRQ, the Joint
Commission, the Leapfrog Group, the RAND corporation, the John Woods Richardson
Foundation, etc. next to governmental websites such as the CMS in the USA, NHS
resources in the UK and others in Australia. Although this identified many documents
concerning P4Q, almost none of these passed the phase of quality appraisal. Almost all
of these documents are aimed at a broad audience of P4Q users and therefore omit
detailed methodological specifications. Many grey literature sources about P4Q even
lack the use of references.
In addition, sources managed by health plans or employer groups showed a conflict of
interest, which was exemplified by a marketing style approach of P4Q. It was deemed
unreliable to include sources that almost sell P4Q as a product.