December 8, 2016 | Author: Clemence Powers | Category: N/A
Download NHS ECONOMIC EVALUATION DATABASE HANDBOOK...
NHS ECONOMIC EVALUATION DATABASE HANDBOOK
April 2007
The internal guidance for abstractors writing for NHS EED has changed, therefore this report no longer reflects current practice and is in the process of being updated.
Centre for Reviews and Dissemination University of York, UK
Contents Acknowledgements
iii
Chapter I: About the NHS Economic Evaluation Database Project
1
Chapter II: NHS EED Process
5
Stage One: Searches Stage Two: Initial decision Stage Three: Obtain papers Stage Four: Final decision Stage Five: Allocate papers Stage Six: Write abstracts Stage Seven: Check abstracts Stage Eight: Edit abstracts Stage Nine: Load abstracts Quality control Chapter III: Guidance for Writing NHS EED Abstracts List of abbreviations Guidance for writing NHS EED abstracts Example F (A) Example F (B) Structure of an abstract for the NHS Economic Evaluation Database Basic principles of abstract writing for NHS EED 1. Subject of study 2. Key elements of study 3. Details about clinical evidence 4. Economic analysis 5. Results 6. Critical commentary 7. Implications 8. Other publications of related interest Figure one – Choice of comparator Figure two – Modelling Figure three – Type A study Figure four – Type B study Figure five – Benefit measure and utilities Figure six – Cost overview Figure seven – Resource use and prices/unit costs Figure eight – Other issues Chapter IV: Explanatory Notes E1 Terminology for the subject of study E2 Terminology for the key elements of study E3 Clinical evidence E4 Economic analysis E5 Methods of presenting results
6 9 9 10 11 11 11 12 12 12 13 13 15 17 21 25 27 29 31 33 39 47 51 53 55 57 58 59 60 61 62 63 64 65 65 67 71 83 89
Chapter V: Using the Database 1. How to access and search NHS EED 2. Marketing and dissemination
91 91 93
Chapter VI: Research Using NHS EED
95
References
101
Further Reading
107
ii
Acknowledgements This third edition of CRD Report 6 has been edited by Dawn Craig and Stephen Rice at CRD. The first edition was written by Alessandra Vanoli and Trevor Sheldon at CRD and Michael Drummond at the Centre for Health Economics (CHE), and the second edition was edited by Boyka Stoykova and John Nixon at CRD. We express our gratitude to members of the Advisory Group to the NHS EED project for their comments on earlier drafts of this report and to those abstractors who assisted in piloting the revised guidelines and abstract structure. We also greatly appreciate the contribution of CRD and CHE Staff to the project and of all commissioned abstractors, past and present: Raquel Aguiar Ibáñez Farhad Alirezanezhad Gohardani Federico Augustovski Anne-Marie Bagnall Marco Barbieri Gary Barton Lourdes Betegon Karen Bloor Clazien Bouwmans Yolanda Bravo Ken Buckingham Sheelah Connolly Jane Dalton Linda Davies Phil Davis Doriana Delfino Tracy Denby Mark Deverill Jacqueline Dinnes Paul Dolan Tarn Driffield David Epstein Sue Evans Teresa Feighan Elisabeth Fenwick Rachel Fleurence Sean Forrest Bethan George Laura Ginnelly Juan Gonzalez Perez Barbara Graham Susan Griffin Alison Hadnett Rodolfo Hernandez Christiane Hoffman Bruce Hollingsworth Naomi Holman Sarah Howard Cynthia Iglesias Marilyn James Tom Jefferson Emma Jefferys
Karl Kantor Urpo Kiiskinen Mueni Kilonzo Alice Knight Dimitra Lambrelli Jose Leal Ioana Littlewood Mirella Longo Paula Lorgelly Sandrine Loubiere Allan Low Ramon Luengo Fernandez Andrea Manca Roshni Mangalore Rob Manning Giuliano Masiero Lisa Maslove Anne Mason Ifigeneia Mavranezouli Douglas McCulloch David McDaid Marian McDonagh Borislava Mihaylova Paul Miller Zsolt Mogyorosy Zulma Montano Arevalo Antonia Morga Ann Morgan Miranda Mugford Ruben Mujica Jo Ann Mulligan Dominic Munro John Nixon Yumi Nixon Pedro Olivares Sue O'Meara Stephen Palmer Francis Pang Alison Partridge Katherine Payne Paolo Pesci Francisco Pozo-Martin Rimawan Pradiptyo
Jacob Puliyel Charles Qianhui Yan Nirmala Ragbir-Day Tracy Randall Rob Riemsma Oliver Rivero Arias Glen Robert Mairin Ryan Diane Sanderson Christoph Schmidt Louise Schmidt Tony Scott Anna Semlyen Phil Shackley Luca Sichel Turco Steven Simoens Russell Slack David Smith Lee Smith Ann Spencer Andrew Street Gillian Stynes Agota Szende Matthew Taylor Mariamma Thalanany Rod Thomas Sally Thompson Lesley Tilson David Torgerson Tsvetomira Tsenova Bronwyn Tunnage Hege Urdahl Maria Velasco Arantxa Viudes Andrew Walker Helen Weatherly Deborah Wilson Jane Wolstenholme Sarah Wordsworth YHEC (past & present members)
iii
An international advisory group assisted in the development of this project and continue to provide advice and guidance when required:
Martin Buxton Karl Claxton Jonathan Cooke Bernard Crump Cameron Donaldson Michael Drummond David Eddy
iv
Matthias Graf von der Schulenberg Simon Harding John Henderson Akinore Hisashige Dee Kyle Bryan Luce Miranda Mugford
Nick Phin John Posnett Joan Rovira Frans Rutten Trevor Sheldon Alan Shiell George Torrance Peter West
Chapter I About the NHS Economic Evaluation Database Project Jimmy Christie and Julie Glanville
Health technologies should only be considered for general use when their effectiveness and safety have been demonstrated. However, decision-makers are aware that, because of finite resources, the resource implications of health care interventions also need to be considered. If information on both costs and effectiveness is available, then resources should be allocated with a view to increasing (social) efficiency, taking into account policy on equity. Economic evaluations are conducted to provide decision-makers with information, which may help the development of more efficient health care programmes. However, reviews of economic evaluations have demonstrated that the quality of these studies is highly variable and their reporting often inadequate. Health care decision-making should be informed by the most reliable studies. Since 1994 the Centre for Reviews and Dissemination at the University of York has been commissioned by the English NHS R&D Programme to improve access to information about the costeffectiveness of health technologies and to provide assessments of the quality of such studies through the NHS Economic Evaluation Database (NHS EED). NHS EED contains structured abstracts of economic evaluations of health technologies in all languages published from 1994 onwards (selected studies are available for 1992 and 1993). Papers are abstracted if they are full economic evaluations of health technologies, relevant to the NHS. A detailed abstract structure has been developed to assist in the critical appraisal of each study and to ensure that the information is of maximum use to the intended audience of UK health care professionals, managers, policy-makers and academics. The database also includes other records:
116 selected records originally included in the DH Register of Cost-Effectiveness Studies (1994), and which have been mapped into the CRD abstract structure records which link through to abstracts of selected European economic evaluations written as part of the EURONHEED project (mostly for the period 2000-2005)
Bibliographic details only of the following types of study are included on the database for information:
partial economic evaluations (since August 2003) outcome valuation studies (since February 2004) costing studies (since 1994) methodological papers (since 1994) reviews of economic evaluations (since 1994)
To enhance the visibility of papers which have been selected for abstract, but for which abstracts have not yet been written, their bibliographic details are included in NHS EED as provisional abstracts. It is possible for health care professionals and researchers in the UK to request that particular provisional abstracts are ‘fast-tracked’. NHS EED is a companion to the CRD Database of Abstracts of Reviews of Effects (DARE), which is a collection of abstracts of good quality research reviews about the effects of health
1
care interventions and the organisation of health services. Both databases are accessible free of charge on the Internet (http://www.crd.york.ac.uk/crdweb/). NHS EED is a useful resource for health care decision-makers and researchers but requires promotion, awareness raising, feedback and research to fulfil its potential as a tool to promote the use of reliable results in health care decision-making. CRD staff have contributed to this process by marketing the database to relevant user groups, and raising awareness with gatekeeper groups such as librarians. We have also undertaken surveys and other research to assess the use and usefulness of NHS EED and we have carried out research activities on the methods and quality of economic evaluations to exploit the data held within the database. This report summarises the methods used to develop NHS EED, the progress made so far, and ways to access and search the database. It provides details of how studies are identified for inclusion, criteria for inclusion/exclusion of studies, and the guidance that the NHS EED project has developed for reporting critical abstracts of economic evaluations.
Development of the database Currently some 250 new records are added to the public database each month of which approximately 33% are full abstracts, and the remainder are partial economic evaluations, outcome valuation studies, cost, review or methodology studies. Chart 1 shows the growth in number of full abstracts and bibliographic references since 1995. It is anticipated that abstract numbers will continue to increase by some 60 per month for the foreseeable future.
16000
14000
12000
10000
8000
6000 4000 2000 0 End 95 Dec-96 Dec-97 Dec-98 Dec-99 Dec-00 Dec-01 Dec-02 Dec-03 Dec-04 Dec-05 Dec-06
Ye ar Ending
Full Abstracts Chart 1: Growth of NHS EED since 1995
2
Bibliographic References
Current content of the database A snapshot of the database in April 2006 shows the following breakdown of records (Chart 2).
30% Full Abstracts 0.3% EURONHEED Links 9% Methodology Studies 0.5% Outcome Valuation Studies 2% Partial Economic Evaluations 3.2% Provisional Abstracts 42% Cost Studies 13% Review Articles
Chart 2: NHS EED: breakdown by record type
3
4
Chapter II NHS EED Process Steven Duffy and Jimmy Christie
Maintaining and developing NHS EED is a complex process requiring the combined efforts of information officers, researchers, abstractors and the Database Administrator. The following diagram provides an overview of the main stages of this process:
NHS ECONOMIC EVALUATION DATABASE FLOW PROCESS InfO
1.SEARCHES
RFs
2. INITIAL DECISIONS
DbA
Search: MEDLINE, CINAHL, EMBASE, PsycINFO & Handsearch
Determine whether or not to obtain full papers. Classify identifiable Cost, Review, Methodology (CRM) studies.
3. OBTAIN PAPERS
Papers are ordered and obtained. Papers are ordered and obtained.
RFs
4.FINAL DECISIONS
Confirm final classification of papers, suitable for abstract or CRM studies.
5. ALLOCATE PAPERS
Papers, selected for abstract, are allocated to abstractors according to priorities.
6. WRITE ABSTRACTS
Abstracts are written, using pre-defined template and in accordance with guidelines in this report
RFs/QAG
7. CHECK ABSTRACTS
Abstracts are checked for accuracy and technical quality. Unsatisfactory abstracts returned for amendment.
InfO/DbA
8. PROOF READ ABSTRACTS
Abstracts are proof-read and MeSH indexing is added.
InfO
CAs
InfO
Abstracts are loaded onto public database.
9. LOAD ABSTRACTS
Abbreviations
Legend Person(s) responsible
PROCESS STAGE
Brief description
InfO – Information officer RFs – Research Fellows DbA – Database Administrator CAs – Commissioned Abstractors QAG – Quality Assurance Group members
Diagram 1: NHS EED flow process
5
Stage One: Searches In the first stage of the process, the Information Officer identifies potential economic evaluations by searching a variety of electronic and paper based sources, details of which are as follows:
1.1. MEDLINE The Ovid CD-ROM version of MEDLINE is used, producing, on average 160 references per month or approximately 1920 per annum. The search strategy used is as follows:
Economics/ exp "Costs and Cost Analysis"/ “Value of Life”/ Economics, Dental/ exp Economics, Hospital/ Economics, Medical/ Economics, Nursing/ Economics, Pharmaceutical/ 1 or 2 or 3 or 4 or 5 or 6 or 7 or 8 (econom$ or cost or costs or costly or costing or price or prices or pricing or pharmacoeconomic$).ti,ab.. (expenditure$ not energy).ti,ab. (value adj1 money).ti,ab. budget$.ti,ab. 10 or 11 or 12 or 13 9 or 14 letter.pt. editorial.pt. historical article.pt. 16 or 17 or 18 15 not 19 Animals/ Humans/ 21 not (21 and 22) 20 not 23 (metabolic adj cost).ti,ab. ((energy or oxygen) adj cost).ti,ab. 24 not (25 or 26)
1.2. Cumulative Index of Nursing and Allied Health Literature (CINAHL) The Ovid CD-ROM version of CINAHL is searched monthly resulting in 56 retrievals per month (approx. 650 per annum). The search strategy is as follows:
exp Economics/ exp Financial Management/ exp Financial Support/ exp Financing, Organized/ 6
exp Business/ 2 or 3 or 4 or 5 1 not 6 Health Resource Allocation/ Health Resource Utilization/ 8 or 9 7 or 10 (cost or costs or economic$ or pharmacoeconomic$ or price$ or pricing$).ti,ab. 11 or 12 editorial.pt. letter.pt. 14 or 15 13 not 16 Animal Studies/ 17 not 18 cochrane library.so. 19 not 20
1.3. EMBASE The Ovid online version of EMBASE is used, producing, on average 380 references per month or approximately 4560 per annum. The search strategy used is as follows:
Health Economics/ exp Economic Evaluation/ exp Health Care Cost/ exp Pharmacoeconomics/ 1 or 2 or 3 or 4 (econom$ or cost or costs or costly or costing or price or prices or pricing or pharmacoeconomic$).ti,ab. (expenditure$ not energy).ti,ab. (value adj2 money).ti,ab. budget$.ti,ab. 6 or 7 or 8 or 9 5 or 10 letter.pt. editorial.pt. note.pt. 12 or 13 or 14 11 not 15 (metabolic adj cost).ti,ab. ((energy or oxygen) adj cost).ti,ab. ((energy or oxygen) adj expenditure).ti,ab. 17 or 18 or 19 16 not 20 exp Animal/ exp Animal Experiment/ Nonhuman/ (rat or rats or mouse or mice or hamster or hamsters or animal or animals or dog or dogs or cat or cats or bovine or sheep).ti,ab. 22 or 23 or 24 or 25 exp Human/ exp Human Experiment/ 7
27 or 28 26 not (26 and 29) 21 not 30
1.4. PsycINFO The Ovid online version of PsycINFO is used producing, on average, 70 references per month or approximately 840 per annum. The search strategy used is as follows:
"Costs and Cost Analysis"/ "Cost Containment"/ (economic adj2 evaluation$).ti,ab,id. (economic adj2 analy$).ti,ab,id. (economic adj2 (study or studies)).ti,ab,id. (cost adj2 evaluation$).ti,ab,id. (cost adj2 analy$).ti,ab,id. (cost adj2 (study or studies)).ti,ab,id. (cost adj2 effective$).ti,ab,id. (cost adj2 benefit$).ti,ab,id. (cost adj2 utili$).ti,ab,id. (cost adj2 minimi$).ti,ab,id. (cost adj2 consequence$).ti,ab,id. (cost adj2 comparison$).ti,ab,id. (cost adj2 identificat$).ti,ab,id. (pharmacoeconomic$ or pharmaco-economic$).ti,ab,id. 1 or 2 or 3 or 4 or 5 or 6 or 7 or 8 or 9 or 10 or 11 or 12 or 13 or 14 or 15 or 16 (task adj2 cost$).ti,ab,id. (switch$ adj2 cost$).ti,ab,id. (metabolic adj cost).ti,ab,id. ((energy or oxygen) adj cost).ti,ab,id. ((energy or oxygen) adj expenditure).ti,ab,id. 18 or 19 or 20 or 21 or 22 (animal or animals or rat or rats or mouse or mice or hamster or hamsters or dog or dogs or cat or cats or bovine or sheep or ovine or pig or pigs).ab,ti,id,de. editorial.dt. letter.dt. dissertation abstract.pt. 24 or 25 or 26 or 27 17 not (23 or 28)
1.5. Handsearch In addition to the electronic sources listed above, a wide range of journals and grey literature sources are handsearched by the Information Staff. On average, handsearching retrieves approximately 350 references per annum. The journals and other sources searched are listed in Appendix 1 to this section. The search results are passed to the Database Administrator who prepares them for the next stage in the process – Initial Decision.
8
Stage Two: Initial decision In stage two of the process, one of the project Research Fellows scrutinises each record retrieved from the searches with the aim of deciding which category it belongs in, namely: whether or not the paper is likely to be a full economic evaluation or whether it should be classified as a cost study, methodology study, review article or outcome valuation study, or whether it should be rejected. In making this decision, the following criteria are applied:
Inclusion criteria for papers to be abstracted: Papers in any language are selected for complete abstracts if they are full economic evaluations. These are studies in which a comparison of two or more alternatives is undertaken and costs and outcomes are examined for each alternative. They are classified as cost-benefit analysis, cost-utility analysis or cost-effectiveness analysis (including costconsequences analysis).
Exclusion criteria for papers to be abstracted: Studies which are not full economic evaluations are not abstracted. For example: methodological papers; review articles; discursive analysis of costs/benefits; partial evaluation studies, such as cost analyses; efficacy or effectiveness evaluations; policy papers; cost of treatment/burden of illness papers; letters; editorials; book reviews; notes etc. not a relevant topic: e.g., diseases/interventions/forms of organisation of health care which are not relevant to the NHS.
Bibliographic details included in the Database The following papers are included in the Database as references: methodological papers costing studies reviews articles outcome valuation studies partial economic evaluations On average some 24% of references retrieved are rejected at the initial decision stage and a further 20% are pre-classified as cost, review, methodology or outcome valuation studies. In the next stage of the process, copies of full papers are obtained for the remaining 56%.
Stage Three: Obtain papers Following scrutiny by the Research Fellows, the results of the initial decision stage are passed to the Database Administrator who orders copies of the full papers for:
References which are assessed as likely to be suitable for full abstract References for which insufficient information was available for classification to be possible without reference to the full paper
9
(Note that full papers are not obtained for those references which, at this stage, can be firmly categorised as cost studies, methodology studies, review articles or outcome valuation studies). Papers are obtained from the following sources:
British Library British Medical Association Library Royal College of Surgeons Library JB Morrell Library, University of York Nursing Union List of Journals Electronic Journals available on the Internet
As papers are received from the various sources they are passed to one of the project Research Fellows for the next stage in the process – Final Decision.
Stage Four: Final decision At the final decision stage, the project Research Fellows scrutinise each paper and decide on its final classification, applying the inclusion criteria described previously. Based on all papers which have been subject to final decision, the percentage breakdown by classification is as follows: Selected for full abstract Cost studies Methodology studies Review articles Outcome valuation studies Partial economic evaluations Rejected
23% 32% 7% 10% 0.5% 1.5% 26%
Following final decision, all papers are passed to the Database Administrator who disposes of them as follows: a. The bibliographic details of all papers selected for full abstract are loaded onto the public database as provisional abstract records. Any database user with a special interest in the subject matter of a provisional abstract may request that the writing of the full abstract be given enhanced priority. Papers selected for abstract are then stored while awaiting allocation to an abstractor. b. Bibliographic details of cost, review, methodology, outcome valuation studies and partial economic evaluations are loaded onto the public database and the original papers are recycled. c. Rejected papers are registered on an internal database with the reason for rejection.
10
Stage Five: Allocate papers
Papers selected for abstract are allocated to abstractors by the Database Administrator according to the following priorities: Priority 1. Any provisional abstracts for which a high-priority abstract has been requested are allocated immediately to an appropriate abstractor. The aim is to ensure that the person initiating the request receives an advance copy of the abstract within two to three weeks and that, if possible, the final version is loaded onto the public database with the next data upload. Priority 2. Any studies completed in the UK (and relevant to the UK NHS context) are allocated to the first available abstractor. Priority 3. In the absence of any Priority 2 papers, any studies not completed in the UK, but that are of particular relevance or importance to the NHS, are allocated to the first available abstractor. Priority 4. All other papers are allocated to abstractors in order of date of publication (earliest first).
Stage Six: Write abstracts On receipt of the paper(s) the abstractor writes an abstract for each paper in accordance with the guidelines contained in Chapter III of this report by summarising the relevant material in the paper in the appropriate section of the template. On completion, the abstractor returns the original papers and the completed abstract templates to the Database Administrator who arranges payment, carries out a virus check of the disks and prepares hard copies of each abstract for checking by the project Research Fellows.
Stage Seven: Check abstracts All written abstracts are checked by one or other of the project Research Fellows for quality technical accuracy and compliance with the guidelines laid down in this report. Based on this technical check, the Research Fellow will, where appropriate, provide feedback and guidance to the abstractor for future reference. In addition, any abstracts which are considered to be of an unsatisfactory quality are returned to the appropriate abstractor for amendment. Matters arising from individual abstracts that are considered to have wider implications for the quality of abstracts in general are discussed by the Quality Assurance Group (QAG). Checked abstracts are returned to the Database Administrator for the next stage in the process. 11
Stage Eight: Edit abstracts
Once checked by the Research Fellow, abstracts are passed to the Database Administrator who checks that they conform to layout rules, then adds indexing terms, carries out a spell check and a further virus check, reformats the templates ready for loading onto the production database and passes the completed abstracts to the proof reader for final editing.
Stage Nine: Load abstracts Once the proof reader has edited the abstracts, they are loaded onto an internal database (the production database) to await final loading onto the public database. Records are loaded onto the public database in bulk on the last working day of each calendar month, following which a full list of all records added in that month is forwarded to everyone who has subscribed to the NHS EED e-mail alert service. After loading, all papers, disks and edited abstracts are returned to the Database Administrator for archiving. Once abstracts are available on the public database, the Database Administrator sends a copy of the abstract to the original author with a letter inviting comment and feedback.
Quality control A number of strategies are in place to ensure the quality of the Database:
12
A strict process of quality control has been implemented to ensure that the structured abstracts are accurate, comprehensive and understandable. When the abstract has been completed, it is checked and edited by a CRD health economist. If the quality of the abstract is not satisfactory, the abstract is returned to the original abstractor for revision.
To deal with regular quality issues, a Quality Assurance Group has been formed. It is chaired by Professor Michael Drummond and the group is staffed by health economists who meet regularly to discuss methodological and quality issues related to the compilation of NHS EED abstracts.
In addition, abstractors are kept informed of quality issues via the project e-group, training sessions and regular feedback.
The input of the Advisory Board is also useful for assistance and advice in resolving methodological and policy issues.
Following completion of abstracts, copies are sent to the original authors for information. Authors are invited to reply with corrections to factual errors, further information and details of related research. Where applicable this information is added to the abstracts.
Chapter III Guidance for Writing NHS EED Abstracts Dawn Craig, Stephen Rice, Raquel Aguiar-Ibáñez, Julie Glanville, Jos Kleijnen, Michael Drummond
LIST OF ABBREVIATIONS CBA CCA CEA CHE CI CINAHL CODECS CRD CRM CUA DALY DARE DH EQ-5D EURONHEED HYE ITT NHS NHS EED NICE NNT QAG QALY RCT RD RR SD WTP
Cost-benefit analysis Cost-consequences analysis Cost-effectiveness analysis Centre for Health Economics, University of York Confidence interval Cumulative Index of Nursing and Allied Health Literature COnnaissances et Décision en ÉConomie de la Santé Centre for Reviews and Dissemination Cost, review and methodology studies Cost-utility analysis Disability-adjusted life year Database of Abstracts of Reviews of Effects, CRD Department of Health EuroQol European Network of Health Economic Evaluation Databases Healthy-Years Equivalent Intention to treat National Health Service National Health Service Economic Evaluation Database National Institute for Health and Clinical Excellence Number needed to treat Quality Assurance Group Quality-adjusted life year Randomised controlled trial Risk difference Relative risk Standard deviation Willingness-to-Pay
13
14
Guidance for writing NHS EED abstracts NHS EED makes information on published economic evaluations accessible in a userfriendly format. The aim of the database is to ensure that its contents are of maximum use to its potential audience of health care managers, policy-makers, researchers, academics and other professionals. In order to appreciate the potential value of research evidence such as economic evaluations, it is important to critically appraise each study and not just to accept the authors’ conclusions at face value. The critical analysis of each study to be written in the form of an abstract is undertaken systematically, following the guidelines in this section. The guidelines are broadly based on existing criteria (Drummond and Jefferson, 1996) for judging the quality of economic evaluations of health care technologies. The aim of a NHS EED abstract is to provide a structured summary of the study and to facilitate an understanding of the methods used as well as giving an assessment of the study’s overall quality. The NHS EED abstract also enables a comparison of studies and highlights any features of special interest. Commentary fields contain summaries of key features that affect the usefulness of the evaluation and give an independent critical view on the study’s conclusions and implications. The structure of the NHS EED abstract allows separate consideration of the quality of the evidence of clinical effectiveness as well as economic evidence. It also distinguishes between economic evaluations carried out alongside a single effectiveness study and those that use multiple sources of effectiveness evidence, which is often the case in studies with economic models. The abstract structure has been developed to be very general and therefore usable with studies with different methodologies.
How to use the guidance The guidance follows the structure of the abstract and is presented under the same headings and subheadings. A detailed description is given of what should be included in each field. Further guidance is available in the appendix. All terms marked with an asterisk (*) are briefly explained in the explanatory notes; in addition further reading is suggested in the “Useful References” list.
Classifications of full economic evaluations All full economic evaluations (F) are classified by the NHS EED team into one of the following categories: F (A) or F (B). One of these classifications is written at the top of each paper supplied for abstract. An (A) study derives all its clinical and epidemiological evidence from a single study. A (B) study derives its clinical and epidemiological evidence from more than one source. The sources of clinical data for a (B) study may be varied, e.g. literature, from hospital records, non-published literature or an expert panel. If the study is an (A) study then the Single Source fields should be completed in section 3 for clinical evidence. If the study is a (B) study then the Multiple Source fields should be completed. Abstractors are encouraged to check the classification before completing the abstract. Occasionally, errors do occur. If in doubt, confirm the correct classification by contacting the
15
NHS EED team. We have included some common classification queries below. The aim of these is to help you to think about where your paper may fit into the classification system, and why. If you are still in any doubt, please contact an NHS EED team member.
A few query scenarios
A study that extrapolates outcomes beyond the end of a clinical trial, using only data from the trial: An F (A) study: explain the extrapolation methods in the modelling field.
A study that is predominantly based on an RCT but derives data from a cohort study for the base line natural history of disease of one strategy in the model or derives treatment incidence or compliance data from an expert: An F (B) study.
A cohort study that monitors the prescribing patterns of pharmacists without guidelines and uses a panel of experts to determine the appropriate prescription given guidelines: An F (A) study: explain the approach in the Study Design field.
Where to report information: In all cases only complete those fields relevant to the classification of the paper. The pen symbol indicates that you need to report the requested information. You should also state when the requested information is missing or not explicitly stated in the paper.
16
Example F (A) NHS Economic Evaluation Database (NHS EED) - Full record display Single dose brachytherapy versus metal stent placement for the palliation of dysphagia from oesophageal cancer: multicentre randomised trial Homs M V, Steyerberg E W, Eijkenboom W M, Tilnaus H W, Stalpers L J A, Bartelsman J F , van Lanschot J J, Wijderman H K, Mulder C J, Reinders J G, Boot H, Aleman B M, Kuipers E J, Siersema P D Source Year published Volume Record Status
Health technology
Type of intervention Hypothesis/study question
Economic study type Study population
Modelling and statistical extrapolation
Setting
Lancet 2004 364 This record was compiled by CRD commissioned reviewers according to a set of guidelines developed in collaboration with a group of leading health economists. The study compared the use of single-dose brachytherapy and selfexpanding metal stents in the palliative treatment of patients with oesophageal cancer. Palliative care. The study compared the effect of alternative treatments on dysphagia, health related quality of life and on costs. The health technologies evaluated were reported to be commonly used for palliation of oesophageal obstruction due to inoperable cancer, but their relative merits were unknown. The authors undertook a clinical trial to compare the outcomes achieved. Although the authors stated that they had adopted a societal perspective, the costing reported in the paper appeared to suggest that the perspective was actually that of a hospital. Cost-effectiveness analysis The study population comprised patients with inoperable cancer of the oesophagus or oesophagogastric junction resulting from metastatic disease, or patients in a poor medical condition with a dysphagia score of 2 to 4. The exclusion criteria included: a tumour length greater than 12 cm; a tumour growth within 3 cm of the upper oesophageal sphincter; deep ulceration or trachea oesophageal fistula; and macroscopic or microscopic tumour growth into the tracheal lumen. Further exclusion criteria were the presence of a pacemaker and previous radiation or stent placement. To allow a comparison of the outcomes between groups at different time points, a linear regression model (with a restricted cubic spline function and an interaction term between time and treatment groups) was used to estimate the time profile of dysphagia score for each group. Survival was calculated and compared using Kaplan Meier curves and the log-rank test. Survival was combined with dysphagia score to calculate dysphagia-adjusted survival. The interventions were provided by a secondary care provider in an inpatient setting. The geographical location of this study was the Netherlands. The effectiveness and resource data were collected from December 1999 to February 2003. The price year was 2002. The costing was undertaken prospectively on the same patient sample that provided the effectiveness data.
Dates to which data relate Link between effectiveness and cost data Study sample The authors reported that their study had 90% power to detect improvement in dysphagia score by one grade at 30 days for an improvement in dysphagia of 89% after stent placement and 70% after brachytherapy. All patients meeting the inclusion criteria were eligible.
17
Study design
Analysis of effectiveness
Effectiveness results
The initial sample comprised 253 consecutive patients of whom 44 (17%) refused to participate. The remaining 209 were randomly assigned to a treatment group, 101 being allocated to brachytherapy and 108 to stent placement. No patients were reported as having been excluded from the initial sample. However, post randomisation, 3 patients died before treatment began, one was declared unfit on the day of surgery, three had problems during endoscopy, and 2 did not fulfil the inclusion criteria. This resulted in 95 patients finally receiving brachytherapy and 101 receiving stents. The study was a multi-centre randomised controlled trial. Randomisation was stratified according to the location of the tumour and by the administration (or not) of chemotherapy prior to treatment within the trial. Randomisation was undertaken by telephone to members of the trial office. Blinding of the patients and clinicians does not appear to have been possible. Recruitment to the trial took place between December 1999 and July 2002, and all patients were followed-up until February 2003, resulting in variable lengths of followup. No patients were lost to follow up. The primary outcome was the dysphagia score at follow up stages. Other outcomes included survival, rates of complications, and health related quality of life scores. The analysis was conducted on an intention to treat basis. Differences at baseline were adjusted for in the regression analysis. This was achieved by including the baseline scores of the dependent variable. At 30 days post treatment, the improvement in dysphagia score did not differ significantly between the two interventions. The score had increased by at least one grade in 73% of the brachytherapy patients and in 76% of the stent placement patients, (p=0.61). At 60 days, the dysphagia scores among patients who received brachytherapy were better than those for patients who received stent placement. This apparent superiority was maintained until about 350 days after surgery, when the 95% confidence intervals (CIs) for the two treatments again overlapped. In terms of survival, patients who received brachytherapy also experienced more days without dysphagia than patients who received a stent placement (115 days compared with 82 days. The difference was 33 days (95% CI: 1 to 64 days; p=0.15)). In addition, median survival was longer with brachytherapy than with stent placement (155 days versus 145 days; p=0.23)
Clinical conclusions
Measure of benefits used in the economic analysis
Direct costs
18
Complications tended to be more frequent after stent placement, (p=0.02). Brachytherapy showed superiority over time on the majority of health related quality of life scales. The average difference over time was 6.5 points in favour of brachytherapy for role functioning and 2.7 points in favour of brachytherapy for global health status. The time profiles of the various quality of life changes were complex and are best viewed graphically. Although there was no difference between the groups at 30 days, brachytherapy subsequently showed greater improvements in dysphagia scores than stent placement. The authors did not derive a summary measure of benefit. In effect, a cost consequence analysis was performed. See ‘Analysis of Effectiveness’ for the clinical outcomes measured. The benefits were not discounted. The study reported the direct costs to the health service. These were the initial cost of treatment, hospital stay, re treatment costs, medical services and costs in the community. Resource use was determined using a micro costing methodology. Neither the source of the unit costs nor their values were reported. The costs were reported as the
Productivity costs Currency Statistical analysis of costs
Methods used to allow for uncertainty Estimated benefits used in the economic analysis Cost results
Synthesis of costs and benefits Author’s conclusions
CRD Commentary
average cost per patient. The costs were not discounted. The price year was 2002. Productivity costs were not considered. Euros (EUR) A bootstrap technique with 2000 replicates was used to determine the significance of the difference, which was reported as a p-value. Cost differences were reported for cost categories, specifically, total treatment, total intramural care, medical procedures, extramural care, and total costs. The examination of uncertainty was restricted to the bootstrapping of the per capita cost. (See ‘Statistical Analysis of Costs’ reported previously.) See the ‘Effectiveness Results’ section. The cost of subsequent stents after the initial treatment was separately reported, as was the cost of treatment in the community after discharge from hospital. The average cost per patient was EUR 8,135 for brachytherapy and EUR 8,215 for stent placement was. The costs and benefits were not combined. Brachytherapy gave better relief of dysphagia over longer follow-up periods and with fewer complications than stent placement. The longterm costs were comparable for both treatments and therefore costs should not play an important role in the choice of treatment. Selection of comparators: A justification was provided for the technologies compared. They are both commonly used in the authors’ setting but their relative merits were unknown. You should decide if these represent valid comparators in your own setting. Validity of estimate of measure of effectiveness: The analysis was based on a randomised controlled trial. Power calculations were performed to ensure that the size of the study sample was adequate. The authors described the study sample in some detail, thus enabling readers to form a reasonable impression of the extent to which their own patients are comparable with those in the study. The fact that the patients were enrolled sequentially (rather than selectively) may provide the reader with some reassurance that the sample was typical of the study population. Methods of randomisation, length of stay and loss to follow-up were all reported, suggesting that the internal validity of the study is likely to be reasonably good. However, while the nature of the intervention would have prevented blinding of the patients and clinicians, the blinding of those researchers who collected the outcome information should have been possible. This possibility was not discussed, but it would have helped to minimise potential biases in the data analysis. The inclusion of baseline observations in the regression analysis conforms to recommended practice for handling potential confounders. Validity of estimate of measure of benefit: The authors did not derive a summary measure of benefit. In effect, a cost consequence analysis was performed. Validity of estimate of costs: The study reported that costs were collected from a 'societal perspective’; however, there was no attempt to consider productivity costs. In fact, a hospital perspective appeared to have been adopted. The detailed micro costing methodology may have precluded the 19
possibility of detailing all the cost components, with the result that the source of the price information was not specified. It is therefore not possible to comment on the generalisability of the costing. The fact that the study was undertaken in the Netherlands would imply that you, as the reader of this abstract, should consider whether Dutch costs are likely to be similar to the costs in your own health setting. Additionally, although the costs were incurred during a 3.5 year period, the authors did not carry out discounting. As the authors observed, the use of bootstrapping in the analysis of hospital costs does address the problems that can arise from highly skewed cost data. Other issues:
Implications of the study
Country code Subject index terms status Subject index terms
Funding body Accession number Database entry date Language published in Address for correspondence
The time profile during which health changes materialised was quite complex and appears to have been dealt with in a relatively sophisticated and appropriate manner. The authors pointed out the importance of obtaining longitudinal data on quality of life. In addition, they compared their findings with those from other studies. The authors noted that their study specified a particular regimen for the brachytherapy, pointing out that alternative regimens require further study. The authors also considered the effect that stents from different manufacturers might have had, but concluded that there was no evidence of differences in stent performance with respect to functional outcome. Although the authors said little about the limitations of their findings, they appear to have provided a balanced discussion. Brachytherapy appears to be the preferred option for the treatment of patients with progressive dysphagia due to carcinoma of the oesophagus or oesophagogastric junction, the exception being, those patients with short life expectancies for whom stent placement might provide better immediate relief. No recommendations for further research were made by the authors. Netherlands Subject indexing assigned by NLM: Aged; Brachytherapy/ae (adverse effects); Brachytherapy; Comparative Study; Deglutition Disorders/et (etiology); Deglutition Disorders/th (therapy); Esophageal Neoplasms/co (complications); Esophageal Stenosis/et (etiology); Esophageal Stenosis/th (therapy); Female; Humans; Male; Metals; Palliative Care; Quality of Life; Recurrence; Research Support, Non U.S. Gov't; Stents/ae (adverse effects); Stents This study was financially supported by the Health Care Insurance Council. 22004008386 31 January 2007 English Dr Peter D Siersema, Department of Gastroenterology and Hepatology, Erasmus MC/University Medical Centre, Rotterdam, PO Box 2040, 3000 CA Rotterdam, Netherlands.
[email protected]
NHS Economic Evaluation Database (NHS EED) Produced by the Centre for Reviews and Dissemination Copyright © 2007 University of York
20
Example F (B) NHS Economic Evaluation Database (NHS EED) - Full record display Potential cost-effectiveness of a family-based program in mild Alzheimer's disease patients Shelledy D C, McCormick S R, LeGrand T S, Cardenas J, Peters J I Source Year published Volume Record Status
Health technology
Type of intervention Hypothesis/study question
Economic study type Study population Modelling and statistical extrapolation
Setting Dates to which data relate
Clinical and epidemiological data
Data sources
Methods used to obtain
European Journal of Health Economics 2004 5 This record was compiled by CRD commissioned reviewers according to a set of guidelines developed in collaboration with a group of leading health economists. The use of a cognitive behavioural family intervention (CBFI) programme combined with current practice was compared with current practice alone in helping informal caregivers to postpone the need to transfer Alzheimer's disease (AD) patients to a nursing home. The CBFI programme consisted of short courses in rehabilitation centres with the support of dementia family care coordinators. The courses included physical and recreational training for AD patients, and psychological as well as educational support and counselling for the caregivers. Current practice consisted of different forms of community services and periodic institutional care. Information and education in health. The objective of the study was to examine the cost effectiveness of the CBFI programme combined with current practice compared with current practice alone. The outcomes of interest occurred over a period of time longer than the duration of any clinical trial, so the outcomes were modelled using a Markov model. The authors did not explicitly report a perspective for their study. However, they did state that the study could inform policy decisions at the societal level, indicating that a societal perspective had been adopted. Cost-utility analysis. The study population included patients with mild AD. No further details of the study population were provided. A Markov model was used to model the disease progression. The model was based, with slight modifications, on one developed in Neumann et al. (1999), and Claxton et al. (2001) (see ‘Other Publications of Related Interest’ below for bibliographic details). The time horizon was not explicitly stated in this paper. However, a 5-year care-giving period was mentioned. The health states were reported. The setting for the study was the community and institutional care. The economic study was carried out in Finland. The effectiveness data used to populate the model came from studies published between 1996 and 2001. The price year was 2001. The dates to which the resource use referred were not reported. The clinical parameters associated with the programme included the transition probabilities for the different stages of AD (including death), and the effect of the CBFI programme on delay to nursing home admission. The baseline transition probabilities for a patient receiving current care were derived from a published economic model of Alzheimer=s disease (Neumann et al. 1999). The effectiveness data for CBFI combined with current care were derived from a published study (Mittelman et al. 1996, see ‘Other Publications of Related Interest’ below for bibliographic details). The design of the clinical study reported in Mittelman et al. (1996) was not clear. The process used to identify the data was not reported. No inclusion 21
data
criteria were specified for any parameters. The method used to select the estimates was neither reported, nor discussed.
Measure of benefits used in the economic evaluation
The measure of benefit used was the quality-adjusted-life-years (QALYs). Quality of life weights were derived from the published economic model on which this analysis was based (Neumann et al. 1999). The benefits were discounted at a rate of 5%. The community and institutional health service costs were included in the analysis. Separate patient expenditure does not appear to have been included. Costs related to informal carers were considered. However, the authors valued informal carers according to productivity loss to firms and, as the informal carers were pensioners, did not attribute costs to them. Institutional care, day care, home nursing, visits to the neurologist, home help service and meals on wheels were costed. The resources used were derived from two Finnish municipal health centres. The unit costs were mainly taken from national Finnish published unit costs. The costs were discounted at an annual rate of 5%. The price year was 2001. Resources and unit costs were reported separately. No productivity losses were considered. Euro (EUR) No statistical analyses of the costs were conducted as the objective of the study was to produce a cost-utility measure. Parameter uncertainty was investigated through probabilistic sensitivity analysis. All the parameters in the model were assigned prior probability distributions. The authors gave detailed descriptions of the derivation of these distributions. No expected-value-ofinformation analysis was performed. The mean QALYs were 1.88 (95% confidence interval, CI: 1.71 to 2.06) for the CBFI programme and 1.87 (95% CI: 1.72 to 2.05) for current practice. The average cost of the CBFI programme was EUR 43,933 (95% CI: 19,785 to 71,026) and the average cost of the comparator was EUR 46,925 (95% CI: 19,073 to 75,740). The authors showed that the intervention was the dominant strategy, as it was less costly and more effective than the alternative. However, based on the 95% CIs, the differences between the mean estimates were not statistically significant. Further, the CBFI programme slightly decreased the quality of life of informal caregivers. Again, the difference between the quality of life values was not statistically significant.
Direct costs
Productivity costs Currency Statistical analysis of costs Methods used to allow for uncertainty
Estimated benefits used in the economic analysis Cost results
Synthesis of costs and benefits
The results of the 1,000 simulations from the AD model with 95% uncertainty showed that 71.3% of iterations were in the quadrant that showed that the CBFI programme offered more QALYS at lower costs.
Author’s conclusions
22
The use of the acceptability curve approach showed that the probability that the CBFI programme was cost effective for AD patients was over 0.9 at all values of willingness-to-pay per gained QALY. Based on current information, the cognitive-behavioural family intervention (CBFI) programme is a potentially cost saving option, whilst being as equally effective as current practice.
CRD Commentary
Selection of comparators: Although no explicit justification was provided for the comparator used, it would appear to represent current practice in the authors’ setting. You should decide if the comparator represents current practice in your own setting. Modelling: The model structure was presented graphically. Details such as the time horizon and cycle length were not reported, although the authors referred to the original model (Neumann et al. 1999) that they modified for use in this paper. Both the model parameters and their sources were fully reported. The authors investigated uncertainty in the model parameters through the use of prior distributions about the model parameters. All parameter distributions were clearly defined and justified. The authors presented their results in full and provided a cost-effectiveness plane and a cost-effectiveness acceptability curve. Validity of estimate of measure of effectiveness: The authors combined data from an existing model with data from a single, published clinical study. No systematic search for data was reported. The parameters for the model were mainly derived from a published economic evaluation and a single, published clinical study. It is not possible to judge the validity of the data given the information reported in this paper (see Neumann et al. 1999 and Mittelman et al. 1996 for further details). Validity of estimate of measure of benefit: The estimation of health benefits (QALYs) was modelled using a Markov model. The methods to estimate the utility weights were not described as they were taken from a published paper. Validity of estimate of costs: Though not explicitly stated, a societal perspective appears to have been adopted. The authors considered all costs apart from productivity loss associated with the patients, perhaps because of the nature of the condition. The sources of the resource use and unit cost data were reported. The costs were discounted at an annual rate of 5%, which would appear appropriate in this instance. This may have implications for the generalisability of the study beyond the study setting. The utilisation data estimates were assigned prior distributions to characterise their uncertainty. The cost data were well reported. Other issues: The authors did not compare their findings with those from other studies, so the extent to which their results agree, or disagree, with other published results cannot be determined. The authors did not address the issue of the generalisability of their results to other settings. The results of the study do not appear to have been presented selectively. The authors’ conclusions would appear to be an adequate reflection of the scope of the analysis. The authors reported a number of further limitations to their study. First, there were problems with data availability, especially for the estimates of uncertainty of the parameters. Second, there were problems with 23
Implications of the study
Other publications of related interest
quality of life estimates for both the caregiver and dementia sufferer. The authors indicate that more research is needed to assess the health-related quality of life of AD caregivers and the preferences of patients. Neumann PJ, Hermann RC, Kuntz KM, Araki SS, Duff SB, Leon J, et al. Cost-effectiveness of donepezil in the treatment of mild or moderate Alzheimer’s disease. Neurology 1999;52:1138-45. Claxton K, Neumann PJ, Araki S, Weinstein MC. Bayesian value-ofinformation analysis: an application to a policy model of Alzheimer’s disease. Int J Technol Assess Health Care 2001;17:38-55.
Country code Subject index terms status Subject index terms
Funding body Accession number Database entry date Language published in Address for correspondence
Mittelman MS, Ferris SH, Shulman E, Steinberg G, Levin B. A family intervention to delay nursing home placement of patients with Alzheimer’s disease. JAMA 1996;276:1725-31. Finland Subject indexing assigned by NLM: Aged; Alzheimer Disease/ec (economics); Bayes Theorem; Caregivers/ec (economics); Caregivers/px (psychology); Comparative Study; Cost Benefit Analysis; Finland; Homes for the Aged; Humans; Nursing Homes; Quality Adjusted Life Years; Research Support, Non U.S. Gov't; Social Support Supported by the Yrjo Jahnsson Foundation. 22004008327 31 January 2007 English J Martikainen, Center for Pharmaceutical Policy and Economics, Department of Social Pharmacy, University of Puopio, P.O. Box 1627, FIN-70211 Kuopio, Finland.
[email protected]
NHS Economic Evaluation Database (NHS EED) Produced by the Centre for Reviews and Dissemination Copyright © 2007 University of York
24
Structure of an abstract for the NHS Economic Evaluation Database Study subject (1)
Key elements (2)
Health technology
Economic study type
Disease
Clinical evidence (3)
Single source Multiple sources
Economic analysis: methods (4)
Benefits {benefit measure, methods of utility valuation studies}
Economic analysis: results (5)
Benefits
Synthesis {C/E ratios,
Sensitivity analysis
Modelling Study question and perspective
Comparators Modelling
Study population Intervention type
Critical commentary (6)
Effectiveness validity Health benefit validity
Sensitive parameters}
Setting
Costs {scope, approach, source, analysis, reporting}
Data dates
Cost validity Costs
Other issues
Effectivenesscost link Implications for practice and research (7)
Clinical evidence classifications Single source (3A)
Multiple sources (3B)
Study sample Study design Effectiveness analysis Effectiveness results Clinical conclusions
Clinical & epidemiological data used in the analysis Data sources Methods used to obtain data
25
26
Basic principles of abstract writing for NHS EED
The overall objective of the abstract is to report succinctly the main features and results of the paper in an informative manner, whilst highlighting the methodological strengths and weaknesses which impact on the findings of the study.
Write as clearly as possible. Think about your target audience, many of whom will not be Health Economists, and therefore may not understand economic terminology.
Avoid copying verbatim; firstly, it is plagiarism and, secondly, it is very likely that the relevant information will not be presented in a reader-friendly way.
Where there are lots of study results, these should be summarised. It is not the purpose of the abstract to repeat the paper.
In the data extraction body of the abstract, ALWAYS state what the authors think they did, e.g. the authors used a societal perspective or the results of an RCT were evaluated using an intention-to-treat method. BUT if you think that the use of terminology was inappropriate or the authors’ statement of the methods was simply wrong then you may indicate that there is an alternative interpretation. The exception to this is in defining the economic evaluations as either, a costeffectiveness analysis, a cost-utility analysis or a cost-benefit analysis; in this case, simply write your interpretation.
An abstract should reflect the content of the economic evaluation paper. As such, reference papers should not be ordered to obtain additional information. Reference can be made to reference papers where appropriate in the abstract to indicate that additional information may be found there.
Always use full sentences except for the ‘Type of Treatment’, ‘Economic Study Type’ and ‘Currency’ fields.
Lists, flow-charts, etc. that provide guidance in classifying or describing methods may not be exhaustive or sufficiently detailed. So if you think the methods used in the paper you are abstracting are not adequately described, either make a statement to indicate that this was the case or use your own words to convey the methods used.
When reporting confidence intervals use: (95% CI: -2 to 3)
When reporting ranges use: (range: 6 to 59)
The pen symbol indicates that you need to report the requested information. You should also state when the requested information is missing or not explicitly stated in the paper.
All terms marked with an asterisk (*) are briefly explained in the explanatory notes.
27
28
1. Subject of study 1.1. Health technology*
Describe the intervention(s) and comparator(s) studied by providing relevant details. E.g. dosage, frequency, health care provider.
Example: The health technology examined was mirtazapine, a noradrenergic and specific serotonergic antidepressant, at doses of 30 to 45 mg/day. Example: The health technology assessed was screening for hepatocellular carcinoma (HCC) using strategies that combined abdominal ultrasonography (US) or computerised tomography (CT) with alpha-fetoprotein (AFP) levels at 6-12 month intervals. 1.2. Disease This is a broad disease category derived from National Library of Medicine Subject Headings and it is assigned by the NHS EED team. For more details see Appendix 2.
1.3. Type of intervention
Report the intervention type in broad terms, using one or more of the following phrases:
Primary prevention*
Secondary prevention*
Screening*
Diagnosis*
Treatment
Rehabilitation*
Palliative care*
Health professional training
Other: ‘specify’ (i.e. public health policies, information and education in health*, integrated care*)
29
1.4. Hypothesis/study question
Summarise the general objective of the study, the hypothesis/question(s) posed and the rationale for the methods chosen.
Report the justification given for the choice of health technology.
Report the perspective adopted for the economic analysis as stated by the authors. If the actual perspective is different to that stated by the authors, report this and explain why (if possible).
Note: The perspective can be ‘societal’ (in which case both direct costs and indirect costs (productivity losses) should be reported in the direct costs and productivity costs fields) ‘patient’ (when the patient makes payment for which no reimbursement or free cover is available) ‘single provider’ (includes health service, hospital, clinic, etc.) 'insurer' (includes third-party payer) 'healthcare system’
Example: The objective of the study was to determine the cost-effectiveness of the newly developed treatment x compared with the current treatment y. As the effectiveness of treatment x had not widely been evaluated in a hospital setting, a clinical trial with an economic evaluation alongside was conducted. The perspective of the study was societal.
Example: The objective of the study was to determine the cost-utility of the newly developed treatment x compared with the current treatment y over the lifetime of the patient. Since no randomised trials exist with adequate duration of follow-up, a model was used to incorporate the progression of the disease beyond the end of any trial period. The perspective of the study was societal.
30
2. Key elements of study 2.1. Economic study type
To define the study type, enter one or more of the following: Cost-effectiveness analysis* (this includes any economic analysis with a clinical effectiveness outcome)
Cost-utility analysis* Cost-benefit analysis*
Note: If the authors have defined the study type incorrectly, the correct type of analysis should be written here and the correction should be justified in the commentary field, Other Issues.
2.2 Study population
Outline the main characteristics of the patient population from which the study sample was drawn; e.g. age, sex, health status (e.g. duration and severity of disease or comorbidity), socio-economic status, etc. Briefly summarise any inclusion/exclusion criteria reported by the authors.
Where studies include economic models, the study population may be hypothetical but defined by the authors. This should be reported. Example: A hypothetical cohort of 50-year old males with single vessel coronary artery disease, and without any other significant comorbidities.
If there is a target population that is broader than the study population for which the authors want to generalise the results, report the target population as well as the study population. However, this is rarely, if ever, reported in economic evaluations. For further explanation see explanatory notes page 67.
2.3. Modelling and statistical extrapolation* This field should just include technical details of any models used. The rationale of the model should have already been stated in the hypothesis/study question field as it is inherently related to the objectives of the study.
Report the type of model (including statistical extrapolation, survival analysis and epidemiological studies) and the time horizon considered.
Make a statement about the level of reporting surrounding the key facts, details and assumptions of the model. Example: A Markov model with a lifetime horizon was developed. The health state, cycle length and time-dependent transition probabilities were presented in full in the paper, along with a number of modelling assumptions which were fully justified.
31
Example: Owing to the limited time horizon of the trial, data were extrapolated using the declining exponential approximation of life expectancy model (DEALE). Example: An epidemiological model was used to combine the available clinical data with the population’s risk factors in order to estimate clinical outcomes for the study population. Details of the clinical data and risk factors were presented in full in the paper.
2.4. Setting
Specify the practice setting using one or more of the following terms: Outpatient care* Inpatient care* Home care*
Community care* Institutional care* Other; ‘specify’
If you find it is relevant and appropriate, specify the level of health care using one of the following terms: Primary care* Secondary care* Tertiary care*
State the country or specific place which is the setting for the economic study (e.g. Sheffield, UK).
2.5. Dates to which data relate Report the year(s) during which the data were collected, for:
The effectiveness analysis.
Note: If more than one source has been used to derive the effectiveness evidence, provide the range of publication years of the studies included in the review, eg 1999 to 2003.
The resources used, such as equipment, manpower, medicines, etc. If different dates are given for different components or interventions, report them separately.
The prices used.
2.6. Link between effectiveness and cost data (This should be completed only if a single study has been used to derive effectiveness).
State whether the costing was undertaken on the same patient sample as that used in the effectiveness study.
State whether the costing (i.e. the collection of resource data) was undertaken prospectively (alongside the effectiveness study) or retrospectively (after the effectiveness results were known).
32
3. Details about clinical evidence Complete sections A or B according to the classification that has been allocated to the paper by the NHS EED team: F(A), F(B). For example, if the study has been classified as F(B) complete section B only. Ensure that you have read the paper and confirmed the classification prior to abstracting. If you are unhappy or unsure about the classification please contact an NHS EED team member for clarification. In this section only the clinical evidence should be reported. Any information about the implications for the use of resources should be reported in fields “4.2 Direct costs”, “4.3 Productivity costs”, “5.2 Cost results”, and “5.3 Synthesis of costs and benefits”, as appropriate. N.B. Reference papers should not be ordered to obtain additional information. Reference can be made to the additional paper, where appropriate, in this section to indicate that additional information may be found there or to explain why the authors did not report many methodological details.
Section A 3A. Single study For this subsection, first identify the study design type using the study design scheme provided on page 76. Guidance to reporting common study designs follows in this section. Abstracts should report the study sample, study design, analysis of effectiveness and summarise the authors’ clinical conclusions for all F(A) papers. Guidance for rarer study designs is provided on pages 77-79 of the explanatory notes. It is not necessary to have a different statement for each issue and you may find reporting easier if you combine some issues in one sentence.
33
3A.1. Study sample
Experimental studies: RCT, non-RCT, cross-over RCT, etc.
Cohort study
Retrospective cohort study
Comparative study Within-group with historical control comparison
Diagnostic accuracy study*
State whether the sample size* was determined in the planning phase of the study to assure a certain power*.
Summarise the sample selection method* (e.g. convenience sample vs random selection).
Report if, and how, the authors justified the choice of the patient sample with respect to the characteristics of the disease and/or treatment under investigation and with respect to the generalisability of the findings.
State if any people refused to participate and what percentage they represent of the original sample selection.
State if any people refused to participate and what percentage they represent of the original sample selection.
State if any people refused access to their data.
For the current group, state if any people refused to participate. For the retrospective group, state if any people refused access to their data.
State if any people refused to participate for the current group and what percentage they represent of the original sample selection.
State if any people refused to participate and what percentage they represent of the original sample selection.
Report the number of patients that started the study.
Report the number of patients that started the study.
Report the percentage of patients excluded from the initial sample and the reasons for the exclusions.
Report the number of patients that started the trial and the number of patients in each group.
34
Report the number of patients that started the study and the number of patients in each study group.
Report the number of patients included in the analysis and the number of patients in each study group.
Report the number of patients in both the current and historical groups that started the study.
3A.2. Study design Experimental studies: RCT, Cohort study non-RCT, cross-over RCT, etc.
Retrospective cohort study
Comparative study Within-group with historical control comparison
Diagnostic accuracy study
Specify the study type: refer to the study design flow chart, p76 of explanatory notes.
State whether the study was single or multi centred. Give details (e.g. number of centres or number of sites). For randomised allocations, report how the random sequence was generated and whether and how the randomised allocation was concealed*.
Not relevant.
Not relevant.
Not relevant.
Not relevant.
See below.
Was blinding plausible/relevant and, if so, specify the blinding method*.
If blinding was plausible/ relevant, specify who was blinded.
Were the data analysers blinded to the groups?
Was blinding plausible/relevant and, if so, specify the blinding method*.
Was blinding plausible/relevant and, if so, specify the blinding method*.
Were the test result evaluators ignorant of the results from the other test?* State whether every patient who was entered into the study received both tests or did those receiving the second test depend on the results of the first?*
Not relevant.
Not relevant.
Not relevant.
For pharmaceutical studies, was there a wash-out period and how long was it?
Report the duration of follow-up.
Report the duration of follow-up.
Report the time-frame for which data were collected.
Report the time-frames for which data were collected for the current and historical study groups.
Report the timeframes for which data were collected for the 2 interventions.
State whether the tests were done concurrently. If not, report the time gap between them.* This includes the reference test follow-up*.
Quote the loss to follow-up* (report the reasons for withdrawals if provided), the % of the overall sample and of each group.
Quote the loss to followup* (report the reasons for withdrawals if provided), the % of the overall sample and of each group.
Not relevant.
Quote the loss to followup* (report the reasons for withdrawals if provided), the % of the overall sample and of each group.
Quote the loss to follow-up* (report the reasons for withdrawals if provided), the % of the overall sample and of each group.
Not relevant.
For a pharmaceutical, cross-over RCT, was there a wash-out period and how long was it?
35
3A.3. Analysis of effectiveness Experimental studies: RCT, non-RCT, cross-over RCT, etc.
Cohort study
Retrospective cohort study
Comparative study Within-group with historical control comparison
Diagnostic accuracy study
Report the primary health/diagnostic outcomes used in the analysis (e.g. CHD deaths, quality of life scores, side effects1, test sensitivities). A definition of the measure is sometimes helpful, e.g. disease progression is defined at a certain clinical measure. Utilities should not be reported here. The valuation of utilities should be dealt with in field 4.1 and QALY results (but not utilities) may be reported in field 5.1. Specify any particular instruments used to evaluate these data (e.g. quality of life questionnaire). N.B. this is rare for a diagnostic cohort study. State whether the analysis was based on intention-to-treat or treatment completers only/per protocol.
State whether all the patients included in the study were accounted for in the analysis.
At analysis, state if the groups At analysis, state if there were comparable in terms of demographics and prognostic was adjustment for confounding factors. features or, if not, if there was adjustment for confounding factors.
State whether patients were excluded for incomplete data.
State whether all the patients included in the study were accounted for in the analysis.
At analysis, state if there was adjustment for confounding factors.
At analysis, state if there was adjustment for confounding factors.
State whether all the patients included in the study were accounted for in the analysis.
Not relevant.
Report if there were any indeterminate test results and how they were handled*.
Not relevant.
1 In some studies, the principal difference between health technologies is in terms of the incidence and severity of side-effects. Side-effects are negative clinical benefits and should be reported in this field. The impact of side-effects on the use of resources should be reported in field ‘4.2 Direct costs’ (or eventually in field ‘4.3 Indirect costs’) and in fields ‘5.2 Cost results’ and ‘5.3 Synthesis of costs and benefits’, as appropriate.
36
3A.4. Effectiveness results
Summarise the results of the trial or study, including the side-effects or adverse effects (give quantitative results when reported).
Tip: If you consider there are too many outcomes to report, prioritise the primary endpoints of the clinical study.
Quote the 95% confidence interval* and p-values* of primary outcomes, using the following formats: (95% CI: 1.23 to 4.56; p=0.001), or (p
