Appendix 2: Handbook development

Please note: due to continuous updates being made to the Immunisation Handbook, the page number on the electronic version will not always match the hard copy version.


PDF printable version of Appendix 2: Handbook Development - Updated in April 2009 (PDF 56 KB)

Introduction

The Handbook has been developed by the Australian Technical Advisory Group on Immunisation (ATAGI), which provides advice to the Federal Minister for Health and Ageing on the Immunise Australia Program and other related issues. In addition to technical experts, ATAGI’s membership includes a consumer representative and general practitioners. ATAGI consulted with other expert bodies, and with the National Health and Medical Research Council, throughout the development of the 9^th edition of the Handbook. Staff of the National Centre for Immunisation Research and Surveillance (NCIRS) provided technical assistance to ATAGI in the development of the Handbook.

The Handbook does not address the cost-effectiveness of different vaccines or different regimens; since January 2006, the cost-effectiveness of vaccines has been assessed by the Pharmaceutical Benefits Advisory Committee (PBAC), which advises government on the funding of vaccines.¹

The Handbook is designed as a general guide to inform clinicians on the safest and most effective vaccination strategies, using the highest quality evidence available in peer-reviewed literature, according to NHMRC guidelines.^2-5In the absence of evidence at the highest level (well conducted randomised trials and meta-analyses), recommendations were based on lower levels of evidence such as case series. Where clinical guidelines were available on specific topics, these were used to frame recommendations, if relevant, in the Australian setting. Immunisation handbooks produced by comparable countries were also consulted. If published sources were inadequate, recommendations were based on expert opinion.

The draft of the 9^th edition of the Handbook was available for public consultation over a 6-week period from February to April 2007. The public comments received were reviewed during an extraordinary meeting of ATAGI and, where necessary, changes to the Handbook were incorporated. Subsequent public consultation periods were provided when amendments to the electronic version of the published 9^th edition of the Handbook were made (see ‘Handbook amendments’ below).

The 9^th edition of the Handbook is disseminated directly to all registered medical practitioners. Additional hard copies are distributed to other immunisation service providers via their State or Territory health authority. An electronic version of the Handbook is accessible on the Immunise Australia Program website http://immunise.health.gov.au/. Implementation of the recommendations as stated in the Handbook is undertaken by immunisation providers in conjunction with their State or Territory health authority, and the Immunise Australia Program of the Australian Government Department of Health and Ageing.

Handbook literature search methodology

For each chapter, broad literature searches were conducted for the years since the last Handbook searches were performed using up to 18 databases, listed in Table 1. The purpose of these searches was to ensure that technical writers and chapter sponsors had access to all relevant information from the latest medical literature in identifying important issues relating to the updating of all Handbook chapters.

Table 1: Electronic databases searched
Electronic database	Time period
MEDLINE	2002–2006
PREMEDLINE (when required)	2002–2006
Cochrane Library—including Cochrane Database of Systematic Reviews, Database of Abstracts of Reviews of Effects, the Cochrane Central Register of Controlled Trials (CENTRAL), the Health Technology Assessment Database, the NHS Economic Evaluation Database	2002–2006
Cumulated Index Nursing & Allied Health Literature (CINAHL) (when required)	2002–2006
EMBASE	2002–2006
Australian focused Informit databases (AMI, APA-FT, APAIS, APAIS – Health, ATSIhealth, Ausport Med, CINCH – Health, DRUG, Informit e-library, Health and Society, HIV A, Meditext, RURAL).	2002–2006

For specific questions arising in individual chapters, other databases and additional resources such as Clinical Evidence were searched as necessary. The scope and nature of the review differed for updating of existing chapters in the Handbook and new chapters for the 9^th edition.

Top of page

• Existing chapters in the Handbook

The medical and health literature was searched to identify relevant studies and reviews regarding individual diseases and the vaccines available using the electronic databases in Table 1. The search period was from 2002 to September 2006. The scope of the searches was broad, to ensure maximum retrieval and minimise the exclusion of items of interest. Previous Handbook searches were examined to determine the scope required for the new searches, and similar search strategies were employed to ensure consistency of information retrieval.

Various search methods were tested, including ‘explode’ and ‘focus’ options. ‘Exploded’ terms retrieve citations containing the term being searched and all the narrower related terms in the database. ‘Focus’ searches retrieve citations that have the search term as the major focus of the item. In the trial searches, some items of interest were missed using the ‘focus’ method, thus ‘exploded’ searches were utilised. All subheadings assigned to the subject headings were generally included.

In general, the search strategy consisted of the disease topic combined with the terms immunisation/preventive health. Boolean operators AND, OR, and NOT were used.

To ensure relevant and accurate retrieval, thesaurus terms (the controlled vocabulary terms used in the database) were used whenever possible. Keyword searching was used only in the absence of an appropriate thesaurus term or if the database did not have thesaurus terms. To facilitate relevant retrieval and to limit what, in some instances, are very large search result sets, the following limits were applied to the disease topic searches:

• Publication year – searches were generally limited to items published from 2002–2006, in order to retrieve items published since the searches completed for the 8^th edition of theHandbook.
• Language – searches were limited to items in English.
• Human – items discussing only animals were removed.
• In vitro – items discussing only in vitro studies were removed.
• Abstracts – search results restricted to items containing abstracts.

The search limits were slightly modified for some of the other searches. For example, the Australian-specific searches did not have search results limited to abstract only, to ensure that all Australian items were retrieved, including items such as letters to the editor and editorials. In addition, the search result files were edited to remove irrelevant items. Files were edited, where possible, to remove duplication across the databases. There was often substantial duplication across databases, as databases index some of the same journals.

More specific searches were undertaken when requested by chapter sponsors or technical writers to provide additional information on a particular aspect of a topic.

• New vaccine chapters

For the new vaccine chapters, Chapter 3.7, Human papillomavirus and Chapter 3.18, Rotavirus, search strategies employed for the existing chapters (as above) were used. In addition, recommendations were formulated using a structured clinical question using the PICO (Patient/population, Intervention, Comparison and Outcome) format as per the NHMRC guidelines.³ Where possible, these structured clinical questions informed both the searching and subsequent data extraction processes. Search limits used for the new vaccine chapters differed from those used for the existing chapters. To maximise retrieval and minimise publication or other potential biases, no limitation by date, language or abstract was applied. All papers identified as relevant from the PICO literature review were critiqued using a standard proforma, according to whether they were classified as studies of an intervention, diagnostic tests, prognosis, aetiology or related to screening. Completion of a separate detailed data extraction form was undertaken for each included study considered in the body of evidence for each recommendation. These processes were completed in consultation with the NHMRC-appointed Guideline Assessment Register consultant.

Complete details of the systematic literature review for the new vaccine chapters in the Handbook may be found on the Immunise Australia website www.immunise.health.gov.au

Top of page

The evidence presented in the included studies was assessed and classified as described by the NHMRC.³ The consideration of important aspects of the evidence supporting the intervention or recommendations included 3 main domains:

• strength of the evidence (level, quality and statistical precision),
• size of the effect (including clinical importance), and
• relevance of the evidence.

Grades of evidence were assigned to the recommendations in the new vaccine chapters using the NHMRC considered judgment approach for assessing a body of evidence (see Table 2). Grade A and B recommendations are generally based on a body of evidence which can be trusted to guide clinical practice, whereas Grade C and D recommendations must be applied carefully to individual clinical and organisational circumstances and should be followed with care.

Table 2: Grades of recommendations4
Grade of recommendation	Description
A	Body of evidence can be trusted to guide practice
B	Body of evidence can be trusted to guide practice in most situations
C	Body of evidence provides some support for recommendation(s) but care should be taken in its application
D	Body of evidence is weak and recommendation must be applied with caution

The level of evidence for the included studies for each recommendation is also given, according to the NHMRC additional levels of evidence. Table 3 shows the levels that apply to intervention studies (for other study types, such as aetiology, prognosis or diagnosis, see http://www.nhmrc.gov.au/consult/index.htm). The level of evidence indicates the study design used by the investigators to assess the effectiveness of an intervention. The level assigned to a study reflects the degree to which bias has been eliminated by the study design.³

Table 3: NHMRC Levels of evidence for intervention studies4
Intervention	Level of evidence
A systematic review of level II studies	I
A randomised controlled trial (RCT)	II
A pseudo-randomised controlled trial (eg. alternate allocation or some other method)	III-1
A comparative study with concurrent controls: Non-randomised, experimental trial Cohort study Case-control study Interrupted time series with a control group	III-2
A comparative study without concurrent controls: Historical control study Two or more single arm study Interrupted time series without a parallel control group	III-3
Case series with either post-test or pre-test/post-test outcomes	IV

Top of page

Further information of how these grades were applied is given in the detailed description of the systematic literature reviews for the new vaccine chapters, which are available on the Immunise Australia website, www.immunise.health.gov.au.

Establishing Selected Dissemination Information (SDI) searches

Selected Dissemination Information (SDI) searches were established to enable the ongoing collection of new relevant items on the search topics. This process used the same search strategies as the previous searches. Search results are automatically generated each time the databases (MEDLINE, EMBASE) are updated, and a report is automatically sent to a nominated e-mail address. This allowed inclusion of the most recent literature to be evaluated and referenced throughout the revision and updating of the Handbook, where applicable.

Handbook amendments

The Australian Technical Advisory Group on Immunisation has established processes for reviewing and updating sections of the 9^th edition of the Handbook when important new information for immunisation providers is required prior to publication of the next edition. The chapter, 3.26 Zoster (Herpes zoster), was a new chapter in the 9^th edition of the Handbook and contained background information on herpes zoster and vaccine development, but did not contain recommendations for use of the zoster vaccine in Australia because a vaccine was not available at the time of publication.

A zoster vaccine became available in Australia in 2008. During 2007–2008 a full version of Chapter 3.26, Zoster (Herpes zoster) and an accompanying “Systematic review of the safety, immunogenicity and efficacy of herpes zoster vaccines” were developed by ATAGI using the processes for the development of new vaccine chapters, described in detail above and in the methods section of the systematic review.

The full version of Chapter 3.26, Zoster (Herpes zoster) and the accompanying systematic review were available for public consultation over a 4-week period from April to May 2008. The public comments (5 comments from 2 submissions) were received and reviewed and, where necessary, changes to the chapter were incorporated. The chapter and systematic review were approved by NHMRC in March 2009, and published as an amendment to the 9^th edition Handbook electronically via the Immunise Australia website www.immunise.health.gov.au. Minor amendments to other sections of the electronic version of the Handbook, such as Part 1, Part 2, and the appendices and various tables, were made where indicated to reflect the availability of zoster vaccine for the prevention of herpes zoster in Australia.

References

1. Toms L. New immunisation advisory structures. Poster presented at the 10th National Immunisation/2nd PHAA Asia Pacific Vaccine Preventable Diseases Conference - Successes in Immunisation. 2006. Available at: http://immunise.health.gov.au/internet/immunise/publishing.nsf/Content/advisory-bodies (accessed May 2007).
2. National Health and Medical Research Council (NHMRC). How to review the evidence: systematic identification and review of the scientific literature. Canberra: NHMRC, 1999.
3. National Health and Medical Research Council (NHMRC). How to use the evidence: assessment and application of scientific evidence. Canberra: NHMRC, 2000.
4. National Health and Medical Research Council (NHMRC). NHMRC additional levels of evidence and grades for recommendations for developers of guidelines. Pilot program 2005-2007. 2005. Available at: http://www.nhmrc.gov.au/consult/index.htm (accessed Feb 2007).
5. National Health and Medical Research Council (NHMRC). Using socioeconomic evidence in clinical practice guidelines. Canberra: NHMRC, 2003

Top of page