ENDORSED BY

Knowledge management in medicine – are comprehensive electronic clinical libraries the answer ?

  Prof. Jeremy Wyatt

  Medical Informatics Department, Academic Medical Centre, Univ. of Amsterdam

UCL Knowledge Management Centre, London www.ucl.ac.uk/kmc

   
The need to provide clinicians with up-to-date evidence-based point of care knowledge resources is widely accepted [1-3], and a number of “comprehensive electronic clinical library” projects costing millions of euros already do this (eg. the National electronic Library for Health, NeLH and Knowledge Access 24 in the UK, the Clinical information Access Project in Australia).
 
However, we also know that doctors rarely use electronic library services [4] and that there are many barriers to knowledge use by clinicians [5]. These barriers include the time taken by clinicians to find answers to their questions and low success rates [6]. For example, of 1200 questions volunteered by US family physicians in Ely’s study, the doctors searched for answers to only 444. However, they used electronic resources to answer only 10 (2%) of these 444 questions, and took a median of 3 minutes to search such resources, finding an answer only twice [7]. This suggests that the electronic resources themselves might be a significant barrier to knowledge access.
 
Clearly no individual resource can answer all clinical questions, so the UK NHS has now recommended 12 specific resources as being suitable to answer clinical questions [8]. All of these are now provided either free on the web or through contracts placed by the NeLH.
 
However, we suspected that even this large number of resources might not answer many clinical questions and were unsure how long it would take to search them. We therefore studied the ability of expert librarians to search for answers to 96 well formulated clinical questions, half from primary and half from secondary care. We deliberately biased our study in favour of the electronic clinical library model, in the following ways:

•  The searchers were volunteer medical librarians familiar with the NHS-commended resources (ie. faster, more expert searchers than doctors)

•  We only selected well formed clinical questions (ie. easier to search for answers)

• All questions were volunteered in the last 2 years by clinicians working in UK general practice or hospitals (ie. the questions were thought to be answerable)

Our results were disappointing, with a 60% failure rate and only 16% of questions well answered. This was despite expert librarians spending a median of 54 minutes searching a minimum of 5 comprehensive evidence based resources.
 
Our results demonstrate that the currently favoured model of a comprehensive electronic clinical library is unlikely to answer many clinical questions at the point of care - even if the doctor uses it once her patients have gone home. Worse, providing clinicians with access to such knowledge resources may introduce searching delays, and could even lead to inappropriate tests or treatment for patients.
 
Improvements to the electronic library model might include adding ever more comprehensive resources, or providing a better search function to enable doctors and others to find what they need fast, without wasting time on unproductive searches. However, we believe that a broader knowledge management strategy is needed which explore alternatives to the comprehensive electronic clinical library, such as:
   

•  A clinical question help line, analogous to NHSDirect, with 24 X 7 access to librarians searching on demand and a regularly updated store of FAQs available on a web site. This could be a development of, and possibly based on, current informal question answering services in various parts of the UK such as ATTRACT [9]

• Intelligent personal search agents to which a clinician could email a question and which would return seconds or minutes later with a short answer and web links to further detail. Such information agents are the subject of much current information science research [eg. 10], but do not yet appear ready for clinical field testing.

In conclusion, we believe that our study casts serious doubt on the extent to which comprehensive electronic clinical libraries can answer a sufficient proportion of clinical questions in time to influence clinical decisions and, most importantly, actions. We therefore suggest that new thinking is needed, such as a 24 X 7 librarian-mediated search and FAQ facility to complement direct user searching.
 
 
Acknowledgements
NHS R&D programme, the BUPA Foundation, Claire O’Brien.
 
 
References

1. Smith R. What clinical information do doctors need ? BMJ 1996; 313: 1062-8  

2. Wyatt JC. Clinical knowledge and practice in the information age: a handbook for health professionals. RSM Press. ISBN 1-85315-483-0, 2001.

3. J A Muir Gray and Simon de Lusignan. National electronic Library for Health (NeLH). BMJ 1999; 319: 1476-9

4. William R. Hersh; David H. Hickam. How Well Do Physicians Use Electronic Information Retrieval Systems? A Framework for Investigation and Systematic Review. JAMA. 1998;280:1347-1352

5. John W Ely, Jerome A Osheroff, Mark H Ebell, M Lee Chambliss, Daniel C Vinson, James J Stevermer, and Eric A Pifer. Obstacles to answering doctors' questions about patient care with evidence: qualitative study. BMJ 2002; 324: 710

6. McKibbon KA, Haynes RB, Walker Dilks CJ, Ramsden MF, Ryan NC, Baker L, Flemming T, Fitzgerald D. How good are clinical Medline searches ? A comparative study of end-user and librarian searches. Comp Biomed Res 1990; 23: 583-93

7. John W Ely, Jerome A Osheroff, Mark H Ebell, George R Bergus, Barcey T Levy, M Lee Chambliss, Eric R Evans. Analysis of questions asked by family doctors regarding patient care. BMJ 1999; 319: 358-361

8. NHS Centre for Reviews and Dissemination, The University of York. Accessing the Evidence on Clinical Effectiveness. Effectiveness Matters February 2001; 5 (1)

10. Dertouzos ML. The future of computing. Sci Am Aug ’99: 36-39