Quality in healthcare – Part 2 Audit

1 Consultant Obstetrician and Gynaecologist. Correspondence: Asoka Weerakkody E-mail: asokanimal@gmail.com All trainees now are quite conversant with the theory of audit process. Unfortunately this knowledge does not often translate to practice and to performance in vivas; even less so to presentations. This article intends to focus on the practical side of audit and hopes to help the trainees to improve the quality of audits done, presented and/or published.

The first 'audit' (although it was not called such) was done by Florence Nightingale in the 1850s. She was able to produce dramatic improvement in the mortality figures amongst the British troops in the battle front (see appendix 1).

Lessons from Crimean experience
Florence Nightingale had a clear idea of her mandate. She quantified the problem, identified the causes and found solutions. Most importantly, she had the clout to get the resources she needed in order to implement the changes (very often nowadays, many audits fail at this point). Her re-audit proved the success of her methods. Rather than fight or argue with her critics i.e. doctors, she won them over by letting the results speak for themselves. She used this base in order to bring about widespread change in practice.

How many audits done since could boast of the similar achievements?
Common mistakes in doing audits 1. Not having clear aims or objectives, leading to vague or non-existent standards.

CONTINUOUS PROFESSIONAL DEVELOPMENT
2. Having too many criteria.
One gets data overload. The central message tends to get diluted.
3. Not realising potentially serious problems in getting access to clinical notes and obtaining accurate data there from (in retrospective audits).
4. Miscalculating the time required to access sufficient numbers (in prospective audits); resulting in partly or hastily finished audits.
5. Conclusions and recommendations not being based on your own results. Forgetting the original question at the end.
6. Having no idea how to implement the changes you propose.
7. Not having clear plans to re-audit, or re-auditing for the sake of it, without implementing recommendations (and getting the same bad results!)

Before you do an audit
The above mistakes could be avoided by asking the following questions, before you start:

Why am I doing it?
The reasons which generate the need for audits are called the 'drivers'. They induce people to do audits.
You must clearly identify the driver for your particular audit. ("My boss asked me to do it" or "I need one for my book", are not valid reasons.) There are several drivers: a. There is an obvious problem: This is clearly what set Nightingale on her mission. In your setting, it could be a large number of burst abdomens or infected wounds in a short space of time. In these situations, the audit should be much focussed and concentrated Asoka Weerakkody on quantifying the problem, finding reasons and (feasible) solutions.

b. To highlight a problem:
Either the problem is hidden and most people have not noticed it or most people know there is a problem, but nobody wants to talk about it (for whatever reason). A very good example is the Bristol heart scandal in 2000.
In these situations it is very important to be very objective and forget personalities. In fairness, the people involved must be pre-informed (if possible) and the final results shared with them before 'going public'.

c. To highlight a need
This is much more common. You might need some new equipment to improve the service, for e.g. fetal blood sampler for the labour ward. If you support your claim by the results of an audit, you are more likely to be taken seriously.

d. To assess a new service
Any new service, for e.g. an early pregnancy assessment unit, must be audited to test whether it delivers in practice what it promised.

e. Organisational needs
These are usually mandatory, for e.g. maternal and perinatal mortality, as they are vital for assessment of the quality of healthcare.

f. Educational needs
For the trainees, there is nothing like 'doing one to know one'! In the examinations, it is very easy to distinguish those who have actually done an audit, from those who are talking theory.

What am I trying to find out?
In particular, what questions am I trying to answer?
Obviously this depends on why. There are different aspects you could look at: a. Structure: this refers to the facilities, buildings, equipment and personnel. These are the responsibilities of the organisation, which are not easily moved. The best way to do so may be by doing a good audit. In the UK, these are considered very important (though not yet in SL).
Frequently it is a combination of above.
It is sensible to focus the questions to a few important ones.

How?
Prospective or retrospective: each has its own advantages and disadvantages.

Notes-based or questionnaire/interview
(Exercise: List the advantages and disadvantages of the above methods.)

What information do I need to answer the questions in 2 above?
Prepare a well thought out proforma, listing in rational order information you require. You must have a clear idea why you need a certain bit of information. Avoid 'information overload'.
Run a pilot of a few cases, before you finalise the audit proforma. Omissions discovered late, can be expensive in terms of time and effort.

How many cases do I need?
In theory, numbers required could be calculated using the power calculations used for research studies. In practice and by convention, one settles for a reasonable number between 30 and 100. Obviously in outcome audits, it would depend on the frequency of the outcome.
Practical point: ask for 10% more cases than the number you require.

Feasibility?
Could it be done within a reasonable time using available resources? Is it worth the effort? Would the system allow me to make the recommended changes? Audit C P D

Standards?
Remember -no standards, no audit! Your standards must be authoritative, and yet practical and not Utopian. National standards may well have to be modified according to local needs and resources. If none exists, you should make your own (ideally by consensus).

Appendix 1 Florence Nightingale and the Crimean War
The British were fighting along with the French, against Russians in the Crimean war. The mortality rate amongst the injured British soldiers was deemed very high. Questions were raised in the Parliament, ultimately leading to the fall of the government. The new Secretary at War was a personal friend of Florence Nightingale who was a highly regarded society lady, reputed to have an interest in nursing. He sent her to the front to investigate.
The conditions she found were wretched. Basic supplies and personnel were lacking, as was any regard to basic principles of hygiene and anti-sepsis.
She collected figures carefully which showed the mortality amongst the British soldiers was over 40%, whereas that amongst the French was only 5% (quantifying the problem). Then she set about enumerating problems and dictated solutions (recommendations).
Fortunately, she 'had the ear' of the Secretary at War; hence whatever she asked was provided. She then set about implementing changes incorporating basic standards of hygiene (implementing changes). Initially, the doctors on site considered her an imposter and a nuisance, and were very sceptical about her methods. She did not fight with them, but simply asked for her own patient load, which the doctors happily divested, as they had enough on their plate.
After a few months she checked the mortality rate again (re-audit).
In her patients, from over 40% it had fallen to 2%.
Given this dramatic improvement, the doctors could not, but take notice. They too changed their practice and all the patients benefitted (achieving a change in practice).
She did not stop there. Upon returning to England, she used this experience to establish proper standards of nursing on a wider scale.
The rest is history.
The emphasis therefore, must be on spending enough time planning, before you start. Most of the poor audits are a result of people rushing to do it, without thinking it through.
During analysis, do not forget the original questions of why and what. Your recommendations must be based on your own results.
Good luck with your next audit!