How to really do quality, part 2: run it every week

← Back to Blog

Most organisations collect numbers. Few control their processes. The difference is simple. Collecting is looking back. Control is knowing, every week, whether your promises are being kept, and reacting when they are not.

A. Write a control plan for each critical step

In manufacturing, a control plan lists every check that keeps a product right. Your version is one card for each critical step you chose in part 1.

Look at two things.

First, the last column. A check without a reaction is only watching. Every row says what happens, the same day, when the check fails.

Second, "how many". When harm is high and numbers are small, check every case. That is why every "not eligible" decision is checked. When numbers are large and harm is lower, check a sample. That is why only five "accepted" decisions a month are checked.

B. Use three kinds of measures

Quality improvement teams use a simple family of three. You need at least one of each.

• Process measure. Did we do the step right? Example: real answers within 3 days, 176 of 220.
• Outcome measure. Did things get better for the person? Example: goals met at the three-month check.
• Balancing measure. Did fixing one thing break another? Example: answers got faster, but did more people have to call twice?

The balancing measure is the one most often missing. Without it, you can hit every target and make the service worse.

Write the counting rule

For every measure, answer five questions in one line each. The statistician W. Edwards Deming called this an operational definition. It means everyone counts the same way.

1. What counts as X? A real answer. An automatic "we got your message" does not count.
2. What counts as Y? Everyone due an answer in the period, including people still waiting.
3. When does the clock start and stop? When the request arrives. When the real answer is sent.
4. Who is counted? Each person once, even if they called five times.
5. Which decision will this number help us make? If none, drop it.

C. Count honestly

Count what was due, not what was done

Say 220 people were due an answer this quarter. 176 got one on time. 28 got one late. 16 are still waiting.

• Honest: 176 of 220. That is 80%.
• Flattering: 176 of the 204 who got an answer. That is 86%. And the 16 people still waiting have vanished.

The people still waiting are exactly the people you need to see. So Y is always everyone who was due.

Show what you don't know

Never count unknown as success. Never count it as failure. Put it beside the number: "52 of 80 arrivals confirmed. 10 unknown."

Time: three numbers, not one

• Typical wait. Put all waits in order and take the middle one. That is the median.
• Slowest tenth. How long the slowest 10% waited.
• Oldest open case. How long the person still waiting has waited so far.

A typical wait of 2 days can hide one person who has waited 18. The other two numbers bring that person back.

Check the data before you trust it

Before a number goes anywhere, check three things in its records:

• Complete. Does every record have the dates and fields you need?
• On time. Was it entered within a week, or reconstructed later?
• Accurate. Take five records and compare them with the paper file or the email. Do they match?

Report data problems openly: "8 of 60 refusal records had no reason written." That is a finding in itself.

D. Read the line, not the dot

Numbers move a little every month even when nothing has changed. Reacting to every small move makes things worse, because you keep changing a process that was fine.

So plot each measure as a line, one point per month. Draw a flat line through the middle value. Quality managers call this a run chart. Then use three simple rules:

1. Six points in a row above, or below, the middle line: something has really changed.
2. Five points in a row all going up, or all going down: a real trend.
3. One point far away from all the others: look at what happened that month.

Anything else is normal noise. Leave the process alone and keep watching. And remember: two quarters are only two dots.

E. Audit real cases

Numbers tell you how often. Cases tell you why. An internal audit means taking real cases and checking, with evidence, whether every promise was kept.

Plan it

Write a simple audit plan for the year. Every critical step is audited at least once a year. Steps with problems are audited more often.

Choose the cases

1. Ten at random. Number the people from the period and pick by chance, for example every twentieth. These show what is normal.
2. Five on purpose. People whose output was "not eligible", and people who left early. These show where the risk is.
3. Keep the two groups apart. Never mix them into one percentage.

Who audits

Never the person who did the work. Not because they are dishonest. Because nobody sees their own blind spots. In a small organisation, swap with a partner organisation. You audit theirs, they audit yours.

Trace each case

Follow the case through every step, from the first contact to today. At every step ask three questions. Was the promise kept? Is there a record? Does the record match what really happened? If you can, ask the person.

Case	Receive first contact	Decide eligibility	Deliver support	Review progress
1	Kept	Kept. Output: accepted	Kept	Kept
2	Late	Kept. Output: accepted	Referral made, no record of arrival	Not done
3	Kept	Wrong. Output: not eligible, old checklist used	Did not reach this step	Did not reach this step

Sort what you find

• Nonconformity. A promise was not kept. Example: a referral never followed up.
• Observation. The promise was kept, but it is at risk. Example: it only worked because one colleague remembered.
• Good practice. Something worth copying to other steps.

Then look across the rows. A problem that repeats across cases is rarely about one person. It is about the process.

One more check: before any number goes into the report, a second person recounts it from the same records. If the counts differ, find out why first.

F. When a promise breaks: correct, find the cause, act, verify

We do not hide mistakes. We turn them into clarity.

Keep one log of nonconformities. One line each: date, step, what happened, who was affected. Then four moves, in this order.

1. Correction. Help the person now. If someone was wrongly told "not eligible", reconsider their case this week.
2. Find the cause. Ask "why?" until you reach something you can change. Look in six places: people (training, workload), method (the procedure), documents and forms, systems (the database), environment (premises, time pressure), and measurement (are we even counting it right?).
3. Corrective action. Change the process, so the same thing does not happen to the next person.
4. Verify. Agree in advance how you will know it worked. Check later, on new cases. Only then close the line in the log.

Two mistakes are very common. The first is stopping after the correction: the person gets help, and three months later it happens again. The second is closing the log after the corrective action: "we changed the procedure" is something you did, not proof that anything improved.

G. Hear from the people you serve

Feedback and complaints are quality data. Treat them that way.

• Ask what happened, not only how happy they were. "Did you know what would happen next?" tells you more than "Were you satisfied?". People with low expectations are often satisfied with poor service.
• Ask everyone. Including people whose output was "not eligible", and people who left.
• Offer more than one route. Phone, paper, in person, someone speaking for them.
• Few complaints is not good news by itself. It can mean people are afraid, or cannot find the way.

You now have measures, audit findings and a log of corrective actions. In part 3 you put them into a two-page report, and take it to the meeting where decisions are made.

Next: Part 3. Review it and improve →