Evidence before the decision
Evidence for decisions and unintended consequences.
- Level
- Bitesize
- Learning time
- 1 hour 15 minutes
- Where
- Complexity, decisions and innovation
- Price
- £39
When you step up to a higher level, what you paid in the last 12 months for lower-level courses inside the bigger course comes off its price, in proportion to how much of each is inside it, whichever topic it's from. Courses at the same or a lower level don't give credit against each other. A Focus course or above always costs at least £15. Read the full terms
Certificate of Completion
Overview and Outcomes.
Overview
A decision is only as good as its evidence, and every measure changes the behaviour it was meant to observe. The pairing starts with Deming's standard of accurate, valid, precise and reliable data, the main types of data and what each can support, and why a survey is only as good as its design. The 2020 A Level grading algorithm shows capable people producing a confident, precise and wrong answer. It then explores unintended consequences through Robert Merton, Goodhart's and Campbell's laws, Bevan and Hood's three kinds of gaming, the four-hour A&E target and a Welsh home care example from Cartrefi Cymru.
What you'll develop
In this Bitesize, you will develop the ability to:
- Apply Deming's four tests of accurate, valid, precise and reliable to a figure you use in your own work.
- Explain why measures change behaviour, using Merton's causes of unintended consequences and Goodhart's and Campbell's laws.
- Distinguish the major types of data and identify which calculations, such as averaging, each can honestly support.
- Identify ratchet effects, threshold effects and output distortion in a target, and recognise the fixes that fail pattern.
- Assess the evidence behind a decision by asking what the data is, where it came from, what it is asked to prove, and what happens if it is wrong.
- Design a small basket of measures for learning that starts with purpose, pairs numbers with stories and adapts when measures distort behaviour.
The Modules.
1 hour 15 minutes of learning.
- 10Understanding Data: The Evidence Base for Good DecisionsEvery technique in this programme depends on the evidence feeding it. This module applies Deming's standard of accurate, valid, precise and reliable data, sets out the main types of data and what each can legitimately support, and covers how primary data is collected, including why a survey is only as good as its design. The 2020 A Level grading algorithm shows how capable people produced a confident, precise and wrong answer.39 min
- 08Determining Outcomes and Unintended ConsequencesEvery measure changes the behaviour it set out to see. This module draws on Robert Merton's causes of unintended consequences, Goodhart's and Campbell's laws, and Gwyn Bevan and Christopher Hood's three kinds of gaming, then examines the four-hour A&E target in England and a Welsh home care example from Cartrefi Cymru. It closes with the fixes that fail archetype and a process for designing measures for learning rather than control.39 min