Database results:
examBoard: AQA
examType: GCSE
lessonTitle: Interobserver Reliability
When psychologists observe behaviour, how do we know their observations are accurate? What if two researchers watching the same thing see it differently? This is where interobserver reliability becomes crucial in psychological research.
Key Definitions:
Imagine two researchers watching children in a playground to count aggressive behaviours. If one counts 15 incidents but the other counts only 5, we have a problem! Without agreement between observers, we can't trust the data. Good interobserver reliability means our research findings are more likely to be valid and can be trusted.
Psychology aims to be a scientific discipline. This means our research methods must be reliable and produce consistent results. When multiple observers agree on what they've seen, it suggests the observations reflect reality rather than subjective opinions or biases.
There are several ways to calculate interobserver reliability. The simplest is to work out the percentage of agreement between observers.
Percentage agreement = (Number of agreements ÷ Total number of observations) × 100
Example: Two observers watch a child for 30 minutes, recording every time they share a toy. Observer A records 12 instances, Observer B records 14. They agree on 10 of these instances.
Percentage agreement = (10 ÷ 16) × 100 = 62.5%
Note: The total observations is 16 (not 26) because we count each unique event (10 they both saw, plus 2 only A saw, plus 4 only B saw).
Generally, researchers aim for at least 80% agreement between observers. Anything below 70% is considered poor reliability. The higher the percentage, the more confident we can be in the data.
A more sophisticated measure is Cohen's Kappa, which accounts for agreements that might happen by chance. This gives a value between -1 and +1:
Kappa below 0.40
Observers aren't reliably seeing the same things
Kappa between 0.40-0.75
Reasonable reliability but room for improvement
Kappa above 0.75
Very good reliability between observers
There are several ways to improve agreement between observers:
The most important step is to clearly define exactly what behaviours count. For example, if studying "aggressive behaviour," you need to specify whether this includes verbal aggression, pushing, or just hitting.
Rather than just asking observers to record "helping," specify exactly what counts:
This clarity makes it much more likely that different observers will record the same events.
Before the actual study begins, observers should be trained together. This involves:
Using well-designed observation sheets with clear categories and checkboxes helps observers focus on the same aspects of behaviour. This reduces the chance they'll interpret events differently or miss important behaviours.
Recording observations on video allows multiple viewings of the same behaviour. Observers can watch events multiple times if needed and researchers can check reliability after the fact by having new observers code the same footage.
Researchers wanted to study how often children help each other in classroom settings. Two observers watched the same classroom for 1 hour, recording each instance of helping behaviour.
Initially, their agreement was only 45% - Observer A recorded 22 instances while Observer B recorded 18, with only 9 agreements.
After refining their definition of "helping" and training together with video examples, they repeated the observation. This time they achieved 85% agreement, making their data much more reliable.
Over time, observers may gradually change how they interpret behaviours, leading to decreased reliability. Regular retraining sessions can help prevent this.
Observers may be influenced by what they expect to see or by knowing the research hypothesis. Using "blind" observers who don't know the study's purpose can reduce this.
Some behaviours are inherently difficult to observe reliably, especially subjective states like "anxiety" or "interest." These require especially clear operational definitions.
In exams, you might be asked to:
Remember to use specific examples to support your answers!
Log in to track your progress and mark lessons as complete!
Login NowDon't have an account? Sign up here.