Sign up to access the complete lesson and track your progress!
Unlock This CourseImagine two headlines. One says, "Most teenagers in the city love the new bus service." The other says, "Teenagers hate it." Both claim to be based on research. So who is right? A good sociologist does not just accept a result. They ask how the research was done, and whether anything might have pushed the results in the wrong direction.
This lesson gives you a toolkit for judging research. You will use it all through Paper 1, because you must be able to analyse and evaluate research choices, not just describe them.
Key terms:
Bias can creep in from two sides. The researcher plans the study, and the respondents take part in it.
The researcher may ask leading questions, choose a sample that suits their idea, notice only what supports their view, or leave out results they dislike. They may not do this on purpose. A researcher who loves a bus service might word every question so that it sounds good.
People may not tell the truth. They might want to look good by giving the answer they think is "right", want to please the researcher, forget details, or exaggerate. A student asked how much homework they do may say more than they really do.
Bias lowers the quality of research. If the evidence is biased, the findings may not be true, and nobody should build a conclusion on them.
Who asks matters: the interviewer effect means age, clothes or reactions can change the answers people give
An interview is a conversation, and conversations are shaped by who is in the room. The interviewer effect happens when something about the interviewer changes the answers. This might be their age, gender, ethnicity, accent, clothes or the way they react.
For example, a young woman might feel less comfortable talking about family arguments with an older male interviewer than with someone her own age. She may hold back, or say what she thinks he wants to hear. The answers then show the effect of the interviewer, not just her real views.
The interviewer effect is a type of bias. It is mostly a problem for interviews, but a researcher who reads out a questionnaire face to face can cause it too.
Workers who know they're being watched may work harder - that's the Hawthorne (observer) effect
The Hawthorne effect is named after a series of studies in a factory in the USA, where workers changed how they behaved when they knew they were being studied. It is also called the observer effect.
Think about your own behaviour. Would you act exactly the same in class if a stranger sat at the back writing notes about you? Most people would not. They may work harder, behave better or act differently without even realising.
If people know they are being observed, what the researcher sees is not natural behaviour. This is a particular problem for observations, but it can happen in any method where people know they are being studied.
You already know what validity and reliability mean. Now you use them as tools to judge a study.
Ask: does this study give a true picture of what people really think or do? Bias, the interviewer effect and the Hawthorne effect all reduce validity, because the data is not a true picture.
Ask: could another researcher repeat this study in the same way and get the same results? Standard questions and clear steps make a study easier to repeat. A very personal, unplanned approach is harder to repeat.
This gives you a simple way to write evaluation: name the problem, say which tool it affects, and explain why.
Most researchers cannot study everyone, so they study a smaller group called a sample. The key question is: is the sample like the wider group?
A sample is representative if it has the same mix of people as the whole group, for example the same balance of ages, genders, ethnic groups and social classes. A study of "teenagers in the city" that only asks pupils from one private school is not representative.
If a sample is representative, the findings can be generalised, which means they can fairly be applied to the whole group. If it is not, the findings only tell us about the people in the sample. Large, well-chosen samples are usually more representative. Small, in-depth studies of a few people are usually less so.
Large surveys with hundreds or thousands of respondents can show patterns across a whole population.
Small interview or observation studies give deep, valid detail, but the findings may not apply to everyone.
How samples are chosen is covered in a later lesson, Sampling Frames and Sampling Techniques. For now, remember the link: representative sample, so generalisable findings.
Use this simple chain whenever you are asked about a study:
A teacher researcher interviews 12 students at her own school about bullying. Because she is a teacher at the school, students may not be honest (interviewer effect and respondent bias), which lowers validity. Only 12 students from one school were chosen, so the sample is not representative and the findings cannot be generalised to all schools.
Students often write "the sample was too small" without saying why that matters. Always add that it is not representative, so the results cannot be generalised. Another mistake is mixing up the Hawthorne effect (people know they are watched) with the interviewer effect (the interviewer influences the answers).
Source A: A researcher wants to find out how teenagers feel about social media. She interviews 20 pupils from one school in her home town. The interviews take place in the head teacher's office. She is a former teacher at the school, and she later writes that most teenagers feel social media is good for them.
(a) From Source A, identify two problems with the way the research was carried out. [2 marks]
(b) Using information in Source A, explain two reasons why the findings may not be valid. [4 marks]
(c) Explain one strength and one limitation of using a sample of 20 pupils from one school, as in Source A. [4 marks]
(a) Any two: only 20 pupils from one school were interviewed (1). The interviews were held in the head teacher's office (1). The researcher was a former teacher at the school (1).
(b) Reason 1: the interviews took place in the head teacher's office, so pupils may have felt nervous or afraid to criticise social media (1), which is respondent bias and lowers validity (1). Reason 2: the researcher is a former teacher, so there may be an interviewer effect, as pupils may say what a teacher wants to hear (1), meaning the answers are not a true picture (1).
(c) Strength: a small sample of 20 pupils is quick and cheap to interview (1), and each pupil can give detailed answers about their own feelings (1). Limitation: the sample is only 20 pupils from one school, so it is not representative of all teenagers (1), which means the findings cannot be generalised to other places or schools (1).
In part (b), do not just say "it is biased". Pick a specific point from Source A, name the effect, and say what it does to validity. That is how you reach all four marks.