How do you judge which sources and evidence to trust in an inquiry, and how do you guard against your own confirmation bias?
Explain how to evaluate sources and evidence in the Independent Study, applying criteria of reliability and relevance and guarding against bias and cherry-picking
A focused answer on evaluating sources and evidence for the Independent Study. Criteria for source reliability and relevance, primary versus secondary sources, the strength and quality of evidence, and guarding against confirmation bias and cherry-picking.
Reviewed by: AI editorial process; not yet individually human-reviewed
Have a quick question? Jump to the Q&A page
Jump to a section
What this dot point is asking
SEAB wants you to be able to evaluate the sources and evidence on which an Independent Study draws: to judge which to trust, how strongly, and how relevant they are, and to guard against the biases that distort selection. This applies the source-evaluation skills of the reasoning area to the practical task of building a credible inquiry. Your task is to set out the criteria for assessing sources and evidence and to explain how a rigorous inquirer tests their own thesis rather than merely defending it.
The answer
Criteria for source reliability
Sources are not equal, and reliability is judged across several criteria, as a matter of degree. The main ones are: the relevant expertise of the author; the track record and editorial standards of the publication (peer-reviewed journal versus anonymous post); independence (whether the source has an interest in the conclusion); whether it is primary or secondary; transparency of method and data (can the claim be checked?); recency, where the field moves quickly; and corroboration by independent sources. No single criterion is decisive; a confident assessment triangulates several.
Primary and secondary sources
A primary source is first-hand evidence: original data, a direct testimony, the text or artefact itself. A secondary source reports, interprets or analyses primary material. Both have uses, but they answer different needs. For establishing what was said or found, primary sources are stronger; for context and interpretation, good secondary sources are valuable. A study that relies only on secondary reports of primary evidence inherits any distortion in the chain, so tracing claims back to their primary source is part of rigour.
Assessing the evidence, not just the source
Source reliability and evidence quality are distinct and must be assessed separately. A reliable source can present weak evidence (a single anecdote, a tiny sample, a correlation mistaken for causation), and a biased source can present strong evidence. So after judging the source, judge the evidence on its own terms: the size and representativeness of any sample, whether causal claims are supported, whether the data actually bear on the question (relevance), and how strong the inference from evidence to conclusion is. Conflating the two leads either to over-trusting a reliable source's weak evidence or dismissing a suspect source's strong evidence.
Bias is a reason to scrutinise, not to dismiss
A source with an interest in the conclusion warrants extra scrutiny, but bias does not make its claims false. To reject a source's evidence solely because of who produced it is the genetic fallacy. The correct response to bias is to look harder for independent corroboration and to check the method, not to discard the content. Crucially, this principle must be applied even-handedly: it is illegitimate to wave away disagreeing sources as biased while exempting agreeing ones from the same test.
Guarding against your own bias
The most insidious threats come from the inquirer. Confirmation bias is the tendency to favour evidence that supports a prior view and to discount evidence against it. Cherry-picking is selecting only confirming sources, which guarantees apparent support regardless of the truth. These corrupt an inquiry from within. The safeguards are deliberate and procedural: actively seek disconfirming evidence and the strongest opposing sources; apply the same reliability criteria to confirming and disconfirming material alike; weigh the whole body of evidence rather than isolated favourable items; keep an audit trail of sources considered, including those not used; and represent opposing views in their strongest form. A rigorous inquiry tests its thesis against the best contrary evidence rather than assembling a case for a predetermined conclusion.
Examples in context
Example 1. Tracing a claim to its source. A student finds a striking statistic repeated across many blogs and prepares to cite it. Tracing it back reveals that all the blogs draw on a single secondary report, which itself misread the original primary study. The statistic, once checked at source, does not say what the blogs claim. The example shows why distinguishing primary from secondary sources, and following claims to their origin, is part of rigorous evidence evaluation rather than optional thoroughness.
Example 2. Testing rather than defending a thesis. Two students hold the same initial view. One searches only for supporting articles and assembles a confident-looking case. The other searches neutrally, reads the strongest opposing studies, and finds the evidence more mixed than expected, revising the thesis accordingly. The second study is far more credible, because it tested its thesis against disconfirming evidence. The contrast illustrates the difference between genuine inquiry and confirmation bias.
Try this
Q1. State four criteria for assessing the reliability of a source. [6 marks]
- Cue. Any four of: relevant expertise, publication track record and standards, independence (no interest in the conclusion), primary versus secondary status, transparency of method, recency, and corroboration by independent sources.
Q2. Explain why source reliability and evidence quality must be assessed separately. [6 marks]
- Cue. A reliable source can present weak evidence (small sample, anecdote, correlation mistaken for cause), and a biased source can present strong evidence; judging only the source over-trusts or wrongly dismisses the actual evidence.
Q3. Explain confirmation bias and two safeguards against it in an inquiry. [8 marks]
- Cue. Confirmation bias favours evidence supporting a prior view; safeguards (any two): actively seek disconfirming evidence and the strongest opposing sources, apply uniform criteria to both sides, weigh the whole body of evidence, and keep an audit trail of sources considered.
Exam-style practice questions
Practice questions written in the style of SEAB exam questions on this dot point, with worked answer explainers. The year tag is the paper they imitate, not the source.
Original12 marksExplain the criteria you would use to evaluate the reliability of sources in an independent study, and how you would guard against bias in your own selection of evidence.Show worked answer →
A strong answer gives reliability criteria: relevant expertise of the author, the track record and editorial standards of the publication, independence (absence of an interest in the conclusion), whether the source is primary or secondary, transparency of method and data, recency where it matters, and corroboration by independent sources. It stresses that reliability is a matter of degree assessed across these, not a binary.
It distinguishes assessing the source from assessing the evidence: even a reliable source can present weak evidence (a small sample, an anecdote), and a biased source can present strong evidence; so source and evidence must be judged separately, and bias is a reason to scrutinise, not to dismiss (avoiding the genetic fallacy).
On guarding against one's own bias, it names confirmation bias (favouring evidence that supports a prior view) and cherry-picking (selecting only confirming sources). Safeguards: actively seek disconfirming evidence and the strongest opposing sources, weigh the body of evidence rather than isolated items, keep an audit trail of sources considered, and represent opposing views in their strongest form.
Judgement-style close: rigorous inquiry uses explicit criteria and deliberately tests its own thesis against the best contrary evidence. Markers reward the reliability criteria, the source-versus-evidence distinction, the bias-is-not-refutation point, and concrete safeguards against confirmation bias.
Original12 marksCritically assess the following research practice. 'For my study I searched online for articles supporting my thesis, found a dozen that agreed with me, and cited them as my evidence. I left out the articles that disagreed because they were clearly biased against my view.'Show worked answer →
The expected answer diagnoses two linked faults. First, cherry-picking: selecting only confirming sources guarantees apparent support regardless of the truth, so the resulting evidence base is worthless as a test of the thesis. Second, confirmation bias dressed up as source criticism: dismissing the disagreeing articles as biased simply because they disagree is the genetic fallacy, and conveniently exempts the agreeing articles from the same scrutiny.
It explains why this undermines the inquiry: a thesis that has not been tested against the best contrary evidence is unsupported, however many friendly citations it has; selection by conclusion is not evaluation.
It prescribes the fix: search neutrally for the strongest evidence on both sides, apply the same reliability criteria to confirming and disconfirming sources alike, weigh the whole body of evidence, and engage the strongest opposing arguments rather than dismissing them.
Judgement: the practice is methodologically invalid because it selects evidence by agreement and applies bias-criticism selectively; a credible study must seek and confront disconfirming evidence. Markers reward naming cherry-picking and confirmation bias, the genetic-fallacy point about the selective dismissal, and the corrective procedure.
Related dot points
- Explain how to frame a research question for the Independent Study, distinguishing good from poor questions and refining scope, contestability and answerability
A focused answer on framing a research question for the Independent Study. The marks of a good question (focused, answerable, contestable, significant), how to narrow a broad topic, and the common faults of vague, loaded or unanswerable questions.
- Explain how to choose and justify a methodology for the Independent Study, matching method to question across conceptual, empirical and mixed approaches and addressing rigour and ethics
A focused answer on selecting a methodology for the Independent Study. Matching method to question, conceptual versus empirical (qualitative and quantitative) approaches, criteria of rigour such as validity and reliability, and ethical and practical constraints.
- Explain how to construct a sustained argument for a thesis in the Independent Study and defend it by anticipating and answering the strongest objections
A focused answer on building and defending a thesis in the Independent Study. Moving from question to thesis, structuring premises and evidence into a sustained argument, steelmanning objections, replying to them, and reaching a qualified, defensible conclusion.
- Explain how to structure and write the Independent Study dissertation, from introduction and methodology to argument, evaluation and conclusion, with sound referencing and academic integrity
A focused answer on writing the Independent Study dissertation. The standard structure, the function of each section, signposting and clarity, referencing and avoiding plagiarism, and reflecting on limitations and significance.