Intermediate course · $40

Research Literacy: Finding and Judging Trustworthy Sources

Decisions in this work get made on evidence: studies, statistics, news, and government reports. This course gives you a practical, repeatable method for telling solid sources from shaky ones, so the case you make and the choices you support rest on ground that holds.

Intermediate · for practitioners

Section One

Why the source matters before the number does

I quoted a statistic in a funding meeting once, and someone asked me where it came from. I had no answer.

In this work, evidence is not academic. A number in a grant application can move money. A study cited in a case note can shape what happens to a person. A statistic repeated in a public meeting can change a policy. When the source under that number is weak, the harm is not just to your credibility. It can send resources to the wrong place, or away from someone who needed them.

Source literacy The habit of asking where a claim comes from, and how it was produced, before you repeat it or act on it.

Source literacy is not about distrusting everything. A practitioner who dismisses all data is as stuck as one who believes any number with a decimal point. The goal is calibration: trusting strong evidence more, weak evidence less, and being able to tell which is which.

Two failure modes, not one

Credulity accepts a claim because it is printed, official-looking, or confirms what you already believe. Cynicism rejects a claim because the source is imperfect, which every source is. Both skip the actual work, which is judging the specific claim on its specific merits.

Quick check-in

Check your understanding before moving on.

1. Source literacy is best described as:

2. Why does a weak source carry real risk in homeless-services work?

Practice scenario: The statistic in the grant application

The situation A caseworker is finishing a grant application and wants to open with a strong line: "Studies show that 90 percent of people in our program stay housed after one year." The number came from a slide in a conference talk last year. No study, author, or date is attached to it.
Reading it through this lens The claim might even be true, but right now it is unsourced. If a funder asks for the citation and there is none, the whole application looks shaky. The fix is not to drop the idea, it is to find the actual study behind the slide, confirm it says what the slide said, and cite it. If no such study can be found, the number does not go in.

Section Two

What makes a source trustworthy

Peer-reviewed sounded like a magic word until I learned what it actually checks for.

Not all sources carry the same weight, and the differences are learnable. A few distinctions do most of the work.

Primary vs secondary source A primary source reports original research or first-hand data (the study itself, the government dataset). A secondary source describes or interprets a primary one (a news article about the study, an advocacy brief summarizing it). Always try to reach the primary source, because summaries drift.
Peer review Before publication in a research journal, other experts in the field review the work for sound method and reasoning. It raises the floor on quality. It does not certify that the conclusion is true.

When you reach a primary study, four questions tell you most of what you need: Who was studied, and how many (a finding from 30 people is not a finding about a city)? How was it measured? When was it done (a 2009 housing-market finding may not hold now)? And who conducted and funded it (covered in the next section)?

Government data: authoritative, not flawless

Sources like the Census, HUD's Annual Homeless Assessment Report, and local Point-in-Time counts are among the most reliable you will use, and they are appropriate to cite. They also have documented limits, especially undercounts of people who are unsheltered or doubled up. Reliable and complete are not the same thing. Cite them, and know their known gaps.

Walk upstream to the primary source
A claim reaches you: a statistic on a slide, a number in a brief, a line in a news story.
Find what it cites. Most claims travel through a secondary source that is summarizing something else.
Reach the primary source: the original study or dataset. Judge the claim there, where the method is visible.

Quick check-in

Check your understanding before moving on.

1. Peer review primarily certifies that:

2. You find a news article describing a study. The best next step is to:

Practice scenario: Two sources, one claim

The situation A colleague hands you two documents that both say rapid rehousing reduces returns to homelessness: a peer-reviewed study in a housing journal with a sample of several thousand households, and a four-page PDF from an organization that sells rapid-rehousing software, citing "internal data."
Reading it through this lens Both can be read, but they do not carry equal weight. The peer-reviewed study used a large sample and an outside method check. The PDF may be accurate, but it is a secondary summary, the underlying data is not shown, and the publisher benefits from the conclusion. Lead with the study. If you mention the PDF, name it as what it is.

Section Three

Spotting bias, funding conflicts, and advocacy dressed as research

The report had footnotes and charts. It still only ever pointed one direction.

Footnotes and charts are not proof of neutrality. They are formatting. To judge a source you have to look at who made it, why, and whether the method could have produced a result they did not want.

Conflict of interest When the people producing a claim stand to gain from a particular answer (funding, sales, reputation, a policy they already back). A conflict does not make a finding wrong. It does mean you check the method harder before you trust it.
Advocacy vs research Research investigates a question and reports what it finds, including inconvenient results. Advocacy argues a position using evidence. Both are legitimate and useful. The problem is advocacy presented as if it were neutral research, with the arguing hidden.

A few honest warning signs: the source only ever cites evidence pointing one way, it leads with the conclusion and works backward, it reports a percentage but hides the raw numbers, or it would not change its claim no matter what the data showed. That last one is the deepest tell.

Watch your own confirmation bias too

Confirmation bias is the pull to accept claims that fit what you already believe and to scrutinize the ones that do not. It is easiest to spot in a report you disagree with, and hardest to spot in one you like. Apply the same test to both: if a study supports your position, ask whether you are accepting it because the method is strong, or only because the conclusion is comfortable.

Quick check-in

Check your understanding before moving on.

1. A study was funded by a group that benefits from its conclusion. This means:

2. The deepest sign that a document is advocacy dressed as neutral research is that it:

Practice scenario: The "independent" evaluation

The situation A software vendor offers your agency a case-management tool and provides an "independent study" showing agencies that adopted it cut paperwork time in half. The study was commissioned and paid for by the vendor, and the agencies in it were selected by the vendor.
Reading it through this lens This is not automatically false, but it has two conflicts: the vendor funded it, and the vendor chose which agencies to include, which can quietly stack the result. Before relying on it, you would want an outside evaluation, or at least the full method and a sample the vendor did not hand-pick. Treat the vendor study as a claim to verify, not as settled evidence.

Section Four

Reading statistics without being fooled

Forty percent of what? That question has saved me more than once.

Most statistical mistakes are not lies. They are missing context that changes the meaning. A handful of questions catch the common ones.

Rate vs count A count is a raw number (1,200 people). A rate is that number against a population (1,200 out of 50,000, or 2.4 percent). A rising count in a growing city can still be a falling rate. Always ask for the denominator: the "out of what."

Two more traps worth naming. Correlation is not causation: two things moving together does not show one caused the other, since a third factor may drive both. And base rates matter: a "50 percent increase" in a rare event can still be a tiny number of people, while a small percentage of a huge population can be enormous.

Point-in-Time (PIT) count A one-night annual count of sheltered and unsheltered people, used widely in homelessness data. It is valuable and standard, but it is a snapshot, it is known to undercount unsheltered and hidden homelessness, and methods can change year to year. Compare years carefully, and say "counted" rather than implying it captured everyone.

Be precise about people, especially with health data

When a statistic touches mental health, resist collapsing it into "the mentally ill." The study almost always measured something specific, such as a share of people with a diagnosed condition, or people reporting a particular symptom. Name what was actually measured. Vague language invites stigma and usually overstates what the data can support.

Quick check-in

Check your understanding before moving on.

1. A report says homeless deaths rose 50 percent. Before repeating it, the most important question is:

2. Two trends rise together over the same period. On its own, this shows:

Practice scenario: The headline that doubled homelessness

The situation A local news headline reads "Homelessness Doubles in County." The article is based on this year's Point-in-Time count compared to last year's. Buried lower down: the county changed its counting method this year and added volunteers who reached encampments not surveyed before.
Reading it through this lens The count went up, but part of the increase is almost certainly better counting, not only more people becoming homeless. A method change breaks a clean year-over-year comparison. The honest version is that this year counted more people, with some of the rise driven by improved methods, and the true change is harder to state precisely. "Doubled" overstates what the data can support.

Section Five

A practitioner’s quick-evaluation method

I needed something I could actually run in the five minutes before a meeting, not a graduate seminar.

You will not run a full literature review every time someone hands you a number. You need a fast, repeatable check you can apply to any source in a few minutes. Run these in order, and stop early if it fails badly.

The five-minute source check

1. Source: Can I find the primary source, or only a summary of a summary? 2. Maker and money: Who produced and funded it, and do they benefit from the answer? 3. Method and sample: Who was studied, how many, and how was it measured? 4. Recency: Is it current enough to still apply? 5. Fit: Does the headline claim actually match what the evidence shows, or does it overstate it? 6. Falsifiability: Would this source change its claim if the data went the other way?

None of these requires a statistics degree. They require the discipline to ask before you cite. A source that passes most of them is usable, and you can say why. A source that fails several is one you set down, or flag clearly as weak when you have nothing better.

Citing honestly when evidence is thin Sometimes the strong source does not exist yet. That is allowed, if you say so. "Early data suggests" and "in our experience" are honest framings. "Studies prove" is not, unless studies actually do.

Quick check-in

Check your understanding before moving on.

1. The five-minute source check is mainly designed to:

2. When the strong source you want does not exist, the honest move is to:

Practice scenario: Running the check live

The situation In a team meeting, someone proposes ending a warming-shelter program, citing "a study showing warming shelters do not reduce winter deaths." You have ninety seconds before the group starts nodding along.
Reading it through this lens You do not need to refute it on the spot, you need the check. Ask the fast questions out loud: Where is the study, who ran it, who was studied, and how recent is it? Often the claim cannot survive the first question, because no one can name the source. If it can, you now know whether it is strong enough to act on. Either way, the decision rests on evidence instead of on whoever spoke most confidently.

Your certificate of completion

Answer 80% of the check-in questions correctly across all 5 sections to unlock your certificate.

0 of 0 answered correctly 0%

Keep going. You need 80% to unlock your certificate. You can retry any question above. You have passed. Enter your name below to generate your certificate.