Test Design — Event Type vs Victim Identity

Navigation

Hub: 00 - Fact-Check MOC · Previous: 02 - Data Sources · Next: 04 - Results Stage 1

The event-type vs victim-identity test, with its interpretation rules fixed before any result was seen.

Tests an objection the book never states: that emotive terms like “massacre” and “slaughter” attach to a kind of violence rather than to the identity of the victims.

If that objection holds, Johnson’s central finding is largely an artifact of comparing a discrete close-range attack (October 7) against a diffuse aerial and siege campaign, and the asymmetry he reports measures event structure rather than editorial bias.

The hypothesis, stated precisely

H1 (event-type). Massacre-vocabulary tracks violence form: close-range, deliberate, against defenseless civilians, ideally with identifiable victims and a discrete time signature. Victim identity is incidental.

H2 (victim-identity). Massacre-vocabulary tracks victim identity: Israeli and Ukrainian civilian deaths receive it, Palestinian civilian deaths do not, holding violence-form constant.

These make opposed predictions, which is what makes the test worth running.

Design

Hold violence-form roughly constant; vary victim group.

Seven-day window following each event. CNN and MSNBC, matching the book’s TV corpus.

Bucha is the load-bearing case

Bucha is close-range killing of civilians by a state military, discovered and covered as a discrete event. It is the closest available structural match to October 7 with a different victim group, and unlike the Gaza cases it is not confounded by contested attribution. If Bucha draws massacre-language at rates comparable to October 7 while the flour convoy and Nuseirat do not, H1 fails and the objection collapses.

Mariupol serves the same function for the aerial comparison against Jabalia.

Normalization

Raw counts are useless here. October 7 dominated airtime, so of course it accumulates more of every word. Two denominators, both reported:

1. Percentage of event context. Term-matching clips divided by all clips in the window mentioning the relevant victim group on that network. This controls for total coverage volume, which is the confound that would otherwise sink the comparison.
2. Clips per 100 deaths. Controls for event magnitude.

Neither is perfect. Report both; if they disagree, that disagreement is itself informative.

Running it

python3 gdelt_event_type_test.py --dry-run          # inspect the 98 queries first
python3 gdelt_event_type_test.py --out ./results    # stage 1, ~3 minutes
python3 gdelt_event_type_test.py --out ./results --sample   # stage 2, clip export

GDELT’s TV API is free and needs no key. Responses are cached in results/.cache, so re-runs cost nothing. Must run somewhere with open network access.

Limitations, which are not minor

Stage 1 output is a screen, not a result

A clip matching “massacre” AND “gaza” may well be describing October 7 inside a segment that also mentions Gaza. Co-occurrence within a 15-second clip does not establish that the term refers to the victim group. This is exactly why Johnson hand-coded a Victim column, and why stage 2 exists.

Stage 2 exports clip-level rows with empty VICTIM_CODE and IS_QUOTE columns for manual coding. Code a random sample of at least 100 per event before treating any stage-1 number as a finding. IS_QUOTE matters because Johnson excludes terms inside quoted official statements from his counts; to compare against his figures, the same exclusion has to be applied here.

Further constraints worth stating plainly:

• Bucha’s date is discovery, not occurrence. The killings happened over March; coverage begins around 1 April. The window is anchored to coverage, which is what is being measured, but it is not strictly parallel to the other events.
• al-Ahli had contested attribution from the first hours. Editorial hedging there has an obvious non-bias explanation. Analyze it separately and do not fold it into the aggregate.
• Casualty figures are approximate and contested, particularly Mariupol (estimates span 12 to 600) and al-Ahli. They are denominators, so they drive the per-100-deaths column directly. Verify each against primary sources before publishing anything.
• Two networks only. Extends to print only if a comparable corpus is found.
• This tests the vocabulary objection, not the book’s thesis. Even a decisive result here leaves the causal claim — that coverage enabled rather than merely accompanied — untouched.

How to read the outcome

If close-range Palestinian events draw massacre-language at rates comparable to Bucha and October 7: the objection succeeds, and Johnson’s headline asymmetry is substantially an artifact of comparing unlike events. His Chapter 7 Ukraine comparison would survive, since it holds violence-form constant by design, but the Chapter 2 finding would need heavy qualification.

If Bucha draws it and the Palestinian close-range events do not: event-type is ruled out as the explanation, and Johnson’s finding is stronger than his own methodology demonstrates, since he never ran this control.

If everything is near-zero outside October 7: the terms may simply be rare enough outside singular events that the comparison cannot support either conclusion. A null result. Report it as one rather than reaching for the more interesting reading.

Pre-commit to these interpretations before looking at the output. The whole value of the test is that it can come out either way.