Reliability & Scale
Why three of you code every comment, and how your work trains ReHuman
3
Coders per comment
≥70%
Target exact agreement
Millions
Comments daily, worldwide
Every comment is scored independently by three coders. You never discuss a comment until all three of you have finished. Only then do you open the Reliability tab, compare, and adjudicate the flagged disagreements together. Independence first, consensus second — that order is what makes the agreement statistics meaningful.
Your team aims for 70% exact agreement or higher (Krippendorff's α ≥ 0.70). Agreement below that is not a personal failure — it is a signal that an indicator's anchors need sharpening, which is itself a finding the methodology depends on.
Millions of comments are posted every day. Even a global network of student coders can only ever read a tiny fraction of them by hand. Human coding alone cannot measure discourse at the scale on which democracy is actually contested.
The comments your team adjudicates become ground truth — a gold-standard dataset of human judgements. ReHuman, the project's AI model, is trained to reproduce those judgements and then validated against comments it has never seen. Where it matches expert human coders, it can be trusted to score millions of comments automatically; where it fails, that failure points straight back to the indicators you found hardest.
This makes your coding doubly valuable. Every carefully adjudicated batch both measures the discourse in front of you and teaches the model that will eventually read what no human team could. The care you take is the ceiling on how good ReHuman can become.