Common Mistakes & How to Avoid Them
Learn from the most frequent coding errors to improve your accuracy
Scoring your agreement with the post's politics
The single most common error. You read a post arguing for a political position you disagree with and your scores creep downward. Every time you enter a score, ask: 'Am I scoring this indicator, or am I scoring whether I agree?'
Treating short posts as automatically low-scoring
A three-word post can still score 0 on Empathy if those three words are cruel. A two-sentence post can score 4 on Solidarity. Score what is there, not what is missing due to length.
Using SKIP when you should score 0
SKIP means the dimension cannot be applied. It does not mean the post scores badly. If a post is hostile, it scores 0 on Anti-Hostility — you don't skip that indicator.
Rushing through to finish
Content analysis requires careful, deliberate reading. Budget at least 3-5 minutes per post. Read the post twice before scoring — once for overall impression, then again indicator by indicator.
Not reading the Score Guide
The Score Guide provides anchor descriptions and examples for every indicator. Keep it open while you code. When in doubt, re-read the relevant anchor descriptions.
Coding Challenges & Guidance
Score what is present, not what is missing due to brevity. A one-word insult scores 0 on Respectful Tone regardless of its length. A three-word expression of solidarity ('We stand together') may score 3 on Solidarity. If a comment doesn't contain enough material to assess an indicator, record Low confidence.
Score the overall balance of the comment on each indicator. Don't average mechanically — ask which quality is more dominant, more sustained, and more structurally central to the comment's purpose.
The DDI does not evaluate whether political positions are correct. It evaluates how they are expressed. A comment arguing for a position you strongly disagree with can score 4 on every dimension. If you find yourself scoring a comment low because of what it says rather than how it says it, that is a coding error.
Score every comment against the absolute anchors, not against the other comments in the batch. A run of vicious comments can make a merely rude one feel civil; a run of thoughtful ones can make an average comment feel weak. Reset to the rubric for each comment — score it as if it were the only one you had read that day.
Source Type is a field to record, not a scoring shortcut. Do not raise or lower a score because of where the comment appeared. The whole point of the study is to discover how discourse differs across source types — that discovery is only valid if you score each comment on its own words, blind to your expectations.
Use SKIP when a dimension is structurally inapplicable — when there is simply no content in the comment that could be scored on that dimension. Don't SKIP because scoring would be inconvenient or because the comment scores 0. Score 0 is meaningful data. When in doubt, score rather than skip, and record Low confidence.
- Score discourse quality, not political content
- Read the initiating post first, then the comment twice
- Keep the Score Guide open while coding
- Be honest about uncertainty with Low confidence
- Score each comment against the absolute anchors
- Score only what is present in the comment itself
- Score based on whether you agree with the comment
- Rush through comments to finish quickly
- Code without checking the anchor descriptions
- Let the previous comment set your baseline
- Adjust a score because of the Source Type
- Compare scores with other coders before everyone is done
Remember: Low reliability is information, not failure. Every piece of professional content analysis research goes through a reliability-testing and rubric-refinement stage. Finding that an indicator is hard to apply consistently is a valuable scientific finding — it tells researchers that the indicator needs to be more precisely defined.