Author: paul.seyringer

The last blogpost of this series was the “tool” post: the actual coding sheet. This final part is about whether that tool is worth using. The first big question is reliability: what happens if two people watch the same villain and still end up with completely different Big Five scores? If this framework is supposed to support systematic comparison and statistical analysis, it can’t rely entirely on one person’s subjective intuition. It needs to be stable across observers.

Making it trustworthy: coder agreement explained step-by-step

The solution is inter-rater reliability testing. That’s standard scientific practice: you get two different people to code the exact same character independently, and then you measure how similar their results are. The simplest way to understand it:

Imagine Coder A and Coder B both rate the Joker’s Neuroticism using the four Neuroticism items from the sheet.

Coder A gives scores of 7, 6, 7, 7 (average 6.75).
Coder B gives scores of 6, 6, 7, 6 (average 6.25).

Their final trait scores are only 0.5 points apart on a 1–7 scale. That means both people basically saw the same emotional profile: high instability, visible distress, volatility, and conflict. Small differences like this are normal. The “scientific way” (takes 30 seconds in Excel): Put the five averaged trait scores (E, A, C, N, O) from Coder A into one row and Coder B into the row below. Then use the formula:

=CORREL(A1:A5,B1:B5)

This returns a number between 0.0 (total disagreement) and 1.0 (perfect agreement). Research shows that values above 0.70 are usually considered acceptable (Gosling et al., 2003). If you clear that threshold consistently, your measurement is stable enough to trust. What this looks like in the analysis database Once you start building a database, you can add coder agreement as a quality-control column:

Character | Media | E | A | C | N | O | Coder Agreement

Joker | The Dark Knight | 6.5 | 1.5 | 2.0 | 7.0 | 7.0 | 0.82

Thanos | MCU | 5.0 | 2.5 | 7.0 | 3.0 | 6.0 | 0.91

Thomas Shelby | Peaky Blinders | 6.0 | 3.5 | 7.0 | 6.25 | 6.5 | test needed

Why this works across every type of antagonist and antihero

The entire point of this coding sheet is universality. I designed the 20 items to work across different genres, archetypes, and narrative roles. They don’t require special rules for “crime boss vs. fantasy villain vs. horror monster.” They simply capture observable patterns.

Mastermind types (Thanos, Hannibal Lecter, Littlefinger)

These tend to cluster around extremely high Conscientiousness, high Openness, and rock-bottom Agreeableness.

Chaos tricksters (Joker, Loki)

Usually high Openness, high Neuroticism, low Conscientiousness.

Silent brutes (Michael Myers, Grundy)

Low Extraversion, low Openness, low Neuroticism.

Tragic antiheroes (Walter White, Anakin Skywalker)

Often show changing profiles over time – especially rising Conscientiousness and surging Neuroticism as their arc progresses.

Same sheet, same math, different profiles. Perfect for pattern-finding. Thesis-ready and scalable.
From a project-planning perspective, the method is built for scale:

Time investment:

• 10 – 15 minutes per main character/antagonist
• 3 – 5 minutes for minor villains

Minimum viable sample: ~20 characters for early pattern detection and basic clustering

Reliability target: coder agreement above 0.70

Evidence confidence rating: I also plan to track evidence confidence based on screen time (e.g., 1 star = limited screen time, 3 stars = full narrative visibility).

That’s it for the Big Five part of the framework for now.

With the Personality Profile layer operationalized, I now have a concrete way to compare villains by personality structure without turning every analysis into a full essay. Next up, I’ll return to the other layers of the framework – especially Observable Traits (ACIS/visual coding) and Symbolism & Motivation – and then eventually start applying the full combined method to actual villains.

Until then — see ya!

Literature:

1. Costa, Paul T., and Robert R. McCrae. Revised NEO Personality Inventory (NEO PI-R) and NEO Five-factor Inventory (NEO-FFI). Psychological Assessment Resources (PAR), 1985.
2. Gosling, Samuel D., Peter J. Rentfrow, and William B. Swann Jr. “A very brief measure of the Big-Five personality domains.” Journal of Research in personality 37.6 (2003): 504-528.
3. John, Oliver P., Laura P. Naumann, and Christopher J. Soto. “Paradigm shift to the integrative big five trait taxonomy.” Handbook of personality: Theory and research 3.2 (2008): 114-158.
4. Soto, Christopher J., and Oliver P. John. “The next Big Five Inventory (BFI-2): Developing and assessing a hierarchical model with 15 facets to enhance bandwidth, fidelity, and predictive power.” Journal of personality and social psychology 113.1 (2017): 117.
   

In the last post, I explained why the Personality Profile layer of my framework needed a practical method. In theory, it’s easy to say “use the Big Five.” In practice, it’s a lot harder to apply it consistently to fictional characters, especially if you want the results to be comparable across a large sample. This post is the core of the entire Big Five part of my project: the universal 20-item coding sheet. The goal here is not to capture every nuance of a character’s psyche. That would take an entire dissertation per villain or antagonist. The goal is to create a fast, repeatable profile that can be compared, visualized, and eventually used for statistical clustering.

The universal coding sheet: 20 concrete questions

The system is simple:

• 20 positively worded behavioral statements
• 4 items per trait (Extraversion, Agreeableness, Conscientiousness, Neuroticism, Openness)
• Rated on a 1-7 scale:
  1 = never or almost never shown on screen
  7 = this behavior strongly defines the character

After rating all 20 statements, I average the four items per trait. The result is five values between 1 and 7. And the best part: no reverse scoring, no confusing “this is high but actually means low.” I wanted the sheet to be usable without a calculator brain.

How I code in practice

When I analyze a character, I watch the scenes that best represent their defining moments: majorconfrontations, negotiations, emotional breakdowns, moral dilemmas, power displays, andinterpersonal dynamics.Then I rate the statements using only observable evidence:

• What the character says (dialogue, monologues, tone)
• What the character does (decisions, violence, restraint, strategy)
• How they behave physically (gestures, posture, movement)
• How they react emotionally (fear, anger, guilt, stress)

If I genuinely can’t judge an item because there isn’t enough evidence, I assign a 4. That’s not “average personality.” It’s “neutral / unknown.” This keeps the data usable even when screen time is limited, which matters a lot for minor antagonists.

Time investment

In practice, this takes about 10–15 minutes for a major character with enough screen time.For minor villains with limited scenes, it can take 3–5 minutes. That efficiency is not a bonus. It is the entire point. Without it, this project wouldn’t scale. Now, here is the actual coding sheet.

Extraversion: Commands Attention or Lurks in Shadows?

Extraversion in personality research refers to energy, assertiveness, dominance, and the tendency to seek attention and control social situations. For villains and antiheroes, this trait often manifests in how they physically enter scenes, whether they dominate group conversations, and how they position themselves spatially relative to others.

1. Does the character draw or hold the attention of others during group scenes, negotiations, confrontations, or standoffs? Think of someone who interrupts conversations, makes everyone turn toward them, or commands the room simply by their presence.

2. Does the character naturally take control of situations and begin giving orders or directions to other people? This includes leading meetings, expecting obedience from subordinates, or stepping into leadership roles without hesitation.

3. Does the character dominate conversations and interactions through strategic use of their voice, whether through threats, persuasion, speeches, or memorable monologues? They control the verbal flow rather than reacting to others.

4. Does the character use physical presence, gestures, posture, or dramatic movement to intimidate, impress, or assert dominance? Examples include looming over others, expansive hand gestures during speeches, or deliberate slow strides into a room.

Agreeableness: Shows Loyalty to Someone, or Hurts Everyone Indiscriminately?

Agreeableness measures cooperation, empathy, concern for others’ feelings, and willingness to compromise. Low Agreeableness correlates with antagonism, manipulation, and callousness. What makes this trait fascinating for character analysis is that many sympathetic villains score low overall but show intense loyalty or protectiveness toward a small in-group, family members, loyal followers, or even pets, which creates audience empathy.

5. Does the character demonstrate genuine concern, worry, or protectiveness when specific allies, followers, subordinates, or family members are threatened, injured, or in danger? This goes beyond strategic necessity; they seem personally invested in these people’s well-being.

6. Does the character ever compromise their plans, hold back from violence, or adjust their behavior because they actually care about maintaining a specific relationship, rather than purely for tactical advantage? Genuine relationships influence their decisions.

7. Does the character display any understanding, mercy, forgiveness, or empathy toward an opponent, enemy, rival, or even subordinate at any point during their arc? Even a single authentic moment counts as evidence.

8. Does the character demonstrate moral boundaries by avoiding sadism, gratuitous cruelty, or unnecessary suffering, suggesting there are certain things they simply will not do? They have limits, even if those limits seem strange to us.

Conscientiousness: Calculated Long-Term Operator or Impulsive Reactor?

Conscientiousness encompasses organization, self-discipline, persistence, reliability, and goal-directed behavior. Mastermind-type villains and antiheroes typically score extremely high here, while chaotic or rage-driven antagonists score lower.

9. Does the character construct plans, schemes, or strategies that clearly involve multiple deliberate steps, phases, or moving parts working together? Their actions follow structured logic rather than random violence.

10. Does the character demonstrate persistence by continuing to pursue their long-term objectives even after experiencing major failures, betrayals, or significant setbacks? They adapt but don’t quit.

11. Does the character maintain or create organized operations, structures, or systems, such as gangs, armies, criminal networks, political machines, or conspiracy plots, that function with clear hierarchy and coordination? Chaos suggests low Conscientiousness.

12. Does the character deliberately delay or sacrifice short-term victories, immediate gratification, personal revenge, or easy wins in order to position themselves for larger strategic payoffs later? Classic long-game thinking.

Neuroticism: Emotionally Stable Machine or Cracking Under Pressure?

Neuroticism tracks proneness to negative emotions including fear, anxiety, anger, guilt, sadness, and emotional instability. High Neuroticism often humanizes villains by making their pain and vulnerability visible to the audience.

13. Does the character show visible signs of fear, hesitation, panic, emotional distress, or internal conflict during high-pressure situations? Physical tells like trembling hands, wide eyes, or frozen reactions count.

14. Does the character express or voice concerns, worries, or paranoia about threats to their safety, power, control, relationships, or position? They articulate vulnerability.

15. Does the character experience noticeable emotional volatility including rage outbursts, crying, sudden mood shifts, or unpredictable changes in behavior? Emotional control breaks down.

16. Does the character appear burdened or haunted by guilt, traumatic memories, past failures, regrets, or moral conflicts that visibly affect their present actions? Flashbacks, brooding, or haunted expressions provide evidence.

Openness to Experience: Creative Schemer or Simple Brute Force?

Openness measures intellectual curiosity, creativity, imagination, appreciation for complexity, and willingness to explore new ideas. High Openness villains often speak philosophically and use clever, unconventional tactics.

17. Does the character employ creative, indirect, or unconventional tactics and strategies rather than relying solely on straightforward violence or simple brute force solutions? They solve problems in unexpected ways.

18. Does the character demonstrate curiosity or fascination with new ideas, technologies, cultures, philosophies, weapons, or unusual alliances that fall outside their familiar world? Intellectual engagement with novelty.

19. Does the character use symbolic, visionary, philosophical, metaphorical, or abstract language rather than only concrete, literal descriptions? They think and communicate in complex conceptual terms.

20. Does the character demonstrate mental flexibility by adapting effectively to completely unfamiliar situations, environments, rules, worlds, or technologies without becoming confused or rigid? They roll with the strange.

So what’s next?

Now that the coding sheet exists, the next obvious question is: how do we know it actually produces stable results? In the next post, I’ll explain how I make this trustworthy through coder agreement and why this sheet works across very different villain archetypes without needing special rules for genre or medium.

See ya!

Literature:

1. Costa, Paul T., and Robert R. McCrae. Revised NEO Personality Inventory (NEO PI-R) and NEO Five-factor Inventory (NEO-FFI). Psychological Assessment Resources (PAR), 1985.
2. Gosling, Samuel D., Peter J. Rentfrow, and William B. Swann Jr. “A very brief measure of the Big-Five personality domains.” Journal of Research in personality 37.6 (2003): 504-528.
3. John, Oliver P., Laura P. Naumann, and Christopher J. Soto. “Paradigm shift to the integrative big five trait taxonomy.” Handbook of personality: Theory and research 3.2 (2008): 114-158.
4. Soto, Christopher J., and Oliver P. John. “The next Big Five Inventory (BFI-2): Developing and assessing a hierarchical model with 15 facets to enhance bandwidth, fidelity, and predictive power.” Journal of personality and social psychology 113.1 (2017): 117.
   

If you’ve read my first five Blogposts, you already know the basic structure of what I’m trying to build here: a framework that lets me systematically analyze villains and antiheroes, compare them across genres, and eventually gather enough data to find patterns in the kinds of “bad characters” audiences actually root for.

So far, I’ve defined four layers that make up a complete villain profile:

1. Observable Traits (visual/audio presence, presentation, measurable surface-level coding)
2. Personality Profile (Big Five values, used for comparison and clustering)
3. Symbolism & Motivation (what the character represents, why they act)
4. Creation Context (when and why this character exists culturally)

In theory, that framework is solid. But after writing the first five posts, I noticed something uncomfortable: the Personality Profile layer still sounded a little too clean on paper. The Big Five is the one part of my model that is unquestionably “scientific.” It has decades of empirical support. It has well-established measurement tools. It is widely used across psychology. And it gives me exactly what I need for statistical analysis: five distinct values that can be averaged, compared, visualized, and fed into cluster analysis.

But when it comes to fictional characters, the Big Five has a problem. It is usually measured through self-report questionnaires. Real people are asked how they feel, how they behave in everyday life, what they believe about themselves, and how they react across contexts. Fictional characters don’t get that luxury. They don’t fill out surveys. They don’t have off-screen daily routines. And even if they did, we wouldn’t see them. So the question I kept running into was simple: how do you measure a scientifically established personality model using nothing but on-screen behavior?

The problem with existing approaches: Before I wrote this post, I did what anyone would do: I searched online for examples. I wanted to see how others used the Big Five to analyze characters in movies or TV shows. And what I found was… not great.

Most examples fell into two categories:

1. Clinical questionnaires copied and pasted into fandom analysis.
   These were the classic “100-item Big Five test” formats. They technically work for measuring personality, but they are absolutely unusable for my goals. If I want to analyze 50 villains, I cannot spend hours per character. I need this to be fast.
2. Vague descriptive labels without method.
   This was the opposite extreme: people saying things like “this character is high in Neuroticism” or “this villain has low Agreeableness,” but never explaining why, never showing which scenes count as evidence, and never providing a replicable scoring system.

For casual fandom discussion, that’s fine. But for a framework that aims at building a database and eventually running comparisons and statistical clustering, it’s a dead end. What I needed was something in between: a system that is scientifically grounded, but also practical enough to apply repeatedly.

Where it comes from: the science, explained simply

The key breakthrough came when I stopped looking for “character Big Five analysis” examples and instead went straight back to the original research and measurement tools. And what I found there was surprisingly reassuring: you don’t need 100 items to get a meaningful Big Five profile. The Big Five is not a single test. It’s a trait taxonomy. The tests are just tools for measuring it. My coding sheet is built around four cornerstone sources that are basically unavoidable in Big Five research:

Costa and McCrae (1992) – NEO-PI-R Manual

This is the professional manual for one of the most widely used personality inventories. What makes it useful here is that it breaks each trait down into facets and ties them to concrete behavioral examples. It is much easier to translate “makes detailed plans and follows through” into a screen-observable indicator than to translate something like “I often feel stressed.”

Gosling, Rentfrow, and Swann (2003) – Ten-Item Personality Inventory (TIPI)

This paper was essential for confidence. It showed that extremely short Big Five measures can still be reliable. They used only two adjectives per trait and still got valid results. That doesn’t mean longer measures are useless – it just means that brevity doesn’t automatically destroy reliability.

John, Naumann, and Soto (2008) – Handbook chapter

This is one of the clearest descriptions of high vs. low Big Five traits in everyday life. It helped me sharpen the difference between “low Agreeableness as cold manipulation” versus “low Agreeableness as blunt tough-mindedness,” for example. Those nuances matter a lot for villains.

Soto and John (2017) – BFI-2 update

Their work reflects a modern approach: wording items to reflect observable behavior rather than vague self-perception. This is exactly what fictional character analysis needs. When you can’t ask someone what they feel, you focus on what they do. So the idea behind this coding sheet is not to reinvent the Big Five. It’s to translate it. And since my entire framework already emphasizes observable traits (especially through ACIS), this fits perfectly into the “only code what is on screen” logic I’ve been using from the start.

So what’s next?

In the next post, I’ll finally present the actual 20-item coding sheet and explain how I rate characters in practice – including what “neutral” looks like when evidence is missing, and why I use a 1-7 scale instead of a simple yes/no.

Until then – see ya!

Literature:

1. Costa, Paul T., and Robert R. McCrae. Revised NEO Personality Inventory (NEO PI-R) and NEO Five-factor Inventory (NEO-FFI). Psychological Assessment Resources (PAR), 1985.
2. Gosling, Samuel D., Peter J. Rentfrow, and William B. Swann Jr. “A very brief measure of the Big-Five personality domains.” Journal of Research in personality 37.6 (2003): 504-528.
3. John, Oliver P., Laura P. Naumann, and Christopher J. Soto. “Paradigm shift to the integrative big five trait taxonomy.” Handbook of personality: Theory and research 3.2 (2008): 114-158.
4. Soto, Christopher J., and Oliver P. John. “The next Big Five Inventory (BFI-2): Developing and assessing a hierarchical model with 15 facets to enhance bandwidth, fidelity, and predictive power.” Journal of personality and social psychology 113.1 (2017): 117.