On Sunday 29 July, we will be sharing the results of our Twitter research in a talk at DEF CON, the world’s largest hacker conference, held annually in Las Vegas, Nevada. Although it’s termed a hacker conference, it attracts speakers and thinkers from a wide range of backgrounds, including respected privacy researchers such as Danah Boyd, Christopher Soghoian, and security experts such as Bruce Schneier. DEF CON audiences contain critical thinkers from a variety of disciplines, from deep technical to policy making and law enforcement. It is for these reasons that we chose to submit our talk to DEF CON over other conferences and we’re honoured to have been accepted.
The Big Five Experiment (on Twitter)
Since the beginning of the year, we’ve been investigating what someone’s Twitter activity might reveal about their personality, looking at a broad range of personality traits. After tweets from celebrities such as Stephen Fry and Tony Hawk, we had over 3,000 people take part in our study, exceeding our expectations.
We’ve analysed over 3 million tweets from a final set of 2,927 tweeters from over 80 countries. We examined profile information, number of tweets, re-tweets and replies, a user’s klout score and we also examined the words used in people’s tweets.
As we progressed with our research, we also observed a growing public interest in Psychopaths, attributable perhaps to the popularity of Jon Ronson’s book “The Psychopath Test”. However, one news article stood out amongst all others and it is this article which provided us with the title for our DEF CON talk.
“Can twitter help expose psychopath killers’ traits?”
The article leads with the following statement:
“Scientists are trying to discover if we can spot a psychopathic killer using anything he or she has written, from a tweet to a blog post. The researchers want to take methods used to analyze the language of psychopaths and apply it to the general population using social media, reports CBS News”
We therefore decided that out of all the personality traits examined, our initial focus would be on the more anti-social personality traits of Psychopathy, Machiavellianism and Narcissism, often referred to as ‘The Dark Triad‘ in order to critically examine the question posed in the news article. Additionally, we had some concerns regarding the portrayal of social media personality prediction in mainstream media.
Concerns
The key concerns that we aim to address in the talk and the paper include:
- Public understanding of psychopathy
- General public focus on whether we can spot psychopaths and therefore predict crime
- Public perception that detecting personality from social media is infallible
Public understanding of psychopathy: Readers would be forgiven for thinking there are two types of people, Psychopaths (1%) and everyone else (99%) with everyone else having no psychopathic personality traits at all. The reality is a little bit more complicated. Psychopathic personality traits are generally measured on a scale from low to severe, a line on which all of us fit somewhere. Where we fit on that continuum will depend in the social circumstances and some biological birth gifts (nature and nurture). The oft quoted 1% figure relates to people who meet the definition of a Psychopath (That is. for research purposes, obtaining a score above 30 +-3 on the PCL-R scale), but that doesn’t mean the remaining 99% have no psychopathic traits at all. Further, while psychopathic traits are generally considered bad, it is entirely possible that some degree of psychopathic traits may offer an evolutionary advantage (how much is as yet an unanswered question).
General public focus on whether we can spot psychopaths and therefore predict crime: The media article which caught our attention, focuses on crime prediction, a topic that is fascinating in it’s own right, but one that also causes a great deal of angst, not least amongst privacy campaigners. There are issues with this school of thought, issues which we’ll raise in the talk. Those issue include:
- What we think are a high level of false-positives when examining on an individual basis.
- Psychopathy doesn’t mean an individual will necessarily turn to crime.
- Using social media in such a manner is an unfair invasion of privacy.
- Social media personality research is very much in its infancy.
However, there are likely bigger and more important/practical questions for society, so we should be careful in suggesting that research in this area can only do harm. For instance, if anti-social traits were found to be increasing in certain groups (i.e. The finance industry as this theory posits), can we measure that using social media and other measures? Linked to that is the important question of “should we?”. Why might we want to measure anti-social traits in the first place? Well, if they’re found to be on the rise, why? and is this a bad thing, a good thing or somewhere in between? If it is considered a bad thing, perhaps there is something positive we can do to reduce anti-social behaviour in society. It might, for instance, be possible to educate our children in empathy in addition to mathematics and languages. While many people would baulk at the idea of trying to influence such a social change we should also consider that anti-social personality disorders may be a result of current social conditioning or norms. José Sánchez discusses this very topic in “Social Crises and Psychopathy: Toward a Sociology of the Psychopath” (In the book ‘Unmasking the Psychopath‘ by Reid, Dorr, Walker and Bonner). Although in a different context, intervening with personality to improve the quality of someones life was a topic touched on by Professor B.W.Roberts at the 16th European Conference on Personality Psychology last week. While there are clearly negative implications from the use of social media to measure personality, we should also consider that social media may provide researchers with an excellent opportunity to identify both positive and negative social trends. We believe that understanding anti-social personality traits may be “the most important question you never cared about” (to borrow a quote from Stephen D. Hart).
A possible public perception that detecting personality from social media is infallible: We have a specific concern over the way in which studies focused on personality and social media are portrayed in the popular press; chiefly that press articles may lead people to conclude the social media personality prediction is infallible. With headlines such as “Facebook Profiles Accurately Predict Job Performance” and “Facebook beats personality tests for predicting job success, NIU management professor finds” who can blame them? This type of coverage can make people feel justified in the observations they make about others online, possibly leading to Confirmation Bias and/or an Anchoring Effect, i.e. they put too much emphasis on a single source of information to govern their thought processes. In reality, personality prediction via social media is more complicated than the headlines suggest, but is predicitve performance good enough (that will depend on the context of the use)?
Judgments are abound when it comes to social media and hiring. The 2012 JobVite survey found that 48% of those recruiters consulted always review a candidate’s online profile, and that references to illegal drugs generate the highest negative reaction to a candidate at 78%. This is troublesome not least because there are issues relating to discrimination and there is no consideration as to whether this practice constitutes an unfair invasion of privacy.
Our research
We teamed up with Kaggle.com, Florida Atlantic University and a handful of curious data scientists to explore to what degree it’s possible to determine the ‘Dark Triad’ of Psychopathy, Narcissism and Machiavellianism through Twitter usage.
Kaggle is the global leader in running predictive modeling competitions. “Companies, governments and researchers present datasets and problems – the world’s best data scientists then compete to produce the best solutions. At the end of a competition, the competition host pays prize money in exchange for the intellectual property behind the winning model.”
113 teams submitted 1071 models to examine Psychopathy (as measured by the Short Dark Traid or SD3) in Twitter usage. Tackling machine learning together with Kaggle has demonstrated the value in crowd-sourcing such problems. A small number of the competitors have since worked with us in writing up the results, including Gregory Park who wrote this blog post about his experiences in the competition.
Florida Atlantic Univeristy’s (FAU) ‘Department of Computer and Electrical Engineering and Computer Science Data Mining and Machine Learning and Empirical Software Engineering Lab’ have focused on some specific challenges related to the imbalanced nature of the data set. You can learn more about the work of researchers at FAU in this YouTube video and at this address. After meeting researchers from FAU at DEF CON 19 last year, we’re pleased to have them join us in this years talk.
Where you can see more
We will be presenting our research and results as it applies to Psychopathy at DEF CON, in a talk scheduled for Sunday 29 July in Track 1.
A full paper covering Psychopathy, Narcissism and Machiavellianism will be released roughly one month after the conference.