The Secret Service is on the hunt for software that, among other things, has the “ability to detect sarcasm” on social media. It’s obvious why this would be beneficial: The agency is obligated to investigate any threats made against the president, but it would be useful to sift genuine ones from obviously sarcastic ones on platforms like Twitter. The problem is that this is a very tough thing for computers to do — partly because it’s a very tough thing for humans to do.
In regular speech, humans can rely on subtle cues that someone is being sarcastic, even if they’re sometimes missed — I remember some culture shock from living in Michigan during college and finding that Midwesterners, as wonderful and friendly as they were, were generally a lot less attuned to sarcasm. These cues obviously aren’t present in text, which explains why jokes often don’t translate over SMS or Twitter.
So it’s no surprise that computer scientists haven’t yet been all that successful in training software programs to recognize sarcasm. One study from 2011 (PDF) used tweets that had been specifically hashtagged #sarcasm or #sarcastic, stripped those hashtags, and then dumped them into a virtual pile with a bunch of other straightforwardly positive and negative tweets. At their best performance, the computer programs the researchers used could only correctly separate sarcastic from non-sarcastic tweets about 65 percent of the time — and this was in a rather controlled setting.
Bing Liu, a University of Illinois at Chicago computer scientist who authored a book about sentiment analysis (that is, extracting emotional context from text), expressed skepticism that anyone yet has a good handle on this problem. “I am not aware that anyone has a satisfactory algorithm or system that can detect sarcastic sentences,” he said in an email. And the stuff the Secret Service would be looking at would be a particularly uphill battle: “In discussions about politics [sarcasm] is fairly common and very hard to deal with because it often requires some background knowledge which computers are not good at.”
So good luck to the Secret Service. Not just because sarcasm-detecting software could help keep the president safe, but because once it trickles down to the masses, it will save us all a lot of awkwardness in interpreting weird text messages. How helpful would it be if a little Bender popped up in the corner when your smartphone suspected your friend was just being a sarcastic jerk?