…what we ended up finding out was that the human judges actually performed pretty much at chance at detecting deception. And this is not particularly surprising; in fact, a lot of previous deception detection work has suggested that humans are actually very bad at detecting deception, and even when trained they don’t necessarily do any better. They might become more skeptical, but they won’t necessarily perform any better overall.
[Our software] starts with a sort of blank slate and tries to find, given these sets of reviews, what words and what features are sort of common to the deceptive reviews and likewise what are common to the truthful reviews. And it assigns in fact a weight to each of these features so that we can later analyze and determine what the most important features in the deceptive reviews and what the most important features in the truthful reviews are.