We examine the first large real-world data set on personal knowledge question's security and memorability from their deployment at Google. Our analysis confirms that secret questions generally offer a security level that is far lower than user-chosen passwords. It turns out to be even lower than proxies such as the real distribution of surnames in the population would indicate. Surprisingly, we found that a significant cause of this insecurity is that users often don't answer truthfully. A user survey we conducted revealed that a significant fraction of users (37%) who admitted to providing fake answers did so in an attempt to make them "harder to guess" although on aggregate this behavior had the opposite effect as people "harden" their answers in a predictable way. On the usability side, we show that secret answers have surprisingly poor memorability despite the assumption that reliability motivates their continued deployment. From millions of account recovery attempts we observed a significant fraction of users (e.g 40% of our English-speaking US users) were unable to recall their answers when needed. This is lower than the success rate of alternative recovery mechanisms such as SMS reset codes (over 80%). Comparing question strength and memorability reveals that the questions that are potentially the most secure (e.g what is your first phone number) are also the ones with the worst memorability. We conclude that it appears next to impossible to find secret questions that are both secure and memorable. Secret questions continue have some use when combined with other signals, but they should not be used alone and best practice should favor more reliable alternatives.
Secrets, lies, and account recovery: lessons from the use of personal knowledge questions at google
Available Media
Conference
22nd international conference on World Wide Web 2015
Authors
Joseph Bonneau , Elie Bursztein , Ilan Caron , Rob Jackson , Mike Williamson
Award Best Student Paper Award
Citation
Selected Press articles
Huffington Post | Damon Beres | May 2015
Your Password Security Questions Are Terrible, And They’re Not Fooling AnyoneTime | Victor Luckerson | May 2015
Stop Using This Painfully Obvious Answer For Your Security QuestionsABC News | Alyssa Newcomb | May 2015
Google Reveals the Problem With Password Security QuestionsCBS News | Michael Casey | May 2015
Those secret security answers may not be so secureTech Crunch | Frederic Lardinois | May 2015
Google Study Shows Security Questions Aren’t All That Secure