TL;DR: OKCupid’s study on male dating photos fails reproducibility
If you’re a guy who uses online dating sites/apps, you’ve probably heard this one: don’t smile in your picture. Better yet, don’t smile and look away from the camera.
This tip originated on the OkCupid’s OkTrends blog in January 2010. The post was called The 4 Big Myths of Profile Pictures.
Growing skepticism
Since OkCupid published their data in support of not smiling in pics, the tip has been quoted as gospel truth on dating advice blogs, PUA podcasts, dozens of dating subreddits, forums, everywhere.
But, at the same time, widespread skepticism rose too. One Reddit commenter put it this way:
“Despite what OkCupid may say, I believe that in Tinder’s last roundup of the top 100 male profiles, basically all of them were smiling in their lead picture.”
Further — everywhere men floated the “don’t smile in pictures” advice, many women decried how much they hated these nonsmiling pictures. But, the men countered, women don’t necessarily know what they want. After all, OkCupid’s findings were based on behavior, not just talk, right?
Photofeeler steps in
This is where we at Photofeeler come in. Like everyone else, we believed in OkCupid’s conclusions. But the more data we collected about men’s dating photo attractiveness, the more it became undeniable: OkCupid’s advice wasn’t raising men’s photo scores.
At first, this realization was frustrating. We even worried our tool could be flawed. But every time we looked into this, we found the same thing: daters who used Photofeeler for photo testing were getting right-swipes like never before. In fact, users reported 3-5x (200-400%) more matches! So the opinions on our site were translating directly into behavior.
We decided it was time for someone to challenge the OkCupid study.
Using the massive stores of data on our platform, we set out to reproduce OkCupid’s process (as laid out by the Myths of Profile Pictures post). We narrowed the demographics of our data set accordingly, matching their 7,140-photo sample. Then we ran each picture through a variety of analysis scripts (in our case, neural nets that detected smiles and eye contact) as well as tagged each one by hand until total agreement was reached. Finally, we used Photofeeler attractiveness ratings to gauge the success of the various photo types (smiling, not smiling, eye contact, no eye contact).
What We Found (Results/Findings)
Here is OkCupid’s study results vs. our own:
OkCupid’s data said that not smiling and not making eye contact was better.
Ours says that whether you smile or not makes no statistically-significant difference (except in the case of eye contact and no smile, which is harmful).
Here are some possible reasons behind the differing results.
I. Evidence suggests OkCupid manipulated their data set to get a click-worthy result
Let’s be real here: research can be a racket. Back in 2010, no one would publish “smiling works great in dating photos!!1” because most people assumed that already. Affirming what people already know is boring. A counter-intuitive result (e.g. “smiles are actually unattractive!”) was far more sensational.
Most people think data ought to speak for itself. But data can be manipulated. To put it frankly, data can be manipulated to show practically any result that the scientist would like it to. And in OkCupid’s case, it’s reasonable to assume that they got the interesting result they wanted, in part, by cutting out particular populations from their data set.
OkCupid used a sample of 7,140 photographs from users aged 18-32, in big cities, possessing average attractiveness (that is, they lopped off the top and bottom 20%), and who had profiles containing only one photo and no text. Then “after a bit more sifting” (!), they landed on their study’s data set.
Why did OkCupid eliminate users outside of the ages of 18 and 32? Why did they eliminate users who were most and least attractive? The explanation given (that they “[feared it] would skew [their] results”) is no explanation at all. They didn’t have to “fear” anything because, in all likelihood, they first ran their numbers with these populations included. They just didn’t get as interesting of a result that way.
The over-sifting of the data set likely obscured other trends that were more responsible for profile success than the photo characteristics the study claimed to be measuring. For instance, it’s possible that the men in the top 20% of attractiveness were attractive and smiling, and the men in the bottom 20% of attractiveness were unattractive and not smiling. This leaves only smiling ugly guys at the bottom of the spectrum and grumpy hot guys at the top, making it look like being grumpy makes you hot.
II. OkCupid had far too little data to draw conclusions
Building on the previous point, there’s the question of how many pictures of men not smiling and not making eye contact were in the data set to begin with.
Giving OkCupid the benefit of the doubt, let’s say their sample was 50/50 male and female (even though it would likelier have skewed female). 7,140 photos is only about 3,570 per gender.
This is a good sample if you’re measuring a condition that will be present in all of the photos. But the number of men who were not smiling and looking away (especially in early 2010, before OkCupid advised it) would be in the hundreds at most. Even today, less than 15% of photos have no eye contact. Before OkCupid declared it superior, it was likely 5-10% (200-300 photos split into 3 groups: smiling/not/flirty).
We know for certain that OkCupid knowingly made claims based on too little data because they had approximately 7 photos of male âflirty faceâ with no eye contact and they still drew conclusions about its effectiveness.
Not to mention, the metric they were using to gauge a male dater’s profile effectiveness (“women met per attempt”) is a wildly varying and unbounded metric; one guy with a particularly interesting photo that gets one unsolicited message per day could have easily made their whole result.
III. OkCupid had a biased population sample
Trends change. In data science, we know it can be difficult to find consistent trends even between visitors of the same website from one week to the next.
Is it likely that trends found among a very specific niche of male daters long ago — those who chose to upload only one photo and no profile text to OkCupid in 2009 — could translate to a viable Tinder strategy for all men in 2017?
The truth is that societal and dating norms have changed a lot in this amount of time. For instance, dating online used to be taboo. So perhaps giving an air of “I’m too good for this” with a non-smiling, looking away photo and no profile text appealed slightly more to women at the time.
Further, what kind of man uploads one photo and no profile text? Perhaps it’s this personality type (not the photo’s characteristics out of context) that speak to which photo strategies worked best for him.
IV. The OkCupid study itself spawned a new class of bad photos
Finally, let’s chat for a moment about what happens when a highly popular dating site disseminates misinformation about what works best in guy’s dating pics: a new class of male dating photos is born.
We hypothesize that the publicity of OkCupid’s results gave rise to a new type of dating photo within which the subject is purposefully avoiding smiling and eye contact, in which the subject seems to be awkwardly looking in another direction for no apparent reason. Internally, we labeled these photos as “avoidant” because they tend to come across to strangers as if the subject is too timid to make eye contact.
Fact is, these avoidant photos just didn’t exist before the 2010 OkCupid study. Before this advice was made popular by OkCupid, the only instances of not smiling and not making eye contact were at least somewhat contextually relevant. After: not so much.
Conclusions
OKCupid’s study on male dating photos did not pass the reproducibility test in 2017.
Our data suggests that there is no one-size-fits-all photo strategy for all men. Rather, itâs important to determine whatâs appropriate and natural in context, and to consider what characteristics you individually have to offer a mate.
Truthfully, even if a particular photo strategy showed a slight difference in average effectiveness, the individual photos score all over the map.
That’s because certain strategies may work better for certain people or in certain contexts. If you want to know what works: try it, test it.
Photofeeler is a tool for testing profile pics, as seen in Time, Forbes, The Today Show, and more. Know for certain how youâre coming across in your dating pictures. Itâs free to use here.