« Girls (and Boys) Gone Wild: Commitment and Infidelity during Spring Break | Main | Five Risk Factors for Divorce »

Evidence to Support a Valid Online Dating Matching Algorithm: My Wish List

By Dr. Paul Eastwick, in collaboration with Drs. Eli Finkel, Ben Karney, Harry Reis, & Sue Sprecher*

*The latter four authors are listed alphabetically

For centuries, entrepreneurial types have claimed to possess the secrets of romantic attraction, promising that their charms, potions, or drugs are the solution to the search for a perfect mate. With the rise and now ubiquity of the Internet, those offers have moved online and become increasingly sophisticated, but the promise remains the same.

Now, the potions come in the form of mathematical matching algorithms, a new alchemy that takes the basic elements of people—their attitudes, values, and personalities—and promises to transform them into golden dyads.

As part of a larger piece about online dating, my colleagues and I have expressed serious doubts that these algorithms are effective. To the extent that we can ascertain the workings of these mysterious algorithms (which have not been disclosed to the scientific community), they appear to rely on principles (e.g., similarity and complementarity) that have very small effects on relationship well-being. At the same time, the algorithms completely ignore principles that have large effects on relationship well-being (e.g., stress and interaction quality).

(If you are interested in reading more about the problems with these algorithms, click here for a NYT editorial by Finkel and Karney.)

Here, we wish to describe precisely what would constitute compelling evidence that a matching algorithm is indeed effective. That is, what would it take to convince skeptics like us that the use of an algorithm improves people’s chances of forming and maintaining a satisfying relationship relative to traditional offline ways of meeting a romantic partner?

The tool that would illuminate the modern-day love drug is actually the very same tool used to evaluate the efficacy of all drugs and interventions: randomized experiments. Below, we describe a set of hypothetical experimental conditions that would provide compelling evidence in support of a fictitious site’s matching algorithm (ePerfectChemistry.com).

In our hypothetical scenario, single adults volunteer for a study. All of them complete the ePerfectChemistry.com matching algorithm questionnaire. The participants in the algorithm condition periodically receive lists of matches produced by the ePerfectChemistry.com algorithm. In this condition, participants are like true users of modern-day matching sites. As time progresses, we track the relationships that the participants form with their matches: Do they enjoy their dates, do they form a relationship, do their relationships last? For the sake of discussion, imagine that ePerfectChemistry.com couples have wonderful, lasting relationships, and that they rarely get divorced.

This is not enough to excite a skeptic. Rather, the algorithm condition must produce better, longer-lasting relationships than three distinct control conditions:

1. A wait-list control condition

In the real world, self-selection effects are a major impediment when researchers wish to draw causal conclusions. For example, one reason that ePerfectChemistry.com users form lasting relationships could be that the users who gravitate to ePerfectChemistry.com are especially motivated to find a serious relationship. In fact, there is an endless list of possible differences between people who self-select into a site like ePerfectChemistry.com and people who do not. 

The solution to this problem is simple: Some participants who sign up for the study are randomly assigned to the algorithm condition, whereas others are randomly assigned to a wait-list control group. This means that they can join ePerfectChemistry.com only after the study has concluded. In the meantime, they presumably date through other traditional means. If the algorithm condition produces benefits above and beyond the offline dating activities of individuals in this wait-list control condition, it suggests that self-selection effects cannot account for the success of the matching algorithm.

2. A placebo control condition

It is also possible that ePerfectChemistry.com users have great relationship success because they are told that they are especially compatible with their matches by experts who claim to wield the power of science and mathematics. That is, increased relationship success could arise through expectancy effects; sometimes when people think they are receiving an efficacious drug, for example, they experience the drug’s “effects” even if it is an inert placebo. Thus, in this condition, participants receive lists of matches from ePerfectChemistry.com, but these matches are actually generated randomly (i.e., not using the matching algorithm). If the algorithm condition produces benefits above and beyond this condition, it suggests that expectancy effects cannot account for the success of the matching algorithm. 

3. A high-relationship-aptitude control condition

To understand this control condition, we must distinguish between two things that a matching algorithm can calculate. The first is that an algorithm could determine who in a dating pool has poor relationship aptitude (e.g., high neuroticism, low self-control, sociopathy). If your responses to the ePerfectChemistry.com questionnaire indicate that you are unlikely to provide others with a satisfying relationship, ePerfectChemistry.com can tell you that they do not have any matches for you and eliminate you from the dating pool. Sorry!

If you are fortunate enough to pass this screening, you might find that the potential partners at ePerfectChemistry.com are more delightful than partners in the general population. But ePerfectChemistry.com promises more than that. They actually promise to use the principles of similarity and complementarity to pair you with someone who is uniquely compatible with you (and not with others).

Thus, in this condition, participants would receive matches from ePerfectChemistry.com, but these matches would be prescreened only for individual differences in relationship aptitude, not dyadic compatibility. If the algorithm condition produces benefits above and beyond any benefits observed in this condition, it suggests that ePerfectChemistry.com couples are especially compatible with each other, not that ePerfectChemistry.com eliminates undesirable people from the dating pool.

There is one massive barrier that prevents ePerfectChemistry.com from conducting and publishing this study: In order for ePerfectChemistry.com to publish the results of the second and third control conditions, they would have to reveal the details of their proprietary matching algorithm. As long as sites can market themselves effectively by claiming to have a scientific foundation, such peer-review will never happen. Dr. Art Aron offers part of a solution: Have scientific organizations appoint a panel of experts to evaluate the scientific credibility of online dating sites. This is very similar to the way that the FDA approves drugs, and it would even allow some proprietary information to be kept confidential. It would give sites a chance to market their actual scientific proficiencies in lieu of inventing them.

We remain hopeful that matching sites will ultimately rise to the challenge, both by conducting and publishing the appropriate studies and by drawing upon current relationship science to improve their methods. The readers of Science of Relationships should be disappointed until those with the resources to make an actual difference in people’s romantic lives demonstrate some bona fide vision.

Until then, we recommend that consumers be wary of promises on the bottles of Internet love potions.

Interested in learning more about relationships? Click here for other topics on the Science Of Relationships. Like us on Facebook to get our articles delivered directly to your NewsFeed.

Finkel, E. J., Eastwick, P. W., Karney, B. R., Reis, H. T., & Sprecher, S. (in press). Online dating: A critical analysis from the perspective of psychological science. Psychological Science in the Public Interest. doi: 10.1177/1529100612436522

Dr. Paul Eastwick - Website/CV
Dr. Eastwick is an Assistant Professor in the Psychology Department at Texas A&M University. His research has explored sex differences in initial attraction processes, the theories people hold about their romantic lives, and the importance of attachment in early relationship development. He is also interested in the intersection of race and romantic attraction, the potential for anthropology and archaeology to inform evolutionary psychology, and the use of both (a) speed-dating and (b) virtual environments to test social psychological hypotheses.

image sources: futurity.org/technorati.com Related Posts Plugin for WordPress, Blogger...

PrintView Printer Friendly Version

EmailEmail Article to Friend

Reader Comments (4)

Excellent article

April 4, 2012 | Unregistered CommenterOnline Dating

In fairness, more and more online dating site are using matching tools. And on the other hand, it is up to the single person how he/she would use the site in finding his/her soulmate or to empower his/her relationship.

May 26, 2012 | Unregistered CommenterLove TV

But how are consumers to know that matching algorithms actually work? Until dating sites provide scientific evidence (along the lines of what Dr. Eastwick and colleagues call for above), there's no way to evaluate the effectiveness of these tools.

Interesting article. I learnt something.
I'm wondering about the 3 control conditions though. For #2, I'm wondering if maybe there needs to be a separate test to help design the placebo. Just like placebo pills still need to be round and look correct, a separate rudimentary algorithm might be necessary just for the placebo matches so they don't arouse suspicion after participants completed a lengthy questionaire but are then bombarded with 'bad' matches.

For #3, I'm reminded of getting rejected by eharmony, only to get 'accepted' after sitting their test again at a later time. This was probably part practicality on their part (don't serve tough customers), clever marketing to appear more exclusive, and maybe financial incentives at play. Whether or not I really passed the screening the 2nd time round, I'll never know. ;)

I've become cynical about dating sites though. Find me a good algorithm to get people to interact more for a start, and then meet "IRL" to see if their date's 3d picture, size, smell, and sound of their voice is attractive.

June 2, 2013 | Unregistered CommenterBob123
Editor Permission Required
Sorry, due to the amount of spam we receive, commenting has been disabled for visitors of this site. Please see our Facebook page for comments on recent articles posted.