Study That Claimed AI Could Determine a Person’s Sexuality Is Under Ethical Review

A pre-released study to be published in the Journal of Personality and Social Psychology went viral last week. First reported by The Economist, It found that a machine learning algorithm could tell from a handful of photos whether a white person identified as gay or straight on a dating website with startlingly accuracy. The work, conducted by Michal Kosinski and Yilun Wang from Stanford University, received immediate and widespread criticism from other academics and LGBTQ activists.

According to The Outline, the paper is now being reexamined by the journal it was supposed to be published in, though it's not clear for what reason. "An ethical review is underway right at this moment," Shinobu Kitayama, the editor of the Journal of Personality and Social Psychology, told The Outline on Monday. Kitayama did not return Motherboard's request for comment. Kosinski also did not immediately reply to a request for comment, nor did Stanford University.

Academics, independent experts, and LGBTQ activists from organizations like GLAAD and the Human Rights Campaign argue that not only is the study's methodology flawed, but that it raises serious ethical concerns. The study's sample of photos were collected from a dating site without participants' consent, and the paper did not include people of color, nor bisexual or transgender individuals. It also ignored cultural and societal factors that contribute to human sexuality. Ultimately, the researchers appeared to base their work on the assumption that the contours of a person's face are fixed, rather than easily manipulated by makeup.

"Technology cannot identify someone's sexual orientation. What their technology can recognize is a pattern that found a small subset of out white gay and lesbian people on dating sites who look similar. Those two findings should not be conflated," Jim Halloran, GLAAD's chief digital officer, said in a joint statement issued by both his organization and the Human Rights Campaign. Their statement echoed the concerns of Sarah Jamie Lewis, a cybersecurity researcher who studies privacy and specializes in privacy issues affecting the LGBTQ community.

"All this paper does is reinforce stereotypes and categories that the queer community is fighting so hard to break," Lewis told me over Twitter direct message when I first wrote about the study. "But even if we accepted the paper's premise that someone can appear visually queer then the paper still has major ethical issues around participant consent and the overall aim of the research," she explained, referring to how the photos used were pulled from a publicly available dating site instead of gathered with the participants' consent.

In a blog post, Greggor Mattson, an associate professor of sociology at Oberlin College, argues that the paper is only the "most recent example of discredited studies attempting to determine the truth of sexual orientation in the body."

He also wrote that paper completely ignored "contributions from the fields of sociology, cultural anthropology, feminism or LGBT studies." Mattson goes on to systematically criticize Wang and Kosinski's methodology, the biases in their work, and criticizes what he sees as their shallow understanding of cultural and societal factors associated with human sexuality.

Mattson also noted that Kosinski, one of the researchers behind the paper, is reportedly an advisor to Faception, a controversial security firm that claims its machine learning technology analyzes "facial images and automatically reveals personalities in real-time."

In another blog post, Philip N. Cohen, a professor sociology at the University of Maryland, writes that the paper has numerous methodological problems. He writes that the study's "dependent variable is poor defined, and then conclusions from studying it are generalized beyond the bounds of the research." Ultimately, he argues that they misinterpreted their own findings.

On Twitter, Kosinski has pushed back against critics of the study. In a Google Doc last updated on September 11, he and his co-author Wang wrote that the Human Rights Council and GLAAD are "engaged in a smear campaign" against them, and that the concerns they raised "were incorrect, misleading, lack merit, or were clearly addressed in our work."

GLAAD and Human Rights Council said in their statement that the paper was not peer-reviewed. As Kosinski and Wang point out in their rebuttal, it in fact was. But the peer-review system is full of problems, and doesn't guarantee that a published paper is scientifically sound.

"It's just a very flawed system," Cohen wrote in his blog post. At least now the Journal of Personality and Social Psychology is taking a second look.