Washington (CNN) — In a tweet on Monday, President Donald Trump made a sensational allegation about Google.
“Wow, Report Just Out! Google manipulated from 2.6 million to 16 million votes for Hillary Clinton in 2016 Election! This was put out by a Clinton supporter, not a Trump Supporter! Google should be sued. My victory was even bigger than thought! @JudicialWatch,” Trump wrote.
But Trump did not describe the research correctly. And the research itself has been called into question.
Facts First: Epstein himself says Trump was wrong about his findings. Epstein did find “bias” in Google search results, but he says there is no evidence Google “manipulated” the results to favor Clinton. Also, critics of the study note that there is no definitive link between search results and voting behavior in presidential elections.
First let’s address what Epstein says Trump got wrong. Then we’ll delve into Epstein’s research.
Trump’s words and numbers were inaccurate
In an interview with CNN on Monday, Epstein said the pro-Clinton bias was “sufficient to have shifted between 2.6 and 10.4 million votes” to Clinton.
There is no basis in Epstein’s research for Trump’s claim that the alleged bias might have affected “16 million” votes. Epstein did testify in July that big tech companies in general could potentially shift “upwards of 15 million votes” in the 2020 election, but he didn’t claim that this happened in 2016.
In the Monday interview, Epstein rejected Trump’s claim that Google “manipulated” votes in 2016. He said he does not have firm evidence even that Google intentionally manipulated its search algorithm or results, let alone votes themselves.
“I don’t have any evidence that Google manipulated anything. I just have evidence that there was this bias — highly statistically significantly bias,” he said.
Google said Epstein was incorrect in his claims of bias.
“This researcher’s inaccurate claim has been debunked since it was made in 2016. As we stated then, we have never re-ranked or altered search results to manipulate political sentiment. Our goal is to always provide people with access to high quality, relevant information for their queries, without regard to political viewpoint,” the company said in an email.
(An aside: Judicial Watch, a conservative legal activist group that is active on elections issues, was not involved in Epstein’s research. The group told CNN that it believed Trump tagged it in the tweet to encourage it to look into the allegations.)
How Epstein determined there was bias
For this study, he had 95 people from 24 states, including 21 self-described undecided voters, conduct election-related searches using search engines Google, Yahoo and Bing. Then he had another group of Americans, hired through the crowdsourcing website Amazon Mechanical Turk, use a point scale to rate the supposed bias of the articles found on the first page of the search results.
An extremely pro-Trump article would get a minus-5, while an extremely pro-Clinton article would get a plus-5.
Using this method, he found that Google’s results were reliably more pro-Clinton, in both red states and blue states, than Yahoo or Bing results. Then, using his previous research from elections in other countries about how search results can affect voter intentions, he came to a broad estimate of 2.6 million to 10.4 million votes potentially affected by search bias in the US in 2016.
“I don’t even care about the human element. There’s a system out there running amok,” he said.
Criticism of the data
Other academics have joined Google in criticizing Epstein’s methodology and conclusions.
One issue is the quality of news sources.
Google says one of its criteria for ranking news results is how authoritative a source is. Using Epstein’s methodology, a search engine whose top results page did not feature an article from a far-right pro-Trump website, such as Breitbart, would be ranked as more biased in favor of Clinton than a search engine that did showcase Breitbart — even if the first search engine highlighted a deeply informative Washington Post investigation about Trump’s past and the second search engine highlighted Breitbart’s pro-Trump puff piece.
Epstein emphasized that he is not a Trump fan. But he argued that Google results should not be treating pro-Trump media as second-class “for whatever reason, whatever your excuse.”
“I’ve certainly met people that say Breitbart is a more reliable news source than the New York Times … there’s a lot of subjectivity that goes into these kinds of determinations,” he said.
Another issue, other academics say, is that Epstein’s study did not establish a link between alleged bias in search results and voter behavior in 2016.
Epstein said he came to the conclusion of bias sufficient to affect 2.6 million to 10.4 million votes based on what he has found in studies of national elections outside the US, including the 2010 Australian prime minister election and a 2014 Indian legislative election.
In other words: Epstein did not test 2016 American voters to see if their Clinton-or-Trump choice had been changed by search results they got. He extrapolated from his previous studies.
McDonald said it is certainly possible that political results from Google and other search engines have been affected by the unconscious biases of the people who wrote their algorithms. But McDonald said Epstein has failed to establish that any such biases have had anywhere near the magnitude of impact on American presidential voting that Epstein suggests.
“It’s just not plausible,” McDonald said.
For example, a strongly anti-abortion voter might be more likely to have their vote affected by abortion-related search results than results about another subject. But Epstein’s analysis did not distinguish between voters’ interest levels in different topics.
“You can’t zero in on Google and have Google be your only factor in your analysis to shape one’s voting outcomes,” he said.
Srinivasan also said that the political value systems of the people who create search algorithms might affect the results.
He said, “Robert and I agree on a lot.” But he said Epstein’s analysis on search results and voting is overly simplistic.