How Russia Polices Yandex, Its Most Popular Search Engine

Google said it would demote Russia Today and Sputnik, but the Russian government has tried to control what news appears on Yandex for years.

|
Nov 24 2017, 4:44pm

Image: Shutterstock

In Russia, when people want to find something on the internet, they often don’t turn to Google or Bing, but to Yandex, a publicly-traded and extremely popular search engine.

“It’s the default search engine for most Russians,” Alexey Kovalev, a Russian journalist who often writes about misinformation and technology, told me over Skype. “It’s also a news aggregator, the biggest in the country. It’s the most coveted real estate for online news media.”

The Russian government has tried to manipulate what news stories appear on Yandex for years, according to Kovalev and Andrei Soldatov, the co-author of The Red Web: The Struggle Between Russia’s Digital Dictators and the New Online Revolutionaries. The fact that the Russian government has meddled with Yandex reads as particularly hypocritical this week, because that same government criticized Google for saying it would demote Russia Today (RT) and Sputnik, two Kremlin-backed news sites, in its search rankings.

Eric Schmidt, executive chairman of Google’s parent company Alphabet, announced last weekend that the company is working to remove Russian propaganda, particularly RT and Sputnik, from Google News after it faced criticism for giving the Kremlin prominent placement on the platform.

Tuesday, the head of Roskomnadzor, Russia’s state media regulator, threatened retaliation against Google for planning to diminish the sites’ reach, according to Interfax, an independent Russian news service. Alexander Zharov wrote a letter to Schmidt asking him to explain his comments. “We will receive an answer and understand what to do next,” he wrote. “It is obvious that we will defend our media.”

Russia is mad Google wants to keep propaganda off Google News, but it has ensured for nearly a decade that state media stays on Yandex.

“The Kremlin has tried to influence what appears on Yandex's front page since the war with Georgia in 2008. Mostly because the front page of Yandex is the main source of news for lots of Russians,” Soldatov told me in an email.

Google has gained market share in Russia in recent years. Google search (and Android) now dominate Russia’s mobile market, and overall, Google and Yandex are about equally popular in search market share, according to Statcounter, an analysis tool that has a tracking script installed on two million websites.

But according to Kovalev, Soldatov, and The Moscow Times, an independent, English-language publication, Yandex.News is much more popular than Google News, and a more important target for the Russian government.

After Russia’s war with Georgia in 2008, two powerful Russian government officials showed up at Yandex’s offices in Moscow, Soldatov said. They wanted access to the algorithms Yandex used to determine what news items it surfaced. Unlike Google, which has a separate news page, Yandex displays five news stories prominently on the Russian version of its front page.

“The company resisted, and instead of granting access, it gave the Kremlin officials a phone number to call and voice their concerns,” Soldatov told me. “For years [the Kremlin] called Yandex and complained. And for years the answer was a polite explanation that nothing could be done, as it was all about the algorithms.”

But over time, the situation changed, and the Russian government found new ways to meddle with the news Yandex promoted. “Because they cannot control Yandex directly, they can control online media,” Kovalev said.

This year, the “news aggregator law” came into effect in Russia. It requires websites that publish links to news stories with over one million daily users (Yandex.News has over six million daily users) to be responsible for all the content on their platform, which is an enormous responsibility.

“Our Yandex.News team has been actively working to retain a high quality service for our users following new regulations that impacted our service this past year,” Yandex told Motherboard in a statement, adding that to comply with new regulations, it reduced the number of sources that it aggregated from 7,000 to 1,000, which have “official media licenses.”

“For now, Google News has avoided Roskomnadzor’s restrictions because it is less popular,” The Moscow Times wrote in April soon after the law went into effect.

In the US, there’s no such law. Social media companies are protected from the legal consequences of users posting harmful material by laws like the Digital Millennium Copyright Act’s “safe harbor” provision and the First Amendment.

Aggregators in Russia can bypass the regulations of the news aggregator law by only sourcing their material from media outlets approved by the Roskomnadzor. In essence, Yandex has undergone a purge.

Read More: ‘Russia’s Google’ Is Collecting a Database of Rare Languages

Many sites with state licenses are Kremlin-owned local newspapers that essentially publish the same stories in unison, tricking Yandex’s algorithm into thinking news outlets are overwhelmingly reporting a specific pro-Kremlin story. “When it wants to manipulate the news agenda, [the Kremlin] commands all these newspapers release one news article praising the mayor for his glorious achievements, for example,” Kovalev said.

Yandex has tried to diminish the amount of influence that Kremlin-owned sites have on its news aggregator, but it’s an uphill battle. “They did downgrade these identical websites in their algorithm but the problem is that there is just too many of them,” Kovalev told me.

The news aggregator law is having a real effect on what gets surfaced on Yandex. In March, anti-corruption demonstrations broke out across the country. Thousands of protesters poured into Moscow’s streets. It was arguably the biggest story Russia had witnessed in months.

But if you checked Yandex, you wouldn’t know about it. The search engine briefly surfaced a story about the demonstrations, but it soon disappeared, according to The Moscow Times.

The day the protests happened, “Yandex displayed on its front page some news about a flower festival in Moscow,” Soldatov said.

Update: This piece has been updated with comment from Yandex.

Correction: An earlier version of this piece misspelled Andrei Soldatov's name as Solatov. The piece has been updated.

Got a tip? You can contact this reporter securely on Signal at +1 201-316-6981, or by email at louise.matsakis@vice.com

Get six of our favorite Motherboard stories every day by signing up for our newsletter.