“Recommended for you”: the fight to decide what you read next

Content engines are a breed of company that few have heard of but nearly everyone as used– typically by clicking on a story in the “read next” or “Recommended for you” boxes that are springing up around the web. The companies, such as Outbrain and Taboola, are flush with tens of millions in investor money and are in a growing battle with each other for space on publishers’ pages.

While content engines have been around for a while, their growing presence is influencing how readers explore the web. They are also taking on a new importance as vanguards of “native advertising,” a trend that many hope will reinvigorate the online ad economy.

Here’s an overview of the content engine industry and what it means for publishers, advertisers and readers.

Keeping readers close to home

New York magazine uses a variety of tactics to increase visits to its website and to induce readers to stay on the site. These include boxes showing its most-popular stories as well as cross-promotion arrangements with other news sites. New York has also recently signed on with content engine nRelate, which uses algorithms to display other stories a reader may find interesting; the stories come from New York’s own site or one of its affiliates.

nRelate promises publishers more pageviews but also a new revenue stream as a result of “sponsored stories” it inserts in the list of recommended content – if a reader clicks on the advertiser’s “story” (typically a promotional article written like a news story) the publisher and nRelate share the money.

Michael Silberman, the GM of digital media at New York, says he welcomes the chance for extra revenue but that the main purpose of the nRelate experiment is to get readers to stay on the site. And, like so much in publishing these days, the outcome will be data driven. “We’ll do A/B testing to see if there’s an overall lift. If all it does is redirect traffic we would have had anyways, it’s not worth it. If it leads to more pageviews per visit, then yes,” said Silberman, adding that publishers have to take care that tools like nRelate don’t clutter or slow down the site.

Tools like nRelate promise more money or traffic but they also represent a growing strategic importance for publishers. The reason is that, in age of the social media, fewer and fewer readers arrive at a website directly through a publisher’s home page; this means that publishers are doing more than ever to persuade them to stick around.

“We find [Outbrain] quite useful as a tool to drive traffic to our sites and circulate traffic through our sites. The numbers suggest that our users Breaking Media screenshot find it useful as well,” said John Lerner, CEO of Breaking Media, which owns sites like Above the Law. (see screenshot at right). He did not provide specific numbers.

Meanwhile, the content engines are also a source of money. Joshua Albertson, the President of Curbed Media, says in a good year, they can bring in six figures of revenue through its sites Curbed, Eater and Racked.

While tools like nRelate hold promise for publishers, they’re also being pitched as a way for advertisers to break through to readers who tune out conventional online ads. The idea is that, by packaging the ad as a story to “read next,” a reader is far more likely to click on it and digest the marketing message.

This ad format – call it “native advertising” or “content marketing” or whatever you wish – has produced a growing list of companies that want to serve up sponsored stories. The biggest of these is Outbrain, which has already received $ 64 million in funding and claims to be on more than 90,000 blogs and websites like Slate, whose editor, David Plotz, says he has been pleased with the three-year partnership.

“We’ve been really happy with it. We needed to provide related links and Outbrain combines it with a tool that makes quite a lot of money for us.”

Outbrain’s hold on the market is far from firm, however, as competitors have been peeling away some of its clients. According to Curbed’s Albertson, the company decided to switch after it found Outbrain’s performance declining and rival Taboola offered it a bigger share of advertising revenue. Here is a screenshot of Taboola, which serves both stories and video and recently received $ 15 million in new funding:

Taboola screenshot

As the rivals move in, Outbrain has begun slapping its logo on its boxes of recommended stories in an effort to be seen as a brand. It also recently fired a quarter of its advertising clients because of the poor quality of their story content. The company explained the decision this way:

“If companies coming into this space are not respectful of the audience who are clicking on these links, then the user trust will decline over time. That means this space — which is currently so valuable to the audience and publisher — will be just like the the display ad market. People will become ‘content link blind,’ just like they became ‘banner blind.’”

Meanwhile, other companies are pivoting from their core business to get into the content engine game too. These include dictionary company Reverb, which believes its experience with syntax will make it good at finding content, and the commenting platform, Disqus.

In an interview, Disqus CEO Daniel Ha said the company has an advantage due to its existing relationship with thousands of publishers, and the ability to offer brands a way to reach hyper-engaged comment communities. Ha added that the Disqus strategy is based on the “independent web” and “the middle-tail of websites,” pointing to examples like a Toronto Maple Leafs fan forum and coupon site Southern Savers.

A boom or a bubble?

Content engines may be a hot topic but is there enough money and web real estate to go around? Neil Mody, the CEO of nRelate, says there is not. “There’s a classic network effect. You’ll get one big winner. Numbers two through five will do OK. Numbers 6 thru 20 will go niche or fail.”

Mody says there is already evidence of a coming shakeout. He claims some content engines are paying publishers to get on their sites even though they have no ad revenue. He also says the industry is over-capitalized and that there is risk of a backlash as some content engines flood the space with junky “lose your belly” type of articles.

Another challenge for content engines is the low cost of switching for publishers. The engines are built so that it’s easy for publishers to put them up, but also take them down — a situation that doesn’t favor long-term relationships. The result is that publishers can simply sign up with whomever offers the best content or most money. This means the winner is likely to be content engine with the best algorithms and the highest quality advertisements.