The other thing you must know before starting an aggregator

by Jason Preston on September 25, 2008

The other day I wrote a post about what you need to know before launching an aggregator, and Tracy from West Seattle Blog chimed in with a comment reminding me about what I should have put in that post: don’t scrape.

Or as bikehugger also put it: “don’t rip off bloggers.”

There is a fine line between aggregation and scraping, and unfortunately it can be a little bit like “adult cinema” – you know it when you see it. This is scraping. This is not.

So what’s the big difference?


As Tracy pointed out in her comment, a big part of it is how content shows up in search results. An obvious no-no is pulling entire posts from a blog and republishing them on your site without obvious attribution.

A less obvious no-no is formatting your page meta so that when your pages turn up in search results, it appears as though you are carrying original content.

Not giving a blogger attribution is a little bit like not paying the AP—the blogger doesn’t want you to cut them a check (in the traditional sense), they want payment in the form of traffic and prestige.

Every inbound link improves their traffic and their online profile (meaning some mushy aggregate of page rank, alexa rank, search engine result placement, etc), which are two of the most important things for a blogger looking to build their brand.

You get the content for free…why steal the brand??


A good way to make sure that what you’re building is based around aggregation instead of scraping and spamming is to have the right intentions when you’re building it.

Build an aggregator to make it easy for your readers to find cool and interesting content elsewhere on the web.

As long as your intention is to funnel people’s attention to the web sites that deserve it, and you build your technology to facilitate that process, you should be in the clear.

Jason writes about the future of publishing. To learn more, subscribe to the Eat Sleep Publish RSS feed.

{ 1 trackback }

Piggyback authority — Eat Sleep Publish
10.15.08 at 9:16 am


1 Tracy @ WSB 09.25.08 at 9:35 am

Bless you.

2 Godius 09.26.08 at 1:40 am

Great article, keep it up. I found you guys via the new website indexing service over at

Comments on this entry are closed.