In his this morning's rant on RSS,
Dave mentions polling frequency for aggregators and the usual pull vs.
push issue. My opinion as an application developer is that the beauty
of RSS is that it's not
instant. Current aggregator users simply don't expect a post to appear
in their aggregators instantly. Having to wait on average 30 minutes
(considering the traditional hourly scan) is a perfectly acceptable
delay; it might be different on intranets but usually intranets don't
have bandwidth limitations.
Developing an application which does not require instant feedback allows to scale it in a much nicer way: backend agents can be set up to routinely perform tasks, update pages, find relations, ping profiles and do all kind of activities without any hurry: we always have an hour to go.
For our own k-collector aggregator we took a slightly different approach to the hourly scan: all feeds are polled within one hour, but not all of them are scanned at the same time. The aggregation activity is dynamically scattered in the whole hour, granting prompt updates but preventing the usual processor and bandwidth spikes triggered by hourly scans (active users with a Radio or a MovableType client ping the server when they update, meaning that their posts get on the site quicker).
I do understand why somebody would want to make RSS as quick as email or instant messaging but it would need radical rethinking of the current approach and infrastructure, for example downloading RSS using something like bit torrent, spreading the overall bandwidth usage on clients and not only on servers.
PS: I do like the name RSS... but since I'm a nerdy user who has been using this name for years my opinion doesn't count much. Compared to HTTP, "html" or "tcp/ip", RSS seems a reasonably easy name for everybody to use and remember. Marketing wise I would probably change the orange XML icon from "XML" to \"RSS" (I understand why it reads XML, but it's probably a little hard to explain to the everyday user).
Developing an application which does not require instant feedback allows to scale it in a much nicer way: backend agents can be set up to routinely perform tasks, update pages, find relations, ping profiles and do all kind of activities without any hurry: we always have an hour to go.
For our own k-collector aggregator we took a slightly different approach to the hourly scan: all feeds are polled within one hour, but not all of them are scanned at the same time. The aggregation activity is dynamically scattered in the whole hour, granting prompt updates but preventing the usual processor and bandwidth spikes triggered by hourly scans (active users with a Radio or a MovableType client ping the server when they update, meaning that their posts get on the site quicker).
I do understand why somebody would want to make RSS as quick as email or instant messaging but it would need radical rethinking of the current approach and infrastructure, for example downloading RSS using something like bit torrent, spreading the overall bandwidth usage on clients and not only on servers.
PS: I do like the name RSS... but since I'm a nerdy user who has been using this name for years my opinion doesn't count much. Compared to HTTP, "html" or "tcp/ip", RSS seems a reasonably easy name for everybody to use and remember. Marketing wise I would probably change the orange XML icon from "XML" to \"RSS" (I understand why it reads XML, but it's probably a little hard to explain to the everyday user).
8:30:00 PM
comments: trackback: