• Senior Project Manager - Multi...
• MANAGER STRATEGIC INDUSTRY...
• Project Manager - Trading...
• C#ASP.NET Developers
• Project Analyst

French Window - RSS: For Geeks Only?

Bill French, Information Age

17/06/2004 14:23:10

When I first encountered RSS [Really Simple Syndication] it was in relation to some research I was doing concerning ICE, an early form of syndication sponsored by Vignette Corporation in 1999. I was trying to understand new methods of moving summary content out to multiple servers. Following that research project RSS faded into the background for me for about nine months, when in mid 2000 I realised that I was having difficulty staying ahead of the constant bombardment of content pursuant to my work and interests.

RSS seemed to provide a solution that would allow some relief managing personal flows of information and I began to seek out content sources that were available in XML, preferably RSS. At the time, there were only a few tools for reading semantic formats, but they were good enough to help me sift through information quicker than I was able to before adopting this practice. Given the emergence of RSS over the past three years you can imagine how happy I must be today. You can’t go a week now without seeing stories about RSS adoption, new tools announcements, and an unending flow of new feeds becoming available from major companies.

Geek or survivor? I presently subscribe to about a hundred RSS feeds and my inbox is running at about 20 to 40 messages on a daily basis; very manageable. I no longer receive email newsletters and I specifically seek out newsletter sources that are RSS-based. To put it bluntly, I have no time to surf the Web; the Web has to come to me.

Is RSS useful to me because I’m a geek? Or is my interest in RSS driven by a requirement for me to survive in the profession that I love? Until recently, I thought that RSS adoption (mine as well other people in information-based professions) was based on the common belief that early adopters, who possess the skills to use nascent technologies, choose them simply because they can. I think this is a flawed argument. It’s all about time.

As I reflect on my decision to use this less-than-mature model for staying abreast of information, I have come to realise that many of my colleagues and other early adopters of RSS have come to depend on RSS for one very simple reason: they value their time. Many people think that RSS is simply a new model for browsing, and while it is a new way to browse information, it’s far from just that. RSS provides a way for you to save time and depending on how much time you presently spend going to Web sites or hunting down updates to information sources. The more you use the Web, the more time you will likely save if you adopt RSS.

For people that work in information technology, the last few years have produced an overwhelming onslaught of information; overload like we’ve never seen before. Our inboxes are clogged and difficult to manage. Press releases for new or competitive systems (that we must understand or at least be aware of) are not getting read. Spam is adding complexities to our ability to sift and discover important messages. The velocity of information [in addition] to the volume is simply too much to handle.

IT people stand at the shoreline of an approaching tsunami of content partly because producing and consuming information is an integral part of their job. As such, they feel the wave hit first, and with a force that causes them to look for information survival alternatives. RSS is one of many information survival alternatives that are being adopted at an alarming pace; so rapid that the present polling nature of RSS will need to change.

The news Is not all good Imagine a very large swamp, teaming with orange alligators as far as the eye can see, and they're all hungry. They eat like clockwork, every 60 minutes. They devour food without any regard to the available resources. And in the middle of the swamp, perched on a small rock outcropping with open crates labelled “Bandwidth Sandwiches”, we pitch them in as fast as possible; our only hope is that the alligators will eat us last. This is an ugly scenario. What happens when everyone discovers the power of alligators; er, I mean aggregators? I agree with the premise of this Wired Magazine article – an Internet society that polls for everything will probably not work very well.

Seriously – this is a problem. We serve about 5000 RSS feeds off a single Intel CPU, but not without some tense moments. Imagine 100,000 readers each requesting a single feed at 60 minute intervals (many at the top of the hour). That’s 2,400,000 HTTP requests per day. Now add 20 feeds.

RSS content can be delivered via FTP and Web services to file systems and into BitTorrent (theoretically). But the future requires a Web services approach that elevates intelligent process between the client and server. Additionally, the RSS server must reward readers for good behavior and punish them when they waste bandwidth. We have many RSS delivery mechanisms that we’re considering for the future and we already have algorithms that punish certain readers that are blatantly wasteful. The general polling architecture behind RSS and adopted by virtually all RSS readers (while popular today) must give way to smarter approaches soon. "Once all the covered wagons show up and an RSS reader is folded into Microsoft's Internet Explorer or Outlook, we're doomed," -- Gary Lawrence Murphy

RSS implementations must evolve Actually we’re doomed if reader tools and technologies don’t get smarter and listen to servers as much as they ask for information. The issue of bandwidth consumption will likely give rise to a new type of server; an RSS server. And a new reader capability; the ability to arbitrate with a server. The MyST platform is designed to handle transactions like this and many others types of processes that embrace loose information couplings in intelligent ways.

"The trouble is, aggregators are greedy. They constantly check Websites that use RSS, always searching for new content." -- Ryan Singel And this is exactly why RSS is so useful. People need machines to help them wade through information and determine the delta (i.e., what's different from yesterday). Machines work best with machine code and that's what RSS is - a semantically useful format that an RSS reader (i.e., your agent) can use to help you deal with the information tsunami that is approaching.

The nice thing about the relationship that an RSS server has with its reader agents – it can augment the feed to inform users that they will not be serviced [with content updates] until they upgrade their reader client to something that doesn’t suck the life out of the information architecture. In fact, I wouldn't be surprised if certain highly demanded feeds will be served rapidly to registered customers and a bit slower to non-paying content consumers. Imagine paying a premium for high-velocity performance of feeds important to your business. Or paying a surcharge to dine on RSS content at a peak demand period. A whole new world of RSS services will be required to perform the myriad of semantic content acrobatics that will be required.

The swamp is teaming with orange beasts and the growth rate (in the near term) is likely to benefit those making bandwidth hoagies. In the meantime, be careful what you syndicate; it may require an extra server or two.

The adoption rate I don’t have any statistics concerning the adoption rate of RSS and I clearly have a biased perspective. However, when someone asks me why I feel so strongly that RSS is an important trend to watch and consider in IT solutions, I point to these facts – a seemingly non-technical appreciation for the use of RSS in business that are traditionally cautious about technology.

About.com

Yahoo.com

Oracle

Borland

Amazon

CNet

Feedroom

Washington Post

Microsoft

Sun Microsystems

National Government of Canada

In 1994, if someone told you that in 10 years there would be just shy of a billion Internet users, more than 100 billion Web pages, and trillions of dollars of commerce based on a globally connected network, you might have thought the person insane. It was difficult to imagine then, as it is difficult to imagine now, that more than 100 billion RSS feeds will emerge in the next six years. It is difficult to imagine a different model of search where information (that you really want) searches for you. It’s difficult to imagine anything as geeky as RSS (i.e., machine-code in a URL) impacting non-technical users in every aspect of information and their daily personal and business lives. But it was difficult to envision a Web page 10 years ago.