C# Code, Tutorials and Full Visual Studio Projects

RumorMill 5 – RSS, RDF and ATOM Reader

Posted by on Oct 4, 2011 in Code Snippets, Featured, Projects, WPF | 29 comments

RumorMill 5 – RSS, RDF and ATOM Reader

RumorMill 5 is a C# WPF MVVM application that polls RSS feeds. It also supports the idea of categorizing your feeds by color and name so you can identify the topic of the feed simply by the color.

If a picture is worth a thousand words what a video worth? This should help clarify how it works:

 

 

Another thing I didn’t mention in the video (yes I’m too lazy to re-record it, is that if you click on the categories at the top it filters the view. So if I clicked “Games”, then all the items in the games category would disappear. If I clicked it again they would reappear. Also the time on the side that displays how old the article is, is dynamic. There is a timer in the background that changes the time so it always shows how old the article is while it’s on the display.

Download the Source Code
So what happened to RumorMill 1-4? They were all posted online at one point or another on my old site CodeFreezer.com, which no longer exists.

Configuration

RumorMill5 is rather simple. It consists of a single View, one ViewModel, 3 Models and a bunch of parsers to handle RSS, RDF, and ATOM feeds.

It has three files it uses.

  • Categories.xml – defines a list of categories
  • Feeds.xml – defines the feeds,polling intervals and categories to use per feed
  • Cache.dat – a cache of all the unique id’s so we can tell what is new and old when we poll

Categories

RumorMill has a grouping concept called categories. Each feed you assign to a category. The background color of the feed is then displayed in the color that was chosen for the category. This way if you have a category of “Finance” and “Games” you can tell the types of feeds by simply looking at the color.

The Categories.xml file defines the names and colors of the categories.

Feeds

RumorMill5 supports RSS, RDF and ATOM feeds. The feeds.xml file defines what feeds you want the program to poll and the interval you would like to poll them. It also assigns a CategoryName which links it to the categories.xml file listed above.

Cache

When RumorMill5 downloads a feed it gets the unique id of each item in the feed and adds it to the cache.dat file. The next time it polls it checks the cache.dat file to see if it has seen each item. Only new items are then displayed.

NOTE: RumorMill5 never erases the cache so if you planning on using this in a production scenario you will need to implement a janitor process to clean very old cache entries.

How the Polling Works

RumorMill5 polls each feed listed in the feeds.xml file, based on the interval specified in in the file. It does this via a single master loop. This master loop is called the feedTimer. It ticks each minute and looks for items that need refreshing. Each feed is handled on a separate thread using the .NET task framework. (see FetchFeed Method of ReaderViewModel)

How Parsing Works

Each feed being refreshed is interrogated by the FeedParser which determines what type of feed it is, ie: RSS, RDF, ATOM. Based on the feed a different parser is called. (yes it is really a factory pattern now that I look at it.)

The parsers are probably the most interesting part as they are done from scratch, without using any RSS, RDF or ATOM libraries. Just LINQ for XML which continues to amaze me. (someone give those LINQ devs a raise pls.)

Here is an example of the RssParser

if (doc.Elements().Count() == 0) return null;

XNamespace ns = "http://purl.org/rss/1.0/modules/slash/";

List items = (from item in doc.Descendants("item")
                        select new NewsItem
                {
                    Id = (item.Element("guid") != null ? item.Element("guid").Value : item.Element("link").Value),
                    Title = item.Element("title").Value,
                    Timestamp = Rfc822DateTime.Parse(Rfc822DateTime.Clean(item.Element("pubDate").Value) ?? DateTime.Now.ToString()),
                    Url = item.Element("link").Value,
                    Description = (item.Element("description").Value ?? ""),
                    NewsFeed = feed,
                    FetchTime = DateTime.Now.ToUniversalTime(),
                    Source = feed.Name
                }).ToList();

return items;

29 Comments

Join the conversation and post a comment.

  1. Andrea

    Hi, I’d like to know if i can integrate RumorMill5 in my application. Please note that this is a production appliccation and will be sold to customers, so ibasivally i’m asking if i can use your good work with a commercial license.
    Furhermore, please kindly tell the schema for xml files as my visual studio did’nt create them when building.
    Thanks al lot!

  2. admin

    Feel free to use the code in your production application and sell if you wish, just keep in mind I take no responsibility for any problems that might arise due to bugs etc in the code. Please see the Disclaimer.

    I don’t have a formal schema created for the XML files. Should be simple enough for you to make one if you wish, I didn’t have the need to make one.

  3. Andrea

    Thank you so much!
    You are right, the XML is simple enough.
    Again, thanks

  4. Anonymous

    Hi, can you privide a simple feeds.xml and categories.xml files to try?
    I cannot figure out what i’m supposed to do…
    The applicaiton does not show anything !

  5. admin

    Sorry about that, when I packaged it and removed the bin folder I deleted the XML files. I’ve put a new copy of the project up with the XML files included.

  6. ct

    I’ve dabbled in WPF stuff for a few years now and this is the first time I’ve seen anyone use FluidMoveBehavior in a list control to animate fade items coming in. This is so cool cuz now Windows apps can start to look just as good and feel like those slick iPhone apps that. I also like the AnimationHelper idea of fading it to mark it read.

    I’m going to use some of these ideas in a personal log file viewer app I’ve been meaning to write to make it look cool and use FluidMoveBehavior to fade things in as new errors/alerts come in.

    With all the hype of HTML5, etc. this gives me renewed interest in learning more about WPF — still so much to learn. Very cool — thanks again for sharing!

  7. Sunil

    Hi,

    I have been trying to compile and run your app is VS2010.
    But it keeps throwing up “Type universe cannot resolve assembly: System.Windows, Version=2.0.5.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e.”
    Error.

    I added the missing assemblies from the silverlight v4.0 directory to the gac using the gacutil
    Still no success.

    Could you please give any suggestions. ?

    Thanks

  8. admin

    I’m not sure, there is nothing different with the project that should cause an issue. I made it in visual studio professional 10. Do you have Microsoft blend installed? It uses the interaction libraries from there.

  9. Sunil

    Hi,

    I do not have blend installed. But i did install the expression blend sdk to reference the libraries. Maybe i should try installing blend and then try.

    Thanks

  10. Sunil

    Hi,

    I installed blend and it worked without a problem.

    Thanks

  11. Doc

    Great solution! Thanks!

  12. Mike

    Hi, thanks fro this – your stuff is generally great. and I as looking forward to seeing this as I’m trying to write a news aggregator for my Windows Tablet but when I run it it does absolutely nothing!! (Running in VS2010)

    Oh well, guess there’s only one way to know what it’;s doing… debug mode here I come.

  13. Mike

    Ok, worked it out – importing the solution into VS2010 set the .xml files back to not being Content so they weren’t being copied to the execution folder… changed that and it’s working well now. I would also like to get images from the stories for my aggregator so that’s my next challenge. Many thanks for the code!

  14. Howard Camp

    @admin – neat tool and good article.

    You said:

    <>

    I’m looking in your code to see how you determine the unique id of each item but I can’t find it. Can you please point me to it?

    Also, there isn’t a cache.xml included with your project and doing a text search for “cache.xml” in your source code comes up with nothing. Where does that exist?

    Thanks!

  15. Howard Camp

    Whoops – I put something in double angle brackets and the html parser ate it. I meant to say:

    You said:

    “When RumorMill5 downloads a feed it gets the unique id of each item in the feed and adds it to the cache.xml file. The next time it polls it checks the cache.xml file to see if it has seen each item. Only new items are then displayed.”

  16. admin

    The cache.xml is generated by the application and will be created in the output folder. Run the application with a feed and you will see it appear. It’s only necessary to prevent showing the same articles each time the feed is polled. So you can simply delete it anytime.

    The unique id is already in the feed as its part of the rss spec. If its omitted I believe I use the URL of the rss article. (sorry I would be more precise but I’m typing this on my phone and going by memory as I’m not near a computer.)

  17. Howard Camp

    Thanks for the quick response!

    Curious, where in the code is the cache.xml created? I’m searching and don’t see it.

    I’m looking in your code to see how you determine the unique id of each item but I can’t find it. Can you please point me to it?

    Thanks

  18. admin

    I just looked at it’s cache.dat instead of xml (my fault had a typo above which I’ve corrected). The name is declared in the ReaderViewModel as a constant, but the file is created when the Save() method is called. That calls the PersistanceManager.SaveRows() method which does a File.WriteAllLines(). If the file is present this will append to the file. If the file is not present it will be automatically created.
    The Save() method is called when the application is shutting down, in the ReaderView.Closing event.

    Hope this helps.

  19. Howard Camp

    Gotcha! Hey sorry to be a nag but please see my other question about how you determine the unique ID of a feed item, and how you determine that the feed has new items in it.

    Thanks again

  20. admin

    The unique Id is set in the RssBuilder, RdfParser and AtomParser.
    The RSS one uses the GUID that is returned in the feed, or if that is null it uses the link.
    The ATOM one uses the ID or again the Link if the ID is null.
    And the RDF one uses the link as there is no other unique ID I could consistently find.

  21. Howard Camp

    Good, there’s no magic I’ve missed then. In this article http://www.therssweblog.com/?guid=20070407124931 the author uses the combination of , and to determine uniqueness.

    ps Thanks for being responsive – it’s rare nowadays.

  22. Howard Camp

    I screwed up again and used as string literals which are actually html tags, so they got removed. The author uses the combo of link + title + description.

  23. Sean

    Thanks for the code! Starting with your syndication parsing code has proven to be more robust than using built in dotnet SyndicationFeed.

  24. admin

    Yeah I also started with the built-in .net feed but had so many issues, expecially with dates that I made my own. Keep in mind though that I haven’t used this code in a production scenario so it’s bound to have a few issues.

  25. ray

    Your solution is grt .

    How to affect a change in direction in Ticker.cs file?

  26. Sam

    I’m brand new to c# and this will be the first source code I’ll compile. I originally opened the project 5 minutes ago to find that I didn’t have some reference assemblies that were required for the program to run. I figured out as much that I needed to install expression blend and that fixed 2-3 of the unlinked references though I’m now dealing with the following error “Warning 1 The referenced component ‘AttachedCommandBehavior’ could not be found.” I’ve done a fair amount of google searching but to no avail. Am I to obtain this assembly from a specific website? am I to compile something before I compile the application which would then create this file? I’d love to analyze this project and have been looking for an RSS feed such as this for a long time so I’d appreciate being pointed in the right direction, thank-you.

  27. Sam

    I assume I’m to reference AttachedCommandBehavior.dll version 2.0 by first destroying the defunct reference and re-instating it? I’ve happily got the program to launch however it displays nothing but a blank state, nor do the toolbar options seem to respond. I’m not sure what I may or may not be doing wrong at present, nor do I wish to turn this into a lengthy troubleshooting session. I’m executing the application using the solid green play icon, am I to perform some kind of pre-compilation before I do so? Thank-you in advance.

  28. Kelly Elias

    My bad, thank you for pointing this out. As the article states the solution needs two XML files. One is a list of groups, the other is a list of RSS feeds. But I never showed the format!
    This has been corrected. I updated the ZIP to include a ReadMe.txt with instructions on what to do and included an example of the missing XML files to get you started. Sorry about that.

Leave a Comment

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>