Building on the idea of open data tables, I made a very quick table (see it on GitHub) which plugs into Feedburner’s API and allows us, with a few different options, to grab data about a specific Feedburner feed by using queries like select * from feedburner.feed where uri="cowburn"¹.

Querying against the table

Since the table is just my own, and not part of the main collection of contributed tables, we have to tell YQL about its location before we can do any queries on it. To do that, simply use use pointing to the url of the XML file: use "http://github.com/salathe/yql-tables/raw/master/feedburner/feedburner.feed.xml";

Once YQL knows about the table, we can query against it (by default the table name is constructed from the file name, feedburner.feed.xml means we use feedburner.feed as the table name). The table is set up to allow three different methods of grabbing the feed data.

1. Up-to-date Statistics

If just the feed name is provided then only the most up-to-date information about that feed will be returned. Feedburner does not let you get stats about today, so the most recent data will always be from yesterday. So select * from feedburner.feed where uri="cowburn"¹ gives one result containing yesterdays statistics; subscriber count (circulation), the date, the number of hits the feed got and the reach of the feed.

2. Get last n days of stats

If you’re after more than just the latest figures, simply add a days clause to the where portion of the query: … where uri="cowburn" and days="7"²

3. Get a specific set of dates

If the last n days is too recent for you, there’s also the option of passing along a range or series of dates with, conveniently enough, a dates clause. This value follows the format for formatting date ranges of the Feedburner API. So to select feed data between July first and seventh from 2009 you would simply add … where uri="cowburn" and dates="2009-07-01,2009-07-07"³

Note that you cannot combine the days and dates in the same query as that would be silly.

Adding Subscriber Count to the Blog

To add the subscriber count to the page (next to the “subscribe” link in the header), just takes a few lines of JavaScript (I used jQuery) to pull a JSON-formatted response back from YQL. The console even automatically creates an URL for you to copy and paste in the section entitled “The REST query“.

The actual jQuery/JS code is not particularly noteworthy, all it does is prepare the URL to the YQL API, asks that URL for some JSON and then parses the response into a HTML span tag which is injected within the “subscribe” link of the navigation. This blog post is more about the data table, than grabbing JSON and injecting elements into the DOM (which is super-easy with jQuery anyway). Would you like that process described a bit more thoroughly? My feed widget with YQL and jQuery post shows you some actual code.

Well that’s all there is to say really. If you want to use the Feedburner data table then by all means feel free—it’s up there on GitHub so you can link to the file or fork and make your own changes.

Footnotes

1. Try select * from feedburner.feed where uri="cowburn" in the YQL console
2. Try select * from feedburner.feed where uri="cowburn" and days="7" in the YQL console
3. Try select * from feedburner.feed where uri="cowburn" and dates="2009-07-01,2009-07-07" in the YQL console

P.S. I promise (with crossed fingers just in case) that the next blog post will not be about YQL!

INSERT/UPDATE/DELETE

Now we can have a full CRUD stack to play with enabling use to write to the web in addition to the reading we’ve had thus far! If you’re failing to see the juiciness, think that you can now do INSERT INTO web.service (a, b, c) VALUES ("a", "b", "c") where web.service can be pretty much any open API (bit.ly and twitter examples are in the documentation) or, to put it plainly, pretty much any web site which accepts user input.

post/put/delete methods on the y.rest() method

A huge thank you for this since, tied into the above on I/U/D, this enables us to push back data to the web. I asked for this back in May since I needed it to log into a website via a POST request, saving cookies for the main request after login.

Json table now accepts top-level arrays.

A small bug fix but particularly useful for me. See my YQL forum post on this bug which arose due to my trying to query a delicious.com JSON feed.

Preamble

I could’ve gone the server-side route; use PHP to grab the feed from here (well, Feedburner actually), parse it (probably with SimpleXML for simplicity) and display it with my desired HTML. And that’s a perfectly acceptable approach. However, since the feed content is only really an added extra I didn’t want to add the feed content to the HTML page and have that overwhelm the main content on there (the introduction and contact info). Because of that, I decided to take the client-side approach using JavaScript to grab, parse and inject our feed into the page.

The Feedburner feed, as is usual for feeds, comes in XML format and due to the security restrictions of browsers I can’t just go and grab that with JavaScript. Besides, I didn’t want the whole feed anyway: who wants to load in a 38kb feed just to grab some headings? (What I do end up grabbing is around 4kb.)

Introducing YQL

To grab only what I want, and to provide a format that I can actually get at, as well as to satisfy a niggling urge to put it into some practical use, I headed over to Yahooland and made use of the awesome tool that is YQL. YQL provides a SQL-like syntax for getting at the structured data (feeds are structured data!) around the web. It allows me to ask for specific items of data from the web, such as “give me the title and permalink for the last 10 items from Yahoo news” for example. There’s a huge range of data that YQL can get its hands on (stopping just short of the entire web) but that’s beyond the scope of this blog entry. The service also allows me to get responses in XML or JSONP, the latter being important because I can grab this from a remote domain name without the security restrictions coming into play.

Querying for data

So to get at my feed, I can simply issue the query:
Try it yourself

SELECT * FROM feed WHERE url='http://feeds2.feedburner.com/cowburn'

For my little widget, I don’t really need everything brought back by that query. Firstly, I only want the latest 5 items instead of the 10 that Feedburner provides. Secondly, I only really need the title, date, link and excerpt for each blog entry. The Feedburner feed provides extra information like the comments link, tags, and the full HTML content of the entry. Grabbing only what I need shrinks down the size of the data coming down the pipe and results in things getting done faster. I mentioned before that I wanted the results in JSONP format which is a simple enough task: just click the JSON radio button in the YQL console, or add “format=json&callback=cbfunc” (where cbfunc is your callback function name) to the REST query URL. So to grab only what I want, the following YQL query is used: Try it yourself

SELECT title, description, link, pubDate 
FROM feed(0,5) 
WHERE url='http://feeds2.feedburner.com/cowburn'

Aside: the (0,5) limits YQL to the first 5 items in the Feedburner feed.

Right, now we have the data that we want and can get it in JSONP format. How do we go about doing that exactly? JavaScript time!

Retrieving the JSONP with jQuery

I really like using jQuery for adding in extra goodness to sites and I was already using it for a little piece of DOM manipulation on the page (interesting but not to be covered in this post). Plus, jQuery makes grabbing JSONP content very, very simple indeed using its jQuery.getJSON (docs) function. Basically we provide that function with an URL to the JSON document and a callback function to execute once jQuery has grabbed it. See the docs for examples.

Creating a widget

Generally, there’s a logical structure to creating a widget like this one. Steps involve grabbing the required data, parsing it, building the HTML and attaching it to the page. Since this would take an inordinately long time to explain step by step (perhaps some other intrepid blogger has already done it) I’ll just show you with code. Below is the JavaScript—in jQuery plugin form though that isn’t particularly necessary or important (just means it’s all wrapped up neatly and only concerned with itself). There are a number of variables to hold HTML, the YQL URL and a number of functions which parse the JSON object into HTML form and a little bit of effects to fade in the HTML block when it’s all ready. There’s also a function to parse the dates from “Mon, 05 Jan 2009 16:28:34 +0000″ into “5 Jan” form.

Put simply, the widget code just says, “when the page is loaded, ask YQL for the filtered feed, parse the response items into a HTML list and inject it into my page just above the footer.”

The widget code

// Grab food!
(function($){
	jQuery(function(){
 
		var wrapper = $('<div id="widget_blog">'
		            + '<h4>From the <a href="http://cowburn.info" rel="me">blog</a>:</h4>'
		            + '<ul></ul>'
		            + '</div>');
		var list    = wrapper.children('ul:first');
		var style   = $('<style type="text/css">@import url(assets/feed.css);</style>');
		var service = "http://query.yahooapis.com/v1/public/yql?q=select%20title%2C"
		            + "description%2Clink%2CpubDate%20from%20feed(0%2C5)%20where%20url%3D'"
		            + "http%3A%2F%2Ffeeds2.feedburner.com%2Fcowburn'&format=json&callback=?";
 
		var callback = function(data) {
			if ( ! (data && data.query && data.query.results && (items = data.query.results.item))) {
				// Unexpected response from YQL service
				return;
			}
 
			// Inject HTML before the footer
			$('.footer:first').before(wrapper);
			wrapper.before(style).hide();
			$.each(items, display_item);
			wrapper.fadeIn("slow");	
		};
 
		var parse_date = function(str) {
			var d = new Date(str);
			var m = ['Jan', 'Feb', 'Mar', 'Apr', 'May', 'June', 'July', 'Aug', 'Sept', 'Oct', 'Nov', 'Dec'];
			return d.getUTCDate() + ' ' + m[d.getUTCMonth()];
		};
 
		var display_item = function(index, item){
			var tpl = '<li><span>%date: </span><a href="%link" title="%extract">%title</a></li>';
			var html = tpl.replace(/%link/g, item.link)
			              .replace(/%extract/g, item.description)
			              .replace(/%title/g, item.title)
			              .replace(/%date/g, parse_date(item.pubDate));
			list.append(html);
		};
 
		// Actually grab the feed now
		$.getJSON(service, callback);
 
	});
 
})(jQuery);

As well as the HTML to display the feed items, the widget also adds a CSS stylesheet to add a little bit of formatting: link. The widget code itself (which may change in the future) can be found here.

End result

What we’re left with is a neat little list of my last 5 blog entries.

Salathe.co.uk before/after widget