Entries in 'tools we use' ↓

We’re in the process of setting up our Planet-like website Kaplak Stream. I’ve done some extensive reading and testing of the two most prominent aggregation plugins for WordPress and WordPress MU : Guillermo Rauch’s WP-o-Matic plugin and FeedWordPress by Charles Johnson (aka RadGeek) of Feminist Blogs. This article will examine the pros and cons of both these plugins, in their present state.

Both aggregation tools are open source and distributed under a GPL license, which means that anyone may adjust the workings of these plugins and re-publish their version. They are each however developed and pioneered by one developer only, and rely heavily on the committment of their developers.

WP-o-Matic

WP-o-Matic is developed by 16-years old Argentinian wunderkid Guillermo Rauch, who has done a remarkable job. Schedules are very easy to organize. They are called campaigns, and each campaign can fetch as many feeds as you like. Campaigns are executed by cron, which runs on the server and executes the fetching script at specified intervals. If you can’t get cron from your web host, the WP-o-Matic script can be executed by Webcron. Webcron has been a free online service until recently. Now, the service must be paid for, however (at a very low price, one may add).

Pros

• Wonderfully flexible customization options of each campaign, directly accessible from a brilliantly designed WP admin interface: specified expressions or URL’s can be transformed, and additional custom text or code added to each post in the campaign (such as ads). Great stuff.
• Uses cronjobs for executing the script, which should provide the greatest reliability, if you can get it.

Cons

• Doesn’t use timestamp of fed posts, if they are older than the time window set for the campaign. I.e. if a post is months old and you’ve set your campaign to fetch every hour, posts will be timestamped with the time of feeding it, rather than the original timestamp. This sometimes means older posts are published in the wrong or opposite order of the feed, which messes up the chronology of a blog. This, combined with the bugs which makes it difficult to re-run fetches without completely removing the campaign, makes correcting the timestamps a very tedious affair. If timestamps are important to you, this is a no-no.
• Uses Unix/Linux cronjobs for fetching feeds, which is good if you can get it – and know how to set it up, but not all can or do.
• Seems unreliable when used without Unix cron. Campaigns are not processed at all, or processed at the wrong time intervals.
• Bugridden – small bugs such as campaigns not resetting properly, when reset. Complete campaigns and posts have to be deleted if one wants to re-fetch a feed to test a new configuration.
• Uncertainty if the plugin is supported and developed further by it’s developer. Last release is from October 2007. Guillermo (who has now turned 17) recently announced his continued support for WP-o-Matic and the release of a new version in the near future, along with a new website specifically for this plugin.

FeedWordPress

I initially had problems with feeds from Google Reader (and Twitter, for that matter) – titles showed, but content disappeared. At first I thought this was a general problem with Atom feeds, but it turned out it’s because WordPress (even the latest versions) comes bundled with an outdated Magpie RSS parser. At first glance, the problem wasn’t fixed by exchanging the rss.php and rss-functions.php with the updated ones bundled with FeedWordPress, but reinstalling these files and re-entering the feeds did in fact solve the compatibility problems with Atom feeds. At first, coming from WP-o-Matic’s advanced campaigns setup, I wasn’t impressed with the interface provided by FeedWordPress initially, and the hazzle I had with Atom feeds gave me the impression that this plugin was no match for WP-o-Matic. But as I worked with it, FeedWordPress turned out to be an extremely competent agent for the job.

Pros

• Extensively well documented
• Seems to be the more stable and reliable candidate of the two. Works great with WordPress’ built-in cron alone.
• Built-in API for WP themes and plugins to use
• Maintained, supported and seems to be actively developed by the developer (last build 8 May 2008)
• Works great with timestamps – fetches all timestamps from feeds 100% correctly.

Cons

• Can’t add custom text or code to the posts of each particular feed, except if one utilizes the API. If one utilizes the API from a WP theme, custom changes will apply to all syndicated posts, when they are displayed on the site. This is a solution in cosmetics only, in that the custom layout and text is applied only in the visuals – and not reflected in the actual contents of a post. One has to access the API from within a plugin, which hooks itself up with an action or filter in WordPress, to actually ‘inscribe’ posts with custom text or code, which stays with the post, no matter how it is skinned or re-published by other sites. This requires a bit of PHP coding/hacking skills.
• Can’t import tags. Tags can be imported by FeedWordPress as new categories, however, which somewhat alleviates the problem, but forces you to go with the category system over tagging or both.

Conclusions

Both these plugins reviewed here possess tremendous power, at the point of your fingertips. None of them are perfect, however, and both still need work, but I’m impressed with both. What they can do, and the power and speed of which these plugins work, is impressive. I’d love to have FeedWordPress feature the powerful customization scheme of WP-o-Matic, and I’d really like to have WP-o-Matic use the WordPress cron so reliably and steadily as FeedWordPress does. And I’d really really like to have WP-o-Matic just get timestamps right, with the ease of FeedWordPress.

However much I adore the flexible and powerful customization interface (the ‘campaign’ setup) of WP-o-Matic, we have to go with the more stable candidate of the two, which is FeedWordPress, IMHO. Especially since we can’t get cron right now, and are reluctant to pay for it right now, if we can get something which works great at this level, without paying for it.

We’re going with FeedWordPress, for these reasons mainly :

• It works well, even without setting up cronjobs (using WordPress’ built-in cron).
• It deals well with timestamps. There’s no messing around with the chronology of posts.
• It is the best documented plugin of the two, and it has an API which makes it easy for us to tweek it for our uses.
• And we have greater trust in it’s developer Rad Geek/Charles Johnson to continue support and development for this plugin.

When using free software plugins, I find picking the ones you want to use comes down to what killer feature you really want and which developer you trust the most to deliver it and continue development and support.

Kaplak Stream is based on a Wordpress MU install (currently v2.6.1), where a network of niche sites are fed one or more feeds on a particular subject in the ’stream’ or from particular online services, using feed aggregation tools.

Building the setup for Kaplak Stream so far has revealed a path ridden with challenges (as one might expect). WordPress MU, which is a tremendously powerful package, is not as widely used as it’s popular little sister, and therefore is less well documented and supported, which goes too for the compatibility and effects of various plugins.

One initial thing which gave rise to some trouble, was to get WordPress MU to stop worrying and love embedded stuff such as YouTube videos and widgets. WordPress MU was designed for great environments hosting thousands of blogs, with thousands of different users, and has a higher security threshold than regular WP. And there’s no way to turn this filtering of tags off in the Admin interface.

Now, there’s a plugin called Unfiltered MU which will remove this filtering of posts and thus allow the embedding stuff. Unfortunately this plugin works only with posts actually published using the Admin interface editor. It doesn’t work with imported posts (from your old single-WordPress setup), and apparently it doesn’t work with aggregated posts either. So if you setup MU and want it to import an old blog or set it up to aggregate items from a feed, you still got trouble.

I found out one has to manually edit kses.php to enable the tags used by embedded stuff, at one’s own peril. For our purpose, however, we’re not concerned with security in the sense that we are the only users of our system, for the time being.

At your own peril (I underscore the fact that you may put your setup at risk enabling these HTML tags, but hey, life is dangerous) : Put in these tags and something along the lines of the below code into your “allowed” arrays in kses.php : object, embed, param, script.

'object' => array (
			'id' => array (),
			'classid' => array (),
			'data' => array (),
			'type' => array (),
			'width' => array (),
			'height' => array (),
			'allowfullscreen' => array ()),
'param' => array (
			'name' => array (),
			'value' => array ()),
'embed' => array (
			'id' => array (),
			'style' => array (),
			'src' => array (),
			'type' => array (),
			'height' => array (),
			'width' => array (),
			'quality' => array (),
			'name' => array (),
			'flashvars' => array (),
			'allowscriptaccess' => array (),
			'allowfullscreen' => array ()),
'script' => array (
			'type' => array ()),

Pick the ones which you need for your videos or other embedded media to work. Allowing the ones listed will allow video embeds from most providers, incl. YouTube, Google Video, Viddler, Blip.tv and others as well as widgets from a lot of sources. It works on posts aggregated by FeedWordpress for instance, which was my problem with the “Unfiltered MU” plugin.