Entries in 'feedwordpress' ↓

Aggregation tools such as WP-o-Matic and FeedWordPress just got a promising little brother, and I’m currently playing a little with it in the Kaplak Labs. The name of this nice little plugin for WordPress is Yet Another Autoblogger or YAAB in short. It is developed by Satheesh Kumar, who was kind enough to post a note on the blog about it just recently:

I too have made a similar but better plugin called YAAB-Autoblogger. Yaab has all features of wp-o-matic and in addition it can create automatic blog carnivals in your site. Also it supports SMS blogging and Youtube cloning. Ebay product syndication and automated content rewriting are upcoming features. After all I myself is a doctor ( not a programmer ). I started making this plugin for my personal use, but when I doveloped it, it was highly impressing and I have planned to release it for public. Kindly download it from http://www.psypo.com/yaab , try it and if possible please review it in your valuable blog

I have only just played around with this plugin a little, but it looks fairly promising. Here are my initial comments and feedback for further improvement (which I also posted on Satheesh’s blog) :

• I can’t get YAAB to fetch multiple posts in separate posts, like FWP or WP-o-Matic does. It fetches only the latest post or saves the complete feed into a single post, no matter what values I provide it with. I’m sure this is easily fixed or explained.
• YAAB is very userfriendly and has an almost cartoony tutorial-like quality. I like the little character who helps guide setting up a feed for aggregation. Neat stuff, but it makes me wonder how flexible the plugin will be for more “unusual” type feeds.
• I also like the template very much. It’s very similar to what Guillermo did in WP-o-Matic, and I liked it there too :-)
• However, there are no variables for author, date posted, permalinks back to the source, or other data included in the feeds. Would be nice to be able to extract all the information in the feed, and place it where I want in the post. Also would be nice to have a regex like functionality to replace terms or code in a feed item, like the one used in WP-o-Matic. But especially the author and source/permalink information is crucial, IMHO.
• There are no functionality for tagging incoming posts, or fetching the tags included in the feed. Also a bit crucial in my book.
• YAAB has some very promising YouTube feeds functionality which makes it easy to set up an autoblog with automatically embedded YouTube videos. I haven’t played with it yet – but I will :-)

As previously stated, I have absolutely no idea how flexible this plugin is yet when it comes to feeds from Twitter Search and other such weird Atom sources. But as this is the first version, I’ll worry about that later :-) Keep up the good work, Satheesh!

FeedWordPress has received an extensive update. Latest version of November 5th 2008 (including subsequent interface bugfixes) is available here.

Great to see this round of improvements! Some of the most important features seem to be support for tags and formatting filters. The plugin has also been removed from beta status and supports all the latest versions of WordPress (2.5 and 2.6).

Find our earlier review of FeedWordPress and WP-o-Matic here.

We’re in the process of setting up our Planet-like website Kaplak Stream. I’ve done some extensive reading and testing of the two most prominent aggregation plugins for WordPress and WordPress MU : Guillermo Rauch’s WP-o-Matic plugin and FeedWordPress by Charles Johnson (aka RadGeek) of Feminist Blogs. This article will examine the pros and cons of both these plugins, in their present state.

Both aggregation tools are open source and distributed under a GPL license, which means that anyone may adjust the workings of these plugins and re-publish their version. They are each however developed and pioneered by one developer only, and rely heavily on the committment of their developers.

WP-o-Matic

WP-o-Matic is developed by 16-years old Argentinian wunderkid Guillermo Rauch, who has done a remarkable job. Schedules are very easy to organize. They are called campaigns, and each campaign can fetch as many feeds as you like. Campaigns are executed by cron, which runs on the server and executes the fetching script at specified intervals. If you can’t get cron from your web host, the WP-o-Matic script can be executed by Webcron. Webcron has been a free online service until recently. Now, the service must be paid for, however (at a very low price, one may add).

Pros

• Wonderfully flexible customization options of each campaign, directly accessible from a brilliantly designed WP admin interface: specified expressions or URL’s can be transformed, and additional custom text or code added to each post in the campaign (such as ads). Great stuff.
• Uses cronjobs for executing the script, which should provide the greatest reliability, if you can get it.

Cons

• Doesn’t use timestamp of fed posts, if they are older than the time window set for the campaign. I.e. if a post is months old and you’ve set your campaign to fetch every hour, posts will be timestamped with the time of feeding it, rather than the original timestamp. This sometimes means older posts are published in the wrong or opposite order of the feed, which messes up the chronology of a blog. This, combined with the bugs which makes it difficult to re-run fetches without completely removing the campaign, makes correcting the timestamps a very tedious affair. If timestamps are important to you, this is a no-no.
• Uses Unix/Linux cronjobs for fetching feeds, which is good if you can get it – and know how to set it up, but not all can or do.
• Seems unreliable when used without Unix cron. Campaigns are not processed at all, or processed at the wrong time intervals.
• Bugridden – small bugs such as campaigns not resetting properly, when reset. Complete campaigns and posts have to be deleted if one wants to re-fetch a feed to test a new configuration.
• Uncertainty if the plugin is supported and developed further by it’s developer. Last release is from October 2007. Guillermo (who has now turned 17) recently announced his continued support for WP-o-Matic and the release of a new version in the near future, along with a new website specifically for this plugin.

FeedWordPress

I initially had problems with feeds from Google Reader (and Twitter, for that matter) – titles showed, but content disappeared. At first I thought this was a general problem with Atom feeds, but it turned out it’s because WordPress (even the latest versions) comes bundled with an outdated Magpie RSS parser. At first glance, the problem wasn’t fixed by exchanging the rss.php and rss-functions.php with the updated ones bundled with FeedWordPress, but reinstalling these files and re-entering the feeds did in fact solve the compatibility problems with Atom feeds. At first, coming from WP-o-Matic’s advanced campaigns setup, I wasn’t impressed with the interface provided by FeedWordPress initially, and the hazzle I had with Atom feeds gave me the impression that this plugin was no match for WP-o-Matic. But as I worked with it, FeedWordPress turned out to be an extremely competent agent for the job.

Pros

• Extensively well documented
• Seems to be the more stable and reliable candidate of the two. Works great with WordPress’ built-in cron alone.
• Built-in API for WP themes and plugins to use
• Maintained, supported and seems to be actively developed by the developer (last build 8 May 2008)
• Works great with timestamps – fetches all timestamps from feeds 100% correctly.

Cons

• Can’t add custom text or code to the posts of each particular feed, except if one utilizes the API. If one utilizes the API from a WP theme, custom changes will apply to all syndicated posts, when they are displayed on the site. This is a solution in cosmetics only, in that the custom layout and text is applied only in the visuals – and not reflected in the actual contents of a post. One has to access the API from within a plugin, which hooks itself up with an action or filter in WordPress, to actually ‘inscribe’ posts with custom text or code, which stays with the post, no matter how it is skinned or re-published by other sites. This requires a bit of PHP coding/hacking skills.
• Can’t import tags. Tags can be imported by FeedWordPress as new categories, however, which somewhat alleviates the problem, but forces you to go with the category system over tagging or both.

Conclusions

Both these plugins reviewed here possess tremendous power, at the point of your fingertips. None of them are perfect, however, and both still need work, but I’m impressed with both. What they can do, and the power and speed of which these plugins work, is impressive. I’d love to have FeedWordPress feature the powerful customization scheme of WP-o-Matic, and I’d really like to have WP-o-Matic use the WordPress cron so reliably and steadily as FeedWordPress does. And I’d really really like to have WP-o-Matic just get timestamps right, with the ease of FeedWordPress.

However much I adore the flexible and powerful customization interface (the ‘campaign’ setup) of WP-o-Matic, we have to go with the more stable candidate of the two, which is FeedWordPress, IMHO. Especially since we can’t get cron right now, and are reluctant to pay for it right now, if we can get something which works great at this level, without paying for it.

We’re going with FeedWordPress, for these reasons mainly :

• It works well, even without setting up cronjobs (using WordPress’ built-in cron).
• It deals well with timestamps. There’s no messing around with the chronology of posts.
• It is the best documented plugin of the two, and it has an API which makes it easy for us to tweek it for our uses.
• And we have greater trust in it’s developer Rad Geek/Charles Johnson to continue support and development for this plugin.

When using free software plugins, I find picking the ones you want to use comes down to what killer feature you really want and which developer you trust the most to deliver it and continue development and support.

Kaplak Stream is based on a Wordpress MU install (currently v2.6.1), where a network of niche sites are fed one or more feeds on a particular subject in the ’stream’ or from particular online services, using feed aggregation tools.

Building the setup for Kaplak Stream so far has revealed a path ridden with challenges (as one might expect). WordPress MU, which is a tremendously powerful package, is not as widely used as it’s popular little sister, and therefore is less well documented and supported, which goes too for the compatibility and effects of various plugins.

One initial thing which gave rise to some trouble, was to get WordPress MU to stop worrying and love embedded stuff such as YouTube videos and widgets. WordPress MU was designed for great environments hosting thousands of blogs, with thousands of different users, and has a higher security threshold than regular WP. And there’s no way to turn this filtering of tags off in the Admin interface.

Now, there’s a plugin called Unfiltered MU which will remove this filtering of posts and thus allow the embedding stuff. Unfortunately this plugin works only with posts actually published using the Admin interface editor. It doesn’t work with imported posts (from your old single-WordPress setup), and apparently it doesn’t work with aggregated posts either. So if you setup MU and want it to import an old blog or set it up to aggregate items from a feed, you still got trouble.

I found out one has to manually edit kses.php to enable the tags used by embedded stuff, at one’s own peril. For our purpose, however, we’re not concerned with security in the sense that we are the only users of our system, for the time being.

At your own peril (I underscore the fact that you may put your setup at risk enabling these HTML tags, but hey, life is dangerous) : Put in these tags and something along the lines of the below code into your “allowed” arrays in kses.php : object, embed, param, script.

'object' => array (
			'id' => array (),
			'classid' => array (),
			'data' => array (),
			'type' => array (),
			'width' => array (),
			'height' => array (),
			'allowfullscreen' => array ()),
'param' => array (
			'name' => array (),
			'value' => array ()),
'embed' => array (
			'id' => array (),
			'style' => array (),
			'src' => array (),
			'type' => array (),
			'height' => array (),
			'width' => array (),
			'quality' => array (),
			'name' => array (),
			'flashvars' => array (),
			'allowscriptaccess' => array (),
			'allowfullscreen' => array ()),
'script' => array (
			'type' => array ()),

Pick the ones which you need for your videos or other embedded media to work. Allowing the ones listed will allow video embeds from most providers, incl. YouTube, Google Video, Viddler, Blip.tv and others as well as widgets from a lot of sources. It works on posts aggregated by FeedWordpress for instance, which was my problem with the “Unfiltered MU” plugin.

As I mentioned in an earlier post, syndicating stuff is also one huge grey area of legal hazzle. Stumbled over this discussion from a couple of years back (as well as this one), which airs not at all uncommon concerns. You risk being called a scraper, a spammer and a splogger, if you pursue the path of syndication.

Pariah S. Burke wrote :

RSS feeds are published for individual, private consumption; they are not a blanket license to, or waiver of, reprint rights. Taking and republshing content—no matter how much or how little—without the original author’s permission is a violation of U.S. and international Copyright laws. There are exceptions, of course, detailed in the Fair Use doctrine, but such exceptions are very specific and do not apply to the vast majority of sites using FeedWordPress, Autoblog, and the like. In fact, Charles Johnson, the creator of FeedWordPress is in constant and frequent violation of copyright law because the apparent majority of his blog’s content is stolen without the original authors’ permission.

In that case, Google, which enables users to very easily tag and share (i.e. republish) feeds they find interesting via their popular service Google Reader, is guilty of same said constant and frequent violation of copyright law, or at least, in willful and assisting infringement. The same of course goes for YouTube and any web service, which allows anyone to embed their videos, images and games on your own local site.

Who says a tool has to be used in one way only? Let’s get creative! That’s how problems are solved and new business models are developed!

Here’s another POV, from a guide on setting up an automatic blog which automatically generates a ’shitload of traffic’ and is ‘just about hands free’ :

To be honest, I’m not a big fan of people scraping content that people have sweated over. However, one thing I don’t mind doing is thieving from thieves.

You’re on the hunt for “disposable” content – generally not text based. Think along the lines of Flash games, funny videos, funny pictures, hypnomagical-optical-illusions – that kind of thing. The Internet is awash with blogs that showcase this stuff. Check out Google blogsearch and try a search like funny pictures blog. There’s hundreds of the leeching bastards showcasing other peoples pictures, videos, games and hypnomagical-optical-illusions for their website. They can hardly call it “their” content. With this ethical pebble tossed aside, we can go and grab some content.

There’s loads of ways you can hunt down potential content. You’re on the lookout for RSS feeds with this rich media. So you could try; Google Blogsearch, Technorati, MyBlogLog – basically any site that lets you search the blogosphere.

My personal point of view (this is also Kaplak’s stand) is that the problem of visibility for sites and products is larger than the largely fictional problem of “theft”. If you make syndicated feeds publicly available, you implicitly want and ask for syndication, because you want your message out. Syndication will help your site or product become visible in places and contexts it would not otherwise be seen in, and that’s why you use it and why you should use it. If you do not want your message out in other contexts and do not want to see your articles appear on other websites in a syndicated format, you can simply choose not to make articles available for syndication. The benefits however, in the Google Juice and traffic which syndication brings back to your sites and products, are in most cases much greater than the disadvantages.

Accusing syndication sites and services for theft and copyright infringement is IMHO ridiculous at best, as these services actually help your site become seen and achieve better rankings in search engines. It helps your interested readers and users find you in the first place. And if you don’t want to be read – why publish to the web?

At worst, these allegations are harmful, as they instill an atmosphere of fear and create distrust of using RSS, feeds and aggregation tools. Instead, we need to urge and encourage syndication and use of syndicated feeds, as it enables rich web contexts, which would otherwise not be possible, and makes it easier to direct interest and relevant traffic to sites and subjects of interest. It is above all a tool, which can be used for our mutual benefits – or for spamming and creating yet more “get rich quick” mentality kind of sites filled with stuff the world could care less about (but apparently doesn’t). I am of the opinion that these types of sites may provide their owners with short-term rewards, but ultimately will fade to authentic sites of much stronger lasting value. How to build lasting value, and help these sites and products build lasting value, is what we’re interested in here.