Per-group/node RSS feed aggregation?

Question

I am in the process of building out a community site where each 'network' (like an OG group) can set up it's own RSS feed (simply by pasting the URL into their network node's 'network_feed' field).

I am in the early stages of figuring out how to build and implement a system whereby I can run through all those feeds (for networks that have them) and aggregate them, posting new nodes that contain node references back to the main network for each new item in the feeds.

I have had some limited experience with the Feeds module, and it seems the best candidate for this (Aggregator in D7 core seems a bit less hardy), but I didn't know if there was something I was missing that would make this much easier.

I'm thinking, right now, that I'll build some sort of system that queues up all the URLs contained in 'network_feed' fields, then maybe hook into some of the feeds.module's methods to retrieve and parse the nodes. Instead of doing this manually, does anyone know if there's something prewritten for Feeds, Aggregator, or another solution that does this kind of thing?

Answer 1

I eventually wrote an entire module for this purpose. Basically, I have a 'feed' field for the group-type content type, and my module tracks that field and any changes through core hook_node_update, _delete, _insert, etc. hooks. Then I have a cron-based queue consumer that grabs the 5 feeds that were updated furthest in the past and checks for new content, then posts it into nodes under the feed's group.

I used SimplePie to help with the parsing, because even SimpleXML is somewhat of a pain. Anyways, I might try to make this functionality more formalized and port it to work with something like Organic Groups, but it's really purpose-built. If anyone would like some help doing something similar, give me a shout.