1

I am developing a website which basically has the main data source from wikipedia. The website currently needs users to copy paste text from various data sources (like wikipedia). But the problem which I realized out of it is that soon the data in the primary datasource will be outdated when people enter

Instead, I am looking in developing a module for wikipedia interface with a following workflow:

  1. User will create a new page (a species page in this instance) and link it with its corresponding wikipedia page.

  2. The local database will have 2 more tables (lets say) apart from the content type information of the current page of the Drupal powered website's node, in this case a species node.

    • a. A table for storing the RAW wikitext of the linked wiki page with the most recent timestamp.
    • b. A table for storing the processed wikitext which will be mapped into the Drupal node's appropriate fields.
  3. Provide a button in the species page which will check if the 2(a) wikitext is the current in the wikipedia's corresponding article. If not, 2(a) will be updated and user will have to perform the task of 2(b) which is mapping the new wikitext into Drupal node's fields - This can be accomplished by injecting tags in appropriate way.

  4. Provide a module where the wikitext of 2(b) is pulled into the various fields of drupal's content node.

Now my questions:

Is it feasible?
How should I get going?
Any scope for enhancement in the requirement analysis?
Loopholes?

Please see, I know my question is very broad but I would need suggestions, pointers and guidance for going ahead. I believe a proper plan is much necessary before starting to implement it.

Thank you.

EDIT:

We have released IndianSnakes application in both Android and iOS. This is possible coz we had information of species formatted across different fields which can be visualized in the above links which I have pasted. Now, the whole point of my question stands, I need some ideas in developing a middleware with which the wealth of wiki information can be formatted across different fields for each species.

5
  • It may be feasible, but parsing the wikitext yourself might be a nightmare. Also, what's the added value of this? Won't wikipedia/wikispecies/wikidata be better?
    – Pere
    Mar 6, 2013 at 0:21
  • @Pere - The data set I am targeting at currently is not yet migrated completely to Wikispecies. Even if it is migrated, say, the structuring of data in API response would not be the same as we expect, so there has to be an intermediate module which translates the data as per the needs of our design. What do you think? Mar 13, 2013 at 6:43
  • I would import the data automatically from wikispecies dumps rather than depending on humans. Also, mediawiki might be a better option than drupal here.
    – Pere
    Mar 13, 2013 at 7:45
  • Hi Raj, I'm afraid this isn't the right place to ask for general ideas, or get a broad overview of building a complex site/feature from scratch. When you're further along in development, and have specific, narrowly scoped questions about a specific part of the feature you're implementing, that's when to ask here. Please have a flick through the help center for more info on what types of question are appropriate here. Many thanks
    – Clive
    Feb 19, 2016 at 12:22
  • Unfortunately true and I agree :( I also mentioned the same fact in my question too. But, can I get my bounty back? Also, which is the platform to ask such questions then? Feb 20, 2016 at 5:19

1 Answer 1

2

First recommendation: Just use MediaWiki for the whole thing. It's not as nicely modular as Drupal but it has a lot of facilities. And as a bonus it's likely that upgrading will be easier (Dn->Dn+1 has always been a pain to me, MW upgrades have always been very smooth for me).

If you still want to use Drupal, I would use MediaWiki's parsing. E.g. fetch articles through index.php?action=render&title=Page_Name. It's probably best to simply fetch data when needed but with a long cache time (e.g. 1 week) to avoid Wikipedia blocking your server's IP.

Not the answer you're looking for? Browse other questions tagged or ask your own question.