0

After a few attempts browsing the web, I don't figure out how to download HTML from a given url, and create node content from it.

What I would achieve is quite simple :

  • Have a link-like field in my content type
  • The source code of this page is downloaded either in the Body of the content, either in a MHT-like file
  • Viewing the node displays it either in the bodyeither` in an IFrame.

The content creation would happen mostly from a dedicated script creating a new content through Drush, filling only the Title and the link field.

The main goal of this is to save short-lived webpages in a drupal website, and ideally being able to index them.

I'm open to any ideas : module, coding, whatever. Don't hesitate to give me advice !

Thanks

2
  • When would you want the node to be populated with the content of the URI in the field? When someone loads the node for the first time?
    – Brady
    Jul 16, 2015 at 13:30
  • I thunk it could more likely happen when saving the node. Please continue what you mean :)
    – Benj
    Jul 16, 2015 at 13:49

1 Answer 1

1

This would have to be done using a custom module. I do not know of a contributed module that would perform this. Probably because a module like this could be seen as a huge copyright infringement lawsuit waiting to happen.

But if I were to do this I would use GuzzleHttp as the engine to pull down the content. Then take that content and create a new entity using entity_create(). Where you have to focus your efforts will be sanitizing the information you "guzzle" off the web so that you do not create security issues in your site from XSS, SQLi, or other vulnerabilities. The process would be triggered on node-save.

This is just the basic idea, I cannot explain how to write the entire module as that would be pages of content.

1
  • Seems promising, I will try that as soon as possible then come back here. Developing a custom module was indeed the most obvious way.
    – Benj
    Jul 16, 2015 at 14:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.