0

I want to scrap location data from google. So for example; If I search for "Restaurants in Miami". I get over 14K results. I reviewed; How to import Nodes and Node Location data using Feeds module?. But that doesn't really help me with importing a query term from google.

  1. How would I be able to scrap each one of the locations that the query gives as a result and import them as a node into drupal?
  2. Are there any pre-built modules that might assist in this process?
  3. I'm I going to need to build a custom module for this use case?

UPDATE: I've found this page; http://drupal.org/project/example_web_scraper and will studying the info to see if this is what I'm looking for.

AH HA MOMENT:

Improve this question
2
  • What exactly do you want to do?
    – BetaRide
    Commented Jul 14, 2012 at 7:10
  • It's scrape, not scrap.
    – user7667
    Commented Jul 16, 2012 at 18:14

1 Answer 1

Reset to default
5

Not to put a damper on things (I've written more scrapers than I care to remember and it's always fun), but such an act is specifically against Google's Terms of Service:

Do not misuse our Services, for example, do not interfere with our Services or try to access them using a method other than the interface and the instructions that we provide.

I'm sure you'll be ok if you're doing this for fun/educational purposes (don't take my word for it though, I'm not a lawyer!), but if you make something commercial out of this you could find yourself in a tricky position.

Improve this answer
5
  • I really just want to pull the location data to create nodes so I don't need to do it by hand. Doesn't Google have its MAPS API to do this very thing?
    – Digital fire
    Commented Jul 16, 2012 at 16:59
  • Also found this; code.google.com/p/gempaq/wiki/GMaps_GeoRSS_Feeds_parser. I know its for D6 but its similar to what I need to do.
    – Digital fire
    Commented Jul 16, 2012 at 17:03
  • Absolutely...but that's a stand-alone API, governed by its own set of rules (it's not related to search in their opinion, and definitely wouldn't be considered scraping, which has its own set of connotations). If you want to get location data in accordance with the terms of service then one of the APIs is the only way to go really. There's a dev version of Google Maps API v3 that might be worth a look
    – Clive ♦
    Commented Jul 16, 2012 at 17:04
  • Again, that's fine as you're parsing data that Google have specifically shared, it's not scraping as such. Sorry if I've got hung up on that word too much :)
    – Clive ♦
    Commented Jul 16, 2012 at 17:07
  • Interesting. Going to take a look at that later today. Thank you @Clive.
    – Digital fire
    Commented Jul 16, 2012 at 17:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.