1

I have a simple three fielded (name, address, country) content in drupal. I have a csv containing 1,000,000 row lines of data. I need to fill up all drupal content type I made with the csv data.

First, I used FEEDS module but the importing is unacceptably too long. I tried to insert the data directly into MYSQL (ex: field_data_field_name and field_data_revision_name). It is successfully inserted into MYSQL but it does not show/update in the DRUPAL content.

How to make the Drupal synchronize with the newly added data in the table of MYSQL?

  • have you tried to use Batch API? – Aboodred1 Apr 18 '14 at 4:32
  • Thanks for all the comments. After going through some searching I have successfully uploaded the huge csv file using the following: 1) in the same MYSQL database I created a new table (say ABC) 2) uploaded the 1000000 row data via MYSQL statement "LOAD DATA CONCURRENT LOCAL INFILE…" 2) then in drupal I used the sql parser, mapped the ABC table to the Drupal node and imported the MYSQL ABC table to the DRUPAL content type that I made. These address the a) synchronization of the new data inserted directly into MYSQL and the DRUPAL nodes b) speeded up the importing of data. – RT Tan Apr 20 '14 at 6:14
0

Not sure but could you design your view fields [name, address, country] the way you want to use them. Then enter unique data.

Then look in the DB for the unique data to be sure you have the right fields.

After backup and site shut down you could import your csv right into mysql. With the db interface. Make sure you backup The tables before starting.

A bigger question perhaps is how will you work with the data.

Not the answer you're looking for? Browse other questions tagged or ask your own question.