Best method of importing a large csv file with fields containing multiple values and sub-fields like paragraphs and field collections

Question

I exported an old Drupal 7 site using the Views data export module. Now I need to import this back into my new Drupal 7 site, but some of my fields contain multiple values and fields that have sub fields in them (field collections).

• What would be the best method of importing this?
• Do I have to write my own custom module using the migrate module or use feeds?
• And how do I deal with importing data into a field collection or a field with multiple values?

Answer 1

You could try using Field Collection Feeds

Provide feeds integration for field collection module.

Answer 2

This took a good 2 weeks to figure out. Here it is. Comment if you need advice.

I had to import 20k records and each record had some fields with multi-value along with sub-values like the ones used in field collections and paragraphs. Also I need taxonomy terms to be populated on the fly as it was importing the paragraphs.

On my new Drupal 7 site, I did the following using Paragraphs instead of field collections due to it being deprecated in the coming year or so:

• Create the content type your making with all the fields matched up to the old field.
• Create all the paragraphs that match up with the sub-fields and make sure to set them with proper limit values as it would error out if u limit the values to 1.
• Setup your taxonomies
• Be sure to install: https://www.drupal.org/sandbox/shaney/2862236 --- Major recoding is needed to fit how you want it to fit your data. I had to modify alot of the code there as it wasn't fully finished.
• In the feeds importer. Pick processor as Node processor and pick the content type your targeting
• Start mapping out your fields. The source column MUST match what you have in the csv. It could be in any order, it doesn't matter as it will know by the column name to target name.
• The trickiest part was the paragraph mappings as it involves formatting the data properly with properly used deliminator's if you have subfields of subfields. Below is a quick screen shot of what I have when mapping paragraph fields.
• Also I had to split up my 20k record file into 1500 records per a file. As my import was crashing if it went over 2000 records. I used the terminal command "split -l 15000 myfile.csv" to get it split up. then do each one at a time and cross your fingers it doesnt mess up.

I had to modify the paragraphs mapper to look like this. By default the mapper can only do 1 field. I just needed max 5 fields so I added them in manually. The proper way to do this is provide an add more fields functionality. I didn't get to do this as I only needed 5 fields mapped.

This is how it looks when an import is complete: