1

I am trying to import a csv file using parsecsv-for-php library.The given csv file is in Catalan language.

The thing is that when i just try to create nodes with given csv data it is throwing me this error like this while importing PDOException: in field_sql_storage_field_storage_write() (line 448 of /modules/field/modules/field_sql_storage/field_sql_storage.module).

To resolve this issue, what i did is that used utf8_encode function for each fields and avoided that error.

After successfully importing, the issue over here is that, this is the body text in csv

Thomas Alva Edison va anunciar la invenció del fonògraf l’any 1877, fent una demostració aquest mateix any. L’any 1878 el va patentar als Estats Units. Inicialment per enregistrar va fer servir cilindres de cartró recoberts d’estany, per finalment utilitzar cilindres de cera sòlida. Aquests van ser comercialitzats l’any 1889. La competència amb els discos sonors va acabar per eliminar els cilindres de cera cap els anys 1920s. Tot i així van sobreviure uns anys més com a suports d’enregistrament a oficines amb el dictàfon.

and the imported body text in the drupal i have displayed as image

enter image description here

If u compare both u will know there are certain differences in the body text such as

l`any in csv body text is getting imported in drupal as lany

d`estany is getting imported in drupal as destany

d`enregistrament is getting imported in drupal as denregistrament

I just want the same text to imported in drupal too with out missing any symbols or special characters. what am i doing wrong.

I have attached the my uploaded csv file formats too.

Only this character(') is not coming up properly in drupal.Any help on this is really appreciated.Thanks in advance

enter image description here

2 Answers 2

1

I found a way to overcome this issue. What I exactly did is that, in the imported view, I just added another charset=ISO-8859-1 which retrieves back the symbols or characters from db and displayed them properly.

<?php
drupal_add_http_header('Content-Type', 'text/html; charset=ISO-8859-1,utf-8');
$new_output = utf8_decode($output);
echo $new_output;
?>

That put back all special characters or symbols which was missing.

0

Can you check what markup gets saved to database? The table field_data_body seems to be in utf8 by default. That would rule down the possibility of field_formatters messing the characters in presentation layer.

If the data in database is corrupted, try getting the output of utf8_encode logged to some file for example. Is it possible to encode the csv to utf8 with some external tool before using the import tool?

File encodings are tricky and even using wrong encoding in module file can alter the markup. I need to output some xml-files as ISO-8859-1 to serve another system updated content from my service and even using ide-with default encoding can break the modules output by chanching file format. So narrow down more and you will find the break point.

The original pdo-exception could be indication that ´-char was read correctly, but assumed to be text separator. Using escaping should work in that case. In that case run the original importion with using something like $row_field = '"'.addslashes ( $row_field ).'"'; before saving to database.

Alternative to addslashes would be check_plain($row_field) .

If these dont work then its quite possible that the database doesnt support 'utf8mb4, which is "real" '.

5
  • Check is it Lany after utf_encode but before saving to database.
    – user20301
    Sep 12, 2013 at 6:08
  • The corruption seems to happen inside utf8_encode. You need to make sure that the source file is correctly using ISO-8859-1 to be able use utf8_encode. Alternatively i would suggest to use utf8 in the csv-file and dropdown any conversion. This way it wouldnt transform at all during processes at the server. But this can bring back the original PDO-exception if the csv is not encapsulated correctly. "title 1", "...invenció del fonògraf l’any ..."
    – user20301
    Sep 12, 2013 at 7:05
  • From personal experience "i think its utf8" is not nearly enough. Second, you cannot utf8_encode utf8 to utf8. 3 possibilities remain(1. csv row fields are not encapsulated correctly with eg "-char, ´-characters need to be escaped in csv or php php.net/manual/en/function.addslashes.php, your file is not really utf8 when running without utf8_encode or not really iso when using utf8_encode)
    – user20301
    Sep 12, 2013 at 7:22
  • I will update the answer while you edit the question, to avoid this becoming a chat.
    – user20301
    Sep 12, 2013 at 7:35
  • if i do like this $row_field = addslashes($row_field); it is showing the same error. PDOException: in field_sql_storage_field_storage_write() (line 448 of /modules/field/modules/field_sql_storage/field_sql_storage.module). If i do like this $row_field = utf8_encode ( $row_field ) i can able to avoid that error. Back to square one
    – Ram
    Sep 12, 2013 at 7:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.