2

To the best of my knowlege, I could not find a answer that applies to my situation, so here it goes:

I am using the feeds import to add a series of roughly 6500 employees as nodes (not preferred solution, I know... Blame my project manager).

I am importing a XML:

<OS2>
  <Persons>
    <Person>
      <Brugernavn>UsernameFromExternalSource</Brugernavn> **<<-- Never Empty, but also not unique**
      <Navn>RealName</Navn> **<<-- Never Empty, but is not unique. Serves as node title**
      <Stilling></Stilling> **<<-- Can be empty**
      <Telefon></Telefon> **<<-- Can be empty**
      <Mobil></Mobil> **<<-- Can be empty**
      <Afdeling></Afdeling /> **<<-- Can be empty, usually is not. Inserts into Taxonomy**
      <Email>some@name.com<Email /> **<<-- GUID as it is the only one that will always be unique**
      <Kompetencer></Kompetencer> **<<-- Can be empty**
      <Opgaver></Opgaver> **<<-- Can be empty**
    </Person>
  </persons>
</OS2>

I've mapped the items with the proper Xpath, but when I run the import, all the items are being imported properly, but sometimes, very randomly, some items are being imported twice as seperate nodes (meaning I will have /employee/name and /employee/name-0 )

I checked the feeds_item table, and there are several instances of the guid column, as well as the hash, having the same value twice, wich as I understand it, should'n happen.

I am at a loss, and I've tried to convice the people who have the final say to let me use alternatives, but that is sadly out of the question, as well as getting the XML altered to have it's own numeric unique-ID.

Futher data that might help:

  • I recive no error messages in the apache log, or the Drupal log
  • As far as I can tell, all elements are being imported properly, in some cases with said duplicates
  • The server does not timeout, and has 8GB of ram assigned to handle the import

Does anyone have an idea of what is going wrong?

Thanks in Advance.

  • Do you have duplicates in your source data or not? That's the biggest question. If you do not, it is simply a feeds import bug. If you do, well, you need to write some workarounds. It is not impossible to have two kids with the same first and last name in one classroom, I've seen that happening, so two identical employees may be legit. But identical guid seems weird. – Mołot May 14 '13 at 9:52
  • 1
    There are no duplicates emails, wich are the ones I am using to identify uniques, in the XML file (Just made a check to be certain). And as you said, for some reason the import allows for duplicate GUID's, wich is why I am also puzzled. – Roald Jean Storbank May 14 '13 at 9:58
  • Not a solution but a debug idea - make column unique in the database, run import, see where it will fail. – Mołot May 14 '13 at 10:00
  • 1
    Either my lack of MySQL management knowledge is to blame, or I can't seem to make that col unique, as I get this when I try to convert it into a unique: "#1170 - BLOB/TEXT column 'guid' used in key specification without a key length" What is really odd, is the fact that there seems to be no obvious pattern to when duplicates are created and when not. – Roald Jean Storbank May 14 '13 at 10:36
  • My bad - MySQL cannot index blobs. My idea was to make import fail by abusing constrains, in hope that error log would provide more data to investigate randomness. This approach worked for me couple of times already, but it seems this module is not going to make it easy. The weird workaround is to create varchar column with default CONVERT(VarChar(40), `guid`) and make THAT unique. But I'm not sure if it's worth it. and of course tailor varchar's length to longest mail you expect, 40 is just a guess. – Mołot May 14 '13 at 10:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.