Massive conditionals in field_sql_storage_field_storage_load query

Question

In Drupal 7, I've been struggling with a slow local (and remote dev) site for the past few days. If I try to edit a field on a Message entity type, it hits my 720 sec execution time limit (normally 180 secs, I wanted to see if it would ever finish.)

I have traced the slowness back to field_sql_storage_field_storage_load in field_sql_storage.module.

That function loops over fields to obtain entity ids;

foreach ($fields as $field_id => $ids) {

Heres were the fun starts. I have a handful of field_id's (100-odd) but on the field_id's that I've var_dump'ed, there have been over 30,000 id's!

Ie;

array (size=85)
  2 => 
    array (size=38942)
      0 => string '1' (length=1)
      1 => string '2' (length=1)
      2 => string '3' (length=1)
      3 => string '4' (length=1)
      4 => string '7' (length=1)
      5 => string '8' (length=1)
      6 => string '9' (length=1)
      7 => string '10' (length=2)
      8 => string '11' (length=2)
      9 => string '12' (length=2)
      <snip> ... goes for 30,000 id's

Another way to look at it, is inside the query that it is trying to execute;

AND (entity_id IN (:db_condition_placeholder_1, :db_condition_placeholder_2, :db_condition_placeholder_3, :db_condition_placeholder_4, <snip> all the way to db_condition_placeholder_38838

Unfortunately, the site is still slow to output my var_dump's, so there is something slowing it down a lot before it even gets here.

What is happening here? Why are we having to perform such massive conditions in our queries?

It doesn't appear to happen on the production site, but maybe thats because production site uses memcache? Might not even use this function at all.

Would love any help, thanks.