What does it mean if an item in Search API has a "-1" flag and is never indexed?

Question

Search API uses flags in the changed column of the search_api_item table to store item-level information relating to indexing. The -1 flag (something to do with queuing items?) appears to be crucial to a problem I'm facing, where it is set on items which are then never indexed.

I've studied the code and all the documentation and relevant issues I can find, and I can't figure out in nearly enough detail what the practical implications of this flag are: what it being set causes or prevent from happening (and therefore, where I should focus next in trying to solve my problem). Since Search API is an API module which plenty of developers work with, and these flags are a core part of its logic and an important part of the contrib-accessible data it records, I'm hoping that someone will be able to help explain what a -1 flag means and what it causes to happen (or what it prevents).

For context, the specific problem spurring me to ask (with Search API using Search API Solr on Drupal 7) is that indexing stops abruptly - cron runs continue to work fine, cron requests reach the search server with no problem, but a large portion of my indexable items are never indexed. There are no error messages or relavant information in any of the appropriate logs (drupal watchdog, cron, apache solr, tomcat). It's a classic debugging nightmare: something is failing silently and leaving no trace. But there's one clue: in the search_api_item table on the database, I can see that, when Cron reaches those items where indexing is failing, the changed column changes to a '-1'. The notes for that column say:

Either a flag or a timestamp to indicate if or when the item was changed since it was last indexed.

After much digging in code I found what appears to be the place these flags are set ( in search_api/includes/datasource.inc ) and there appear to be three such flags:

• -1, set in trackItemQueued(): the one which looks to be crucial to my problem. According to the comments it means "queued" - but I can't find the crucial details of what this causes to happen (or prevents from happening) next. If I've understood this right, then this flag being set should be a sign that everything is working fine: but the items aren't indexed, and I can't see what the expected next step should be.
• 0, which apparently means already indexed (presumably causing these items to be passed over)
• 1, set in trackItemInsert(), and I can't figure out anything about what this one means
• (and timestamps apparently signify "changed/dirty", which I imagine means "no longer matches the index data as of [timestamp], needs to be queued for re-indexing")

Search API is an API module, so there will be a good number of Drupal developers who know how it works on the inside. Do you know what the -1 flag indicates - in terms of what then happens to items flagged ("queued", or, "marked to be queued") in this way? What is the role of this flag in the process and what happens next?

---

(bonus points for any expert advice on the next step to take in debugging a problem like mine: but the focus of my question is general, about understanding the Search API item flags. My issue and attempts at debugging it are intended mainly as context. In case it's relevant further context, I'm also running Ultimate Cron and its dependency Background Process to handle the cron runs, to reduce search server strain by allowing lots of small indexing batches called by cron every few minutes without overdoing other drupal cron functions. This part of the process seems to work fine: the search server is reached perfectly in sync with the Ultimate Cron schedule as expected. It's what happens after the item flags are checked and updated that is leaving me stumped)

---

(I've also posted a related support request on the Search API issue queue, without a response. I'm judging it's okay to ask here as well because a) this meta discussion explicitly says that also posting questions that are support questions also asked in issue queues here as well is okay and also a desirable way to take the pressure off busy module maintainers, and b) this is particularly true for Search API as the maintainer has said in the past that he doesn't have much time available to help with support issues. As adivsed on Meta, I'll update both threads with any relevant information)

Answer 1

-1 means that it's added to the queue system. By default, that means the queue table in the database. It should create a queue task for groups of id's matching your batch size.

If they aren't run, this either means that the queue system does not run at all (It should be executed in the default cron implementation) or that some other stuff is filling up the queue. The following query should give you an idea of what's in the queue: SELECT name, COUNT(*) FROM queue GROUP BY name.

Some hints:

• There was a quite major bug in the update.module that caused it to create possibly thousands of duplicate queue tasks. If you can see many fetch_tasks with the above query, you can delete them and clean the cache_update table to have them re-added once. Make sure you're running an up to date Drupal core version to prevent this from re-appearing.
• Maybe one of your cron implementations is running into a timeout or fatal error. You could try a module like Elysia cron to get detailed stats about how much time each cron implementation uses. It probably is something that is run after search_api.module because it seems to be able to create the queue tasks. xmlsitemap.module is a candidate that can take a huge amount of time if you have many nodes.
• You're using an alternative cron module, that is not properly executing queue tasks.

You can also try http://drupal.org/project/queue_ui, it provides an UI to manage the default queue implementation and provides the same information as the above query, but it's not actively maintained and broken without applying a patch from the issue queue.