0

I have a site that will need to deal with thousands of nodes. it's an internal site, not public facing, with a daily concurrent user base of no less than 80 and no more than 100 users.

I am trying to figure out how to deal with search and indexing. The Big problem is I need facets and whatnot, but I also need instant data retrieval. So users can't wait for indexes to catch up. can one index via SearchAPI near instantly?

On average, there might be 2-300 new nodes created per day, every day, by users, and an additional 5-600 nodes created per day+ through automated imports (Feeds).

I was thinking of just using straight views, and trying to do search filters, but it's not scalable and after a few months to a year it would be quickly unmanageable.

TL:DR: Many many nodes per day need indexed search with facets (SearchAPI and FacetAPI using ElasticSearch). Near instant retrieval required. If a user creates a node, they need to be able to see that node in the index within a few seconds or less.

Alternatively, thinking out loud, could I have it submit the new node to the index on every new node addition?

1

Regarding Search, I would suggest to go with external Search System, like ApacheSolr, Elastic Search, Algolia, etc. This will make sure that database will not be overloaded with content index.

I'm working on a project where we currently have almost 2 Lac nodes and around 30-500 nodes created every day. The number of concurrent users are 1-20 max. We're using ApacheSolr (Solr hosted on different server) with Search API and Search API Solr Module and results are retrieved in less than 1-2 Seconds (The Project is internal of Client's Company so can't share the link here).

I was thinking of just using straight views, and trying to do search filters, but it's not scalable and after a few months to a year it would be quickly unmanageable.

Using view for rendering search results will not help much in terms of instant results. Preferred way will be writing custom code to query based on keyword and show results based on requirement.

Alternatively, thinking out loud, could I have it submit the new node to the index on every new node addition?

Indexing of Node after Creation/Updates are managed by module. The search indexing is normally done when running Cron. So setting up your site Cron to run every minute will make sure that the nodes are indexed instantly.

Hope this helps.

  • This basically confirms my thoughts on this. Thanks. – Remog Sep 16 '16 at 14:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.