1

We are working in website and integrating the Drupal Commerce in our website. We are also adding multiple product per product display to allow user to select products while adding product to cart.

The website will have large data set with more than 10 million product display (node) and more than 30 million products (drupal commerce).

We have also integrated apache solr with search api and allowing user to keyword search and filtering the search result.

Can anyone suggest that if we would stored such large data into single database, will it effect our performance or we need to stored data into 'master/slave' concept of database?

As we have large dataset, so we are preparing architecture of our database, so it would not effect performance of our website.

2
  • 1
    FYI: The answers below might not fully reflect this question because it's been edited.
    – nvahalik
    Dec 9 '15 at 17:44
  • The only change to the question's body was adding the link to the project page for Drupal commerce. I doubt that changes the meaning of the question.
    – apaderno
    Dec 9 '15 at 21:49
4

Talk to your hosting provider. If they can't answer these questions, get a new hosting provider.

These really are concerns you should bring to your hosting provider. A quality hosting provider will be able to meet these needs and tailor a solution that ensures smooth operation of a site as a whole and within the individual components: Solr, MySQL, Apache/Nginx/Litespeed/etc.

How will they do this?

  • Benchmarking - Before launch, they'll launch a version of the site and then work to ensure that certain performance criteria are met. They'll tweak and scale to meet these numbers.
  • Infrastructure - Just having a master/slave DB isn't necessarily a solution. You have to have good infrastructure and equipment backing it up. It also has to be architected well. Good providers will have this.
  • Multiple Layers of Scalability - Having multiple strategies to handle load or large sites is key. This means not just having a huge DB but also effective content distribution (CDN) and internal Drupal caching (Redis) in addition to strong infrastructure.
3
  • Just to add, hosting providers are not the only people who can help you with this. If you are using a platform solution such as AWS, there's lots of independent consultants who can put a solution together. Dec 9 '15 at 16:03
  • 1
    If you're using AWS and have questions like this, then you likely shouldn't be using AWS.
    – nvahalik
    Dec 9 '15 at 16:28
  • 1
    True. They certainly need some help from somewhere. Dec 9 '15 at 16:38
0

Performance in drupal have many aspects to deal with as it pops up you will Implement the needed solution personally I don't think the numbers you mentioned scared you to go with drupal.

Suppose you have a big number of products as you mentioned but still your store is not famous to have big traffic so every thing will be ok with default settings as a regular drupal website.

Search api with solr is a great step will serve you a lot, varnish, mongodb or redis will reduce load on DB servers.

0

The database layer and query builder allows drivers to specify case-insensitive operators. This allows MySQL to use LIKE() for auto-complete and user name lookups instead of LOWER(), allowing these queries to use indexes rather than doing a full table scan.

There is also built in master/slave support with the database layer, although Drupal can’t yet set up your database replication for you (or fold your laundry).

Proper planning of your database structure and optimization techniques will lead you to a very good solution.

I would suggest to have a look at this.

Not the answer you're looking for? Browse other questions tagged or ask your own question.