I am working on a web application which will primarily be used by authenticated users. The web application imports records from CSV files using the Feeds module, MongoDB was chosen to store this data because it is much faster to do the writes rather than in MySQL. The application is importing thousands of records every day really fast. The records are referenced to the users and when they log in, they can consult their information. The application is also used as an API for the mobile app.
I have a 17gb AWS EC2 Instance as the server.
Varnish is install working perfectly serving content for anonymous users. I'm getting around 3000-4000 requests per second which is more than enough.
My problem is when benchmarking for authenticated users, on API pages that are only json content I get only 24 requests per second. And other pages I get like 9 - 11 requests per second. Here is some apache benchmark output.
Concurrency Level: 100 Time taken for tests: 9.154 seconds Complete requests: 100 Failed requests: 0 Write errors: 0 Total transferred: 1790900 bytes HTML transferred: 1731600 bytes Requests per second: 10.92 [#/sec] (mean) Time per request: 9154.469 [ms] (mean) Time per request: 91.545 [ms] (mean, across all concurrent requests) Transfer rate: 191.05 [Kbytes/sec] received
Memcache is installed, using memcache_storage module, for some reason I got slightly higher requests per second using this module rather than memcache. 4Gb are set as Memcache cache. Here is some Memcache statistics output.
Cache Information Current Items(total) 10730 (95109) Hits 644064 Misses 74324 Request Rate (hits, misses) 1.44 cache requests/second Hit Rate 1.29 cache requests/second Miss Rate 0.15 cache requests/second Set Rate 0.19 cache requests/second
APC is also installed, working perfectly.
I don't have many modules installed:
block, field, field sql storage, field UI, file, filter, image, list, locale, menu, node, number, options, path, system, taxonomy, text, user,
content access, varnish, ctools, eck, feeds, feeds ui, feeds tamper, feeds tamper ui, email, field permissions, hsm field, node reference, references, user reference, jquery update, mongodb, mongodb storage, mongodb watchdog, entity api, disable messages, image url formatter, job scheduler, libraries, quicktabs, quicktabs styles, apc, eck entitycache, entitycache, memcache storage, services, rest server, entityfieldquery views, views, views json, and views ui.
As for the Devel output, usually I get something like this. (This output was after I refresh the page)
Executed 57 queries in 10.79 ms. Queries exceeding 5 ms are highlighted. Page execution time was 167.27 ms. Memory used at: devel_boot()=1.36 MB, devel_shutdown()=13.96 MB, PHP peak=14.25 MB.
_drupal_session_write query is always the longest, it never executes in less than 8 ms, and sometimes it reaches 500 ms like in the output below.
Executed 57 queries in 607.49 ms. Queries exceeding 5 ms are highlighted. Page execution time was 2395.87 ms. Memory used at: devel_boot()=1.36 MB, devel_shutdown()=13.96 MB, PHP peak=14.25 MB.
These are my PHP settings in settings.php
/** *MongDB configuration */ $conf['mongodb_connections'] = array( // Connection name/alias 'default' => array( // Omit USER:PASS@ if Mongo isn't configured to use authentication. 'host' => 'mongodb://localhost', // Database name 'db' => 'mydb', ), ); $conf['mongodb_watchdog'] = 'mydb_logs'; $conf['field_storage_default'] = 'mongodb_field_storage'; /** *Memcached */ $conf['cache_backends'] = 'sites/all/modules/memcache_storage/memcache_storage.inc'; $conf['cache_default_class'] = 'MemcacheStorage'; $conf['cache_class_cache_form'] = 'DrupalDatabaseCache'; $conf['session_inc'] = 'sites/all/modules/memcache_storage/includes/session.inc'; $conf['lock_inc'] = 'sites/all/modules/memcache_storage/includes/lock.inc'; /** *Varnish */ $conf['cache_backends'] = 'sites/all/modules/varnish/varnish.cache.inc'; $conf['cache_class_cache_page'] = 'VarnishCache'; /** * Add APC Caching. */ $conf['cache_backends'] = 'sites/all/modules/apc/drupal_apc_cache.inc'; $conf['cache_class_cache'] = 'DrupalAPCCache'; $conf['cache_class_cache_bootstrap'] = 'DrupalAPCCache';
Configurations in my.cnf file.
key_buffer_size = 12M query_cache_size = 32M query_cache_limit = 2M query_cache_type = 1 # InnoDB caches. innodb_buffer_pool_size = 500M innodb_flush_method = O_DIRECT max_connections = 75 table_cache = 96 sort_buffer_size = 12M tmp_table_size = 12M
I really feel lost at this moment, I have read numerous articles about performance and scalability and I feel like I've done most of the things that are mentioned, but don't seem to work at all. I'm stressed because I feel like the server will crash really fast with this numbers. I hope someone can give me advices.