I tried Tokudb as well and below are the stats comparison for complex SUM and COUNT query.
Please note that the number of rows and size is exactly the same and there is a partition on a Date column
Table size 10GB and rows 12 Million
Also note that I am using the default settings of Tokudb
With Tokudb - 1 min 41.38 sec
With Innodb - 58.17 sec
Recently i have got the task to work on a 10 GB and 12 Million rows table to make it work with complex aggregation queries like SUM or COUNT.
Server - Ubuntu 14.0.4
Mysql - Percona XtraDB Cluster 5.6
RAM - 16 GB
CPU - 8 Cores - 2.95 Ghz
SSD - 100 GB
I tried many options --
1) Increased the innodb_buffer_pool_size to 8GB on a 16GB RAM machine -- It helped but nothing magical here
2) Add few more Keys on Date based columns and force the users in front end to select at least one date range column -- Seen some performance gain here
3) Tried MyISAM engine -- I would say in present days MyISAM is history since Innodb itself is pretty much comparable to MyISAM -- so didn't see amy much performance gain .. one disadvantage was that loading of this 12 Million rows took ages .. and also "SELECT table_rows FROM information_schema.tables" had hanged , so i was not able to to figure out that how many rows have been loaded in my Table.
4) Finally I tried Partitioning the Table based on a DATE column -- Massive Performance gain .. If the user has selected a date range which falls under one or two partitions , then you can get the results very fast , even if it falls in many partitions still the performance is acceptable .. Useful ref - http://www.slideshare.net/datacharmer/mysql-partitions-tutor... -- remember that its Mandatory that you add that date column as part of Primary Key if you are Partitioning on that date column .
yeah we can wait for later stages for the implementation in demo but in the meantime any documentation or pointers on how to achieve the direct download of results..
In Demo it says to get see the data in browsers console.. thats cool .. but how do we Download the data , like the results should directly prompt a Save dialog window..
Cool , this looks very good. In the page https://www.empatica.com/info.php there are few projects been mentioned something like "Parkinson's disease early detection" ..are there any corresponding links for this to get more info as in how does it detect the Parkinsons disease in early stages.
This is rocksolid basic script..one quick question..if you post on every event then dont you think so that it will generate huge traffic..how did you handle that
Few basic things which can be done is like .. install Fail2ban , change the default SSH port to something else, and use PermitRootlogin as No in sshd_config file
Does such Webscraping is allowed legally. Since it is not done directly from our servers and if any legal action will be taken by the scraped website , will it be on kimonolabs..or the user..