MongoDB 3.0: Benefits of Compression with Limited IO

Posted on Feb 5, 2015

#mongodb

I re-ran my compression testing from November to show the on-disk savings that compression in WiredTiger allows in my last post. The other part of that testing was to see if we could show a benefit to having compressed data from a read perspective. As I explained back in November, one of the scenarios where compression can help is when there is only limited IO capacity available.

Once again, to simulate that, I used a relatively slow external USB 3.0 drive to simulate reasonable IO constraints and then recorded how long it took to read my entire data set from the on-disk testing into memory from that disk. The results show that benefit, though the results in WiredTiger are somewhat hampered by the issue described in SERVER-16444. I may attempt to re-run these tests with a more random, index driven read load – as well as being more representative of a normal workload, it should also show greater benefits by avoiding the table scan related overheads.

In any case, as you can see in the graph below, the benefits are clear with a 23.8% and 34.6% reduction in the time taken to read in the compressed data using snappy and zlib respectively (over the non-compressed data):

Time taken to load data from disk compressed versus uncompressed

Once again, I think this reinforces the choice of snappy as a good default for the WiredTiger Storage engine in MongoDB.

<div class="sd-content">
  <ul>
    <li class="share-facebook">
      <a rel="nofollow" data-shared="sharing-facebook-2214" class="share-facebook sd-button share-icon no-text" href="https://comerford.cc/2015/02/05/mongodb-3-0-benefits-of-compression-with-limited-io/?share=facebook" target="_blank" title="Click to share on Facebook"><span></span><span class="sharing-screen-reader-text">Click to share on Facebook (Opens in new window)</span></a>
    </li>
    <li class="share-twitter">
      <a rel="nofollow" data-shared="sharing-twitter-2214" class="share-twitter sd-button share-icon no-text" href="https://comerford.cc/2015/02/05/mongodb-3-0-benefits-of-compression-with-limited-io/?share=twitter" target="_blank" title="Click to share on Twitter"><span></span><span class="sharing-screen-reader-text">Click to share on Twitter (Opens in new window)</span></a>
    </li>
    <li class="share-google-plus-1">
      <a rel="nofollow" data-shared="sharing-google-2214" class="share-google-plus-1 sd-button share-icon no-text" href="https://comerford.cc/2015/02/05/mongodb-3-0-benefits-of-compression-with-limited-io/?share=google-plus-1" target="_blank" title="Click to share on Google+"><span></span><span class="sharing-screen-reader-text">Click to share on Google+ (Opens in new window)</span></a>
    </li>
    <li class="share-email">
      <a rel="nofollow" data-shared="" class="share-email sd-button share-icon no-text" href="https://comerford.cc/2015/02/05/mongodb-3-0-benefits-of-compression-with-limited-io/?share=email" target="_blank" title="Click to email this to a friend"><span></span><span class="sharing-screen-reader-text">Click to email this to a friend (Opens in new window)</span></a>
    </li>
    <li class="share-end">
    </li>
  </ul>
</div>

Share this: