zlib库优化性能测试(port from facebook and intel)

背景

facebook基于zlib 1.2.5 做了些改进,并port了intel的一些相关补丁。本文主要目的是验证其压缩效果

主要有两个补丁:

https://github.com/facebook/mysql-5.6/commit/8ac9e7d219c8679b75da26e64d702b5517d05ea8 (Optimize zlib for a non-sliding window.)

https://github.com/facebook/mysql-5.6/commit/ed46ec9b4ef367f3335a531999f39cb1d9d40341 (port from intel)

intel补丁的主要修改:

1) By default, use CRC32 as the hash, computed using SSE4.2

2) Also provide a better fallback hash in case of no SSE

3) By default use double-byte comparisons by enabling UNALIGNED_OK

4) Use best_len_minus_1 instead of best_len in longest_match()

5) Minor optimization in deflate_slow()

修改MySQL

为cmake选修-DWITH_ZLIB增加新值

-DWITH_ZLIB= SYSTEM | BUNDLED | ENHANCED

增加目录zlib_enhanced,基于zlib-1.2.5版本,并打上补丁

对innodb块压缩的影响

测试环境:

某个典型的空间类型库, 300w行记录

使用如下语句测试导入速度:

load data infile ‘t1.data’ into table t1 character set gbk;

每组测试三次,取最优值

key_block_size = 8

zlib 1 min 37.81 sec
Zlib_enhanced 1 min 38.68 sec

除去误差因素,load数据的时间基本持平,原因是数据能够很容易压缩到8k,几乎无压缩失败

key_block_size = 4

zlib 6 min 0.88 sec
Zlib_enhanced 7 min 1.59 sec)

Load数据的性能提升约14.4%

对列压缩场景的性能影响

使用某个大字段数据集场景,大字段定义为压缩类型

load data infile ‘t1.data’ into table t1;

每组测三次,取最优值

zlib 4 min 2.18 sec
zlib_enhanced 3 min 50.50 sec

在该场景下,load数据性能提升较少,约4.8% ,压缩比无损失

结论

从测试的结果来看,zlib补丁的效果是有一定的提升的,测试过程中也验证了和老版本zlib的兼容性

原创文章,转载请注明: 转载自Simple Life

本文链接地址: zlib库优化性能测试(port from facebook and intel)

Post Footer automatically generated by wp-posturl plugin for wordpress.


Comments

Leave a Reply

Your email address will not be published. Name and email are required


Current month ye@r day *