Does compression in ZFS improve performance
First I grabbed a bunch of files from out of my documents folder. Combination of pdf, docx, txt, sql, etc.. I just ket grabbing files until I got to 3.9GB. I then put them in a 7zip archive using no compression, store only. So nowIi have one single file called test that is 3.9GB in size. This is the file I tested with.
First test was to transfer the files from my SSD OS drive to the zpool called raid0. Btw it is not a raid0, it is simply two 500GB WD blue drives mirrored. I use rsync --progress so i can see whats happening during the transfer. Results for no compression on zpool raid0:
[root@test raid0]# rsync --progress /root/test /raid0/test
test
4190667460 100% 162.84MB/s 0:00:24 (xfer#1, to-check=0/1)
sent 4191179086 bytes received 31 bytes 171068535.39 bytes/sec
total size is 4190667460 speedup is 1.00
As you can see I get 162.84MB/s transfer. Not bad
Next test is I remove the file from the zpool and turn on compression, and repeat transfer
[root@test raid0]# rm *
rm: remove regular file `test'? y
[root@test raid0]# zfs set compression=lz4 raid0
[root@test raid0]# ls
[root@test raid0]# rsync --progress /root/test /raid0/test
test
4190667460 100% 176.71MB/s 0:00:22 (xfer#1, to-check=0/1)
sent 4191179086 bytes received 31 bytes 186274627.42 bytes/sec
total size is 4190667460 speedup is 1.00
As you can see my transfer was s bit faster. This time 176MB/s. I also checked my compression ratio after the transfer on the zpool.
[root@test raid0]# zfs get compressratio raid0
NAME PROPERTY VALUE SOURCE
raid0 compressratio 1.19x -
So yes I got better speed and saved disk space. A win win I would think. But that compression on the fly does cost. I am running this on an Intel i5-4690k CPU @3.50/4.0 GHZ with 16GB DDR3 1600 RAM (8GBx2). Granted I just eyeballed the CPU load in htop but without compression I was looking at approximately 20% load. With compression a whopping 40+%.
I added the third 500GB WD blue drive and did a bit more testing
In a 3 drive raid0 using three 500GB WD blue drives with no compression I got 300MB/s transfer speed from seperate SSD drive to zpool. Usable space 1.5TB. Can stand ZERO failed drives.
In a 3 drive raidz using three 500GB WD blue drives with no compression I
got 260MB/s transfer speed from seperate SSD drive to zpool. Usable space 1TB, Can stand ONE failed drive.
In a 3 drive raidz2 using three 500GB WD blue drives with no compression I
got 160MB/s transfer speed from seperate SSD drive to zpool. Usable space 500GB. Can stand TWO failed drives
No comments:
Post a Comment