Effect of copying on latency
Test
The goal of this test was to show, how copying the message data (on the application level) affects the latency. One test was standard ØMQ latency test (local_lat and remote_lat) the other one was modified version of the test that copied the message data once on each peer.
The test results were provided by Michael Santy.
Environment
- ØMQ/2.0 beta 2
- CPU: Intel Core i7 940 @ 2.93 GHz
- Memory: 12GB DDR3 @ 1333 MHz
- IB HCA: Mellanox MT25204 — InfiniHost III Lx HCA
- IB Switch: Mellanox MT47396 Infiniscale III
- OS: Centos 5.4 x86_64
- OFED: 1.4.1
- libSDP: 1.1.99
Results
| message size | latency with copying | latency w/o copying |
|---|---|---|
| 4 kB | 45.856 us | 45.471 us |
| 8 kB | 55.690 us | 57.462 us |
| 16 kB | 81.309 us | 79.635 us |
| 32 kB | 99.423 us | 97.009 us |
| 64 kB | 164.521 us | 152.227 us |
| 128 kB | 227.577 us | 192.544 us |
| 256 kB | 371.325 us | 307.026 us |
| 512 kB | 682.949 us | 528.641 us |
| 1 MB | 1464.738 us | 1125.205 us |
| 2 MB | 2697.648 us | 2085.429 us |
| 4 MB | 5045.856 us | 3714.495 us |
| 8 MB | 9662.857 us | 7199.121 us |
| 16 MB | 20898.646 us | 13861.691 us |
| 32 MB | 44297.914 us | 27565.414 us |
| 64 MB | 87754.242 us | 54715.375 us |
| 128 MB | 167838.859 us | 108787.906 us |
| 256 MB | 327360.406 us | 216642.000 us |
Here are the results of the test in the form of graph. Red line represents latency with copying, while the black line represents latency without copying:
The same results displayed in log/log graph to make the smaller message sizes more readable:
