Achieving Top Network Performance - Red Hat Summit
Achieving Top Network Performance - Red Hat Summit Achieving Top Network Performance - Red Hat Summit
CSTATE default – C7 on this configpk cor CPU %c0 GHz TSC %c1 %c3 %c6 %c7 %pc2 %pc3 %pc6 %pc7 SMIs0.04 1.43 2.19 0.08 0.00 0.00 99.89 4.46 0.00 93.94 0.00 00 0 0 0.41 1.28 2.19 0.93 0.01 0.00 98.66 3.13 0.01 93.91 0.00 00 1 1 0.04 1.66 2.19 0.06 0.00 0.00 99.91 3.13 0.01 93.91 0.00 00 2 2 0.01 1.73 2.19 0.01 0.00 0.00 99.98 3.13 0.01 93.92 0.00 00 3 3 0.01 1.72 2.19 0.02 0.01 0.00 99.96 3.13 0.01 93.92 0.00 00 4 4 0.01 1.85 2.19 0.01 0.00 0.00 99.98 3.13 0.01 93.92 0.00 00 5 5 0.01 1.94 2.19 0.01 0.00 0.00 99.98 3.13 0.01 93.91 0.00 00 6 6 0.01 1.92 2.19 0.02 0.00 0.00 99.98 3.13 0.01 93.91 0.00 00 7 7 0.01 1.76 2.19 0.01 0.00 0.00 99.98 3.13 0.01 93.91 0.00 01 0 8 0.01 1.71 2.19 0.02 0.01 0.00 99.96 5.80 0.00 93.96 0.00 01 1 9 0.01 1.69 2.19 0.02 0.01 0.00 99.97 5.80 0.00 93.96 0.00 01 2 10 0.01 1.75 2.19 0.02 0.00 0.00 99.97 5.80 0.00 93.96 0.00 01 3 11 0.01 1.83 2.19 0.02 0.00 0.00 99.97 5.80 0.00 93.96 0.00 01 4 12 0.01 1.84 2.19 0.02 0.00 0.00 99.97 5.80 0.00 93.96 0.00 01 5 13 0.01 1.91 2.19 0.02 0.00 0.00 99.98 5.80 0.00 93.96 0.00 01 6 14 0.01 1.96 2.19 0.02 0.00 0.00 99.98 5.80 0.00 93.96 0.00 01 7 15 0.01 2.38 2.19 0.03 0.00 0.00 99.96 5.80 0.00 93.96 0.00 0
CSTATE disabled – Note speedpk cor CPU %c0 GHz TSC %c1 %c3 %c6 %c7 %pc2 %pc3 %pc6 %pc7 SMIs100.00 2.69 2.19 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 00 0 0 100.00 2.69 2.19 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 00 1 1 100.00 2.69 2.19 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 00 2 2 100.00 2.69 2.19 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 00 3 3 100.00 2.69 2.19 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 00 4 4 100.00 2.69 2.19 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 00 5 5 100.00 2.69 2.19 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 00 6 6 100.00 2.69 2.19 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 00 7 7 100.00 2.69 2.19 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01 0 8 100.00 2.69 2.19 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01 1 9 100.00 2.69 2.19 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01 2 10 100.00 2.69 2.19 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01 3 11 100.00 2.69 2.19 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01 4 12 100.00 2.69 2.19 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01 5 13 100.00 2.69 2.19 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01 6 14 100.00 2.69 2.19 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01 7 15 100.00 2.69 2.19 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0
- Page 2 and 3: Achieving Top NetworkPerformanceMar
- Page 4 and 5: Take Aways●●●Awareness of the
- Page 6 and 7: Some Quick Disclaimers●●●We d
- Page 8 and 9: Agenda● Why Bother ?●40 gbit, g
- Page 10 and 11: Teaser 1 - Glustereffect of net.cor
- Page 12 and 13: Teaser 3 - latency● Font size 28
- Page 14 and 15: Memory Characteristics●●●Memo
- Page 16 and 17: “Issues” that NUMA makes visibl
- Page 18 and 19: NUMA - Latency[root@perf ~]# numact
- Page 20 and 21: PCI Bus - and related issues●●
- Page 22 and 23: 40 Gbit Gen3 vs 10 Gbit PCI Gen2 la
- Page 24 and 25: CPU Characteristics - Basics●●
- Page 26 and 27: CPU - Performance Governors●echo
- Page 30 and 31: NPtcp latency vs cstates - c7 vs c0
- Page 32 and 33: RHEL6 “tuned-adm” profiles# tun
- Page 34 and 35: Kernel Bypass Technologies - Pros a
- Page 36 and 37: Offload - Solarflare OpenOnloadAver
- Page 38 and 39: KVM Network ArchitectureVirtioConte
- Page 40 and 41: KVM Network Architecture - vhost_ne
- Page 42 and 43: Latency comparison - RHEL 6Network
- Page 44 and 45: Host CPU Consumption virtio vs vhos
- Page 46 and 47: KVM Network Architecture - PCI Devi
- Page 48 and 49: KVM Network Architecture - SR-IOV
- Page 50 and 51: KVM Architecture - Device Assignmen
- Page 52 and 53: RHEL6 - new features●●●●●
- Page 54 and 55: RHEL6 - new features●●Add getso
- Page 56 and 57: Receive Steering - improved message
- Page 58 and 59: Tuning Knobs - Overview●●●By
- Page 60 and 61: sysctl - View and set /proc/sys set
- Page 62 and 63: sysctl - TCP related settings●TCP
- Page 64 and 65: Why Bother ? - Teaser 1effect of ne
- Page 67 and 68: lspci - details# lspci -vvvs 81:00.
- Page 69 and 70: Why Bother - A quick teaser● ifco
- Page 71 and 72: Tuning- first pass bottleneck resol
- Page 73 and 74: Tuning - second pass setup●●●
- Page 75 and 76: Tuning - irqbalance disabled, netpe
- Page 77 and 78: Tuning - second pass●mpstat on th
CSTATE disabled – Note speedpk cor CPU %c0 GHz TSC %c1 %c3 %c6 %c7 %pc2 %pc3 %pc6 %pc7 SMIs100.00 2.69 2.19 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 00 0 0 100.00 2.69 2.19 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 00 1 1 100.00 2.69 2.19 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 00 2 2 100.00 2.69 2.19 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 00 3 3 100.00 2.69 2.19 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 00 4 4 100.00 2.69 2.19 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 00 5 5 100.00 2.69 2.19 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 00 6 6 100.00 2.69 2.19 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 00 7 7 100.00 2.69 2.19 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01 0 8 100.00 2.69 2.19 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01 1 9 100.00 2.69 2.19 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01 2 10 100.00 2.69 2.19 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01 3 11 100.00 2.69 2.19 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01 4 12 100.00 2.69 2.19 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01 5 13 100.00 2.69 2.19 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01 6 14 100.00 2.69 2.19 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01 7 15 100.00 2.69 2.19 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0