Cisco Nexus9K/7K/6K/5K/3K , ACI, UCS, VMware Jumbo Frame 구성 가이드 입니다.

1. Cisco Systems Korea
DC TSA / (whchoi@cisco.com)

2. Jumbo Frame 설정 고려사항
2
VM
VMkernel
Storage
vSwitch
vSwitch
UCS FI
VM
VMkernel
Storage
vSwitch
vSwitch
UCS FI
Switch
(Nexus or ACI)
J
J J J
J J
J J J
JJ J


J

3. Nexus 7K/9K/3K Jumbo Frame 설정 방법
3
N9396-DC-OOB# sh running-config interface ethernet 1/1
Interface Ethernet1/1
description To-LAB-Backbone
switchport access vlan 2
mtu 9216
interface Ethernet1/46
no switchport
mtu 9216
ip address 1.1.1.1/24
interface Vlan118
no shutdown
mtu 9216
ip address 172.16.100.1/24

4. Nexus 7K/9K/3K Jumbo Frame 설정 방법
4
## L2 Interface ##
N9396-DC-OOB# show interface ethernet 1/1
Ethernet1/1 is up
admin state is up, Dedicated Interface
Hardware: 100/1000/10000 Ethernet, address: 58f3.9ca3.deb2 (bia 58f3.9ca3.deb2)
Description: "To-LAB-Backbone"
MTU 9216 bytes, BW 1000000 Kbit, DLY 10 usec
N9396-DC-OOB# show interface vlan 118
Vlan118 is up, line protocol is up, autostate enabled
Hardware is EtherSVI, address is 58f3.9ca3.deb1
Internet Address is 172.16.100.1/24
MTU 9216 bytes, BW 1000000 Kbit, DLY 10 usec
….
RX
9953327437 unicast packets 7854337 multicast packets 4271679 broadcast packets
0 jumbo packets 0 storm suppression packets

5. Nexus 6K/5K/3K Jumbo Frame 설정 방법
5
N6001-A(config)#policy-map type network-qos jumbo
N6001-A(config-pmap-nq)# class type network-qos class-default
N6001-A(config-pmap-nq-c)#mtu 9216
N6001-A(config-pmap-nq-c)#system qos
N6001-A(config-sys-qos)#service-policy type network-qos jumbo
policy-map type network-qos jumbo
class type network-qos class-fcoe
pause no-drop
mtu 2158
class type network-qos class-default
mtu 9216
system qos
service-policy type network-qos jumbo
interface Ethernet1/5
description "TO-POD1-N5K-A-5"
no switchport
mtu 9216
vrf member IPN
ip address 56.56.56.2/30

6. Nexus 6K/5K/3K Jumbo Frame 설정 방법
6
## L2 Interface ##
N6001-A# show queuing interface ethernet 1/5
Ethernet1/5 queuing information:
TX Queuing
qos-group sched-type oper-bandwidth
0 WRR 100
1 WRR 0
Statistics:
Mcast pkts tail dropped on this egress asic : 0
RX Queuing
qos-group 0
q-size: 100160, q-size-40g: 100160, HW MTU: 9216 (9216 configured)
drop-type: drop, xon: 0, xoff: 0
Statistics:
Pkts received over the port : 1059
Ucast pkts sent to the cross-bar : 1053
Mcast pkts sent to the cross-bar : 6
Ucast pkts received from the cross-bar : 90936
Pkts sent to the port : 90936
Pkts discarded on ingress : 0
Per-priority-pause status : Rx (Inactive), Tx (Inactive)

7. Nexus 6K/5K/3K Jumbo Frame 설정 방법
7
## L3 Interface ##
N6001-A# show interface ethernet 1/5
Ethernet1/5 is up
Dedicated Interface
Hardware: 1000/10000 Ethernet, address: 002a.6a96.5cbc (bia 002a.6a96.5c6c)
Description: "TO-POD1-N5K-A-5"
Internet Address is 56.56.56.2/30
MTU 9216 bytes, BW 10000000 Kbit, DLY 10 usec
....
RX
12982 unicast packets 242004 multicast packets 6 broadcast packets
254995 input packets 34035648 bytes
71 jumbo packets 0 storm suppression bytes

8. ACI Jumbo Frame 설정 방법
8
DC1-LEAF-101# show interface ethernet 1/6
Ethernet1/6 is up
admin state is up, Dedicated Interface
MTU 9000 bytes, BW 10000000 Kbit, DLY 1 usec
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation ARPA, medium is broadcast
Port mode is trunk
RX
packets
130208122 input packets 42866109702 bytes
11545414 jumbo packets 0 storm suppression bytes

9. ACI Jumbo Frame 설정 방법
9
DC1-LEAF-101# show interface ethernet 1/6
Ethernet1/6 is up
admin state is up, Dedicated Interface
Belongs to po1
MTU 9216 bytes, BW 10000000 Kbit, DLY 1 usec
packets
11576870 jumbo packets 0 storm suppression bytes

10. UCS FI Jumbo Frame 설정 방법
10

11. UCS FI Jumbo Frame 설정 방법
11

12. UCS FI Jumbo Frame 설정 방법
12
DC2-FI-A(nxos)# show policy-map
Type qos policy-maps
====================
policy-map type qos default-in-policy
class type qos class-default
set qos-group 0
policy-map type qos system_qos_policy
class type qos class-fcoe
set qos-group 1
class type qos class-platinum
set qos-group 2
class type qos class-silver
set qos-group 4
class type qos class-bronze
set qos-group 5
class type qos class-gold
set qos-group 3
class type qos class-default
set qos-group 0

13. UCS FI Jumbo Frame 설정 방법
13
DC2-FI-B(nxos)# show queuing interface ethernet 1/13
Ethernet1/13 queuing information:
TX Queuing
qos-group sched-type oper-bandwidth
0 WRR 6
1 WRR 32
2 WRR 25
3 WRR 25
4 WRR 6
5 WRR 6
RX Queuing
qos-group 3
q-size: 29760, HW MTU: 9216 (9216 configured)
drop-type: drop, xon: 0, xoff: 29760
Statistics:
Pkts received over the port : 28833
Ucast pkts sent to the cross-bar : 4
Mcast pkts sent to the cross-bar : 28829
Ucast pkts received from the cross-bar : 0
Pkts sent to the port : 105148
Pkts discarded on ingress : 0
Per-priority-pause status : Rx (Inactive), Tx (Inactive)

14. VMware ESX vSwitch Jumbo 설정 방법
14

15. VMware ESX vSwitch Jumbo 설정 방법
15

16. Jumbo Frame Test
16
[root@DC1-ESX05-35:~] esxcfg-vswitch -l
Switch Name Num Ports Used Ports Configured Ports MTU Uplinks
vSwitch0 6912 7 128 9000 vmnic0,vmnic1
PortGroup Name VLAN ID Used Ports Uplinks
VM Network 2 1 vmnic0,vmnic1
Management Network 2 1 vmnic0,vmnic1
Switch Name Num Ports Used Ports Configured Ports MTU Uplinks
vSwitch1 6912 29 128 9000 vmnic2,vmnic3
[root@DC1-ESX05-35:~] esxcfg-vmknic -l
Interface Port Group/DVPort/Opaque Network IP Family IP Address Netmask Broadcast MAC Address MTU TSO MSS Enabled Type NetStack
vmk0 Management Network IPv4 10.72.86.35 255.255.255.0 10.72.86.255 00:25:b5:11:15:00 1500 65535 true STATIC defaultTcpipStack
vmk0 Management Network IPv6 fe80::225:b5ff:fe11:1500 64 00:25:b5:11:15:00 1500 65535 true STATIC, PREFERRED defaultTcpipStack
vmk1 vmotion IPv4 192.168.3.35 255.255.255.0 192.168.3.255 00:50:56:65:d2:9e 9000 65535 true STATIC defaultTcpipStack
vmk1 vmotion IPv6 fe80::250:56ff:fe65:d29e 64 00:50:56:65:d2:9e 9000 65535 true STATIC, PREFERRED defaultTcpipStack
[root@DC1-ESX05-35:~] vmkping -I vmk1 192.168.3.123 -d -s 8972
PING 192.168.3.123 (192.168.3.123): 8972 data bytes
8980 bytes from 192.168.3.123: icmp_seq=0 ttl=64 time=0.330 ms
8980 bytes from 192.168.3.123: icmp_seq=1 ttl=64 time=0.280 ms
8980 bytes from 192.168.3.123: icmp_seq=2 ttl=64 time=0.262 ms
ping 192.168.1.1 –l 8972 –f
ping 192.168.1.1 –M do –s 8972

17. 감사합니다.