VMware EMC Service Talk

3. Copyright © 2013 EMC Corporation. All Rights Reserved. This lesson covers the following topics: • FC protocol make-up • FC components Lesson 1: Fibre Channel Protocol Overview Fibre Channel and FCoE Technology 3

4. Copyright © 2013 EMC Corporation. All Rights Reserved. Fibre Channel Layered Architecture: FC-0 ULP Upper Layer Protocol SCSI-3, IP, ESCON/FIPS, etc. FC-4 FCP (SCSI-3 Fibre Channel Link Encapsulation), Single-Byte Command Code Sets (FICON), etc. FC-3 Common Services (Not Used) FC-2 Exchange and Sequence Management, Frame Structure ,Flow Control FC-1 Encode/Decode, Link Control Protocols, Ordered Sets FC-0 Physical Interface, Optical and Electrical Interfaces, Cables, Connectors, etc. Fibre Channel and FCoE Technology 4

5. Copyright © 2013 EMC Corporation. All Rights Reserved. Transceivers Cable Connectors Comments SFP, SFP+ LC • Most popular for Fibre Channel today • SFPs up to 8 Gbps • SFP+ > 8Gbps • versions for different distances mSFP mLC • mini-SFP and mini-LC • Designed by Brocade for 64-port blade GBIC SC • Technology found in older 1 Gbps equipment • Replaced by LC technology Optical Transceivers and Cable Connectors Fibre Channel and FCoE Technology 5

6. Copyright © 2013 EMC Corporation. All Rights Reserved. Fibre Channel Cables and Distances Transceive r Type Speed Multimode media maximum distance 62.5 um (OM1) 50 um (OM2) 50 um (OM3) 50 um (OM4) SW 2 Gbps 150m 300m 500m 500m 4 Gbps 70m 150m 380m 400m 8 Gbps 21m 50m 150m 190m 10 Gbps 33m 82m 300m 550m 16 Gbps 15m 35m 100m 125m Single-mode media maximum distance (9 um core optic fiber) LWL 4 Gbps 10 km to 40 km depending on vendor 8 Gbps 10 Gbps 16 Gbps Note: For greater distances, you may connect using DWDM / CWDM between switches . Fibre Channel and FCoE Technology 6

7. Copyright © 2013 EMC Corporation. All Rights Reserved. Basic Port Types Node Port: Transmits and receives Fibre Channel data frames in a Switched Fabric environment (N_Port) Fabric Port: Port on a switch that N_Port connects to. (F_Port) Expansion Port: Port on a switch that has another switch plugged into it (two switches that are merged) (E_Port) Node Loop Port: Transmits and receives Fibre Channel frames in an Arbitrated Loop (private) environment (NL_Port) Fabric Loop Port: Allows a Fabric to communicate with an Arbitrated Loop (public) (FL_Port) Generic Port: Port on a switch that detects the attached port type and auto-configures to match that port type (G_Port) Fibre Channel and FCoE Technology 7

8. Copyright © 2013 EMC Corporation. All Rights Reserved. The Host Bus Adapter Fabric Target Storage Array Server HBA NIC LAN See EMC Support Matrix at https://elabnavigator.emc.com Fibre Channel and FCoE Technology 8

9. Copyright © 2013 EMC Corporation. All Rights Reserved. Fibre Channel Layered Architecture: FC-1 ULP FC-4 FC-3 FC-2 FC-1 Encode/Decode, Link Control Protocols, Ordered Sets FC-0 Data encoding/decoding • Less than 10 Gbps (20% overhead for encoding)  8-bits  10-bits • 10 Gbps and 16 Gbps (3% overhead for encoding)  64-bits  66-bits • Special Characters  Used with data characters to create ordered sets  Ordered sets include: Idle, SOF, EOF, R_RDY Fibre Channel and FCoE Technology 9

10. Copyright © 2013 EMC Corporation. All Rights Reserved. Fibre Channel Layered Architecture: FC-2 ULP FC-4 FC-3 FC-2 Exchange and Sequence Management, Frame Structure ,Flow Control FC-1 FC-0 • Defines structure of Fibre Channel frame  Frames, Sequences, Exchanges • Flow control mechanisms • Error handling and recovery • Provides class of services  Class 1, Class 2, Class 3, Class 4, Class 6, and Class F Fibre Channel and FCoE Technology 10

11. Copyright © 2013 EMC Corporation. All Rights Reserved. FC-2 • Defines the structure of the Fibre channel frame  How information to large to fit into a single frame is transferred by using a sequence of frames  How operations that consist of exchanges of sequences of frames are managed • Describes how the rate of frame transmission is controlled by the flow control mechanisms • Error handling and recovery • Class of Service  Class 1, Class 2, Class 3, Class 4, Class 6, and Class F Fibre Channel and FCoE Technology 11

12. Copyright © 2013 EMC Corporation. All Rights Reserved. Exchange, Sequence, Frame • Exchange is an operation manager  SCSI task, an IP connection  Has an Originator  Has a Responder  Has one or more Sequences • Sequence is a ULP object to be transmitted in one direction that may be one frame or longer • Frame is the unit of information transfer • Manages flow control (buffer-to-buffer and end-to-end) Fibre Channel and FCoE Technology 12

13. Copyright © 2013 EMC Corporation. All Rights Reserved. Conversations Session Exchange Exchange Exchange Seq Sequence Sequence SequenceSeqSeq Frame #1 Frame #1 Frame #2 Frame #3 Frame #1 Frame #2 Frame #3 Frame #4 Frame #5 Frame #6 Frame #1 Frame #2 Frame #3 Frame #4 Frame #1 Frame #2 Frame #1 Fibre Channel and FCoE Technology 13

14. Copyright © 2013 EMC Corporation. All Rights Reserved. Fibre Channel Frame Data Field End-of-Frame CRC Start-of-Frame Header Optional Header Fill Bytes Payload (information being transported) Optional Header R_CTL CS_CTL TYPE SEQ_ID DF_CTL SEQ_CNT Frame Control (F_CNT) Source Address (S_ID) Destination Address (D_ID) OX_ID RX_ID Parameter Field (PARM) Fibre Channel and FCoE Technology 14

15. Copyright © 2013 EMC Corporation. All Rights Reserved. Credit-Based Flow Control Fibre Channel Switch Physical ISL FC Frame R_RDY Credit = 161516 Fibre Channel Switch Long Distance links may require additional BB_Credit Fibre Channel and FCoE Technology 15

16. Copyright © 2013 EMC Corporation. All Rights Reserved. Fibre Channel Layered Architecture: FC-4 ULP FC-4 FCP (SCSI-3 Fibre Channel Protocol) mapping, Single-Byte Command Code Sets (used for FICON) mapping FC-3 Common Services (Not Used) FC-2 FC-1 FC-0 • Mapping Layer  Maps Upper Level Protocol to lower levels  SAN typically uses SCSI FCP mapping Fibre Channel and FCoE Technology 16

17. Copyright © 2013 EMC Corporation. All Rights Reserved. FC-4 and Upper Level Protocol Exchange, Sequence, and Frame ...Information Unit # 1 Information Unit # 3Information Unit # 2...FC-4 FC-2 Sequence # 1 Sequence # 3Sequence # 2... ... Frame 1 ... Frame 1 Frame 2 Frame N Frame 1 Payload: SCSI CDB DATA Payload: Status Exchange (Operation) DATA DATA SCSI Command SCSI ResponseSCSI DATA ULP SCSI-3: One SCSI Read I/O Operation Fibre Channel and FCoE Technology 17

18. Copyright © 2013 EMC Corporation. All Rights Reserved. EMC Fibre Channel Implementations • Point-to-Point – Link between two ports using Arbitrated Loop • Arbitrated Loop (FC-AL) – Shared interconnect between 2 to 126 nodes • Switched Fabric (FC-SW) – Dynamic connectivity support for more than 15 million nodes Fibre Channel and FCoE Technology 18

20. Copyright © 2013 EMC Corporation. All Rights Reserved. Fibre Channel Address (FCID) • Required to route frames from source to target • 24-bits (three-bytes) - specified in Hex  Assigned at Fabric Login (FLOGI) • Divided into three parts: Domain_ID Area_ID Port_ID Domain (01-EF) (1-239 in decimal) 00-FF 00-FF Example: 6b 04 00 Fibre Channel and FCoE Technology 20

21. Copyright © 2013 EMC Corporation. All Rights Reserved. Common Fabric Services: Well Known Addresses Service Name Well Known Address Description Login Service (F_Port Server) FFFFFE All nodes register in login service when performing FLOGI Name Service FFFFFC Stores information about nodes attached to fabric during the Fabric Login process Fabric Controller FFFFFD State change notification to nodes registered in the fabric and inter-switch communications Fibre Channel and FCoE Technology 21

22. Copyright © 2013 EMC Corporation. All Rights Reserved. World Wide Name • A World Wide Name (WWN) is a 64-bit value used in Fibre Channel networks to uniquely identify each element in the network • Usually assigned to a Host Bus Adapter (HBA), switch port, or storage port by the vendor at the time of manufacture • Often linked within a component, such as an HBA Node WWN – A name for all ports within the single HBA Port WWN – A name for each individual port Fibre Channel and FCoE Technology 22

23. Copyright © 2013 EMC Corporation. All Rights Reserved. WWPN Examples EMC VNX World Wide Port Name Example 5 0 0 6 0 1 6 0 4 7 2 0 5 5 9 0 EMC Company ID (24-bits) Port VNX Seed (from SPE/DPE midplane) EMC VMAX World Wide Port Name Example 5 0 0 0 0 9 7 2 0 8 1 3 4 9 a d EMC Company ID for VMAX (24-bits) VMAX Serial Number - Decodes to country where manufactured/Model / etc. (HK192601234) CPU/Dir/P (12g port B) Emulex World Wide Port Name Example 1 0 0 0 0 0 0 0 c 9 2 0 d c 4 0 Reserved (12-bits) Company OUI (24-bits) Company Specific (24-bits) Fibre Channel and FCoE Technology 23

24. Copyright © 2013 EMC Corporation. All Rights Reserved. FC Port Initialization Process - Step 11 - Link online 2 - Fabric Login - get FCID 3 - Port Login to fabric name server 4 - Registrations with name server 5 - Query Name server for FC-4 devices 6 - State Change Registration Repeat for each Target:  7 Port Login to Target port  8 Process Login to Target port  9 SCSI INQ Fabric 1 5 1 2 3 4 HBA WWN = 50:06:04:8a:d5:2e:76:c4 FCID = 040600 WWN = 10:00:00:00:c9:48:f1:c7 FCID = ?????? 6 Fibre Channel and FCoE Technology 24

25. Copyright © 2013 EMC Corporation. All Rights Reserved. FC Port Initialization Process - Step 21 - Link online 2 - Fabric Login - get FCID 3 - Port Login to fabric name server 4 - Registrations with name server 5 - Query Name server for FC-4 devices 6 - State Change Registration Repeat for each Target:  7 - Port Login to Target port  8 - Process Login to Target port  9 - SCSI INQ Fabric 1 5 1 2 3 4 HBA FLOGI ACC Accept frame includes FCID for HBA 030500 FLOGI Frame Destination Address = FFFFFE Source Address = 000000 WWN = 50:06:04:8a:d5:2e:76:c4 FCID = 040600 WWN = 10:00:00:00:c9:48:f1:c7 FCID = ?????? 6 Fibre Channel and FCoE Technology 25

26. Copyright © 2013 EMC Corporation. All Rights Reserved. FC Port Initialization Process - Step 31 - Link online 2 - Fabric Login - get FCID 3 - Port Login to fabric name server 4 - Registrations with name server 5 - Query Name server for FC-4 devices 6 - State Change Registration Repeat for each Target:  7 - Port Login to Target port  8 - Process Login to Target port  9 - SCSI INQ Fabric 1 WWN = 50:06:04:8a:d5:2e:76:c4 FCID = 040600 WWN = 10:00:00:00:c9:48:f1:c7 FCID = 030500 5 1 2 3 4 HBA PLOGI ACC PLOGI Frame Destination Address = FFFFFC Source Address = 030500 6 Fibre Channel and FCoE Technology 26

27. Copyright © 2013 EMC Corporation. All Rights Reserved. FC Port Initialization Process - Step 41 - Link online 2 - Fabric Login - get FCID 3 - Port Login to fabric name server 4 - Registrations with name server 5 - Query Name server for FC-4 devices 6 - State Change Registration Repeat for each Target:  7 - Port Login to Target port  8 - Process Login to Target port  9 - SCSI INQ Fabric 1 5 1 2 3 4 HBA WWN = 50:06:04:8a:d5:2e:76:c4 FCID = 040600 WWN = 10:00:00:00:c9:48:f1:c7 FCID = 030500 6 Fibre Channel Services Frames • Register Class of Service • Register FC-4 Type • Register Port type • Register other (vendor dependent) FC Serv ACC FC Serv ACC Fibre Channel and FCoE Technology 27

28. Copyright © 2013 EMC Corporation. All Rights Reserved. FC Port Initialization Process - Step 51 - Link online 2 - Fabric Login - get FCID 3 - Port Login to fabric name server 4 - Registrations with name server 5 - Query Name server for FC-4 devices 6 - State Change Registration Repeat for each Target:  7 - Port Login to Target port  8 - Process Login to Target port  9 - SCSI INQ Fabric 1 5 1 2 3 4 HBA WWN = 50:06:04:8a:d5:2e:76:c4 FCID = 040600 WWN = 10:00:00:00:c9:48:f1:c7 FCID = 030500 6 Fibre Channel Services Frames • Query: Get PIDs • Query: Get Port WWN for PID1 • Query: Get Port WWN for PIDn FC Serv ACC FC Serv ACC Name Server Database: • PID = 040600 • FC-4 = SCSI FCP • WWPN = 50:06:04:8a:d5:2e:76:c4 • . . . • PID = 030100 . . . • PID = 040400 . . . Fibre Channel and FCoE Technology 28

29. Copyright © 2013 EMC Corporation. All Rights Reserved. FC Port Initialization Process - Step 61 - Link online 2 - Fabric Login - get FCID 3 - Port Login to fabric name server 4 - Registrations with name server 5 - Query Name server for FC-4 devices 6 - State Change Registration Repeat for each Target:  7 - Port Login to Target port  8 - Process Login to Target port  9 - SCSI INQ Fabric 1 5 1 2 3 4 HBA WWN = 50:06:04:8a:d5:2e:76:c4 FCID = 040600 WWN = 10:00:00:00:c9:48:f1:c7 FCID = 030500 6 SCR ACC SCR Frame Destination Address = FFFFFD Source Address = 030500 Fibre Channel and FCoE Technology 29

30. Copyright © 2013 EMC Corporation. All Rights Reserved. FC Port Initialization Process - Step 71 - Link online 2 - Fabric Login - get FCID 3 - Port Login to fabric name server 4 - Registrations with name server 5 - Query Name server for FC-4 devices 6 - State Change Registration Repeat for each Target:  7 - Port Login to Target port  8 - Process Login to Target port  9 - SCSI INQ Fabric 1 5 1 2 3 4 HBA WWN = 50:06:04:8a:d5:2e:76:c4 FCID = 040600 WWN = 10:00:00:00:c9:48:f1:c7 FCID = 030500 6 Fibre Channel and FCoE Technology 30

31. Copyright © 2013 EMC Corporation. All Rights Reserved. FC Port Initialization Process - Step 81 - Link online 2 - Fabric Login - get FCID 3 - Port Login to fabric name server 4 - Registrations with name server 5 - Query Name server for FC-4 devices 6 - State Change Registration Repeat for each Target:  7 - Port Login to Target port  8 - Process Login to Target port  9 - SCSI INQ Fabric 1 5 1 2 3 4 HBA WWN = 50:06:04:8a:d5:2e:76:c4 FCID = 040600 WWN = 10:00:00:00:c9:48:f1:c7 FCID = 030500 6 SCSI Process SCSI Process Fibre Channel and FCoE Technology 31

32. Copyright © 2013 EMC Corporation. All Rights Reserved. FC Port Initialization Process - Step 91 - Link online 2 - Fabric Login - get FCID 3 - Port Login to fabric name server 4 - Registrations with name server 5 - Query Name server for FC-4 devices 6 - State Change Registration Repeat for each Target:  7 - Port Login to Target port  8 - Process Login to Target port  9 - SCSI INQ Fabric 1 5 1 2 3 4 HBA WWN = 50:06:04:8a:d5:2e:76:c4 FCID = 040600 WWN = 10:00:00:00:c9:48:f1:c7 FCID = 030500 6 SCSI Process SCSI Process Fibre Channel and FCoE Technology 32

33. Copyright © 2013 EMC Corporation. All Rights Reserved. Fibre Channel Login at a Glance Process FLOGI PLOGI PRLI Requirements - Link initialization - Cable - HBA and driver - Switch port - FLOGI - Zoning - Persistent binding - Driver setting - PLOGI - Device masking (Target) - Device mapping (Initiator) - Driver setting (Initiator) Information passed - WWN - S_ID - Protocol - Class - Zoning - WWN - S_ID - ULP - Class - BB Credit - LUN Session members - N_Port to F_Port - N_Port to N_Port - ULP (SCSI-3 to SCSI-3) Verification UNIX WINDOWS - Syslog - Switch Utilities - Syslog - Driver Utilities - Syslog - Host based volume management - Event Viewer - Switch Viewer - Driver utilities - Driver utilities - Host based volume management - Device manager Fibre Channel and FCoE Technology - 33

34. Copyright © 2013 EMC Corporation. All Rights Reserved. Zone Set: Fabric 1 Zone A: Initiator A; Target w Zone B: Initiator B; Target x Zone C: Initiator C; Target y; Target z . . . Zoning Overview Storage (Targets) Hosts (Initiators) I T Initiator (HBA) Target Zone A Fabric 1 Fibre Channel and FCoE Technology 34

35. Copyright © 2013 EMC Corporation. All Rights Reserved. Fabric 1 How to Define Zone Members I T Target Port WWPN: 50:06:04:8a:d5:2e:76:c4 Zone A Initiator Port WWPN: 10:00:00:00:c9:48:f1:c7 5 3 1 2 3 4 Zone A 10:00:00:00:c9:48:f1:c7; 50:06:04:8a:d5:2e:76:c4 Zone Members Defined by WWPN Zone A 0305ef; (Domain_ID 03, port_05) 0403ef; (Domain_ID 04, port_03) Zone Members Defined by Domain/Port Fibre Channel and FCoE Technology 35

36. Copyright © 2013 EMC Corporation. All Rights Reserved. This lesson covers the following topics: • Architecture of an Ethernet solution Lesson 2: Ethernet Overview Fibre Channel and FCoE Technology 36

37. Copyright © 2013 EMC Corporation. All Rights Reserved. Ethernet and the OSI Model 7 - Application 6 - Presentation 5 - Session 4 - Transport 3 - Network 2 - Data Link 1 - Physical LLC - Logical Link Control MAC - Media Access Control Ethernet is defined at Layer 2 of the OSI model Fibre Channel and FCoE Technology 37

38. Copyright © 2013 EMC Corporation. All Rights Reserved. Ethernet PDU Destination MAC 6 bytes Source MAC 6 bytes 802.1Q tag (optional) 4 bytes Type 2 bytes Data Payload variable # bytes FCS 4 bytes Ethernet_II Frame Format Header Fibre Channel and FCoE Technology 38

39. Copyright © 2013 EMC Corporation. All Rights Reserved. Ethernet MAC Address I G G L Organizationally Unique Identifier (OUI) (Assigned by IEEE) Vendor assigned 24 bits 24 bits Examples: Broadcast address: ff:ff:ff:ff:ff:ff Physical address of port on PC: 00-26-5E-82-3E-6E 47 46 45 24 23 0 Fibre Channel and FCoE Technology 39

40. Copyright © 2013 EMC Corporation. All Rights Reserved. MAC Address Used to Direct Frame Through Network MAC: 00-26-5E-82-3E-6E MAC: 00-26-5E-77-2F-58 …2F:58 …3E:6E 08-00 Payload FCS Ethernet Frame This is the source address This is the destination address Fibre Channel and FCoE Technology 40

41. Copyright © 2013 EMC Corporation. All Rights Reserved. Switching Frames Using MAC Addresses - Step 1 MAC Table Port 1 -Empty Port 2 - Empty Port 3 - Empty Port 4 - Empty PC A PC C PC B PC D Port 1 Port 2 Port 3 Port 4 Frame Dest Src PC A Fibre Channel and FCoE Technology 41

42. Copyright © 2013 EMC Corporation. All Rights Reserved. Switching Frames Using MAC Addresses - Step 2 MAC Table Port 1 -PC A Port 2 - Empty Port 3 - Empty Port 4 - Empty PC C PC B PC D Port 1 Port 2 Port 3 Port 4 No entry for PC D, so flood the frame out all ports PC A Fibre Channel and FCoE Technology 42

43. Copyright © 2013 EMC Corporation. All Rights Reserved. Switching Frames Using MAC Addresses - Step 3 MAC Table Port 1 -PC A Port 2 - Empty Port 3 - Empty Port 4 - Empty PC A PC C PC B PC D Port 1 Port 2 Port 3 Port 4 Frame Dest Src PC A PC D PC D Fibre Channel and FCoE Technology 43

45. Copyright © 2013 EMC Corporation. All Rights Reserved. How STP Solves the Problem • STP is a loop-prevention protocol • Uses spanning tree algorithm to elect a Root Bridge and block certain ports. Switch A Switch C Switch B Block Fibre Channel and FCoE Technology 45

46. Copyright © 2013 EMC Corporation. All Rights Reserved. How Does STP Protocol Elect a Root Bridge? Switch A Switch C Switch B Priority = 32768 MAC = 0000.1111.2222 Priority = 32768 MAC = 0000.2222.3333 Priority = 32768 MAC = 0000.2222.2222 • Switch with lowest Bridge ID (BID) wins election • If all switches have equal priority, then switch with lowest MAC address wins Since all switches are set to default priority, this switch will be elected Root Bridge because it has the lowest MAC address Bridge ID Fibre Channel and FCoE Technology 46

47. Copyright © 2013 EMC Corporation. All Rights Reserved. Result of running STP Protocol Switch A Switch C Switch B Root Bridge Designated Port (F) Root Port (F) Alternate Port (D) Nonroot Bridge Nonroot Bridge Root Port (F) Designated Port (F) Designated Port (F) • Forwarding ports  Root ports  Designated ports • Discarding port  Alternate Fibre Channel and FCoE Technology 47

48. Copyright © 2013 EMC Corporation. All Rights Reserved. Ethernet Hardware Overview • Ethernet hardware supports the following speeds  10 Mbps  100 Mbps  1 Gbps  10 Gbps  40 Gbps  100 Gbps Hardware Popular Implementations NIC (Network Interface Controller) • Stand-alone card • Built into host motherboard • Wireless Cables • Copper UTP (Unshielded Twisted Pair) • Copper Twinax • Optic Connectors • RJ45 (for copper UTP cables) • SFP+ (for copper twinax cables) • LC (for fiber optic cables) Most common Data Center Core (Backbone) Fibre Channel and FCoE Technology 48

49. Copyright © 2013 EMC Corporation. All Rights Reserved. Network Interface Controller - NIC • Provides access to the network • Different implementations include:  Stand-alone card (could have multiple ports)  Built into motherboard (most common now)  Wireless • Ports can be “teamed” to act as a single logical interface for failover NIC LAN Fibre Channel and FCoE Technology 49

50. Copyright © 2013 EMC Corporation. All Rights Reserved. Example Ethernet Cabling UTP (Unshielded Twisted Pair) Twinax with SFP+ Fiber Optic Cable with LC Connector Media Copper Copper Optic IEEE Standard 1000BaseT (802.3ab) 10GSFP+CU (Direct Attach) 1000BaseLX (802.3z) Distance/speed 100 meters @1 Gbps 5 meters @ 10 Gbps 10 km @ 1 Gbps Cat 5 RJ45 Twinax / SFP+ 9/125 SM Fiber LC Connector Fibre Channel and FCoE Technology 50

52. Copyright © 2013 EMC Corporation. All Rights Reserved. VLAN 200 VLAN 200 VLAN 100 VLAN 100 VLAN Overview • Each VLAN is a separate broadcast domain • Each switch port is assigned to a single VLAN • Devices in different VLANs must use a router to communicate • VLAN Trunks allow the data from multiple VLANs to pass on a single link between switches Fibre Channel and FCoE Technology 52

53. Copyright © 2013 EMC Corporation. All Rights Reserved. How a VLAN Solves the Problems of the Flat Network • Smaller broadcast domains • Security between VLANs VLAN 100 VLAN 200 VLAN 100 VLAN 200 VLAN 100 VLAN 200 Trunk Broadcast Broadcast Fibre Channel and FCoE Technology 53

54. Copyright © 2013 EMC Corporation. All Rights Reserved. How it Works: VLAN Tagging • 802.1Q - IEEE Standard  Each port on switch assigned to specific VLAN ID  Unassigned ports automatically belong to Default VLAN (VLAN ID = 0001) • When port receives frame:  Compare VLAN ID to filter table.  If valid, and MAC on another switch - forward to trunk port  If destined to exit an access port on switch, remove VLAN ID • Trunk carries both tagged and untagged traffic Destination MAC 6 bytes Source MAC 6 bytes 802.1Q tag (optional) 4 bytes Type 2 bytes Data Payload variable # bytes FCS 4 bytes Ethernet_II Frame Format VLAN Tag Fibre Channel and FCoE Technology 54

55. Copyright © 2013 EMC Corporation. All Rights Reserved. This lesson covers the following topics: • FCoE Protocol and FCoE Fabric design basics Lesson 3: FCoE Protocol Overview Fibre Channel and FCoE Technology 55

56. Copyright © 2013 EMC Corporation. All Rights Reserved. FCoE Protocol Stack Volume managers, File systems, etc. SCSI Command Descriptor Blocks, data, and responses FCoE frame encapsulation / de-capsulation, FCoE entities Frame switching, MAC address, transport Connection to the Physical Layer Fibre Channel Protocol frame construction FC-4, FC-3, FC-2 Application SCSI FCP FCoE Enhanced Ethernet FCoE Protocol Stack Fibre Channel and FCoE Technology 56

57. Copyright © 2013 EMC Corporation. All Rights Reserved. FCoE Encapsulation SOF EOF Ethernet Header FCoE Header Encapsulated FC Frame (including FC-CRC) FCS FC Header Payload Fill Dest. MAC Address Source MAC Address Version Reserved VLAN TAG CRC Ether Type Fibre Channel Frame Ethernet Frame FCoE PDU Fibre Channel and FCoE Technology 57

58. Copyright © 2013 EMC Corporation. All Rights Reserved. FCoE Frames • Frame Efficiency  iSCSI, FCIP, iFCP rely on IP over Ethernet which result in additional overhead in frames  FCoE frames only use Ethernet protocol  Efficiency can be calculated by the formula • Jumbo Frames  Ethernet frames with more than 1500 bytes of payload  Considered unreliable on regular Ethernet  CEE allows reliable use of Jumbo Frames  FCoE must use jumbo frames (2240 bytes) Number of Data Bytes Total Bytes Sent Fibre Channel and FCoE Technology 58

60. Copyright © 2013 EMC Corporation. All Rights Reserved. • CNA is both a  Network Adapter  Fibre Channel adapter • Host sends I/O to appropriate adapter:  NIC – NFS, Web, Email, etc.  HBA – SCSI block data • Host receives data through lossless Ethernet MAC  Directs I/O to appropriate adapter  Based on Ethernet “Type” Converged Network Adapter Architecture 2 x 10 Gbps Ethernet Ports Ethernet NIC Host Networking Drivers FCoE Encapsulation Existing FC Host Drivers CNA Adapter Lossless Ethernet MAC Is it FCoE?NO YES HBA Architecture Fibre Channel and FCoE Technology 60

61. Copyright © 2013 EMC Corporation. All Rights Reserved. Fibre Channel Forwarder – FCF • FCF function provided by FCoE switch or FCoE blade  Services FCoE Login requests  Provides services associated with a Fibre Channel switch Stand-alone FCoE Switch Fibre Channel Switch with FCoE blade Fibre Channel and FCoE Technology 61

62. Copyright © 2013 EMC Corporation. All Rights Reserved. • Lossless Ethernet:  FCoE requires reliable frame delivery  Can’t drop frames from buffer overflow • Improves regular Ethernet:  Eliminates Ethernet loss, therefore more reliable  Speed increased to 10 Gbps • Additions to Ethernet:  Priority-based Flow Control (PFC)  Enhanced Transmission Selection (ETS)  DCB Capability Exchange (DCBX)  Notify neighbor node of DCB support Data Center Bridging (DCB) Fibre Channel and FCoE Technology 62

63. Copyright © 2013 EMC Corporation. All Rights Reserved. Enhanced Ethernet Features Feature Enhancement Priority Flow Control IEEE 802.1Qbb – class of service flow control by enabling PAUSE functionality on IEEE 802.1p lanes Data Center Bridging Exchange Auto-negotiation of Enhanced Ethernet capabilities DCBX (switch to NIC) Bandwidth Management IEEE 802.1Qaz Enhanced Transmission Selection – manage bandwidth and assign priorities to groups of IEEE 802.1p lanes based on class of traffic. Fibre Channel and FCoE Technology 63

64. Copyright © 2013 EMC Corporation. All Rights Reserved. Priority-based Flow Control STOP Transmit Queues Receive Buffers 0 1 2 3 4 5 6 7 Buffers are almost full - send “PAUSE” Priority Fibre Channel and FCoE Technology 64

65. Copyright © 2013 EMC Corporation. All Rights Reserved. Enhanced Ethernet - DCBX • DCBX runs between the switch and CNA • DCBX negotiates capabilities • Switch distributes configuration to all attached adapters • Devices need to discover the capabilities of its peers. • DCBCX utilizes the link-layer discovery protocol and handles local operational configuration for each feature Priority Flow Control Data Center Bridging Exchange Bandwidth Management Rules for Capabilities Result Capability and configuration match Enabled Adapter configured to accept switch config. Enabled Adapter not configured to accept switch config. Disabled Adapter does not support/implement DCBX Disabled Fibre Channel and FCoE Technology 65

66. Copyright © 2013 EMC Corporation. All Rights Reserved. Enhanced Ethernet – Bandwidth Management • Enables consistent management of QoS • Provides consistent scheduling of different traffic types Priority Flow Control Data Center Bridging Exchange Bandwidth Management Fibre Channel and FCoE Technology 66

67. Copyright © 2013 EMC Corporation. All Rights Reserved. Addressing MAC = A FCID = 050100 MAC = B Domain ID = 05 Domain ID = 01 FCID = 011C00 FC SAN Converged Network FC Frame D_ID = 011C00 S_ID = 050100 Dest = MAC B Source = MAC A D_ID = 011C00 S_ID = 050100 Ethernet Frame Fibre Channel and FCoE Technology 67

68. Copyright © 2013 EMC Corporation. All Rights Reserved. A Note About CNA MAC Addresses FCoE Controller Lossless Ethernet MAC FCoE_LEP VN_Port FC-3 / FC-4 Layers Ethernet_Port Ethernet NIC Host Networking Drivers Existing FC Host Drivers Physical MAC ENode MAC FPMA MAC CNA Fibre Channel and FCoE Technology 68

69. Copyright © 2013 EMC Corporation. All Rights Reserved. How are MAC Addresses Assigned to the VN_Port? • Server-provided MAC Address (SPMA):  Uses single MAC Address for all FCoE traffic  FCoE has MAC address independent of other protocols  Support for different MAC addresses for each Virtual Port  This method isn’t used • Fabric-provided MAC Address (FPMA):  Uses virtual port MAC addresses  Each virtual port MAC address is supplied by the fabric  Most common method  Have the following format: Fibre Channel and FCoE Technology 69

70. Copyright © 2013 EMC Corporation. All Rights Reserved. Server Provided MAC Addresses (SPMA) • Adapter uses burned-in or configured MAC address  Consistent with the Ethernet model • FCF needs a table to map between MAC addresses and FC_IDs FCoE Node Port MAC 00.05.69.00.00.03 MAC Address Fibre Channel and FCoE Technology 70

71. Copyright © 2013 EMC Corporation. All Rights Reserved. Fabric Provided MAC Addresses (FPMA) • MAC address assigned for each FC_ID:  Consistent with the Fibre Channel model • Uses OUIs with U/L = 1 FC-MAP (Local addressing) • No table needed for encapsulation FC-MAP (0E-FC-00) FCID 07.08.09 24 bits 24 bits 48 bit MAC Address FC-MAP (0E-FC-00) FCID 07.08.09 Fibre Channel and FCoE Technology 71

72. Copyright © 2013 EMC Corporation. All Rights Reserved. FCoE Initialization Protocol (FIP) and FC Logins CNA CNA (ENode) Lossless Ethernet Switch (FIP Snooping Bridge) FCoE Switch (FCF) Storage FCoE Switch (FCF) FC SAN FC SAN Storage ENode (FCoE Controller) Send FIP VLAN Request Send FIP Solicitation Send FIP FLOGI FCF Reply FIP VLAN Notification Reply FIP Advertisement Reply FIP FLOGI ACC FIP Multicast Fibre Channel and FCoE Technology 72

82. Copyright © 2013 EMC Corporation. All Rights Reserved. VAAI – Hardware Assisted Locking • Scalable Lock Management – Atomic Test and Set (ATS) • A number of VMFS operations cause the LUN to temporarily become locked for exclusive write use by one of the ESXi nodes, including: – Moving a VM with vMotion – Creating a new VM or deploying a VM from a template – Powering a VM on or off – Creating a template – Creating or deleting a file, including snapshots • VAAI feature, atomic_test_and_set (ATS) allows the ESXi host to offload the management of the required locks to the storage and avoids locking the entire VMFS file system

83. Copyright © 2013 EMC Corporation. All Rights Reserved. Atomic Test & Set • Original file locking technique 1. Acquire SCSI reservation 2. Acquire file lock 3. Release SCSI reservation 4. Do work on VMFS file/metadata 5. Release file lock • New file locking technique 1. Acquire ATS lock 2. Acquire file lock 3. Release ATS lock 4. Do work on VMFS file/metadata 5. Release file lock Note - The main difference with using the ATS lock is that it does not affect the other ESXi hosts sharing the datastore

84. Copyright © 2013 EMC Corporation. All Rights Reserved. VMFS Scalability with Atomic Test and Set Makes VMFS more scalable overall, by offloading block locking mechanism Using Atomic Test and Set (ATS) capability provides an alternate option to use of SCSI reservations to protect the VMFS metadata from being written to by two separate ESXi hosts at one time. Normal VMware Locking (No ATS) Enhanced VMware Locking (With ATS)

85. Copyright © 2013 EMC Corporation. All Rights Reserved. How to resolve SCSI Reserves on VMware ESXi servers • Refer to the following VMware KB articles: – SCSI Reservation Issue with Fibre Channel HBAs (4365932) – Troubleshooting SCSI Reservation failures on Virtual Infrastructure 3.x and vSphere 4.x (1005009) – VMware KB Article 1002293 • Warning! DO NOT release the reservation with INLINES when customer is using vSphere. • It is also NOT recommended to try to release via Solutions Enabler • The CORRECT way to release a SCSI Reserve in ESXi is to ask the customer to use OS tools as described in the VMware documentation and KB articles that are listed.

87. Copyright © 2013 EMC Corporation. All Rights Reserved. During this lesson the following topics are covered: • PowerPath configuration and PowerPath’s function in the I/O path • How PowerPath supports EMC and non-EMC arrays • Key functionality included in current releases of PowerPath • How PowerPath is licensed • Hardware, software, clusters, and storage arrays that PowerPath supports Lesson 1: PowerPath Architecture 87

88. Copyright © 2013 EMC Corporation. All Rights Reserved. EMC PowerPath - A Family of Products • PowerPath traditionally viewed only as “Path Management Software”  Performed Multipathing, load balancing, failover, etc. • Functionality Expanded  PowerPath Family now has 4 distinct products  PowerPath can now be used for solutions previously handled by separate software  Common Interface allows for simplified management • Optimized for heterogeneous host / storage environments 88 PowerPath Multipathing Path Management / OptimizationPowerPath/VE PowerPath Migration Enabler Data Protection PowerPath Encryption

89. Copyright © 2013 EMC Corporation. All Rights Reserved. • PowerPath sits between the applications and the HBA driver • Applications direct I/O to PowerPath • PowerPath directs I/O to an optimal path based on current workload and path availability PowerPath Configuration 89 OpenSystemsHost Logical volume manager File system PowerPath Driver Applications Management utilitiesDBMS Storage Array Host bus adapter Host bus adapter Host bus adapter Host bus adapter Storage Area Network

90. Copyright © 2013 EMC Corporation. All Rights Reserved. • PowerPath keeps a table of paths • Presents a single device to the application • Combines equivalent paths into a single “Path Set” Volume Path Set 90 PowerPath HBAHBA SERVERSTORAGE Front-end Ports Storage Area Network Volume Path Set Application HBAHBA POWERPATH “Power” Device

91. Copyright © 2013 EMC Corporation. All Rights Reserved. PowerPath Array Support Concepts 91 SERVERSTORAGE SERVERSTORAGE Active-Active Arrays Active-Passive Arrays Owned by SP-A HBA HBA HBAHBA SAN SP-A Active Active SP-B Passive Passive PowerPath 0 1 0 1 Host Application(s) Mapped to all Front-end ports Active Active Active Active SAN Front-end Ports HBA HBA HBAHBA PowerPath Host Application(s)

92. Copyright © 2013 EMC Corporation. All Rights Reserved. • Dead  indicates the path is not usable.  PowerPath driver will not direct user I/O to this logical I/O path. • Alive  indicates the path is usable  The PowerPath driver can direct I/O to this logical I/O path. Device (LUN) Path States 92 SERVERSTORAGE Front-end Ports Storage Area Network Host Application(s) HBA HBA HBAHBA Host Bus Adapter PowerPath A B

93. Copyright © 2013 EMC Corporation. All Rights Reserved. PowerPath Licensing • PowerPath  Unlocks the full load balancing and path failover capabilities of PowerPath and supports all EMC and certain non-EMC arrays  Configures optimized default load balancing policies for EMC VMAX, VNX and supported third-party arrays • PowerPath/VE  Full feature set specifically for virtual server environments based on VMware vSphere and Windows Hyper-V • PowerPath SE  Backend (between switch and array) failover only  Single HBA supported only  No load balancing available  Path Failover across two backend paths 93 PowerPath SE

94. Copyright © 2013 EMC Corporation. All Rights Reserved. During this lesson the following topics are covered: • Benefits of PowerPath/VE • Features and functionality of PowerPath/VE • PowerPath/VE configuration in an ESXi environment Lesson 2: Multipathing in ESXi 94

95. Copyright © 2013 EMC Corporation. All Rights Reserved. What is PowerPath/VE? • PowerPath advanced multipathing capabilities for virtual environments hosted on VMware vSphere • Features include:  Dynamic load balancing  Device prioritization  Automated optimization of server, storage, and path utilization  Dynamic path failover and path recovery  Monitoring performance and path status, I/O statistics and reports  Automatic path testing  Automatic path restore 95 x86 Architecture PowerPath

96. Copyright © 2013 EMC Corporation. All Rights Reserved. PowerPath/VE Key Benefits • Automates the virtual server-storage connection:  Assures tunable, predictable performance  Optimizes server, storage, and data-path utilization  Maximizes data availability • Uses standard management features across physical and virtual environments:  PowerPath/VE experience similar to rest of environment (physical servers)  Common user experience across all classes of supported platforms • Saves Money:  Simple to implement application service levels for performance and availability  Eliminates need to monitor and rebalance the dynamic environment  Reduces number of multi-pathing software variants to manage 96

97. Copyright © 2013 EMC Corporation. All Rights Reserved. • PowerPath manages complexity:  Constantly adjusts I/O path usage and changes in I/O loads from VMs  Simplifies provisioning by pooling all connections  Optimizes overall I/O performance in VMware ESX environments PowerPath/VE Load Balancing 97 OS APP OS APP OS APP OS APP OS APP OS APP OS APP OS APP OS APP OS APP OS APP OS APP OS APP OS APP OS APP OS APP STORAGE SAN PowerPath PowerPath PowerPath PowerPath

98. Copyright © 2013 EMC Corporation. All Rights Reserved. PowerPath/VE supports: • Changes to Permanent Device Loss (PDL) handling • VMFS 5 Filesystem vSphere 5.0 Kernel Support 98 EMC and Non-EMC Storage Host Bus Adapte r Host Bus Adapte r Host Bus Adapte r Host Bus Adapte r vSphere Kernel PowerPath/VE Guest OS Applications DBMS Management Utilities File System Logical Volume Manager PowerPath/VE runs in the kernel

99. Copyright © 2013 EMC Corporation. All Rights Reserved. PowerPath/VE – Two Major Components 99 PowerPath Remote Management Server ESX Hosts  PowerPath remote tools (rpowermt) installed  Used to remotely manage PowerPath/VE on any number of ESX hosts  PowerPath software is installed

100. Copyright © 2013 EMC Corporation. All Rights Reserved. PowerPath Remote Management Server • Supported platforms:  Can be a physical or virtual host  Check ESM for version compatibility • Software:  rpowermt  Sends PowerPath management commands to a remote ESX host  Is the only way to manage PowerPath/VE (no local commands)  VMware vSphere CLI  Installs PowerPath remotely onto an ESX host  License file  Uses rpowermt to register license with an ESX host • Can manage any number of ESX hosts. 100 PowerPath Remote Management Server

101. Copyright © 2013 EMC Corporation. All Rights Reserved. PowerPath Remote Tools • Are installed on the PowerPath Remote Management Server • Provide remote versions of existing PowerPath commands:  Example:  Local command used by previous versions of PowerPath: powermt display dev=all  Remote command, supported by PowerPath/VE: rpowermt host=<hostname> display dev=all • Note: Remote Tools version and PowerPath version must be compatible:  Possible to update either Remote Tools or PowerPath, without updating the other 101

102. Copyright © 2013 EMC Corporation. All Rights Reserved. Installation and Configuration Steps 1. Upload Files to ESXi Server 2. Install PowerPath software on the ESXi server 3. Install PowerPath Remote Tools on PowerPath Remote Management Server (Optional) 4. Configure and register license file 102

104. Copyright © 2013 EMC Corporation. All Rights Reserved. Install PowerPath on ESXi 104 # cd /vmfs/volumes/Storage1/PPVE/ # ls PowerPath_VE_5_7_for_VMWARE_vSphere-Install_SW_Bundle.zip # unzip PowerPath_VE_5_7_for_VMWARE_vSphere-Install_SW_Bundle.zip . . . # esxcli software vib install --depot=/vmfs/volumes/Storage1/PPVE/ PowerPath_VE_5_7_for_VMWARE_vSphere-Install_SW_Bundle/EMCPower. VMWARE.5.7.b173.zip Installation Result Message: The update completed successfully, but the system needs to be rebooted for the changes to be effective. Reboot Required: true VIBs Installed: EMC_bootbank_powerpath.cim.esx_5.7.0.00.00-b173, EMC_bootbank_powerpath.lib.esx_5.7.0.00.00-b173, EMC_bootbank_powerpath.plugin.esx_5.7.0.00.00-b173 VIBs Removed: VIBs Skipped:

106. Copyright © 2013 EMC Corporation. All Rights Reserved. Verify PowerPath /VE Installation 106 # cd /opt/emc/powerpath/bin /opt/emc/powerpath/bin # ls powermt /opt/emc/powerpath/bin # ./powermt version EMC powermt for PowerPath (c) client Version 5.7 (build 173) EMC PowerPath (c) Version 5.7 (build 173) Warning: License not installed. /opt/emc/powerpath/bin #

VMware EMC Service Talk

VMware EMC Service Talk

Recomendados

Más contenido relacionado

La actualidad más candente

La actualidad más candente(20)

Similar a VMware EMC Service Talk

Similar a VMware EMC Service Talk(20)

Más de Yunchao (Kevin) Wang

Más de Yunchao (Kevin) Wang(8)

VMware EMC Service Talk