Introducing and Implementing IBM FlashSystem V9000

Redbooks
Front cover
Introducing and Implementing
IBM FlashSystem V9000
Karen Orlando
Christophe Fagiano
Detlef Helmbrecht
Jeffrey Irving
Arne Lehfeldt
Corne Lottering
Alexander Watson

International Technical Support Organization
Introducing and Implementing IBM FlashSystem V9000
July 2015
SG24-8273-00

© Copyright International Business Machines Corporation 2015. All rights reserved.
Note to U.S. Government Users Restricted Rights -- Use, duplication or disclosure restricted by GSA ADP Schedule
Contract with IBM Corp.
First Edition (July 2015)
This edition applies to IBM FlashSystem V9000 Version 7.4.1.x
Note: Before using this information and the product it supports, read the information in “Notices” on
page xi.

© Copyright IBM Corp. 2015. All rights reserved. iii
Contents
Notices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi
Trademarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xii
IBM Redbooks promotions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii
Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .xv
Authors. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xvii
Now you can become a published author, too! . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xviii
Comments welcome. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xix
Stay connected to IBM Redbooks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xix
Chapter 1. FlashSystem V9000 introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1 FlashSystem V9000 storage overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.2 Why flash matters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.3 IBM FlashSystem family: Product differentiation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.4 FlashSystem V9000: IBM Tier 1 storage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.5 FlashCore technology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.5.1 Hardware accelerated I/O . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.5.2 IBM MicroLatency module. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.5.3 Advanced flash management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.5.4 Flash wear guarantee . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.6 Architectural design overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
1.6.1 Hardware-only data path. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
1.6.2 The 20nm flash card memory chips . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
1.6.3 Flash module capacities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
1.6.4 Gateway interface FPGA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
1.6.5 Flash controller FPGA. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
1.6.6 IBM Variable Stripe RAID and two-dimensional flash RAID overview . . . . . . . . . 11
1.6.7 Fixed and scalable building blocks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
1.6.8 Scale up and scale out solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
1.7 Advanced software features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
1.7.1 Advanced functions for data reduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
1.7.2 Data migration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
1.7.3 Advanced copy services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
1.7.4 External virtualization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
1.7.5 Easy Tier. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
1.7.6 Licensing. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
Chapter 2. FlashSystem V9000 architecture. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
2.1 Introduction to IBM FlashSystem V9000. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
2.1.1 Capacity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
2.1.2 Performance and latency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
2.1.3 IBM FlashCore technology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
2.1.4 Overview of IBM Variable Stripe RAID and 2D Flash RAID . . . . . . . . . . . . . . . . . 21
2.1.5 Scalability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
2.1.6 Protocol support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
2.1.7 Encryption support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
2.1.8 Comparison of IBM FlashSystem models V840 and V9000 . . . . . . . . . . . . . . . . . 34
2.1.9 Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

iv Implementing IBM FlashSystem V9000
2.2 Architecture of IBM FlashSystem V9000 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
2.2.1 Overview of architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
2.2.2 Hardware components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
2.2.3 Power requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
2.2.4 Physical specifications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
2.3 Control enclosure (AC2) of the FlashSystem V9000 . . . . . . . . . . . . . . . . . . . . . . . . . . 45
2.3.1 I/O connectivity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
2.3.2 Compression Acceleration Card . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
2.3.3 Technician port . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
2.3.4 Battery backup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
2.4 Storage enclosure (AE2) of the FlashSystem V9000 . . . . . . . . . . . . . . . . . . . . . . . . . . 51
2.4.1 Interface cards . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
2.4.2 MicroLatency modules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
2.4.3 Battery modules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
2.5 Administration and maintenance of FlashSystem V9000 . . . . . . . . . . . . . . . . . . . . . . . 57
2.5.1 System management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
2.5.2 Software and licensing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
2.5.3 Serviceability and software enhancements. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
2.6 Support matrix for the FlashSystem V9000 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
2.7 Warranty information. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
Chapter 3. Advanced software functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
3.2 Advanced features for storage efficiency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
3.2.1 IBM Easy Tier . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
3.2.2 Thin provisioning. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
3.2.3 Real-time Compression software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
3.3 Data migration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
3.3.1 Migration operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
3.3.2 Migrating data from an image mode volume. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
3.4 Advanced copy services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
3.4.1 FlashCopy. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
3.4.2 Volume mirroring and migration options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
3.4.3 Remote Copy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
3.5 Data encryption . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
Chapter 4. Planning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
4.1 General planning introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104
4.2 Physical planning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106
4.2.1 Racking considerations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
4.2.2 Power requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
4.2.3 Network cable connections . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113
4.3 Logical planning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117
4.3.1 Management IP addressing plan . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117
4.3.2 SAN zoning and SAN connections . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118
4.3.3 iSCSI IP addressing plan . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
4.3.4 Call home option . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120
4.3.5 V9000 system configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120
4.3.6 EasyTier version 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123
4.3.7 Volume configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124
4.3.8 Host mapping (LUN masking). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126
4.3.9 SAN boot support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127
4.4 License features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127

Contents v
4.4.1 Encryption feature. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127
4.4.2 External virtualized storage configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128
4.4.3 Advanced copy services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131
4.4.4 Real-time Compression. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137
4.5 Data migration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137
4.6 V9000 configuration backup procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138
Chapter 5. Scalability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139
5.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140
5.2 Building block for scaling. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140
5.3 Scaling concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141
5.3.1 Scale up for capacity. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142
5.3.2 Scale out for performance. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142
5.3.3 FlashSystem V9000 scaled configurations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142
5.4 Adding a V9000 FlashSystem storage enclosure . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143
5.5 Set up of a scalable building block with extra storage enclosure . . . . . . . . . . . . . . . . 149
5.6 Adding a second scalable building block . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152
5.7 Set up of scale out system: Two building blocks and one extra storage enclosure . . 157
5.8 Planning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160
5.9 Installing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160
5.10 Operations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160
5.11 Concurrent code load in a scaled out system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 168
Chapter 6. Installation and configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173
6.1 Installation overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174
6.1.1 Tasks for the IBM SSR or IBM lab-based services. . . . . . . . . . . . . . . . . . . . . . . 174
6.1.2 First customer involvement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175
6.2 IBM FlashSystem V9000 physical specifications . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175
6.3 Installing the hardware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175
6.4 Connecting the components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 176
6.4.1 Connecting the components in a fixed building block . . . . . . . . . . . . . . . . . . . . . 176
6.4.2 Connecting the components in a scalable building block . . . . . . . . . . . . . . . . . . 177
6.4.3 Ethernet cabling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180
6.4.4 Scaling from one to two, three, or four building blocks . . . . . . . . . . . . . . . . . . . . 180
6.5 Initial customer setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181
6.5.1 License agreement and password change . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181
6.5.2 System Setup wizard . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183
6.5.3 System name change . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184
6.5.4 Date and time setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185
6.5.5 Licensed functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187
6.5.6 Email Event Notifications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 188
6.5.7 Summary of changes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193
6.5.8 Encryption setting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193
Chapter 7. Host configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197
7.1 Host attachment overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 198
7.2 IBM FlashSystem V9000 setup. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199
7.2.1 Fibre Channel and SAN setup overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199
7.2.2 Fibre Channel SAN attachment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201
7.2.3 Fibre Channel direct attachment. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202
7.3 iSCSI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202
7.3.1 Initiators and targets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203
7.3.2 iSCSI nodes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203
7.3.3 iSCSI qualified name . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203

vi Implementing IBM FlashSystem V9000
7.3.4 iSCSI set up of FlashSystem V9000 and host server . . . . . . . . . . . . . . . . . . . . . 204
7.3.5 Volume discovery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205
7.3.6 Authentication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205
7.3.7 Target failover . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205
7.3.8 Host failover . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 206
7.4 File alignment for the best RAID performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207
7.5 AIX: Specific information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207
7.5.1 Optimal logical unit number configurations for AIX . . . . . . . . . . . . . . . . . . . . . . . 207
7.5.2 Configuring the AIX host . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 208
7.5.3 Configuring fast fail and dynamic tracking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 208
7.5.4 Installing the 2145 host attachment support package. . . . . . . . . . . . . . . . . . . . . 210
7.5.5 Subsystem Device Driver Path Control Module (SDDPCM). . . . . . . . . . . . . . . . 210
7.5.6 Configuring the assigned volume by using SDDPCM. . . . . . . . . . . . . . . . . . . . . 211
7.5.7 Using SDDPCM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213
7.5.8 Creating and preparing volumes for use with AIX V6.1 and SDDPCM. . . . . . . . 214
7.5.9 Expanding an AIX volume. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215
7.5.10 Running FlashSystem V9000 commands from AIX host system . . . . . . . . . . . 215
7.6 Windows: Specific information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 216
7.6.1 Configuring Windows Server 2008 and 2012 hosts . . . . . . . . . . . . . . . . . . . . . . 216
7.6.2 Configuring Windows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 216
7.6.3 Hardware lists, device driver, HBAs, and firmware levels. . . . . . . . . . . . . . . . . . 217
7.6.4 Installing and configuring the host adapter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217
7.6.5 Changing the disk timeout on Windows Server . . . . . . . . . . . . . . . . . . . . . . . . . 217
7.6.6 Installing the SDDDSM multipath driver on Windows . . . . . . . . . . . . . . . . . . . . . 218
7.6.7 Attaching FlashSystem V9000 volumes to Windows Server 2008 R2 and
Windows Server 2012. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221
7.6.8 Implementing 4 KB alignment for Windows Server 2003 . . . . . . . . . . . . . . . . . . 225
7.6.9 Extending a volume . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 226
7.6.10 Removing a disk on Windows. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227
7.6.11 Using FlashSystem V9000 CLI from a Windows host. . . . . . . . . . . . . . . . . . . . 229
7.6.12 Microsoft Volume Shadow Copy. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 230
7.7 Linux (on x86/x86_64): Specific information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 230
7.7.1 Configuring the Linux host . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 231
7.7.2 Configuration information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 231
7.7.3 Multipathing in Linux . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 231
7.8 VMware: Configuration information. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234
7.8.1 Configuring VMware hosts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234
7.8.2 Operating system versions and maintenance levels. . . . . . . . . . . . . . . . . . . . . . 234
7.8.3 HBAs for hosts that are running VMware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235
7.8.4 VMware storage and zoning guidance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235
7.8.5 Setting the HBA timeout for failover in VMware . . . . . . . . . . . . . . . . . . . . . . . . . 236
7.8.6 Multipathing in ESX. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 236
7.8.7 Attaching VMware to volumes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 236
7.8.8 Volume naming in VMware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 239
7.8.9 Setting the Microsoft guest operating system timeout . . . . . . . . . . . . . . . . . . . . 240
7.8.10 Extending a VMFS volume . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 240
7.8.11 Removing a data store from an ESX host. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 241
7.9 Oracle (Sun) Solaris hosts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 242
7.9.1 MPxIO dynamic pathing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 242
7.10 Hewlett-Packard UNIX: Configuration information . . . . . . . . . . . . . . . . . . . . . . . . . . 243
7.10.1 Operating system versions and maintenance levels. . . . . . . . . . . . . . . . . . . . . 243
7.10.2 Supported multipath solutions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 243
7.10.3 Clustered-system support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 243

Contents vii
7.10.4 Support for HP-UX with greater than eight LUNs . . . . . . . . . . . . . . . . . . . . . . . 243
7.11 Using SDDDSM, SDDPCM, and SDD web interface . . . . . . . . . . . . . . . . . . . . . . . . 243
7.12 More information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 244
Chapter 8. Using FlashSystem V9000 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 245
8.1 Overview of FlashSystem V9000 management tool . . . . . . . . . . . . . . . . . . . . . . . . . . 246
8.1.1 Access to the GUI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 246
8.1.2 Main GUI window . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 247
8.1.3 Actions menu . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 249
8.1.4 Function icons. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 250
8.1.5 Status indicators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251
8.1.6 See which user is logged in, get help, and get overview information . . . . . . . . . 252
8.1.7 System details. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253
8.1.8 Multiple selections. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253
8.2 Actions menu . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 254
8.2.1 Rename System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 255
8.2.2 Update System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 256
8.2.3 Power Off . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 256
8.3 Monitoring menu . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 257
8.3.1 System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 257
8.3.2 Events . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 258
8.3.3 Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 264
8.4 Access menu. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 269
8.4.1 Users. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 269
8.4.2 Audit log . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273
8.5 Pools menu . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 274
8.5.1 Volumes by Pool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 275
8.5.2 Creating storage pools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 276
8.5.3 Renaming a storage pool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 278
8.5.4 Deleting a storage pool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279
8.6 Volumes menu . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279
8.6.1 Opening the Volumes menu . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 280
8.6.2 Volumes window . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 280
8.6.3 Volumes by Pool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 291
8.6.4 Volume by Host. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 291
8.7 Hosts menu . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 291
8.7.1 Opening the Hosts menu . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 292
8.7.2 Host Mappings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 298
8.7.3 Ports by Hosts. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 298
8.7.4 Volumes by Host. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 299
8.7.5 Backing up data: IBM lab-based services task . . . . . . . . . . . . . . . . . . . . . . . . . . 299
8.8 Copy Services menu . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 300
8.8.1 FlashCopy. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 301
8.8.2 Consistency Groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 302
8.8.3 FlashCopy mappings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 304
8.8.4 Remote copy. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 304
8.8.5 Partnerships . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 306
Chapter 9. Configuring settings. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 307
9.1 Settings menu . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 308
9.1.1 Opening the Settings menu . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 308
9.2 Notifications menu. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 308
9.2.1 Email and call home . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 308

viii Implementing IBM FlashSystem V9000
9.2.2 SNMP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 316
9.2.3 Syslog . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 317
9.3 Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 318
9.3.1 Management IP address. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 318
9.3.2 Service IP addresses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 319
9.3.3 Ethernet ports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 320
9.3.4 Fibre Channel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 320
9.3.5 Fibre Channel ports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 321
9.4 Security menu . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 321
9.4.1 Configure remote authentication. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 322
9.4.2 Encryption . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 332
9.4.3 Enable the encryption license using V9000 GUI without Volumes . . . . . . . . . . . 334
9.4.4 Handling encryption using CLI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 340
9.5 System menu . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 341
9.5.1 Date and Time option . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 341
9.5.2 Licensed functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 342
9.5.3 Update software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 343
9.6 Support menu . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 353
9.6.1 Download support package . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 353
9.6.2 Download individual log files. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 355
9.6.3 Deleting log files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 356
9.6.4 GUI Preferences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 358
Chapter 10. Service Assistant Tool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 361
10.1 Accessing Service Assistant Tool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 362
10.2 Log in to Service Assistant Tool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 363
10.3 Home page . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 364
10.4 Collect Logs page . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 365
10.5 Manage System page . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 366
10.6 Recover System page. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 367
10.7 Re-install Software page . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 368
10.8 Update Manually page . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 369
10.9 Configure Node page . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 370
10.10 Change Service IP page . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 370
10.11 Configure CLI Access page . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 371
10.12 Restart Service page. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 371
Chapter 11. ISVs and use cases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 373
11.1 Use cases and ISV overview and considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . 374
11.2 VMware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 374
11.3 Database acceleration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 375
11.3.1 Oracle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 375
11.3.2 Microsoft SQL Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 377
11.3.3 DB2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 378
11.3.4 Architecture for SAP landscapes featuring IBM FlashSystem . . . . . . . . . . . . . 379
11.3.5 Epic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 382
11.4 Deduplication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 383
11.4.1 Atlantis ILIO . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 383
11.4.2 Permabit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 384
11.4.3 IBM ProtecTIER . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 384
11.5 VMware vCloud integration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 385
11.5.1 FlashSystem V9000 in a VMware vCloud environment . . . . . . . . . . . . . . . . . . 386
11.5.2 Storage Integration Server: Delivering the function of Spectrum Control . . . . . 387

Contents ix
11.5.3 vCloud Suite . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 388
11.5.4 FlashSystem V9000 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 389
11.5.5 Use case: Provisioning FlashSystem V9000 volumes using VMware . . . . . . . 389
11.5.6 Single-site HA: Volume mirroring to another storage system . . . . . . . . . . . . . . 393
11.5.7 Cross-site HA: Extended distance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 394
11.6 Running FlashSystem V9000 in an IBM Virtual Storage Center environment . . . . . 397
11.6.1 Licensing. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 399
Chapter 12. Hints and tips . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 401
12.1 Performance data and statistics gathering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 402
12.1.1 FlashSystem V9000 controller performance overview . . . . . . . . . . . . . . . . . . . 402
12.1.2 Performance monitoring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 408
12.2 Command-line hints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 417
12.2.1 Running commands on the FlashSystem V9000 . . . . . . . . . . . . . . . . . . . . . . . 417
12.2.2 Creating connections . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 423
12.2.3 V9000 command-line scripting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 427
12.2.4 Sample commands of mirrored VDisks. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 430
12.2.5 Backup V9000 configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 432
12.2.6 Using the V9000 Software Upgrade Test Utility . . . . . . . . . . . . . . . . . . . . . . . . 433
12.2.7 Secure erase of data. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 434
12.3 Call home process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 441
12.3.1 Call home details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 441
12.3.2 Email alert . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 442
12.3.3 Inventory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 442
12.4 Service support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 442
12.4.1 IBM Storage Technical Advisor. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 442
12.4.2 How a FlashSystem V9000 is entitled for support . . . . . . . . . . . . . . . . . . . . . . 443
12.4.3 Providing Logs to IBM ECuRep . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 444
12.4.4 Downloading from IBM Fix Central . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 449
Appendix A. Guidelines: Port utilization in a V9000 scalable environment. . . . . . . . 455
A.1 Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 456
A.2 Guidelines: The performance method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 457
A.3 Guidelines: The infrastructure savings method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 462
A.4 Guidelines: Zoning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 463
A.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 466
Related publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 467
IBM Redbooks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 467
Other publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 467
Online resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 467
Help from IBM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 468

x Implementing IBM FlashSystem V9000

© Copyright IBM Corp. 2015. All rights reserved. xi
Notices
This information was developed for products and services offered in the U.S.A.
IBM may not offer the products, services, or features discussed in this document in other countries. Consult
your local IBM representative for information on the products and services currently available in your area. Any
reference to an IBM product, program, or service is not intended to state or imply that only that IBM product,
program, or service may be used. Any functionally equivalent product, program, or service that does not
infringe any IBM intellectual property right may be used instead. However, it is the user's responsibility to
evaluate and verify the operation of any non-IBM product, program, or service.
IBM may have patents or pending patent applications covering subject matter described in this document. The
furnishing of this document does not grant you any license to these patents. You can send license inquiries, in
writing, to:
IBM Director of Licensing, IBM Corporation, North Castle Drive, Armonk, NY 10504-1785 U.S.A.
The following paragraph does not apply to the United Kingdom or any other country where such
provisions are inconsistent with local law: INTERNATIONAL BUSINESS MACHINES CORPORATION
PROVIDES THIS PUBLICATION "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR
IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF NON-INFRINGEMENT,
MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Some states do not allow disclaimer of
express or implied warranties in certain transactions, therefore, this statement may not apply to you.
This information could include technical inaccuracies or typographical errors. Changes are periodically made
to the information herein; these changes will be incorporated in new editions of the publication. IBM may make
improvements and/or changes in the product(s) and/or the program(s) described in this publication at any time
without notice.
Any references in this information to non-IBM websites are provided for convenience only and do not in any
manner serve as an endorsement of those websites. The materials at those websites are not part of the
materials for this IBM product and use of those websites is at your own risk.
IBM may use or distribute any of the information you supply in any way it believes appropriate without incurring
any obligation to you.
Any performance data contained herein was determined in a controlled environment. Therefore, the results
obtained in other operating environments may vary significantly. Some measurements may have been made
on development-level systems and there is no guarantee that these measurements will be the same on
generally available systems. Furthermore, some measurements may have been estimated through
extrapolation. Actual results may vary. Users of this document should verify the applicable data for their
specific environment.
Information concerning non-IBM products was obtained from the suppliers of those products, their published
announcements or other publicly available sources. IBM has not tested those products and cannot confirm the
accuracy of performance, compatibility or any other claims related to non-IBM products. Questions on the
capabilities of non-IBM products should be addressed to the suppliers of those products.
This information contains examples of data and reports used in daily business operations. To illustrate them
as completely as possible, the examples include the names of individuals, companies, brands, and products.
All of these names are fictitious and any similarity to the names and addresses used by an actual business
enterprise is entirely coincidental.
COPYRIGHT LICENSE:
This information contains sample application programs in source language, which illustrate programming
techniques on various operating platforms. You may copy, modify, and distribute these sample programs in
any form without payment to IBM, for the purposes of developing, using, marketing or distributing application
programs conforming to the application programming interface for the operating platform for which the sample
programs are written. These examples have not been thoroughly tested under all conditions. IBM, therefore,
cannot guarantee or imply reliability, serviceability, or function of these programs.

xii Implementing IBM FlashSystem V9000
Trademarks
IBM, the IBM logo, and ibm.com are trademarks or registered trademarks of International Business Machines
Corporation in the United States, other countries, or both. These and other IBM trademarked terms are
marked on their first occurrence in this information with the appropriate symbol (® or ™), indicating US
registered or common law trademarks owned by IBM at the time this information was published. Such
trademarks may also be registered or common law trademarks in other countries. A current list of IBM
trademarks is available on the Web at http://www.ibm.com/legal/copytrade.shtml
The following terms are trademarks of the International Business Machines Corporation in the United States,
other countries, or both:
AIX®
AIX 5L™
DB2®
DB2 Universal Database™
developerWorks®
DS4000®
DS5000™
DS8000®
Easy Tier®
FlashCopy®
FlashSystem™
GPFS™
HyperFactor®
IBM®
IBM FlashCore™
IBM FlashSystem®
IBM SmartCloud®
IBM Spectrum™
IBM Spectrum Storage™
Informix®
MicroLatency®
NetView®
Passport Advantage®
POWER®
Power Systems™
POWER8™
PowerPC®
ProtecTIER®
pureScale®
Real-time Compression™
Redbooks®
Redpaper™
Redbooks (logo) ®
Storwize®
System i®
System p®
System Storage®
Tivoli®
Tivoli Enterprise Console®
Variable Stripe RAID™
XIV®
z/OS®
The following terms are trademarks of other companies:
Intel, Intel Xeon, Intel logo, Intel Inside logo, and Intel Centrino logo are trademarks or registered trademarks
of Intel Corporation or its subsidiaries in the United States and other countries.
Linux is a trademark of Linus Torvalds in the United States, other countries, or both.
Microsoft, Windows, and the Windows logo are trademarks of Microsoft Corporation in the United States,
other countries, or both.
Java, and all Java-based trademarks and logos are trademarks or registered trademarks of Oracle and/or its
affiliates.
UNIX is a registered trademark of The Open Group in the United States and other countries.
Other company, product, or service names may be trademarks or service marks of others.

IBM REDBOOKS PROMOTIONS
Find and read thousands of
IBM Redbooks publications
Search, bookmark, save and organize favorites
Get up-to-the-minute Redbooks news and announcements
Link to the latest Redbooks blogs and videos
Download
Now
Get the latest version of the Redbooks Mobile App
iOS
Android
Place a Sponsorship Promotion in an IBM
Redbooks publication, featuring your business
or solution with a link to your web site.
Qualiﬁed IBM Business Partners may place a full page
promotion in the most popular Redbooks publications.
Imagine the power of being seen by users who download
millions of Redbooks publications each year!
®
®
Promote your business
in an IBM Redbooks
publication
ibm.com/Redbooks
About Redbooks Business Partner Programs
IBM Redbooks promotions

THIS PAGE INTENTIONALLY LEFT BLANK

© Copyright IBM Corp. 2015. All rights reserved. xv
Preface
Storage capacity and performance requirements are growing faster than ever before, and the
costs of managing this growth are depleting more of the IT budget.
The IBM® FlashSystem™ V9000 is the premier, fully integrated, Tier 1, all-flash offering from
IBM and has changed the economics of today's data center by eliminating storage
bottlenecks. Its software-defined storage features simplify data management, improve data
security, and preserve your investments in storage.
IBM FlashSystem® V9000 FlashCore technology and advanced software-defined storage are
available in one solution in a compact 6U form factor. FlashSystem V9000 improves business
application availability and delivers greater resource utilization so you can get the most from
your storage resources, and achieve a simpler, more scalable, and cost efficient IT
Infrastructure.
Using IBM Storwize® family functions, management tools, and interoperability combines the
performance of FlashSystem architecture with the advanced functions of software-defined
storage to deliver performance, efficiency, and functions that meet the needs of enterprise
workloads that demand IBM MicroLatency® response time.
This IBM Redbooks® publication introduces the IBM FlashSystem V9000. It describes the
product architecture, software and hardware, implementation, and hints and tips. It illustrates
use cases and independent software vendor (ISV) scenarios that demonstrate real-world
solutions, and also examples of the benefits gained by integrating the FlashSystem storage
into business environments.
This book offers FlashSystem V9000 scalability concepts and guidelines for planning,
installing, and configuring, which can help environments scale up and out to add more flash
capacity and expand virtualized systems. Port utilization methodologies are provided to help
maximize the full potential of performance and low latency of IBM FlashSystem V9000 in your
scalable environment.
This book is intended for pre-sales and post-sales technical support professionals, storage
administrators, and anyone who wants to understand how to implement this exciting
technology.
The following overview is of the new IBM Spectrum™ Storage family of offerings that are
described and referenced throughout this book:
IBM Spectrum Storage™
The IBM Spectrum Storage family is based on proven technologies and is designed
specifically to simplify storage management, scale to keep up with data growth, and
optimize data economics. It represents a new, more agile way of storing data, and helps
organizations prepare for new storage demands and workloads. The software-defined
storage solutions included in the IBM Spectrum Storage family can help organizations
simplify their storage infrastructures, cut costs, and start gaining more business value from
their data.
For details about the entire IBM Spectrum Storage family, see this web page:
http://www.ibm.com/systems/storage/spectrum/

xvi Implementing IBM FlashSystem V9000
IBM Spectrum Control
This provides efficient infrastructure management for virtualized, cloud, and
software-defined storage to simplify and automate storage provisioning, capacity
management, availability monitoring, and reporting.
The functionality of IBM Spectrum Control is provided by IBM Data and Storage
Management Solutions and includes functionality delivered by IBM SmartCloud® Virtual
Storage Center, Tivoli® Storage Productivity Center, IBM Storage Integration Server, and
others.
For more information, see this web page:
http://www.ibm.com/software/tivoli/csi/cloud-storage/
IBM Spectrum Virtualize
IBM Spectrum Virtualize is industry-leading storage virtualization that enhances existing
storage to improve resource utilization and productivity in order to achieve a simpler, more
scalable, and cost-efficient IT infrastructure. The functionality of IBM Spectrum Virtualize
is provided by IBM SAN Volume Controller.
For details about IBM Spectrum Virtualize - SAN Volume Controller see this web page:
http://www.ibm.com/systems/storage/software/virtualization/svc/
IBM Spectrum Scale
IBM Spectrum Scale is a proven, scalable, high-performance data and file management
solution, based upon IBM General Parallel File System or GPFS™. IBM Spectrum Scale
technology is a high-performance enterprise file management platform; it can help you
move beyond simply adding storage to optimize data management.
For more information, see this web page:
http://www.ibm.com/systems/storage/spectrum/scale/
IBM Spectrum Accelerate
IBM Spectrum Accelerate is a software-defined storage solution, which is born of the
proven IBM XIV® integrated storage offering, and which designed to help speed delivery
of data across the organization and add extreme flexibility to cloud deployments
IBM Spectrum Accelerate delivers hotspot-free performance, easy management scaling,
and proven enterprise functionality such as advanced mirroring and flash caching to
different deployment platforms.
For more information, see the IBM Spectrum Accelerate web page:
http://www.ibm.com/systems/storage/spectrum/accelerate/

Preface xvii
Authors
This book was produced by a team of specialists from around the world working at the
International Technical Support Organization, Poughkeepsie Center.
Karen Orlando is a Project Leader at the International
Technical Support Organization, Tucson Arizona Center. Karen
has over 25 years in the IT industry with extensive experience
in open systems management, and information and software
development of IBM hardware and software storage. She holds
a degree in Business Information Systems from the University
of Phoenix and is Project Management Professional (PMP),
certified since 2005.
Christophe Fagiano is a Technical Advisor supporting XIV,
FlashSystem, SONAS, and IBM ProtecTIER®. Christophe
began his career with IBM in 1991 working in development of
communication controllers, then in Level 3 support. Christophe
also worked in development of Power PC chips with his area of
expertise in video/sound interfaces. Other development roles
include development of virtualization environments for IBM
AIX® and Linux platforms. Christophe works in France.
Detlef Helmbrecht is an Advanced Technical Skills (ATS) IT
Specialist working for the IBM Systems and Technology Group.
He at the EMEA Storage Competence Center (ESCC),
Germany. Detlef has over 25 years of experience in IT,
performing numerous different roles, including software design,
sales, and solution architect. His areas of expertise include
high performance computing, disaster recovery, archiving,
application tuning, and FlashSystem.
Jeffrey Irving is an IBM Storage Technical Advisor supporting
multiple storage products. He has more than 30 years of
experience in IT. Beginning his career with AT&T Bell Labs
testing UNIX operating systems, he moved on to
supercomputers, leading several test software test teams in the
process. Jeff has been with IBM 15 years, specializing in
storage virtualization and customer support, and he leads the
Technical Advisor team for FlashSystem products. Jeff works in
Wisconsin, US.
Arne Lehfeldt is an IBM Client Technical Specialist for storage
products and has more than 10 years of experience in storage.
Arne started in the storage support center and then moved
through various roles within IBM including the services
organization where he lead two big storage migration projects.
Arne has been with IBM for 12 years and works in Mainz,
Germany. He is an IBM Certified IT Specialist and has obtained
Brocade BCFP and IBM SVC certifications.

xviii Implementing IBM FlashSystem V9000
Thanks to the following people for their contributions to this project:
Dave Gimpl, Glen Jaquette, Steven Keller, Justin Haggard, Jon Herd, Steven Hurley,
Matt Key, Mike Mehall, Kim Miller, Jackson Shea, Roger Strommen, Bobby Sumners,
IBM Systems
Special thanks to James Thompson from IBM Systems Performance, for his contribution of
Appendix A, “Guidelines: Port utilization in a V9000 scalable environment” on page 455.
Now you can become a published author, too!
Here’s an opportunity to spotlight your skills, grow your career, and become a published
author—all at the same time! Join an ITSO residency project and help write a book in your
area of expertise, while honing your experience using leading-edge technologies. Your efforts
will help to increase product acceptance and customer satisfaction, as you expand your
network of technical contacts and relationships. Residencies run from two to six weeks in
length, and you can participate either in person or as a remote resident working from your
home base.
Find out more about the residency program, browse the residency index, and apply online at:
ibm.com/redbooks/residencies.html
Corne Lottering is a FlashSystem Client Technical Specialist
in the US, focusing on technical sales in Texas and Louisiana.
He has been with IBM for more than 14 years, and has
experience in a wide variety of storage technologies, including
the IBM System Storage® DS5000™, DS8000®, Storwize,
XIV, FlashSystem, IBM SAN switches, IBM Tape Systems, and
storage software. Since joining IBM, he has fulfilled roles in
support, implementation and pre-sales support across various
African and Middle Eastern countries. Corne is the author of a
number of IBM Redbooks publications related to the midrange
IBM DS Storage range of products.
Alexander Watson is a Product Engineer for IBM
FlashSystem Storage in the US. He is a Subject Matter Expert
on SAN switches, IBM Midrange system storage products, and
IBM Storwize Virtual Storage products. Al has over 20 years of
experience in planning, managing, designing, implementing,
problem analysis, and tuning of SAN environments and storage
systems. He has worked at IBM for 15 years. Al’s areas of
expertise include SAN fabric networking, Open System
Storage IO, Virtualization, IBM FlashSystem Storage solutions.

Preface xix
Comments welcome
Your comments are important to us!
We want our books to be as helpful as possible. Send us your comments about this book or
other IBM Redbooks publications in one of the following ways:
Use the online Contact us review Redbooks form found at:
ibm.com/redbooks
Send your comments in an email to:
redbooks@us.ibm.com
Mail your comments to:
IBM Corporation, International Technical Support Organization
Dept. HYTD Mail Station P099
2455 South Road
Poughkeepsie, NY 12601-5400
Stay connected to IBM Redbooks
Find us on Facebook:
http://www.facebook.com/IBMRedbooks
Follow us on Twitter:
http://twitter.com/ibmredbooks
Look for us on LinkedIn:
http://www.linkedin.com/groups?home=&gid=2130806
Explore new Redbooks publications, residencies, and workshops with the IBM Redbooks
weekly newsletter:
https://www.redbooks.ibm.com/Redbooks.nsf/subscribe?OpenForm
Stay current on recent Redbooks publications with RSS Feeds:
http://www.redbooks.ibm.com/rss.html

xx Implementing IBM FlashSystem V9000

© Copyright IBM Corp. 2015. All rights reserved. 1
Chapter 1. FlashSystem V9000 introduction
This chapter introduces the IBM FlashSystem V9000 storage system and its core value,
benefits, and technological advantages.
This chapter includes the following topics:
FlashSystem V9000 storage overview
Why flash matters
IBM FlashSystem family: Product differentiation
FlashSystem V9000: IBM Tier 1 storage
FlashCore technology
Architectural design overview
Advanced software features
1

2 Implementing IBM FlashSystem V9000
1.1 FlashSystem V9000 storage overview
The IBM FlashSystem V9000, as shown in Figure 1-1, delivers high capacity and fully
integrated management for the enterprise data center. FlashSystem V9000 uses a fully
featured and scalable all-flash architecture that performs at up to 2.5 million IOPS with
IBM MicroLatency, is scalable to 19.2 GBps, and delivers an effective capacity of up to
2.28 PB. Using its flash-optimized design, FlashSystem V9000 can provide response times
of 200 microseconds. It delivers better acquisition costs than a high-performance spinning
disk for the same effective capacity while achieving five times the performance, making it
ideal for environments that demand extreme performance.
Figure 1-1 IBM FlashSystem V9000
With IBM Real-time Compression™ technology, FlashSystem V9000 (V9000) further extends
the economic value of all-flash systems. The V9000 provides up to two times the
improvement in Real-time Compression over the model it is replacing. Using the optional
Real-time Compression and other design elements, the V9000 provides up to 57 TB usable
capacity and up to 285 TB effective capacity in only 6U. This scales to 456 TB usable capacity
and up to 2.28 PB effective capacity in only 36U.
FlashSystem V9000 delivers enterprise-class advanced storage capabilities, including these
among others:
IBM Real-time Compression
IBM EasyTier
Thin provisioning
Copy services
Data virtualization
Highly available configurations
Advanced data services that are provided include copy services, mirroring, replication,
external virtualization, and VMware vSphere Storage APIs - Array Integration (VAAI) support.
Host interface support includes 8 Gb and 16 Gb FC and 10 Gb FCoE/iSCSI. AES 256
hardware-based encryption adds to the rich feature set.
IBM FlashSystem V9000, including its MicroLatency module (flash modules), is covered by
up to seven years of total hardware support through the applicable warranty period.

Chapter 1. FlashSystem V9000 introduction 3
1.2 Why flash matters
Flash is a vibrant and fast growing technology. Clients are looking to solve data center
problems, optimize applications, reduce costs, and grow their businesses.
Here are several reasons why flash is a must in every data center, and why an IBM
FlashSystem changes storage economics:
Reduces application and server licensing costs, especially those related to databases and
virtualization solutions.
Improves application efficiency, that is, an application’s ability to process, analyze, and
manipulate more information, faster.
Improves server efficiency. Helps you get more out of your existing processors, use less
RAM per server, and consolidate operations by having server resources spend more time
processing data as opposed to waiting for data.
Improves storage operations. Helps eliminate costly application tuning, wasted developer
cycles, storage array hot spots, array tuning, and complex troubleshooting. Decreases
floor space usage and energy consumption by improving overall storage environment
performance.
Enhances performance for critical applications by providing the lowest latency in the
market.
Almost all technological components in the data center are getting faster, including central
processing units, network, storage area networks (SANs), and memory. All of them have
improved their speeds by a minimum of 10x; some of them by 100 times (100x), such as data
networks. However, spinning disk has only increased its performance 1.2x.
The IBM FlashSystem V9000 provides benefits that include a better user experience, server
and application consolidation, development cycle reduction, application scalability, data
center footprint savings, and improved price performance economics.
Flash improves the performance of applications that are critical to the user experience, such
as market analytics and research applications, trading and data analysis interfaces,
simulation, modeling, rendering, and so on. Server and application consolidation is possible
because of the increased process utilization resulting from the low latency of flash memory,
which enables a server to load more users, more databases, and more applications. Flash
provides or gives back time for further processing within the existing resources of such
servers. Clients soon realize that there is no need to acquire or expand server resources as
often or as soon as was previously expected.
Development cycle reduction is possible because developers spend less time designing an
application to work around the inefficiencies of hard disk drives (HDDs) and less time tuning
for performance.
Data center footprint savings are because of the high density and high performance per
density flash solutions that are replacing racks of spinning HDDs. Reducing the data center
footprint also translates into power and cooling savings, making flash one of the greenest
technologies for the data center.
Note: Improved price:performance economics are because of the low cost for performance
from the IBM FlashSystem. The cost savings result from deploying fewer storage
enclosures, fewer disk drives, fewer servers with fewer processors, and less RAM while
using less power, space, and cooling. Flash is one of the best tools for the data center
manager for improving data center economics.

1.3 IBM FlashSystem family: Product differentiation
Flash is used widely in the data center, either within a server (Peripheral Component
Interconnect Express (PCIe) cards or internal solid-state drives (SSDs)), in storage arrays
(hybrid or all-flash), appliances, or platform solutions (hardware, software, network). Flash
can be used as cache or as a data tier. Because of the vast and wide adoption of flash, there
are several flash architectures and, therefore, criteria that can be applied to compare flash
options. See Figure 1-2.
Figure 1-2 The various deployments of flash
Most storage vendors use and promote flash. The difference is how it is implemented, and the
impact that such implementation has on the economics (cost reduction and revenue
generation) for clients.
Flash technology is used to eliminate the storage performance bottleneck. The IBM
FlashSystem family is a key shared-storage market leader and provides extremely low
latency and consistent response times. It is designed and built specifically for flash.
Some other vendors create flash appliances based on commodity server platforms and use
software-heavy stacks. Some suppliers use hardware technologies designed and created for
disk, not flash. Some hybrid arrays combine existing storage designs, spinning HDDs, and
SSD. The IBM storage portfolio includes SSD and flash on a variety of storage platforms;
however, these alternate solutions do not have the same low latency (MicroLatency) as the
hardware-accelerated FlashSystem.
IBM FlashSystem family versus SSD-based storage arrays
Flash memory technologies appeared in the traditional storage systems some time ago.
These SSD-based storage arrays help to successfully address the challenge of increasing
I/Os per second (IOPS) needed by applications, and the demand for lower response times in
particular tasks. An implementation example is the IBM Easy Tier® technology. For an
overview of this technology, see 3.2.1, “IBM Easy Tier” on page 71.
However, these technologies typically rely on flash in the format of Fibre Channel (FC),
serial-attached SCSI (SAS), or Serial Advanced Technology Attachment (SATA) disks, placed
in the same storage system as traditional spinning disks, and using the same resources and
data paths. This approach can limit the advantages of flash technology because of the
limitations of traditional disk storage systems.

IBM FlashSystem storage provides a hardware-only data path that realizes all of the
potential of flash memory. These systems differ from traditional storage systems, both in
the technology and usage.
An SSD device with an HDD disk form factor has flash memory that is put into a carrier or tray.
This carrier is inserted into an array, such as an HDD. The speed of storage access is limited
by the following technology because it adds latency and cannot keep pace with flash
technology:
Array controllers and software layers
SAS controllers and shared bus
Tiering and shared data path
Form factor enclosure
IBM FlashSystem products are fast and efficient. The hardware-only data path has a
minimum number of software layers, which are mostly firmware components, and
management software that is separated from the data path (out-of-band). The only other
family of products with hardware-only access to flash technology is the PCI Express (PCIe)
flash product family, where products are installed into a dedicated server. With the
appearance of the IBM FlashSystem, the benefits of PCIe flash products to a single server
can now be shared by many servers.
1.4 FlashSystem V9000: IBM Tier 1 storage
The market for all-flash arrays is saturated with products aiming to replace enterprise storage
arrays but consistently failing to deliver the breadth of data lifecycle, storage services, or the
scalability delivered by incumbent solutions. Alternatively, hybrid arrays loaded with storage
services consistently lack the low latency and performance scalability delivered by all-flash
arrays.
FlashSystem V9000 merges IBM software-defined storage with the scalable performance of
IBM FlashSystem storage to accelerate critical business applications and decrease data
center costs simultaneously. As a result, your organization can gain a competitive advantage
through a more flexible, responsive, and efficient storage environment.
The FlashSystem V9000 provides a true paradigm shift in enterprise storage with the focus
on the dimensions of value, scalable performance, enduring economics, and agile Integration,
as shown in Figure 1-3.
Figure 1-3 IBM FlashSystem V9000 dimensions of value

1.5 FlashCore technology
The IBM FlashCore™ technology, used in the IBM FlashSystem V9000, employs several new
and patented mechanisms, to achieve greater capacity and throughput, at a lower cost than
the previous IBM FlashSystem V840. Figure 1-4 shows the three major areas within IBM
FlashCore technology and the unique IBM attributes of each one.
Figure 1-4 IBM FlashCore Technology
1.5.1 Hardware accelerated I/O
IBM FlashSystem V9000 hardware design offers several unique IBM components including
Hardware RAID, Non-blocking Crossbar Switch, Hardware Only Data Path, Single Box Highly
Available Architecture, Concurrent Code Load and Concurrent Maintenance.
1.5.2 IBM MicroLatency module
IBM FlashSystem V9000 uses the new 20nm multi-level cell (MLC) flash card memory chips
and either 1.2 TB, 2.9 TB, or 5.7 TB capacity IBM MicroLatency modules, as shown in
Figure 1-5 on page 7. The IBM FlashSystem V9000 design also employs the use of IBM
Engineered Massively Parallel Design, FPGAs in the Data Path, Distributed RAM and
High-Speed Interfaces plus Hardware-based Data-at-Rest Encryption.

Figure 1-5 IBM MicroLatency module
1.5.3 Advanced flash management
The IBM FlashSystem V9000 has unique patented designs to ensure maximum availability.
These include IBM Variable Stripe RAID™, IBM engineered error correction code (ECC), IBM
Optimized Over-provisioning, Advanced Wear Leveling on IBM MicroLatency modules, Write
Buffer And Hardware Offload, and Garbage Collection. See “Terminology” on page 20.
All this is made possible because of the following IBM patented and world class innovations:
ECC algorithms that correct very high bit-error rates
Variable voltage and read level shifting that help to maximize flash endurance
Health binning and heat segregation, which continually monitor the health of flash blocks
and performs asymmetrical wear leveling and sub-chip tiering.
This all results in providing up to 57% improvement in endurance with a potential 45%
reduction in write amplification.
1.5.4 Flash wear guarantee
Through close collaboration between IBM Research, software development and flash
engineering, IBM created an advanced flash characterization platform to test thousands
of flash devices over a lifetime of wear.
As a result, IBM has made major strides in advanced flash management. This improves MLC
flash endurance 9x over standard implementations, and provides enterprise reliability and
performance with IBM MicroLatency.
The IBM FlashSystem V9000, including its IBM MicroLatency modules, is covered by up to
seven years of total hardware support through the applicable warranty period plus up to six
years of optional post-warranty hardware maintenance. Clients may purchase the
post-warranty hardware maintenance either at the time of system purchase or up until IBM
announces withdrawal from marketing or withdrawal from service.

1.6 Architectural design overview
The IBM FlashSystem V9000 consists of two control enclosures (AC2), one storage
enclosure (AE2), and software and hardware features, to make up a building block. Building
blocks can be either fixed or scalable. You can combine scalable building blocks to create
larger clustered systems in such a way that operations are not disrupted.
A scalable building block can be scaled up by adding IBM FlashSystem V9000 AE2 storage
enclosures for increased storage capacity. You can add a maximum of four extra storage
enclosures and one extra storage enclosure per building block.
A scalable building block can be scaled out by combining up to four building blocks to provide
higher IOPS and bandwidth needs for increased performance.
Figure 1-6 illustrates the scalable capacity of FlashSystem V9000. It also shows that extra
FlashSystem V9000 storage enclosures (SEs) can be added to a single building block (BB),
and also to two, three, or four BBs.
Figure 1-6 FlashSystem V9000 scalability
The FlashSystem V9000 AC2 control enclosures combines software and hardware into a
comprehensive, modular appliance that uses symmetric virtualization.
Symmetric virtualization is achieved by creating a pool of managed disks (MDisks) from
the internal storage and from attached storage systems. Those storage systems are then
mapped to a set of volumes for use by attached host systems. System administrators can
view and access a common pool of storage on the storage area network (SAN). With this
functionality, administrators can use storage resources more efficiently and it provides a
common base for advanced functions.

Also with FlashSystem V9000, you can migrate data across MDisks without interfering with
any host applications that are simultaneously accessing or writing data.
The FlashSystem V9000 includes a single easy-to-use management graphical user interface
(GUI) to help you monitor, manage, and configure your system.
The AE2 storage enclosure components include flash modules, battery modules, and
canisters.
The AE2 storage enclosure, with an all-hardware data path using field programmable-gate
array (FPGA) modules, is engineered to deliver the lowest possible latency. The modules
incorporate proprietary flash controllers and use numerous patented technologies. The flash
controllers have a proprietary logic design, firmware, and system software.
There are no commodity 2.5-inch SSDs, PCIe cards, or any other significant non IBM
assemblies within the system. The flash chips, FPGA chips, processors, and other
semiconductors in the system are carefully selected to be consistent with the purpose-built
design, which is designed from the ground up for high performance, reliability, and efficiency.
The AE2 storage enclosures offer the following notable architectural concepts:
Hardware-only data path.
Use of FPGAs extensively.
Field-upgradable hardware logic.
Less expensive design cycle.
Extremely high degree of parallelism.
Intelligent flash modules.
Distributed computing model.
Low-power IBM PowerPC® processors.
Interface and flash processors run thin real-time operating systems.
With minimal management communication, the management processor communicates
with the interface and flash processors through an internal network.
1.6.1 Hardware-only data path
The hardware-only data path design of the AE2 storage enclosures eliminates software-layer
latency. To achieve extremely low latencies, the IBM FlashSystem advanced software
functions are carefully assessed and implemented.
In the AE2 storage enclosures, data traverses the array controllers through FPGAs and
dedicated, low-power CPUs. There are no wasted cycles on interface translation, protocol
control, or tiering.
The AE2 storage enclosures, with an all-hardware data path design, have an internal
architecture that differs from other hybrid (SSD and HDD) or SSD-only based disk systems.
1.6.2 The 20nm flash card memory chips
The flash chip is the basic storage component of the IBM MicroLatency module. A maximum
of 64 multi-level cell (MLC) flash chips can exist on each flash module. To maintain consistent
wearing and reliability, combining flash chips of different flash technologies is not supported in
the same flash module or storage system.
The IBM FlashSystem V9000 employs the new 20nm MLC chips which are of a higher
density than the 24nm eMLC chips used in the IBM FlashSystem V840. This new design of

chips allows the IBM FlashSystem V9000 to package greater densities of flash memory per
card while retaining the same if not better performance and wear.
IBM patented ECC correction and checking-algorithms ensure the same or greater
performance from the MCL based chips, with a greater capacity for the same footprint
and at a lower cost per terabyte.
1.6.3 Flash module capacities
The IBM FlashSystem V9000 uses either 1.2 TB. 2.9 TB, or 5.7 TB IBM MicroLatency
modules. This is a 40% increase in capacity per module over the IBM FlashSystem V840.
They must be of the same capacity throughout the AE2 storage enclosure and cannot be
intermixed with the older 24nm flash modules.
Only RAID 5 is supported on the IBM FlashSystem V9000 with configurations of 4, 6, 8, 10,
and 12 modules when using the 1.2 TB IBM MicroLatency modules and 6, 8, 10, and 12
modules when using the 2.9 TB or 5.7 TB IBM MicroLatency modules
If fewer than 12 modules are installed, flash module fillers must be installed in the empty bays
to maintain cooling airflow in the system enclosure.
1.6.4 Gateway interface FPGA
The gateway interface FPGA is responsible for providing I/O to the flash module and direct
memory access (DMA) path. It is located on the flash module and has two connections to the
backplane.
1.6.5 Flash controller FPGA
The flash controller FPGA of the flash module provides access to the flash chips and is
responsible for the following functions:
Provides data path and hardware I/O logic
Uses lookup tables and a write buffer
Controls 13 or 16 chips (module-size-dependent)
Operates independently of other controllers
Maintains write ordering and layout
Provides write setup
Maintains garbage collection
Provides error handling
Figure 1-7 on page 11 shows the flash controller design details.

Figure 1-7 Flash controller design
The concurrent operations performed on the flash chips include moving data in and out of the
chip through DMA, and by internally moving data and performing erasures. While actively
transferring user data in the service of host-initiated I/O, the system can simultaneously run
garbage collection activities without affecting the I/O. The ratio of transparent background
commands running concurrent to active data transfer commands is 7:1.
A maximum of four flash controllers per IBM MicroLatency module exist: two for each primary
board and two for each expansion board.
1.6.6 IBM Variable Stripe RAID and two-dimensional flash RAID overview
Storage systems of any kind are typically designed to perform two main functions: to store
and protect data. The IBM FlashSystem V9000 includes the following options for data
protection.
RAID data protection:
– IBM Variable Stripe RAID
– Two-dimensional (2D) Flash RAID
Flash memory protection methods
Optimized RAID rebuild times
Table 1-1 lists the various methods of protection.
Table 1-1 Various types of IBM FlashSystem protection
Layer Managed by Protection
System-level RAID 5 Centralized RAID controllers Module failure
Module-level RAID 5 Each module across the chips Chip failure and page failure
Module-level Variable Stripe
RAID
Each module across the chips Sub-chip, chip, or multi-chip
failure
Chip-level error correction code
(ECC)
Each module using the chips Bit and block error

1.6.7 Fixed and scalable building blocks
FlashSystem V9000 can be configured as a fixed building block or a scalable building block.
A fixed building block contains one FlashSystem V9000. The AE2 storage enclosure is cabled
directly to each AC2 control enclosure using 8 Gb links, and each AC2 control enclosure is
connected to switches or to a host. The AC2 control enclosures are directly connected,
without the use of switches or a SAN fabric, to form the cluster links. A fixed building block can
be upgraded to a scalable building block, but the upgrade process is disruptive to operations.
Scalable building blocks can contain multiple AC2 control enclosure pairs and multiple AE2
storage enclosures. In a scalable building block, AC2 control enclosures are not cabled to
each other. This infrastructure means that you can add building blocks or storage enclosures
nondisruptively. Fibre Channel switches are used to create a private storage fabric. The Fibre
Channel switch fabric is dedicated, and is not shared with hosts or server-side storage area
networks (SANs). After connecting the components in a scalable building block, no physical
cable connects any host to any switch in the internal Fibre Channel switch fabric. This private
fabric is therefore not impacted by traditional host-side SAN traffic, saturation issues, or
accidental or intentional zoning issues, thus providing maximum availability and maximum
cluster performance.
The Fibre Channel switch fabric is isolated from the host or server-side SAN for these
reasons:
So that any host or server does not access AE2 storage enclosures directly
So that no congestion to the host or server-side SAN can cause potential performance
implications for both the host or server-side SAN and the FlashSystem V9000
1.6.8 Scale up and scale out solution
We can start with a fixed building block, or we can opt for a scalable building block that
includes two 16 Gb FC switches, which allows you to add extra storage enclosures and
building blocks.
IBM gives you the flexibility and the ability to scale up for increasing capacity and scaling out
as needed for increases in performance and capacity.
Note: The proprietary 2D Flash RAID data protection scheme of the IBM FlashSystem
V9000 storage system combines system-level RAID 5 and module-level Variable Stripe
RAID (not only module-level RAID).

Figure 1-8 shows a scale out solution with four FlashSystem V9000 building blocks, using the
16 Gb FC switches for interconnections.
Figure 1-8 Scale out FlashSystem V9000 solution
Figure 1-9 shows a scale up solution with one FlashSystem V9000 scalable building block
and four FlashSystem V9000 AE2 storage systems.
Figure 1-9 Scale up FlashSystem V9000 solution
Figure 1-10 on page 14 shows a scale up and scale out solution with four FlashSystem
V9000 building blocks and four FlashSystem V9000 AE2 storage systems, indicating a
maximum supported configuration.
Fibre Channel Switches
FlashSystem V9000
FlashSystem V9000
FlashSystem V9000
FlashSystem V9000
FlashSystem AE2 V9000
FlashSystem V9000

Figure 1-10 Scale up and scale out FlashSystem V9000 solution
For more details about FlashSystem V9000 scale up or scale out solutions, see Chapter 5,
“Scalability” on page 139.
1.7 Advanced software features
The FlashSystem V9000 can function as a feature-rich, software-defined storage layer that
virtualizes and extends the functionality of all managed storage. These include Real-time
Compression, dynamic tiering, thin provisioning, snapshots, cloning, replication, data copy
services and high-availability configurations. In this capacity, it acts as the virtualization layer
between the host and other external storage systems, providing flexibility and extending
functionality to the virtualized external storage capacity. Up to 32 PB of storage can be
managed by a single FlashSystem V9000 array, and because the storage is virtualized,
volumes can be nondisruptively moved between external and internal storage capacity. This
functionality enables agile integration into existing storage environments with seamless data
migration between FlashSystem V9000 and legacy storage systems. When using Real-time
Compression for active data sets, FlashSystem V9000 can increase the effective capacity of
your flash storage up to five times.
Note: The Fibre Channel internal connection switches are ordered together with the first
FlashSystem V9000 scalable building block. IBM also support the use of customer
supplied Fibre Channel switched and cables, if it is supported by IBM. See the latest
supported Fibre Channel switches at the following web page:
http://www.ibm.com/systems/support/storage/ssic/interoperability.wss
FlashSystem V9000
FlashSystem V9000
FlashSystem V9000
FlashSystem V9000

1.7.1 Advanced functions for data reduction
The FlashSystem V9000 employs several features to assist with the reduction of data and the
increase in effective capacity.
IBM Real-time Compression
The IBM Real-time Compression software that is embedded in the FlashSystem V9000
addresses the requirements of primary storage data reduction, including performance with
the use of dedicated compression acceleration hardware. It does so by using a purpose-built
technology that is called Real-time Compression that uses the RACE engine. Customers can
expect data reduction and effective capacity increases of up to 5x for relevant data sets.
Thin provisioning
In a shared storage environment, thin provisioning is a method for optimizing the use of
available storage. It relies on allocation of blocks of data on demand versus the traditional
method of allocating all of the blocks up front. This methodology eliminates almost all white
space, which helps avoid the poor usage rates (often as low as 10%) that occur in the
traditional storage allocation method where large pools of storage capacity are allocated to
individual servers but remain unused (not written to).
Thin-provisioned flash copies
Thin-provisioned IBM FlashCopy® (or Snapshot in GUI) uses disk space only when updates
are made to the source or target data and not for the entire capacity of a volume copy.
1.7.2 Data migration
The IBM FlashSystem V9000 provides online volume migration while applications are
running, which is possibly the greatest single benefit for storage virtualization. This capability
allows data to be migrated on and between the underlying storage subsystems without any
impact to the servers and applications. In fact, this migration is performed without the
knowledge of the servers and applications that it even occurred. The FlashSystem V9000
delivers these functions in a homogeneous way on a scalable and highly available platform
over any attached storage and to any attached server.
1.7.3 Advanced copy services
Advanced copy services are a class of functionality of storage arrays and storage devices that
allow various forms of block-level data duplication. By using advanced copy services, you can
make mirror images of part or all of your data eventually between distant sites. Copy services
functions are implemented within a FlashSystem V9000 (FlashCopy and Image Mode
Migration) or between one FlashSystem V9000 and another FlashSystem V9000 (Metro
Mirror and Global Mirror).
FlashCopy
FlashCopy is the IBM branded name for Point-in-Time copy, which is sometimes called
Time-Zero, or T0 copy. This function makes a copy of the blocks on a source volume and can
duplicate them on 1 - 256 target volumes.

Remote mirroring
Global Mirror and Metro Mirror are implemented at the volume layer within the FlashSystem
V9000. They are collectively referred to as remote copy. In general, the purpose of both
functions is to maintain two copies of data. Often, but not necessarily, the two copies are
separated by distance. The remote copy can be maintained in one of two modes:
synchronous or asynchronous.
Metro Mirror is the IBM branded term for synchronous remote copy function.
Global Mirror is the IBM branded term for the asynchronous remote copy function.
At the time of writing, stretch cluster configurations were not supported for
FlashSystem V9000.
1.7.4 External virtualization
The FlashSystem V9000 includes data virtualization technology to help insulate applications
from physical storage. This enables applications to run without disruption, even when
changes are made to the storage infrastructure. The FlashSystem V9000 functions benefit
all virtualized storage. For example, Easy Tier and Real-time Compression help improve
performance and increase effective capacity while high-performance thin provisioning helps
automate provisioning. These benefits can help extend the useful life of existing storage
assets, reducing costs. And because these functions are integrated into the FlashSystem
V9000, they can operate smoothly together, reducing management effort.
1.7.5 Easy Tier
Easy Tier is a performance function that automatically migrates or moves extents off a volume
to or from one storage tier to another storage tier. Easy Tier supports three kinds of tier
attributes:
Easy Tier monitors the host I/O activity and latency on the extents of all volumes with the
Easy Tier function that is turned on in a multitier storage pool over a 24-hour period.
It creates an extent migration plan that is based on this activity and then dynamically
moves high-activity or hot extents to a higher disk tier within the storage pool.
It also moves extents whose activity dropped off or cooled from the high-tier disk back to a
lower-tiered disk.
1.7.6 Licensing
The base license that is provided with your system includes the use of its basic functions.
However, extra licenses can be purchased to expand the capabilities of your system.
Administrators are responsible for purchasing extra licenses and configuring the systems
within the license agreement, which includes configuring the settings of each licensed
function on the system.
The base 5639-RB7 license entitles FlashSystem V9000 (machine type 9846/9848) to all the
licensed functions such as Virtualization, FlashCopy, Global Mirror and Metro Mirror, and
Real-Time Compression. Any connected storage that is not a FlashSystem V9000 requires
the External Virtualization license that is a per terabyte (TB) capacity unit of metric. You use
the Licensed Functions window in the System Setup wizard to enter External Virtualization
licenses purchased for your system.
For more details about licensing, see 2.5.2, “Software and licensing” on page 65.

© Copyright IBM Corp. 2015. All rights reserved. 17
Chapter 2. FlashSystem V9000 architecture
This chapter describes FlashSystem V9000 architecture, detailing the components,
capabilities and features that make up this product. An introduction to the IBM FlashSystem
V9000, product features, a comparison to the IBM FlashSystem V840, and an overview of the
architecture and hardware are included.
This chapter includes the following topics:
Introduction to IBM FlashSystem V9000
Architecture of IBM FlashSystem V9000
Control enclosure (AC2) of the FlashSystem V9000
Storage enclosure (AE2) of the FlashSystem V9000
Administration and maintenance of FlashSystem V9000
Support matrix for the FlashSystem V9000
Warranty information
For more details about the IBM FlashSystem architecture, see the IBM FlashSystem V9000
web page in the IBM Knowledge Center:
https://ibm.biz/fs_V9000_kc
2

2.1 Introduction to IBM FlashSystem V9000
IBM FlashSystem V9000 is an all-flash storage array that provides extreme performance and
large capacity while also delivering enterprise class reliability and “green” data center power
and cooling requirements. The IBM FlashSystem V9000 building block holds up to twelve
5.7 TB IBM MicroLatency modules in only 6U of rack space, making it an extremely dense
all-flash storage array solution.
FlashSystem V9000 uses a fully featured and scalable all-flash architecture that performs
at up to 2.5M IOPS with IBM MicroLatency, is scalable up to 19.2 GBps, and delivers up
to 2.28 PB effective capacity. Using its flash-optimized design, FlashSystem V9000 can
provide response times of 200 microseconds. This high capacity, extreme performance and
enterprise reliability are powered by the patented IBM FlashCore Technology. Advanced data
services that are provided include copy services, mirroring, replication, external virtualization,
and VAAI support. Host interface support includes 8 Gb and 16 Gb FC, and 10 Gb Fibre
Channel over Ethernet (FCoE) and iSCSI. AES 256 hardware-based encryption adds to the
rich feature set.
IBM FlashSystem V9000 is made up of the two control enclosures, referred to as AC2s, and
one storage enclosure, referred to as AE2.
The IBM FlashSystem V9000 core attributes are described next. Figure 2-1 shows the front
view of the IBM FlashSystem V9000.
Figure 2-1 IBM FlashSystem V9000
2.1.1 Capacity
IBM FlashSystem V9000 supports a maximum of four building blocks and four additional
storage enclosures. Each building block or storage enclosure can accommodate up to twelve
5.7 TB IBM MicroLatency modules, which provide a capacity of 57 TB (RAID 5). The
FlashSystem V9000 therefore supports a maximum physical capacity of 456 TB. Using the
optional Real-time Compression and other design elements, the FlashSystem V9000
provides up to 57 TB usable capacity and up to 285 TB effective capacity in only 6U.
This scales to 456 TB usable capacity and up to 2.28 PB effective capacity in only 36U
of rack space.

Chapter 2. FlashSystem V9000 architecture 19
Each IBM FlashSystem V9000 building block can be ordered with 4, 6, 8, 10, or 12
MicroLatency modules. The MicroLatency modules available are either 1.2 TB, 2.9 TB, or
5.7 TB storage capacity.
IBM FlashSystem V9000 supports RAID 5 configurations.
IBM FlashSystem V9000 supports the creation of up to 2,048 logical unit numbers (LUNs) per
building block. The size of the LUNs can be 1 MiB - 51.8 TiB in size (not to exceed the total
system capacity). The IBM FlashSystem V9000 supports up to 2,084 host connections and
up to 256 host connections for each interface port. The IBM FlashSystem V9000 allows the
mapping of multiple LUNs to each host for Fibre Channel, Fibre Channel over Ethernet
(FCoE), and iSCSI protocols.
IBM FlashSystem V9000 supports up to 256 host connections for the iSCSI protocol.
Table 2-1 lists all the combinations of storage capacities for various configurations of the IBM
FlashSystem V9000 building block.
Table 2-1 IBM FlashSystem V9000 capacity in TB and TiB for RAID 5
Important: 1.2 TB, 2.9 TB, and 5.7 TB IBM MicroLatency modules cannot be intermixed in
the same IBM FlashSystem V90000 storage enclosure.
Note: The maximum usable capacity of IBM FlashSystem V9000 in RAID 5 mode is
51.8 TiB per building block.
IBM FlashSystem 900 configuration RAID 5 TB RAID 5 TiB
Four 1.2 TB flash modules 2.4 2.1
Six 1.2 TB flash modules 4.8 4.3
Eight 1.2 TB flash modules 7.2 6.5
Ten 1.2 TB flash modules 9.6 8.3
Twelve 1.2 TB flash modules 12 10.9

2.1.2 Performance and latency
IBM FlashSystem V9000 uses all hardware field-programmable gateway array (FPGA)
components in the AE2 storage enclosure data path, which enables fast I/O rates and low
latency. IBM FlashSystem V9000 provides extreme performance of up to 2.5 M IOPS and up
to 19.2 GBps in bandwidth. The IBM FlashSystem V9000 provides response times as low as
200 μs.
2.1.3 IBM FlashCore technology
IBM FlashSystem V9000 provides enterprise class reliability and serviceability that are
unique for all-flash storage arrays. FlashSystem V9000 uses the patented IBM FlashCore
Technology to provide data protection and maximum system uptime:
IBM Advanced Flash Management improves flash endurance 9x over standard
implementations.
– Proprietary garbage collection, relocation, and block-picking algorithms that were
invented by IBM.
– Flash wear leveling includes the following functions:
• ECC algorithms that correct very high bit error rates.
• Variable voltage and read-level shifting to maximize flash endurance.
• Health binning and heat segregation continually monitor the health of flash blocks
and perform symmetrical wear leveling and sub-chip tiering.
• Hot-data placement provides up to 57% improvement in endurance. Heat-level
grouping provides up to 45% reduction in write amplification.
Variable Stripe RAID is a patented IBM technology that provides an intra-module RAID
stripe on each flash module.
With two-dimensional (2D) Flash RAID, system-wide RAID 5 along with Variable Stripe
RAID helps reduce downtime and maintain performance and allows the provisioning of an
entire flash module as a spare to be used in another flash module failure.
Terminology
The following terms are mentioned in this book:
Wear leveling An algorithm that assures even usage of all blocks.
Garbage collection Erasing blocks, which are not used anymore, so that they can be
rewritten.
Relocation Moving a block to another location.
Block picking The first step of the garbage collection process. Using proprietary
algorithms, the best block is picked for garbage collection.

Introducing and Implementing IBM FlashSystem V9000

Introducing and Implementing IBM FlashSystem V9000

Recomendados

Recomendados

Más contenido relacionado

La actualidad más candente

La actualidad más candente (20)

Similar a Introducing and Implementing IBM FlashSystem V9000

Similar a Introducing and Implementing IBM FlashSystem V9000 (20)

Más de Michael Martin

Más de Michael Martin (6)

Introducing and Implementing IBM FlashSystem V9000