To date, the implementation of enterprise storage systems has evolved around traditional storage array architectures. There are many situations where having the option to scale the same enterprise storage system out, up or both is a better way forward than continuing to rely on the traditional scale up model. Here we compare the approaches, pointing out the significant operational and economic advantages of the new scale out paradigms.

1. Technology Insight Paper
Defining the Value of a Modular,
Scale out Storage Architecture
By John Webster
January 2014
Enabling you to make the best technology decisions

2. Defining the Value of a Modular, Scale Out Storage Architecture
1
© 2014 Evaluator Group, Inc. All rights reserved. Reproduction of this publication in any form
without prior written permission is prohibited.

3. Defining the Value of a Modular, Scale Out Storage Architecture
2
© 2014 Evaluator Group, Inc. All rights reserved. Reproduction of this publication in any form
without prior written permission is prohibited.
Executive Summary
To date, the implementation of enterprise storage systems has evolved around traditional storage array
architectures. Typically, these are single or dual controller systems with cache and disk in RAID
configurations. Enterprise IT administrators scale storage capacity upward by simply adding more disks
to the existing array controller or controller pair—a technique now referred to as “scale up.”
Clustering technology applied to storage system architectures now offers the “scale out” alternative:
scale a single storage system in both capacity and performance by simultaneously adding or replacing
modular controllers and disks. In fact, one can actually combine the ability to scale out—adding nodes to
the cluster—with the option to scale up individual nodes, a technique we call “bi‐directional” scaling.
There are many situations where having the option to scale the same enterprise storage system out, up
or both is a better way forward than continuing to rely on the traditional scale up model. Here we
compare the approaches, pointing out the significant operational and economic advantages of the new
scale out paradigms. These include:
 Lower TCO resulting from an increased use of existing capacity and a reduction in management
overhead. The ability to manage storage as a single entity vs. a collection of disparate arrays that
must be managed separately, often using different management interfaces and operational
practices
 Having the flexibly to deploy storage resources in varying ways to match the variability of today’s
business workloads
 The ability to non‐disruptively add capacity and performance in a modular fashion that also
allows IT administrators to significantly extend the useful life of a storage system (7‐10 years) vs.
that of the traditional (usually three years) there by increasing ROI
Defining the Storage Scaling Alternatives
Scale up storage systems are the most commonly used type of storage implementation and well
understood by enterprise IT administrators. Enterprise IT traditionally deploys either a new scale up
storage system or adds capacity to an existing scale up array as a way to accommodate stored data
growth as seen in Figure 1. below:

4. Defining the Value of a Modular, Scale Out Storage Architecture
3
© 2014 Evaluator Group, Inc. All rights reserved. Reproduction of this publication in any form
without prior written permission is prohibited.
Figure 1: Adding Disk Capacity to the Scale up System
However, relying only on scale up storage system architectures to accommodate growth creates a
number of downstream issues:
Operational complexity—Adding more scale up storage systems to the data center environment
in order to keep up with application data growth forces IT to manage and protect a growing
number of often disparate systems when the maximum performance or capacity is reached.
Performance bottlenecks—As capacity increases behind scale up controllers, IT is challenged to
maintain consistent application service levels.
System downtime—Scale up architectures can force operations staff to take a storage system
out of service for updating or configuration changes. This practice is no longer acceptable for
storage systems that support critical business applications.
Scaling Out
An increasingly advantageous alternative is the use of scale out systems for SAN attached block storage
as well as network attached file storage (NAS). These have the flexibility to simultaneously scale in
performance and capacity by adding more by adding more storage nodes as shown in Figure 2 below:

5. Defining the Value of a Modular, Scale Out Storage Architecture
4
© 2014 Evaluator Group, Inc. All rights reserved. Reproduction of this publication in any form
without prior written permission is prohibited.
Figure 2: Scale out storage system based on the clustering of storage nodes
The modular units containing both the storage controllers and storage devices (SSD and/or rotating disk)
are typically called “nodes”
Scale out storage is architected using clustered nodes connected over a high performance network and
storage devices that are either SAN attached or captive to the nodes. Implementations vary across
vendors. One significant difference relates to whether data is distributed across nodes or if each node is
responsible for its own data. Data distribution and protection across nodes can facilitate non‐disruptive
technology upgrades and replacement of nodes as needed. Another that we examine here in more
detail is an ability to scale out by adding more nodes to the cluster combined with the flexibility of
scaling up individual nodes when needed to target specific application workloads.
Bi-Dimensional Scaling—Scaling Up and Out
Because not all applications exhibit the same I/O patterns and not all workloads stress storage systems
the same way, having a scale out system that supports nodes with varying configurations allows IT
administrators to respond to these differences in a more targeted and efficient way. (See Figure 3
below)

6. Defining the Value of a Modular, Scale Out Storage Architecture
5
© 2014 Evaluator Group, Inc. All rights reserved. Reproduction of this publication in any form
without prior written permission is prohibited.
Figure 3. Bi‐dimensional scaling—Up and Out
Having the ability to scale storage systems out and up at the same time allows IT administrators to
maximize primary storage efficiency and ROI by targeting resources to the application environment. It
also gives IT administrators the ability to quickly respond workload variability—critical in virtualized
server and cloud services environments.
An important attribute for bi‐dimensional scaling is an ability to tailor the configuration of nodes
according to expected operating conditions and workloads. IT administrators should be able to add solid
state storage devices (SSD) for performance and/or high capacity disk on a per node basis for example.
Storage resources that could vary on a per‐node basis should include:
 SSD
 High performance disk
 Large capacity disk
 Connectivity options (number and type including FC, GbE, iSCSI, FCoE and Optical SAS)
Having the flexibly to deploy these resources in varying ways becomes important when one considers
the variability of today’s business workloads. As business activity increases for example, there is a
corresponding increase in the data created that must be stored and managed. There is also a mixture of

7. Defining the Value of a Modular, Scale Out Storage Architecture
6
© 2014 Evaluator Group, Inc. All rights reserved. Reproduction of this publication in any form
without prior written permission is prohibited.
highly active data requiring consistent service levels and greater performance vs. much less active data
that over time consumes larger amounts of capacity.
The same imbalance can be seen with Decision Support applications. As business activity increases, the
need for storage performance increases linearly while the sizes of the target data sets increase non‐
linearly. An additional difficulty here is determining the access requirements for the different types of
data (structured, unstructured, etc.). Workloads that require collaboration is another area where an
imbalance can occur. More data is put into the collaborative pool, but the amount of data accessed is
more localized and grows at different rate. Bi‐directional scaling addresses the imbalance.
Operational Value
Management Efficiency
A large, scale out system can be managed as a single entity vs. a collection of disparate arrays that must
be managed separately and often using different management interfaces and operational practices.
Greater management efficiency can be gained even if SRM software is used to aggregate the
management of individual arrays. The scale out management interface can also be used to administer
individual nodes—particularly desirable when the storage system supports bi‐dimensional scaling. The
efficiency of managing a single scale out system, particularly one that can encompass heterogeneous
arrays as compared to many individual heterogeneous systems with an equivalent combined useable
capacity, results in reduced administrative costs and lowers operational impact when addressing new
capacity demands. These are quantified in another Evaluator Group Technical Insight paper entitled
“The Changing Face of Storage Acquisition.”
Capacity and Performance Utilization
When compared to traditional scale up arrays, scale out systems offer the storage administrator an
ability to greatly increase existing capacity utilization. Because traditional arrays are often run at 40‐50%
of available capacity for performance reasons, the majority of an array’s available capacity remains
unused and becomes trapped behind the controller. In a scale out system, multiple controllers can tap
unused capacity so storage is never trapped behind a single controller. And because performance scales
upward with the addition of nodes, utilization rates can be significantly increased vs. traditional arrays.
The realizable percentage increase can easily be 30%, yielding capacity growth without all of the costs
(hardware, software, operational) connected to adding a new array or replacing an obsolete one. As an
additional benefit, capacity planning becomes easier because a scale out system provides greater
visibility and flexibility for managing growth.
When the storage system offers bi‐dimensional scaling, capacity and performance characteristics can be
targeted for even greater efficiency. For applications requiring higher performance, some nodes could

8. Defining the Value of a Modular, Scale Out Storage Architecture
7
© 2014 Evaluator Group, Inc. All rights reserved. Reproduction of this publication in any form
without prior written permission is prohibited.
be configured with solid state disk modules (SSD, typically Flash). But because not all of the workloads
require premium performance, not all nodes need to be configured with SSD. Indeed, some can be
configured for greater capacity by emphasizing higher capacity, lower performance SAS disk.
Performance can be balanced within the same cluster, eliminating the need to acquire some arrays for
performance and others for capacity.
Economic Value
The ability to non‐disruptively add capacity and performance in a modular fashion to scale out storage
clusters allows IT administrators to significantly extend the useful life of storage systems (7‐10 years) vs.
that of traditional arrays which are usually three years. This has two major economic benefits:
1. Fewer system replacements: Capacity refreshes involving the replacement of an entire storage
system become less numerous over time. In fact, the useful life of a scale out system can be
twice that of a traditional array. This cuts the costs associated with technology refresh cycles in
half while more closely aligning refresh cycles with amortization schedules. The ability to non‐
disruptively add/replace modules as new technology emerges and enters mainstream usage (SSD,
higher performance I/O ports, etc.) also helps to ensure long‐term quality of service (QoS).
2. Possible elimination of data migration costs: The significant costs associated with migrating data
from an obsolete system to a new system are greatly reduced over time and possibly eliminated
when data can be moved non‐disruptively from an old to new system. These include system
downtime and a corresponding loss in productivity, as well as an extension of lease terms at
premium rates when the replaced system is leased and not returned on schedule.
With bi‐dimensional scaling, potentially greater savings result from the ability of to buy storage in a finer
granularity for performance and capacity; buy what is needed when it is needed. Additional nodes can
be acquired when more capacity and performance are needed. Add more storage devices to individual
nodes when just more capacity is needed.
To effectively demonstrate the economic value gained from scale out systems, the use of Total Cost of
Ownership or TCO calculations are recommended. For that reason, Evaluator Group has written an
additional Technical Insight paper that presents a model for making these calculations as well as
discussing the operational simplification that can result from pursuing scale out storage systems.
Value in the Cloud
Bi‐dimensional storage system scaling can be of particular value to cloud services providers as well as
enterprise IT departments to the extent that they are adopting private and hybrid cloud deployment
models. Aspects of cloud computing that map well to bi‐dimensional scaling include:

9. Defining the Value of a Modular, Scale Out Storage Architecture
8
© 2014 Evaluator Group, Inc. All rights reserved. Reproduction of this publication in any form
without prior written permission is prohibited.
Multi-tenancy
Having node configurations that vary in performance and capacity gives a cloud service provider the
ability to match the requirements of a customer’s application set to the right storage performance and
capacity characteristics. This results in better control over quality of service (QoS) levels and a charge
back function that more accurately reflects actual resource usage.
Workload variability and QoS
In a cloud computing environment, it is common to see significant workload variability. This can be
observed when comparing different customers that are supported on the same storage system in a
multi‐tenant environment as well. But it is also true that specific customer workloads can vary over time.
After all cloud computing is supposed to offer an “as needed” model that accounts for both increases
and decreases in resource consumption. Bi‐dimensional scaling makes it easier for administrators to
respond to the multiple aspects of workload variability while maintaining QoS and efficient resource
utilization by controlling the performance allocated to a specific user and/or workload.
Storage Connectivity
Cloud services administrators typically want to take advantage of new developments in storage
connectivity as they become available. For example, an ability to upgrade from 8Gb to 16Gb FC with
minimum operational disruption is desirable. Support for Optical SAS allows them to use pre‐existing
optical infrastructure without the need for SAS to Fibre Channel bridges.
NetApp FAS8000 and Clustered Data ONTAP 8.2.1
New NetApp FAS8000 series models 8020, 8040, and 8060 have been announced that deliver a
clustered, modular hardware architecture that is designed specifically for Clustered Data ONTAP and
offers bi‐directional scaling. Over time the FAS8000 system’s architectural modularity will allow NetApp
to reduce the number of models in the FAS system portfolio. Customers will see fewer model variations
but will have the ability to scale the models up and out. The net effect will be that NetApp customers
can expect longer system life cycles from the FAS8000 series as well as future generations while having
the ability to upgrade technology on an as needed basis without disruption. In addition, administrators
can delay or even avoid the typical cadence of three‐year technology refresh cycles.
With version 8.2.1 of Data ONTAP, organizations now have the ability to add or replace storage shelves
without disruption to business operations. This new release also improves data security and
management of Microsoft CIFS environments, and automates flash pool sizing.

10. Defining the Value of a Modular, Scale Out Storage Architecture
9
© 2014 Evaluator Group, Inc. All rights reserved. Reproduction of this publication in any form
without prior written permission is prohibited.
Increased Performance
The FAS8000 system architecture offers up to twice the processing power of the previous generation;
delivering predictable performance across a wider range of SAN and NAS workloads and operating
demands. Enhancements to NetApp’s flash‐based storage options, including an up to three times
increase in the amount of flash as cache, further increasing performance. Active data is automatically
promoted to flash. Internal bandwidth consumption is also monitored and managed automatically and
in real time. FAS8000 performance enhancements may also allow customers to scale performance and
capacity using even fewer nodes than before vs. other scale out storage systems currently available.
I/O Connectivity
The FAS8000 Series offers a broad range of embedded connectivity options including the new Universal
Target Adapter (UTA2) ports that can be configured as either 10GbE or 16Gb FC. This gives
administrators the flexibility to handle a variety of protocols including FC, FCoE, iSCSI and change
between them over time as business needs change. Administrators can connect the same FAS8000
system to servers and cloud service providers using a mix of protocols and change over to new protocols
without juggling adapter cards. They also ease the transition for those moving from FC to Ethernet.
External Storage Virtualization
To help consolidate storage operations under a single management framework, all FAS8000 systems
have the unique ability to virtualize and incorporate external third party block and file‐based arrays
within the system image. External storage virtualization is an ONTAP 8.2.1 software‐based option that
can be purchased and activated at any time. Supported external arrays include NetApp E‐Series, EMC
VNX and VMAX, and HDS AMS, HUS, and VSP. (A complete listing of supported third party arrays can be
found on the NetApp website and changes as new support is added.)
Hybrid Cloud Integration
The FAS8000 series can be integrated with the storage infrastructures of cloud‐based service providers
specializing in backup and disaster recovery, as well as hyperscalers for cloud bursting. For cloud
bursting administrators can replicate data and then create zero capacity clones of VMs for development
and testing efforts.
Additional Capabilities
 Encryption‐based storage security for data at rest
 Support for 10K and 7.2K RPM disk
 Support for various SSD drive capacities
 Optical SAS connectivity
Conclusion

11. Defining the Value of a Modular, Scale Out Storage Architecture
10
© 2014 Evaluator Group, Inc. All rights reserved. Reproduction of this publication in any form
without prior written permission is prohibited.
Advances made in developing and delivering scale out storage systems—including those that offer bi‐
dimensional scalability—now allow IT administrators address the growth of business information
storage while consolidating and perhaps even reducing the number of scale up systems that have
traditionally been incrementally acquired as needed. The benefits realized from deploying this new
generation of scale out systems can therefore be economically quantified. These include
 Cost reductions resulting from increased capacity utilization
 Enhanced application user productivity from consistent QoS levels
 Fewer disruptive configuration changes
 The ability to purchase additional capacity as needed
 Consolidated administration
 Reduction in disruptive operational impacts.
And from an application user perspective, the ability to modularly scale performance and capacity bi‐
directionally means that IT administrators can deliver expected service levels while simultaneously
responding to storage capacity demand.
Technology refresh is an ongoing activity for IT professionals, but the impact of replacing systems,
migrating data, and changing operational procedures has made it a burden. Moving to a scale out
system that supports technology updates bi‐directionally, modularly and transparently relieves this
pressure and allows IT administrators to bring the best technology into usage when it is needed.
About Evaluator Group
Evaluator Group Inc. is dedicated to helping IT professionals and vendors create and implement strategies that make the most of the value
of their storage and digital information. Evaluator Group services deliver in‐depth, unbiased analysis on storage architectures,
infrastructures and management for IT professionals. Since 1997 Evaluator Group has provided services for thousands of end users and
vendor professionals through product and market evaluations, competitive analysis and education. www.evaluatorgroup.com Follow us
on Twitter @evaluator_group
Copyright 2014 Evaluator Group, Inc. All rights reserved.
No part of this publication may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying and recording,
or stored in a database or retrieval system for any purpose without the express written consent of Evaluator Group Inc. The information contained in this
document is subject to change without notice. Evaluator Group assumes no responsibility for errors or omissions. Evaluator Group makes no expressed or
implied warranties in this document relating to the use or operation of the products described herein. In no event shall Evaluator Group be liable for any
indirect, special, inconsequential or incidental damages arising out of or associated with any aspect of this publication, even if advised of the possibility of
such damages. The Evaluator Series is a trademark of Evaluator Group, Inc. All other trademarks are the property of their respective companies.