In politics, one often hears: “Something must be done.” Invariably, the “something” refers to a significant, known set of problems and a paucity of answers, and it is simply apparent that things cannot go on as they are. In the world of SMBs and enterprise storage, the first part of the statement holds true. There are indeed abundant known challenges—data growth, consumerization, clouds, big data, and so on. And often, they come with even bigger expectations. The silver lining to the dark clouds threatening the storage ecosystem is that plenty of tools and methods either exist or are well understood and can control or remove much of the problem … if both vendors and users are willing to admit the issues and embrace change before it is too late.

1. © 2012, The Enterprise Strategy Group, Inc. All Rights Reserved.
Storage Challenges Are Mounting
“Faced with the choice of changing one’s mind
and proving that there is no need to do so,
almost everybody gets busy on the proof.” —J.K. Galbraith
The challenges that managers of organizational data and storage face today are daunting. First, there is the constantly
increasing need for additional resources to support growing data volumes stemming from natural application growth
and new workloads. (Think of the impact of the web, social media, mobility, and big data.) In many cases, organizations
try to handle it by simply throwing more hardware at the problem, often ending up with massively underutilized assets.
Next, operational processes often have not caught up with technology innovations, so as IT service delivery becomes
more agile (and cloud enters the fray, for instance), administrators struggle to manage with the same old, inflexible
processes.
Budget constraints are almost always an issue, typically even more so in recent years. In addition, as virtual servers and
cloud computing are improving provisioning and providing a higher level of service, users are beginning to expect
“instant IT.” In the old days (five years ago!), if a user wanted to launch a new application to support a business process,
he or she had an expectation that it would take a while—weeks or months—to get through the normal channels. But
with fast, easy provisioning made possible by virtualization, IT can spin up a new virtual machine in minutes.
However, having the correct, appropriate, available, and affordable storage infrastructure behind that VM and
application can be a whole different story. As with any problem in life, the first two stages typically require
acknowledging the problem and establishing the desire to address it. Storage needs “something to be done” before an
unsustainable model—the one largely deployed today—begins to cause real damage to IT and the business.
Key Challenges
• Data Growth—Data growth is probably the only absolute constant in IT, and arguably, it is the inevitable
cause of most operating problems. Data growth never abates; it occurs whether the economy is up or down.
One can have a delightful academic debate about whether the demand for capacity will always exceed and
drive the technologies (in terms of both capacity and performance), or whether a declining raw price drives
latent demands that become economically viable at the new cost level. Frankly, it does not matter. When
you’re at the top of a roller coaster, regardless of whether you were pulled or pushed up the incline, massive
acceleration and g-forces are coming. The whole ecosystem is on the ride. We now live in a zettabyte world,
and it strains systems, people, opex, and capex.
• New IT Architectures—Virtualization drives and stresses storage capacity and performance needs equally—
and a poor storage infrastructure can be an anchor on the success of server virtualization and VDI projects.
At the same time, the use of cloud models is increasing. (ESG research pins cloud expenditures in 2012 at 7%
Storage Systems Brief
Challenges and Opportunities for the Storage Industry
Date: May 2012 Author: Mark Peters, Senior Analyst
Abstract: In politics, one often hears: “Something must be done.” Invariably, the “something” refers to a significant,
known set of problems and a paucity of answers, and it is simply apparent that things cannot go on as they are. In the
world of SMBs and enterprise storage, the first part of the statement holds true. There are indeed abundant known
challenges—data growth, consumerization, clouds, big data, and so on. And often, they come with even bigger
expectations. The silver lining to the dark clouds threatening the storage ecosystem is that plenty of tools and
methods either exist or are well understood and can control or remove much of the problem … if both vendors and
users are willing to admit the issues and embrace change before it is too late.

2. Storage Systems Brief: Challenges and Opportunities for the Storage Industry 2
© 2012, The Enterprise Strategy Group, Inc. All Rights Reserved.
of all IT spending.) While the attraction of both virtualization and cloud is beguiling, users all too often find
that storage and its overall management can cause a type of “Ponzi” complexity behind the scenes. Very
often, specific storage exists for a specific application, and huge operational issues arise in making data
available to others within or outside an organization. (Imagine if you had to change laptops to go from Word
to PowerPoint!) And yet, that is how storage is often deployed.
• Uncertainty—With much of the world’s new data produced outside of data centers—unstructured and rich
media being the new norm—the ability to predict what storage is needed within those data centers is crucial
but increasingly difficult. On the other hand, it is certain that users have high expectations in terms of speed,
availability, and immediacy … irrespective of geography and time zones.
• Operational Struggles—The lack of flexibility in, and between, many storage systems simply makes
completing the jigsaw puzzle harder. Everything must still be protected and backed up … whether it’s a
scale-up or scale-out environment, distributed or converged. Many users have resorted (perhaps a better
choice of words would be “are resigned”) to overprovisioning and underutilization just to get the job done,
even knowing that such an approach equates to throwing money down the drain.
The Crux of the Problem
Disk technology is 56 years old this year, and tape is 60. The underlying essence of what we’re using for storage hasn’t
changed. There’s an old joke about giving someone directions that ends with, “But if I were you, I wouldn’t start from
here!” But is that so with storage? Are we really just rearranging the deckchairs on the Titanic, or is there room for
optimism? The truth is that storage is complex and tough. And getting the job done effectively has for years often
trumped getting it done efficiently.
But two things have changed lately: The economic downturn has put a laser focus on IT efficiency in general and storage
in particular, and the lockstep between new storage technologies and new demands for capacity and performance has
been well and truly broken. Without change and innovation, storage costs will escalate to an unacceptable percentage
of IT budgets, probably without providing acceptable service levels either. As the saying goes, “Something has to
change.”
The good news is that there are plenty of reasons to be optimistic. Before examining them, here’s a reminder of why we
are where we are.
How We Got Here, and Where We Will Go Next
How We Got Here
Commercial computing took hold when one infrastructure stack executed one specific application for one specific
purpose. The original mainframe was a glorified calculator. Centralized computing was predictable and controllable,
albeit expensive. It was manageable: one processor system and one I/O subsystem.
Decentralized (or distributed) computing was developed largely to try to solve the economic (essentially capex)
challenges of centralized computing. It yielded low-cost, commodity servers—which we promptly plugged into
proprietary, large, expensive, monolithic storage boxes. Servers became cheaper and more interoperable, while storage
remained proprietary and expensive. In the old days, the server was the thing that cost all the money. You picked your
server by your OS. You picked your OS by your application. Storage was a “peripheral.”
Today, servers are cheap and interoperable, while storage is still expensive, complex, incompatible, and difficult. In
many respects, it is the last bastion of IT awkwardness: the peripheral tail wagging the purposeful dog!
Where to Next?
Let’s take for granted that we want to virtualize IT in general because we can gain efficiencies in asset utilization, take
advantage of the commoditization of hardware, leverage common infrastructures, provide seamless mobility options,
etc.

3. Storage Systems Brief: Challenges and Opportunities for the Storage Industry 3
© 2012, The Enterprise Strategy Group, Inc. All Rights Reserved.
If we do this, then we need to provide an infrastructure that:
• Self-tunes: Storage that manages/optimizes/reconfigures itself for the workloads that are presented
• Self-heals: Infrastructure that handles faults autonomously, so that applications are not affected
• Scales dynamically: Devices that can extend—virtually—to whatever requirements the workload(s) presents
• Self-manages: Adapts to changing scenarios based on policy and enforces those policies via automation
This set of values is nirvana when it comes to IT and storage infrastructures today, but it is exactly what we are moving
toward. In so doing, we will address the levels of manual intervention, inflexibility, and wasted resources that bedevil
today’s storage world. If we do not adopt these changes, then the storage scenario is simply unsustainable.
In layman’s terms, what is needed is storage that has more automation, flexibility, application and business linkages,
resource utilization, and management /tuning ease.
There is, however, one thing we consistently need storage to have less of. Cost!
The Importance of Economics
Nothing in storage makes sense if it doesn’t make economic sense. We have storage tiers because storage isn’t free. And
using them efficiently has been an aim (that we’re finally beginning to deliver against) for decades. Different types of
storage exist only because of cost differentials; after all, if storage were free (or all the same price, at least), we would
naturally put everything on the fastest devices possible. Only because storage has a price—indeed, a range of prices—do
we think about where to put different data. We often disguise that as a conversation about performance, but that’s a
symptom rather than a cause. The cause is money. Figure 1 shows very simply, yet starkly, how we arrived where we
are. It also shows why a need exists for more complete tiering at all levels of the data hierarchy.
Figure 1. The Evolution of the Storage Hierarchy
Source: Enterprise Strategy Group, 2012.

4. Storage Systems Brief: Challenges and Opportunities for the Storage Industry 4
© 2012, The Enterprise Strategy Group, Inc. All Rights Reserved.
The first picture displayed in Figure 1 portrays the “perfect story” spun for decades; however, it has never properly
existed and would actually be better represented by the wedding-cake diagram shown in the second picture, in which
the “steps” from one storage tier to another are large. In the real world, such moves (such as that from disk to tape)
were for decades also awkward, largely manual, and hence, to be avoided as much as possible. As shown in Picture 3,
the moves were pretty much happening in one direction, creating what could almost be called a storage “lower-archy!”
Over the years, we have added more gradations for active data (Picture 4) in terms of differing types of disk drive
(speed, capacity, caching, etc.), but the top and bottom layers of the data “cake”—solid-state disk and tape—have been
essentially left alone. What the industry needs is to extend additional layers (see Picture 5) in these places, too.
What you end up with is a gradual smoothing of the layer cake from top to bottom so that it begins to look (as Picture 6
shows) rather like the original vision we had decades ago. In some instances, it will be abundantly clear (for applications
known to have extreme I/O performance or latency-sensitive needs) that certain data should reside permanently on an
ultra-performance tier. Another option is to employ some form of tiering software that moves the most deserving or
demanding data at any given time to the highest available storage tier. Both options have their place. The former is very
specific and absolutely guarantees the best service for certain data. The latter is a more generic approach that generates
less performance improvement across a wider amount of data. The final element is shown in Picture 7, in which
technologies such as deduplication, thin provisioning, and solid-state/caching shrink the pyramid for a given workload.
Now that we’ve seen where we have come from in storage, and what’s needed to avoid going over the precipice of
inefficiency and financial disaster, should we throw our hands up in defeat? No. And users don’t have to wait for
artificial-DNA-based storage or the next storage-class-memory technology to appear. Future technologies hold much
promise, but the storage challenge is real today, and a number of technologies are available or emerging that hold the
promise to alleviate the issues.
Areas of Immediate Opportunity
While nothing alone is a panacea, there is reason for optimism among users who will embrace some change … and
among vendors that have the wherewithal to develop and improve the technologies that can make storage a useful—
hopefully almost an invisible—tool, rather than a pain-in-the-IT-neck. Some of the main opportunities are:
1. Embracing an automated storage hierarchy: While any mention of terms such as “information lifecycle
management” must be made in hushed tones (because its marketing preceded a real delivery capability years
ago), the value of using tiers and caching for appropriate data placement is irrefutable—both in terms of logic
and finances.
2. Storing less data: This means not only a backend reduction (although this is still far from endemic), but also
using newer technology on the front end of storage (in-line compression and deduplication, for instance) to
prevent unnecessary blocks of data ever being written anywhere in a storage system.
3. Store the data you have more efficiently: This step includes using straightforward tools such as thin
provisioning (which can apply to prime and replicated data), that are still deployed by a surprisingly small group
of users (around 50%). Unified storage platforms can also reduce the overall capacities needed by establishing
pools of storage for better utilization and more flexibility. Additionally, the use of sophisticated management
tools allows for efficient deletion of unnecessary data.
4. Storage virtualization: This step can occur within devices (enabling thin provisioning, snapshots, etc.) and across
(heterogeneous) platforms. The latter approach can be termed a “storage hypervisor,” and it is another method
enabling users to deploy less actual storage to store the same amount of data—mainly via better utilization.
5. Automation and analytics: Having systems that are “application aware” and can implement policies is another
emerging route to better storage efficiency. It is entirely possible that the “storage administrator” function as
we know it will not exist in five to ten years. Storage will be given specific requirements by the overall system
administrator, and it will then—automatically, autonomically—set performance, protection, tiering, and
recovery policies. Even “set and forget” will be surpassed by “get and forget.”

5. Storage Systems Brief: Challenges and Opportunities for the Storage Industry 5
© 2012, The Enterprise Strategy Group, Inc. All Rights Reserved.
The Bigger Truth
The storage world is dealing with a “burning platform.” The fuel for the fire is the unrelenting growth in demand for
data, and the flames are fanned by the complexity and inefficiency that is endemic in many storage infrastructures
deployed today.
There has been massive consolidation on the vendor side (with well over $10 billion spent on acquisitions in the last two
to three years) as big vendors seek to have the right tools in their bag … which only goes to show that many of the
necessary tools are available already to at least ameliorate—and possibly extinguish—the flames. The situation just
needs users to be more aggressive in using what’s available, and vendors to invest ever-more in the orchestration and
automated management of advanced storage functionality … so that storage can become a valuable, unseen foundation
rather than an expensive frustration.
If neither of those changes happen, then we are headed for a catastrophe. If only one or the other happens, then we will
have at least bought our industry some time. If both happen, then we should be able to return to a balanced storage
system.
While the list of requirements for the “good outcome” to be achieved seems onerous (make storage less expensive;
easier; more self-managing; more scalable; and more sharable within and across applications, people, and
organizations), it is all eminently achievable. The value for users, and therefore the opportunity for vendors, is so great
that it will—hopefully—ensure these advances occur. But the time for both users and vendors to act is now.
All trademark names are property of their respective companies. Information contained in this publication has been obtained by sources The Enterprise Strategy
Group (ESG) considers to be reliable but is not warranted by ESG. This publication may contain opinions of ESG, which are subject to change from time to time. This
publication is copyrighted by The Enterprise Strategy Group, Inc. Any reproduction or redistribution of this publication, in whole or in part, whether in hard-copy
format, electronically, or otherwise to persons not authorized to receive it, without the express consent of The Enterprise Strategy Group, Inc., is in violation of U.S.
copyright law and will be subject to an action for civil damages and, if applicable, criminal prosecution. Should you have any questions, please contact ESG Client
Relations at 508.482.0188.