Major Disruptions in Data Storage Technology: What This Shake-Up Means for the Enterprise

In this special guest feature, Dave Donald, Founder and CEO of Keeper Technology, looks at some of the biggest disruptors we will see, or continue to see, shaking up the storage industry in 2019 including: SDS, NVMe, hyper-converged architectures,  edge computing, and portable storage architectures. Dave is responsible for defining the company’s strategic vision and leading all go-to-market strategy and execution. He brings more than 25 years of experience in answering enterprise storage’s most critical data challenges and building high performance teams at innovative software and storage companies. Dave holds a Bachelor of Science Degree in Electrical and Computer Engineering from the University of Colorado.

Good technology is evolutionary. Great technology is revolutionary. Of the two only the second is disruptive. One of the single greatest factors driving disruption in IT infrastructure is the growth of data and data storage technologies, fueled by initiatives in big data, Internet of Things (IoT) and data analytics.

Today’s enterprises are generating and storing more data than ever before. This trend shows no sign of slowing down. This uptick is pushing enterprises to expand their infrastructures, especially around data storage. Market pressures on storage providers are escalating and it looks like the entire ecosystem is in for a shake-up.

Enterprises are beginning to understand that they need to harness the power of data to remain competitive. To support their digital strategy, companies are planning to invest in more storage hardware in their data centers, store more data in the cloud and continue to investigate emerging, disruptive technologies.

Prior to these events, disruption involved the early adoption of Software Defined Storage (SDS) services. The newer players in SDS are challenging established brands. The dominant companies are beginning to realize that their existing solutions met a very loose definition of SDS, despite their aggressive attempts to market them as something more than they were.

In this respect, definitions matter a lot. For example, a more traditional definition of SDS includes commodity hardware in the form of network-attached servers. This is significantly different from the proprietary architectures of the so-called “Big Iron” providers.

Context is key to understanding disruption, because there has been more change in computing and applications. The reasons are simple: Storage systems must meet higher standards of safety, compliance and consistency. On the other hand, more ephemeral commodities—like processing and memory—have enhanced the importance of storage.

Considering these factors, I believe that some of the biggest disruptors we will see, or continue to see, shaking up the storage industry in 2019 are:

#1 – Software Defined Storage (SDS)

SDS will continue to play a large role given its ability to maximize performance and capacity with zero downtime. SDS also matches enterprise strategies involving cloud storage.

Efficiency is a quick sign of disruption. Better security is not as fast, because it is complicated and often has a negative impact on the end-user experience. None of which diminishes the progress SDS has accomplished in this area.

Companies can apply a scale-up strategy in which they buy small numbers of new and more powerful servers, or they can choose a scale-out approach. The latter uses large numbers of less powerful machines, which allows for clustering and redundancy. It is less expensive, too. Clustering built on standard commodity hardware replicates data across very long distances. If, for example, the primary data center fails, services automatically failover to a working data center.

#2 – Non-Volatile Memory Express-based storage (NVMe)

If we look at NVMe, there is enormous buzz about its sheer potential for performance. The excitement is justifiable, while the cost-effectiveness of NVMe is still a challenge.

NVMe is a non-volatile storage media attached via a PCI Express (PCIe) bus. While PCIe predates NVMe by several years, older data transfer protocols—such as SATA, AHCI and SCSI—were created when the hard drive was the height of storage technology. NVMe provides low-latency commands and multiple queues. These advantages are important, because data is transferred to SSDs in chips and blocks. NVMe is also ideal for the fastest flash storage technologies, because it offers thousands of parallel queues. That is an exponential increase in performance.

With up to 6 x better performance compared to classic SSD disks, NVMe drives are the new benchmark for fast storage.

#3 – Hyper-converged

Hyper-converged architectures are disruptive, too. They place fast storage close to the CPU to increase performance. As computer clusters grow, the need to add storage outstrips the need to add CPU, requiring the insertion of a data-tiering strategy into the architecture.

NVMe integrated architecture is ideal for hyper-converged environments. These environments have intense data operations, high performance applications, and other exceptional performance requirements. NVMe connections can also lessen challenges involving data locality issues. Solving latency concerns will expand the use of hyper-converged environments to larger data set workloads—which will be major draw for larger enterprises.

#4 – Edge computing

Edge computing is a third factor which will have a major effect on computing, data distribution and storage architectures. And yet, there is not a clear vision—right now—about what should be stored at the edge versus what should be forwarded to a more central storage system.

IoT is the catalyst behind edge computing, though there are currently no data formatting standards. Nor is security where it should be.

IoT and edge computing must further normalize data across devices. Companies like Apple, Google, and Amazon are in the midst of pursuing this goal within their respective ecosystems.

#5 – Portable storage architectures

Portable storage architectures can provide a cost-effective alternative to flash storage. To a lesser extent, or perhaps for specific industries that need to store and transport massive amounts of encrypted data quickly, portable storage architectures will be extremely disruptive. In fact, they could disrupt the wholesale data movement in its entirety especially in environments where storage density, high-performance, and portability are paramount.

In terms of benefits, disruption increases performance and lowers costs. The most successful players will be the ones who meet the current and future needs of clients. That means they will have identified needs before clients know what they need.

The degree of disruption may vary—it will vary. But the event itself will be too impressive to dismiss and too influential to deny. It will be measurable in heightened performance and lower pricing. It will be evident to consumers and a compelling testimony in its own right.

Sign up for the free insideBIGDATA newsletter.