The Evolution of Storage
By Paul Speciale- Chief Product Office, Giorgio Regni- CTO, Jérôme Lecat-CEO, Erwan Girard- VP for Product Management, Brad King- Co-founder and Chief Architect, and Pierre Derome- VP Enginnering of Scality.
Storage is big! It’s a huge $40B+ industry that accelerated by 14% in Q3 2017, according to IDC. Storage capacity shipped increased to 72.5 exabytes for that quarter alone. Exabytes are almost beyond comprehension. How big is an exabyte? Well, it’s been said that 5 exabytes would be equal to all the words ever spoken by mankind. We’re still trying to wrap our heads around that.
All of this begs the question: where is this storage industry going, and how can it keep up? What trends are driving this booming industry? What affect do regulatory changes like GDPR have on storage? How is the cloud changing storage? What other technology trends could drive the industry?
Here’s how we see it
Software-Defined Storage will ship more storage in 2018 than any other architecture
Software-defined storage adoption has continued to grow rapidly, due to its flexibility and agile architecture. 2018 will be the first year in which software-defined storage capacity shipped will exceed all of the traditional storage appliances combined (time to sell that Dell-EMC, IBM and Netapp stock).
It makes sense. Software-defined storage allows for an always-on system, on-demand scaling and hardware flexibility for lower cost. Companies can take advantage of that flexibility to mix and match hardware generations and vendors, driving down cost. It’s a no-brainer.
Multi-Cloud data storage will gain traction into the largest Enterprise IT organizations
Cloud is a reality in the Enterprise IT world. IT organizations need to be in control of their data and will want a degree of cloud independence. The leading enterprises will become multi-cloud service organizations, using private and public cloud to transform and accelerate their business.
A new Data Control Layer is emerging, which can add governance and assurance capability to the data for enterprise businesses – ensuring a seamless and consistent experience in terms of secure encryption, access control and rich metadata services across multi-cloud environments.
The Rise of Metadata
As the volume of data grows, metadata is critical. Metadata is king, and we’re seeing a rise in storage systems that will store both the data itself and relevant metadata about that data. Traditional storage approaches like NAS, and even worse SAN storage, had no context knowledge of the data that they stored. Given the growth of data volumes, we need storage to be smarter; storing information and not just a stream of bytes. We will see growing usage of data management via metadata for industries like Healthcare, Financial and Media/Entertainment. The metadata will be indexed and searchable as part of the data storage layer.
Healthcare presents a great example of the importance of metadata. High-level patient information, including Patient Name, Address, Demographics, Admission Date, etc., stored with radiologic images, lab data and other health records information makes data easier to find and use. Metadata makes the data more searchable, more usable..
Convergence of object storage and analytics
Somewhat tied to the above point, storage systems must change to understand the data they are storing. That will apply to the metadata tagging and also to the data itself.
Today, people are adopting more object storage for their 100s Terabytes to Petabytes of storage capacity needs. Moving this data outside of the storage to be analyzed is inefficient. Instead you will see the embedding of analytics layers into the storage technologies. Object Storage vendors will take the lead in this space.
Last days for the Storage Administrator – application-driven DevOps takes over
DevOps is now commonplace, and as companies build applications, they design into those applications the ability to control the infrastructure. This will mean that the traditional role of storage administrator will start to disappear.
Cloud Storage (in the private or public cloud) provides API driven controls to manage data by applications (for example the Amazon S3 API) helping to automate many of the previous mundane tasks of a storage admin.
Privacy regulations like GDPR require data to be always-on and available
Lastly, but certainly not the least, in 2018, data controls and privacy management got teeth. New regulations like the European Community’s General Data Privacy Regulations (GDPR), require data to be always available and indexed. You can’t tell a person what data you have on them if that data is sitting in some offsite tape archive collecting dust.
This kind of data access requirement will drive the further decline and demise of the offline tape. With GDPR, data needs to be always accessible, easily indexed and rapidly retrievable. This is not possible with traditional offsite tape. Traditional bulk data backup via tapes and trucks will finally disappear.
The decline of tape won’t be driven by cost; it will be driven by the need for continuous, always-on access. So long tape; you’ve had a good run.