Most enterprises now run applications in containers, so they need to pay attention to how they store and manage data for containerized applications.
Nutanix Enterprise Cloud IndexA study compiled for cloud software provider VansonBourne and published earlier this year found that 54% of companies have containerized all their applications and a whopping 98% use at least one instance of Kubernetes.
However, this creates a number of challenges for IT architects when it comes to data storage. Containers were designed to be ephemeral or temporary in nature. This works quite well for microservices. But core enterprise applications need to process and store data. This required developers adapt container technology to support persistent storage.
Containers bring a number of benefits to businesses. Containerized applications run independently of the host operating system, making them easily portable. This helps companies that run applications in hybrid or multi-cloud environments.
Containers are also “lightweight”, requiring fewer resources, especially storagethan regular virtual machines. Containers are more efficient and spin up in seconds rather than minutes.
And although not all container applications microservicesThe efficiency of containers allows them to run and build complex applications from small, reusable, and efficient parts.
How are containers and storage related?
The first generation of containers was intended for stateless persons. This had advantages in speed of deployment and efficiency. But stateless or non-persistent applications cannot retain data beyond the lifetime of the container.
Stateless applications work in some situations, such as web services or microservices that do not require persistent storage and access to data. But this ability to process data is critical to many, if not most, enterprise applications.
As a result, container technology has adapted by adding persistent storage. Persistent storage is located outside the container and can be located on local or cloud hardware in the form of file, block or object storage.
The container orchestration layer manages persistent storage. In case Kubernetes – The most common container orchestration system, data is stored in persistent volumes (PVs) and exposed through persistent volume claims (PVCs), which are portable and can move with the container.
PVs are independent of any pod and are not portable between Kubernetes clusters. However, they both serve to separate container and storage so that “regular” storage works with containerized applications.
The challenge for IT teams, however, is that this is far from a simple solution. Containerized applications, the orchestration layer, and storage must work together for an enterprise application to work.
How does CSI help with data storage for containers?
To simplify and standardize connecting containers to storage, the industry has developed container storage interface (CSI) and container storage.
CSI works with cloud, local and hybrid storage, as well as file, block and object storage. This allows developers to tailor storage to their workloads.
CSI is a set of standards that allow storage providers to connect their technologies to Kubernetes. There are currently over 100 different CSI drivers available for conventional and software-defined storage.
CSI continues to evolve, adding support for more storage formats and more providers. Another benefit of CSI is that it helps IT teams manage storage consistently, even across multiple vendor infrastructure.
What is container storage?
Container storage, on the other hand, is software-defined storage that runs inside a container in Kubernetes. Container storage offers the ability to allocate storage to a container only when the container needs it, making it more flexible than other forms of storage.
Vendors offering container storage include Red Hat's OpenShift Data Foundation (ODF), Portworx Pure, and Nutanix Unified Storage.
According to industry analyst Gartner, by 2029, 95% of organizations worldwide will use containerized applications.
cloud storage however, the market is less mature. Industry estimates put Portworx and Red Hat ODF together at less than 30% of the market, although analysts expect the market to more than double by the end of this decade. This suggests that businesses are currently adhering to CSI.
How do storage providers support container storage and backup?
Suppliers are working to container storage Easier to manage and works better with different storage technologies. This is even more important for enterprises operating hybrid clouds. Some companies want to keep storage in-house or in private clouds, but still want to take advantage of cloud and container applications.
As a result, vendors including Dell EMC, HPE, Hitachi Vantara, IBM, NetApp and Pure each of them worked to improve container support. Hyperscale cloud providers also continue to expand their support for containers.
These technologies are now quite mature and, as such, should allow enterprises to run containerized applications with persistent storage without changing hardware or providing cloud storage.
On-premises and cloud service providers have also improved support for backup and recovery of container environments. Reliable backup is an important feature for the deployment of corporate production.
What container management products do vendors offer?
However, the challenge remains to further eliminate the complexities associated with persistent storage and containers.
Tools like Pure's Portworx and NetApp's Trident are designed to make it easier to deliver Kubernetes applications while also improving portability and security.
NetApp's Trident is open source, free, and uses CSI. It supports automatic provisioning of NetApp OnTap storage as a PV for Kubernetes. Trident also offers data management, data protection, disaster recovery and business continuity for container environments.
Portworx also provides automated data services and policy-based management. Portworx also uses CSI and combines the underlying storage into a single data structure. This is then distributed among the clusters.
Pure notes that Portworx provides a consistent data storage model across cloud, hybrid and on-premises storage with “cloud-like agility and agility” for on-premises environments.
Pure recently integrated its intelligent Fusion control plane into Portworx and added an artificial intelligence (AI) co-pilot that it says can control Kubernetes clusters at scale. Portworx also integrates backup and recovery and automated capacity management into its platform.
All of these developments should make it easier for developers to build containerized applications that require persistent storage, while also removing some of the overhead of storing and managing data, as well as disaster recovery once containerized software goes into production.
