Bringing Enterprise Data Services to Containers and Microservice Architectures
October 26, 2016
Atlantis partners with Rancher Labs
Atlantis has partnered with Rancher Labs to deliver a fully-automated, unified platform that can orchestrate and
provision Docker containers and deliver purpose built application defined persistent storage for Docker containers. It provides a single platform with a rich set of data services, provisioning and management of compute, storage and containers by combining in-memory and persistent storage for container runtime and data storage for Docker volumes.
Atlantis USX Overview
- Atlantis USX is a software defined storage platform with data services such as in line data reduction, data protection with distributed persistence, distributed in memory storage, compression, inline IO acceleration, snapshots and data reduction aware backup and DR.
- Atlantis USX enables enterprises to pool and abstract all storage (SAN, NAS, Flash, RAM, DAS, Hybrid Arrays, and Public Cloud) and instantly deliver virtual storage volumes to any application. By pooling storage, both storage capacity and performance are used more efficiently in the datacenter, lowering cost, increasing performance and making datacenters more agile.
- Atlantis USX includes integrated Content-Aware Data Services that leverage Atlantis real-time data reduction technology, IO acceleration, provisioning, data mobility, security and business continuity for any storage.
- Atlantis USX provides policy-based management of virtual storage volumes that allows administrators to define the capacity, performance and availability requirement for individual VMs or applications.
- Atlantis USX delivers automation of all storage functions through REST APIs that are leveraged by third-party orchestration systems to provide Quality of Service (QoS), self-healing to provide resiliency, autonomics to create the optimal storage volume configurations, and event-based management to automate self-provisioning of applications.
- The figure below illustrates Atlantis USX’s major components and features.
- Rancher is an open source software platform that enables organizations to run containers in production. With Rancher, organizations no longer have to build a container services platform from scratch using a distinct set of open source technologies. Rancher supplies the entire software stack needed to manage containers in production.
- Rancher takes in raw computing resources from any private or public cloud in the form of Linux hosts. Each Linux host can be a virtual machine or physical machine. Rancher does not expect more from each host than CPU, memory, local disk storage, and network connectivity. From Rancher’s perspective, a VM instance from a cloud provider and a bare metal server hosted at a colo facility are indistinguishable.
- Rancher implements a portable layer of infrastructure services designed specifically to power containerized applications. Rancher infrastructure services include networking, storage, load balancer, DNS, and security. Rancher infrastructure services are typically deployed as containers themselves, so that the same Rancher infrastructure service can run on any Linux hosts from any cloud.
- Many users choose to run containerized applications using a container orchestration and scheduling framework. Rancher includes a distribution of all popular container orchestration frameworks today, including Docker Swarm, Kubernetes, and Mesos. The same user can create multiple Swarm or Kubernetes clusters.
- In addition to Swarm, Kubernetes, and Mesos, Rancher supports its own container orchestration and scheduling framework called Cattle. Cattle was originally designed as an extension to Docker Swarm. Cattle is used extensively by Rancher itself to orchestrate infrastructure services as well as setting up, managing, and upgrading Swarm, Kubernetes, and Mesos clusters.
- Rancher supports flexible user authentication plugins and comes with pre-built user authentication integration with Active Directory, LDAP, and Github. Rancher supports role based access control (RBAC) at the level of environments, allowing users and groups to share or deny access to, for example, development and production environments.
- The figure below illustrates Rancher’s major components and features.
Source Rancher Labs.
How Rancher and Atlantis USX work together
- We run Docker on our management appliance, USX Manager which orchestrates, monitors and manages the provisioning of USX volumes. Rancher Server is also deployed to run as a container on Docker and we integrate our USX management stack directly into Rancher Server. All management of containers, their applications running in the containers and the storage that these containers and applications consume are managed by Rancher Server. Atlantis USX is transparent to Rancher as all management is through the Atlantis USX machine driver.
- Rancher Server uses an Atlantis USX machine driver to provision, monitor and manage USX volumes and USX Container OS (USX OS) all within the Rancher Server web interface. USX Volumes provide the storage volumes which are used as Docker volumes for storing persistent data which can be shared between containers.
- USX OS is an Atlantis custom kernel that runs the Docker engine which is configured and registered by Rancher Server as a Rancher host. The USX OS is a powerful custom built minimal footprint container ready OS that utilizes the Atlantis USX in-memory engine to provide high performance and efficiency to enable the running of high numbers containers on a single machine instance.
- USX OS is based on Atlantis USX with the Docker engine installed by default. New Docker containers are run directly in memory and can be cloned using USX fast-clone technology by Rancher Server's USX machine driver. USX OS provides all the data services that are available today with Atlantis USX and Atlantis HyperScale. Which means that the containers running in memory are protected using our Fast Clone technology and benefit from the performance provided by running the container in server memory and also the compression to save memory space. The in-line data reduction and IO acceleration provide instant provisioning and single digit microsecond latency, even when running hundreds of containers on a single machine instance.
With Rancher and USX integration, a user is able to provision a new container host directly from Rancher Server and create persistent storage for new applications to use. The storage can be mounted as a Docker volume and made available to the containers to utilize. You will then be able to initiate many of the USX data services directly from the Rancher Server UI such as storage replication of the persistent volumes, instantly cloning a MySQL database, using Teleport to migrate or backup a MySQL database from one cloud to another or scheduling hourly backups of any object running in a persistent volume such as a MySQL database.
In addition, a USX volume can be provisioned by Rancher Server to be used as a local and persistent image library for Docker images. This enables fast local access to an image library which benefits from all of the USX data services, such as data reduction and I/O acceleration for fast and efficient storage and access of Docker images. Updates to images will be near instantaneous due to the in-line data reduction provided by the USX volume. Any forking of images or commits of images will benefit from the data reduction, therefore consuming very little additional storage capacity.
Containers are ideal for stateless applications that require high performance and low latency, USX enables Docker containers to run 100% in-memory.
For stateful applications, USX provides persistent Docker volumes in any infrastructure, including private cloud and public clouds.
- Simple integrated infrastructure – Delivering compute, memory and storage for containers
- Create an environment with disaster recovery using container replication, Docker volume or storage volume replication.
- Container management for on premises and cloud-based containers.
- Create a fast and efficient on premises Docker Image Registry.
- Replicate Docker image library between environments.
- Perform fast cloning of Docker volumes for development, test or quality assurance or backup or DR purposes.
- Perform SnapShot and replication of a USX storage volume hosting Docker volumes for dev, test, QA or backup and DR purposes
Data Reduction aware replication of Docker volumes and USX storage volumes
Data reduction aware backup of Docker volumes
Docker Image Library Replication
Container and Volume Instant Clones
USX for Containers Technical Preview
I’ve captured some screenshots of the Atlantis USX and Rancher integration below, you will be able to download and play with the Tech Preview in November when we will make this available to the USX Community Edition.