Behind the scenes - Kubernetes, Nomad & Fargate

While the container orchestration technology is still evolving, many tools have come into existence in the last 3-4 years, I’m picking Kubernetes, Nomad and Fargate for a detailed comparison. Before jumping into the comparison of the tools, I have a quick overview of the architecture of all the three tools below. Jump to Comparison directly.

Kubernetes Architecture:

Kubernetes is a container orchestration platform that enables you to deploy and manage containerized applications at scale. It is made up of a number of components, each of which plays a vital role in its overall functionality.

The Kubernetes control plane is responsible for maintaining the desired state of the system and responding to events  that occur within it. It is made up of a number of components, including the api server, scheduler, and controller manager.

The Kubernetes api server is the central point of interaction with the Kubernetes control plane. It is responsible for processing all API requests and responding to them accordingly.

The Kubernetes scheduler is responsible for scheduling pods to run on nodes in the cluster. It ensures that all pods are properly placed and have adequate resources available to them.

The Kubernetes controller manager is a collection of controllers that work together to maintain the desired state of the system. These include the replication controller, endpoint controller, and namespace controller.

Kubernetes nodes are the workers in a Kubernetes system. They are responsible for running the pods that have been scheduled to them. Each node must have a copy of the kubelet, which is responsible for communication with the Kubernetes control plane and ensuring that the pods on the node are running as desired.

Kubernetes pods are the smallest deployable units in a Kubernetes system. They are groups of one or more containers, each of which shares a common network namespace and storage volume. Pods provide a way to tightly couple containers that need to communicate with each other.

Kubernetes services provide a way to expose pods to the outside world. They are logical groups of pods that are exposed through a common endpoint. Services can be used to load balance traffic across a group of pods or to provide a single point of entry into a system.

Kubernetes deployments provide a way to manage the lifecycle of a group of pods. They provide a declarative way to define the desired state of a group of pods and ensure that the correct number of pods are always running.

Kubernetes ingress provides a way to expose services to the outside world. It is a collection of rules that map external addresses to services within a Kubernetes cluster.

Kubernetes secrets provide a way to store sensitive information, such as passwords and API keys, in a Kubernetes cluster. They are encrypted and stored in the Kubernetes api server.

Kubernetes config maps provide a way to store configuration information for use by pods in a Kubernetes cluster. They can be used to store things like database connection strings and API keys.

Kubernetes persistent volumes provide a way to persist data in a Kubernetes cluster. They are usable by any pod in the cluster and can be used to store things like databases or application logs.

Kubernetes namespaces provide a way to isolate resources in a Kubernetes cluster. They can be used to limit the scope of a service or deployment.

Kubernetes RBAC provides a way to control access to Kubernetes resources. It is based on roles and permissions that are assigned to users and groups.

Kubernetes networking provides a way to connect pods and services together. It is based on the Container Networking Interface (CNI) and provides a number of plugins that can be used to create different network topologies.

Kubernetes storage provides a way to store data in a Kubernetes cluster. It is based on the Container Storage Interface (CSI) and provides a number of plugins that can be used to create different storage backends.

Kubernetes addons provide additional functionality for a Kubernetes cluster. They include things like monitoring, logging, and DNS. Addons are not installed by default and must be installed separately.

Nomad Architecture:

Hashicorp Nomad is a container orchestration tool that enables you to manage and deploy containers across multiple hosts. It is designed to be scalable and fault-tolerant, making it ideal for running mission-critical applications.

At the core of Hashicorp Nomad is a distributed system that consists of a set of servers that coordinate with each other to manage the scheduling and execution of jobs. Jobs are units of work that are submitted to the Nomad cluster by users. Each job is made up of one or more tasks, which are executed by Nomad agents that are deployed on hosts in the cluster.

Nomad servers use a raft consensus algorithm to elect a leader and replicate state. This ensures that the cluster can continue to function even in the event of server failures. The Nomad server members also coordinate with each other to perform leader election and to gossip about job status changes.

Nomad clients submit jobs to the Nomad servers. They also monitor the state of running jobs and can be used to run tasks locally on the client host.

Nomad agents are deployed on hosts in the cluster. They register with Nomad servers and receive jobs that have been scheduled for execution on their host. The agents then execute the tasks associated with those jobs.

The Hashicorp Nomad architecture is designed to be scalable and fault-tolerant. It can be used to manage and deploy containers across multiple hosts, making it ideal for running mission-critical applications.

AWS Fargate

AWS Fargate is a container orchestration service that makes it easy to run and manage containers at scale. Fargate eliminates the need to provision and manage servers, making it a cost-effective way to run containers. Fargate is like the no-code version of container orchestrators 😂.

Fargate is made up of two key components:

• Container Orchestration Service: This is the core service that manages and schedules containers on Fargate.
• Container Runtime: This is the software that runs containers on Fargate. It is based on open source container runtime technologies such as Docker and Kubernetes.

Fargate is designed to be scalable and highly available. It can run containers on a  large number of servers across multiple Availability Zones. Fargate also has built-in redundancy and can automatically recover from failures.

Comparison

Ease of Use: Nomad is the clear winner here. It has a very simple UI that makes it easy to deploy and manage applications. AWS Fargate is a bit more difficult to use, but still manageable. Kubernetes can be quite difficult to use, especially for beginners.

Multi Region  Federation: Nomad and Kubernetes both support multi region federation out of the box. AWS Fargate does not currently support this feature.

Scalability: Nomad is the most scalable of the three options, followed by Kubernetes. AWS Fargate is the least scalable option.

Multi-Cloud: Nomad and Kubernetes both support multi-cloud deployments. AWS Fargate only supports deployment on Amazon Web Services.

Vendor Lockin: Nomad is the least likely to cause vendor lockin, followed by Kubernetes. AWS Fargate is the most likely to cause vendor lockin.

Batch Jobs: Nomad and Kubernetes both support batch jobs. AWS Fargate does not currently support this feature. But Nomad beats the other two in this aspect as it is widely used for the batch jobs kind of uses cases mainly. Kubernetes support for Batch jobs still has issues and is evolving.

Community: Kubernetes has the largest community, followed by Nomad. AWS Fargate has a smaller community, but is growing.

Granular Control on Infrastructure: Nomad gives you the most control over your infrastructure, followed by Kubernetes. AWS Fargate gives you the least control over your  infrastructure.

Below is a detailed comparison sheet considering all possible attributes for picking the right container orchestration tool for your needs.

‍

‍Conclusion

Kubernetes is versatile and can be used for a variety of purposes. It is also scalable, so it can grow with your business. However, it can be more complex to set up and use than the other options.

Nomad is a great option if you're looking for something lightweight  and easy to use. It's also affordable, making it a good choice for small businesses or startups. However, Nomad doesn't offer as much customization or features as some of the other options.

AWS Fargate is a good choice if you're already using Amazon Web Services (AWS) and want to use their container orchestration services. Fargate is also a good option if you need more customization and features than Nomad offers. However, it's important to note that AWS Fargate is more expensive than Nomad.

Nomad is the clear winner in most aspects for simple container orchestration and cluster scaling considering the ease of use and the support for multiple workloads, with Kubernetes coming in a close second. AWS Fargate is a good option if you are already using AWS and do not need the advanced features offered by Nomad and Kubernetes.

‍