Skip to main content

AWS

Stateful Nodes for AWS provide the following key capabilities:

  • Persistent stateful applications: Run stateful applications on stateful nodes that utilize spot nodes while providing persistence for the root volume, data volumes and network interface, in addition to other launch specifications during node replacements.

  • Solution for spot node challenges: Address the challenge that spot nodes are transient and can be interrupted at any moment by AWS. While this is not an issue for stateless applications as they are designed to easily scale horizontally, it brings a great challenge to stateful applications.

  • Support for fault-tolerant applications: Designed to support any fault-tolerant applications, such as databases and stream-processing servers. If you have a stateful application or an application that is designed to withstand node failure, you may have decided in the past that Spot nodes were not the best use case for you. With Stateful Nodes you can maintain your configurations with zero data loss while taking advantage of the cost savings provided by spot nodes.

Common Use Cases​

Every fault-tolerant application that requires data persistency can benefit from the stateful capabilities offered by Stateful Nodes.

Single Non-prod Server Database​

Non-production database environments can accommodate periodic maintenance windows and brief interruptions, making them ideal candidates for Stateful Node.

Cassandra​

When a Cassandra node is interrupted, Stateful Node launches a replacement instance with the original node's data volumes and network interface preserved, allowing the node to rejoin the cluster without data loss. The Cassandra cluster the node is connected to will behave as if the node was down for some time. Preserving the original node's state ensures that cluster IOPs are not wasted on rebuilding a new node from scratch.

Development Nodes​

Non-production nodes can be run on spot nodes with occasional downtime. When nodes are interrupted, they are automatically replaced with new instances that preserve the original state within a few minutes.

Elastic.co​

Elasticsearch node recovery will take a fraction of the time required to provision a new node. From the standpoint of the Elasticsearch cluster the node was only down for a period of time (depending on the size of the data volumes attached). The cluster requires no configuration changes to handle this recovery, as long as sufficient nodes remain available to maintain quorum.

Kafka​

Kafka's architecture is designed based on several components and each component has its unique role. Stateful Node supports Brokers and ZooKeeper cluster nodes, as well as the consumers, which can run seamlessly on Spot Nodes.

MongoDB​

When running a MongoDB cluster, it's important to preserve the data, when the data is missing a rebuild process is initiated and the data is resynced in its whole. Stateful Nodes allow the retaining of the data volumes of the machine. Any EBS volume that is attached to the node will be continuously snapshotted while the machine is running, and the final snapshot will be used as part of the block device mapping configuration upon replacement.

ML/AI​

Running a cluster of fast.ai ImageNet processing nodes on spot nodes would normally incur heavy speed penalties due to the loading times associated with stopping and starting compute jobs. Stateful Nodes enable EBS data volume persistence by keeping the original data volume ready for use and reattaching it to the replacement node. This eliminates the warm-up time that would otherwise be required, and enables fast launch times even for heavy applications.

Big Data​

Running Redis cluster secondaries on the Stateful Node platform reduces the cluster's cloud compute costs by leveraging AWS Spot Nodes. Due to the fact that Redis cluster nodes are stateful applications that require data continuity, it is possible to deploy the agent servers by leveraging Stateful Nodes' stateful features.

Stateful Persistence Options​

Stateful Nodes utilize a combination of snapshots, volumes and ENIs to ensure state persistence.

  • Persist root volume: Selecting to persist the root volume ensures that Operating System and root volume configurations are maintained during node replacements.

  • Persist data volumes: Data devices are maintained using one of the following methods:

    • Reattach Volumes: The same EBS volumes are detached from the original node and reattached to the new node.
    • Snapshot Backups: During the launch of a new node, new EBS volumes are created and attached to it as part of the launch specifications.

  • Persist Network Interface: New nodes are provisioned with the same private IP using the same Elastic Network Interface.

    • Private IP Pool – Specify a pool of private IPs to be used by the Stateful nodes.