Hands-on Guide: How to Find and Remove Orphaned EBS Snapshots

Intro

In this hands-on guide, we will share how to properly maintain your Amazon Elastic Block Store (EBS) snapshots by identifying and deleting or archiving old snapshots. By performing basic snapshot maintenance, you can greatly reduce your AWS storage costs.

EBS Snapshots Overview

Amazon Elastic Block Store (EBS) snapshots are point-in-time copies of an EBS volume. They are stored in Amazon S3 and can be used to create new EBS volumes, or to restore data to an existing volume.

EBS snapshots are incremental, which means that only the blocks that have changed since the last snapshot are saved, resulting in lower storage costs. Snapshots can be used to create new EBS volumes, or to restore data to an existing volume. They can also be used as a backup solution and enable users to create multiple versions of a volume, or to move data between regions or accounts.

Snapshots can be created manually or scheduled to run automatically at specified intervals. They can also be protected from accidental deletion and users can set up retention policies for them.

The monthly bill for Amazon Elastic Block Store (EBS) snapshots varies depending on the amount of data stored in the snapshots and the region where the snapshots are stored.

The storage cost for an EBS snapshot is based on the amount of data stored per month, measured in gigabytes (GB). The cost varies by region, but in general, it is very low, typically a few cents per GB per month.
As of today, a standard EBS snapshot storage pricing is $0.05/GB-month,
While archive EBS snapshot storage is $0.0125/GB-month.
While this unit cost may not sound like much, here’s a simple calculation to demonstrate what your team may be paying for unused EBS snapshots:  

Let’s say you have 200 EBS snapshots with 250 GB of data each, that are not in use. You might be paying 200 x 250 x $0.05 = $2,500/month per team.

When you pay for snapshot storage you don't need, you may be adding to your organization's increasing cloud costs.

Important notes:

#1 Prices for storage are subject to change, so make sure you check the Amazon EBS pricing page for the most current pricing information.

#2 Copying snapshots to other regions holds data transfer cost associated with it, which is based on the amount of data transferred and the region to which the data is being transferred.
‍
#3 Deleting a snapshot might not necessarily reduce your organization's data storage costs. Other snapshots might reference that snapshot's data, and referenced data is always preserved. If you delete a snapshot containing data being used by a later snapshot, costs associated with the referenced data are allocated to the later snapshot.

‍What are Orphaned EBS Snapshots?

Snapshots are incremental backups, meaning that only the blocks on the device that have changed is saved on the "newer" snapshot been taken. The snapshots also contain information references to a previous snapshot regarding the data that has not been changed.

Here’s Amazon’s guidance on how snapshots work.  

An EBS snapshot would be considered as "old" if you have not used it in X time/days/months - depends what you consider as "old" per policy. Some can say it's 3 months while others 1 year. It really depends on the user and use-case.

An old snapshot may still be in use or referenced from newer snapshots (again, depending on policy), but it becomes redundant when there’s a new snapshot that covers the same data (disk sections). Old snapshots that are no longer referenced are called orphaned snapshots (aka orphans or unused EBS snapshots). When an instance in AWS is deleted (terminated), the volumes of this instance are also deleted (usually), but its snapshots remain in the cloud.

How to Find and Delete Orphaned EBS Snapshots

You can delete EBS snapshots using AWS console or AWS CLI.
When you delete a snapshot, only the data that is referenced exclusively by that snapshot is removed. Unique data is only deleted if all the snapshots that reference it are deleted.  

Finding and Deleting Orphaned EBS Snapshots via AWS Console

To find and delete a snapshot using the console:

1. Open Amazon EC2 console

2. In the navigation pane, choose Snapshots.

3. Select the snapshot(s) you would like to delete, and then choose Actions, Delete snapshot.
   (You can filter the list of snapshots using tags and other snapshot attributes to refine your search. In the Search field, select the attribute field, and then select or enter the attribute value. For example, to view only encrypted snapshots, select Encryption, and then enter true)

4. Choose Delete.

Finding and Deleting Orphaned EBS Snapshots via AWS CLI

AWS CLI allows you to search for specific types of snapshots.

1. Use the describe-snapshots command to get a list of snapshots available to you, including public snapshots, private snapshots that you own, and private snapshots owned by other Amazon Web Services accounts for which you have explicit create volume permissions.
   To filter snapshots created before a given time, you can add a JMESPath expression.

aws ec2 describe-snapshots  
--query "Snapshots[?(StartTime<=`2022-06-01`)].[SnapshotId]" --output text

2. To find old snapshots while also using tags, you can add a filter to the command.  
   The example command below contains a “Team” tag for getting back only snapshots owned by “Infra” team.

aws ec2 describe-snapshots --filter Name=tag:Team,Values=Infra
--query "Snapshots[?(StartTime<=`2020-03-31`)].[SnapshotId]" --output text

3. With the list of old snapshots or snapshots associated with a specific tag, you can execute the delete-snapshot command to delete them.

aws ec2 delete-snapshot --snapshot-id <value>

As previously mentioned, snapshots are incremental, meaning that if you delete a snapshot that has data referenced by another snapshot, that data will be transferred to the next snapshot. This means that the reduction in storage may not be as significant as you expect, as the data is still stored in other snapshots. However, if there are block data changes that were captured by snapshots that are no longer needed, deleting those will save storage space, thus lowering costs.

‍Archiving EBS Snapshots

Archiving snapshots is another way to potentially lower costs for long-term storage of rarely accessed snapshots that do not need frequent or fast retrieval.
While archived snapshots are stored at a much lower cost (up to 75% lower) than standard snapshots, archived snapshots are always full snapshots, meaning, they will likely be larger than the incremental snapshot from which they were created from.

Some typical use cases for archiving snapshots include:

• Archiving the only snapshot of a volume, such as end-of-project snapshots.
• Archiving full, point-in-time incremental snapshots for compliance reasons.
• Archiving monthly, quarterly, or yearly incremental snapshots.  
  

To evaluate if snapshot archiving is a potential solution you can take to lower your storage costs, I would recommend reviewing archiving snapshots considerations and limitations and  AWS guidelines and best practices for archiving snapshots.

Using Amazon Data Lifecycle Manager for EBS Snapshots

Amazon Data Lifecycle Manager (DLM) is a service provided by Amazon Web Services (AWS) that allows you to automate the management of the lifecycle of your Amazon Elastic Block Store (EBS) and Amazon Relational Database Service (RDS) snapshots.

DLM can be used to automatically create snapshots, schedule the creation of snapshots, and set retention policies for snapshots. It also provides options for controlling the number of snapshots kept, and the time period for which snapshots are retained. You can also use DLM to automatically delete or archive snapshots that are no longer needed, helping you to reduce your storage costs.

DLM is a simple and cost-effective way to manage your snapshots, it allows you to back up your data, and recover data from snapshots without any manual effort. You can also use it to schedule automated backups, and to set up retention policies for backups.

Amazon Data Lifecycle Manager (DLM) can help you reduce your cloud costs by automating the management of the lifecycle of your Amazon Elastic Block Store (EBS) and Amazon Relational Database Service (RDS) snapshots.
Here are a few ways that DLM can help you reduce your cloud costs:

1. Automated snapshot creation: DLM can be configured to automatically create snapshots of your EBS and RDS volumes at specified intervals, eliminating the need for manual backups. This can save on operational costs.
   

2. Retention policies: DLM allows you to set retention policies for snapshots, which can be used to automatically delete or archive snapshots that are no longer needed. This can help to reduce storage costs.
   

3. Schedule snapshots: DLM can be used to schedule snapshots to be created at specific times, allowing you to control when snapshots are taken, and when they are deleted. This can be used to minimize the amount of storage used by snapshots.
   

4. Cost optimization: DLM allows you to optimize your storage costs by reducing the amount of data stored in snapshots. Since snapshots are incremental, deleting older snapshots will free up storage space and lower your storage costs.
   

5. Compliance: With DLM you can also set up policies to ensure that you are keeping backups for compliance with regulatory requirements, and you can do it in an automated way, which may save you money on compliance related costs.

It's important to note that DLM can only be used for EBS and RDS snapshots, and it does not support other types of data, such as files stored in Amazon S3.

‍The New & Easy Way: Find Old EBS Snapshots with Lightlytics

Using Lightlytics Architectural Standards, you can easily find potential old EBS snapshots to be deleted. You can use Lightlytics’s out-of-the-box EBS snapshots cost rules or you can create your own custom rules for tags and various EBS snapshot attributes.
By using these capabilities, you can review the total predicted cost of each rule, and the breakdown of cost per violated resource within each rule.

Example architectural standard: Ensure there are no EBS snapshots older than a month
This rule checks for EBS snapshots older than 1 month within your AWS account, so you can remove them in order to lower the cost of your monthly bill. The threshold for the retention period is 1 month, which means that all incremental snapshots older than 1 month should be deleted. Since EBS snapshots are incremental, deleting previous (older) snapshots does not affect the ability to restore the volume data from later snapshots which allows you to keep just the necessary backup data and lower your AWS monthly costs.
This rule can also help you work with the AWS Well-Architected Framework.

This architectural standard’s conditions:

‍

Review rule violations: When there are violations for this rule (or any rule in our architectural standards), this view shows each violated resource including the related predicted cost.

You can also create your own custom rules using the rule creation wizard on Lightlytics.
Here’s a custom rule example:
The below custom rule checks for EBS snapshots older than 3 months owned by Team Infra. EBS snapshots offer teams the ability to back up their EBS volumes and easily create a new volume with point-in-time data. Using the “Team” tag is helpful in identifying who owns the snapshots, in order to identify and delete snapshots that are no longer needed.

‍Conclusion  

1. Why and how removing EBS snapshots reduces cost
   Removing unnecessary EBS snapshots can lower storage costs by reducing the amount of data stored in Amazon S3. Similarly, it can lower data transfer costs by reducing the amount of data transferred. Setting retention policies for snapshots can also help to reduce storage costs by automatically deleting or archiving them after a certain period of time. It's important to keep in mind that removing snapshots must be done with care, ensuring that they are not still needed for recovery or that they are not a part of a snapshot retention policy.
   

2. Where to start managing EBS snapshots?
   Use AWS Cost Explorer to view and analyze your EBS snapshots costs and usage.
   This tool can help you visualize cost and usage at a high level or at the resource level.
   After you get a clearer picture of your EBS snapshots costs and usage, you can define which EBS snapshots can be deleted.  
   

3. Which tools to use?
   Find and delete orphaned (aka old or unused) EBS snapshots using AWS console or AWS CLI.
   Alternatively, Lightlytics Architectural Standards can easily help you find potential old EBS snapshots to be deleted using out-of-the-box EBS snapshots Cost rules or your own custom rules.
   Consider archiving snapshots when you want to retain snapshots for long-term storage at a lower cost than standard snapshots storage.
   Use Data Lifecycle Manager (DLM) to reduce your cloud costs by automating the management of the lifecycle of your Amazon Elastic Block Store (EBS) and Amazon Relational Database Service (RDS) snapshots.

4. Final tips, & mistakes to avoid
   a. Tag Your EBS Snapshots
   Tagging EBS snapshots allows grouping snapshots by project, application, environment or other criteria. By tagging them, you can easily identify and delete snapshots that are no longer needed, which can help to lower your storage costs. Additionally, tagging allows for compliance, automation, cost allocation, and searchability, making it easier to manage and track the cost of your snapshots.
   b. Practical tips
   i. AWS Console shows the size of each snapshot as the size of the volume at the time the snapshot was taken, giving the impression that they are full backups. However, this is not entirely accurate as the storage footprint of snapshots is much smaller in reality. To obtain more accurate and detailed storage information for cost estimation and optimization, you can use the EBS Direct API.
   ii. Copying snapshots between regions incurs data transfer and storage costs, and it can increase the complexity of managing and tracking your costs. Additionally, it may increase the time required to restore data from that snapshot, which can be a problem in case of an emergency. It's important to weigh the costs and benefits of copying snapshots between regions, consider alternative solutions and evaluate if the additional costs outweigh the benefits.  
   iii. Deleting a snapshot might not necessarily reduce your organization's data storage costs as other snapshots may reference the data contained in the deleted snapshot. As a result, the referenced data will be preserved and the cost of storing the data will be allocated to the later snapshot. In other words, if you delete a snapshot that contains data being used by a later snapshot, the cost of storing that data will not be reduced.
   c. Measure twice, cut/delete once: It's important to be mindful of what snapshots are used for and to not delete snapshots that are needed for recovery, compliance, disaster recovery, creating new volumes or testing.

Found this useful? 

- Read Tal's second blog post in this series: Hands-on Guide: How to Find and Remove Unattached Elastic IPs

- Reach out to Tal on LinkedIn if you'd like to suggest other topics, tips & tricks to reduce AWS cost.