Photo by Christian Wiediger Unsplash.
Want to use AWS reservations to save money, but have trouble matching your reservations to your EC2 servers? Or maybe you want to spread out your reservations, but need an easy way to get a list of servers that currently don't have a reservation? AWS reporting doesn't currently make this possible, but Request Tracker’s asset system can.
Understanding AWS Reservations
If you run services like EC2 and RDS on AWS, you probably know that there are several levels of pricing based on how long you can commit to using the services. On-demand is the most flexible because you can deprovision it at any time, so it's also the most expensive. If you can commit to a longer term, like a year or more, you can sign up for "reserved instances" and get a lower price.
When you sign up for these reservations, AWS asks for a bunch of information about the servers you are committing to. These are details like the region, the type of server (t3.medium, etc.), and the platform (Linux, etc.). For the reservation to get credited on your bill correctly, all of the reservation information must match up with your running servers.
When you start running more than a few services, tracking all of this information to make sure your reservations are being used can become a challenge. AWS has a utilization report that will tell you in aggregate if your reservations are being used or not. But this report doesn't connect specific EC2 or RDS instances with a reservation. This can be good if you remove an instance and then create a new one because the reservation will transfer. But if you want to review all active services and figure out which reservations are being used where, the AWS reporting doesn't provide that.
RT Asset Tracking
RT's asset tracking system is designed to record this kind of information, so we decided to start tracking our AWS resources and reservations in RT to help us match the two together.
To start, we needed to get the AWS assets created in RT. AWS has extensive APIs to access information about your resources, so we created a utility to fetch data about EC2 and RDS instances and create an asset record for each. We put all of these in one catalog called "AWS Resources".
Next we wanted to track our purchased reservations. We created a new catalog called "AWS Reserved Instances" to track those, since they aren't really the same thing as an EC2 or RDS instance. We also wanted to track different asset details on reservations, like start and end dates. Reservations also have "states" in AWS, so we were able to map that to the asset status and create a custom asset lifecycle with all of the AWS state values like "pending payment", "active", and "retired".
Once we had all of the information sync'd from AWS, we could start making sense of it. We started by creating some categories of assets using a few saved searches. We put them all in a dashboard like the one below so we could see what we had.
AWS asset dashboard in Request Tracker
Assigning Existing Reservations
The first thing we wanted to do is connect reservations with a matching resource. This mapping is very useful for tracking what we have purchased, but it just doesn't exist in the AWS interface.
Assets have links, similar to tickets, so we decided to link each resource asset with a reservation asset. To make that easy, we created a saved asset search called "Unlinked AWS Reservations", which is the third search above. This is all reservations without a linked asset, which at first was all of them.
In the search results, we added a custom column called "Link to Resource". Each reservation in the results then had a link to a new page that showed all AWS assets (EC2s, RDSs) that matched the details of that reservation. So a reservation for an EC2 t3.medium in us-east-1 would show all matching EC2 assets that also were not linked yet. This is important, because we need to make sure our linking aligns with the AWS rules for counting reservations in our bill.
To link them, we pick one from the list and click Link. We worked through all of our existing reservations until all were linked and our "Unlinked AWS Reservations" was empty.
Buying New Reservations
In our case, we didn't have a reservation for every AWS asset, so when we were done linking, our "EC2 Instances without Reservations" and "RDS Instances without Reservations" still had some assets. These two lists now became our shopping list for new reservations.
The tricky part about reservations is you need to manually fill out all of the fields in the AWS interface. If you get any fields wrong, your reservation won't match an existing resource and you'll end up paying for the unused reservation.
Previously we would keep an EC2 or RDS AWS console page open in one browser window, then open another browser window with the “buy” form, and carefully check every value. This was tricky because the values are at different locations all over the EC2 and RDS pages.
With our new asset searches, we modified the search layout to show all of the information needed to buy a reservation, and we put the values in the same order as the reservation form. This makes it much easier to buy new reservations with less stress.
We used this process to buy some new reservations now that we could easily see what was needed. Then we re-ran our import utility and the new reservations showed up in "Unlinked AWS Reservations". We followed the previous process and got them all linked until we had the coverage level we wanted.
Expiring Reservations
When you buy a reservation, you commit to some term, like 1, 2, or 3 years. Based on what you picked, the reservation record will have a duration. We use that duration to figure out the reservation end date and store that on the asset. This information helps populate the top two searches on our dashboard, "Recently Retired AWS Reservations" and "Expiring AWS Reservations".
The first is reservations that are now retired, so assets that previously had a reservation are no longer covered. That means our bill just went up and we need to buy a new reservation to get savings for another year.
To help plan ahead, "Expiring AWS Reservations" looks ahead 60 days to let us know which ones will be expiring soon.
Using these two searches, we can plan to buy new reservations at regular intervals. We can also watch the EC2 and RDS searches to see any new instances that might be added.
Connecting AWS with Your Business
The visibility we gained from linking the reservations was a huge win for us and saved a bunch of time we previously spent trying to understand what systems had reservations and where more were needed. But now that we have AWS asset records in RT, we can do even more. We can link them to customer tickets, link them to change management activity, and link them to sales and contract activity which is also tracked on tickets. Our AWS resources are now integrated into all of our other processes, which is a huge win given how important AWS is in our hosting business.
Does any of the above sound familiar? If you are trying to manage AWS resources via the AWS console, but having a hard time linking that to your other business processes, drop us a line. We'd love to share our experience and talk about how Request Tracker can bring some organization to your AWS infrastructure.