Cutting Costs in AWS: Look beyond the obvious to Tame your AWS budget

By | January 25, 2016

A version of the following article appeared on TechTarget SearchAWS as Four principles of AWS cost management


Money-IIThe cost calculus of moving applications from on-premise infrastructure to cloud services like AWS is akin to going from a fixed-price, all-you-can-eat buffet to an à la carte cafeteria. Gone is the incentive to consume as much as possible in order to maximize ROI (or food per dollar). Instead, the goal is to use (or eat) exactly what you need, but no more. It’s the classic economic trade-off between fixed and variable costs.

AWS has embraced pay-as-you-go as a defining feature, but it’s but one of four basic principles defining Amazon’s pricing philosophy, the others being:

  • pay less per unit by using more
  • pay less when you reserve
  • pay less when AWS itself grows

Although users have no direct control over the last tenet, albeit collectively they’re doing a great job of growing Amazon’s business, effectively exploiting the first three elements is key to getting the most value from AWS.

Start with the Basics

The obvious way to cut AWS costs is by using less. Unfortunately, that advice is about as useful as saying the way to investment riches is to buy low and sell high: the devil’s in the details. Indeed, the tendency is to over consume since the abstract, ephemeral nature of AWS, with resources that can be instantly and programmatically instantiated gives rise to a phenomenon that’s long plagued internal virtual infrastructure: VM, or in the case of AWS, resource sprawl. Much like the U.S. legal code, where laws and regulations once added never go away even long after they’re obsolete, there’s a tendency on AWS for machine instances and storage buckets to proliferate and persist long after they’re no longer needed. However, unlike internal systems where zombie VMs merely waste system resources, on AWS, the meter is always running, meaning that these dormant instances rack up the bill.

pricing-page_cycle

The key to reducing AWS usage is assiduously monitoring and auditing usage looking for orphaned, underutilized or over-speced resources. Although monitoring and inventory of virtual infrastructure is a niche targeted by a number of third-party products, for AWS users the easiest (and cheapest) place to start is with Trusted Advisor, a software wizard that automatically checks one’s AWS portfolio and provides advice in four areas: cost optimization, security, fault tolerance and performance.

Among its cost optimization features, Trusted Advisor can find EC2 instances with low CPU utilization, idle load balancers and RDS databases, underutilized EBS volumes and orphaned IP addresses. It also checks historical EC2 usage to identify candidates with steady workloads appropriate for reserved instances, which is key to exploiting another of AWS’s pricing strategies: lower rates for pre-committed, contracted usage.

Pay Now and Save Later

Like any contractual subscription with tiered pricing, whether a cell phone data plan or cloud sync and share storage service, reserved instances (RIs) require understanding one’s usage and buying just enough to cover the need without either overspending or hitting a usage cap. On AWS, this means a hybrid deployment strategy can often save money, with RIs used for baseline, steady-state workloads and on-demand or spot instances to cover usage spikes. Spot instances, which often go for 10-15% the cost of on-demand equivalents, are particularly appropriate for batch jobs or time-insensitive workloads. Note that RIs aren’t limited to EC2 but are also available for databases (RDS, DynamoDB, ElastiCache) and CDN (CloudFront).

RIs can be purchased three ways: with monthly installments, partial or all payment up-front. Note however that the price difference between partial and all up-front isn’t that great, so a hybrid up-front/monthly payment schedule typically provides the best balance between capital commitment and price discount. For example, recent pricing for m4.large EC2 instances showed just a 1% difference between the partial and all up-front discount for a one-year term.

AWS Reserved Instance Pricing

AWS Reserved Instance Pricing

Although RIs do lock buyers into a contractual one- or three-year commitment, you still have some flexibility. RIs can be redeployed to other apps, moved to another zone within the same region and (for Linux/Unix instances) even resized all without charge. Buyers finding that they no longer need RIs can also resell them in the AWS Marketplace.

Cost-effective Cloud Architecture

Designing cloud-savvy infrastructure is the most effective way to minimize AWS costs over the long haul. Given the variety of AWS services, there is generally more than one way to achieve the same thing and the most obvious solution of deploying another EC2 instance or S3 bucket, isn’t alway the best given the rich variety higher-level AWS services. Cost optimization techniques include:

  • Using S3 and CloudFront for content caching to offload the handling of static content in applications like WordPress from EC2.
  • Using load balancing (ELB) and auto-scaling to reduce the average number of EC2 instances by only bursting capacity when needed.
  • Using AWS managed services instead of self-managed equivalents on generic EC2 instances. Examples include replacing things like Rabbit MQ with SQS (message queuing), Exchange or Sendmail with Simple Email Service, a NoSQL cluster with DynamoDB, Memcached with ElastiCache and a media encoder like HandBrake with the Elastic Transcoder service.
  • Properly sizing EC2 instances by selecting the right mix of CPU power and memory from the dozens of EC2 instance types.
  • Using Reduced Redundancy Storage (RRS) and Glacier for derived copies (non-originals) of data, log files, archives and anything that doesn’t require 100% uptime. Note however that Glacier should only be used for truly archival data since recovery takes hours, not minutes.
  • Scaling out databases with read-only replicas on ElastiCache instead of new RDS instances.
  • Using the AWS free tier for small, short-term dev/test projects and only moving a paid tier if and when the application goes to production and demonstrates significant customer demand.

Conclusion

No one wants to waste money on AWS, so cost management and wise usage is imperative, however it’s important to remember that running in the cloud isn’t really a cost minimization exercise for most organizations, it’s about increasing IT and developer agility and innovation. Still, by designing infrastructure for AWS, using a mix of reserved, on-demand and spot instances of the right size and assiduously pruning orphaned or underutilized resources organizations can invariably cut their AWS bill.