Skip to content
ANNOUCEMENT Announcing self-paced Karpenter Lab series - START NOW
Identify any Amazon EC2 instances that appear to be under-utilised and downsize (resize) them to help lower the cost of your monthly AWS bill. By default, an EC2 instance is considered under-utilised when matches the following criteria (to declare the instance under-utilised both conditions must be met):
By default, AWS CloudWatch doesn’t capture an EC2 instance memory utilisation because the necessary metric cannot be implemented at the hypervisor level. In order to report the memory utilisation using CloudWatch you need to install an agent (script) on the instance and create a custom metric (let’s name it EC2MemoryUtilization) on the AWS CloudWatch dashboard. The instructions required for installing the monitoring agent depends on the Operating System used by the instance. Please refer to this URL for more details.
This rule can help you with the following compliance standards:
This rule can help you work with the AWS Well-Architected Framework
Downsizing under-utilised EC2 instances to meet the capacity needs at the lowest cost represents an efficient strategy to reduce your monthly AWS costs. For example, resizing a c5.xlarge instance provisioned in the US-East (N. Virginia) region to a c5.large-type instance due to CPU and memory underuse, you can roughly reduce your AWS costs by half.
1. Sign in to the AWS Management Console.
2. Navigate to EC2 dashboard at https://console.aws.amazon.com/ec2/.
3. In the navigation panel, under INSTANCES section, choose Instances.
4. Select the EC2 instance that you want to examine.
5. Select the Monitoring tab from the dashboard bottom panel.
6. Within the CloudWatch metrics section, perform the following actions:
a. Expand the CPU Utilisation (Percent) usage graph thumbnail to open the instance CPU usage details box. Inside the CloudWatch Monitoring Details dialog box, set the following parameters:
b. Once the monitoring data is loaded, verify the instance CPU usage for the last 7 days. If the average usage (percent) has been less than 30%, e.g.
7. Now determine the EC2 instance memory utilisation by reading the EC2MemoryUtilization metric data reported by the CloudWatch agent (Perl script) installed on the selected EC2 instance (this rule assumes that the script has been successfully installed and it has returned memory usage data in the past 7 days). To verify the instance memory usage reported by the custom CloudWatch metric, perform the following actions:
a. Navigate to Cloudwatch dashboard at https://console.aws.amazon.com/cloudwatch/.
b. In the navigation panel, select Metrics to access your existing Cloudwatch metrics.
c. Choose All metrics tab from the dashboard bottom panel, click Linux System then select InstanceId to list any custom metrics installed on your EC2 instances.
d. Select the right EC2 instance from the list (see Audit section part I, step no. 4), click the Action dropdown button from the dashboard top-right menu then choose Add to dashboard option.
e. On Add to dashboard dialog box, perform the following:
f. Click the Custom dropdown button from the dashboard top-right menu, select Relative then choose the 1 Weeks option to return the data recorded in the past week.
g. Once the monitoring data is available within the widget, verify the instance memory usage for the last 7 days. If the average usage (percent) has been less than 30%, e.g.
If the rule conditions are met, based on the usage data outlined at step no. 6 and 7, the selected AWS EC2 instance is considered underutilised and can be safely downsized in order to stop incurring charges for the EC2 compute resources that you don’t use.
8. Repeat steps no. 4 – 7 to verify the CPU and memory usage data available in the last 7 days for the rest of the EC2 instances provisioned in the current region.
9. Change the AWS region from the navigation bar and repeat the audit process for other regions.
1. Run describe-instances command (OSX/Linux/UNIX) using necessary filtering to list the IDs of all active (running) EC2 instances provisioned in the selected region:
aws ec2 describe-instances \\
--region us-east-1 \\
--filters Name=instance-state-name,Values=running \\
--output table \\
--query 'Reservations[*].Instances[*].InstanceId'2. The command output should return a table with the requested instance IDs:
[
"i-0e2131e74da4fb0a2",
"i-06badd7b814657cd2",
"i-0898d6b29959828c9",
"i-0354a63317fd7db5b",
"i-0908eedce41f43f1c",
"i-06c01f0780dfe3d1f",
...
"i-0e735c3d186a476b9",
"i-0284674295b30fa02",
"i-04096936dff888674",
"i-020526d484b07cbf2",
"i-084871ed8ee1d3034",
"i-05552ffac630881ae"
]3. Run get-metric-statistics command (OSX/Linux/UNIX) to get the statistics recorded by AWS CloudWatch for the CPUUtilization metric representing the CPU usage of the selected EC2 instance.
The following command example returns the average CPU utilization for an EC2 instance identified by the ID i-0e2131e74da4fb0a2, usage data captured during a 7-day time frame, using a time interval of 1 hour as the granularity for the returned datapoints:
aws cloudwatch get-metric-statistics \\
--region us-east-1 \\
--metric-name CPUUtilization \\
--start-time 2021-08-10T17:00:00 \\
--end-time 2021-08-17T17:00:00 \\
--period 3600 \\
--namespace AWS/EC2 \\
--statistics Average \\
--dimensions Name=InstanceId,Value=i-0e2131e74da4fb0a24. The command output should provide the CPU Usage details as shown below
{
"Label": "CPUUtilization",
"Datapoints": [
{
"Timestamp": "2021-08-13T13:00:00+00:00",
"Average": 0.27319447917781636,
"Unit": "Percent"
},
...
{
"Timestamp": "2021-08-12T20:00:00+00:00",
"Average": 0.2344444467613811,
"Unit": "Percent"
},
{
"Timestamp": "2021-08-16T03:00:00+00:00",
"Average": 0.23875008798240735,
"Unit": "Percent"
}
]
}If the average CPU usage data returned is less than 30%, the selected EC2 instance qualifies as candidate for the underused instance.
5. Determine the EC2 instance memory usage by querying the EC2MemoryUtilization metric data (or whatever name you have used for your custom metric) reported by the CloudWatch script installed on the selected EC2 instance (this rule assumes that the script has been successfully installed and it has recorded memory usage data within the past 7 days).
Run get-metric-statistics command (OSX/Linux/UNIX) using the metric name as identifier.
The following command example returns the average memory utilization for an EC2 instance identified by the ID i-0e2131e74da4fb0a2, from the usage data captured by a CloudWatch metric named EC2MemoryUtilization, during a 7-day time frame, using a time interval of 1 hour as the granularity for the returned data points:
aws cloudwatch get-metric-statistics \\
--region us-east-1 \\
--metric-name EC2MemoryUtilization \\
--start-time 2021-08-10T17:00:00 \\
--end-time 2021-08-17T17:00:00 \\
--period 3600 \\
--namespace AWS/EC2 \\
--statistics Average \\
--dimensions Name=InstanceId,Value=i-0e2131e74da4fb0a26. The command output should return the memory usage details requested:
{
"Label": "EC2MemoryUtilization",
"Datapoints": [
{
"Timestamp": "2021-08-13T13:00:00+00:00",
"Average": 5.2085,
"Unit": "Percent"
},
...
{
"Timestamp": "2021-08-12T20:00:00+00:00",
"Average": 6.382,
"Unit": "Percent"
},
{
"Timestamp": "2021-08-16T03:00:00+00:00",
"Average": 7.376,
"Unit": "Percent"
}
]
}If the average memory utilization data returned is less than 30%, the selected EC2 instance qualifies as candidate for the underused instance.
If the usage data returned at steps no. 3 – 6 satisfy the conditions set by the rule (i.e. average CPU and memory usage less than 30%), the selected EC2 instance is considered “underutilized” and should be downsized in order to reduce your AWS EC2 usage costs.
7. Repeat steps no. 3 – 6 to verify the required CPU and memory usage data for the rest of the EC2 instances available within the current region.
8. Change the AWS region by updating the –region command parameter value and repeat steps no. 1 – 7 to perform the entire audit process for other regions.
Option 1: Downsize (resize) the underused EC2 instances provisioned within your AWS account.
To resize any EC2 instance that is currently running in “underutilized” mode, perform the following commands:
The following process assumes that the EC2 instances selected for downsize are NOT currently used in production or for critical operations. To resize production instances without any downtime, you should create a snapshot of your current image and launch a new instance from that snapshot using the required instance type.
1. Sign in to the AWS Management Console.
2. Navigate to EC2 dashboard at https://console.aws.amazon.com/ec2/.
3. In the navigation panel, under INSTANCES section, choose Instances.
4. Select the underused EC2 instance that you want to resize (see Audit section part I to identify the right resource).
5. Click Actions button from the dashboard top menu, select Instance State, then select Stop.
6. Inside the Stop Instances dialog box, review the action details and click Yes, Stop to confirm the action.
7. Once the instance is stopped (i.e. Instance State set to stopped), use again the Actions button from the dashboard top menu, select Instance Settings, then select Change Instance Type.
8. In the Change Instance Type dialog box, perform the following:
a. From the Instance Type dropdown list, select the instance type to downsize to (e.g. c4.large – see EC2 Instance Types page available at this URL to help you choose the right instance type).
b. (Optional) Select EBS-optimized to enable EBS optimization or deselect EBS-optimized to disable EBS optimization. This feature provides dedicated throughput to your AWS EBS volumes for best I/O performance (additional charges apply).
c. Click Apply to resize the selected instance.
9. Now click Actions button from the dashboard top menu, select Instance State, then select Start.
10. In the Start Instances dialog box, click Yes, Start to restart the instance. Once the booting process is complete, the EC2 instance status should change from pending to running (this may take few minutes).
11. Repeat steps no. 4 – 10 to downsize (resize) any other underutilized EC2 instances provisioned in the current region.
12. Change the AWS region from the navigation bar and repeat the remediation process for other regions.
1. Run stop-instances command (OSX/Linux/UNIX) using the resource ID as identifier to stop the underused EC2 instance that you want to resize (see Audit section part II to identify the right instance):
aws ec2 stop-instances \\
--region us-east-1 \\
--instance-ids i-0e2131e74da4fb0a22. The command output should return the stop request metadata:
{
"StoppingInstances": [
{
"InstanceId": "i-0e2131e74da4fb0a2",
"CurrentState": {
"Code": 64,
"Name": "stopping"
},
"PreviousState": {
"Code": 16,
"Name": "running"
}
}
]
}3. Once the instance is stopped (should take one minute), run modify-instance-attribute command (OSX/Linux/UNIX) to resize the selected EC2 instance to the desired type. The following command example downsize an EC2 instance identified by the IDi-0e2131e74da4fb0a2by resizing it from ac5.xlargeinstance to ac5.large` instance. If successful, no output is returned:
aws ec2 modify-instance-attribute \\
--region us-east-1 \\
--instance-id i-0e2131e74da4fb0a2 \\
--instance-type "{\\"Value\\": \\"c5.large\\"}"4. Run start-instances command (OSX/Linux/UNIX) to restart the EC2 instance resized at the previous step (it may take few minutes until the instance enters the running state):
aws ec2 start-instances \\
--region us-east-1 \\
--instance-ids i-0e2131e74da4fb0a25. The command output should return the start request information:
{
"StartingInstances": [
{
"InstanceId": "i-0e2131e74da4fb0a2",
"CurrentState": {
"Code": 0,
"Name": "pending"
},
"PreviousState": {
"Code": 80,
"Name": "stopped"
}
}
]
}6. Repeat steps no. 1 – 5 to downsize (resize) any other underused EC2 instances available within the current region.
7. Change the AWS region by updating the –region command parameter value and repeat the entire process for other regions.
Option 2: Override the rule check.
If the selected underused EC2 instance configuration must remain unchanged (some workload scenarios can result in low resource utilization by design), you should override the nOps rule check for the specified instance from the nOps console.
nOps is an AI-powered FinOps platform that makes it easy manage and optimize your cloud usage, commitments, and spend.
© nOps 2024. All Rights Reserved.