Skip to content
ANNOUCEMENT Announcing self-paced Karpenter Lab series - START NOW
Identify any Amazon EC2 instances that appear to be idle and stop or terminate them to help lower the cost of your monthly AWS bill. By default, an EC2 instance is considered ‘idle’ when meets the following criteria (to declare the instance ‘idle’ both conditions must be true):
It is important that your EC2 instances are tagged with correct tags which provide visibility into their usage profile and help you decide whether it’s safe or not to stop or terminate these resources. For Example, knowing the role and the owner of an EC2 instance before you take the decision to stop/terminate it is very important and can avoid unwanted termination of actually used workloads.
This rule can help you with the following compliance standards:
This rule can also help you work with the AWS Well-Architected Framework.
Idle instances represent a good candidate to reduce your monthly AWS costs and avoid accumulating unnecessary EC2 usage charges.
To identify any idle EC2 instances currently available in your AWS account, perform the following:
1. Sign in to the AWS Management Console.
2. Navigate to EC2 dashboard at https://console.aws.amazon.com/ec2/.
3. In the left navigation panel, under INSTANCES section, choose Instances.
4. Select the EC2 instance that you want to examine.
5. Select the Monitoring tab from the dashboard bottom panel.
6. Within the CloudWatch metrics section, perform the following actions:
a. Expand CPU Utilization (Percent) usage graph thumbnail to open the instance CPU usage details box.
Inside the CloudWatch Monitoring Details dialog box, set the following parameters:
Once the monitoring data is loaded, verify the instance CPU usage for the last 7 days. If the average usage (percent) has been less than 2%, e.g., the selected EC2 instance qualifies as candidate for an idle instance. Click Close to return to the dashboard.
b. Expand Network In (bytes) usage graph thumbnail to open the instance network usage details box.
Inside the CloudWatch Monitoring Details dialog box, set the following parameters:
Once the monitoring data is loaded, verify the incoming network traffic for the last 7 days. If the average traffic has been less than 5 MB, e.g., the selected EC2 instance qualifies as candidate for an idle instance. Click Close to exit.
c. Expand Network Out (Percent) usage graph thumbnail to open the instance network usage details box.
Inside the CloudWatch Monitoring Details dialog box, set the following parameters:
Once the monitoring data is loaded, verify the incoming network traffic for the last 7 days. If the average traffic has been less than 5 MB, e.g., the selected EC2 instance qualifies as candidate for an idle instance. Click Close to exit.
7. Now examine the tags on the instance to determine if the Instance can be stopped or terminated. To check for the necessary tags, perform the following.
a. Select the Tags tab from the dashboard bottom panel.
b. Verify the requested tags and their values.
8. If all conditions outlined at step no. 6 (a, b + c) and 7 are met, the selected EC2 instance is considered “idle” and can be shut down or terminated in order to stop incurring charges for this resource.
9. Repeat steps no. 4 – 7 to verify the CPU usage, the Network In/Out traffic and the role for the rest of the EC2 instances provisioned in the current region.
10. Change the AWS region from the navigation bar and repeat the audit process for the other regions.
1. Run describe-instances command (OSX/Linux/UNIX) using appropriate filtering to list the IDs of all the existing EC2 instances available in the selected region:
aws ec2 describe-instances \\
--region us-east-1 \\
--output table \\
--query 'Reservations[*].Instances[*].InstanceId'
2. The command output should return a table with the requested instance IDs:
-------------------------
| DescribeInstances |
+-----------------------+
| i-0e2131e74da4fb0a2 |
| i-06badd7b814657cd2 |
| i-0debafbfb2d39a7ca |
| i-05134e709061a2399 |
| i-0898d6b29959828c9 |
| i-0354a63317fd7db5b |
| i-0908eedce41f43f1c |
| i-06c01f0780dfe3d1f |
| i-01c3d73007eda95ee |
| i-086ecb2e166a10cba |
3. Run get-metric-statistics command (OSX/Linux/UNIX) to get the statistics recorded by AWS CloudWatch for the CPUUtilization metric representing the CPU usage of the selected EC2 instance.
a. Change the –start-time (start recording date) and –end-time (stop recording date) parameters value to choose your own time frame for recording the resource CPU usage.
b. Set the –period parameter value based on your requirements to define the granularity – in seconds – of the returned data points. A period can be as short as one minute (60 seconds) or as long as one day (86400 seconds).
The following command example returns the average CPU usage of an EC2 instance identified by the ID i-0e2131e74da4fb0a2, usage data captured during a 7 days period (set by the –start-time and –end-time command parameters), using 1 hour period as the granularity of the returned data points (set by the –period parameter):
aws cloudwatch get-metric-statistics \\
--region us-east-1 \\
--metric-name CPUUtilization \\
--start-time 2021-07-31T22:30:00 \\
--end-time 2021-08-06T22:30:00\\
--period 3600 \\
--namespace AWS/EC2 \\
--statistics Average \\
--dimensions Name=InstanceId,Value=**i-0e2131e74da4fb0a2**
4. The command output should return the CPU usage details requested:
{
"Label": "CPUUtilization",
"Datapoints": [
{
"Timestamp": "2021-08-02T00:30:00+00:00",
"Average": 0.2394444444444444,
"Unit": "Percent"
},
{
"Timestamp": "2021-08-05T07:30:00+00:00",
"Average": 0.2395833333333333,
"Unit": "Percent"
},
{
"Timestamp": "2021-08-03T09:30:00+00:00",
"Average": 0.24513889352307097,
"Unit": "Percent"
},
...
{
"Timestamp": "2021-08-06T16:30:00+00:00",
"Average": 0.23361111111111107,
"Unit": "Percent"
},
{
"Timestamp": "2021-08-05T20:30:00+00:00",
"Average": 0.23680556018966048,
"Unit": "Percent"
},
{
"Timestamp": "2021-08-04T11:30:00+00:00",
"Average": 0.2380555555555555,
"Unit": "Percent"
}
]
}
If the average CPU usage data returned is less than 2%, the selected EC2 instance qualifies as candidate for the idle instance.
5. Now query the NetworkIn metric in a similar manner by changing the —metric-name to NetworkIn :
aws cloudwatch get-metric-statistics \\
--region us-east-1 \\
--metric-name NetworkIn \\
--start-time 2021-07-31T22:30:00 \\
--end-time 2021-08-06T22:30:00\\
--period 3600 \\
--namespace AWS/EC2 \\
--statistics Average \\
--dimensions Name=InstanceId,Value=**i-0e2131e74da4fb0a2**
6. The command output should return the instance network traffic data requested:
{
"Label": "NetworkIn",
"Datapoints": [
{
"Timestamp": "2021-08-04T05:30:00+00:00",
"Average": 10.5,
"Unit": "Bytes"
},
{
"Timestamp": "2021-08-02T07:30:00+00:00",
"Average": 9.0,
"Unit": "Bytes"
},
{
"Timestamp": "2021-08-01T18:30:00+00:00",
"Average": 82.5,
"Unit": "Bytes"
},
{
"Timestamp": "2021-08-05T01:30:00+00:00",
"Average": 170.21666666666667,
"Unit": "Bytes"
},
...
{
"Timestamp": "2021-08-03T22:30:00+00:00",
"Average": 12.0,
"Unit": "Bytes"
},
{
"Timestamp": "2021-08-02T00:30:00+00:00",
"Average": 12.0,
"Unit": "Bytes"
},
{
"Timestamp": "2021-08-05T07:30:00+00:00",
"Average": 22.5,
"Unit": "Bytes"
}
]
}
If the average incoming traffic values returned are less than 5 MB, the selected EC2 instance qualifies as candidate for the idle instance.
7. Now query the Networkout metric in a similar manner by changing the —metric-name to NetworkOut :
aws cloudwatch get-metric-statistics \\
--region us-east-1 \\
--metric-name NetworkOut \\
--start-time 2021-07-31T22:30:00 \\
--end-time 2021-08-06T22:30:00\\
--period 3600 \\
--namespace AWS/EC2 \\
--statistics Average \\
--dimensions Name=InstanceId,Value=i-0e2131e74da4fb0a2
8. The command output should return the instance network traffic data requested:
{
"Label": "NetworkOut",
"Datapoints": [
{
"Timestamp": "2021-08-04T05:30:00+00:00",
"Average": 24.0,
"Unit": "Bytes"
},
{
"Timestamp": "2021-08-02T07:30:00+00:00",
"Average": 22.5,
"Unit": "Bytes"
},
{
"Timestamp": "2021-08-05T14:30:00+00:00",
"Average": 55.5,
"Unit": "Bytes"
},
...
{
"Timestamp": "2021-08-03T16:30:00+00:00",
"Average": 49.5,
"Unit": "Bytes"
},
{
"Timestamp": "2021-08-01T04:30:00+00:00",
"Average": 25.5,
"Unit": "Bytes"
},
{
"Timestamp": "2021-08-01T18:30:00+00:00",
"Average": 97.5,
"Unit": "Bytes"
}
]
}
If the average outgoing traffic values returned are less than 5 MB, the selected EC2 instance qualifies as candidate for the idle instance.
9. Run describe-tags command (OSX/Linux/UNIX) to describe the tags for the selected EC2 instance.
aws ec2 describe-tags \\
--region us-east-1 \\
--filters "Name=resource-id,Values=i-0e2131e74da4fb0a2"
10. The command output should return the tags (key-value pairs) applied to the instance. These tags can be used to determine the resource role within the application stack and to contact its owner for more information in order to decide whether the EC2 instance can be shutdown/terminated or not:
{
"Tags": [
{
"Key": "Name",
"ResourceId": "i-0e2131e74da4fb0a2",
"ResourceType": "instance",
"Value": "siq_jenkins-siq-jenkins-dev"
},
{
"Key": "aws:autoscaling:groupName",
"ResourceId": "i-0e2131e74da4fb0a2",
"ResourceType": "instance",
"Value": "siq_jenkins-siq-jenkins-dev"
}
]
}
If the data returned for the steps no. 3 – 10 satisfy the conditions set by the conformity rule (instance role, instance owner, CPU and Network In + Network Out usage), the selected EC2 instance is considered “idle” and can be stopped or terminated in order to reduce AWS EC2 usage costs.
11. Repeat steps no. 3 – 10 to verify the CPU usage, the Network In/Out traffic and the role for the rest of the EC2 instances provisioned in the current region.
12. Change the AWS region by updating the –region command parameter value and repeat steps no. 1 – 11 to perform the audit process for other regions.
To shutdown/terminate any AWS EC2 instances that are currently running in idle mode, perform the following commands:
1. Sign in to the AWS Management Console.
2. Navigate to EC2 dashboard at https://console.aws.amazon.com/ec2/.
3. In the left navigation panel, under INSTANCES section, choose Instances.
4. Select the idle EC2 instance that you want to stop or terminate (see Audit section part I to identify the right resource).
5. Based on the instance attachment to an AWS Auto Scaling Group (ASG), choose one of the following sets of instructions:
a. To stop/terminate an EC2 instance that is not running within an AWS ASG, click the Actions dropdown button from the dashboard top menu, select Instance State and choose one of the following options:
I. To stop the instance click Stop. In the Stop Instances dialog box, review the instance details then click Yes, Stop to confirm your action. The resource status should change to stopping and then to stopped as the shutdown process progress.
II. To terminate the instance click Terminate. In the Terminate Instances dialog box, review the instance details then click Yes, Terminate to confirm your action. The resource status should change to shutting-down and then to terminated as the removal process progress.
b. To stop/terminate an EC2 instance that is currently running within an AWS ASG perform the following:
I. If In the navigation panel, under AUTO SCALING, choose Auto Scaling Groups.
II. Select the ASG that holds the idle EC2 instance that you want to stop/terminate.
III. Choose the Instances tab from the dashboard bottom panel and select the right instance.
IV. Click on the Actions dropdown button available on the panel then select Detach.
V. In the Detach Instance dialog box, uncheck Add a new instance to the Auto Scaling group to balance the load to decrement the ASG desired capacity then click Detach Instance to confirm the action.
VI. Once the idle instance is detached from your ASG you will be able to stop or terminate it without triggering a Scale In event for replacing the instance. Now that the EC2 instance is no longer running within an AWS ASG, follow the instructions outlined at step 5.a. (AWS Console) to stop/terminate the resource.
6. Repeat step no. 4 and 5 to stop/terminate any other idle EC2 instances provisioned within the current region.
7. Change the AWS region from the navigation bar and repeat the process for other regions.
1. Based on the EC2 instance attachment to an AWS Auto Scaling Group, choose one of the following sets of commands:
a. To stop/terminate an EC2 instance that is not running within an AWS ASG, perform the following:
aws ec2 stop-instances \\
--region us-east-1 \\
--instance-ids i-0e2131e74da4fb0a2
{
"StoppingInstances": [
{
"InstanceId": "i-0e2131e74da4fb0a2",
"CurrentState": {
"Code": 64,
"Name": "stopping"
},
"PreviousState": {
"Code": 16,
"Name": "running"
}
}
]
}
aws ec2 terminate-instances \\
--region us-east-1 \\
--instance-ids i-0e2131e74da4fb0a2
{
"TerminatingInstances": [
{
"InstanceId": "i-0e2131e74da4fb0a2",
"CurrentState": {
"Code": 32,
"Name": "shutting-down"
},
"PreviousState": {
"Code": 16,
"Name": "running"
}
}
]
}
b. To stop/terminate an EC2 instance that is currently running within an AWS ASG perform the following:
aws autoscaling describe-auto-scaling-instances \\
--region us-east-1 \\
--instance-ids i-0e2131e74da4fb0a2 \\
--query 'AutoScalingInstances[*].AutoScalingGroupName'
[
"siq_jenkins-siq-jenkins-dev"
]
aws autoscaling detach-instances \\
--region us-east-1 \\
--instance-ids i-0e2131e74da4fb0a2 \\
--auto-scaling-group-name "siq_jenkins-siq-jenkins-dev" \\
--should-decrement-desired-capacity
2. Repeat step no. 1 to stop/terminate any other idle EC2 instances provisioned within the current region.
3. Change the AWS region by updating the –region command parameter value and repeat the entire process for other regions.
Come see why we are the #1 cloud management platform and why companies like Uber, Dickey’s BBQ Pit and Norwegian Cruise Line trust nOps to manage their cloud.
nOps is an AI-powered FinOps platform that makes it easy manage and optimize your cloud usage, commitments, and spend.
© nOps 2024. All Rights Reserved.