Monitoring EC2 Applications

1. Metric Insights Alarms ▾

EC2 workloads can have numerous resources (e.g. 1000+ instances). This can present challenges when:

  • monitoring large clusters
  • monitoring a subset of instances which may not be part of an Autoscaling Group
  • managing alarms for Autoscaling Group with frequent scaling actions

CloudWatch Alarms based on Metric Insights will automatically update the resources being monitored by the alarm as resources change. This reduces the administrative effort to ensure CloudWatch Alarms are monitoring active resources.

Here are references for creating alarms for AWS Services:

2. Sample Metric Insights Queries ▸

3. Alarms based on CloudWatch Agent Metrics ▾

  • If the EC2 instances are using the CloudWatch Agent to publish OS and application metrics, corresponding queries can be written.
  • Below returns the Memory % Committed Bytes In Use for all instances in the Autoscaling group.

3.1 CPU Utilization > 80%

CloudWatch Alarm Console
Fig 1: Metric Insights Alarm listing the individual EC2 Instances being monitored
SQL
SELECT AVG("Processor % Processor Time")
FROM SCHEMA(IIS_Web_Server, InstanceId)
GROUP BY InstanceId
ORDER BY AVG() DESC
LIMIT 10

3.2 Memory Utilization > 80%

SQL
SELECT AVG("Memory % Committed Bytes In Use")
FROM SCHEMA(IIS_Web_Server, AutoScalingGroupName,ImageId,InstanceId,InstanceType,objectname)
GROUP BY InstanceId
ORDER BY AVG() DESC
LIMIT 10

3.3 Disk Utilization < 5GB

  • Monitoring disk utilization on a specific disk
SQL
SELECT AVG("LogicalDisk Free Megabytes")
FROM SCHEMA(IIS_Web_Server, AutoScalingGroupName,ImageId,InstanceId,InstanceType,instance,objectname)
WHERE instance = 'C:'
GROUP BY InstanceId
ORDER BY AVG() DESC
LIMIT 10

3.4 IIS Process DOWN

SQL
SELECT MIN("procstat_lookup pid_count")
FROM SCHEMA(IIS_Web_Server, AutoScalingGroupName, ImageId, InstanceId, InstanceType, exe, pid_finder)
WHERE exe = 'w3wp'
GROUP BY InstanceId
ORDER BY MIN() ASC
LIMIT 10

4. Alarm Notification ▸

5. Composite & Metric Alarms ▾

A deployment may include both Composite and Metric Alarms.

In this way, the DevOps Team has the flexibility to determine:

  • which alarms to notification on
  • which combination of alarms warrant a page

The figure below includes alarms for EC2 web application as well as the load balancer fronting it.

Fig 2: CloudWatch Alarm Hierarchy: Composite & Metric Alarms
CloudWatch Alarms Console: Composite Alarms
Fig 3: CloudWatch Alarms Console: Composite Alarms