Analyzing Embedded Metric Format (EMF) Logs
1. Timeseries: EMF Log Events vs All Log Events (5-minute)
Generates a graph comparing EMF logs against the total log volume over time. A 5-minute period bin(5m) is applied. Use bin(1m), bin(1h), or bin(1d) for different granularities.
stats sum(@message like "_aws" and @message like "CloudWatchMetrics") as EMF_Log_Events, count(*) as Total_Log_Events by bin(5m)
2. Percentage of EMF Logs
Calculates the percentage of log events that are EMF-formatted relative to the total log volume. Useful for understanding how much of your log data is generating metrics.
stats
count(*) as Total_Log_Events,
sum(@message like "_aws" and @message like "CloudWatchMetrics") as EMF_Log_Events,
(sum(@message like "_aws" and @message like "CloudWatchMetrics") / count(*)) * 100 as EMF_Percentage
3. Summarize EMF Logs by Dimension
Breaks down EMF events by their dimension values to show how metrics are being derived. Returns the unique dimension combinations generating metrics.
filter @message like "_aws" and @message like "CloudWatchMetrics"
| stats
count(*) as eventCount
by Environment, uri, method, status
| sort eventCount desc
| Environment | uri | method | status | eventCount |
|---|---|---|---|---|
| prod | /health | GET | 200 | 2 |
| prod | UNKNOWN | GET | 200 | 1 |
| prod | /store | POST | 200 | 1 |
4. Summarize Per-Dimension Cardinality
Returns the total unique values for each dimension independently. Illustrates the cardinality contribution of each dimension to help identify high-cardinality culprits.
filter @message like "_aws" and @message like "CloudWatchMetrics"
| stats
count_distinct(uri) as uniqueUris,
count_distinct(method) as uniqueMethods,
count_distinct(status) as uniqueStatuses,
count_distinct(Environment) as uniqueEnvironments,
count(*) as totalEMFEvents
| uniqueUris | uniqueMethods | uniqueStatuses | uniqueEnvironments | totalEMFEvents |
|---|---|---|---|---|
| 4 | 2 | 1 | 1 | 6 |
5. Summary by URI
Breaks down EMF events by uri dimension, highlighting which endpoints generate the most metric data.
filter @message like "_aws" and @message like "CloudWatchMetrics"
| stats count(*) as eventCount by uri
| sort eventCount desc
6. Summary by URI: /health
Calculates the percentage of EMF logs where uri = "/health" versus all other endpoints. Identifies whether health checks are a major contributor to metric volume.
# Health Check EMF Percentage
filter @message like "_aws" and @message like "CloudWatchMetrics"
| stats
count(*) as totalEMFEvents,
sum(uri = "/health") as healthEvents,
(sum(uri = "/health") / count(*)) * 100 as healthPercentage,
sum(uri != "/health") as nonHealthEvents,
(sum(uri != "/health") / count(*)) * 100 as nonHealthPercentage
7. Timeseries: EMF Health Check Events (5-minute)
Generates a graph of EMF logs with uri = "/health" received every 5 minutes. The granularity can be modified using bin(1m) or bin(1h).
filter @message like "_aws" and @message like "CloudWatchMetrics" and uri = "/health"
| stats count(*) as healthEMFEvents by bin(5m)
| sort @timestamp desc
8. Filter for EMF Logs
Returns EMF log events exclusively for export or further investigation.
filter @message like "_aws" and @message like "CloudWatchMetrics"
Usage Notes
- These queries use the CloudWatch Logs Insights query language (not SQL mode).
- Replace dimension names (
Environment,uri,method,status) with the dimensions defined in your EMF log events. - The EMF filter pattern
@message like "_aws" and @message like "CloudWatchMetrics"identifies EMF-formatted log events. - The
bin()function in timeseries queries can be adjusted:bin(1m),bin(5m),bin(1h), orbin(1d). - Results can be visualized directly in a CloudWatch Dashboard using Log Insights widgets.