Analyzing EventBridge Logs

1. Filter for ERROR & INVOCATION_ATTEMPT_THROTTLE

Returns error events with INVOCATION_ATTEMPT_THROTTLE message type, showing the rule, target, AWS service error, HTTP status code, and error message.

SQL
--Filter on ERROR & INVOCATION_ATTEMPT_THROTTLE
SELECT event_bus_name,
       log_level,
       message_type,
       `details.invocation_status`,
       `details.rule_arn` AS Rule_ARN,
       `details.target_arn` AS Target_ARN,
       `error.aws_service`,
       `error.http_status_code`,
       `error.error_message`,
       request_id
FROM `LogGroup`
WHERE log_level = 'ERROR'
  AND message_type = 'INVOCATION_ATTEMPT_THROTTLE'

2. Filter for a Specific EventBridge Rule

Returns all log events for a specific rule ARN. Useful for investigating why a particular rule is failing or being throttled.

SQL
--Filter for a Specific EventBridge Rule
SELECT `@timestamp`,
       message_type,
       `details.invocation_status` AS Invocation_Status,
       `details.rule_arn` AS Rule_ARN,
       `details.target_arn` AS Target_ARN,
       `error.aws_service`,
       `error.http_status_code`,
       `error.error_message`,
       request_id
FROM `LogGroup`
WHERE `details.rule_arn` LIKE '%rule/SSM-Automation%'
ORDER BY `@timestamp` DESC

3. Filter for a Specific Target

Filters error events for a specific target ARN. Helps isolate issues with a particular Lambda function, SNS topic, or other target.

SQL
--Filter for a Specific Target
SELECT `@timestamp`,
       message_type,
       `details.invocation_status` AS Invocation_Status,
       `details.rule_arn` AS Rule_ARN,
       `details.target_arn` AS Target_ARN,
       `error.aws_service`,
       `error.http_status_code`,
       `error.error_message`,
       request_id
FROM `LogGroup`
WHERE `details.target_arn` LIKE '%function:Wexim_HTTP_404_Log_Analysis'
  AND log_level = 'ERROR'
ORDER BY `@timestamp` DESC

4. Filter for an EventBridge Request ID

Traces a single event through the system using its request ID. Shows all log entries related to one invocation attempt.

SQL
--Filter for an EventBridge Request ID
SELECT `@timestamp`,
       message_type,
       `details.invocation_status` AS Invocation_Status,
       `details.rule_arn` AS Rule_ARN,
       `details.target_arn` AS Target_ARN,
       `error.aws_service`,
       `error.http_status_code`,
       `error.error_message`,
       request_id
FROM `LogGroup`
WHERE request_id = 'ccb4f960-fc18-4db0-afc2-bad457712c9c'
ORDER BY `@timestamp` DESC

5. Filter for Errors with HTTP 400 or 403 Response

Surfaces invocation errors where the target returned a 400 (bad request) or 403 (forbidden) HTTP status. Often indicates permission or payload issues.

SQL
--Filter for Errors with HTTP 400 or 403 Response
SELECT `@timestamp`,
       log_level,
       message_type,
       `details.invocation_status` AS Invocation_Status,
       `details.rule_arn` AS Rule_ARN,
       `details.target_arn` AS Target_ARN,
       `error.aws_service`,
       `error.http_status_code`,
       `error.error_message`,
       request_id
FROM `LogGroup`
WHERE `error.http_status_code` IN (400, 403)
ORDER BY `@timestamp` DESC

6. Filter for INVOCATION_ATTEMPT_PERMANENT_FAILURE

Returns events where the invocation permanently failed (no retries will occur). Critical for identifying broken rules that require manual intervention.

SQL
--Filter for INVOCATION_ATTEMPT_PERMANENT_FAILURE
SELECT `@timestamp`,
       event_bus_name,
       log_level,
       message_type,
       `details.invocation_status` AS Invocation_Status,
       `details.rule_arn` AS Rule_ARN,
       `details.target_arn` AS Target_ARN,
       `error.aws_service`,
       `error.http_status_code`,
       `error.error_message`,
       request_id
FROM `LogGroup`
WHERE message_type = 'INVOCATION_ATTEMPT_PERMANENT_FAILURE'
ORDER BY `@timestamp` DESC

7. Filter for Invocations with Latency greater than 100ms

Surfaces invocations where target duration exceeded 100ms. Helps identify slow targets that may be contributing to throttling or timeouts.

SQL
--Filter for Invocations with Latency greater than 100ms
SELECT `@timestamp`,
       message_type,
       `details.invocation_status` AS Invocation_Status,
       `details.rule_arn` AS Rule_ARN,
       `details.target_arn` AS Target_ARN,
       `details.target_duration_ms` AS Target_Duration,
       `error.aws_service`,
       `error.http_status_code`,
       `error.error_message`
FROM `LogGroup`
WHERE `details.target_duration_ms` > 100
ORDER BY `details.target_duration_ms` DESC

8. Filter for Recently Created EventBridge Rules (CloudTrail)

Queries CloudTrail for PutRule API calls to identify recently created EventBridge rules, including who created them and from where.

SQL
--Filter for recently created EventBridge Rules
SELECT eventTime,
       eventName AS API,
       `requestParameters.name` AS ruleName,
       sourceIPAddress,
       userAgent,
       `requestParameters.description` AS ruleDescription,
       `userIdentity.arn` AS User
FROM `LogGroup`
WHERE eventName = 'PutRule'
ORDER BY eventTime DESC

9. Filter for PutPermission (Apply Resource Policy to Event Bus)

Queries CloudTrail for PutPermission API calls which apply resource-based policies to event buses. Important for cross-account event routing audits.

SQL
--Filter for PutPermission (Apply Resource Policy to Event Bus)
SELECT eventTime,
       eventID,
       awsRegion,
       eventName,
       `requestParameters.eventBusName` AS EventBus,
       errorCode,
       errorMessage
FROM `LogGroup`
WHERE eventName = 'PutPermission'
ORDER BY eventTime DESC

10. Summarize EventBridge Logs by Logging Level & Message Type

High-level overview of all EventBridge log events grouped by log level and message type. Quickly shows the distribution of INFO vs ERROR events.

SQL
--Summarize EventBridge Logs by Logging Level & Message Type
SELECT log_level,
       message_type,
       COUNT(*) AS TotalEvents
FROM `LogGroup`
GROUP BY log_level, message_type
ORDER BY TotalEvents DESC

11. Summarize EventBridge Errors by Message Type

Counts error events by message type. Shows whether failures are due to throttling, permanent failures, or DLQ issues.

SQL
--Summarize EventBridge Errors by Message Type
SELECT log_level,
       message_type,
       COUNT(*) AS Errors
FROM `LogGroup`
WHERE log_level = 'ERROR'
GROUP BY log_level, message_type
ORDER BY Errors DESC

12. Summarize EventBridge Errors by Message Type, Invocation Status, AWS Service & HTTP Status Code

Detailed error breakdown by message type, invocation status, AWS service, HTTP response code, and error message. The most comprehensive error summary.

SQL
--Summarize EventBridge Errors: Message Type, Invocation Status, AWS Service, HTTP Status Code, Error Message
SELECT message_type,
       `details.invocation_status` AS Invocation_Status,
       `error.aws_service`,
       `error.http_status_code` AS HTTP_Response_Code,
       `error.error_message` AS Error_Message,
       COUNT(*) AS TotalErrors
FROM `LogGroup`
WHERE log_level = 'ERROR'
GROUP BY message_type, `details.invocation_status`, `error.aws_service`, `error.http_status_code`, `error.error_message`
ORDER BY TotalErrors DESC

13. Summarize EventBridge Events for a Specific Rule

Aggregates error events for a specific rule by target, invocation status, and HTTP status code. Helps quantify how often a rule is failing.

SQL
--Summarize EventBridge Events for a Specific Rule
SELECT `details.rule_arn` AS Rule_ARN,
       `details.target_arn` AS Target_ARN,
       log_level,
       `details.invocation_status` AS Invocation_Status,
       `error.aws_service`,
       `error.http_status_code`,
       COUNT(*) AS NumofEvents
FROM `LogGroup`
WHERE `details.rule_arn` LIKE '%rule/SSM-Automation-Stop-EC2%'
  AND log_level = 'ERROR'
GROUP BY `details.rule_arn`, `details.target_arn`, log_level, `details.invocation_status`, `error.aws_service`, `error.http_status_code`
ORDER BY NumofEvents DESC

14. Summarize INVOCATION_ATTEMPT_PERMANENT_FAILURE Events

Aggregates permanent failure events by rule, target, AWS service, and HTTP status code. Identifies which rule/target combinations are permanently failing most.

SQL
--Summarize INVOCATION_ATTEMPT_PERMANENT_FAILURE Events
SELECT `details.rule_arn` AS Rule_ARN,
       `details.target_arn` AS Target_ARN,
       `error.aws_service`,
       `error.http_status_code`,
       COUNT(*) AS Failures
FROM `LogGroup`
WHERE message_type = 'INVOCATION_ATTEMPT_PERMANENT_FAILURE'
GROUP BY `details.rule_arn`, `details.target_arn`, `error.aws_service`, `error.http_status_code`
ORDER BY Failures DESC

15. Summarize INVOCATION_ATTEMPT_THROTTLE Events

Aggregates throttle events by rule, target, and service. Identifies which targets are being throttled most and may need concurrency increases.

SQL
--Summarize INVOCATION_ATTEMPT_THROTTLE Events
SELECT `details.rule_arn` AS Rule_ARN,
       `details.target_arn` AS Target_ARN,
       `error.aws_service`,
       `error.http_status_code`,
       COUNT(*) AS TotalErrors
FROM `LogGroup`
WHERE message_type = 'INVOCATION_ATTEMPT_THROTTLE'
GROUP BY `details.rule_arn`, `details.target_arn`, `error.aws_service`, `error.http_status_code`
ORDER BY TotalErrors DESC

16. Summarize INVOCATION_DLQ Events

Aggregates dead-letter queue events by rule, target, and service. Shows which rules are sending events to the DLQ after exhausting retries.

SQL
--Summarize INVOCATION_DLQ Events
SELECT `details.rule_arn` AS Rule_ARN,
       `details.target_arn` AS Target_ARN,
       `error.aws_service`,
       `error.http_status_code`,
       COUNT(*) AS TotalErrors
FROM `LogGroup`
WHERE message_type = 'INVOCATION_DLQ'
GROUP BY `details.rule_arn`, `details.target_arn`, `error.aws_service`, `error.http_status_code`
ORDER BY TotalErrors DESC

Usage Notes

  • Replace `LogGroup` with the name of your EventBridge log group (for queries 1–7, 10–16) or CloudTrail log group (for queries 8–9)
  • Replace rule ARNs, target ARNs, and request IDs with values relevant to your environment
  • These queries use the SQL query language — ensure your Log Insights console is set to SQL mode
  • EventBridge logging must be enabled on your event bus for queries 1–7 and 10–16 to return results
  • Results can be visualized directly in a CloudWatch Dashboard using Log Insights widgets