About the Company
Implementation of Cloudwatch and Cloudtrail for Monitoring and Logging
- Client: IN10 Media BCCI
- Industry: Media and News
- AWS Services Used: AWS CloudWatch, AWS CloudTrail, AWS Lambda, AWS SNS, AWS EventBridge
Challenges: IN10 Media BCCI operates a dynamic news platform that requires real-time monitoring and comprehensive logging to ensure high availability, security, and performance.
The previous infrastructure lacks comprehensive monitoring and logging capabilities, resulting in difficulties in tracking application performance, identifying security issues, and maintaining compliance, delayed Incident Response, Manual Monitoring, Limited Insight into Changes, Difficulty Diagnosing Performance Issues.
Objectives
1.Enhance Monitoring: Implement AWS CloudWatch to provide real-time monitoring of the infrastructure and applications.
2.Improve Logging: Implement AWS CloudTrail to log all API activities and track user actions for security and compliance.
3.Optimize Performance: Use the insights from monitoring and logging to optimize the performance of the infrastructure and applications.
4.Ensure Security: Enhance the security posture by tracking and analysing access and activity logs.
5.Facilitate Troubleshooting: Enable faster and more efficient troubleshooting by providing detailed logs and metrics.
Optimizing the migration process to minimize downtime and ensure data integrity while transferring large volumes of data (100 GB) securely and efficiently from on-premises servers to AWS.
Configuring and managing a robust network infrastructure to
establish secure and reliable connections between on-premises data centers and AWS infrastructure, ensuring minimal latency and maximum uptime.
Designing and implementing a scalable storage architecture that can seamlessly accommodate the expected growth of data volumes into terabytes, while ensuring high availability and performance
Maintaining high data availability and performance consistency across distributed networks
AWS CloudWatch Implementation
1.Real-Time Monitoring:
• We have set up CloudWatch dashboards to visualize system performance metrics.
• Configured CloudWatch Alarms to notify the operations team of any anomalies or threshold breaches.
2.Custom Metrics:
• We have created custom CloudWatch metrics for specific application parameters.
• Integrated CloudWatch with existing applications to push custom logs and metrics.
3.Logs and Metrics Analysis:
• We have utilized CloudWatch Logs to aggregate, monitor, and store log files from various sources.
• We implemented CloudWatch Log Insights for querying and analysing log data.
Optimizing the migration process to minimize downtime and ensure data integrity while transferring large volumes of data (100 GB) securely and efficiently from on-premises servers to AWS.
Configuring and managing a robust network infrastructure to
establish secure and reliable connections between on-premises data centers and AWS infrastructure, ensuring minimal latency and maximum uptime.
Designing and implementing a scalable storage architecture that can seamlessly accommodate the expected growth of data volumes into terabytes, while ensuring high availability and performance
Maintaining high data availability and performance consistency across distributed networks
AWS CloudTrail Implementation
1.API Activity Logging:
• Enabled CloudTrail across all AWS accounts to log API calls: We implemented CloudTrail across all AWS accounts to comprehensively record all API activity.
• We have configured CloudTrail to capture details about API requests: CloudTrail is configured to capture granular details about API requests, including the source IP address, timestamp, and request parameters.
2.Security and Compliance:
• We have set up CloudTrail logs to monitor for security threats and compliance breaches: CloudTrail logs are continuously monitored to detect potential security threats and ensure compliance with relevant regulations.
• We integrated CloudTrail with AWS Config to track resource configurations and changes: CloudTrail is integrated with AWS Config to provide a comprehensive view of resource configurations and track any changes made.
3.Centralized Logging:
• We have aggregated CloudTrail logs in a centralized S3 bucket for easy access and long-term storage: CloudTrail logs are aggregated within a centralized S3 bucket for efficient access and long-term archival purposes.
• Enabled log file validation to ensure the integrity and authenticity of log files: Log file validation is enabled to guarantee the integrity and authenticity of CloudTrail logs.
4.Analysis and Alerting:
• We have used AWS Lambda to process CloudTrail logs and trigger alerts based on specific events: AWS Lambda functions are utilized to process CloudTrail logs and trigger automated alerts based on predefined security events.
• We have integrated CloudTrail with AWS SNS to notify the security team of any suspicious activities: CloudTrail is integrated with AWS SNS to deliver real-time notifications to the security team regarding any suspicious activities identified in the logs.
CloudWatch Alarms for BCCI Account
• To enhance infrastructure monitoring and ensure proactive management of the BCCI account, we have implemented a comprehensive set of CloudWatch alarms. These alarms are designed to alert the team to critical changes in various metrics, helping to maintain optimal performance and quickly address any issues.
Instance Health and Performance:
• For CPU utilization, alarms were configured with thresholds at different levels for various servers: one alarm was set to trigger at greater than 90%, another at greater than 80%, and a third at 50%. This tiered approach allows for proactive management of server load and helps prevent potential performance degradation.
• Memory utilization alarms were established with thresholds at greater than 90% and greater than 80%. These alarms enable timely identification and resolution of memory-related issues, ensuring smooth operation of applications and services.
• To monitor disk space, root disk utilization alarms were set with thresholds at greater than 90% and greater than 80%. This ensures that disk usage is kept in check, preventing storage-related disruptions.
• Additionally, alarms for HTTP errors were configured to monitor the health of web services. An alarm was set for 4XX errors with a threshold of 50 errors, and another for 5XX errors with a threshold of 10 errors. These alarms help quickly identify and address client-side and server-side issues, respectively, maintaining a high level of service availability and user satisfaction.
• Email notifications for IN10 Media BCCI’s pipeline actions and stages, ensuring that all critical events are promptly communicated via AWS SNS (Simple Notification Service) by email. The notifications cover various aspects of pipeline execution, including: Succeeded, Failed, canceled, Approved etc. These notifications ensure that IN10 Media BCCI’s team stays informed about the status of their CI/CD pipelines, enabling them to take prompt action when necessary to maintain seamless and efficient operations.
• We have developed a series of CloudWatch dashboards specifically designed for our customer, BCCI, to enhance their infrastructure monitoring capabilities. These dashboards provide comprehensive insights into various aspects of their system, enabling them to maintain optimal performance and quickly address any issues that arise. Below is a summary of the dashboards we have created for BCCI:
• BCCI-PreProd-Dashboard: Monitors the pre-production environment, providing visibility into the system’s health and performance before any changes are deployed to the production environment.
• BCCI-PROD: Focuses on the production environment, offering real-time monitoring and alerting to ensure the live system runs smoothly.
• BCCI-Prod-Dashboard: Another key dashboard for the production environment, providing detailed metrics and visualization to help in analyzing the production system’s performance.
• EC2-Uptime: Tracks the uptime and availability of EC2 instances, ensuring that the virtual servers are operational and performing as expected.
• IN10Media-Cloudwatch-Dashboard: Custom dashboard tailored for the IN10Media service, offering monitoring and insights relevant to its specific infrastructure needs.
• IPL-CloudWatch-Dashboard: Designed for the IPL infrastructure, this dashboard helps in monitoring the various components and services associated with the IPL operations.
• IPL-POLLS: Provides monitoring for polling services related to IPL, offering insights into their performance and reliability.
• IPL-PROD: Focuses on the IPL production environment, ensuring that all live services are running smoothly and providing real-time performance metrics.
• IPL-PreProd-Dashboard: Monitors the IPL pre-production environment, giving visibility into system performance and stability before changes are rolled out to production.
These dashboards are accessible via the shared link: CloudWatch Dashboards for BCCI, where you can view and interact with them to gain detailed insights into the performance and health of AWS infrastructure.
EC2 Instance State Change Notification Automation using Cloudtrail API
We have implemented a sophisticated automation solution using Amazon EventBridge, AWS Lambda, and Amazon SNS. This setup ensures that any changes in the state of EC2 instances such as starting, stopping, or terminating—are promptly communicated to the relevant stakeholders via email.
EventBridge Configuration:
• We have set up Amazon EventBridge (formerly known as CloudWatch Events) to monitor API calls made to AWS CloudTrail. This enables us to capture detailed events related to EC2 instance state changes.
• Specifically, EventBridge rules are configured to listen for EC2 state transition events, such as when an instance is started, stopped, or terminated.
CloudTrail Integration:
• AWS CloudTrail captures API activity across the AWS environment, including actions related to EC2 instances. CloudTrail logs are used as the event source for EventBridge, providing detailed context about the state changes.
Lambda Function:
• When EventBridge detects an EC2 state change event, it triggers an AWS Lambda function. This Lambda function processes the event data, extracting key details such as the instance ID, previous state, and new state.
• The function then formats this information into a structured message suitable for notification.
Amazon SNS Notification:
• The Lambda function publishes the formatted message to an Amazon SNS topic. SNS is used to send notifications via email to a predefined list of recipients.
Success Metrics:
Performance Optimization:
• Reduced Downtime: Real-time monitoring and alerts reduced system downtime by 40%.
• Improved Performance: Insights from custom metrics and logs helped in optimizing application performance, resulting in a 30% improvement in response times.
Enhanced Security:
• Improved Threat Detection: Continuous monitoring of API activities and access logs improved threat detection and response time.
• Compliance: Ensured compliance with industry standards by maintaining detailed logs of all activities.
Operational Efficiency:
• Faster Troubleshooting: Detailed logs and real-time monitoring facilitated faster identification and resolution of issues, reducing troubleshooting time by 50%.
• Scalability: The scalable nature of CloudWatch and CloudTrail allowed IN10 Media BCCI to handle increased traffic and expand its infrastructure seamlessly.
• Reduction in Manual Monitoring Effort: Manual monitoring efforts have been reduced by 75%, as automated notifications provide immediate awareness of EC2 state changes.
• Number of EC2 State Change Events Captured: 100% of EC2 state change events (start, stop, terminate) are accurately captured by EventBridge.
Conclusion
The implementation of AWS CloudWatch and CloudTrail by Galaxy Office Automation Pvt Ltd significantly enhanced IN10 Media BCCI’s monitoring and logging capabilities. This project not only improved system performance and security but also ensured compliance and operational efficiency. The successful deployment of these AWS services has positioned IN10 Media BCCI to better handle its growing user base and dynamic content demands.