Amazon GuardDuty is a threat detection service that uses machine learning and other techniques to identify malicious activity and unauthorized behavior in your AWS accounts and workloads. We share a step by step guide to enable GuardDuty and SNS alerts on this blog post.
In a previous article posted on the Stream Security's blog, we dove into VPC flow logs, explaining how they capture metadata of our network traffic and help us understand patterns and issues in our network, whether it’s on the network interface level, the subnet level or even more holistically at the VPC level. Definitely a great capability! That said, VPC flow logs give you low-level raw data, leaving it up to our network or DevSecOps teams to analyze it and derive actionable insights. And being able to spot a distributed denial of service attack by staring at the VPC flow logs in Amazon CloudWatch is a rare gift. Wouldn’t it be nice if there was a tool that could do this analysis for us, do the heavy lifting of identifying threats and only notify us in case of an actual threat while not falling for the false positives? What if that tool could also easily look at our VPC flow logs without us having to write ETL code at all? Well, there actually is and it's called Amazon GuardDuty.
In this blog, we will introduce Amazon GuardDuty and see how it can help us determine that someone broke into our instance and used it maliciously as a port scanner.
Amazon GuardDuty is a threat detection service that uses machine learning and other techniques to identify malicious activity and unauthorized behavior in your AWS accounts and workloads. It integrates with other AWS security services to provide a comprehensive view of your security posture and helps Security, DevOps, Compliance and Incident response teams to quickly respond to security threats.
The service integrates with other AWS security services, such as Amazon CloudTrail, Amazon VPC flow Logs, and AWS WAF, to provide a comprehensive view of your security posture.
Amazon GuardDuty integrates with other AWS security services, to provide a comprehensive view of your security posture, such as:
The combination of the above sources allows Amazon GuardDuty to detect plenty of threats from a backdoor in your network to someone using your compute capacity to mine bitcoins! And a very cool feature of Amazon GuardDuty is that prior to detecting some of those threats, the service can establish a baseline of the activity in your AWS account, subsequently detect deviations from that baseline and eventually flag those deviations as abnormal behavior. Imagine the complexity that we would face if we had to code a detection mechanism to flag a resource being accessed from an unusual location at that specific time of the day! And the gift keeps on giving. Over the years, Amazon GuardDuty has evolved dramatically, and it can now centralize under one single pane of glass the findings coming from all of your AWS accounts.
In the example below, we have a VPC with a single public subnet in which lives an EC2 instance. At this point, we do not know that a third-party actor gained access to that instance and installed malicious software to conduct port scanning on our behalf, and by that tarnishing our reputation. We definitely want to catch this as quickly as possible to avoid being sanctioned for taking part in a port scan attack!
As mentioned earlier, Amazon GuardDuty relies on several data sources, one of them being VPC flow logs. So the first thing we will have to do is enable VPC flow logs. And prior to that, we also need to create an IAM role. This role will be used a little bit later and all it does is give permissions to the VPC flow logs servers to write in CloudWatch; this is what will make our VPC flow logs show an Active status.
In your IAM dashboard, create a new EC2 IAM role called vpc-flowlogs (check the AWS service and EC2 radio buttons) and create an IAM policy called vpc-flowlogs-to-cloudwatch.
Update the JSON as follows:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"logs:CreateLogGroup",
"logs:CreateLogStream",
"logs:PutLogEvents",
"logs:DescribeLogGroups",
"logs:DescribeLogStreams"
],
"Resource": "*"
}
]
}
Finally, update the Trust relationships of this role so that it reflects the below:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": "vpc-flow-logs.amazonaws.com"
},
"Action": "sts:AssumeRole"
}
]
}
That’s it for roles and permissions; let’s move on to the VPC flow logs.
You may remember that VPC flow logs can be enabled at various points in your VPC. In this example, we elect to enable VPC flow Logs at the subnet level.
In the VPC console, select your subnet, navigate to the Flow logs tab, and click “Createflow log”
We give our flow logs a name and we set it up so that it captures all the traffic in our subnet, both accepted and rejected traffic. We specify a 1 minute aggregation period and ask that the logs be streamed to CloudWatch logs. If you type a new name for the destination log group, it will be created in CloudWatch logs for you so no need to visit CloudWatch logs during the prerequisite phase. Finally, we specify the IAM role we created earlier and leave all the other settings untouched.
If you created the IAM role properly, your VPC flow logs should show as Active.
And now onto the simplest part of this process. With the VPC flow logs enabled, we can browse to the Amazon GuardDuty console and appreciate the streamlined process that consists only in clicking on the “Enable GuardDuty” button.
After that step, we drop directly in the Findings page and obviously there isn’t much to look at just yet.
In order to simulate the scenario by which our instance has been compromised and running malicious software, let’s ssh into our EC2 instance and install the nmap utility, which is commonly used for port scanning.
And with that installed, As an apology to our friends at Google, we are shamefully scanning one of their well-known servers with a port scanner.
The scan concluded and to no surprise, two ports are open on this DNS server, one for DNS (port 53) and one for HTTPS (port 443).
Now let’s head back to Amazon GuardDuty and see if it noticed anything interesting.
And sure enough after a couple of minutes, Amazon GuardDuty detected in the VPC flow logs that our instance was taking part in a port scan. And by clicking on the finding, you can see that Amazon GuardDuty has already done most of the forensics for you, displaying a lot of valuable insights as well as recommendations to remediate the issue! Just like that, by enabling Amazon GuardDuty on top of VPC flow logs, we went from raw data to valuable and actionable insights!
Instead of manually reviewing GuardDuty findings, you can configure an Amazon EventBridge rule to send SNS notifications upon a trigger of a specific GuradDuty event type.
Amazon EventBridge is a fully managed event bus service that allows you to route events from a variety of sources to multiple targets such as AWS Lambda functions, Amazon SNS topics, and more, filter events based on attributes, transform events, and replay events for testing and debugging.
For our scenario, I created an EventBridge rule , while configuring the event source as GuardDuty and event type as GuardDuty Finding,
And using the following event pattern code:
{
"source": [
"aws.guardduty"
],
"detail": {
"type": [
"Recon:EC2/Portscan"
]
}
}
As for the Target, I’ve selected an SNS topic which is subscribed to my Email address, that I created beforehand.
Note: EventBridge single rule can route incoming events to multiple targets.
And here’s the outcome:
To have this step's configuration, I've got help from the following AWS support article.
After being notified with GuardDuty's findings, you can start troubleshoot the alert using Stream Security.
With Stream Security's Discovery you can easily view the EC2 instance's exposure to the internet and other AWS resources, which lets you benefit from better security and more efficient resource management. By understanding the instance's network configuration, you can ensure that it is properly secured and only accessible to the intended users and services. You can also optimize the instance's performance by making sure that it has the necessary network connectivity to communicate with other AWS resources.
In addition, with Stream Security's Network Traffic Activity logs, you can review, analyze and troubleshoot network configurations with unmatched visibility of your cloud activity and resource usage. With enriched and detailed logs across VPCs, services, clusters, workloads and network components, you can quickly detect network issues within your cloud environments.
Amazon GuardDuty is an extremely easy way to turn raw data into actionable insights, especially in the context of security and threat detection. It feeds off multiple data sources such as Amazon CloudTrail, DNS queries and even VPC flow logs if you enable them. Armed with this data and the capability to establish baselines in your account, Amazon GuardDuty can raise the flag on countless different potential threats such as malware, brute force attacks, data exfiltration and many more, improving our security stance drastically for very little effort.
Check our blog on Troubleshoot AWS Reachability Issues with VPC Flow Logs.
