Passing a penetration test, or pentest, is a good sign that you have the right controls in place to detect a real security threat.
When you engage with a pentesting or vulnerability scanning service, you may be wondering how your SIEM will respond. Will you be inundated with alerts? Will you be able to detect a pentester’s attempts to access your environment?
Before we dive into what you should expect, it’s important to understand the differences between a pentest, a vulnerability scan and a red team engagement.
You may or may not have experience with penetration tests; if you do, it’s most likely because of a compliance framework like PCI. However, you might hear other terms thrown around like red team engagement or vulnerability assessment.
Do these things mean the same thing? No; let’s take a moment to clarify the differences.
| Term | Description | Goal |
|---|---|---|
| Vulnerability Assessment |
An assessment used to identify the adequacy of security measures, identify security deficiencies, and confirm the mitigations in place. | Reduce attack surface |
| Penetration Test | An attack against a system, network or application designed to identify and measure risks associated with the exploitation of a target’s attack surface. | Reduce attack surface |
| Red Team Engagement | The process of using Tactics, Techniques, and Procedures (TTPs) to emulate a real-world threat. | Train and measure the effectiveness of the people, process, and technology |
Source: Red Team Development and Operations (Joe Vest and James Tubberville)
At Blumira, we lean most heavily towards developing rules to catch real-world TTPs that you would see in a red team engagement or an attack by a threat actor in the wild, such as a ransomware gang.
This doesn’t mean you shouldn’t expect any alerts from your SIEM during a penetration test. But you can mitigate many techniques used in a standard penetration test with various security hygiene measures and a secure default configuration.
Let’s review a few examples of alerts you’re likely to encounter during a penetration test.
This discovery technique is favored by many testers, as they can often begin the test on a device without any of the organization’s security tools present on them. This is a great way to begin enumeration without any existing access.
Another common enumeration technique in a network is port scanning. However, this is often not alerted on because the detections require an intrusion detection system (IDS) and network segmentation to pick up and alert on the scanning activity.
If you do not have networks segmented with intrusion protection system (IPS) devices between segments, you’ll be unlikely to catch any of this activity.
You can test your SIEM to ensure that it will detect password spraying.
Depending on the attack path or level of logging an organization has set up, the next alerts can vary. To provide the best visibility, we highly recommend the deployment of sysmon for process logging and the GPOs in Logmira to enable the most effective logging options not enabled by default in Windows.
However most, if not all, penetration tests usually end with the following alert.
Before a pentest is conducted, you can test your SIEM to ensure that it will detect domain account creation.
Before the test, run some tests on your own. There are some good tools on GitHub to test your security stack with:
When the test is complete, review the results with your testers and discuss what configuration changes you can make to secure the environment and ensure future pentesters can’t expose weak points as easily. The penetration testers should be there to help you, so ask what configurations can make you more secure.
Once that’s done, you’ll need to secure the organizational buy-in to make the changes — otherwise you’re likely to wind up with the same results the next year.
Some security tools can be overly noisy, leading security teams and individuals to become overworked and strained by alert fatigue. The sheer volume of alerts and an inability to investigate them all can result in overlooking real security threats.
We’ve also seen SIEMs that are too quiet in the event of a pentest or vuln scan, failing to detect behaviors and techniques that would result in a threat actor gaining access to an environment.
That’s why our detection mindset at Blumira is to create meaningful, actionable alerts. Part of this means that we model our detections off of known attacker tactics, techniques, and procedures. Another component of this is delivering actionable, contextualized findings so you know how to interpret them and what you should do next.
In our upcoming webinar with Brian Johnson, president of pentesting firm 7 Minute Security, you’ll learn more tests you can run to ensure that your SIEM will help you pass a pentest with flying colors.
You’ll also see Blumira in action, catching several behaviors that you’re likely to see in a pentest. Sign up here.
