January 11, 2022

SIEM Alerts To Expect During a Pentest

Passing a penetration test, or pentest, is a good sign that you have the right controls in place to detect a real security threat.

When you engage with a pentesting or vulnerability scanning service, you may be wondering how your SIEM will respond. Will you be inundated with alerts? Will you be able to detect a pentester’s attempts to access your environment?

Before we dive into what you should expect, it’s important to understand the differences between a pentest, a vulnerability scan and a red team engagement.

Pentest, Vulnerability Scan and Red Team Engagement Defined

You may or may not have experience with penetration tests; if you do, it’s most likely because of a compliance framework like PCI. However, you might hear other terms thrown around like red team engagement or vulnerability assessment.

Do these things mean the same thing? No; let’s take a moment to clarify the differences.

Term	Description	Goal
Vulnerability Assessment	An assessment used to identify the adequacy of security measures, identify security deficiencies, and confirm the mitigations in place.	Reduce attack surface
Penetration Test	An attack against a system, network or application designed to identify and measure risks associated with the exploitation of a target’s attack surface.	Reduce attack surface
Red Team Engagement	The process of using Tactics, Techniques, and Procedures (TTPs) to emulate a real-world threat.	Train and measure the effectiveness of the people, process, and technology

Source: Red Team Development and Operations (Joe Vest and James Tubberville)

At Blumira, we lean most heavily towards developing rules to catch real-world TTPs that you would see in a red team engagement or an attack by a threat actor in the wild, such as a ransomware gang.

This doesn’t mean you shouldn’t expect any alerts from your SIEM during a penetration test. But you can mitigate many techniques used in a standard penetration test with various security hygiene measures and a secure default configuration.

Let’s review a few examples of alerts you’re likely to encounter during a penetration test.

1. Null Session

The Null Session alert is probably the most common detection we see during a penetration test of a Windows environment. Many Windows domains have legacy gear or have not been treated to the latest security best practices.

This discovery technique is favored by many testers, as they can often begin the test on a device without any of the organization’s security tools present on them. This is a great way to begin enumeration without any existing access.

Another common enumeration technique in a network is port scanning. However, this is often not alerted on because the detections require an intrusion detection system (IDS) and network segmentation to pick up and alert on the scanning activity.

If you do not have networks segmented with intrusion protection system (IPS) devices between segments, you’ll be unlikely to catch any of this activity.

2. Password Spraying

With unprivileged network access, many penetration testers then start with a password spray attack to attempt to locate a user account with the infamous SeasonYearSpecialChar pattern (ie: Winter2021!). This is often the second most common alert we see during a penetration test.

You can test your SIEM to ensure that it will detect password spraying.

Depending on the attack path or level of logging an organization has set up, the next alerts can vary. To provide the best visibility, we highly recommend the deployment of sysmon for process logging and the GPOs in Logmira to enable the most effective logging options not enabled by default in Windows.

However most, if not all, penetration tests usually end with the following alert.

3. Added Admin Account

This is normally the end goal for most testers: the access to or creation of a Domain Administrator account, which you’ll hear referred to as the proverbial “keys to the kingdom.”

Before a pentest is conducted, you can test your SIEM to ensure that it will detect domain account creation.

Best Practices For a Successful Pentest

Before the test, run some tests on your own. There are some good tools on GitHub to test your security stack with:

When the test is complete, review the results with your testers and discuss what configuration changes you can make to secure the environment and ensure future pentesters can’t expose weak points as easily. The penetration testers should be there to help you, so ask what configurations can make you more secure.

Once that’s done, you’ll need to secure the organizational buy-in to make the changes — otherwise you’re likely to wind up with the same results the next year.

Pass Your Pentest With Blumira

Some security tools can be overly noisy, leading security teams and individuals to become overworked and strained by alert fatigue. The sheer volume of alerts and an inability to investigate them all can result in overlooking real security threats.

We’ve also seen SIEMs that are too quiet in the event of a pentest or vuln scan, failing to detect behaviors and techniques that would result in a threat actor gaining access to an environment.

That’s why our detection mindset at Blumira is to create meaningful, actionable alerts. Part of this means that we model our detections off of known attacker tactics, techniques, and procedures. Another component of this is delivering actionable, contextualized findings so you know how to interpret them and what you should do next.

Learn More

In our upcoming webinar with Brian Johnson, president of pentesting firm 7 Minute Security, you’ll learn more tests you can run to ensure that your SIEM will help you pass a pentest with flying colors.

You’ll also see Blumira in action, catching several behaviors that you’re likely to see in a pentest. Sign up here.

Tag(s): Security Trends and Info , SIEM XDR , Blog

Brian Laskowski

Brian has 5 years of experience in IT, with prior work including linux systems administration to most recently leading the threat intelligence program at the State of Michigan security operations center. Other areas of focus have included, incident response, threat hunting, memory analysis, adversary emulation, and...