Logs are everywhere — which is good because they’re a critical piece of operations work in computing, whether you work in DevOps, security operations, or IT operations.
Analyzing logs helps with many different aspects of the technical support of devices, users, applications, and more. They can determine where authentication is happening, how applications work, whether a system crashed because of malicious actions or some other malfunction.
To understand the value in an event log, first let’s take a look at what they are.
Event logs are a specific type of file that store information about significant actions or occurrences in a computer system. The concept of event logs are universal across operating systems and devices. These files are chronologically ordered lists of the recorded events that store significant actions or occurrences.
There are multiple types of event logs from operating systems, databases, and cloud services. Windows event logs, for example, are generated on any computer running the Windows operating system. Linux based operating systems have a system log (syslog) process that records events in applications and on the operating system. There are also event logs for hardware devices, applications, and cloud services like AWS and GCP.
The type of events tracked in an event log changes depending on which type of system is creating the log. Operating systems like Windows and Linux tend to collect the following types of information in their event logs:
There are also database event logs where, for example, SQL server stores events such as:
There are many different opportunities for logging in the networking space.
Lastly, cloud services like AWS CloudTrail, CloudWatch Log, or AWS Config record different service logs for situations like RDS instances or the output of a serverless function.
Event logs often have different settings for how in depth the information should be. For example, there are debug logs that add extra verbosity when you are troubleshooting a specific issue. These logs aren’t normally turned on by default, but can come in handy in specific instances.
There are a few common fields to be found in event log files. The specific names of the fields may change depending on whether you’re dealing with a system event log or a cloud event log, but the concepts remain the same.
These fields include:
Understanding the fields that are most common is important then for event management.
Event logs are commonly stored in log files. These are dedicated files with sequential entries, which are often deleted when they get too large. Windows stores logs in .evtx files for the majority of logs, which is a proprietary binary format that can only be viewed in Windows Event Viewer. In other instances they are stored in a flat file. When using IIS they are stored in %SystemDrive%\inetpub\logs\LogFiles or Windows Firewall that are stored in %windir%\system32\logfiles\firewall\pfirewall.log.
In many non-Windows systems, the logs are stored in the /var/log directory or other subdirectories in that location. The most common way to navigate through Linux log files are with commands like less, grep, and tail. These commands can be very powerful for searching and formatting the data to be more human readable. Application event logs are governed by the developers who built the apps. There is thus greater flexibility in where the logs are stored, and what the file may or may not be called.
Event logs are a crucial component of most systems. They are records of events that occur on the system, such as system start and stop times, user logins and logouts, and software or hardware errors. Event logs are used to track and troubleshoot system issues, monitor for security threats, and comply with regulatory requirements.
One of the primary uses of event logs is to use them to troubleshoot system issues. When a problem occurs on a computer or network, the event log can provide valuable information that can help IT professionals identify the cause of the problem and find a solution. For example, if a system like the domain controller shows an error, the event log can show when the error occurred and what caused it. This information can be used to fix the problem and prevent it from happening again in the future.
The other main use of event logs is to use them for monitoring information and system security and risk. When a configuration change occurs, risky behavior happens, or malicious activity happens, logging is one of the only ways that there can be visibility into those events.
Event logs can provide information about attempted or successful security breaches. This information can be used to identify and respond to potential security threats, such as by blocking access to the system from a specific IP address, running malware scans, or declaring and responding to a full incident.
Compliance is another important reason why event logs matter. Many industries and government agencies have regulations that require organizations to keep records of certain types of events, such as user logins and logouts. They can provide this information, which is necessary for compliance with these regulations.
A main reason event logs are used for auditing and compliance purposes is because they provide an unchangeable record of all the activities that occur on a system, which can be used to determine who did what and when. This information is essential for organizations that need to comply with regulations such as Sarbanes-Oxley, HIPAA, and PCI-DSS, as well as for internal auditing and compliance efforts.
Windows systems have a few common security event logs. Each has a dedicated combination of event channel and event ID that can be used for identification, and each reflects a distinct type of security event. These include:
By default, many devices and operating systems are configured to supply the bare minimum in regards to logging. Sometimes logging isn’t even enabled at all! In general it is best to check what logging options are available for each piece of software and hardware, and every service that is running in an enterprise environment.
We recommend installing and configuring osquery for linux deployments where possible. It is a robust solution that can capture disk, process, and other operating system events. Osquery has a wide adoption and userbase with very thorough documentation and additional plug-ins to fit every environment.
Linux logs are an important tool for maintaining the reliability, availability, and security of computer systems. They provide valuable information that can be used to troubleshoot system issues, monitor for security threats, comply with regulatory requirements, and improve the overall performance and availability of the system. It is important for organizations to have a robust logging infrastructure in place to ensure that they can take full advantage of the benefits that event logs provide.
For windows environments there are a large number of resources available for both on prem and cloud deployments. For on prem devices we recommend starting with both sysmon or the Blumira Agent, and Logmira for group policy settings. The combination of these two things will enable the greatest amount of security logging on your endpoints. For Azure cloud deployments you have the option to enable logging settings at almost every console and send through the Event Hub.
Blumira’s free edition integrates directly with your Microsoft 365 tenant to detect suspicious activity in your environment — at no cost. Get your free account and see the value of Blumira today.
Event logs are an important tool for maintaining the reliability, availability, and security of computer systems. They provide valuable information that can be used to troubleshoot system issues, monitor for security threats, comply with regulatory requirements, and improve the overall performance and availability of the system. It is important for organizations to have a robust logging infrastructure in place. This ensures that the business can take full advantage of the benefits that they provide.