Security Logging and Monitoring: What to Log, Where to Store It, and How to Alert

Effective security logging is the difference between detecting an attack in hours versus discovering a breach months later. Yet most organizations either log too little (missing critical events) or log everything (drowning in noise). This guide gives you a practical framework for what to log, how to store it, and how to build meaningful alerts.

The Logging Triad: What, Where, How Long

# What to log (minimum):
Authentication:  All login attempts (success + failure), MFA events, account lockouts
Authorization:   Access to sensitive data, privilege escalation, permission changes
System changes:  New accounts, group membership changes, software installs
Network:         Firewall blocks, DNS queries, connection records (NetFlow)
Application:     HTTP access logs, API calls, error events
Endpoint:        Process execution, file changes, registry modifications

# Where to send logs:
# Centralized SIEM (not local files attackers can delete!)
# Options: Wazuh (free), Splunk, Microsoft Sentinel, Elastic SIEM

# How long to retain:
# Regulatory minimum: HIPAA/PCI = 1 year, GDPR = purpose-limited
# Security recommendation: 90 days hot (searchable) + 1 year cold (archived)
# Reason: Average breach detection time was 194 days (2023) - you need 6 months minimum

Critical Windows Events to Monitor

# Authentication events:
4624  Successful logon          4625  Failed logon
4634  Logoff                    4648  Explicit credential logon (RunAs)
4771  Kerberos pre-auth failed  4776  NTLM auth attempt

# Account management:
4720  Account created           4726  Account deleted
4728  Added to security group   4732  Added to local admins
4723  Password change attempt   4724  Admin password reset

# Privilege use:
4672  Special privileges assigned at logon (admin login)
4673  Privileged service called
4674  Operation on privileged object

# Process events (enable Sysmon for better data):
4688  Process created           4689  Process terminated
1     Sysmon: Process create    3     Sysmon: Network connection

# Collect these with Wazuh:
# /var/ossec/etc/ossec.conf - add Windows event channels:
# <localfile>
#   <location>Security</location>
#   <log_format>eventchannel</log_format>
# </localfile>

Linux Auditd: Critical Rules

# /etc/audit/rules.d/security.rules

# Monitor privileged command execution:
-a always,exit -F arch=b64 -S execve -F euid=0 -k priv_exec
-a always,exit -F arch=b32 -S execve -F euid=0 -k priv_exec

# Monitor authentication files:
-w /etc/passwd -p wa -k identity
-w /etc/shadow -p wa -k identity
-w /etc/sudoers -p wa -k sudoers

# Monitor SSH configuration:
-w /etc/ssh/sshd_config -p wa -k sshd_config

# Monitor cron:
-w /etc/crontab -p wa -k cron
-w /etc/cron.d/ -p wa -k cron
-w /var/spool/cron/ -p wa -k cron

# Monitor kernel modules (rootkit indicator):
-a always,exit -F arch=b64 -S init_module -S delete_module -k modules

# Apply rules:
augenrules --load
auditctl -l  # List active rules

Building Effective Alerts

# Alert quality = Signal-to-Noise Ratio
# Alert fatigue = too many alerts = analysts ignore them all

# Alert priority framework:
# P1 (Immediate response, 15 min):
# - Active ransomware (high disk I/O + mass file extension changes)
# - Domain admin account created outside change window
# - LSASS memory access from non-system process

# P2 (Respond same day):
# - Multiple failed MFA attempts on executive account
# - Large outbound data transfer to unknown IP
# - New scheduled task or service created

# P3 (Investigate within 72 hours):
# - Account created then immediately deleted
# - Successful login from new country
# - Unusual port scanning from internal host

# Wazuh correlation rule example:
# Alert when user fails login 5 times then succeeds:
# Rule 5710: Multiple authentication failures
# Rule 5715: Authentication success after failures
# <rule id="100001" level="12" frequency="5" timeframe="120">
#   <if_matched_sid>5710</if_matched_sid>
#   <description>Multiple auth failures followed by success - possible brute force</description>
# </rule>

Log Integrity

# Attackers delete logs to cover tracks — protect your logs:

# 1. Forward logs to immutable external SIEM immediately (streaming)
# Local log deletion is irrelevant if logs are already in SIEM

# 2. Enable Windows Log Forwarding:
# Group Policy > Computer Config > Windows Settings > Security Settings
# Advanced Audit Policy Configuration > System Audit Policies

# 3. Protect Wazuh from log tampering:
# Wazuh agent runs as root and sends logs to manager before local write
# Manager is isolated from other systems

# 4. Use WORM (Write Once Read Many) storage for archives:
# AWS S3 Object Lock: compliance mode prevents deletion
# Azure Immutable Blob Storage: legal hold or retention period

# 5. Monitor log deletion itself:
# Windows Event ID 1102: Security audit log cleared
# Linux auditd: -w /var/log/ -p wa -k log_changes

Wrap Up

Good logging is your security camera system. You don’t need to record everything, but you need the right cameras in the right places, properly secured storage, and alerts configured to wake someone up when something important happens. Implement this logging foundation and you’ll catch attacks that would otherwise be invisible for months.