Malware Analysis for Beginners: How to Safely Examine Suspicious Files

by

When you receive a suspicious file — a phishing attachment, an unknown executable found on a compromised server, or a sample from a threat intelligence feed — malware analysis tells you what it does, how it works, and what indicators to look for across your environment. This guide covers the fundamentals for analysts just getting started.

Safety first: Never analyze malware on a production system or your personal computer. Always use an isolated, dedicated analysis environment.

Setting Up a Safe Analysis Lab

Option 1: Virtual Machine (Free)

The most accessible option. Use VirtualBox (free) or VMware Workstation (paid) to create an isolated Windows or Linux VM for analysis:

  • Install Windows 10 or Windows 11 in a VM (use a trial version or old license)
  • Critically: disable VM network access or put it on an isolated host-only network — malware phones home and spreads
  • Take a clean snapshot before each analysis — revert to clean state after each session
  • Consider using FlareVM (Mandiant’s free Windows malware analysis distribution) — it pre-installs 70+ analysis tools automatically

Option 2: Online Sandboxes (Easiest)

For quick analysis without setting up infrastructure, online sandboxes automatically execute malware and generate reports:

  • ANY.RUN (app.any.run) — interactive online sandbox, free tier available
  • VirusTotal (virustotal.com) — multi-engine AV scan + basic behavior analysis. Note: files uploaded to VirusTotal are shared with security vendors — don’t upload sensitive or confidential files.
  • Cuckoo Sandbox — open-source, self-hosted sandbox
  • Joe Sandbox — free community tier with limited submissions
  • Hybrid Analysis — free with registration, powered by Falcon Sandbox

Two Approaches to Malware Analysis

Static Analysis

Examining the malware without executing it. Safer and can be done on your normal machine with caution, but limited — you see the code/structure but not necessarily what it does at runtime.

Dynamic Analysis

Executing the malware in a controlled environment and observing its behavior — what files it creates, what processes it spawns, what network connections it makes. Reveals the real behavior but requires an isolated lab.

Static Analysis: Step by Step

Step 1: Calculate File Hashes

Before anything else, calculate the file’s cryptographic hash. This is the malware’s fingerprint — used to search threat intelligence databases.

# Linux/Mac
sha256sum suspicious_file.exe
md5sum suspicious_file.exe

# Windows PowerShell
Get-FileHash suspicious_file.exe -Algorithm SHA256

Search the SHA256 hash in VirusTotal, MalwareBazaar (bazaar.abuse.ch), and Hybrid Analysis. If it’s known malware, you’ll get an immediate identification and detailed report without needing to analyze it yourself.

Step 2: Identify the File Type

# Linux - identify file type by magic bytes, not extension
file suspicious_file.exe

# Check if a "PDF" is actually an executable
file suspicious.pdf

Attackers disguise malware with wrong file extensions. A .pdf or .docx can actually be an executable if the file type tool says otherwise.

Step 3: Extract Strings

Strings are printable character sequences embedded in binary files. Malware often contains revealing strings: IP addresses of C2 servers, URLs, registry keys, file paths, error messages, and hardcoded credentials.

# Basic strings extraction
strings suspicious_file.exe

# More thorough (min 10 char strings, Unicode + ASCII)
strings -n 10 -a suspicious_file.exe | grep -E "(http|ftp|cmd|powershell|registry|HKEY)"

# Windows tool: BinText or Sysinternals Strings

Step 4: PE Header Analysis (Windows Executables)

For Windows PE (Portable Executable) files, examine the header for: compilation timestamp, imported DLLs and functions (reveals capabilities — if it imports CryptEncrypt, it likely encrypts data), exported functions, sections (packed/encrypted malware has unusual section names or entropy), and embedded resources.

Free tools: PEiD (packer detection), pestudio (comprehensive PE analysis), CFF Explorer, Detect-It-Easy (DIE).

Step 5: Check Entropy

Packed or encrypted malware has high entropy (randomness) in its sections. Normal code has lower entropy because instructions follow patterns. High entropy sections (above 7.0/8.0) in a PE file indicate packing/encryption — the real malicious code is hidden and only reveals itself at runtime.

Dynamic Analysis: Step by Step

Step 1: Take a Clean Snapshot

In your isolated VM, take a snapshot of the clean state. You’ll revert to this after analysis. Disconnect or disable all network interfaces.

Step 2: Set Up Monitoring

Launch monitoring tools before executing the malware:

  • Process Monitor (ProcMon) from Sysinternals — logs all file system, registry, and process activity
  • Process Hacker or Process Explorer — monitor running processes and their network connections
  • Wireshark — capture all network traffic (even if network is isolated, you can route through a fake gateway)
  • Autoruns — monitor what starts at boot (malware often adds persistence)
  • Regshot — take a “before” snapshot of the registry; take “after” and compare

Step 3: Execute and Observe

Run the malware. Observe for 5–15 minutes. Look for:

  • New processes spawned (cmd.exe, powershell.exe are common malware children)
  • Files created or modified (especially in AppData, Temp, System32)
  • Registry modifications (persistence keys like HKCUSoftwareMicrosoftWindowsCurrentVersionRun)
  • Network connection attempts (C2 beacon, DNS lookups to unusual domains)
  • Privilege escalation attempts
  • Lateral movement (network shares being accessed)

Step 4: Extract Indicators of Compromise (IoCs)

Document everything you found: file hashes, file paths created, registry keys modified, IP addresses and domains contacted, mutex names (malware uses mutexes to avoid running twice). These IoCs can be used to hunt for the same malware across your environment.

Step 5: Revert Snapshot

Always revert to your clean snapshot after analysis. Never let analysis VMs accumulate infections over time.

Free Malware Samples for Practice

  • MalwareBazaar (bazaar.abuse.ch) — thousands of malware samples with context
  • VirusTotal — with an account, you can download samples
  • theZoo (GitHub) — collection of malware samples for educational purposes
  • Malware Traffic Analysis (malware-traffic-analysis.net) — PCAP files with malware traffic for analysis practice

Summary

Malware analysis is a rewarding skill that makes you a significantly more effective incident responder. Start with static analysis (hashes, strings, PE headers) since it’s safe and quick. Progress to dynamic analysis with an isolated VM once you’re comfortable. The combination of VirusTotal lookups, ProcMon monitoring, and network capture will reveal what most malware does. Practice regularly with samples from MalwareBazaar.