Forensics · Lesson 08
Linux Logs & Journal
Learn how Ubuntu and Linux Mint systems record logins, sudo, SSH, services, package changes, scheduled activity, boots, and system events through the systemd journal and traditional log files.
Linux logging is flexible. Do not assume every machine has the same files. Start with the question, inspect the journal, confirm which text logs exist, and correlate across sources.
Lesson Snapshot
Learning objectives
Use Linux logs as evidence, not assumptions
Understand how Linux systems record authentication, services, boot activity, package changes, and system events.
Use journalctl safely to filter by time, boot, service, priority, and keyword.
Recognize common Ubuntu and Linux Mint log files while remembering that exact logging depends on system configuration.
Interpret sudo, SSH, login, service, and package-management evidence without overclaiming intent.
Correlate Linux logs with users, files, processes, services, cron jobs, and network evidence.
Document the exact source, host, timestamp, user, unit, process, and message needed for a defensible forensic answer.
Core idea
Linux logs are distributed across multiple sources
The systemd journal may contain service, boot, kernel, and authentication events, while traditional files under /var/log can provide text-based records when those logging services are configured.
Competition caution
Do not clear or rotate the evidence you still need
Preserve relevant login, sudo, service, package, cron, and SSH records before changing configuration or restarting services.
Log sources
Know where Linux evidence may be stored
systemd journal
Queried with journalctl
Centralized event records for boots, services, kernel messages, authentication-related activity, and applications that log through systemd.
Caution
Journal retention and persistence depend on configuration. Some records may exist only in memory or for a limited time.
/var/log/auth.log
Common on Ubuntu-family systems when rsyslog-style logging is enabled
Authentication, sudo, SSH, PAM, and account-related messages.
Caution
This file may not exist on every Linux installation or may be incomplete if logging is configured differently.
/var/log/syslog
Common on Ubuntu-family systems when rsyslog is enabled
General system and service messages from many components.
Caution
Some systems rely more heavily on the journal and may not maintain this file.
/var/log/kern.log
Present on some Ubuntu-family systems
Kernel-related messages such as driver, device, and low-level system activity.
Caution
Kernel messages may instead be primarily available through journalctl -k.
/var/log/apt
APT package-management logs
Package installation, removal, upgrade, and command history.
Caution
Package logs explain package-manager activity, not necessarily who physically initiated it.
Application-specific logs
/var/log or application-defined directories
Web servers, databases, security tools, and other services may maintain dedicated log files.
Caution
Paths, formats, and retention vary by application.
Journal fields
Read the metadata around each Linux event
Timestamp
When the event was recorded by the system.
Ask
What time zone is displayed, and does it align with other evidence?
Hostname
The system that generated the event.
Ask
Are records from more than one host being mixed together?
Unit
The systemd service, socket, timer, mount, or other unit associated with the message.
Ask
Is this the service actually relevant to the forensic question?
Process / PID
The process name or process identifier associated with the message.
Ask
Can process evidence or service status corroborate the event?
User / UID
An account or numeric user identifier associated with the event where available.
Ask
Does this represent the caller, service account, target user, or another identity?
Priority
A syslog-style severity from emergency through debug.
Ask
Is severity relevant to the forensic question, or only to operational health?
Boot ID
An identifier that separates events by system boot.
Ask
Did the event occur before or after a reboot?
Message
Human-readable details from the logging component.
Ask
Which facts are directly stated, and which conclusions still require corroboration?
Investigation workflow
Move from Linux question to evidence
Start with the question
Decide whether you need login, sudo, SSH, service, boot, package, kernel, cron, or application evidence.
Choose the likely source
Use the systemd journal first for service and boot context, then check text logs such as auth.log or syslog when they are present and relevant.
Set a time window
Use known file, browser, process, or scenario timestamps to avoid reading thousands of unrelated records.
Filter by unit or keyword
Narrow by service, process, priority, user-related term, or keyword while preserving enough surrounding records for context.
Read neighboring events
Look before and after the target message to identify startup sequences, failures, retries, privilege changes, or related service activity.
Correlate
Compare the log with account state, files, services, cron jobs, processes, package logs, and network listeners.
Document before hardening
Record the source, time, host, unit, user, process, and message before changing accounts, services, SSH, cron, or files.
journalctl
Filter the systemd journal without changing it
Recent journal records
journalctl -n 50
Shows the most recent journal entries.
Current boot
journalctl -b
Shows records from the current boot.
Previous boot
journalctl -b -1
Shows records from the previous boot when retained.
Time window
journalctl --since "2026-10-10 13:00:00" --until "2026-10-10 14:00:00"
Limits results to a defined period.
One service
journalctl -u ssh --since today
Shows events associated with the SSH service unit where the unit name is ssh.
Kernel messages
journalctl -k
Shows kernel-related records for the selected boot context.
Priority filter
journalctl -p warning..emerg
Shows warning-and-higher-priority records, useful for operational review but not a replacement for question-driven filtering.
Reverse order
journalctl -r -n 50
Shows recent records newest first.
Traditional text logs
Use grep and less when the files are present
Review authentication log
sudo less /var/log/auth.log
Reads the authentication log without modifying it when the file exists.
Search sudo activity
grep -i "sudo" /var/log/auth.log
Finds lines that reference sudo activity.
Search SSH activity
grep -i "sshd" /var/log/auth.log
Finds SSH daemon authentication and session records.
Search a username
grep -i "trainee1" /var/log/auth.log
Finds authentication records associated with a specific training username.
Review syslog
sudo less /var/log/syslog
Reads general system messages where that file is present.
Search a service name
grep -i "cron" /var/log/syslog
Finds references to a service or subsystem in a text log.
Authentication patterns
Read login and privilege records carefully
Accepted password / Accepted publickeyAn SSH authentication succeeded for the account and connection described by the log entry.
Caution
A successful authentication does not prove the session was unauthorized.
Failed passwordAn SSH password authentication attempt failed.
Caution
One failure can be normal. Repetition, source, timing, and later success matter.
sudo: user : TTY= ... COMMAND= ...A sudo invocation was recorded with user and command context.
Caution
Read the exact account, command, and result rather than labeling all sudo activity suspicious.
session opened / session closedPAM or another authentication subsystem recorded session lifecycle activity.
Caution
Session type and service context matter.
authentication failureAn authentication component recorded a failed authentication event.
Caution
Check the service, user, source, and nearby events before drawing conclusions.
new user / group changesAccount-management tools may log creation or membership activity through system logging.
Caution
Confirm with /etc/passwd, /etc/group, sudoers, and account state.
SSH evidence
Remote authentication and session clues
Successful SSH login
Record account, source address, authentication method, time, and session-opening records.
Repeated failed SSH attempts
Compare source addresses, target usernames, timing, and whether a success followed.
SSH service restart
Check journal entries, configuration modification time, and whether the restart followed an administrative change.
Authorized key added
Preserve ~/.ssh/authorized_keys metadata, ownership, permissions, hash, and related login records.
Unexpected SSH listener
Correlate service state, sshd configuration, listening port, process ownership, and scenario requirements.
SSH login for privileged account
Review policy and scenario requirements before concluding that the login was unauthorized.
sudo evidence
Privilege-use clues
sudo command recorded
Shows that an account invoked a privileged command through sudo.
Next: Record the exact command, account, time, TTY or session context, and nearby authentication events.
sudo authentication failure
Shows an unsuccessful attempt to use sudo.
Next: Check whether later attempts succeeded and whether the account should have sudo access.
User added to sudo group
Changes the account's potential privilege level.
Next: Check /etc/group, account-management logs, timestamps, and scenario authorization.
sudoers file changed
May change privilege rules without changing group membership.
Next: Preserve file metadata, content, relevant include files, and administrative evidence.
Service evidence
Use systemd events to understand service state changes
Started <service>
systemd reports the service entered a started state.
Next evidence
Check the unit definition, executable path, user, process, and preceding events.
Stopped <service>
systemd reports the service stopped.
Next evidence
Determine whether the stop was expected, manual, failure-related, or dependency-driven.
Failed to start <service>
The service startup failed.
Next evidence
Read preceding and following journal messages, unit status, and configuration errors.
Reloaded / Restarted <service>
The service configuration or process state was refreshed.
Next evidence
Check for nearby configuration changes or administrative commands.
Unit file changed or enabled
Startup behavior may have changed.
Next evidence
Inspect the unit file, enablement state, timestamps, and related package or admin activity.
Package history
Explain software installation and removal
/var/log/apt/history.log
APT package transaction history such as installs, upgrades, and removals where retained.
/var/log/apt/term.log
Terminal-style output produced during APT operations where retained.
journalctl
May contain package-manager, service-start, and post-install activity surrounding a package change.
dpkg logs
Package status and installation actions can appear in /var/log/dpkg.log on Debian-family systems.
APT history
sudo less /var/log/apt/history.log
Reviews package-management transaction history where present.
Search package name
grep -i "openssh" /var/log/apt/history.log
Finds package operations mentioning a specific package.
DPKG history
grep -i " install " /var/log/dpkg.log
Shows package installation records where the log exists and is retained.
Login history
Use session databases as supporting evidence
Recent logins
last
Reads login history from the wtmp database where available.
Failed login history
sudo lastb
Reads failed-login history from btmp where available and permissions allow.
Currently logged-in users
who
Shows current login sessions.
Detailed current sessions
w
Shows logged-in users and current activity summaries.
wtmp can rotate
Older login records may move to rotated files or be unavailable.
btmp may require root
Failed-login history is commonly restricted.
Shell history is incomplete
Commands may not be saved immediately, can be configured differently, and do not capture every action.
Current sessions are snapshots
who and w show present state, not the entire past timeline.
Scheduled activity
Cron and systemd timers leave different clues
cron / CRON messages
Can show scheduled jobs being executed.
Correlate with user crontabs, /etc/crontab, /etc/cron.*, process activity, and the command that ran.
systemd timers
Scheduled execution managed by systemd rather than classic cron.
Use systemctl list-timers and journalctl for the timer and target service.
User crontab
Per-user recurring commands.
Preserve the crontab entry, user, schedule, command, and related logs before changing it.
/etc/cron.*
System-wide periodic job directories.
Check ownership, package source, timestamps, and expected administrative purpose.
Severity
Priority is operational context, not forensic relevance
emergSystem is unusable.
alertImmediate action may be required.
critCritical condition.
errError condition.
warningWarning condition.
noticeNormal but significant condition.
infoInformational message.
debugDetailed debugging information.
An informational SSH login or sudo event may be more important to a forensic question than an unrelated critical hardware error.
Correlation
Combine Linux log evidence with independent artifacts
SSH accepted login at 09:10 + sudo command at 09:13 + configuration file mtime at 09:13
Supported conclusion
The account authenticated remotely and a privileged command aligns with a configuration change.
APT install at 10:05 + new service unit at 10:06 + service started at 10:07
Supported conclusion
The package transaction likely introduced the service and the service started immediately afterward.
Cron execution at 11:00 + script process start at 11:00 + outbound connection at 11:01
Supported conclusion
The scheduled job may explain the recurring process and related connection.
Failed SSH attempts from one address + later successful login from the same address
Supported conclusion
The same source generated failed attempts followed by a success, but intent still requires context.
Service restart at 12:22 + config mtime at 12:21
Supported conclusion
The service restart may have followed a configuration edit.
sudo group membership change + later sudo command
Supported conclusion
The account gained privilege capability before performing a privileged command.
Evidence note
Record Linux events so another teammate can reproduce them
Forensic question
Hostname
Log source
Command or filter used
Timestamp and zone
Unit or process
User or UID
Source address if present
Message or relevant fields
Neighboring events
Corroborating artifact
Supported conclusion
Common mistakes
Linux log-analysis mistakes that weaken forensic answers
Assuming auth.log always exists
Ubuntu-family systems often use it when rsyslog is configured, but journal-only or differently configured systems may not.
Reading only one matching line
Session open, command, restart, failure, and follow-up events may be nearby.
Treating sudo as suspicious by itself
Legitimate administrators and defensive work use sudo constantly.
Using shell history as authoritative
History can be incomplete, delayed, disabled, or edited.
Ignoring boot boundaries
A reboot can separate events into different journal boots and reset process state.
Ignoring source addresses
Remote login evidence becomes much stronger when account, source, method, and time are recorded together.
Changing SSH or cron before preserving evidence
Configuration edits can remove the exact details needed for later questions.
Treating severity as forensic importance
An informational authentication or service event may be more relevant than a critical unrelated error.
Fictional Linux log lab
Reconstruct SSH, sudo, service, and cron activity
Accepted password for trainee5 from 192.0.2.44 via sshd.
trainee5 invokes sudo to edit /etc/ssh/sshd_config.
ssh.service is restarted.
/etc/ssh/sshd_config shows a modification time near the recorded sudo activity.
A fictional user cron job runs /home/trainee5/bin/check.sh.
check.sh launches a process that connects to fictional lab host 203.0.113.70.
Task 01
Which record establishes remote authentication?
The 09:02:11 auth.log record showing an accepted SSH login for trainee5.
Task 02
Which evidence best supports a privileged SSH configuration change?
The sudo record at 09:04:03 plus the sshd_config modification time at 09:06:12.
Task 03
What event supports that the SSH service applied a configuration change?
The 09:05:20 service restart provides timing consistent with applying updated configuration.
Task 04
Which evidence suggests recurring scheduled activity?
The 09:10:00 cron execution of check.sh.
Task 05
Does the outbound connection alone prove malicious behavior?
No. It shows network activity by the scheduled process. Purpose must be established from scenario and additional evidence.
Task 06
What should be preserved before changing the cron job or SSH configuration?
Relevant auth/journal records, configuration contents and metadata, cron entry, script path and hash, process evidence, and network evidence.
Lesson checklist
Before moving on
I start with the exact forensic question.
I know when to use journalctl versus a text log.
I check whether auth.log and syslog actually exist before relying on them.
I can filter journal records by time, boot, unit, and priority.
I record hostname, source, timestamp, unit, user, and process details.
I distinguish successful authentication from proof of unauthorized access.
I correlate sudo activity with the exact command and nearby events.
I correlate service events with unit definitions, files, and process state.
I review package logs when software installation matters.
I use last, lastb, who, and w as supporting login evidence.
I preserve cron, SSH, and service evidence before hardening changes.
I remember that missing logs do not prove an event never happened.
Reflection
Check your Linux log reasoning
Question 1
Why can journalctl be useful even when /var/log/auth.log is missing?
Question 2
What does an accepted SSH login prove, and what does it not prove?
Question 3
Why should a sudo record be correlated with the exact command and nearby configuration evidence?
Question 4
How can a package installation log help explain a newly created service?
Question 5
Why is shell history weaker than system logging?
Question 6
What evidence would you preserve before deleting an unexpected cron job?
Carry this forward
Start with journalctl, verify which text logs exist, filter by time and unit, read authentication and sudo records carefully, correlate with files and services, and preserve evidence before changing SSH, cron, accounts, or service configuration.