Debian Forums Cache - User contributions [en]

Debian First Aid Kit

2026-05-19T14:40:52Z

Distro-nix: Repair filesystem rewrite

'''Author''' '''Profile:''' [https://forums.debian.net/memberlist.php?mode=viewprofile&u=239099 distro-nix] '''Debian Version:''' Debian 13 (Trixie) '''Kernel Versions Tested:''' 6.12.73+deb13-amd64 and 6.16.3+deb13-amd64 (x86_64) '''Created:''' 2025-10-27 '''Last Updated:''' 2026-05-15 '''ID:''' 544201 '''License:''' CC BY-SA 4.0

''This guide is preserved in its original form. Questions and feedback welcome via the author’s <dfn>Debian Forums profile</dfn>.''

== Abstract ==
The Debian First Aid Kit is a practical reference for diagnosing and resolving common system problems on Debian GNU/Linux. It covers commonly occuring categories of failure: system freezes and crashes, boot problems, network issues, package management, disk and filesystem errors, performance degradation, service and application failures, permission and access problems, hardware identification and errors, and quick diagnostics. Each section gives the relevant commands so you can understand what they do and why, rather than a bare list to copy and paste blindly. A dedicated section covers MTR (Matt’s Traceroute) in depth, including how to interpret the output; practical troubleshooting, and how to monitor continuously. The guide also includes a set of shell aliases for faster access to common diagnostic tasks. All commands are verified on Debian 13 (Trixie) running kernel 6.12.73+deb13-amd64 and 6.16.3+deb13-amd64. This document is intended for users with basic command-line familiarity who need a reliable, consolidated reference when something goes wrong.

'''Keywords:''' Debian, troubleshooting, system administration, MTR, network diagnostics, package management, journalctl, filesystem repair
----

== Tested Commands Disclaimer ==
All commands in this document have been verified on Debian 13 (Trixie) with the kernel versions listed above. Results may vary on other releases or hardware configurations. Always test in a safe environment before making changes to a production system.

== Conventions Used in This Document ==
Command examples throughout this guide follow standard Unix prompt conventions. A $ prefix indicates a command run as a regular user. Where elevated privileges are required, sudo is shown explicitly and # with a brief helpful explanation unless it is self-explanatory:
$ journalctl -b -1 # regular user
$ sudo apt install mtr # requires elevation via sudo
In recovery shell contexts, where you are operating directly as root, the prompt is #. In these cases, Debian recommends:
$ sudo -i

== System Freezes & Crashes ==

=== Check System Logs ===

=== Common Freeze Causes to Look For ===

* '''Kernel panics:''' Search for “kernel panic” or “Oops”
* '''Out of Memory (OOM):''' Search for “Out of memory” or “oom-killer”
* '''Hardware errors:''' Look for “MCE” (Machine Check Exception) or “hardware error”
* '''Driver issues:''' Check for module/driver failures
* '''Overheating:''' Check system temperatures

=== Check System Resources ===
$ free -h # memory usage

$ df -h # disk space

$ htop # interactive resource monitor

$ top # alternative resource monitor

$ sudo apt install btop # btop is a more detailed alternative

$ sudo dmesg | grep -i error # check for disk errors

=== Access Recovery Mode ===
Reboot and hold Shift to access the GRUB menu (timing depends on your GRUB configuration). Select “Advanced options”, then choose recovery mode. From the recovery menu, select “root” for a root shell.

=== Common Boot Fixes ===
$ sudo fsck /dev/sdXN # repair filesystem — identify device with lsblk first

$ sudo grub-install /dev/sdX # reinstall GRUB bootloader

$ sudo update-grub # regenerate GRUB configuration

$ cat /etc/fstab # check fstab for mount errors

The following are permanent errors caused by incomplete or buggy ACPI tables in the BIOS. They are harmless and can be safely ignored if they appear on your system:
[0.686554] ACPI Error: No handler for Region [ECRM] (00000000201accc4) [EmbeddedControl] (20250404/evregion-131)
[0.686577] ACPI Error: Region EmbeddedControl (ID=3) has no handler (20250404/exfldio-261)
[0.686594] ACPI Error: Aborting method \_SB.GPIO._EVT due to previous error (AE_NOT_EXIST) (20250404/psparse-529)

== Boot Problems ==

=== Check Boot Process ===
$ systemd-analyze blame # boot time by unit

$ systemd-analyze blame # boot time by unit

$ systemctl --failed # units that failed during boot

$ systemctl status <service-name> # e.g. systemctl status NetworkManager.service

=== Access Recovery Mode ===
Reboot and hold Shift to access the GRUB menu (timing depends on your GRUB configuration). Select “Advanced options”, then choose recovery mode. From the recovery menu, select “root” for a root shell.

=== Common Boot Fixes ===
$ sudo fsck /dev/sdXN # repair filesystem — identify device with lsblk first

$ sudo grub-install /dev/sdX # reinstall GRUB bootloader

$ sudo update-grub # regenerate GRUB configuration

$ cat /etc/fstab # check fstab for mount errors

== Network Issues ==

=== Diagnose Network Connection ===
$ ip addr show # check network interfaces

$ ping -c 4 8.8.8.8 # test IPv4 connectivity

$ ping -c 6 2a00:1450:4007:809::200e # test IPv6 connectivity

$ nslookup google.com # check DNS resolution

$ ip route show # view routing table

$ ss -tuln # check active connections

=== Restart Network Service ===
$ sudo systemctl restart NetworkManager # for systems using NetworkManager

$ sudo systemctl restart networking # for systems using networking service

$ sudo ip link set eth0 down # bring interface down

$ sudo ip link set eth0 up # bring interface back up

== MTR (Matt’s Traceroute) ==
MTR combines the functionality of ping and traceroute into a single real-time network diagnostic tool. It continuously monitors the path between your system and a destination, providing detailed statistics about latency and packet loss at each hop. When you need to demonstrate a network problem that is outside your control—to your hosting provider, for example—MTR gives you the evidence you need.

=== Installation ===
$ sudo apt install mtr

=== Basic Usage ===
$ mtr google.com # interactive mode

$ mtr --report google.com # report mode (10 cycles then exit)

$ mtr --report-cycles 50 google.com # specify number of cycles

$ mtr --tcp google.com # use TCP instead of ICMP

$ mtr --udp google.com # use UDP

$ mtr --no-dns google.com # skip DNS resolution (faster)

$ mtr --show-ips google.com # show both hostnames and IPs

== Understanding MTR Output ==

==== Sample Output ====
HOST: hostname Loss% Snt Last Avg Best Wrst StDev
1.|-- 192.168.1.1 0.0% 10 1.2 1.5 1.0 2.3 0.4
2.|-- 10.0.0.1 0.0% 10 8.5 9.2 7.8 12.1 1.3
3.|-- isp-gateway.net 0.0% 10 15.3 16.1 14.2 19.8 1.8
4.|-- ??? 100.0% 10 0.0 0.0 0.0 0.0 0.0
5.|-- google.com 0.0% 10 25.4 26.8 24.1 32.5 2.4

=== Column Meanings ===
'''HOST''' — Hostname or IP address of each hop in the route.

'''Loss%''' — Percentage of packets lost at this hop.

'''Snt''' — Number of packets sent to this hop.

'''Last''' — Latency of the most recent packet, in milliseconds.

'''Avg''' — Average latency across all packets sent, in milliseconds.

'''Best''' — Lowest latency recorded, in milliseconds.

'''Wrst''' — Highest latency recorded, in milliseconds.

'''StDev''' — Standard deviation of latency measurements—lower values indicate a more stable connection.

=== Interpreting Results ===
A healthy network shows 0% loss on all hops, stable latency with low StDev values, a gradual latency increase as hop count increases, and consistent response times.

==== Problem Indicators ====
'''1. High Packet Loss at a Specific Hop'''
5.|-- problem-router.net 25.0% 10 45.3 48.2 42.1 65.8 8.4
If loss continues to the destination, there is a real problem at that router. If loss appears only at that hop but not beyond, the router is likely rate-limiting ICMP responses—a false positive, not a real problem. The rule of thumb: if packet loss appears at hop N but hops N+1, N+2, and beyond show 0% loss, it’s almost certainly ICMP rate limiting.

'''2. High Latency at a Specific Hop'''
3.|-- slow-link.net 0.0% 10 150.3 155.2 148.1 165.8 5.4
This indicates a network bottleneck, a congested link, geographical distance (intercontinental hops), or slow routing equipment.

'''3. No Response (???)'''
4.|-- ??? 100.0% 10 0.0 0.0 0.0 0.0 0.0
The router at this hop is configured not to respond to ICMP or traceroute packets, or a firewall is blocking diagnostic packets. This is not necessarily a problem if later hops respond normally.

'''4. High Jitter (StDev)'''
6.|-- unstable.net 0.0% 10 35.3 52.8 28.1 95.2 24.7
A high StDev (24.7ms in this example) indicates inconsistent latency caused by network congestion or instability. This is particularly noticeable in real-time applications such as VoIP, gaming, and video calls.

'''5. Sudden Latency Spike'''
1.|-- 192.168.1.1 0.0% 10 1.2 1.5 1.0 2.3 0.4
2.|-- 10.0.0.1 0.0% 10 8.5 9.2 7.8 12.1 1.3
3.|-- problematic-hop.net 0.0% 10 180.5 185.2 178.1 195.8 6.4
4.|-- next-hop.net 0.0% 10 182.3 187.8 180.5 198.2 6.8
Hop 3 introduces approximately 170ms of additional latency (jumping from 9ms to 180ms). Everything from hop 3 onward carries that penalty.

=== Advanced Usage ===

==== Report Mode with Different Output Formats ====
$ mtr --report --csv google.com > network-report.csv # CSV for logging
$ mtr --report --json google.com # JSON for parsing
$ mtr --report --xml google.com # XML format
$ mtr --report-wide google.com # wide report, no abbreviations

==== Protocol Selection ====
$ mtr google.com # ICMP (default)
$ mtr --udp google.com # UDP
$ sudo mtr --tcp --port 443 google.com # specific TCP port
$ sudo mtr --tcp --port 22 remote-server.com # generally the default port

==== Timing and Duration ====
$ mtr --interval 0.5 google.com # interval between pings (default 1s)
$ mtr --report-cycles 100 google.com # extended test, 100 cycles
$ mtr google.com # continuous monitoring (Ctrl+C to stop)
$ mtr --report-cycles 10 google.com # quick 10-cycle report

==== Advanced Options ====
$ mtr --aslookup google.com # show Autonomous System (AS) numbers
$ mtr --max-ttl 20 google.com # set maximum number of hops
$ mtr --psize 1000 google.com # set packet size
$ mtr --show-ips google.com # show both IP and hostname
$ mtr --address 192.168.1.100 google.com # specify source address
$ mtr -4 google.com # IPv4 only
$ mtr -6 google.com # IPv6 only

=== Interactive Mode Commands ===
When running MTR in interactive mode (mtr hostname), the following keys are available:

'''h''' — Display help.

'''d''' — Toggle display mode, cycling through different views.

'''n''' — Toggle between hostnames and IP addresses.

'''r''' — Reset all statistics.

'''p''' — Pause/unpause the display.

'''q''' — Quit MTR.

'''u''' — Switch between ICMP, UDP, and TCP modes.

'''y''' — Switch between IPv4 and IPv6.

'''o''' — Toggle field display options.

'''j''' — Toggle latency display.

=== Practical Troubleshooting Scenarios ===

==== Scenario 1: Diagnosing a Slow Website ====
$ mtr --report-cycles 100 --no-dns example.com
Look for high average latency at specific hops, packet loss at the destination, and high StDev values indicating jitter.

==== Scenario 2: Testing if a Firewall Blocks SSH ====
$ sudo mtr --tcp --port 22 --report-cycles 50 remote-server.com
If the last hop shows 100% loss but earlier hops are fine, port 22 may be filtered. Run a standard ICMP test for comparison.

==== Scenario 3: ISP Performance Issues ====
$ mtr --report-cycles 100 8.8.8.8

$ mtr --report-cycles 100 1.1.1.1

If issues appear in the first three or four hops, the problem is likely within your ISP’s network. If issues appear later, the problem lies beyond your ISP.

==== Scenario 4: VPN Troubleshooting ====
$ mtr --report-cycles 50 --no-dns google.com > before-vpn.txt # before VPN
$ mtr --report-cycles 50 --no-dns google.com > after-vpn.txt # after VPN
$ diff before-vpn.txt after-vpn.txt # compare the two

==== Scenario 5: Gaming/Streaming Performance ====
$ mtr --report-cycles 200 game-server.com
For real-time applications, look for low average latency (under 50ms for gaming), low StDev (under 5ms preferred), and zero packet loss.

==== Scenario 6: Intermittent Connectivity ====
$ mtr --report-cycles 500 --interval 1 target.com > long-test.txt # long-running test
$ mtr target.com # real-time monitoring

=== Continuous Monitoring ===

==== Log Network Performance Over Time ====
while true; do
timestamp=$(date +%Y%m%d-%H%M%S)
mtr --report --report-cycles 50 google.com > "mtr-$timestamp.txt"
sleep 3600
done

==== Monitor Multiple Destinations ====
#!/bin/bash
echo "=== MTR Report $(date) ===" > daily-network-report.txt
echo "" >> daily-network-report.txt

echo "Google DNS:" >> daily-network-report.txt
mtr --report --report-cycles 50 --no-dns 8.8.8.8 >> daily-network-report.txt
echo "" >> daily-network-report.txt

echo "Cloudflare DNS:" >> daily-network-report.txt
mtr --report --report-cycles 50 --no-dns 1.1.1.1 >> daily-network-report.txt
echo "" >> daily-network-report.txt

echo "Your Server:" >> daily-network-report.txt
mtr --report --report-cycles 50 your-server.com >> daily-network-report.txt

=== Useful Aliases for .bashrc ===
Quick network path analysis
alias mtrreport='mtr --report --report-cycles 50 --no-dns'

Monitor connection to Google DNS
alias netcheck='mtr --report-cycles 20 8.8.8.8'

Extended network test
alias mtrlong='mtr --report-cycles 100'

TCP port 443 test (HTTPS)
alias mtrhttps='sudo mtr --tcp --port 443 --report-cycles 30'

Quick comparison of major DNS providers
alias dnstest='echo "Google:" && mtr --report-cycles 20 8.8.8.8 && echo -e "\nCloudflare:" && mtr --report-cycles 20 1.1.1.1'
After adding to ~/.bashrc, reload with:
$ source ~/.bashrc

==== Useful Tip for Aliases ====
Rather than adding aliases directly to <code>~/.bashrc</code>, it's cleaner to keep them in a dedicated file: <code>~/.bashrc_aliases</code> and source that file from <code>~/.bashrc</code>. This keeps your shell configuration tidy and makes it easy to share or back up just your aliases without touching the rest of your environment.

=== Troubleshooting Tips ===
'''Permission issues with TCP mode:'''
$ sudo mtr --tcp --port 443 example.com # Use sudo for TCP on privileged ports
$ sudo setcap cap_net_raw+ep /usr/bin/mtr-packet # Or set capabilities (one-time setup)
'''False positives:''' Packet loss at an intermediate hop but not at the destination is almost always ICMP rate limiting at that router, not a real problem. Example:
3.|-- router.isp.net 20.0% 50 15.3 16.1 14.2 19.8 1.8
4.|-- next-hop.net 0.0% 50 18.5 19.2 17.8 22.1 1.3
5.|-- destination.com 0.0% 50 25.4 26.8 24.1 32.5 2.4
Hop 3 shows 20% loss, but hops 4 and 5 show 0%. Traffic is flowing normally.

'''DNS resolution delays:''' If MTR is slow to start, skip DNS resolution with mtr --no-dns target.com and resolve addresses afterward with host 203.0.113.1 if needed.

'''Comparing results:'''
$ mtr --report --report-cycles 50 example.com > test1.txt
$ sleep 60
$ mtr --report --report-cycles 50 example.com > test2.txt
$ diff test1.txt test2.txt

=== When to Use MTR vs Other Tools ===
'''MTR''' — Best for continuous monitoring, identifying problem hops, and gathering detailed statistics. Requires installation via apt.

'''ping''' — Best for a quick connectivity test or simple latency check. Only tests the endpoint, not the path.

'''traceroute''' — Best for one-time path discovery. Provides no continuous monitoring.

'''ss / netstat''' — Best for checking local connection status. Doesn’t test remote paths.

=== Best Practices ===
Run at least 50–100 cycles for accurate statistics. Use --no-dns for faster results and to avoid DNS resolution issues during testing. Test at different times of day, since network conditions vary. If one protocol shows issues, compare ICMP, UDP, and TCP modes. Save reports with timestamps for trend analysis. Use 8.8.8.8 or 1.1.1.1 as known-good hosts to verify your local network first. Let MTR run for at least 30–60 seconds before drawing conclusions.

=== Reading Between the Lines ===
'''Good network health:'''
HOST: hostname Loss% Snt Last Avg Best Wrst StDev
1.|-- 192.168.1.1 0.0% 50 1.1 1.2 0.9 2.1 0.2
2.|-- 10.0.0.1 0.0% 50 8.2 8.5 7.5 10.2 0.5
3.|-- isp-gateway.net 0.0% 50 15.1 15.5 14.0 18.3 0.8
4.|-- google.com 0.0% 50 24.8 25.2 23.5 28.1 1.1
No packet loss, consistent latency, low jitter throughout.

'''Problematic network:'''
HOST: hostname Loss% Snt Last Avg Best Wrst StDev
1.|-- 192.168.1.1 0.0% 50 1.2 1.5 1.0 2.3 0.3
2.|-- 10.0.0.1 5.0% 50 45.3 52.8 8.1 245.2 45.7
3.|-- ??? 100.0% 50 0.0 0.0 0.0 0.0 0.0
4.|-- destination.com 15.0% 50 95.4 125.8 48.2 385.5 78.2
Packet loss at hop 2 and the destination, high jitter, and a very high worst-case latency all point to a genuine problem at or near hop 2.

=== Summary ===
MTR is your friend when it comes to diagnostics. Use --report-cycles 50 or higher for reliable data. Watch for packet loss at the destination—intermediate losses may be false positives. High StDev indicates an unstable connection. High average latency points to slow links. Use --no-dns for faster results. Compare ICMP, UDP, and TCP modes if issues appear. Test at different times of day for a complete picture.
== Package Management Issues ==

=== Fix Broken Packages ===
$ sudo apt update # update package lists
$ sudo apt --fix-broken install # fix broken dependencies
$ sudo dpkg --configure -a # reconfigure packages (no output = nothing to do)
$ sudo apt clean # clean package cache
$ sudo apt autoclean
$ sudo apt autoremove # remove unused packages

=== Handle Held or Locked Packages ===
$ sudo lsof /var/lib/dpkg/lock-frontend # find process holding the lock
$ sudo rm /var/lib/dpkg/lock-frontend # remove lock (only if no package manager is running)
$ sudo rm /var/lib/apt/lists/lock
$ sudo dpkg --configure -a # reconfigure dpkg

== Disk & Filesystem Issues ==

=== Check Disk Health ===
$ df -h # disk space
$ df -i # inode usage
$ sudo apt install sysstat # sysstat includes iostat, mpstat, sar, pidstat, cifsiostat
$ iostat -x 1 # disk I/O statistics
$ iostat -xm 2 # stats in MB instead of KB
$ iostat -x sda 1 # monitor a specific device
$ sudo dmesg | grep -i "error\|fail" # check for disk errors
$ sudo apt install smartmontools
$ sudo smartctl -a /dev/sdX # SMART disk health - lsblk to determine the target device 

=== Repair Filesystem ===
$ sudo umount /dev/sdXN # unmount the partition first
$ sudo fsck /dev/sdXN # run filesystem check
$ sudo e2fsck -f /dev/sdXN # for ext4 specifically
$ sudo apt --fix-broken install # Fix broken dependencies
$ sudo dpkg --configure -a # Reconfigure packages

# No output means there is nothing to reconfigure

# Clean package cache
sudo apt clean
sudo apt autoclean

# Remove unused packages
sudo apt autoremove

=== Handle Held or Locked Packages ===
# If apt is locked, find the process holding the lock
sudo lsof /var/lib/dpkg/lock-frontend

# Remove the lock (use with caution—only if no package manager is running)
sudo rm /var/lib/dpkg/lock-frontend
sudo rm /var/lib/apt/lists/lock

# Reconfigure dpkg
sudo dpkg --configure -a
Disk & Filesystem Issues

=== Check Disk Health ===
# Check disk space
df -h

# Check inode usage
df -i

# View disk I/O statistics
# iostat is part of the sysstat package, which also includes:
# mpstat (CPU statistics), sar (system activity reporter),
# pidstat (process statistics), and cifsiostat (CIFS statistics)
sudo apt install sysstat

iostat -x 1

# Show stats in MB instead of KB
iostat -xm 2

# Monitor a specific device
iostat -x sda 1

# Check for disk errors in dmesg
sudo dmesg | grep -i "error\|fail"

# SMART disk health (requires smartmontools)
sudo apt install smartmontools
sudo smartctl -a /dev/sda

=== Repair Filesystem ===
# Unmount the partition first
sudo umount /dev/sdXN

# Run filesystem check
sudo fsck /dev/sdXN

# For ext4 specifically
sudo e2fsck -f /dev/sdXN
Performance Issues

=== Identify Resource Hogs ===
# CPU usage by process
top -o %CPU

# Memory usage by process
top -o %MEM

# Disk usage by directory
du -sh /* | sort -h

# Find large files
find / -type f -size +100M 2>/dev/null

# Check running processes sorted by memory use
ps aux --sort=-%mem | head -20

=== System Temperature Monitoring ===
# Install lm-sensors if not already present
sudo apt install lm-sensors
sudo sensors-detect

# View temperatures
sensors

# Real-time temperature monitoring
watch -n 2 sensors
See section 12 for a shell alias that makes temperature monitoring more convenient.

Service & Application Errors

=== Debug Service Problems ===
# Check service status
sudo systemctl status service-name

# View service logs
sudo journalctl -u service-name

# Restart a service
sudo systemctl restart service-name

# Enable service at boot
sudo systemctl enable service-name

# View recent service failures
journalctl -p err -b

=== Application Crash Investigation ===
# Check for core dumps
ls -lh /var/crash/

# View available application logs
ls /var/log/

# Follow syslog for application errors in real-time
sudo tail -f /var/log/syslog
Permission & Access Issues

=== Fix Common Permission Problems ===
# Check file ownership
ls -l /path/to/file

# Change ownership (user:group)
sudo chown user:group /path/to/file

# Change permissions
sudo chmod 644 /path/to/file

# Recursively fix permissions on a directory
sudo chown -R user:group /path/to/directory

=== User & Authentication Issues ===
# Check user information
id username

# View user login history
last -a

# Check failed login attempts
sudo journalctl | grep "authentication failure"

# Reset user password
sudo passwd username
Hardware Issues

=== Identify Hardware ===
# List all hardware (install if needed: sudo apt install lshw)
sudo lshw -short

# PCI devices
lspci -v

# USB devices
lsusb -v

# CPU information
lscpu

# Memory information
sudo dmidecode --type memory

=== Check Hardware Errors ===
# Kernel ring buffer (hardware messages)
dmesg | less
# Press q to quit

# Search for specific hardware issues
dmesg | grep -i "error\|fail\|warn"

# Check for USB issues
dmesg | grep -i usb
Quick Diagnostic Commands

=== System Information at a Glance ===
# Uptime and load average
uptime

# Kernel version
uname -r

# Debian version
cat /etc/debian_version

# Full system summary (install if needed: sudo apt install inxi)
sudo inxi -Fxz

=== Emergency Toolkit ===
# Create a consolidated diagnostic report
sudo journalctl -b > ~/system-report.txt
dmesg >> ~/system-report.txt
systemctl --failed >> ~/system-report.txt
df -h >> ~/system-report.txt
free -h >> ~/system-report.txt

# Watch logs in real-time
sudo journalctl -f

# Monitor system resources continuously
watch -n 1 'free -h && df -h'
Useful Aliases & Shortcuts
Add these to your ~/.bashrc for quick access to common troubleshooting commands:
# Monitor system temperatures in real-time
alias temps="watch -n 2 'for i in /sys/class/hwmon/hwmon*/; do echo -n \"\$(cat \${i}name): \"; cat \${i}temp*_input 2>/dev/null | while read temp; do echo \"scale=1; \$temp/1000\" | bc; done | tr \"\n\" \" \"; echo \"°C\"; done'"

# Quick system status
alias sysstat='echo "=== CPU ===" && uptime && echo -e "\n=== Memory ===" && free -h && echo -e "\n=== Disk ===" && df -h && echo -e "\n=== Top Processes ===" && ps aux --sort=-%mem | head -10'

# View last boot logs
alias lastboot='journalctl -b -1'

# Check failed services
alias failedservices='systemctl --failed'

# Monitor logs in real-time
alias watchlog='sudo journalctl -f'

# Quick network status
alias netstatus='ip addr show && echo -e "\n=== Routes ===" && ip route show'
After adding these, reload your shell configuration:
source ~/.bashrc
Tips for Effective Troubleshooting

'''Check logs first.''' journalctl and dmesg are your best starting point for almost any problem.

'''Change one thing at a time.''' It’s tempting to make several changes at once, but doing so makes it impossible to know what actually fixed—or broke—the problem.

'''Document your changes.''' Keep notes on what you’ve tried and what the results were. You’ll thank yourself later.

'''Use exact error messages.''' Copy error messages verbatim and search for them alongside “Debian” and the version number—for example, “Debian 13” or “Debian 13.1” for point releases.

'''Check what changed recently.''' Did you install something, update packages, or change the kernel before the problem appeared? Start there.

'''Back up before making major changes.''' This goes without saying, but it’s easy to skip when you’re in a hurry.

'''Use verbose mode.''' Adding -v or -vv flags to commands often reveals what would otherwise be silent failures.

'''Use the community.''' The Debian forums, Reddit, Stack Exchange, and the Debian mailing lists are all active and well-indexed. Someone has usually seen your problem before.

''This work was contributed by <dfn>distro-nix</dfn> to the Debian User Forums on 2025-10-27.''

Debian First Aid Kit

2026-05-19T14:16:35Z

Distro-nix: Updating sections

'''Author''' '''Profile:''' [https://forums.debian.net/memberlist.php?mode=viewprofile&u=239099 distro-nix] '''Debian Version:''' Debian 13 (Trixie) '''Kernel Versions Tested:''' 6.12.73+deb13-amd64 and 6.16.3+deb13-amd64 (x86_64) '''Created:''' 2025-10-27 '''Last Updated:''' 2026-05-15 '''ID:''' 544201 '''License:''' CC BY-SA 4.0

''This guide is preserved in its original form. Questions and feedback welcome via the author’s <dfn>Debian Forums profile</dfn>.''

== Abstract ==
The Debian First Aid Kit is a practical reference for diagnosing and resolving common system problems on Debian GNU/Linux. It covers commonly occuring categories of failure: system freezes and crashes, boot problems, network issues, package management, disk and filesystem errors, performance degradation, service and application failures, permission and access problems, hardware identification and errors, and quick diagnostics. Each section gives the relevant commands so you can understand what they do and why, rather than a bare list to copy and paste blindly. A dedicated section covers MTR (Matt’s Traceroute) in depth, including how to interpret the output; practical troubleshooting, and how to monitor continuously. The guide also includes a set of shell aliases for faster access to common diagnostic tasks. All commands are verified on Debian 13 (Trixie) running kernel 6.12.73+deb13-amd64 and 6.16.3+deb13-amd64. This document is intended for users with basic command-line familiarity who need a reliable, consolidated reference when something goes wrong.

'''Keywords:''' Debian, troubleshooting, system administration, MTR, network diagnostics, package management, journalctl, filesystem repair
----

== Tested Commands Disclaimer ==
All commands in this document have been verified on Debian 13 (Trixie) with the kernel versions listed above. Results may vary on other releases or hardware configurations. Always test in a safe environment before making changes to a production system.

== Conventions Used in This Document ==
Command examples throughout this guide follow standard Unix prompt conventions. A $ prefix indicates a command run as a regular user. Where elevated privileges are required, sudo is shown explicitly and # with a brief helpful explanation unless it is self-explanatory:
$ journalctl -b -1 # regular user
$ sudo apt install mtr # requires elevation via sudo
In recovery shell contexts, where you are operating directly as root, the prompt is #. In these cases, Debian recommends:
$ sudo -i

== System Freezes & Crashes ==

=== Check System Logs ===

=== Common Freeze Causes to Look For ===

* '''Kernel panics:''' Search for “kernel panic” or “Oops”
* '''Out of Memory (OOM):''' Search for “Out of memory” or “oom-killer”
* '''Hardware errors:''' Look for “MCE” (Machine Check Exception) or “hardware error”
* '''Driver issues:''' Check for module/driver failures
* '''Overheating:''' Check system temperatures

=== Check System Resources ===
$ free -h # memory usage

$ df -h # disk space

$ htop # interactive resource monitor

$ top # alternative resource monitor

$ sudo apt install btop # btop is a more detailed alternative

$ sudo dmesg | grep -i error # check for disk errors

=== Access Recovery Mode ===
Reboot and hold Shift to access the GRUB menu (timing depends on your GRUB configuration). Select “Advanced options”, then choose recovery mode. From the recovery menu, select “root” for a root shell.

=== Common Boot Fixes ===
$ sudo fsck /dev/sdXN # repair filesystem — identify device with lsblk first

$ sudo grub-install /dev/sdX # reinstall GRUB bootloader

$ sudo update-grub # regenerate GRUB configuration

$ cat /etc/fstab # check fstab for mount errors

The following are permanent errors caused by incomplete or buggy ACPI tables in the BIOS. They are harmless and can be safely ignored if they appear on your system:
[0.686554] ACPI Error: No handler for Region [ECRM] (00000000201accc4) [EmbeddedControl] (20250404/evregion-131)
[0.686577] ACPI Error: Region EmbeddedControl (ID=3) has no handler (20250404/exfldio-261)
[0.686594] ACPI Error: Aborting method \_SB.GPIO._EVT due to previous error (AE_NOT_EXIST) (20250404/psparse-529)

== Boot Problems ==

=== Check Boot Process ===
$ systemd-analyze blame # boot time by unit

$ systemd-analyze blame # boot time by unit

$ systemctl --failed # units that failed during boot

$ systemctl status <service-name> # e.g. systemctl status NetworkManager.service

=== Access Recovery Mode ===
Reboot and hold Shift to access the GRUB menu (timing depends on your GRUB configuration). Select “Advanced options”, then choose recovery mode. From the recovery menu, select “root” for a root shell.

=== Common Boot Fixes ===
$ sudo fsck /dev/sdXN # repair filesystem — identify device with lsblk first

$ sudo grub-install /dev/sdX # reinstall GRUB bootloader

$ sudo update-grub # regenerate GRUB configuration

$ cat /etc/fstab # check fstab for mount errors

== Network Issues ==

=== Diagnose Network Connection ===
$ ip addr show # check network interfaces

$ ping -c 4 8.8.8.8 # test IPv4 connectivity

$ ping -c 6 2a00:1450:4007:809::200e # test IPv6 connectivity

$ nslookup google.com # check DNS resolution

$ ip route show # view routing table

$ ss -tuln # check active connections

=== Restart Network Service ===
$ sudo systemctl restart NetworkManager # for systems using NetworkManager

$ sudo systemctl restart networking # for systems using networking service

$ sudo ip link set eth0 down # bring interface down

$ sudo ip link set eth0 up # bring interface back up

== MTR (Matt’s Traceroute) ==
MTR combines the functionality of ping and traceroute into a single real-time network diagnostic tool. It continuously monitors the path between your system and a destination, providing detailed statistics about latency and packet loss at each hop. When you need to demonstrate a network problem that is outside your control—to your hosting provider, for example—MTR gives you the evidence you need.

=== Installation ===
$ sudo apt install mtr

=== Basic Usage ===
$ mtr google.com # interactive mode

$ mtr --report google.com # report mode (10 cycles then exit)

$ mtr --report-cycles 50 google.com # specify number of cycles

$ mtr --tcp google.com # use TCP instead of ICMP

$ mtr --udp google.com # use UDP

$ mtr --no-dns google.com # skip DNS resolution (faster)

$ mtr --show-ips google.com # show both hostnames and IPs

== Understanding MTR Output ==

==== Sample Output ====
HOST: hostname Loss% Snt Last Avg Best Wrst StDev
1.|-- 192.168.1.1 0.0% 10 1.2 1.5 1.0 2.3 0.4
2.|-- 10.0.0.1 0.0% 10 8.5 9.2 7.8 12.1 1.3
3.|-- isp-gateway.net 0.0% 10 15.3 16.1 14.2 19.8 1.8
4.|-- ??? 100.0% 10 0.0 0.0 0.0 0.0 0.0
5.|-- google.com 0.0% 10 25.4 26.8 24.1 32.5 2.4

=== Column Meanings ===
'''HOST''' — Hostname or IP address of each hop in the route.

'''Loss%''' — Percentage of packets lost at this hop.

'''Snt''' — Number of packets sent to this hop.

'''Last''' — Latency of the most recent packet, in milliseconds.

'''Avg''' — Average latency across all packets sent, in milliseconds.

'''Best''' — Lowest latency recorded, in milliseconds.

'''Wrst''' — Highest latency recorded, in milliseconds.

'''StDev''' — Standard deviation of latency measurements—lower values indicate a more stable connection.

=== Interpreting Results ===
A healthy network shows 0% loss on all hops, stable latency with low StDev values, a gradual latency increase as hop count increases, and consistent response times.

==== Problem Indicators ====
'''1. High Packet Loss at a Specific Hop'''
5.|-- problem-router.net 25.0% 10 45.3 48.2 42.1 65.8 8.4
If loss continues to the destination, there is a real problem at that router. If loss appears only at that hop but not beyond, the router is likely rate-limiting ICMP responses—a false positive, not a real problem. The rule of thumb: if packet loss appears at hop N but hops N+1, N+2, and beyond show 0% loss, it’s almost certainly ICMP rate limiting.

'''2. High Latency at a Specific Hop'''
3.|-- slow-link.net 0.0% 10 150.3 155.2 148.1 165.8 5.4
This indicates a network bottleneck, a congested link, geographical distance (intercontinental hops), or slow routing equipment.

'''3. No Response (???)'''
4.|-- ??? 100.0% 10 0.0 0.0 0.0 0.0 0.0
The router at this hop is configured not to respond to ICMP or traceroute packets, or a firewall is blocking diagnostic packets. This is not necessarily a problem if later hops respond normally.

'''4. High Jitter (StDev)'''
6.|-- unstable.net 0.0% 10 35.3 52.8 28.1 95.2 24.7
A high StDev (24.7ms in this example) indicates inconsistent latency caused by network congestion or instability. This is particularly noticeable in real-time applications such as VoIP, gaming, and video calls.

'''5. Sudden Latency Spike'''
1.|-- 192.168.1.1 0.0% 10 1.2 1.5 1.0 2.3 0.4
2.|-- 10.0.0.1 0.0% 10 8.5 9.2 7.8 12.1 1.3
3.|-- problematic-hop.net 0.0% 10 180.5 185.2 178.1 195.8 6.4
4.|-- next-hop.net 0.0% 10 182.3 187.8 180.5 198.2 6.8
Hop 3 introduces approximately 170ms of additional latency (jumping from 9ms to 180ms). Everything from hop 3 onward carries that penalty.

=== Advanced Usage ===

==== Report Mode with Different Output Formats ====
$ mtr --report --csv google.com > network-report.csv # CSV for logging
$ mtr --report --json google.com # JSON for parsing
$ mtr --report --xml google.com # XML format
$ mtr --report-wide google.com # wide report, no abbreviations

==== Protocol Selection ====
$ mtr google.com # ICMP (default)
$ mtr --udp google.com # UDP
$ sudo mtr --tcp --port 443 google.com # specific TCP port
$ sudo mtr --tcp --port 22 remote-server.com # generally the default port

==== Timing and Duration ====
$ mtr --interval 0.5 google.com # interval between pings (default 1s)
$ mtr --report-cycles 100 google.com # extended test, 100 cycles
$ mtr google.com # continuous monitoring (Ctrl+C to stop)
$ mtr --report-cycles 10 google.com # quick 10-cycle report

==== Advanced Options ====
$ mtr --aslookup google.com # show Autonomous System (AS) numbers
$ mtr --max-ttl 20 google.com # set maximum number of hops
$ mtr --psize 1000 google.com # set packet size
$ mtr --show-ips google.com # show both IP and hostname
$ mtr --address 192.168.1.100 google.com # specify source address
$ mtr -4 google.com # IPv4 only
$ mtr -6 google.com # IPv6 only

=== Interactive Mode Commands ===
When running MTR in interactive mode (mtr hostname), the following keys are available:

'''h''' — Display help.

'''d''' — Toggle display mode, cycling through different views.

'''n''' — Toggle between hostnames and IP addresses.

'''r''' — Reset all statistics.

'''p''' — Pause/unpause the display.

'''q''' — Quit MTR.

'''u''' — Switch between ICMP, UDP, and TCP modes.

'''y''' — Switch between IPv4 and IPv6.

'''o''' — Toggle field display options.

'''j''' — Toggle latency display.

=== Practical Troubleshooting Scenarios ===

==== Scenario 1: Diagnosing a Slow Website ====
$ mtr --report-cycles 100 --no-dns example.com
Look for high average latency at specific hops, packet loss at the destination, and high StDev values indicating jitter.

==== Scenario 2: Testing if a Firewall Blocks SSH ====
$ sudo mtr --tcp --port 22 --report-cycles 50 remote-server.com
If the last hop shows 100% loss but earlier hops are fine, port 22 may be filtered. Run a standard ICMP test for comparison.

==== Scenario 3: ISP Performance Issues ====
$ mtr --report-cycles 100 8.8.8.8

$ mtr --report-cycles 100 1.1.1.1

If issues appear in the first three or four hops, the problem is likely within your ISP’s network. If issues appear later, the problem lies beyond your ISP.

==== Scenario 4: VPN Troubleshooting ====
$ mtr --report-cycles 50 --no-dns google.com > before-vpn.txt # before VPN
$ mtr --report-cycles 50 --no-dns google.com > after-vpn.txt # after VPN
$ diff before-vpn.txt after-vpn.txt # compare the two

==== Scenario 5: Gaming/Streaming Performance ====
$ mtr --report-cycles 200 game-server.com
For real-time applications, look for low average latency (under 50ms for gaming), low StDev (under 5ms preferred), and zero packet loss.

==== Scenario 6: Intermittent Connectivity ====
$ mtr --report-cycles 500 --interval 1 target.com > long-test.txt # long-running test
$ mtr target.com # real-time monitoring

=== Continuous Monitoring ===

==== Log Network Performance Over Time ====
while true; do
timestamp=$(date +%Y%m%d-%H%M%S)
mtr --report --report-cycles 50 google.com > "mtr-$timestamp.txt"
sleep 3600
done

==== Monitor Multiple Destinations ====
#!/bin/bash
echo "=== MTR Report $(date) ===" > daily-network-report.txt
echo "" >> daily-network-report.txt

echo "Google DNS:" >> daily-network-report.txt
mtr --report --report-cycles 50 --no-dns 8.8.8.8 >> daily-network-report.txt
echo "" >> daily-network-report.txt

echo "Cloudflare DNS:" >> daily-network-report.txt
mtr --report --report-cycles 50 --no-dns 1.1.1.1 >> daily-network-report.txt
echo "" >> daily-network-report.txt

echo "Your Server:" >> daily-network-report.txt
mtr --report --report-cycles 50 your-server.com >> daily-network-report.txt

=== Useful Aliases for .bashrc ===
Quick network path analysis
alias mtrreport='mtr --report --report-cycles 50 --no-dns'

Monitor connection to Google DNS
alias netcheck='mtr --report-cycles 20 8.8.8.8'

Extended network test
alias mtrlong='mtr --report-cycles 100'

TCP port 443 test (HTTPS)
alias mtrhttps='sudo mtr --tcp --port 443 --report-cycles 30'

Quick comparison of major DNS providers
alias dnstest='echo "Google:" && mtr --report-cycles 20 8.8.8.8 && echo -e "\nCloudflare:" && mtr --report-cycles 20 1.1.1.1'
After adding to ~/.bashrc, reload with:
$ source ~/.bashrc

==== Useful Tip for Aliases ====
Rather than adding aliases directly to <code>~/.bashrc</code>, it's cleaner to keep them in a dedicated file: <code>~/.bashrc_aliases</code> and source that file from <code>~/.bashrc</code>. This keeps your shell configuration tidy and makes it easy to share or back up just your aliases without touching the rest of your environment.

=== Troubleshooting Tips ===
'''Permission issues with TCP mode:'''
$ sudo mtr --tcp --port 443 example.com # Use sudo for TCP on privileged ports
$ sudo setcap cap_net_raw+ep /usr/bin/mtr-packet # Or set capabilities (one-time setup)
'''False positives:''' Packet loss at an intermediate hop but not at the destination is almost always ICMP rate limiting at that router, not a real problem. Example:
3.|-- router.isp.net 20.0% 50 15.3 16.1 14.2 19.8 1.8
4.|-- next-hop.net 0.0% 50 18.5 19.2 17.8 22.1 1.3
5.|-- destination.com 0.0% 50 25.4 26.8 24.1 32.5 2.4
Hop 3 shows 20% loss, but hops 4 and 5 show 0%. Traffic is flowing normally.

'''DNS resolution delays:''' If MTR is slow to start, skip DNS resolution with mtr --no-dns target.com and resolve addresses afterward with host 203.0.113.1 if needed.

'''Comparing results:'''
$ mtr --report --report-cycles 50 example.com > test1.txt
$ sleep 60
$ mtr --report --report-cycles 50 example.com > test2.txt
$ diff test1.txt test2.txt

=== When to Use MTR vs Other Tools ===
'''MTR''' — Best for continuous monitoring, identifying problem hops, and gathering detailed statistics. Requires installation via apt.

'''ping''' — Best for a quick connectivity test or simple latency check. Only tests the endpoint, not the path.

'''traceroute''' — Best for one-time path discovery. Provides no continuous monitoring.

'''ss / netstat''' — Best for checking local connection status. Doesn’t test remote paths.

=== Best Practices ===
Run at least 50–100 cycles for accurate statistics. Use --no-dns for faster results and to avoid DNS resolution issues during testing. Test at different times of day, since network conditions vary. If one protocol shows issues, compare ICMP, UDP, and TCP modes. Save reports with timestamps for trend analysis. Use 8.8.8.8 or 1.1.1.1 as known-good hosts to verify your local network first. Let MTR run for at least 30–60 seconds before drawing conclusions.

=== Reading Between the Lines ===
'''Good network health:'''
HOST: hostname Loss% Snt Last Avg Best Wrst StDev
1.|-- 192.168.1.1 0.0% 50 1.1 1.2 0.9 2.1 0.2
2.|-- 10.0.0.1 0.0% 50 8.2 8.5 7.5 10.2 0.5
3.|-- isp-gateway.net 0.0% 50 15.1 15.5 14.0 18.3 0.8
4.|-- google.com 0.0% 50 24.8 25.2 23.5 28.1 1.1
No packet loss, consistent latency, low jitter throughout.

'''Problematic network:'''
HOST: hostname Loss% Snt Last Avg Best Wrst StDev
1.|-- 192.168.1.1 0.0% 50 1.2 1.5 1.0 2.3 0.3
2.|-- 10.0.0.1 5.0% 50 45.3 52.8 8.1 245.2 45.7
3.|-- ??? 100.0% 50 0.0 0.0 0.0 0.0 0.0
4.|-- destination.com 15.0% 50 95.4 125.8 48.2 385.5 78.2
Packet loss at hop 2 and the destination, high jitter, and a very high worst-case latency all point to a genuine problem at or near hop 2.

=== Summary ===
MTR is your friend when it comes to diagnostics. Use --report-cycles 50 or higher for reliable data. Watch for packet loss at the destination—intermediate losses may be false positives. High StDev indicates an unstable connection. High average latency points to slow links. Use --no-dns for faster results. Compare ICMP, UDP, and TCP modes if issues appear. Test at different times of day for a complete picture.
== Package Management Issues ==

=== Fix Broken Packages ===
$ sudo apt update # update package lists
$ sudo apt --fix-broken install # fix broken dependencies
$ sudo dpkg --configure -a # reconfigure packages (no output = nothing to do)
$ sudo apt clean # clean package cache
$ sudo apt autoclean
$ sudo apt autoremove # remove unused packages

=== Handle Held or Locked Packages ===
$ sudo lsof /var/lib/dpkg/lock-frontend # find process holding the lock
$ sudo rm /var/lib/dpkg/lock-frontend # remove lock (only if no package manager is running)
$ sudo rm /var/lib/apt/lists/lock
$ sudo dpkg --configure -a # reconfigure dpkg

== Disk & Filesystem Issues ==

=== Check Disk Health ===
$ df -h # disk space
$ df -i # inode usage
$ sudo apt install sysstat # sysstat includes iostat, mpstat, sar, pidstat, cifsiostat
$ iostat -x 1 # disk I/O statistics
$ iostat -xm 2 # stats in MB instead of KB
$ iostat -x sda 1 # monitor a specific device
$ sudo dmesg | grep -i "error\|fail" # check for disk errors
$ sudo apt install smartmontools
$ sudo smartctl -a /dev/sdX # SMART disk health - lsblk to determine the target device 

=== Repair Filesystem ===
$ sudo umount /dev/sdXN # unmount the partition first
$ sudo fsck /dev/sdXN # run filesystem check
$ sudo apt --fix-broken install # Fix broken dependencies
$ sudo dpkg --configure -a # Reconfigure packages

# No output means there is nothing to reconfigure

# Clean package cache
sudo apt clean
sudo apt autoclean

# Remove unused packages
sudo apt autoremove

=== Handle Held or Locked Packages ===
# If apt is locked, find the process holding the lock
sudo lsof /var/lib/dpkg/lock-frontend

# Remove the lock (use with caution—only if no package manager is running)
sudo rm /var/lib/dpkg/lock-frontend
sudo rm /var/lib/apt/lists/lock

# Reconfigure dpkg
sudo dpkg --configure -a
Disk & Filesystem Issues

=== Check Disk Health ===
# Check disk space
df -h

# Check inode usage
df -i

# View disk I/O statistics
# iostat is part of the sysstat package, which also includes:
# mpstat (CPU statistics), sar (system activity reporter),
# pidstat (process statistics), and cifsiostat (CIFS statistics)
sudo apt install sysstat

iostat -x 1

# Show stats in MB instead of KB
iostat -xm 2

# Monitor a specific device
iostat -x sda 1

# Check for disk errors in dmesg
sudo dmesg | grep -i "error\|fail"

# SMART disk health (requires smartmontools)
sudo apt install smartmontools
sudo smartctl -a /dev/sda

=== Repair Filesystem ===
# Unmount the partition first
sudo umount /dev/sdXN

# Run filesystem check
sudo fsck /dev/sdXN

# For ext4 specifically
sudo e2fsck -f /dev/sdXN
Performance Issues

=== Identify Resource Hogs ===
# CPU usage by process
top -o %CPU

# Memory usage by process
top -o %MEM

# Disk usage by directory
du -sh /* | sort -h

# Find large files
find / -type f -size +100M 2>/dev/null

# Check running processes sorted by memory use
ps aux --sort=-%mem | head -20

=== System Temperature Monitoring ===
# Install lm-sensors if not already present
sudo apt install lm-sensors
sudo sensors-detect

# View temperatures
sensors

# Real-time temperature monitoring
watch -n 2 sensors
See section 12 for a shell alias that makes temperature monitoring more convenient.

Service & Application Errors

=== Debug Service Problems ===
# Check service status
sudo systemctl status service-name

# View service logs
sudo journalctl -u service-name

# Restart a service
sudo systemctl restart service-name

# Enable service at boot
sudo systemctl enable service-name

# View recent service failures
journalctl -p err -b

=== Application Crash Investigation ===
# Check for core dumps
ls -lh /var/crash/

# View available application logs
ls /var/log/

# Follow syslog for application errors in real-time
sudo tail -f /var/log/syslog
Permission & Access Issues

=== Fix Common Permission Problems ===
# Check file ownership
ls -l /path/to/file

# Change ownership (user:group)
sudo chown user:group /path/to/file

# Change permissions
sudo chmod 644 /path/to/file

# Recursively fix permissions on a directory
sudo chown -R user:group /path/to/directory

=== User & Authentication Issues ===
# Check user information
id username

# View user login history
last -a

# Check failed login attempts
sudo journalctl | grep "authentication failure"

# Reset user password
sudo passwd username
Hardware Issues

=== Identify Hardware ===
# List all hardware (install if needed: sudo apt install lshw)
sudo lshw -short

# PCI devices
lspci -v

# USB devices
lsusb -v

# CPU information
lscpu

# Memory information
sudo dmidecode --type memory

=== Check Hardware Errors ===
# Kernel ring buffer (hardware messages)
dmesg | less
# Press q to quit

# Search for specific hardware issues
dmesg | grep -i "error\|fail\|warn"

# Check for USB issues
dmesg | grep -i usb
Quick Diagnostic Commands

=== System Information at a Glance ===
# Uptime and load average
uptime

# Kernel version
uname -r

# Debian version
cat /etc/debian_version

# Full system summary (install if needed: sudo apt install inxi)
sudo inxi -Fxz

=== Emergency Toolkit ===
# Create a consolidated diagnostic report
sudo journalctl -b > ~/system-report.txt
dmesg >> ~/system-report.txt
systemctl --failed >> ~/system-report.txt
df -h >> ~/system-report.txt
free -h >> ~/system-report.txt

# Watch logs in real-time
sudo journalctl -f

# Monitor system resources continuously
watch -n 1 'free -h && df -h'
Useful Aliases & Shortcuts
Add these to your ~/.bashrc for quick access to common troubleshooting commands:
# Monitor system temperatures in real-time
alias temps="watch -n 2 'for i in /sys/class/hwmon/hwmon*/; do echo -n \"\$(cat \${i}name): \"; cat \${i}temp*_input 2>/dev/null | while read temp; do echo \"scale=1; \$temp/1000\" | bc; done | tr \"\n\" \" \"; echo \"°C\"; done'"

# Quick system status
alias sysstat='echo "=== CPU ===" && uptime && echo -e "\n=== Memory ===" && free -h && echo -e "\n=== Disk ===" && df -h && echo -e "\n=== Top Processes ===" && ps aux --sort=-%mem | head -10'

# View last boot logs
alias lastboot='journalctl -b -1'

# Check failed services
alias failedservices='systemctl --failed'

# Monitor logs in real-time
alias watchlog='sudo journalctl -f'

# Quick network status
alias netstatus='ip addr show && echo -e "\n=== Routes ===" && ip route show'
After adding these, reload your shell configuration:
source ~/.bashrc
Tips for Effective Troubleshooting

'''Check logs first.''' journalctl and dmesg are your best starting point for almost any problem.

'''Change one thing at a time.''' It’s tempting to make several changes at once, but doing so makes it impossible to know what actually fixed—or broke—the problem.

'''Document your changes.''' Keep notes on what you’ve tried and what the results were. You’ll thank yourself later.

'''Use exact error messages.''' Copy error messages verbatim and search for them alongside “Debian” and the version number—for example, “Debian 13” or “Debian 13.1” for point releases.

'''Check what changed recently.''' Did you install something, update packages, or change the kernel before the problem appeared? Start there.

'''Back up before making major changes.''' This goes without saying, but it’s easy to skip when you’re in a hurry.

'''Use verbose mode.''' Adding -v or -vv flags to commands often reveals what would otherwise be silent failures.

'''Use the community.''' The Debian forums, Reddit, Stack Exchange, and the Debian mailing lists are all active and well-indexed. Someone has usually seen your problem before.

''This work was contributed by <dfn>distro-nix</dfn> to the Debian User Forums on 2025-10-27.''

Debian First Aid Kit

2026-05-16T02:16:27Z

Distro-nix:

'''Author''' '''Profile:''' [https://forums.debian.net/memberlist.php?mode=viewprofile&u=239099 distro-nix] '''Debian Version:''' Debian 13 (Trixie) '''Kernel Versions Tested:''' 6.12.73+deb13-amd64 and 6.16.3+deb13-amd64 (x86_64) '''Created:''' 2025-10-27 '''Last Updated:''' 2026-05-15 '''ID:''' 544201 '''License:''' CC BY-SA 4.0

''This guide is preserved in its original form. Questions and feedback welcome via the author’s <dfn>Debian Forums profile</dfn>.''

== Abstract ==
The Debian First Aid Kit is a practical reference for diagnosing and resolving common system problems on Debian GNU/Linux. It covers commonly occuring categories of failure: system freezes and crashes, boot problems, network issues, package management, disk and filesystem errors, performance degradation, service and application failures, permission and access problems, hardware identification and errors, and quick diagnostics. Each section gives the relevant commands so you can understand what they do and why, rather than a bare list to copy and paste blindly. A dedicated section covers MTR (Matt’s Traceroute) in depth, including output interpretation, practical troubleshooting, and how to monitor continuously. The guide also includes a set of shell aliases for faster access to common diagnostic tasks. All commands are verified on Debian 13 (Trixie) running kernel 6.12.73+deb13-amd64 and 6.16.3+deb13-amd64. This document is intended for users with basic command-line familiarity who need a reliable, consolidated reference when something goes wrong.

'''Keywords:''' Debian, troubleshooting, system administration, MTR, network diagnostics, package management, journalctl, filesystem repair
----

== Tested Commands Disclaimer ==
All commands in this document have been verified on Debian 13 (Trixie) with the kernel versions listed above. Results may vary on other releases or hardware configurations. Always test in a safe environment before applying any procedure to a production system.

== Conventions Used in This Document ==
Command examples throughout this guide follow standard Unix prompt conventions. A $ prefix indicates a command run as a regular user. Where elevated privileges are required, sudo is shown explicitly and # with a brief explanation if needed:
$ journalctl -b -1 # regular user
$ sudo apt install mtr # requires elevation via sudo
In recovery shell contexts, where you are operating directly as root, the prompt is #.

== System Freezes & Crashes ==

=== Check System Logs ===

=== Common Freeze Causes to Look For ===

* '''Kernel panics:''' Search for “kernel panic” or “Oops”
* '''Out of Memory (OOM):''' Search for “Out of memory” or “oom-killer”
* '''Hardware errors:''' Look for “MCE” (Machine Check Exception) or “hardware error”
* '''Driver issues:''' Check for module/driver failures
* '''Overheating:''' Check system temperatures

=== Check System Resources ===
$ free -h # memory usage

$ df -h # disk space

$ htop # interactive resource monitor

$ top # alternative resource monitor

$ sudo apt install btop # btop is a more detailed alternative

$ sudo dmesg | grep -i error # check for disk errors

=== Access Recovery Mode ===
Reboot and hold Shift to access the GRUB menu (timing depends on your GRUB configuration). Select “Advanced options”, then choose recovery mode. From the recovery menu, select “root” for a root shell.

=== Common Boot Fixes ===
$ sudo fsck /dev/sdXN # repair filesystem — identify device with lsblk first

$ sudo grub-install /dev/sdX # reinstall GRUB bootloader

$ sudo update-grub # regenerate GRUB configuration

$ cat /etc/fstab # check fstab for mount errors

The following are permanent errors caused by incomplete or buggy ACPI tables in the BIOS. They are harmless and can be safely ignored if they appear on your system:
[0.686554] ACPI Error: No handler for Region [ECRM] (00000000201accc4) [EmbeddedControl] (20250404/evregion-131)
[0.686577] ACPI Error: Region EmbeddedControl (ID=3) has no handler (20250404/exfldio-261)
[0.686594] ACPI Error: Aborting method \_SB.GPIO._EVT due to previous error (AE_NOT_EXIST) (20250404/psparse-529)

== Boot Problems ==

=== Check Boot Process ===
$ systemd-analyze blame # boot time by unit

$ systemd-analyze blame # boot time by unit

$ systemctl --failed # units that failed during boot

$ systemctl status <service-name> # e.g. systemctl status NetworkManager.service

=== Access Recovery Mode ===
Reboot and hold Shift to access the GRUB menu (timing depends on your GRUB configuration). Select “Advanced options”, then choose recovery mode. From the recovery menu, select “root” for a root shell.

=== Common Boot Fixes ===
$ sudo fsck /dev/sdXN # repair filesystem — identify device with lsblk first

$ sudo grub-install /dev/sdX # reinstall GRUB bootloader

$ sudo update-grub # regenerate GRUB configuration

$ cat /etc/fstab # check fstab for mount errors

== Network Issues ==

=== Diagnose Network Connection ===
$ ip addr show # check network interfaces

$ ping -c 4 8.8.8.8 # test IPv4 connectivity

$ ping -c 6 2a00:1450:4007:809::200e # test IPv6 connectivity

$ nslookup google.com # check DNS resolution

$ ip route show # view routing table

$ ss -tuln # check active connections

=== Restart Network Service ===
$ sudo systemctl restart NetworkManager # for systems using NetworkManager

$ sudo systemctl restart networking # for systems using networking service

$ sudo ip link set eth0 down # bring interface down

$ sudo ip link set eth0 up # bring interface back up

== MTR (Matt’s Traceroute) ==
MTR combines the functionality of ping and traceroute into a single real-time network diagnostic tool. It continuously monitors the path between your system and a destination, providing detailed statistics about latency and packet loss at each hop. When you need to demonstrate a network problem that is outside your control—to your hosting provider, for example—MTR gives you the evidence you need.

=== Installation ===
$ sudo apt install mtr

=== Basic Usage ===
$ mtr google.com # interactive mode

$ mtr --report google.com # report mode (10 cycles then exit)

$ mtr --report-cycles 50 google.com # specify number of cycles

$ mtr --tcp google.com # use TCP instead of ICMP

$ mtr --udp google.com # use UDP

$ mtr --no-dns google.com # skip DNS resolution (faster)

$ mtr --show-ips google.com # show both hostnames and IPs

== Understanding MTR Output ==

==== Sample Output ====
HOST: hostname Loss% Snt Last Avg Best Wrst StDev
1.|-- 192.168.1.1 0.0% 10 1.2 1.5 1.0 2.3 0.4
2.|-- 10.0.0.1 0.0% 10 8.5 9.2 7.8 12.1 1.3
3.|-- isp-gateway.net 0.0% 10 15.3 16.1 14.2 19.8 1.8
4.|-- ??? 100.0% 10 0.0 0.0 0.0 0.0 0.0
5.|-- google.com 0.0% 10 25.4 26.8 24.1 32.5 2.4

=== Column Meanings ===
'''HOST''' — Hostname or IP address of each hop in the route.

'''Loss%''' — Percentage of packets lost at this hop.

'''Snt''' — Number of packets sent to this hop.

'''Last''' — Latency of the most recent packet, in milliseconds.

'''Avg''' — Average latency across all packets sent, in milliseconds.

'''Best''' — Lowest latency recorded, in milliseconds.

'''Wrst''' — Highest latency recorded, in milliseconds.

'''StDev''' — Standard deviation of latency measurements—lower values indicate a more stable connection.

=== Interpreting Results ===
A healthy network shows 0% loss on all hops, stable latency with low StDev values, a gradual latency increase as hop count increases, and consistent response times.

==== Problem Indicators ====
'''1. High Packet Loss at a Specific Hop'''
5.|-- problem-router.net 25.0% 10 45.3 48.2 42.1 65.8 8.4
If loss continues to the destination, there is a real problem at that router. If loss appears only at that hop but not beyond, the router is likely rate-limiting ICMP responses—a false positive, not a real problem. The rule of thumb: if packet loss appears at hop N but hops N+1, N+2, and beyond show 0% loss, it’s almost certainly ICMP rate limiting.

'''2. High Latency at a Specific Hop'''
3.|-- slow-link.net 0.0% 10 150.3 155.2 148.1 165.8 5.4
This indicates a network bottleneck, a congested link, geographical distance (intercontinental hops), or slow routing equipment.

'''3. No Response (???)'''
4.|-- ??? 100.0% 10 0.0 0.0 0.0 0.0 0.0
The router at this hop is configured not to respond to ICMP or traceroute packets, or a firewall is blocking diagnostic packets. This is not necessarily a problem if later hops respond normally.

'''4. High Jitter (StDev)'''
6.|-- unstable.net 0.0% 10 35.3 52.8 28.1 95.2 24.7
A high StDev (24.7ms in this example) indicates inconsistent latency caused by network congestion or instability. This is particularly noticeable in real-time applications such as VoIP, gaming, and video calls.

'''5. Sudden Latency Spike'''
1.|-- 192.168.1.1 0.0% 10 1.2 1.5 1.0 2.3 0.4
2.|-- 10.0.0.1 0.0% 10 8.5 9.2 7.8 12.1 1.3
3.|-- problematic-hop.net 0.0% 10 180.5 185.2 178.1 195.8 6.4
4.|-- next-hop.net 0.0% 10 182.3 187.8 180.5 198.2 6.8
Hop 3 introduces approximately 170ms of additional latency (jumping from 9ms to 180ms). Everything from hop 3 onward carries that penalty.

=== Advanced Usage ===

==== Report Mode with Different Output Formats ====
$ mtr --report --csv google.com > network-report.csv # CSV for logging
$ mtr --report --json google.com # JSON for parsing
$ mtr --report --xml google.com # XML format
$ mtr --report-wide google.com # wide report, no abbreviations

==== Protocol Selection ====
$ mtr google.com # ICMP (default)
$ mtr --udp google.com # UDP
$ sudo mtr --tcp --port 443 google.com # specific TCP port
$ sudo mtr --tcp --port 22 remote-server.com # generally the default port

==== Timing and Duration ====
$ mtr --interval 0.5 google.com # interval between pings (default 1s)
$ mtr --report-cycles 100 google.com # extended test, 100 cycles
$ mtr google.com # continuous monitoring (Ctrl+C to stop)
$ mtr --report-cycles 10 google.com # quick 10-cycle report

==== Advanced Options ====
$ mtr --aslookup google.com # show Autonomous System (AS) numbers
$ mtr --max-ttl 20 google.com # set maximum number of hops
$ mtr --psize 1000 google.com # set packet size
$ mtr --show-ips google.com # show both IP and hostname
$ mtr --address 192.168.1.100 google.com # specify source address
$ mtr -4 google.com # IPv4 only
$ mtr -6 google.com # IPv6 only

=== Interactive Mode Commands ===
When running MTR in interactive mode (mtr hostname), the following keys are available:

'''h''' — Display help.

'''d''' — Toggle display mode, cycling through different views.

'''n''' — Toggle between hostnames and IP addresses.

'''r''' — Reset all statistics.

'''p''' — Pause/unpause the display.

'''q''' — Quit MTR.

'''u''' — Switch between ICMP, UDP, and TCP modes.

'''y''' — Switch between IPv4 and IPv6.

'''o''' — Toggle field display options.

'''j''' — Toggle latency display.

=== Practical Troubleshooting Scenarios ===

==== Scenario 1: Diagnosing a Slow Website ====
$ mtr --report-cycles 100 --no-dns example.com
Look for high average latency at specific hops, packet loss at the destination, and high StDev values indicating jitter.

==== Scenario 2: Testing if a Firewall Blocks SSH ====
$ sudo mtr --tcp --port 22 --report-cycles 50 remote-server.com
If the last hop shows 100% loss but earlier hops are fine, port 22 may be filtered. Run a standard ICMP test for comparison.

==== Scenario 3: ISP Performance Issues ====
$ mtr --report-cycles 100 8.8.8.8

$ mtr --report-cycles 100 1.1.1.1

If issues appear in the first three or four hops, the problem is likely within your ISP’s network. If issues appear later, the problem lies beyond your ISP.

==== Scenario 4: VPN Troubleshooting ====
$ mtr --report-cycles 50 --no-dns google.com > before-vpn.txt # before VPN
$ mtr --report-cycles 50 --no-dns google.com > after-vpn.txt # after VPN
$ diff before-vpn.txt after-vpn.txt # compare the two

==== Scenario 5: Gaming/Streaming Performance ====
$ mtr --report-cycles 200 game-server.com
For real-time applications, look for low average latency (under 50ms for gaming), low StDev (under 5ms preferred), and zero packet loss.

==== Scenario 6: Intermittent Connectivity ====
$ mtr --report-cycles 500 --interval 1 target.com > long-test.txt # long-running test
$ mtr target.com # real-time monitoring

=== Continuous Monitoring ===

==== Log Network Performance Over Time ====
while true; do
timestamp=$(date +%Y%m%d-%H%M%S)
mtr --report --report-cycles 50 google.com > "mtr-$timestamp.txt"
sleep 3600
done

==== Monitor Multiple Destinations ====
#!/bin/bash
echo "=== MTR Report $(date) ===" > daily-network-report.txt
echo "" >> daily-network-report.txt

echo "Google DNS:" >> daily-network-report.txt
mtr --report --report-cycles 50 --no-dns 8.8.8.8 >> daily-network-report.txt
echo "" >> daily-network-report.txt

echo "Cloudflare DNS:" >> daily-network-report.txt
mtr --report --report-cycles 50 --no-dns 1.1.1.1 >> daily-network-report.txt
echo "" >> daily-network-report.txt

echo "Your Server:" >> daily-network-report.txt
mtr --report --report-cycles 50 your-server.com >> daily-network-report.txt

=== Useful Aliases for .bashrc ===
Quick network path analysis
alias mtrreport='mtr --report --report-cycles 50 --no-dns'

Monitor connection to Google DNS
alias netcheck='mtr --report-cycles 20 8.8.8.8'

Extended network test
alias mtrlong='mtr --report-cycles 100'

TCP port 443 test (HTTPS)
alias mtrhttps='sudo mtr --tcp --port 443 --report-cycles 30'

Quick comparison of major DNS providers
alias dnstest='echo "Google:" && mtr --report-cycles 20 8.8.8.8 && echo -e "\nCloudflare:" && mtr --report-cycles 20 1.1.1.1'
After adding to ~/.bashrc, reload with:
$ source ~/.bashrc

==== Useful Tip for Aliases ====
Rather than adding aliases directly to <code>~/.bashrc</code>, it's cleaner to keep them in a dedicated file—<code>~/.bashrc_aliases</code>—and source that file from <code>~/.bashrc</code>. This keeps your shell configuration tidy and makes it easy to share or back up just your aliases without touching the rest of your environment.

=== Troubleshooting Tips ===
'''Permission issues with TCP mode:'''
$ sudo mtr --tcp --port 443 example.com # Use sudo for TCP on privileged ports
$ sudo setcap cap_net_raw+ep /usr/bin/mtr-packet # Or set capabilities (one-time setup)
'''False positives:''' Packet loss at an intermediate hop but not at the destination is almost always ICMP rate limiting at that router, not a real problem. Example:
3.|-- router.isp.net 20.0% 50 15.3 16.1 14.2 19.8 1.8
4.|-- next-hop.net 0.0% 50 18.5 19.2 17.8 22.1 1.3
5.|-- destination.com 0.0% 50 25.4 26.8 24.1 32.5 2.4
Hop 3 shows 20% loss, but hops 4 and 5 show 0%. Traffic is flowing normally.

'''DNS resolution delays:''' If MTR is slow to start, skip DNS resolution with mtr --no-dns target.com and resolve addresses afterward with host 203.0.113.1 if needed.

'''Comparing results:'''
$ mtr --report --report-cycles 50 example.com > test1.txt
$ sleep 60
$ mtr --report --report-cycles 50 example.com > test2.txt
$ diff test1.txt test2.txt

=== When to Use MTR vs Other Tools ===
'''MTR''' — Best for continuous monitoring, identifying problem hops, and gathering detailed statistics. Requires installation via apt.

'''ping''' — Best for a quick connectivity test or simple latency check. Only tests the endpoint, not the path.

'''traceroute''' — Best for one-time path discovery. Provides no continuous monitoring.

'''ss / netstat''' — Best for checking local connection status. Doesn’t test remote paths.

=== Best Practices ===
Run at least 50–100 cycles for accurate statistics. Use --no-dns for faster results and to avoid DNS resolution issues during testing. Test at different times of day, since network conditions vary. If one protocol shows issues, compare ICMP, UDP, and TCP modes. Save reports with timestamps for trend analysis. Use 8.8.8.8 or 1.1.1.1 as known-good hosts to verify your local network first. Let MTR run for at least 30–60 seconds before drawing conclusions.

=== Reading Between the Lines ===
'''Good network health:'''
HOST: hostname Loss% Snt Last Avg Best Wrst StDev
1.|-- 192.168.1.1 0.0% 50 1.1 1.2 0.9 2.1 0.2
2.|-- 10.0.0.1 0.0% 50 8.2 8.5 7.5 10.2 0.5
3.|-- isp-gateway.net 0.0% 50 15.1 15.5 14.0 18.3 0.8
4.|-- google.com 0.0% 50 24.8 25.2 23.5 28.1 1.1
No packet loss, consistent latency, low jitter throughout.

'''Problematic network:'''
HOST: hostname Loss% Snt Last Avg Best Wrst StDev
1.|-- 192.168.1.1 0.0% 50 1.2 1.5 1.0 2.3 0.3
2.|-- 10.0.0.1 5.0% 50 45.3 52.8 8.1 245.2 45.7
3.|-- ??? 100.0% 50 0.0 0.0 0.0 0.0 0.0
4.|-- destination.com 15.0% 50 95.4 125.8 48.2 385.5 78.2
Packet loss at hop 2 and the destination, high jitter, and a very high worst-case latency all point to a genuine problem at or near hop 2.

=== Summary ===
MTR is your friend when it comes to diagnostics. Use --report-cycles 50 or higher for reliable data. Watch for packet loss at the destination—intermediate losses may be false positives. High StDev indicates an unstable connection. High average latency points to slow links. Use --no-dns for faster results. Compare ICMP, UDP, and TCP modes if issues appear. Test at different times of day for a complete picture.
== Package Management Issues ==

=== Fix Broken Packages ===
$ sudo apt update # update package lists
$ sudo apt --fix-broken install # fix broken dependencies
$ sudo dpkg --configure -a # reconfigure packages (no output = nothing to do)
$ sudo apt clean # clean package cache
$ sudo apt autoclean
$ sudo apt autoremove # remove unused packages

=== Handle Held or Locked Packages ===
$ sudo lsof /var/lib/dpkg/lock-frontend # find process holding the lock
$ sudo rm /var/lib/dpkg/lock-frontend # remove lock (only if no package manager is running)
$ sudo rm /var/lib/apt/lists/lock
$ sudo dpkg --configure -a # reconfigure dpkg

== Disk & Filesystem Issues ==

=== Check Disk Health ===
$ df -h # disk space
$ df -i # inode usage
$ sudo apt install sysstat # sysstat includes iostat, mpstat, sar, pidstat, cifsiostat
$ iostat -x 1 # disk I/O statistics
$ iostat -xm 2 # stats in MB instead of KB
$ iostat -x sda 1 # monitor a specific device
$ sudo dmesg | grep -i "error\|fail" # check for disk errors
$ sudo apt install smartmontools
$ sudo smartctl -a /dev/sdX # SMART disk health - lsblk to determine the target device 

# Fix broken dependencies
sudo apt --fix-broken install

# Reconfigure packages
sudo dpkg --configure -a
# No output means there is nothing to reconfigure

# Clean package cache
sudo apt clean
sudo apt autoclean

# Remove unused packages
sudo apt autoremove

=== Handle Held or Locked Packages ===
# If apt is locked, find the process holding the lock
sudo lsof /var/lib/dpkg/lock-frontend

# Remove the lock (use with caution—only if no package manager is running)
sudo rm /var/lib/dpkg/lock-frontend
sudo rm /var/lib/apt/lists/lock

# Reconfigure dpkg
sudo dpkg --configure -a
Disk & Filesystem Issues

=== Check Disk Health ===
# Check disk space
df -h

# Check inode usage
df -i

# View disk I/O statistics
# iostat is part of the sysstat package, which also includes:
# mpstat (CPU statistics), sar (system activity reporter),
# pidstat (process statistics), and cifsiostat (CIFS statistics)
sudo apt install sysstat

iostat -x 1

# Show stats in MB instead of KB
iostat -xm 2

# Monitor a specific device
iostat -x sda 1

# Check for disk errors in dmesg
sudo dmesg | grep -i "error\|fail"

# SMART disk health (requires smartmontools)
sudo apt install smartmontools
sudo smartctl -a /dev/sda

=== Repair Filesystem ===
# Unmount the partition first
sudo umount /dev/sdXN

# Run filesystem check
sudo fsck /dev/sdXN

# For ext4 specifically
sudo e2fsck -f /dev/sdXN
Performance Issues

=== Identify Resource Hogs ===
# CPU usage by process
top -o %CPU

# Memory usage by process
top -o %MEM

# Disk usage by directory
du -sh /* | sort -h

# Find large files
find / -type f -size +100M 2>/dev/null

# Check running processes sorted by memory use
ps aux --sort=-%mem | head -20

=== System Temperature Monitoring ===
# Install lm-sensors if not already present
sudo apt install lm-sensors
sudo sensors-detect

# View temperatures
sensors

# Real-time temperature monitoring
watch -n 2 sensors
See section 12 for a shell alias that makes temperature monitoring more convenient.

Service & Application Errors

=== Debug Service Problems ===
# Check service status
sudo systemctl status service-name

# View service logs
sudo journalctl -u service-name

# Restart a service
sudo systemctl restart service-name

# Enable service at boot
sudo systemctl enable service-name

# View recent service failures
journalctl -p err -b

=== Application Crash Investigation ===
# Check for core dumps
ls -lh /var/crash/

# View available application logs
ls /var/log/

# Follow syslog for application errors in real-time
sudo tail -f /var/log/syslog
Permission & Access Issues

=== Fix Common Permission Problems ===
# Check file ownership
ls -l /path/to/file

# Change ownership (user:group)
sudo chown user:group /path/to/file

# Change permissions
sudo chmod 644 /path/to/file

# Recursively fix permissions on a directory
sudo chown -R user:group /path/to/directory

=== User & Authentication Issues ===
# Check user information
id username

# View user login history
last -a

# Check failed login attempts
sudo journalctl | grep "authentication failure"

# Reset user password
sudo passwd username
Hardware Issues

=== Identify Hardware ===
# List all hardware (install if needed: sudo apt install lshw)
sudo lshw -short

# PCI devices
lspci -v

# USB devices
lsusb -v

# CPU information
lscpu

# Memory information
sudo dmidecode --type memory

=== Check Hardware Errors ===
# Kernel ring buffer (hardware messages)
dmesg | less
# Press q to quit

# Search for specific hardware issues
dmesg | grep -i "error\|fail\|warn"

# Check for USB issues
dmesg | grep -i usb
Quick Diagnostic Commands

=== System Information at a Glance ===
# Uptime and load average
uptime

# Kernel version
uname -r

# Debian version
cat /etc/debian_version

# Full system summary (install if needed: sudo apt install inxi)
sudo inxi -Fxz

=== Emergency Toolkit ===
# Create a consolidated diagnostic report
sudo journalctl -b > ~/system-report.txt
dmesg >> ~/system-report.txt
systemctl --failed >> ~/system-report.txt
df -h >> ~/system-report.txt
free -h >> ~/system-report.txt

# Watch logs in real-time
sudo journalctl -f

# Monitor system resources continuously
watch -n 1 'free -h && df -h'
Useful Aliases & Shortcuts
Add these to your ~/.bashrc for quick access to common troubleshooting commands:
# Monitor system temperatures in real-time
alias temps="watch -n 2 'for i in /sys/class/hwmon/hwmon*/; do echo -n \"\$(cat \${i}name): \"; cat \${i}temp*_input 2>/dev/null | while read temp; do echo \"scale=1; \$temp/1000\" | bc; done | tr \"\n\" \" \"; echo \"°C\"; done'"

# Quick system status
alias sysstat='echo "=== CPU ===" && uptime && echo -e "\n=== Memory ===" && free -h && echo -e "\n=== Disk ===" && df -h && echo -e "\n=== Top Processes ===" && ps aux --sort=-%mem | head -10'

# View last boot logs
alias lastboot='journalctl -b -1'

# Check failed services
alias failedservices='systemctl --failed'

# Monitor logs in real-time
alias watchlog='sudo journalctl -f'

# Quick network status
alias netstatus='ip addr show && echo -e "\n=== Routes ===" && ip route show'
After adding these, reload your shell configuration:
source ~/.bashrc
Tips for Effective Troubleshooting

'''Check logs first.''' journalctl and dmesg are your best starting point for almost any problem.

'''Change one thing at a time.''' It’s tempting to make several changes at once, but doing so makes it impossible to know what actually fixed—or broke—the problem.

'''Document your changes.''' Keep notes on what you’ve tried and what the results were. You’ll thank yourself later.

'''Use exact error messages.''' Copy error messages verbatim and search for them alongside “Debian” and the version number—for example, “Debian 13” or “Debian 13.1” for point releases.

'''Check what changed recently.''' Did you install something, update packages, or change the kernel before the problem appeared? Start there.

'''Back up before making major changes.''' This goes without saying, but it’s easy to skip when you’re in a hurry.

'''Use verbose mode.''' Adding -v or -vv flags to commands often reveals what would otherwise be silent failures.

'''Use the community.''' The Debian forums, Reddit, Stack Exchange, and the Debian mailing lists are all active and well-indexed. Someone has usually seen your problem before.

''This work was contributed by <dfn>distro-nix</dfn> to the Debian User Forums on 2025-10-27.''

Debian First Aid Kit

2026-05-15T23:00:17Z

Distro-nix: Checking/fixing preformattied boxes

'''Author''' '''Profile:''' [https://forums.debian.net/memberlist.php?mode=viewprofile&u=239099 distro-nix] '''Debian Version:''' Debian 13 (Trixie) '''Kernel Versions Tested:''' 6.12.73+deb13-amd64 and 6.16.3+deb13-amd64 (x86_64) '''Created:''' 2025-10-27 '''Last Updated:''' 2026-05-15 '''ID:''' 544201 '''License:''' CC BY-SA 4.0

''This guide is preserved in its original form. Questions and feedback welcome via the author’s <dfn>Debian Forums profile</dfn>.''

== Abstract ==
The Debian First Aid Kit is a practical reference for diagnosing and resolving common system problems on Debian GNU/Linux. It covers commonly occuring categories of failure: system freezes and crashes, boot problems, network issues, package management, disk and filesystem errors, performance degradation, service and application failures, permission and access problems, hardware identification and errors, and quick diagnostics. Each section gives the relevant commands so you can understand what they do and why, rather than a bare list to copy and paste blindly. A dedicated section covers MTR (Matt’s Traceroute) in depth, including output interpretation, practical troubleshooting, and how to monitor continuously. The guide also includes a set of shell aliases for faster access to common diagnostic tasks. All commands are verified on Debian 13 (Trixie) running kernel 6.12.73+deb13-amd64 and 6.16.3+deb13-amd64. This document is intended for users with basic command-line familiarity who need a reliable, consolidated reference when something goes wrong.

'''Keywords:''' Debian, troubleshooting, system administration, MTR, network diagnostics, package management, journalctl, filesystem repair
----

== Tested Commands Disclaimer ==
All commands in this document have been verified on Debian 13 (Trixie) with the kernel versions listed above. Results may vary on other releases or hardware configurations. Always test in a safe environment before applying any procedure to a production system.

== Conventions Used in This Document ==
Command examples throughout this guide follow standard Unix prompt conventions. A $ prefix indicates a command run as a regular user. Where elevated privileges are required, sudo is shown explicitly and # with a brief explanation if needed:
$ journalctl -b -1 # regular user
$ sudo apt install mtr # requires elevation via sudo
In recovery shell contexts, where you are operating directly as root, the prompt is #.

== System Freezes & Crashes ==

=== Check System Logs ===

=== Common Freeze Causes to Look For ===

* '''Kernel panics:''' Search for “kernel panic” or “Oops”
* '''Out of Memory (OOM):''' Search for “Out of memory” or “oom-killer”
* '''Hardware errors:''' Look for “MCE” (Machine Check Exception) or “hardware error”
* '''Driver issues:''' Check for module/driver failures
* '''Overheating:''' Check system temperatures

=== Check System Resources ===
$ free -h # memory usage

$ df -h # disk space

$ htop # interactive resource monitor

$ top # alternative resource monitor

$ sudo apt install btop # btop is a more detailed alternative

$ sudo dmesg | grep -i error # check for disk errors

=== Access Recovery Mode ===
Reboot and hold Shift to access the GRUB menu (timing depends on your GRUB configuration). Select “Advanced options”, then choose recovery mode. From the recovery menu, select “root” for a root shell.

=== Common Boot Fixes ===
$ sudo fsck /dev/sdXN # repair filesystem — identify device with lsblk first

$ sudo grub-install /dev/sdX # reinstall GRUB bootloader

$ sudo update-grub # regenerate GRUB configuration

$ cat /etc/fstab # check fstab for mount errors

The following are permanent errors caused by incomplete or buggy ACPI tables in the BIOS. They are harmless and can be safely ignored if they appear on your system:
[0.686554] ACPI Error: No handler for Region [ECRM] (00000000201accc4) [EmbeddedControl] (20250404/evregion-131)
[0.686577] ACPI Error: Region EmbeddedControl (ID=3) has no handler (20250404/exfldio-261)
[0.686594] ACPI Error: Aborting method \_SB.GPIO._EVT due to previous error (AE_NOT_EXIST) (20250404/psparse-529)

== Boot Problems ==

=== Check Boot Process ===
$ systemd-analyze blame # boot time by unit

$ systemd-analyze blame # boot time by unit

$ systemctl --failed # units that failed during boot

$ systemctl status <service-name> # e.g. systemctl status NetworkManager.service

=== Access Recovery Mode ===
Reboot and hold Shift to access the GRUB menu (timing depends on your GRUB configuration). Select “Advanced options”, then choose recovery mode. From the recovery menu, select “root” for a root shell.

=== Common Boot Fixes ===
$ sudo fsck /dev/sdXN # repair filesystem — identify device with lsblk first

$ sudo grub-install /dev/sdX # reinstall GRUB bootloader

$ sudo update-grub # regenerate GRUB configuration

$ cat /etc/fstab # check fstab for mount errors

== Network Issues ==

=== Diagnose Network Connection ===
$ ip addr show # check network interfaces

$ ping -c 4 8.8.8.8 # test IPv4 connectivity

$ ping -c 6 2a00:1450:4007:809::200e # test IPv6 connectivity

$ nslookup google.com # check DNS resolution

$ ip route show # view routing table

$ ss -tuln # check active connections

=== Restart Network Service ===
$ sudo systemctl restart NetworkManager # for systems using NetworkManager

$ sudo systemctl restart networking # for systems using networking service

$ sudo ip link set eth0 down # bring interface down

$ sudo ip link set eth0 up # bring interface back up

== MTR (Matt’s Traceroute) ==
MTR combines the functionality of ping and traceroute into a single real-time network diagnostic tool. It continuously monitors the path between your system and a destination, providing detailed statistics about latency and packet loss at each hop. When you need to demonstrate a network problem that is outside your control—to your hosting provider, for example—MTR gives you the evidence you need.

=== Installation ===
$ sudo apt install mtr

=== Basic Usage ===
$ mtr google.com # interactive mode

$ mtr --report google.com # report mode (10 cycles then exit)

$ mtr --report-cycles 50 google.com # specify number of cycles

$ mtr --tcp google.com # use TCP instead of ICMP

$ mtr --udp google.com # use UDP

$ mtr --no-dns google.com # skip DNS resolution (faster)

$ mtr --show-ips google.com # show both hostnames and IPs

== Understanding MTR Output ==

==== Sample Output ====
HOST: hostname Loss% Snt Last Avg Best Wrst StDev
1.|-- 192.168.1.1 0.0% 10 1.2 1.5 1.0 2.3 0.4
2.|-- 10.0.0.1 0.0% 10 8.5 9.2 7.8 12.1 1.3
3.|-- isp-gateway.net 0.0% 10 15.3 16.1 14.2 19.8 1.8
4.|-- ??? 100.0% 10 0.0 0.0 0.0 0.0 0.0
5.|-- google.com 0.0% 10 25.4 26.8 24.1 32.5 2.4

=== Column Meanings ===
'''HOST''' — Hostname or IP address of each hop in the route.

'''Loss%''' — Percentage of packets lost at this hop.

'''Snt''' — Number of packets sent to this hop.

'''Last''' — Latency of the most recent packet, in milliseconds.

'''Avg''' — Average latency across all packets sent, in milliseconds.

'''Best''' — Lowest latency recorded, in milliseconds.

'''Wrst''' — Highest latency recorded, in milliseconds.

'''StDev''' — Standard deviation of latency measurements—lower values indicate a more stable connection.

=== Interpreting Results ===
A healthy network shows 0% loss on all hops, stable latency with low StDev values, a gradual latency increase as hop count increases, and consistent response times.

==== Problem Indicators ====
'''1. High Packet Loss at a Specific Hop'''
5.|-- problem-router.net 25.0% 10 45.3 48.2 42.1 65.8 8.4
If loss continues to the destination, there is a real problem at that router. If loss appears only at that hop but not beyond, the router is likely rate-limiting ICMP responses—a false positive, not a real problem. The rule of thumb: if packet loss appears at hop N but hops N+1, N+2, and beyond show 0% loss, it’s almost certainly ICMP rate limiting.

'''2. High Latency at a Specific Hop'''
3.|-- slow-link.net 0.0% 10 150.3 155.2 148.1 165.8 5.4
This indicates a network bottleneck, a congested link, geographical distance (intercontinental hops), or slow routing equipment.

'''3. No Response (???)'''
4.|-- ??? 100.0% 10 0.0 0.0 0.0 0.0 0.0
The router at this hop is configured not to respond to ICMP or traceroute packets, or a firewall is blocking diagnostic packets. This is not necessarily a problem if later hops respond normally.

'''4. High Jitter (StDev)'''
6.|-- unstable.net 0.0% 10 35.3 52.8 28.1 95.2 24.7
A high StDev (24.7ms in this example) indicates inconsistent latency caused by network congestion or instability. This is particularly noticeable in real-time applications such as VoIP, gaming, and video calls.

'''5. Sudden Latency Spike'''
1.|-- 192.168.1.1 0.0% 10 1.2 1.5 1.0 2.3 0.4
2.|-- 10.0.0.1 0.0% 10 8.5 9.2 7.8 12.1 1.3
3.|-- problematic-hop.net 0.0% 10 180.5 185.2 178.1 195.8 6.4
4.|-- next-hop.net 0.0% 10 182.3 187.8 180.5 198.2 6.8
Hop 3 introduces approximately 170ms of additional latency (jumping from 9ms to 180ms). Everything from hop 3 onward carries that penalty.

=== Advanced Usage ===

==== Report Mode with Different Output Formats ====
$ mtr --report --csv google.com > network-report.csv # CSV for logging
$ mtr --report --json google.com # JSON for parsing
$ mtr --report --xml google.com # XML format
$ mtr --report-wide google.com # wide report, no abbreviations

==== Protocol Selection ====
$ mtr google.com # ICMP (default)
$ mtr --udp google.com # UDP
$ sudo mtr --tcp --port 443 google.com # specific TCP port
$ sudo mtr --tcp --port 22 remote-server.com

==== Timing and Duration ====
$ mtr --interval 0.5 google.com # interval between pings (default 1s)
$ mtr --report-cycles 100 google.com # extended test, 100 cycles
$ mtr google.com # continuous monitoring (Ctrl+C to stop)
$ mtr --report-cycles 10 google.com # quick 10-cycle report

==== Advanced Options ====
$ mtr --aslookup google.com # show Autonomous System (AS) numbers
$ mtr --max-ttl 20 google.com # set maximum number of hops
$ mtr --psize 1000 google.com # set packet size
$ mtr --show-ips google.com # show both IP and hostname
$ mtr --address 192.168.1.100 google.com # specify source address
$ mtr -4 google.com # IPv4 only
$ mtr -6 google.com # IPv6 only

=== Interactive Mode Commands ===
When running MTR in interactive mode (mtr hostname), the following keys are available:

'''h''' — Display help.

'''d''' — Toggle display mode, cycling through different views.

'''n''' — Toggle between hostnames and IP addresses.

'''r''' — Reset all statistics.

'''p''' — Pause/unpause the display.

'''q''' — Quit MTR.

'''u''' — Switch between ICMP, UDP, and TCP modes.

'''y''' — Switch between IPv4 and IPv6.

'''o''' — Toggle field display options.

'''j''' — Toggle latency display.

=== Practical Troubleshooting Scenarios ===

==== Scenario 1: Diagnosing a Slow Website ====
$ mtr --report-cycles 100 --no-dns example.com
Look for high average latency at specific hops, packet loss at the destination, and high StDev values indicating jitter.

==== Scenario 2: Testing if a Firewall Blocks SSH ====
$ sudo mtr --tcp --port 22 --report-cycles 50 remote-server.com
If the last hop shows 100% loss but earlier hops are fine, port 22 may be filtered. Run a standard ICMP test for comparison.

==== Scenario 3: ISP Performance Issues ====
$ mtr --report-cycles 100 8.8.8.8

$ mtr --report-cycles 100 1.1.1.1

If issues appear in the first three or four hops, the problem is likely within your ISP’s network. If issues appear later, the problem lies beyond your ISP.

==== Scenario 4: VPN Troubleshooting ====
$ mtr --report-cycles 50 --no-dns google.com > before-vpn.txt # before VPN
$ mtr --report-cycles 50 --no-dns google.com > after-vpn.txt # after VPN
$ diff before-vpn.txt after-vpn.txt # compare the two

==== Scenario 5: Gaming/Streaming Performance ====
$ mtr --report-cycles 200 game-server.com
For real-time applications, look for low average latency (under 50ms for gaming), low StDev (under 5ms preferred), and zero packet loss.

==== Scenario 6: Intermittent Connectivity ====
$ mtr --report-cycles 500 --interval 1 target.com > long-test.txt # long-running test
$ mtr target.com # real-time monitoring

=== Continuous Monitoring ===

==== Log Network Performance Over Time ====
while true; do
timestamp=$(date +%Y%m%d-%H%M%S)
mtr --report --report-cycles 50 google.com > "mtr-$timestamp.txt"
sleep 3600
done

==== Monitor Multiple Destinations ====
#!/bin/bash
echo "=== MTR Report $(date) ===" > daily-network-report.txt
echo "" >> daily-network-report.txt

echo "Google DNS:" >> daily-network-report.txt
mtr --report --report-cycles 50 --no-dns 8.8.8.8 >> daily-network-report.txt
echo "" >> daily-network-report.txt

echo "Cloudflare DNS:" >> daily-network-report.txt
mtr --report --report-cycles 50 --no-dns 1.1.1.1 >> daily-network-report.txt
echo "" >> daily-network-report.txt

echo "Your Server:" >> daily-network-report.txt
mtr --report --report-cycles 50 your-server.com >> daily-network-report.txt

=== Useful Aliases for .bashrc ===
Quick network path analysis
alias mtrreport='mtr --report --report-cycles 50 --no-dns'

Monitor connection to Google DNS
alias netcheck='mtr --report-cycles 20 8.8.8.8'

Extended network test
alias mtrlong='mtr --report-cycles 100'

TCP port 443 test (HTTPS)
alias mtrhttps='sudo mtr --tcp --port 443 --report-cycles 30'

Quick comparison of major DNS providers
alias dnstest='echo "Google:" && mtr --report-cycles 20 8.8.8.8 && echo -e "\nCloudflare:" && mtr --report-cycles 20 1.1.1.1'
After adding to ~/.bashrc, reload with:
$ source ~/.bashrc

==== Useful Tip for Aliases ====
Rather than adding aliases directly to <code>~/.bashrc</code>, it's cleaner to keep them in a dedicated file—<code>~/.bashrc_aliases</code>—and source that file from <code>~/.bashrc</code>. This keeps your shell configuration tidy and makes it easy to share or back up just your aliases without touching the rest of your environment.

=== Troubleshooting Tips ===
'''Permission issues with TCP mode:'''
$ sudo mtr --tcp --port 443 example.com # Use sudo for TCP on privileged ports
$ sudo setcap cap_net_raw+ep /usr/bin/mtr-packet # Or set capabilities (one-time setup)
'''False positives:''' Packet loss at an intermediate hop but not at the destination is almost always ICMP rate limiting at that router, not a real problem. Example:
3.|-- router.isp.net 20.0% 50 15.3 16.1 14.2 19.8 1.8
4.|-- next-hop.net 0.0% 50 18.5 19.2 17.8 22.1 1.3
5.|-- destination.com 0.0% 50 25.4 26.8 24.1 32.5 2.4
Hop 3 shows 20% loss, but hops 4 and 5 show 0%. Traffic is flowing normally.

'''DNS resolution delays:''' If MTR is slow to start, skip DNS resolution with mtr --no-dns target.com and resolve addresses afterward with host 203.0.113.1 if needed.

'''Comparing results:'''
$ mtr --report --report-cycles 50 example.com > test1.txt
$ sleep 60
$ mtr --report --report-cycles 50 example.com > test2.txt
$ diff test1.txt test2.txt

=== When to Use MTR vs Other Tools ===
'''MTR''' — Best for continuous monitoring, identifying problem hops, and gathering detailed statistics. Requires installation via apt.

'''ping''' — Best for a quick connectivity test or simple latency check. Only tests the endpoint, not the path.

'''traceroute''' — Best for one-time path discovery. Provides no continuous monitoring.

'''ss / netstat''' — Best for checking local connection status. Doesn’t test remote paths.

=== Best Practices ===
Run at least 50–100 cycles for accurate statistics. Use --no-dns for faster results and to avoid DNS resolution issues during testing. Test at different times of day, since network conditions vary. If one protocol shows issues, compare ICMP, UDP, and TCP modes. Save reports with timestamps for trend analysis. Use 8.8.8.8 or 1.1.1.1 as known-good hosts to verify your local network first. Let MTR run for at least 30–60 seconds before drawing conclusions.

=== Reading Between the Lines ===
'''Good network health:'''
HOST: hostname Loss% Snt Last Avg Best Wrst StDev
1.|-- 192.168.1.1 0.0% 50 1.1 1.2 0.9 2.1 0.2
2.|-- 10.0.0.1 0.0% 50 8.2 8.5 7.5 10.2 0.5
3.|-- isp-gateway.net 0.0% 50 15.1 15.5 14.0 18.3 0.8
4.|-- google.com 0.0% 50 24.8 25.2 23.5 28.1 1.1
No packet loss, consistent latency, low jitter throughout.

'''Problematic network:'''
HOST: hostname Loss% Snt Last Avg Best Wrst StDev
1.|-- 192.168.1.1 0.0% 50 1.2 1.5 1.0 2.3 0.3
2.|-- 10.0.0.1 5.0% 50 45.3 52.8 8.1 245.2 45.7
3.|-- ??? 100.0% 50 0.0 0.0 0.0 0.0 0.0
4.|-- destination.com 15.0% 50 95.4 125.8 48.2 385.5 78.2
Packet loss at hop 2 and the destination, high jitter, and a very high worst-case latency all point to a genuine problem at or near hop 2.

=== Summary ===
MTR is your friend when it comes to diagnostics. Use --report-cycles 50 or higher for reliable data. Watch for packet loss at the destination—intermediate losses may be false positives. High StDev indicates an unstable connection. High average latency points to slow links. Use --no-dns for faster results. Compare ICMP, UDP, and TCP modes if issues appear. Test at different times of day for a complete picture.
== Package Management Issues ==

=== Fix Broken Packages ===
$ sudo apt update # update package lists
$ sudo apt --fix-broken install # fix broken dependencies
$ sudo dpkg --configure -a # reconfigure packages (no output = nothing to do)
$ sudo apt clean # clean package cache
$ sudo apt autoclean
$ sudo apt autoremove # remove unused packages

=== Handle Held or Locked Packages ===
$ sudo lsof /var/lib/dpkg/lock-frontend # find process holding the lock
$ sudo rm /var/lib/dpkg/lock-frontend # remove lock (only if no package manager is running)
$ sudo rm /var/lib/apt/lists/lock
$ sudo dpkg --configure -a # reconfigure dpkg

== Disk & Filesystem Issues ==

=== Check Disk Health ===
$ df -h # disk space
$ df -i # inode usage
$ sudo apt install sysstat # sysstat includes iostat, mpstat, sar, pidstat, cifsiostat
$ iostat -x 1 # disk I/O statistics
$ iostat -xm 2 # stats in MB instead of KB
$ iostat -x sda 1 # monitor a specific device
$ sudo dmesg | grep -i "error\|fail" # check for disk errors
$ sudo apt install smartmontools
$ sudo smartctl -a /dev/sda # SMART disk health 

# Fix broken dependencies
sudo apt --fix-broken install

# Reconfigure packages
sudo dpkg --configure -a
# No output means there is nothing to reconfigure

# Clean package cache
sudo apt clean
sudo apt autoclean

# Remove unused packages
sudo apt autoremove

=== Handle Held or Locked Packages ===
# If apt is locked, find the process holding the lock
sudo lsof /var/lib/dpkg/lock-frontend

# Remove the lock (use with caution—only if no package manager is running)
sudo rm /var/lib/dpkg/lock-frontend
sudo rm /var/lib/apt/lists/lock

# Reconfigure dpkg
sudo dpkg --configure -a
Disk & Filesystem Issues

=== Check Disk Health ===
# Check disk space
df -h

# Check inode usage
df -i

# View disk I/O statistics
# iostat is part of the sysstat package, which also includes:
# mpstat (CPU statistics), sar (system activity reporter),
# pidstat (process statistics), and cifsiostat (CIFS statistics)
sudo apt install sysstat

iostat -x 1

# Show stats in MB instead of KB
iostat -xm 2

# Monitor a specific device
iostat -x sda 1

# Check for disk errors in dmesg
sudo dmesg | grep -i "error\|fail"

# SMART disk health (requires smartmontools)
sudo apt install smartmontools
sudo smartctl -a /dev/sda

=== Repair Filesystem ===
# Unmount the partition first
sudo umount /dev/sdXN

# Run filesystem check
sudo fsck /dev/sdXN

# For ext4 specifically
sudo e2fsck -f /dev/sdXN
Performance Issues

=== Identify Resource Hogs ===
# CPU usage by process
top -o %CPU

# Memory usage by process
top -o %MEM

# Disk usage by directory
du -sh /* | sort -h

# Find large files
find / -type f -size +100M 2>/dev/null

# Check running processes sorted by memory use
ps aux --sort=-%mem | head -20

=== System Temperature Monitoring ===
# Install lm-sensors if not already present
sudo apt install lm-sensors
sudo sensors-detect

# View temperatures
sensors

# Real-time temperature monitoring
watch -n 2 sensors
See section 12 for a shell alias that makes temperature monitoring more convenient.

Service & Application Errors

=== Debug Service Problems ===
# Check service status
sudo systemctl status service-name

# View service logs
sudo journalctl -u service-name

# Restart a service
sudo systemctl restart service-name

# Enable service at boot
sudo systemctl enable service-name

# View recent service failures
journalctl -p err -b

=== Application Crash Investigation ===
# Check for core dumps
ls -lh /var/crash/

# View available application logs
ls /var/log/

# Follow syslog for application errors in real-time
sudo tail -f /var/log/syslog
Permission & Access Issues

=== Fix Common Permission Problems ===
# Check file ownership
ls -l /path/to/file

# Change ownership (user:group)
sudo chown user:group /path/to/file

# Change permissions
sudo chmod 644 /path/to/file

# Recursively fix permissions on a directory
sudo chown -R user:group /path/to/directory

=== User & Authentication Issues ===
# Check user information
id username

# View user login history
last -a

# Check failed login attempts
sudo journalctl | grep "authentication failure"

# Reset user password
sudo passwd username
Hardware Issues

=== Identify Hardware ===
# List all hardware (install if needed: sudo apt install lshw)
sudo lshw -short

# PCI devices
lspci -v

# USB devices
lsusb -v

# CPU information
lscpu

# Memory information
sudo dmidecode --type memory

=== Check Hardware Errors ===
# Kernel ring buffer (hardware messages)
dmesg | less
# Press q to quit

# Search for specific hardware issues
dmesg | grep -i "error\|fail\|warn"

# Check for USB issues
dmesg | grep -i usb
Quick Diagnostic Commands

=== System Information at a Glance ===
# Uptime and load average
uptime

# Kernel version
uname -r

# Debian version
cat /etc/debian_version

# Full system summary (install if needed: sudo apt install inxi)
sudo inxi -Fxz

=== Emergency Toolkit ===
# Create a consolidated diagnostic report
sudo journalctl -b > ~/system-report.txt
dmesg >> ~/system-report.txt
systemctl --failed >> ~/system-report.txt
df -h >> ~/system-report.txt
free -h >> ~/system-report.txt

# Watch logs in real-time
sudo journalctl -f

# Monitor system resources continuously
watch -n 1 'free -h && df -h'
Useful Aliases & Shortcuts
Add these to your ~/.bashrc for quick access to common troubleshooting commands:
# Monitor system temperatures in real-time
alias temps="watch -n 2 'for i in /sys/class/hwmon/hwmon*/; do echo -n \"\$(cat \${i}name): \"; cat \${i}temp*_input 2>/dev/null | while read temp; do echo \"scale=1; \$temp/1000\" | bc; done | tr \"\n\" \" \"; echo \"°C\"; done'"

# Quick system status
alias sysstat='echo "=== CPU ===" && uptime && echo -e "\n=== Memory ===" && free -h && echo -e "\n=== Disk ===" && df -h && echo -e "\n=== Top Processes ===" && ps aux --sort=-%mem | head -10'

# View last boot logs
alias lastboot='journalctl -b -1'

# Check failed services
alias failedservices='systemctl --failed'

# Monitor logs in real-time
alias watchlog='sudo journalctl -f'

# Quick network status
alias netstatus='ip addr show && echo -e "\n=== Routes ===" && ip route show'
After adding these, reload your shell configuration:
source ~/.bashrc
Tips for Effective Troubleshooting

'''Check logs first.''' journalctl and dmesg are your best starting point for almost any problem.

'''Change one thing at a time.''' It’s tempting to make several changes at once, but doing so makes it impossible to know what actually fixed—or broke—the problem.

'''Document your changes.''' Keep notes on what you’ve tried and what the results were. You’ll thank yourself later.

'''Use exact error messages.''' Copy error messages verbatim and search for them alongside “Debian” and the version number—for example, “Debian 13” or “Debian 13.1” for point releases.

'''Check what changed recently.''' Did you install something, update packages, or change the kernel before the problem appeared? Start there.

'''Back up before making major changes.''' This goes without saying, but it’s easy to skip when you’re in a hurry.

'''Use verbose mode.''' Adding -v or -vv flags to commands often reveals what would otherwise be silent failures.

'''Use the community.''' The Debian forums, Reddit, Stack Exchange, and the Debian mailing lists are all active and well-indexed. Someone has usually seen your problem before.

''This work was contributed by <dfn>distro-nix</dfn> to the Debian User Forums on 2025-10-27.''

Debian First Aid Kit

2026-05-15T22:40:47Z

Distro-nix: inserting sections

'''Author''' '''Profile:''' [https://forums.debian.net/memberlist.php?mode=viewprofile&u=239099 distro-nix] '''Debian Version:''' Debian 13 (Trixie) '''Kernel Versions Tested:''' 6.12.73+deb13-amd64 and 6.16.3+deb13-amd64 (x86_64) '''Created:''' 2025-10-27 '''Last Updated:''' 2026-05-15 '''ID:''' 544201 '''License:''' CC BY-SA 4.0

''This guide is preserved in its original form. Questions and feedback welcome via the author’s <dfn>Debian Forums profile</dfn>.''

== Abstract ==
The Debian First Aid Kit is a practical reference for diagnosing and resolving common system problems on Debian GNU/Linux. It covers commonly occuring categories of failure: system freezes and crashes, boot problems, network issues, package management, disk and filesystem errors, performance degradation, service and application failures, permission and access problems, hardware identification and errors, and quick diagnostics. Each section gives the relevant commands so you can understand what they do and why, rather than a bare list to copy and paste blindly. A dedicated section covers MTR (Matt’s Traceroute) in depth, including output interpretation, practical troubleshooting, and how to monitor continuously. The guide also includes a set of shell aliases for faster access to common diagnostic tasks. All commands are verified on Debian 13 (Trixie) running kernel 6.12.73+deb13-amd64 and 6.16.3+deb13-amd64. This document is intended for users with basic command-line familiarity who need a reliable, consolidated reference when something goes wrong.

'''Keywords:''' Debian, troubleshooting, system administration, MTR, network diagnostics, package management, journalctl, filesystem repair
----

== Tested Commands Disclaimer ==
All commands in this document have been verified on Debian 13 (Trixie) with the kernel versions listed above. Results may vary on other releases or hardware configurations. Always test in a safe environment before applying any procedure to a production system.

== Conventions Used in This Document ==
Command examples throughout this guide follow standard Unix prompt conventions. A $ prefix indicates a command run as a regular user. Where elevated privileges are required, sudo is shown explicitly and # with a brief explanation if needed:
$ journalctl -b -1 # regular user
$ sudo apt install mtr # requires elevation via sudo
In recovery shell contexts, where you are operating directly as root, the prompt is #.

== System Freezes & Crashes ==

=== Check System Logs ===

=== Common Freeze Causes to Look For ===

* '''Kernel panics:''' Search for “kernel panic” or “Oops”
* '''Out of Memory (OOM):''' Search for “Out of memory” or “oom-killer”
* '''Hardware errors:''' Look for “MCE” (Machine Check Exception) or “hardware error”
* '''Driver issues:''' Check for module/driver failures
* '''Overheating:''' Check system temperatures

=== Check System Resources ===
$ free -h # memory usage

$ df -h # disk space

$ htop # interactive resource monitor

$ top # alternative resource monitor

$ sudo apt install btop # btop is a more detailed alternative

$ sudo dmesg | grep -i error # check for disk errors

=== Access Recovery Mode ===
Reboot and hold Shift to access the GRUB menu (timing depends on your GRUB configuration). Select “Advanced options”, then choose recovery mode. From the recovery menu, select “root” for a root shell.

=== Common Boot Fixes ===
$ sudo fsck /dev/sdXN # repair filesystem — identify device with lsblk first

$ sudo grub-install /dev/sdX # reinstall GRUB bootloader

$ sudo update-grub # regenerate GRUB configuration

$ cat /etc/fstab # check fstab for mount errors

The following are permanent errors caused by incomplete or buggy ACPI tables in the BIOS. They are harmless and can be safely ignored if they appear on your system:
[0.686554] ACPI Error: No handler for Region [ECRM] (00000000201accc4) [EmbeddedControl] (20250404/evregion-131)
[0.686577] ACPI Error: Region EmbeddedControl (ID=3) has no handler (20250404/exfldio-261)
[0.686594] ACPI Error: Aborting method \_SB.GPIO._EVT due to previous error (AE_NOT_EXIST) (20250404/psparse-529)

== Boot Problems ==

=== Check Boot Process ===
$ systemd-analyze blame # boot time by unit

$ systemd-analyze blame # boot time by unit

$ systemctl --failed # units that failed during boot

$ systemctl status <service-name> # e.g. systemctl status NetworkManager.service

=== Access Recovery Mode ===
Reboot and hold Shift to access the GRUB menu (timing depends on your GRUB configuration). Select “Advanced options”, then choose recovery mode. From the recovery menu, select “root” for a root shell.

=== Common Boot Fixes ===
$ sudo fsck /dev/sdXN # repair filesystem — identify device with lsblk first

$ sudo grub-install /dev/sdX # reinstall GRUB bootloader

$ sudo update-grub # regenerate GRUB configuration

$ cat /etc/fstab # check fstab for mount errors

== Network Issues ==

=== Diagnose Network Connection ===
$ ip addr show # check network interfaces

$ ping -c 4 8.8.8.8 # test IPv4 connectivity

$ ping -c 6 2a00:1450:4007:809::200e # test IPv6 connectivity

$ nslookup google.com # check DNS resolution

$ ip route show # view routing table

$ ss -tuln # check active connections

=== Restart Network Service ===
$ sudo systemctl restart NetworkManager # for systems using NetworkManager

$ sudo systemctl restart networking # for systems using networking service

$ sudo ip link set eth0 down # bring interface down

$ sudo ip link set eth0 up # bring interface back up

== MTR (Matt’s Traceroute) ==
MTR combines the functionality of ping and traceroute into a single real-time network diagnostic tool. It continuously monitors the path between your system and a destination, providing detailed statistics about latency and packet loss at each hop. When you need to demonstrate a network problem that is outside your control—to your hosting provider, for example—MTR gives you the evidence you need.

=== Installation ===
$ sudo apt install mtr

=== Basic Usage ===
$ mtr google.com # interactive mode

$ mtr --report google.com # report mode (10 cycles then exit)

$ mtr --report-cycles 50 google.com # specify number of cycles

$ mtr --tcp google.com # use TCP instead of ICMP

$ mtr --udp google.com # use UDP

$ mtr --no-dns google.com # skip DNS resolution (faster)

$ mtr --show-ips google.com # show both hostnames and IPs

== Understanding MTR Output ==

==== Sample Output ====
HOST: hostname Loss% Snt Last Avg Best Wrst StDev
1.|-- 192.168.1.1 0.0% 10 1.2 1.5 1.0 2.3 0.4
2.|-- 10.0.0.1 0.0% 10 8.5 9.2 7.8 12.1 1.3
3.|-- isp-gateway.net 0.0% 10 15.3 16.1 14.2 19.8 1.8
4.|-- ??? 100.0% 10 0.0 0.0 0.0 0.0 0.0
5.|-- google.com 0.0% 10 25.4 26.8 24.1 32.5 2.4

=== Column Meanings ===
'''HOST''' — Hostname or IP address of each hop in the route.

'''Loss%''' — Percentage of packets lost at this hop.

'''Snt''' — Number of packets sent to this hop.

'''Last''' — Latency of the most recent packet, in milliseconds.

'''Avg''' — Average latency across all packets sent, in milliseconds.

'''Best''' — Lowest latency recorded, in milliseconds.

'''Wrst''' — Highest latency recorded, in milliseconds.

'''StDev''' — Standard deviation of latency measurements—lower values indicate a more stable connection.

=== Interpreting Results ===
A healthy network shows 0% loss on all hops, stable latency with low StDev values, a gradual latency increase as hop count increases, and consistent response times.

==== Problem Indicators ====
'''1. High Packet Loss at a Specific Hop'''
5.|-- problem-router.net 25.0% 10 45.3 48.2 42.1 65.8 8.4
If loss continues to the destination, there is a real problem at that router. If loss appears only at that hop but not beyond, the router is likely rate-limiting ICMP responses—a false positive, not a real problem. The rule of thumb: if packet loss appears at hop N but hops N+1, N+2, and beyond show 0% loss, it’s almost certainly ICMP rate limiting.

'''2. High Latency at a Specific Hop'''
3.|-- slow-link.net 0.0% 10 150.3 155.2 148.1 165.8 5.4
This indicates a network bottleneck, a congested link, geographical distance (intercontinental hops), or slow routing equipment.

'''3. No Response (???)'''
4.|-- ??? 100.0% 10 0.0 0.0 0.0 0.0 0.0
The router at this hop is configured not to respond to ICMP or traceroute packets, or a firewall is blocking diagnostic packets. This is not necessarily a problem if later hops respond normally.

'''4. High Jitter (StDev)'''
6.|-- unstable.net 0.0% 10 35.3 52.8 28.1 95.2 24.7
A high StDev (24.7ms in this example) indicates inconsistent latency caused by network congestion or instability. This is particularly noticeable in real-time applications such as VoIP, gaming, and video calls.

'''5. Sudden Latency Spike'''
1.|-- 192.168.1.1 0.0% 10 1.2 1.5 1.0 2.3 0.4
2.|-- 10.0.0.1 0.0% 10 8.5 9.2 7.8 12.1 1.3
3.|-- problematic-hop.net 0.0% 10 180.5 185.2 178.1 195.8 6.4
4.|-- next-hop.net 0.0% 10 182.3 187.8 180.5 198.2 6.8
Hop 3 introduces approximately 170ms of additional latency (jumping from 9ms to 180ms). Everything from hop 3 onward carries that penalty.

=== Advanced Usage ===

==== Report Mode with Different Output Formats ====
$ mtr --report --csv google.com > network-report.csv # CSV for logging
$ mtr --report --json google.com # JSON for parsing
$ mtr --report --xml google.com # XML format
$ mtr --report-wide google.com # wide report, no abbreviations

==== Protocol Selection ====
$ mtr google.com # ICMP (default)
mtr --udp google.com # UDP
$ sudo mtr --tcp --port 443 google.com # specific TCP port
$ sudo mtr --tcp --port 22 remote-server.com

==== Timing and Duration ====
$ mtr --interval 0.5 google.com # interval between pings (default 1s)

$ mtr --report-cycles 100 google.com # extended test, 100 cycles

$ mtr google.com # continuous monitoring (Ctrl+C to stop)

$ mtr --report-cycles 10 google.com # quick 10-cycle report

==== Advanced Options ====
$ mtr --aslookup google.com # show Autonomous System (AS) numbers
$ mtr --max-ttl 20 google.com # set maximum number of hops
$ mtr --psize 1000 google.com # set packet size
$ mtr --show-ips google.com # show both IP and hostname
$ mtr --address 192.168.1.100 google.com # specify source address
$ mtr -4 google.com # IPv4 only
$ mtr -6 google.com # IPv6 only

=== Interactive Mode Commands ===
When running MTR in interactive mode (mtr hostname), the following keys are available:

'''h''' — Display help.

'''d''' — Toggle display mode, cycling through different views.

'''n''' — Toggle between hostnames and IP addresses.

'''r''' — Reset all statistics.

'''p''' — Pause/unpause the display.

'''q''' — Quit MTR.

'''u''' — Switch between ICMP, UDP, and TCP modes.

'''y''' — Switch between IPv4 and IPv6.

'''o''' — Toggle field display options.

'''j''' — Toggle latency display.

=== Practical Troubleshooting Scenarios ===

==== Scenario 1: Diagnosing a Slow Website ====
$ mtr --report-cycles 100 --no-dns example.com
Look for high average latency at specific hops, packet loss at the destination, and high StDev values indicating jitter.

==== Scenario 2: Testing if a Firewall Blocks SSH ====
$ sudo mtr --tcp --port 22 --report-cycles 50 remote-server.com
If the last hop shows 100% loss but earlier hops are fine, port 22 may be filtered. Run a standard ICMP test for comparison.

==== Scenario 3: ISP Performance Issues ====
$ mtr --report-cycles 100 8.8.8.8

$ mtr --report-cycles 100 1.1.1.1

If issues appear in the first three or four hops, the problem is likely within your ISP’s network. If issues appear later, the problem lies beyond your ISP.

==== Scenario 4: VPN Troubleshooting ====
$ mtr --report-cycles 50 --no-dns google.com > before-vpn.txt # before VPN
$ mtr --report-cycles 50 --no-dns google.com > after-vpn.txt # after VPN
$ diff before-vpn.txt after-vpn.txt # compare the two

==== Scenario 5: Gaming/Streaming Performance ====
$ mtr --report-cycles 200 game-server.com
For real-time applications, look for low average latency (under 50ms for gaming), low StDev (under 5ms preferred), and zero packet loss.

==== Scenario 6: Intermittent Connectivity ====
$ mtr --report-cycles 500 --interval 1 target.com > long-test.txt # long-running test

$ mtr target.com # real-time monitoring

=== Continuous Monitoring ===

==== Log Network Performance Over Time ====
while true; do
timestamp=$(date +%Y%m%d-%H%M%S)
mtr --report --report-cycles 50 google.com > "mtr-$timestamp.txt"
sleep 3600
done

==== Monitor Multiple Destinations ====
#!/bin/bash
echo "=== MTR Report $(date) ===" > daily-network-report.txt
echo "" >> daily-network-report.txt

echo "Google DNS:" >> daily-network-report.txt
mtr --report --report-cycles 50 --no-dns 8.8.8.8 >> daily-network-report.txt
echo "" >> daily-network-report.txt

echo "Cloudflare DNS:" >> daily-network-report.txt
mtr --report --report-cycles 50 --no-dns 1.1.1.1 >> daily-network-report.txt
echo "" >> daily-network-report.txt

echo "Your Server:" >> daily-network-report.txt
mtr --report --report-cycles 50 your-server.com >> daily-network-report.txt

=== Useful Aliases for .bashrc ===
Quick network path analysis
alias mtrreport='mtr --report --report-cycles 50 --no-dns'

Monitor connection to Google DNS
alias netcheck='mtr --report-cycles 20 8.8.8.8'

Extended network test
alias mtrlong='mtr --report-cycles 100'

TCP port 443 test (HTTPS)
alias mtrhttps='sudo mtr --tcp --port 443 --report-cycles 30'

Quick comparison of major DNS providers
alias dnstest='echo "Google:" && mtr --report-cycles 20 8.8.8.8 && echo -e "\nCloudflare:" && mtr --report-cycles 20 1.1.1.1'
After adding to ~/.bashrc, reload with:
$ source ~/.bashrc

=== Troubleshooting Tips ===
'''Permission issues with TCP mode:'''
Use sudo for TCP on privileged ports
$ sudo mtr --tcp --port 443 example.com

Or set capabilities (one-time setup)
$ sudo setcap cap_net_raw+ep /usr/bin/mtr-packet
'''False positives:''' Packet loss at an intermediate hop but not at the destination is almost always ICMP rate limiting at that router, not a real problem. Example:
3.|-- router.isp.net 20.0% 50 15.3 16.1 14.2 19.8 1.8
4.|-- next-hop.net 0.0% 50 18.5 19.2 17.8 22.1 1.3
5.|-- destination.com 0.0% 50 25.4 26.8 24.1 32.5 2.4
Hop 3 shows 20% loss, but hops 4 and 5 show 0%. Traffic is flowing normally.

'''DNS resolution delays:''' If MTR is slow to start, skip DNS resolution with mtr --no-dns target.com and resolve addresses afterward with host 203.0.113.1 if needed.

'''Comparing results:'''
$ mtr --report --report-cycles 50 example.com > test1.txt
$ sleep 60
$ mtr --report --report-cycles 50 example.com > test2.txt
$ diff test1.txt test2.txt

=== When to Use MTR vs Other Tools ===
'''MTR''' — Best for continuous monitoring, identifying problem hops, and gathering detailed statistics. Requires installation via apt.

'''ping''' — Best for a quick connectivity test or simple latency check. Only tests the endpoint, not the path.

'''traceroute''' — Best for one-time path discovery. Provides no continuous monitoring.

'''ss / netstat''' — Best for checking local connection status. Doesn’t test remote paths.

=== Best Practices ===
Run at least 50–100 cycles for accurate statistics. Use --no-dns for faster results and to avoid DNS resolution issues during testing. Test at different times of day, since network conditions vary. If one protocol shows issues, compare ICMP, UDP, and TCP modes. Save reports with timestamps for trend analysis. Use 8.8.8.8 or 1.1.1.1 as known-good hosts to verify your local network first. Let MTR run for at least 30–60 seconds before drawing conclusions.

=== Reading Between the Lines ===
'''Good network health:'''
HOST: hostname Loss% Snt Last Avg Best Wrst StDev
1.|-- 192.168.1.1 0.0% 50 1.1 1.2 0.9 2.1 0.2
2.|-- 10.0.0.1 0.0% 50 8.2 8.5 7.5 10.2 0.5
3.|-- isp-gateway.net 0.0% 50 15.1 15.5 14.0 18.3 0.8
4.|-- google.com 0.0% 50 24.8 25.2 23.5 28.1 1.1
No packet loss, consistent latency, low jitter throughout.

'''Problematic network:'''
HOST: hostname Loss% Snt Last Avg Best Wrst StDev
1.|-- 192.168.1.1 0.0% 50 1.2 1.5 1.0 2.3 0.3
2.|-- 10.0.0.1 5.0% 50 45.3 52.8 8.1 245.2 45.7
3.|-- ??? 100.0% 50 0.0 0.0 0.0 0.0 0.0
4.|-- destination.com 15.0% 50 95.4 125.8 48.2 385.5 78.2
Packet loss at hop 2 and the destination, high jitter, and a very high worst-case latency all point to a genuine problem at or near hop 2.

=== Summary ===
MTR is your friend when it comes to diagnostics. Use --report-cycles 50 or higher for reliable data. Watch for packet loss at the destination—intermediate losses may be false positives. High StDev indicates an unstable connection. High average latency points to slow links. Use --no-dns for faster results. Compare ICMP, UDP, and TCP modes if issues appear. Test at different times of day for a complete picture.
== Package Management Issues ==

=== Fix Broken Packages ===
$ sudo apt update # update package lists
$ sudo apt --fix-broken install # fix broken dependencies
$ sudo dpkg --configure -a # reconfigure packages (no output = nothing to do)
$ sudo apt clean # clean package cache
$ sudo apt autoclean
$ sudo apt autoremove # remove unused packages

=== Handle Held or Locked Packages ===
$ sudo lsof /var/lib/dpkg/lock-frontend # find process holding the lock
$ sudo rm /var/lib/dpkg/lock-frontend # remove lock (only if no package manager is running)
$ sudo rm /var/lib/apt/lists/lock
$ sudo dpkg --configure -a # reconfigure dpkg

== Disk & Filesystem Issues ==

=== Check Disk Health ===
$ df -h # disk space
$ df -i # inode usage
$ sudo apt install sysstat # sysstat includes iostat, mpstat, sar, pidstat, cifsiostat
$ iostat -x 1 # disk I/O statistics
$ iostat -xm 2 # stats in MB instead of KB
$ iostat -x sda 1 # monitor a specific device
$ sudo dmesg | grep -i "error\|fail" # check for disk errors
$ sudo apt install smartmontools

$ sudo smartctl -a /dev/sda # SMART disk health 

# Fix broken dependencies
sudo apt --fix-broken install

# Reconfigure packages
sudo dpkg --configure -a
# No output means there is nothing to reconfigure

# Clean package cache
sudo apt clean
sudo apt autoclean

# Remove unused packages
sudo apt autoremove

=== Handle Held or Locked Packages ===
# If apt is locked, find the process holding the lock
sudo lsof /var/lib/dpkg/lock-frontend

# Remove the lock (use with caution—only if no package manager is running)
sudo rm /var/lib/dpkg/lock-frontend
sudo rm /var/lib/apt/lists/lock

# Reconfigure dpkg
sudo dpkg --configure -a
Disk & Filesystem Issues

=== Check Disk Health ===
# Check disk space
df -h

# Check inode usage
df -i

# View disk I/O statistics
# iostat is part of the sysstat package, which also includes:
# mpstat (CPU statistics), sar (system activity reporter),
# pidstat (process statistics), and cifsiostat (CIFS statistics)
sudo apt install sysstat

iostat -x 1

# Show stats in MB instead of KB
iostat -xm 2

# Monitor a specific device
iostat -x sda 1

# Check for disk errors in dmesg
sudo dmesg | grep -i "error\|fail"

# SMART disk health (requires smartmontools)
sudo apt install smartmontools
sudo smartctl -a /dev/sda

=== Repair Filesystem ===
# Unmount the partition first
sudo umount /dev/sdXN

# Run filesystem check
sudo fsck /dev/sdXN

# For ext4 specifically
sudo e2fsck -f /dev/sdXN
Performance Issues

=== Identify Resource Hogs ===
# CPU usage by process
top -o %CPU

# Memory usage by process
top -o %MEM

# Disk usage by directory
du -sh /* | sort -h

# Find large files
find / -type f -size +100M 2>/dev/null

# Check running processes sorted by memory use
ps aux --sort=-%mem | head -20

=== System Temperature Monitoring ===
# Install lm-sensors if not already present
sudo apt install lm-sensors
sudo sensors-detect

# View temperatures
sensors

# Real-time temperature monitoring
watch -n 2 sensors
See section 12 for a shell alias that makes temperature monitoring more convenient.

Service & Application Errors

=== Debug Service Problems ===
# Check service status
sudo systemctl status service-name

# View service logs
sudo journalctl -u service-name

# Restart a service
sudo systemctl restart service-name

# Enable service at boot
sudo systemctl enable service-name

# View recent service failures
journalctl -p err -b

=== Application Crash Investigation ===
# Check for core dumps
ls -lh /var/crash/

# View available application logs
ls /var/log/

# Follow syslog for application errors in real-time
sudo tail -f /var/log/syslog
Permission & Access Issues

=== Fix Common Permission Problems ===
# Check file ownership
ls -l /path/to/file

# Change ownership (user:group)
sudo chown user:group /path/to/file

# Change permissions
sudo chmod 644 /path/to/file

# Recursively fix permissions on a directory
sudo chown -R user:group /path/to/directory

=== User & Authentication Issues ===
# Check user information
id username

# View user login history
last -a

# Check failed login attempts
sudo journalctl | grep "authentication failure"

# Reset user password
sudo passwd username
Hardware Issues

=== Identify Hardware ===
# List all hardware (install if needed: sudo apt install lshw)
sudo lshw -short

# PCI devices
lspci -v

# USB devices
lsusb -v

# CPU information
lscpu

# Memory information
sudo dmidecode --type memory

=== Check Hardware Errors ===
# Kernel ring buffer (hardware messages)
dmesg | less
# Press q to quit

# Search for specific hardware issues
dmesg | grep -i "error\|fail\|warn"

# Check for USB issues
dmesg | grep -i usb
Quick Diagnostic Commands

=== System Information at a Glance ===
# Uptime and load average
uptime

# Kernel version
uname -r

# Debian version
cat /etc/debian_version

# Full system summary (install if needed: sudo apt install inxi)
sudo inxi -Fxz

=== Emergency Toolkit ===
# Create a consolidated diagnostic report
sudo journalctl -b > ~/system-report.txt
dmesg >> ~/system-report.txt
systemctl --failed >> ~/system-report.txt
df -h >> ~/system-report.txt
free -h >> ~/system-report.txt

# Watch logs in real-time
sudo journalctl -f

# Monitor system resources continuously
watch -n 1 'free -h && df -h'
Useful Aliases & Shortcuts
Add these to your ~/.bashrc for quick access to common troubleshooting commands:
# Monitor system temperatures in real-time
alias temps="watch -n 2 'for i in /sys/class/hwmon/hwmon*/; do echo -n \"\$(cat \${i}name): \"; cat \${i}temp*_input 2>/dev/null | while read temp; do echo \"scale=1; \$temp/1000\" | bc; done | tr \"\n\" \" \"; echo \"°C\"; done'"

# Quick system status
alias sysstat='echo "=== CPU ===" && uptime && echo -e "\n=== Memory ===" && free -h && echo -e "\n=== Disk ===" && df -h && echo -e "\n=== Top Processes ===" && ps aux --sort=-%mem | head -10'

# View last boot logs
alias lastboot='journalctl -b -1'

# Check failed services
alias failedservices='systemctl --failed'

# Monitor logs in real-time
alias watchlog='sudo journalctl -f'

# Quick network status
alias netstatus='ip addr show && echo -e "\n=== Routes ===" && ip route show'
After adding these, reload your shell configuration:
source ~/.bashrc
Tips for Effective Troubleshooting

'''Check logs first.''' journalctl and dmesg are your best starting point for almost any problem.

'''Change one thing at a time.''' It’s tempting to make several changes at once, but doing so makes it impossible to know what actually fixed—or broke—the problem.

'''Document your changes.''' Keep notes on what you’ve tried and what the results were. You’ll thank yourself later.

'''Use exact error messages.''' Copy error messages verbatim and search for them alongside “Debian” and the version number—for example, “Debian 13” or “Debian 13.1” for point releases.

'''Check what changed recently.''' Did you install something, update packages, or change the kernel before the problem appeared? Start there.

'''Back up before making major changes.''' This goes without saying, but it’s easy to skip when you’re in a hurry.

'''Use verbose mode.''' Adding -v or -vv flags to commands often reveals what would otherwise be silent failures.

'''Use the community.''' The Debian forums, Reddit, Stack Exchange, and the Debian mailing lists are all active and well-indexed. Someone has usually seen your problem before.

''This work was contributed by <dfn>distro-nix</dfn> to the Debian User Forums on 2025-10-27.''

Debian First Aid Kit

2026-05-15T22:22:25Z

Distro-nix:

'''Author''' '''Profile:''' [https://forums.debian.net/memberlist.php?mode=viewprofile&u=239099 distro-nix] '''Debian Version:''' Debian 13 (Trixie) '''Kernel Versions Tested:''' 6.12.73+deb13-amd64 and 6.16.3+deb13-amd64 (x86_64) '''Created:''' 2025-10-27 '''Last Updated:''' 2026-05-15 '''ID:''' 544201 '''License:''' CC BY-SA 4.0

''This guide is preserved in its original form. Questions and feedback welcome via the author’s <dfn>Debian Forums profile</dfn>.''

== Abstract ==
The Debian First Aid Kit is a practical reference for diagnosing and resolving common system problems on Debian GNU/Linux. It covers commonly occuring categories of failure: system freezes and crashes, boot problems, network issues, package management, disk and filesystem errors, performance degradation, service and application failures, permission and access problems, hardware identification and errors, and quick diagnostics. Each section gives the relevant commands so you can understand what they do and why, rather than a bare list to copy and paste blindly. A dedicated section covers MTR (Matt’s Traceroute) in depth, including output interpretation, practical troubleshooting, and how to monitor continuously. The guide also includes a set of shell aliases for faster access to common diagnostic tasks. All commands are verified on Debian 13 (Trixie) running kernel 6.12.73+deb13-amd64 and 6.16.3+deb13-amd64. This document is intended for users with basic command-line familiarity who need a reliable, consolidated reference when something goes wrong.

'''Keywords:''' Debian, troubleshooting, system administration, MTR, network diagnostics, package management, journalctl, filesystem repair
----

== Tested Commands Disclaimer ==
All commands in this document have been verified on Debian 13 (Trixie) with the kernel versions listed above. Results may vary on other releases or hardware configurations. Always test in a safe environment before applying any procedure to a production system.

== Conventions Used in This Document ==
Command examples throughout this guide follow standard Unix prompt conventions. A $ prefix indicates a command run as a regular user. Where elevated privileges are required, sudo is shown explicitly and # with a brief explanation if needed:
$ journalctl -b -1 # regular user
$ sudo apt install mtr # requires elevation via sudo
In recovery shell contexts, where you are operating directly as root, the prompt is #.

== System Freezes & Crashes ==

=== Check System Logs ===

=== Common Freeze Causes to Look For ===

* '''Kernel panics:''' Search for “kernel panic” or “Oops”
* '''Out of Memory (OOM):''' Search for “Out of memory” or “oom-killer”
* '''Hardware errors:''' Look for “MCE” (Machine Check Exception) or “hardware error”
* '''Driver issues:''' Check for module/driver failures
* '''Overheating:''' Check system temperatures

=== Check System Resources ===
$ free -h # memory usage

$ df -h # disk space

$ htop # interactive resource monitor

$ top # alternative resource monitor

$ sudo apt install btop # btop is a more detailed alternative

$ sudo dmesg | grep -i error # check for disk errors

=== Access Recovery Mode ===
Reboot and hold Shift to access the GRUB menu (timing depends on your GRUB configuration). Select “Advanced options”, then choose recovery mode. From the recovery menu, select “root” for a root shell.

=== Common Boot Fixes ===
$ sudo fsck /dev/sdXN # repair filesystem — identify device with lsblk first

$ sudo grub-install /dev/sdX # reinstall GRUB bootloader

$ sudo update-grub # regenerate GRUB configuration

$ cat /etc/fstab # check fstab for mount errors

The following are permanent errors caused by incomplete or buggy ACPI tables in the BIOS. They are harmless and can be safely ignored if they appear on your system:
[0.686554] ACPI Error: No handler for Region [ECRM] (00000000201accc4) [EmbeddedControl] (20250404/evregion-131)
[0.686577] ACPI Error: Region EmbeddedControl (ID=3) has no handler (20250404/exfldio-261)
[0.686594] ACPI Error: Aborting method \_SB.GPIO._EVT due to previous error (AE_NOT_EXIST) (20250404/psparse-529)

== Boot Problems ==

=== Check Boot Process ===
$ systemd-analyze blame # boot time by unit

$ systemd-analyze blame # boot time by unit

$ systemctl --failed # units that failed during boot

$ systemctl status <service-name> # e.g. systemctl status NetworkManager.service

=== Access Recovery Mode ===
Reboot and hold Shift to access the GRUB menu (timing depends on your GRUB configuration). Select “Advanced options”, then choose recovery mode. From the recovery menu, select “root” for a root shell.

=== Common Boot Fixes ===
$ sudo fsck /dev/sdXN # repair filesystem — identify device with lsblk first

$ sudo grub-install /dev/sdX # reinstall GRUB bootloader

$ sudo update-grub # regenerate GRUB configuration

$ cat /etc/fstab # check fstab for mount errors

== Network Issues ==

=== Diagnose Network Connection ===
$ ip addr show # check network interfaces

$ ping -c 4 8.8.8.8 # test IPv4 connectivity

$ ping -c 6 2a00:1450:4007:809::200e # test IPv6 connectivity

$ nslookup google.com # check DNS resolution

$ ip route show # view routing table

$ ss -tuln # check active connections

=== Restart Network Service ===
$ sudo systemctl restart NetworkManager # for systems using NetworkManager

$ sudo systemctl restart networking # for systems using networking service

$ sudo ip link set eth0 down # bring interface down

$ sudo ip link set eth0 up # bring interface back up

== MTR (Matt’s Traceroute) ==
MTR combines the functionality of ping and traceroute into a single real-time network diagnostic tool. It continuously monitors the path between your system and a destination, providing detailed statistics about latency and packet loss at each hop. When you need to demonstrate a network problem that is outside your control—to your hosting provider, for example—MTR gives you the evidence you need.

=== Installation ===
$ sudo apt install mtr

=== Basic Usage ===
$ mtr google.com # interactive mode

$ mtr --report google.com # report mode (10 cycles then exit)

$ mtr --report-cycles 50 google.com # specify number of cycles

$ mtr --tcp google.com # use TCP instead of ICMP

$ mtr --udp google.com # use UDP

$ mtr --no-dns google.com # skip DNS resolution (faster)

$ mtr --show-ips google.com # show both hostnames and IPs

== Understanding MTR Output ==

==== Sample Output ====
HOST: hostname Loss% Snt Last Avg Best Wrst StDev
1.|-- 192.168.1.1 0.0% 10 1.2 1.5 1.0 2.3 0.4
2.|-- 10.0.0.1 0.0% 10 8.5 9.2 7.8 12.1 1.3
3.|-- isp-gateway.net 0.0% 10 15.3 16.1 14.2 19.8 1.8
4.|-- ??? 100.0% 10 0.0 0.0 0.0 0.0 0.0
5.|-- google.com 0.0% 10 25.4 26.8 24.1 32.5 2.4

=== Column Meanings ===
'''HOST''' — Hostname or IP address of each hop in the route.

'''Loss%''' — Percentage of packets lost at this hop.

'''Snt''' — Number of packets sent to this hop.

'''Last''' — Latency of the most recent packet, in milliseconds.

'''Avg''' — Average latency across all packets sent, in milliseconds.

'''Best''' — Lowest latency recorded, in milliseconds.

'''Wrst''' — Highest latency recorded, in milliseconds.

'''StDev''' — Standard deviation of latency measurements—lower values indicate a more stable connection.

=== Interpreting Results ===
A healthy network shows 0% loss on all hops, stable latency with low StDev values, a gradual latency increase as hop count increases, and consistent response times.

==== Problem Indicators ====
'''1. High Packet Loss at a Specific Hop'''
5.|-- problem-router.net 25.0% 10 45.3 48.2 42.1 65.8 8.4
If loss continues to the destination, there is a real problem at that router. If loss appears only at that hop but not beyond, the router is likely rate-limiting ICMP responses—a false positive, not a real problem. The rule of thumb: if packet loss appears at hop N but hops N+1, N+2, and beyond show 0% loss, it’s almost certainly ICMP rate limiting.

'''2. High Latency at a Specific Hop'''
3.|-- slow-link.net 0.0% 10 150.3 155.2 148.1 165.8 5.4
This indicates a network bottleneck, a congested link, geographical distance (intercontinental hops), or slow routing equipment.

'''3. No Response (???)'''
4.|-- ??? 100.0% 10 0.0 0.0 0.0 0.0 0.0
The router at this hop is configured not to respond to ICMP or traceroute packets, or a firewall is blocking diagnostic packets. This is not necessarily a problem if later hops respond normally.

'''4. High Jitter (StDev)'''
6.|-- unstable.net 0.0% 10 35.3 52.8 28.1 95.2 24.7
A high StDev (24.7ms in this example) indicates inconsistent latency caused by network congestion or instability. This is particularly noticeable in real-time applications such as VoIP, gaming, and video calls.

'''5. Sudden Latency Spike'''
1.|-- 192.168.1.1 0.0% 10 1.2 1.5 1.0 2.3 0.4
2.|-- 10.0.0.1 0.0% 10 8.5 9.2 7.8 12.1 1.3
3.|-- problematic-hop.net 0.0% 10 180.5 185.2 178.1 195.8 6.4
4.|-- next-hop.net 0.0% 10 182.3 187.8 180.5 198.2 6.8
Hop 3 introduces approximately 170ms of additional latency (jumping from 9ms to 180ms). Everything from hop 3 onward carries that penalty.

=== Advanced Usage ===

==== Report Mode with Different Output Formats ====
$ mtr --report --csv google.com > network-report.csv # CSV for logging

$ mtr --report --json google.com # JSON for parsing

$ mtr --report --xml google.com # XML format

$ mtr --report-wide google.com # wide report, no abbreviations

==== Protocol Selection ====
$ mtr google.com # ICMP (default)

$ mtr --udp google.com # UDP

$ mtr --tcp google.com # TCP

$ sudo mtr --tcp --port 443 google.com # specific TCP port

$ sudo mtr --tcp --port 22 remote-server.com # default TCP port

==== Timing and Duration ====
$ mtr --interval 0.5 google.com # interval between pings (default 1s)

$ mtr --report-cycles 100 google.com # extended test, 100 cycles

$ mtr google.com # continuous monitoring (Ctrl+C to stop)

$ mtr --report-cycles 10 google.com # quick 10-cycle report

==== Advanced Options ====
$ mtr --aslookup google.com # show Autonomous System (AS) numbers

$ mtr --max-ttl 20 google.com # set maximum number of hops

$ mtr --psize 1000 google.com # set packet size

$ mtr --show-ips google.com # show both IP and hostname

$ mtr --address 192.168.1.100 google.com # specify source address

$ mtr -4 google.com # IPv4 only

$ mtr -6 google.com # IPv6 only

=== Interactive Mode Commands ===
When running MTR in interactive mode (mtr hostname), the following keys are available:

'''h''' — Display help.

'''d''' — Toggle display mode, cycling through different views.

'''n''' — Toggle between hostnames and IP addresses.

'''r''' — Reset all statistics.

'''p''' — Pause/unpause the display.

'''q''' — Quit MTR.

'''u''' — Switch between ICMP, UDP, and TCP modes.

'''y''' — Switch between IPv4 and IPv6.

'''o''' — Toggle field display options.

'''j''' — Toggle latency display.

=== Practical Troubleshooting Scenarios ===

==== Scenario 1: Diagnosing a Slow Website ====
$ mtr --report-cycles 100 --no-dns example.com
Look for high average latency at specific hops, packet loss at the destination, and high StDev values indicating jitter.

==== Scenario 2: Testing if a Firewall Blocks SSH ====
$ sudo mtr --tcp --port 22 --report-cycles 50 remote-server.com
If the last hop shows 100% loss but earlier hops are fine, port 22 may be filtered. Run a standard ICMP test for comparison.

==== Scenario 3: ISP Performance Issues ====
$ mtr --report-cycles 100 8.8.8.8

$ mtr --report-cycles 100 1.1.1.1
If issues appear in the first three or four hops, the problem is likely within your ISP’s network. If issues appear later, the problem lies beyond your ISP.

==== Scenario 4: VPN Troubleshooting ====
$ mtr --report-cycles 50 --no-dns google.com > before-vpn.txt # before VPN

$ mtr --report-cycles 50 --no-dns google.com > after-vpn.txt # after VPN

$ diff before-vpn.txt after-vpn.txt # compare the two

==== Scenario 5: Gaming/Streaming Performance ====
$ mtr --report-cycles 200 game-server.com
For real-time applications, look for low average latency (under 50ms for gaming), low StDev (under 5ms preferred), and zero packet loss.

==== Scenario 6: Intermittent Connectivity ====
$ mtr --report-cycles 500 --interval 1 target.com > long-test.txt # long-running test

$ mtr target.com # real-time monitoring

=== Continuous Monitoring ===

==== Log Network Performance Over Time ====
while true; do
timestamp=$(date +%Y%m%d-%H%M%S)
mtr --report --report-cycles 50 google.com > "mtr-$timestamp.txt"
sleep 3600
done

==== Monitor Multiple Destinations ====
#!/bin/bash
echo "=== MTR Report $(date) ===" > daily-network-report.txt
echo "" >> daily-network-report.txt

echo "Google DNS:" >> daily-network-report.txt
mtr --report --report-cycles 50 --no-dns 8.8.8.8 >> daily-network-report.txt
echo "" >> daily-network-report.txt

echo "Cloudflare DNS:" >> daily-network-report.txt
mtr --report --report-cycles 50 --no-dns 1.1.1.1 >> daily-network-report.txt
echo "" >> daily-network-report.txt

echo "Your Server:" >> daily-network-report.txt
mtr --report --report-cycles 50 your-server.com >> daily-network-report.txt

=== Useful Aliases for .bashrc ===
Quick network path analysis
alias mtrreport='mtr --report --report-cycles 50 --no-dns'

Monitor connection to Google DNS
alias netcheck='mtr --report-cycles 20 8.8.8.8'

Extended network test
alias mtrlong='mtr --report-cycles 100'

TCP port 443 test (HTTPS)
alias mtrhttps='sudo mtr --tcp --port 443 --report-cycles 30'

Quick comparison of major DNS providers
alias dnstest='echo "Google:" && mtr --report-cycles 20 8.8.8.8 && echo -e "\nCloudflare:" && mtr --report-cycles 20 1.1.1.1'
After adding to ~/.bashrc, reload with:
source ~/.bashrc

=== Troubleshooting Tips ===
'''Permission issues with TCP mode:'''
Use sudo for TCP on privileged ports
$ sudo mtr --tcp --port 443 example.com

Or set capabilities (one-time setup)
$ sudo setcap cap_net_raw+ep /usr/bin/mtr-packet
'''False positives:''' Packet loss at an intermediate hop but not at the destination is almost always ICMP rate limiting at that router, not a real problem. Example:
3.|-- router.isp.net 20.0% 50 15.3 16.1 14.2 19.8 1.8
4.|-- next-hop.net 0.0% 50 18.5 19.2 17.8 22.1 1.3
5.|-- destination.com 0.0% 50 25.4 26.8 24.1 32.5 2.4
Hop 3 shows 20% loss, but hops 4 and 5 show 0%. Traffic is flowing normally.

'''DNS resolution delays:''' If MTR is slow to start, skip DNS resolution with mtr --no-dns target.com and resolve addresses afterward with host 203.0.113.1 if needed.

'''Comparing results:'''
$ mtr --report --report-cycles 50 example.com > test1.txt
$ sleep 60
$ mtr --report --report-cycles 50 example.com > test2.txt
$ diff test1.txt test2.txt

=== When to Use MTR vs Other Tools ===
'''MTR''' — Best for continuous monitoring, identifying problem hops, and gathering detailed statistics. Requires installation via apt.

'''ping''' — Best for a quick connectivity test or simple latency check. Only tests the endpoint, not the path.

'''traceroute''' — Best for one-time path discovery. Provides no continuous monitoring.

'''ss / netstat''' — Best for checking local connection status. Doesn’t test remote paths.

=== Best Practices ===
Run at least 50–100 cycles for accurate statistics. Use --no-dns for faster results and to avoid DNS resolution issues during testing. Test at different times of day, since network conditions vary. If one protocol shows issues, compare ICMP, UDP, and TCP modes. Save reports with timestamps for trend analysis. Use 8.8.8.8 or 1.1.1.1 as known-good hosts to verify your local network first. Let MTR run for at least 30–60 seconds before drawing conclusions.

=== Reading Between the Lines ===
'''Good network health:'''
HOST: hostname Loss% Snt Last Avg Best Wrst StDev
1.|-- 192.168.1.1 0.0% 50 1.1 1.2 0.9 2.1 0.2
2.|-- 10.0.0.1 0.0% 50 8.2 8.5 7.5 10.2 0.5
3.|-- isp-gateway.net 0.0% 50 15.1 15.5 14.0 18.3 0.8
4.|-- google.com 0.0% 50 24.8 25.2 23.5 28.1 1.1
No packet loss, consistent latency, low jitter throughout.

'''Problematic network:'''
HOST: hostname Loss% Snt Last Avg Best Wrst StDev
1.|-- 192.168.1.1 0.0% 50 1.2 1.5 1.0 2.3 0.3
2.|-- 10.0.0.1 5.0% 50 45.3 52.8 8.1 245.2 45.7
3.|-- ??? 100.0% 50 0.0 0.0 0.0 0.0 0.0
4.|-- destination.com 15.0% 50 95.4 125.8 48.2 385.5 78.2
Packet loss at hop 2 and the destination, high jitter, and a very high worst-case latency all point to a genuine problem at or near hop 2.

=== Summary ===
MTR is your friend when it comes to diagnostics. Use --report-cycles 50 or higher for reliable data. Watch for packet loss at the destination—intermediate losses may be false positives. High StDev indicates an unstable connection. High average latency points to slow links. Use --no-dns for faster results. Compare ICMP, UDP, and TCP modes if issues appear. Test at different times of day for a complete picture.
== Package Management Issues ==

=== Fix Broken Packages ===
$ sudo apt update # update package lists
$ sudo apt --fix-broken install # fix broken dependencies
$ sudo dpkg --configure -a # reconfigure packages (no output = nothing to do)
$ sudo apt clean # clean package cache
$ sudo apt autoclean
$ sudo apt autoremove # remove unused packages

=== Handle Held or Locked Packages ===

# Fix broken dependencies
sudo apt --fix-broken install

# Reconfigure packages
sudo dpkg --configure -a
# No output means there is nothing to reconfigure

# Clean package cache
sudo apt clean
sudo apt autoclean

# Remove unused packages
sudo apt autoremove

=== Handle Held or Locked Packages ===
# If apt is locked, find the process holding the lock
sudo lsof /var/lib/dpkg/lock-frontend

# Remove the lock (use with caution—only if no package manager is running)
sudo rm /var/lib/dpkg/lock-frontend
sudo rm /var/lib/apt/lists/lock

# Reconfigure dpkg
sudo dpkg --configure -a
Disk & Filesystem Issues

=== Check Disk Health ===
# Check disk space
df -h

# Check inode usage
df -i

# View disk I/O statistics
# iostat is part of the sysstat package, which also includes:
# mpstat (CPU statistics), sar (system activity reporter),
# pidstat (process statistics), and cifsiostat (CIFS statistics)
sudo apt install sysstat

iostat -x 1

# Show stats in MB instead of KB
iostat -xm 2

# Monitor a specific device
iostat -x sda 1

# Check for disk errors in dmesg
sudo dmesg | grep -i "error\|fail"

# SMART disk health (requires smartmontools)
sudo apt install smartmontools
sudo smartctl -a /dev/sda

=== Repair Filesystem ===
# Unmount the partition first
sudo umount /dev/sdXN

# Run filesystem check
sudo fsck /dev/sdXN

# For ext4 specifically
sudo e2fsck -f /dev/sdXN
Performance Issues

=== Identify Resource Hogs ===
# CPU usage by process
top -o %CPU

# Memory usage by process
top -o %MEM

# Disk usage by directory
du -sh /* | sort -h

# Find large files
find / -type f -size +100M 2>/dev/null

# Check running processes sorted by memory use
ps aux --sort=-%mem | head -20

=== System Temperature Monitoring ===
# Install lm-sensors if not already present
sudo apt install lm-sensors
sudo sensors-detect

# View temperatures
sensors

# Real-time temperature monitoring
watch -n 2 sensors
See section 12 for a shell alias that makes temperature monitoring more convenient.

Service & Application Errors

=== Debug Service Problems ===
# Check service status
sudo systemctl status service-name

# View service logs
sudo journalctl -u service-name

# Restart a service
sudo systemctl restart service-name

# Enable service at boot
sudo systemctl enable service-name

# View recent service failures
journalctl -p err -b

=== Application Crash Investigation ===
# Check for core dumps
ls -lh /var/crash/

# View available application logs
ls /var/log/

# Follow syslog for application errors in real-time
sudo tail -f /var/log/syslog
Permission & Access Issues

=== Fix Common Permission Problems ===
# Check file ownership
ls -l /path/to/file

# Change ownership (user:group)
sudo chown user:group /path/to/file

# Change permissions
sudo chmod 644 /path/to/file

# Recursively fix permissions on a directory
sudo chown -R user:group /path/to/directory

=== User & Authentication Issues ===
# Check user information
id username

# View user login history
last -a

# Check failed login attempts
sudo journalctl | grep "authentication failure"

# Reset user password
sudo passwd username
Hardware Issues

=== Identify Hardware ===
# List all hardware (install if needed: sudo apt install lshw)
sudo lshw -short

# PCI devices
lspci -v

# USB devices
lsusb -v

# CPU information
lscpu

# Memory information
sudo dmidecode --type memory

=== Check Hardware Errors ===
# Kernel ring buffer (hardware messages)
dmesg | less
# Press q to quit

# Search for specific hardware issues
dmesg | grep -i "error\|fail\|warn"

# Check for USB issues
dmesg | grep -i usb
Quick Diagnostic Commands

=== System Information at a Glance ===
# Uptime and load average
uptime

# Kernel version
uname -r

# Debian version
cat /etc/debian_version

# Full system summary (install if needed: sudo apt install inxi)
sudo inxi -Fxz

=== Emergency Toolkit ===
# Create a consolidated diagnostic report
sudo journalctl -b > ~/system-report.txt
dmesg >> ~/system-report.txt
systemctl --failed >> ~/system-report.txt
df -h >> ~/system-report.txt
free -h >> ~/system-report.txt

# Watch logs in real-time
sudo journalctl -f

# Monitor system resources continuously
watch -n 1 'free -h && df -h'
Useful Aliases & Shortcuts
Add these to your ~/.bashrc for quick access to common troubleshooting commands:
# Monitor system temperatures in real-time
alias temps="watch -n 2 'for i in /sys/class/hwmon/hwmon*/; do echo -n \"\$(cat \${i}name): \"; cat \${i}temp*_input 2>/dev/null | while read temp; do echo \"scale=1; \$temp/1000\" | bc; done | tr \"\n\" \" \"; echo \"°C\"; done'"

# Quick system status
alias sysstat='echo "=== CPU ===" && uptime && echo -e "\n=== Memory ===" && free -h && echo -e "\n=== Disk ===" && df -h && echo -e "\n=== Top Processes ===" && ps aux --sort=-%mem | head -10'

# View last boot logs
alias lastboot='journalctl -b -1'

# Check failed services
alias failedservices='systemctl --failed'

# Monitor logs in real-time
alias watchlog='sudo journalctl -f'

# Quick network status
alias netstatus='ip addr show && echo -e "\n=== Routes ===" && ip route show'
After adding these, reload your shell configuration:
source ~/.bashrc
Tips for Effective Troubleshooting

'''Check logs first.''' journalctl and dmesg are your best starting point for almost any problem.

'''Change one thing at a time.''' It’s tempting to make several changes at once, but doing so makes it impossible to know what actually fixed—or broke—the problem.

'''Document your changes.''' Keep notes on what you’ve tried and what the results were. You’ll thank yourself later.

'''Use exact error messages.''' Copy error messages verbatim and search for them alongside “Debian” and the version number—for example, “Debian 13” or “Debian 13.1” for point releases.

'''Check what changed recently.''' Did you install something, update packages, or change the kernel before the problem appeared? Start there.

'''Back up before making major changes.''' This goes without saying, but it’s easy to skip when you’re in a hurry.

'''Use verbose mode.''' Adding -v or -vv flags to commands often reveals what would otherwise be silent failures.

'''Use the community.''' The Debian forums, Reddit, Stack Exchange, and the Debian mailing lists are all active and well-indexed. Someone has usually seen your problem before.

''This work was contributed by <dfn>distro-nix</dfn> to the Debian User Forums on 2025-10-27.''

Debian First Aid Kit

2026-05-15T22:21:45Z

Distro-nix: forum profile link

'''Author:''' distro-nix '''Forum Profile:''' [https://forums.debian.net/memberlist.php?mode=viewprofile&u=239099 distro-nix] '''Debian Version:''' Debian 13 (Trixie) '''Kernel Versions Tested:''' 6.12.73+deb13-amd64 and 6.16.3+deb13-amd64 (x86_64) '''Created:''' 2025-10-27 '''Last Updated:''' 2026-05-15 '''ID:''' 544201 '''License:''' CC BY-SA 4.0

''This guide is preserved in its original form. Questions and feedback welcome via the author’s <dfn>Debian Forums profile</dfn>.''

== Abstract ==
The Debian First Aid Kit is a practical reference for diagnosing and resolving common system problems on Debian GNU/Linux. It covers commonly occuring categories of failure: system freezes and crashes, boot problems, network issues, package management, disk and filesystem errors, performance degradation, service and application failures, permission and access problems, hardware identification and errors, and quick diagnostics. Each section gives the relevant commands so you can understand what they do and why, rather than a bare list to copy and paste blindly. A dedicated section covers MTR (Matt’s Traceroute) in depth, including output interpretation, practical troubleshooting, and how to monitor continuously. The guide also includes a set of shell aliases for faster access to common diagnostic tasks. All commands are verified on Debian 13 (Trixie) running kernel 6.12.73+deb13-amd64 and 6.16.3+deb13-amd64. This document is intended for users with basic command-line familiarity who need a reliable, consolidated reference when something goes wrong.

'''Keywords:''' Debian, troubleshooting, system administration, MTR, network diagnostics, package management, journalctl, filesystem repair
----

== Tested Commands Disclaimer ==
All commands in this document have been verified on Debian 13 (Trixie) with the kernel versions listed above. Results may vary on other releases or hardware configurations. Always test in a safe environment before applying any procedure to a production system.

== Conventions Used in This Document ==
Command examples throughout this guide follow standard Unix prompt conventions. A $ prefix indicates a command run as a regular user. Where elevated privileges are required, sudo is shown explicitly and # with a brief explanation if needed:
$ journalctl -b -1 # regular user
$ sudo apt install mtr # requires elevation via sudo
In recovery shell contexts, where you are operating directly as root, the prompt is #.

== System Freezes & Crashes ==

=== Check System Logs ===

=== Common Freeze Causes to Look For ===

* '''Kernel panics:''' Search for “kernel panic” or “Oops”
* '''Out of Memory (OOM):''' Search for “Out of memory” or “oom-killer”
* '''Hardware errors:''' Look for “MCE” (Machine Check Exception) or “hardware error”
* '''Driver issues:''' Check for module/driver failures
* '''Overheating:''' Check system temperatures

=== Check System Resources ===
$ free -h # memory usage

$ df -h # disk space

$ htop # interactive resource monitor

$ top # alternative resource monitor

$ sudo apt install btop # btop is a more detailed alternative

$ sudo dmesg | grep -i error # check for disk errors

=== Access Recovery Mode ===
Reboot and hold Shift to access the GRUB menu (timing depends on your GRUB configuration). Select “Advanced options”, then choose recovery mode. From the recovery menu, select “root” for a root shell.

=== Common Boot Fixes ===
$ sudo fsck /dev/sdXN # repair filesystem — identify device with lsblk first

$ sudo grub-install /dev/sdX # reinstall GRUB bootloader

$ sudo update-grub # regenerate GRUB configuration

$ cat /etc/fstab # check fstab for mount errors

The following are permanent errors caused by incomplete or buggy ACPI tables in the BIOS. They are harmless and can be safely ignored if they appear on your system:
[0.686554] ACPI Error: No handler for Region [ECRM] (00000000201accc4) [EmbeddedControl] (20250404/evregion-131)
[0.686577] ACPI Error: Region EmbeddedControl (ID=3) has no handler (20250404/exfldio-261)
[0.686594] ACPI Error: Aborting method \_SB.GPIO._EVT due to previous error (AE_NOT_EXIST) (20250404/psparse-529)

== Boot Problems ==

=== Check Boot Process ===
$ systemd-analyze blame # boot time by unit

$ systemd-analyze blame # boot time by unit

$ systemctl --failed # units that failed during boot

$ systemctl status <service-name> # e.g. systemctl status NetworkManager.service

=== Access Recovery Mode ===
Reboot and hold Shift to access the GRUB menu (timing depends on your GRUB configuration). Select “Advanced options”, then choose recovery mode. From the recovery menu, select “root” for a root shell.

=== Common Boot Fixes ===
$ sudo fsck /dev/sdXN # repair filesystem — identify device with lsblk first

$ sudo grub-install /dev/sdX # reinstall GRUB bootloader

$ sudo update-grub # regenerate GRUB configuration

$ cat /etc/fstab # check fstab for mount errors

== Network Issues ==

=== Diagnose Network Connection ===
$ ip addr show # check network interfaces

$ ping -c 4 8.8.8.8 # test IPv4 connectivity

$ ping -c 6 2a00:1450:4007:809::200e # test IPv6 connectivity

$ nslookup google.com # check DNS resolution

$ ip route show # view routing table

$ ss -tuln # check active connections

=== Restart Network Service ===
$ sudo systemctl restart NetworkManager # for systems using NetworkManager

$ sudo systemctl restart networking # for systems using networking service

$ sudo ip link set eth0 down # bring interface down

$ sudo ip link set eth0 up # bring interface back up

== MTR (Matt’s Traceroute) ==
MTR combines the functionality of ping and traceroute into a single real-time network diagnostic tool. It continuously monitors the path between your system and a destination, providing detailed statistics about latency and packet loss at each hop. When you need to demonstrate a network problem that is outside your control—to your hosting provider, for example—MTR gives you the evidence you need.

=== Installation ===
$ sudo apt install mtr

=== Basic Usage ===
$ mtr google.com # interactive mode

$ mtr --report google.com # report mode (10 cycles then exit)

$ mtr --report-cycles 50 google.com # specify number of cycles

$ mtr --tcp google.com # use TCP instead of ICMP

$ mtr --udp google.com # use UDP

$ mtr --no-dns google.com # skip DNS resolution (faster)

$ mtr --show-ips google.com # show both hostnames and IPs

== Understanding MTR Output ==

==== Sample Output ====
HOST: hostname Loss% Snt Last Avg Best Wrst StDev
1.|-- 192.168.1.1 0.0% 10 1.2 1.5 1.0 2.3 0.4
2.|-- 10.0.0.1 0.0% 10 8.5 9.2 7.8 12.1 1.3
3.|-- isp-gateway.net 0.0% 10 15.3 16.1 14.2 19.8 1.8
4.|-- ??? 100.0% 10 0.0 0.0 0.0 0.0 0.0
5.|-- google.com 0.0% 10 25.4 26.8 24.1 32.5 2.4

=== Column Meanings ===
'''HOST''' — Hostname or IP address of each hop in the route.

'''Loss%''' — Percentage of packets lost at this hop.

'''Snt''' — Number of packets sent to this hop.

'''Last''' — Latency of the most recent packet, in milliseconds.

'''Avg''' — Average latency across all packets sent, in milliseconds.

'''Best''' — Lowest latency recorded, in milliseconds.

'''Wrst''' — Highest latency recorded, in milliseconds.

'''StDev''' — Standard deviation of latency measurements—lower values indicate a more stable connection.

=== Interpreting Results ===
A healthy network shows 0% loss on all hops, stable latency with low StDev values, a gradual latency increase as hop count increases, and consistent response times.

==== Problem Indicators ====
'''1. High Packet Loss at a Specific Hop'''
5.|-- problem-router.net 25.0% 10 45.3 48.2 42.1 65.8 8.4
If loss continues to the destination, there is a real problem at that router. If loss appears only at that hop but not beyond, the router is likely rate-limiting ICMP responses—a false positive, not a real problem. The rule of thumb: if packet loss appears at hop N but hops N+1, N+2, and beyond show 0% loss, it’s almost certainly ICMP rate limiting.

'''2. High Latency at a Specific Hop'''
3.|-- slow-link.net 0.0% 10 150.3 155.2 148.1 165.8 5.4
This indicates a network bottleneck, a congested link, geographical distance (intercontinental hops), or slow routing equipment.

'''3. No Response (???)'''
4.|-- ??? 100.0% 10 0.0 0.0 0.0 0.0 0.0
The router at this hop is configured not to respond to ICMP or traceroute packets, or a firewall is blocking diagnostic packets. This is not necessarily a problem if later hops respond normally.

'''4. High Jitter (StDev)'''
6.|-- unstable.net 0.0% 10 35.3 52.8 28.1 95.2 24.7
A high StDev (24.7ms in this example) indicates inconsistent latency caused by network congestion or instability. This is particularly noticeable in real-time applications such as VoIP, gaming, and video calls.

'''5. Sudden Latency Spike'''
1.|-- 192.168.1.1 0.0% 10 1.2 1.5 1.0 2.3 0.4
2.|-- 10.0.0.1 0.0% 10 8.5 9.2 7.8 12.1 1.3
3.|-- problematic-hop.net 0.0% 10 180.5 185.2 178.1 195.8 6.4
4.|-- next-hop.net 0.0% 10 182.3 187.8 180.5 198.2 6.8
Hop 3 introduces approximately 170ms of additional latency (jumping from 9ms to 180ms). Everything from hop 3 onward carries that penalty.

=== Advanced Usage ===

==== Report Mode with Different Output Formats ====
$ mtr --report --csv google.com > network-report.csv # CSV for logging

$ mtr --report --json google.com # JSON for parsing

$ mtr --report --xml google.com # XML format

$ mtr --report-wide google.com # wide report, no abbreviations

==== Protocol Selection ====
$ mtr google.com # ICMP (default)

$ mtr --udp google.com # UDP

$ mtr --tcp google.com # TCP

$ sudo mtr --tcp --port 443 google.com # specific TCP port

$ sudo mtr --tcp --port 22 remote-server.com # default TCP port

==== Timing and Duration ====
$ mtr --interval 0.5 google.com # interval between pings (default 1s)

$ mtr --report-cycles 100 google.com # extended test, 100 cycles

$ mtr google.com # continuous monitoring (Ctrl+C to stop)

$ mtr --report-cycles 10 google.com # quick 10-cycle report

==== Advanced Options ====
$ mtr --aslookup google.com # show Autonomous System (AS) numbers

$ mtr --max-ttl 20 google.com # set maximum number of hops

$ mtr --psize 1000 google.com # set packet size

$ mtr --show-ips google.com # show both IP and hostname

$ mtr --address 192.168.1.100 google.com # specify source address

$ mtr -4 google.com # IPv4 only

$ mtr -6 google.com # IPv6 only

=== Interactive Mode Commands ===
When running MTR in interactive mode (mtr hostname), the following keys are available:

'''h''' — Display help.

'''d''' — Toggle display mode, cycling through different views.

'''n''' — Toggle between hostnames and IP addresses.

'''r''' — Reset all statistics.

'''p''' — Pause/unpause the display.

'''q''' — Quit MTR.

'''u''' — Switch between ICMP, UDP, and TCP modes.

'''y''' — Switch between IPv4 and IPv6.

'''o''' — Toggle field display options.

'''j''' — Toggle latency display.

=== Practical Troubleshooting Scenarios ===

==== Scenario 1: Diagnosing a Slow Website ====
$ mtr --report-cycles 100 --no-dns example.com
Look for high average latency at specific hops, packet loss at the destination, and high StDev values indicating jitter.

==== Scenario 2: Testing if a Firewall Blocks SSH ====
$ sudo mtr --tcp --port 22 --report-cycles 50 remote-server.com
If the last hop shows 100% loss but earlier hops are fine, port 22 may be filtered. Run a standard ICMP test for comparison.

==== Scenario 3: ISP Performance Issues ====
$ mtr --report-cycles 100 8.8.8.8

$ mtr --report-cycles 100 1.1.1.1
If issues appear in the first three or four hops, the problem is likely within your ISP’s network. If issues appear later, the problem lies beyond your ISP.

==== Scenario 4: VPN Troubleshooting ====
$ mtr --report-cycles 50 --no-dns google.com > before-vpn.txt # before VPN

$ mtr --report-cycles 50 --no-dns google.com > after-vpn.txt # after VPN

$ diff before-vpn.txt after-vpn.txt # compare the two

==== Scenario 5: Gaming/Streaming Performance ====
$ mtr --report-cycles 200 game-server.com
For real-time applications, look for low average latency (under 50ms for gaming), low StDev (under 5ms preferred), and zero packet loss.

==== Scenario 6: Intermittent Connectivity ====
$ mtr --report-cycles 500 --interval 1 target.com > long-test.txt # long-running test

$ mtr target.com # real-time monitoring

=== Continuous Monitoring ===

==== Log Network Performance Over Time ====
while true; do
timestamp=$(date +%Y%m%d-%H%M%S)
mtr --report --report-cycles 50 google.com > "mtr-$timestamp.txt"
sleep 3600
done

==== Monitor Multiple Destinations ====
#!/bin/bash
echo "=== MTR Report $(date) ===" > daily-network-report.txt
echo "" >> daily-network-report.txt

echo "Google DNS:" >> daily-network-report.txt
mtr --report --report-cycles 50 --no-dns 8.8.8.8 >> daily-network-report.txt
echo "" >> daily-network-report.txt

echo "Cloudflare DNS:" >> daily-network-report.txt
mtr --report --report-cycles 50 --no-dns 1.1.1.1 >> daily-network-report.txt
echo "" >> daily-network-report.txt

echo "Your Server:" >> daily-network-report.txt
mtr --report --report-cycles 50 your-server.com >> daily-network-report.txt

=== Useful Aliases for .bashrc ===
Quick network path analysis
alias mtrreport='mtr --report --report-cycles 50 --no-dns'

Monitor connection to Google DNS
alias netcheck='mtr --report-cycles 20 8.8.8.8'

Extended network test
alias mtrlong='mtr --report-cycles 100'

TCP port 443 test (HTTPS)
alias mtrhttps='sudo mtr --tcp --port 443 --report-cycles 30'

Quick comparison of major DNS providers
alias dnstest='echo "Google:" && mtr --report-cycles 20 8.8.8.8 && echo -e "\nCloudflare:" && mtr --report-cycles 20 1.1.1.1'
After adding to ~/.bashrc, reload with:
source ~/.bashrc

=== Troubleshooting Tips ===
'''Permission issues with TCP mode:'''
Use sudo for TCP on privileged ports
$ sudo mtr --tcp --port 443 example.com

Or set capabilities (one-time setup)
$ sudo setcap cap_net_raw+ep /usr/bin/mtr-packet
'''False positives:''' Packet loss at an intermediate hop but not at the destination is almost always ICMP rate limiting at that router, not a real problem. Example:
3.|-- router.isp.net 20.0% 50 15.3 16.1 14.2 19.8 1.8
4.|-- next-hop.net 0.0% 50 18.5 19.2 17.8 22.1 1.3
5.|-- destination.com 0.0% 50 25.4 26.8 24.1 32.5 2.4
Hop 3 shows 20% loss, but hops 4 and 5 show 0%. Traffic is flowing normally.

'''DNS resolution delays:''' If MTR is slow to start, skip DNS resolution with mtr --no-dns target.com and resolve addresses afterward with host 203.0.113.1 if needed.

'''Comparing results:'''
$ mtr --report --report-cycles 50 example.com > test1.txt
$ sleep 60
$ mtr --report --report-cycles 50 example.com > test2.txt
$ diff test1.txt test2.txt

=== When to Use MTR vs Other Tools ===
'''MTR''' — Best for continuous monitoring, identifying problem hops, and gathering detailed statistics. Requires installation via apt.

'''ping''' — Best for a quick connectivity test or simple latency check. Only tests the endpoint, not the path.

'''traceroute''' — Best for one-time path discovery. Provides no continuous monitoring.

'''ss / netstat''' — Best for checking local connection status. Doesn’t test remote paths.

=== Best Practices ===
Run at least 50–100 cycles for accurate statistics. Use --no-dns for faster results and to avoid DNS resolution issues during testing. Test at different times of day, since network conditions vary. If one protocol shows issues, compare ICMP, UDP, and TCP modes. Save reports with timestamps for trend analysis. Use 8.8.8.8 or 1.1.1.1 as known-good hosts to verify your local network first. Let MTR run for at least 30–60 seconds before drawing conclusions.

=== Reading Between the Lines ===
'''Good network health:'''
HOST: hostname Loss% Snt Last Avg Best Wrst StDev
1.|-- 192.168.1.1 0.0% 50 1.1 1.2 0.9 2.1 0.2
2.|-- 10.0.0.1 0.0% 50 8.2 8.5 7.5 10.2 0.5
3.|-- isp-gateway.net 0.0% 50 15.1 15.5 14.0 18.3 0.8
4.|-- google.com 0.0% 50 24.8 25.2 23.5 28.1 1.1
No packet loss, consistent latency, low jitter throughout.

'''Problematic network:'''
HOST: hostname Loss% Snt Last Avg Best Wrst StDev
1.|-- 192.168.1.1 0.0% 50 1.2 1.5 1.0 2.3 0.3
2.|-- 10.0.0.1 5.0% 50 45.3 52.8 8.1 245.2 45.7
3.|-- ??? 100.0% 50 0.0 0.0 0.0 0.0 0.0
4.|-- destination.com 15.0% 50 95.4 125.8 48.2 385.5 78.2
Packet loss at hop 2 and the destination, high jitter, and a very high worst-case latency all point to a genuine problem at or near hop 2.

=== Summary ===
MTR is your friend when it comes to diagnostics. Use --report-cycles 50 or higher for reliable data. Watch for packet loss at the destination—intermediate losses may be false positives. High StDev indicates an unstable connection. High average latency points to slow links. Use --no-dns for faster results. Compare ICMP, UDP, and TCP modes if issues appear. Test at different times of day for a complete picture.
== Package Management Issues ==

=== Fix Broken Packages ===
$ sudo apt update # update package lists
$ sudo apt --fix-broken install # fix broken dependencies
$ sudo dpkg --configure -a # reconfigure packages (no output = nothing to do)
$ sudo apt clean # clean package cache
$ sudo apt autoclean
$ sudo apt autoremove # remove unused packages

=== Handle Held or Locked Packages ===

# Fix broken dependencies
sudo apt --fix-broken install

# Reconfigure packages
sudo dpkg --configure -a
# No output means there is nothing to reconfigure

# Clean package cache
sudo apt clean
sudo apt autoclean

# Remove unused packages
sudo apt autoremove

=== Handle Held or Locked Packages ===
# If apt is locked, find the process holding the lock
sudo lsof /var/lib/dpkg/lock-frontend

# Remove the lock (use with caution—only if no package manager is running)
sudo rm /var/lib/dpkg/lock-frontend
sudo rm /var/lib/apt/lists/lock

# Reconfigure dpkg
sudo dpkg --configure -a
Disk & Filesystem Issues

=== Check Disk Health ===
# Check disk space
df -h

# Check inode usage
df -i

# View disk I/O statistics
# iostat is part of the sysstat package, which also includes:
# mpstat (CPU statistics), sar (system activity reporter),
# pidstat (process statistics), and cifsiostat (CIFS statistics)
sudo apt install sysstat

iostat -x 1

# Show stats in MB instead of KB
iostat -xm 2

# Monitor a specific device
iostat -x sda 1

# Check for disk errors in dmesg
sudo dmesg | grep -i "error\|fail"

# SMART disk health (requires smartmontools)
sudo apt install smartmontools
sudo smartctl -a /dev/sda

=== Repair Filesystem ===
# Unmount the partition first
sudo umount /dev/sdXN

# Run filesystem check
sudo fsck /dev/sdXN

# For ext4 specifically
sudo e2fsck -f /dev/sdXN
Performance Issues

=== Identify Resource Hogs ===
# CPU usage by process
top -o %CPU

# Memory usage by process
top -o %MEM

# Disk usage by directory
du -sh /* | sort -h

# Find large files
find / -type f -size +100M 2>/dev/null

# Check running processes sorted by memory use
ps aux --sort=-%mem | head -20

=== System Temperature Monitoring ===
# Install lm-sensors if not already present
sudo apt install lm-sensors
sudo sensors-detect

# View temperatures
sensors

# Real-time temperature monitoring
watch -n 2 sensors
See section 12 for a shell alias that makes temperature monitoring more convenient.

Service & Application Errors

=== Debug Service Problems ===
# Check service status
sudo systemctl status service-name

# View service logs
sudo journalctl -u service-name

# Restart a service
sudo systemctl restart service-name

# Enable service at boot
sudo systemctl enable service-name

# View recent service failures
journalctl -p err -b

=== Application Crash Investigation ===
# Check for core dumps
ls -lh /var/crash/

# View available application logs
ls /var/log/

# Follow syslog for application errors in real-time
sudo tail -f /var/log/syslog
Permission & Access Issues

=== Fix Common Permission Problems ===
# Check file ownership
ls -l /path/to/file

# Change ownership (user:group)
sudo chown user:group /path/to/file

# Change permissions
sudo chmod 644 /path/to/file

# Recursively fix permissions on a directory
sudo chown -R user:group /path/to/directory

=== User & Authentication Issues ===
# Check user information
id username

# View user login history
last -a

# Check failed login attempts
sudo journalctl | grep "authentication failure"

# Reset user password
sudo passwd username
Hardware Issues

=== Identify Hardware ===
# List all hardware (install if needed: sudo apt install lshw)
sudo lshw -short

# PCI devices
lspci -v

# USB devices
lsusb -v

# CPU information
lscpu

# Memory information
sudo dmidecode --type memory

=== Check Hardware Errors ===
# Kernel ring buffer (hardware messages)
dmesg | less
# Press q to quit

# Search for specific hardware issues
dmesg | grep -i "error\|fail\|warn"

# Check for USB issues
dmesg | grep -i usb
Quick Diagnostic Commands

=== System Information at a Glance ===
# Uptime and load average
uptime

# Kernel version
uname -r

# Debian version
cat /etc/debian_version

# Full system summary (install if needed: sudo apt install inxi)
sudo inxi -Fxz

=== Emergency Toolkit ===
# Create a consolidated diagnostic report
sudo journalctl -b > ~/system-report.txt
dmesg >> ~/system-report.txt
systemctl --failed >> ~/system-report.txt
df -h >> ~/system-report.txt
free -h >> ~/system-report.txt

# Watch logs in real-time
sudo journalctl -f

# Monitor system resources continuously
watch -n 1 'free -h && df -h'
Useful Aliases & Shortcuts
Add these to your ~/.bashrc for quick access to common troubleshooting commands:
# Monitor system temperatures in real-time
alias temps="watch -n 2 'for i in /sys/class/hwmon/hwmon*/; do echo -n \"\$(cat \${i}name): \"; cat \${i}temp*_input 2>/dev/null | while read temp; do echo \"scale=1; \$temp/1000\" | bc; done | tr \"\n\" \" \"; echo \"°C\"; done'"

# Quick system status
alias sysstat='echo "=== CPU ===" && uptime && echo -e "\n=== Memory ===" && free -h && echo -e "\n=== Disk ===" && df -h && echo -e "\n=== Top Processes ===" && ps aux --sort=-%mem | head -10'

# View last boot logs
alias lastboot='journalctl -b -1'

# Check failed services
alias failedservices='systemctl --failed'

# Monitor logs in real-time
alias watchlog='sudo journalctl -f'

# Quick network status
alias netstatus='ip addr show && echo -e "\n=== Routes ===" && ip route show'
After adding these, reload your shell configuration:
source ~/.bashrc
Tips for Effective Troubleshooting

'''Check logs first.''' journalctl and dmesg are your best starting point for almost any problem.

'''Change one thing at a time.''' It’s tempting to make several changes at once, but doing so makes it impossible to know what actually fixed—or broke—the problem.

'''Document your changes.''' Keep notes on what you’ve tried and what the results were. You’ll thank yourself later.

'''Use exact error messages.''' Copy error messages verbatim and search for them alongside “Debian” and the version number—for example, “Debian 13” or “Debian 13.1” for point releases.

'''Check what changed recently.''' Did you install something, update packages, or change the kernel before the problem appeared? Start there.

'''Back up before making major changes.''' This goes without saying, but it’s easy to skip when you’re in a hurry.

'''Use verbose mode.''' Adding -v or -vv flags to commands often reveals what would otherwise be silent failures.

'''Use the community.''' The Debian forums, Reddit, Stack Exchange, and the Debian mailing lists are all active and well-indexed. Someone has usually seen your problem before.

''This work was contributed by <dfn>distro-nix</dfn> to the Debian User Forums on 2025-10-27.''

Debian First Aid Kit

2026-05-15T22:19:22Z

Distro-nix: Forum profile

'''Author:''' distro-nix '''Forum Profile:''' <dfn>[[Distro-nix|https://forums.debian.net/memberlist.php?mode=viewprofile&u=239099]]</dfn> '''Debian Version:''' Debian 13 (Trixie) '''Kernel Versions Tested:''' 6.12.73+deb13-amd64 and 6.16.3+deb13-amd64 (x86_64) '''Created:''' 2025-10-27 '''Last Updated:''' 2026-05-15 '''ID:''' 544201 '''License:''' CC BY-SA 4.0

''This guide is preserved in its original form. Questions and feedback welcome via the author’s <dfn>Debian Forums profile</dfn>.''

== Abstract ==
The Debian First Aid Kit is a practical reference for diagnosing and resolving common system problems on Debian GNU/Linux. It covers commonly occuring categories of failure: system freezes and crashes, boot problems, network issues, package management, disk and filesystem errors, performance degradation, service and application failures, permission and access problems, hardware identification and errors, and quick diagnostics. Each section gives the relevant commands so you can understand what they do and why, rather than a bare list to copy and paste blindly. A dedicated section covers MTR (Matt’s Traceroute) in depth, including output interpretation, practical troubleshooting, and how to monitor continuously. The guide also includes a set of shell aliases for faster access to common diagnostic tasks. All commands are verified on Debian 13 (Trixie) running kernel 6.12.73+deb13-amd64 and 6.16.3+deb13-amd64. This document is intended for users with basic command-line familiarity who need a reliable, consolidated reference when something goes wrong.

'''Keywords:''' Debian, troubleshooting, system administration, MTR, network diagnostics, package management, journalctl, filesystem repair
----

== Tested Commands Disclaimer ==
All commands in this document have been verified on Debian 13 (Trixie) with the kernel versions listed above. Results may vary on other releases or hardware configurations. Always test in a safe environment before applying any procedure to a production system.

== Conventions Used in This Document ==
Command examples throughout this guide follow standard Unix prompt conventions. A $ prefix indicates a command run as a regular user. Where elevated privileges are required, sudo is shown explicitly and # with a brief explanation if needed:
$ journalctl -b -1 # regular user
$ sudo apt install mtr # requires elevation via sudo
In recovery shell contexts, where you are operating directly as root, the prompt is #.

== System Freezes & Crashes ==

=== Check System Logs ===

=== Common Freeze Causes to Look For ===

* '''Kernel panics:''' Search for “kernel panic” or “Oops”
* '''Out of Memory (OOM):''' Search for “Out of memory” or “oom-killer”
* '''Hardware errors:''' Look for “MCE” (Machine Check Exception) or “hardware error”
* '''Driver issues:''' Check for module/driver failures
* '''Overheating:''' Check system temperatures

=== Check System Resources ===
$ free -h # memory usage

$ df -h # disk space

$ htop # interactive resource monitor

$ top # alternative resource monitor

$ sudo apt install btop # btop is a more detailed alternative

$ sudo dmesg | grep -i error # check for disk errors

=== Access Recovery Mode ===
Reboot and hold Shift to access the GRUB menu (timing depends on your GRUB configuration). Select “Advanced options”, then choose recovery mode. From the recovery menu, select “root” for a root shell.

=== Common Boot Fixes ===
$ sudo fsck /dev/sdXN # repair filesystem — identify device with lsblk first

$ sudo grub-install /dev/sdX # reinstall GRUB bootloader

$ sudo update-grub # regenerate GRUB configuration

$ cat /etc/fstab # check fstab for mount errors

The following are permanent errors caused by incomplete or buggy ACPI tables in the BIOS. They are harmless and can be safely ignored if they appear on your system:
[0.686554] ACPI Error: No handler for Region [ECRM] (00000000201accc4) [EmbeddedControl] (20250404/evregion-131)
[0.686577] ACPI Error: Region EmbeddedControl (ID=3) has no handler (20250404/exfldio-261)
[0.686594] ACPI Error: Aborting method \_SB.GPIO._EVT due to previous error (AE_NOT_EXIST) (20250404/psparse-529)

== Boot Problems ==

=== Check Boot Process ===
$ systemd-analyze blame # boot time by unit

$ systemd-analyze blame # boot time by unit

$ systemctl --failed # units that failed during boot

$ systemctl status <service-name> # e.g. systemctl status NetworkManager.service

=== Access Recovery Mode ===
Reboot and hold Shift to access the GRUB menu (timing depends on your GRUB configuration). Select “Advanced options”, then choose recovery mode. From the recovery menu, select “root” for a root shell.

=== Common Boot Fixes ===
$ sudo fsck /dev/sdXN # repair filesystem — identify device with lsblk first

$ sudo grub-install /dev/sdX # reinstall GRUB bootloader

$ sudo update-grub # regenerate GRUB configuration

$ cat /etc/fstab # check fstab for mount errors

== Network Issues ==

=== Diagnose Network Connection ===
$ ip addr show # check network interfaces

$ ping -c 4 8.8.8.8 # test IPv4 connectivity

$ ping -c 6 2a00:1450:4007:809::200e # test IPv6 connectivity

$ nslookup google.com # check DNS resolution

$ ip route show # view routing table

$ ss -tuln # check active connections

=== Restart Network Service ===
$ sudo systemctl restart NetworkManager # for systems using NetworkManager

$ sudo systemctl restart networking # for systems using networking service

$ sudo ip link set eth0 down # bring interface down

$ sudo ip link set eth0 up # bring interface back up

== MTR (Matt’s Traceroute) ==
MTR combines the functionality of ping and traceroute into a single real-time network diagnostic tool. It continuously monitors the path between your system and a destination, providing detailed statistics about latency and packet loss at each hop. When you need to demonstrate a network problem that is outside your control—to your hosting provider, for example—MTR gives you the evidence you need.

=== Installation ===
$ sudo apt install mtr

=== Basic Usage ===
$ mtr google.com # interactive mode

$ mtr --report google.com # report mode (10 cycles then exit)

$ mtr --report-cycles 50 google.com # specify number of cycles

$ mtr --tcp google.com # use TCP instead of ICMP

$ mtr --udp google.com # use UDP

$ mtr --no-dns google.com # skip DNS resolution (faster)

$ mtr --show-ips google.com # show both hostnames and IPs

== Understanding MTR Output ==

==== Sample Output ====
HOST: hostname Loss% Snt Last Avg Best Wrst StDev
1.|-- 192.168.1.1 0.0% 10 1.2 1.5 1.0 2.3 0.4
2.|-- 10.0.0.1 0.0% 10 8.5 9.2 7.8 12.1 1.3
3.|-- isp-gateway.net 0.0% 10 15.3 16.1 14.2 19.8 1.8
4.|-- ??? 100.0% 10 0.0 0.0 0.0 0.0 0.0
5.|-- google.com 0.0% 10 25.4 26.8 24.1 32.5 2.4

=== Column Meanings ===
'''HOST''' — Hostname or IP address of each hop in the route.

'''Loss%''' — Percentage of packets lost at this hop.

'''Snt''' — Number of packets sent to this hop.

'''Last''' — Latency of the most recent packet, in milliseconds.

'''Avg''' — Average latency across all packets sent, in milliseconds.

'''Best''' — Lowest latency recorded, in milliseconds.

'''Wrst''' — Highest latency recorded, in milliseconds.

'''StDev''' — Standard deviation of latency measurements—lower values indicate a more stable connection.

=== Interpreting Results ===
A healthy network shows 0% loss on all hops, stable latency with low StDev values, a gradual latency increase as hop count increases, and consistent response times.

==== Problem Indicators ====
'''1. High Packet Loss at a Specific Hop'''
5.|-- problem-router.net 25.0% 10 45.3 48.2 42.1 65.8 8.4
If loss continues to the destination, there is a real problem at that router. If loss appears only at that hop but not beyond, the router is likely rate-limiting ICMP responses—a false positive, not a real problem. The rule of thumb: if packet loss appears at hop N but hops N+1, N+2, and beyond show 0% loss, it’s almost certainly ICMP rate limiting.

'''2. High Latency at a Specific Hop'''
3.|-- slow-link.net 0.0% 10 150.3 155.2 148.1 165.8 5.4
This indicates a network bottleneck, a congested link, geographical distance (intercontinental hops), or slow routing equipment.

'''3. No Response (???)'''
4.|-- ??? 100.0% 10 0.0 0.0 0.0 0.0 0.0
The router at this hop is configured not to respond to ICMP or traceroute packets, or a firewall is blocking diagnostic packets. This is not necessarily a problem if later hops respond normally.

'''4. High Jitter (StDev)'''
6.|-- unstable.net 0.0% 10 35.3 52.8 28.1 95.2 24.7
A high StDev (24.7ms in this example) indicates inconsistent latency caused by network congestion or instability. This is particularly noticeable in real-time applications such as VoIP, gaming, and video calls.

'''5. Sudden Latency Spike'''
1.|-- 192.168.1.1 0.0% 10 1.2 1.5 1.0 2.3 0.4
2.|-- 10.0.0.1 0.0% 10 8.5 9.2 7.8 12.1 1.3
3.|-- problematic-hop.net 0.0% 10 180.5 185.2 178.1 195.8 6.4
4.|-- next-hop.net 0.0% 10 182.3 187.8 180.5 198.2 6.8
Hop 3 introduces approximately 170ms of additional latency (jumping from 9ms to 180ms). Everything from hop 3 onward carries that penalty.

=== Advanced Usage ===

==== Report Mode with Different Output Formats ====
$ mtr --report --csv google.com > network-report.csv # CSV for logging

$ mtr --report --json google.com # JSON for parsing

$ mtr --report --xml google.com # XML format

$ mtr --report-wide google.com # wide report, no abbreviations

==== Protocol Selection ====
$ mtr google.com # ICMP (default)

$ mtr --udp google.com # UDP

$ mtr --tcp google.com # TCP

$ sudo mtr --tcp --port 443 google.com # specific TCP port

$ sudo mtr --tcp --port 22 remote-server.com # default TCP port

==== Timing and Duration ====
$ mtr --interval 0.5 google.com # interval between pings (default 1s)

$ mtr --report-cycles 100 google.com # extended test, 100 cycles

$ mtr google.com # continuous monitoring (Ctrl+C to stop)

$ mtr --report-cycles 10 google.com # quick 10-cycle report

==== Advanced Options ====
$ mtr --aslookup google.com # show Autonomous System (AS) numbers

$ mtr --max-ttl 20 google.com # set maximum number of hops

$ mtr --psize 1000 google.com # set packet size

$ mtr --show-ips google.com # show both IP and hostname

$ mtr --address 192.168.1.100 google.com # specify source address

$ mtr -4 google.com # IPv4 only

$ mtr -6 google.com # IPv6 only

=== Interactive Mode Commands ===
When running MTR in interactive mode (mtr hostname), the following keys are available:

'''h''' — Display help.

'''d''' — Toggle display mode, cycling through different views.

'''n''' — Toggle between hostnames and IP addresses.

'''r''' — Reset all statistics.

'''p''' — Pause/unpause the display.

'''q''' — Quit MTR.

'''u''' — Switch between ICMP, UDP, and TCP modes.

'''y''' — Switch between IPv4 and IPv6.

'''o''' — Toggle field display options.

'''j''' — Toggle latency display.

=== Practical Troubleshooting Scenarios ===

==== Scenario 1: Diagnosing a Slow Website ====
$ mtr --report-cycles 100 --no-dns example.com
Look for high average latency at specific hops, packet loss at the destination, and high StDev values indicating jitter.

==== Scenario 2: Testing if a Firewall Blocks SSH ====
$ sudo mtr --tcp --port 22 --report-cycles 50 remote-server.com
If the last hop shows 100% loss but earlier hops are fine, port 22 may be filtered. Run a standard ICMP test for comparison.

==== Scenario 3: ISP Performance Issues ====
$ mtr --report-cycles 100 8.8.8.8

$ mtr --report-cycles 100 1.1.1.1
If issues appear in the first three or four hops, the problem is likely within your ISP’s network. If issues appear later, the problem lies beyond your ISP.

==== Scenario 4: VPN Troubleshooting ====
$ mtr --report-cycles 50 --no-dns google.com > before-vpn.txt # before VPN

$ mtr --report-cycles 50 --no-dns google.com > after-vpn.txt # after VPN

$ diff before-vpn.txt after-vpn.txt # compare the two

==== Scenario 5: Gaming/Streaming Performance ====
$ mtr --report-cycles 200 game-server.com
For real-time applications, look for low average latency (under 50ms for gaming), low StDev (under 5ms preferred), and zero packet loss.

==== Scenario 6: Intermittent Connectivity ====
$ mtr --report-cycles 500 --interval 1 target.com > long-test.txt # long-running test

$ mtr target.com # real-time monitoring

=== Continuous Monitoring ===

==== Log Network Performance Over Time ====
while true; do
timestamp=$(date +%Y%m%d-%H%M%S)
mtr --report --report-cycles 50 google.com > "mtr-$timestamp.txt"
sleep 3600
done

==== Monitor Multiple Destinations ====
#!/bin/bash
echo "=== MTR Report $(date) ===" > daily-network-report.txt
echo "" >> daily-network-report.txt

echo "Google DNS:" >> daily-network-report.txt
mtr --report --report-cycles 50 --no-dns 8.8.8.8 >> daily-network-report.txt
echo "" >> daily-network-report.txt

echo "Cloudflare DNS:" >> daily-network-report.txt
mtr --report --report-cycles 50 --no-dns 1.1.1.1 >> daily-network-report.txt
echo "" >> daily-network-report.txt

echo "Your Server:" >> daily-network-report.txt
mtr --report --report-cycles 50 your-server.com >> daily-network-report.txt

=== Useful Aliases for .bashrc ===
Quick network path analysis
alias mtrreport='mtr --report --report-cycles 50 --no-dns'

Monitor connection to Google DNS
alias netcheck='mtr --report-cycles 20 8.8.8.8'

Extended network test
alias mtrlong='mtr --report-cycles 100'

TCP port 443 test (HTTPS)
alias mtrhttps='sudo mtr --tcp --port 443 --report-cycles 30'

Quick comparison of major DNS providers
alias dnstest='echo "Google:" && mtr --report-cycles 20 8.8.8.8 && echo -e "\nCloudflare:" && mtr --report-cycles 20 1.1.1.1'
After adding to ~/.bashrc, reload with:
source ~/.bashrc

=== Troubleshooting Tips ===
'''Permission issues with TCP mode:'''
Use sudo for TCP on privileged ports
$ sudo mtr --tcp --port 443 example.com

Or set capabilities (one-time setup)
$ sudo setcap cap_net_raw+ep /usr/bin/mtr-packet
'''False positives:''' Packet loss at an intermediate hop but not at the destination is almost always ICMP rate limiting at that router, not a real problem. Example:
3.|-- router.isp.net 20.0% 50 15.3 16.1 14.2 19.8 1.8
4.|-- next-hop.net 0.0% 50 18.5 19.2 17.8 22.1 1.3
5.|-- destination.com 0.0% 50 25.4 26.8 24.1 32.5 2.4
Hop 3 shows 20% loss, but hops 4 and 5 show 0%. Traffic is flowing normally.

'''DNS resolution delays:''' If MTR is slow to start, skip DNS resolution with mtr --no-dns target.com and resolve addresses afterward with host 203.0.113.1 if needed.

'''Comparing results:'''
$ mtr --report --report-cycles 50 example.com > test1.txt
$ sleep 60
$ mtr --report --report-cycles 50 example.com > test2.txt
$ diff test1.txt test2.txt

=== When to Use MTR vs Other Tools ===
'''MTR''' — Best for continuous monitoring, identifying problem hops, and gathering detailed statistics. Requires installation via apt.

'''ping''' — Best for a quick connectivity test or simple latency check. Only tests the endpoint, not the path.

'''traceroute''' — Best for one-time path discovery. Provides no continuous monitoring.

'''ss / netstat''' — Best for checking local connection status. Doesn’t test remote paths.

=== Best Practices ===
Run at least 50–100 cycles for accurate statistics. Use --no-dns for faster results and to avoid DNS resolution issues during testing. Test at different times of day, since network conditions vary. If one protocol shows issues, compare ICMP, UDP, and TCP modes. Save reports with timestamps for trend analysis. Use 8.8.8.8 or 1.1.1.1 as known-good hosts to verify your local network first. Let MTR run for at least 30–60 seconds before drawing conclusions.

=== Reading Between the Lines ===
'''Good network health:'''
HOST: hostname Loss% Snt Last Avg Best Wrst StDev
1.|-- 192.168.1.1 0.0% 50 1.1 1.2 0.9 2.1 0.2
2.|-- 10.0.0.1 0.0% 50 8.2 8.5 7.5 10.2 0.5
3.|-- isp-gateway.net 0.0% 50 15.1 15.5 14.0 18.3 0.8
4.|-- google.com 0.0% 50 24.8 25.2 23.5 28.1 1.1
No packet loss, consistent latency, low jitter throughout.

'''Problematic network:'''
HOST: hostname Loss% Snt Last Avg Best Wrst StDev
1.|-- 192.168.1.1 0.0% 50 1.2 1.5 1.0 2.3 0.3
2.|-- 10.0.0.1 5.0% 50 45.3 52.8 8.1 245.2 45.7
3.|-- ??? 100.0% 50 0.0 0.0 0.0 0.0 0.0
4.|-- destination.com 15.0% 50 95.4 125.8 48.2 385.5 78.2
Packet loss at hop 2 and the destination, high jitter, and a very high worst-case latency all point to a genuine problem at or near hop 2.

=== Summary ===
MTR is your friend when it comes to diagnostics. Use --report-cycles 50 or higher for reliable data. Watch for packet loss at the destination—intermediate losses may be false positives. High StDev indicates an unstable connection. High average latency points to slow links. Use --no-dns for faster results. Compare ICMP, UDP, and TCP modes if issues appear. Test at different times of day for a complete picture.
== Package Management Issues ==

=== Fix Broken Packages ===
$ sudo apt update # update package lists
$ sudo apt --fix-broken install # fix broken dependencies
$ sudo dpkg --configure -a # reconfigure packages (no output = nothing to do)
$ sudo apt clean # clean package cache
$ sudo apt autoclean
$ sudo apt autoremove # remove unused packages

=== Handle Held or Locked Packages ===

# Fix broken dependencies
sudo apt --fix-broken install

# Reconfigure packages
sudo dpkg --configure -a
# No output means there is nothing to reconfigure

# Clean package cache
sudo apt clean
sudo apt autoclean

# Remove unused packages
sudo apt autoremove

=== Handle Held or Locked Packages ===
# If apt is locked, find the process holding the lock
sudo lsof /var/lib/dpkg/lock-frontend

# Remove the lock (use with caution—only if no package manager is running)
sudo rm /var/lib/dpkg/lock-frontend
sudo rm /var/lib/apt/lists/lock

# Reconfigure dpkg
sudo dpkg --configure -a
Disk & Filesystem Issues

=== Check Disk Health ===
# Check disk space
df -h

# Check inode usage
df -i

# View disk I/O statistics
# iostat is part of the sysstat package, which also includes:
# mpstat (CPU statistics), sar (system activity reporter),
# pidstat (process statistics), and cifsiostat (CIFS statistics)
sudo apt install sysstat

iostat -x 1

# Show stats in MB instead of KB
iostat -xm 2

# Monitor a specific device
iostat -x sda 1

# Check for disk errors in dmesg
sudo dmesg | grep -i "error\|fail"

# SMART disk health (requires smartmontools)
sudo apt install smartmontools
sudo smartctl -a /dev/sda

=== Repair Filesystem ===
# Unmount the partition first
sudo umount /dev/sdXN

# Run filesystem check
sudo fsck /dev/sdXN

# For ext4 specifically
sudo e2fsck -f /dev/sdXN
Performance Issues

=== Identify Resource Hogs ===
# CPU usage by process
top -o %CPU

# Memory usage by process
top -o %MEM

# Disk usage by directory
du -sh /* | sort -h

# Find large files
find / -type f -size +100M 2>/dev/null

# Check running processes sorted by memory use
ps aux --sort=-%mem | head -20

=== System Temperature Monitoring ===
# Install lm-sensors if not already present
sudo apt install lm-sensors
sudo sensors-detect

# View temperatures
sensors

# Real-time temperature monitoring
watch -n 2 sensors
See section 12 for a shell alias that makes temperature monitoring more convenient.

Service & Application Errors

=== Debug Service Problems ===
# Check service status
sudo systemctl status service-name

# View service logs
sudo journalctl -u service-name

# Restart a service
sudo systemctl restart service-name

# Enable service at boot
sudo systemctl enable service-name

# View recent service failures
journalctl -p err -b

=== Application Crash Investigation ===
# Check for core dumps
ls -lh /var/crash/

# View available application logs
ls /var/log/

# Follow syslog for application errors in real-time
sudo tail -f /var/log/syslog
Permission & Access Issues

=== Fix Common Permission Problems ===
# Check file ownership
ls -l /path/to/file

# Change ownership (user:group)
sudo chown user:group /path/to/file

# Change permissions
sudo chmod 644 /path/to/file

# Recursively fix permissions on a directory
sudo chown -R user:group /path/to/directory

=== User & Authentication Issues ===
# Check user information
id username

# View user login history
last -a

# Check failed login attempts
sudo journalctl | grep "authentication failure"

# Reset user password
sudo passwd username
Hardware Issues

=== Identify Hardware ===
# List all hardware (install if needed: sudo apt install lshw)
sudo lshw -short

# PCI devices
lspci -v

# USB devices
lsusb -v

# CPU information
lscpu

# Memory information
sudo dmidecode --type memory

=== Check Hardware Errors ===
# Kernel ring buffer (hardware messages)
dmesg | less
# Press q to quit

# Search for specific hardware issues
dmesg | grep -i "error\|fail\|warn"

# Check for USB issues
dmesg | grep -i usb
Quick Diagnostic Commands

=== System Information at a Glance ===
# Uptime and load average
uptime

# Kernel version
uname -r

# Debian version
cat /etc/debian_version

# Full system summary (install if needed: sudo apt install inxi)
sudo inxi -Fxz

=== Emergency Toolkit ===
# Create a consolidated diagnostic report
sudo journalctl -b > ~/system-report.txt
dmesg >> ~/system-report.txt
systemctl --failed >> ~/system-report.txt
df -h >> ~/system-report.txt
free -h >> ~/system-report.txt

# Watch logs in real-time
sudo journalctl -f

# Monitor system resources continuously
watch -n 1 'free -h && df -h'
Useful Aliases & Shortcuts
Add these to your ~/.bashrc for quick access to common troubleshooting commands:
# Monitor system temperatures in real-time
alias temps="watch -n 2 'for i in /sys/class/hwmon/hwmon*/; do echo -n \"\$(cat \${i}name): \"; cat \${i}temp*_input 2>/dev/null | while read temp; do echo \"scale=1; \$temp/1000\" | bc; done | tr \"\n\" \" \"; echo \"°C\"; done'"

# Quick system status
alias sysstat='echo "=== CPU ===" && uptime && echo -e "\n=== Memory ===" && free -h && echo -e "\n=== Disk ===" && df -h && echo -e "\n=== Top Processes ===" && ps aux --sort=-%mem | head -10'

# View last boot logs
alias lastboot='journalctl -b -1'

# Check failed services
alias failedservices='systemctl --failed'

# Monitor logs in real-time
alias watchlog='sudo journalctl -f'

# Quick network status
alias netstatus='ip addr show && echo -e "\n=== Routes ===" && ip route show'
After adding these, reload your shell configuration:
source ~/.bashrc
Tips for Effective Troubleshooting

'''Check logs first.''' journalctl and dmesg are your best starting point for almost any problem.

'''Change one thing at a time.''' It’s tempting to make several changes at once, but doing so makes it impossible to know what actually fixed—or broke—the problem.

'''Document your changes.''' Keep notes on what you’ve tried and what the results were. You’ll thank yourself later.

'''Use exact error messages.''' Copy error messages verbatim and search for them alongside “Debian” and the version number—for example, “Debian 13” or “Debian 13.1” for point releases.

'''Check what changed recently.''' Did you install something, update packages, or change the kernel before the problem appeared? Start there.

'''Back up before making major changes.''' This goes without saying, but it’s easy to skip when you’re in a hurry.

'''Use verbose mode.''' Adding -v or -vv flags to commands often reveals what would otherwise be silent failures.

'''Use the community.''' The Debian forums, Reddit, Stack Exchange, and the Debian mailing lists are all active and well-indexed. Someone has usually seen your problem before.

''This work was contributed by <dfn>distro-nix</dfn> to the Debian User Forums on 2025-10-27.''

Debian First Aid Kit

2026-05-15T22:14:45Z

Distro-nix: Pasting more sections

'''Author:''' distro-nix '''Forum Profile:''' <dfn><nowiki>https://forums.debian.net/memberlist.php?mode=viewprofile&u=239099</nowiki></dfn> '''Debian Version:''' Debian 13 (Trixie) '''Kernel Versions Tested:''' 6.12.73+deb13-amd64 and 6.16.3+deb13-amd64 (x86_64) '''Created:''' 2025-10-27 '''Last Updated:''' 2026-03-11 '''ID:''' 544201 '''License:''' CC BY-SA 4.0

''This guide is preserved in its original form. Questions and feedback welcome via the author’s <dfn>Debian Forums profile</dfn>.''

== Abstract ==
The Debian First Aid Kit is a practical reference for diagnosing and resolving common system problems on Debian GNU/Linux. It covers commonly occuring categories of failure: system freezes and crashes, boot problems, network issues, package management, disk and filesystem errors, performance degradation, service and application failures, permission and access problems, hardware identification and errors, and quick diagnostics. Each section gives the relevant commands so you can understand what they do and why, rather than a bare list to copy and paste blindly. A dedicated section covers MTR (Matt’s Traceroute) in depth, including output interpretation, practical troubleshooting, and how to monitor continuously. The guide also includes a set of shell aliases for faster access to common diagnostic tasks. All commands are verified on Debian 13 (Trixie) running kernel 6.12.73+deb13-amd64 and 6.16.3+deb13-amd64. This document is intended for users with basic command-line familiarity who need a reliable, consolidated reference when something goes wrong.

'''Keywords:''' Debian, troubleshooting, system administration, MTR, network diagnostics, package management, journalctl, filesystem repair
----

== Tested Commands Disclaimer ==
All commands in this document have been verified on Debian 13 (Trixie) with the kernel versions listed above. Results may vary on other releases or hardware configurations. Always test in a safe environment before applying any procedure to a production system.

== Conventions Used in This Document ==
Command examples throughout this guide follow standard Unix prompt conventions. A $ prefix indicates a command run as a regular user. Where elevated privileges are required, sudo is shown explicitly and # with a brief explanation if needed:
$ journalctl -b -1 # regular user
$ sudo apt install mtr # requires elevation via sudo
In recovery shell contexts, where you are operating directly as root, the prompt is #.

== System Freezes & Crashes ==

=== Check System Logs ===

=== Common Freeze Causes to Look For ===

* '''Kernel panics:''' Search for “kernel panic” or “Oops”
* '''Out of Memory (OOM):''' Search for “Out of memory” or “oom-killer”
* '''Hardware errors:''' Look for “MCE” (Machine Check Exception) or “hardware error”
* '''Driver issues:''' Check for module/driver failures
* '''Overheating:''' Check system temperatures

=== Check System Resources ===
$ free -h # memory usage

$ df -h # disk space

$ htop # interactive resource monitor

$ top # alternative resource monitor

$ sudo apt install btop # btop is a more detailed alternative

$ sudo dmesg | grep -i error # check for disk errors

=== Access Recovery Mode ===
Reboot and hold Shift to access the GRUB menu (timing depends on your GRUB configuration). Select “Advanced options”, then choose recovery mode. From the recovery menu, select “root” for a root shell.

=== Common Boot Fixes ===
$ sudo fsck /dev/sdXN # repair filesystem — identify device with lsblk first

$ sudo grub-install /dev/sdX # reinstall GRUB bootloader

$ sudo update-grub # regenerate GRUB configuration

$ cat /etc/fstab # check fstab for mount errors

The following are permanent errors caused by incomplete or buggy ACPI tables in the BIOS. They are harmless and can be safely ignored if they appear on your system:
[0.686554] ACPI Error: No handler for Region [ECRM] (00000000201accc4) [EmbeddedControl] (20250404/evregion-131)
[0.686577] ACPI Error: Region EmbeddedControl (ID=3) has no handler (20250404/exfldio-261)
[0.686594] ACPI Error: Aborting method \_SB.GPIO._EVT due to previous error (AE_NOT_EXIST) (20250404/psparse-529)

== Boot Problems ==

=== Check Boot Process ===
$ systemd-analyze blame # boot time by unit

$ systemd-analyze blame # boot time by unit

$ systemctl --failed # units that failed during boot

$ systemctl status <service-name> # e.g. systemctl status NetworkManager.service

=== Access Recovery Mode ===
Reboot and hold Shift to access the GRUB menu (timing depends on your GRUB configuration). Select “Advanced options”, then choose recovery mode. From the recovery menu, select “root” for a root shell.

=== Common Boot Fixes ===
$ sudo fsck /dev/sdXN # repair filesystem — identify device with lsblk first

$ sudo grub-install /dev/sdX # reinstall GRUB bootloader

$ sudo update-grub # regenerate GRUB configuration

$ cat /etc/fstab # check fstab for mount errors

== Network Issues ==

=== Diagnose Network Connection ===
$ ip addr show # check network interfaces

$ ping -c 4 8.8.8.8 # test IPv4 connectivity

$ ping -c 6 2a00:1450:4007:809::200e # test IPv6 connectivity

$ nslookup google.com # check DNS resolution

$ ip route show # view routing table

$ ss -tuln # check active connections

=== Restart Network Service ===
$ sudo systemctl restart NetworkManager # for systems using NetworkManager

$ sudo systemctl restart networking # for systems using networking service

$ sudo ip link set eth0 down # bring interface down

$ sudo ip link set eth0 up # bring interface back up

== MTR (Matt’s Traceroute) ==
MTR combines the functionality of ping and traceroute into a single real-time network diagnostic tool. It continuously monitors the path between your system and a destination, providing detailed statistics about latency and packet loss at each hop. When you need to demonstrate a network problem that is outside your control—to your hosting provider, for example—MTR gives you the evidence you need.

=== Installation ===
$ sudo apt install mtr

=== Basic Usage ===
$ mtr google.com # interactive mode

$ mtr --report google.com # report mode (10 cycles then exit)

$ mtr --report-cycles 50 google.com # specify number of cycles

$ mtr --tcp google.com # use TCP instead of ICMP

$ mtr --udp google.com # use UDP

$ mtr --no-dns google.com # skip DNS resolution (faster)

$ mtr --show-ips google.com # show both hostnames and IPs

== Understanding MTR Output ==

==== Sample Output ====
HOST: hostname Loss% Snt Last Avg Best Wrst StDev
1.|-- 192.168.1.1 0.0% 10 1.2 1.5 1.0 2.3 0.4
2.|-- 10.0.0.1 0.0% 10 8.5 9.2 7.8 12.1 1.3
3.|-- isp-gateway.net 0.0% 10 15.3 16.1 14.2 19.8 1.8
4.|-- ??? 100.0% 10 0.0 0.0 0.0 0.0 0.0
5.|-- google.com 0.0% 10 25.4 26.8 24.1 32.5 2.4

=== Column Meanings ===
'''HOST''' — Hostname or IP address of each hop in the route.

'''Loss%''' — Percentage of packets lost at this hop.

'''Snt''' — Number of packets sent to this hop.

'''Last''' — Latency of the most recent packet, in milliseconds.

'''Avg''' — Average latency across all packets sent, in milliseconds.

'''Best''' — Lowest latency recorded, in milliseconds.

'''Wrst''' — Highest latency recorded, in milliseconds.

'''StDev''' — Standard deviation of latency measurements—lower values indicate a more stable connection.

=== Interpreting Results ===
A healthy network shows 0% loss on all hops, stable latency with low StDev values, a gradual latency increase as hop count increases, and consistent response times.

==== Problem Indicators ====
'''1. High Packet Loss at a Specific Hop'''
5.|-- problem-router.net 25.0% 10 45.3 48.2 42.1 65.8 8.4
If loss continues to the destination, there is a real problem at that router. If loss appears only at that hop but not beyond, the router is likely rate-limiting ICMP responses—a false positive, not a real problem. The rule of thumb: if packet loss appears at hop N but hops N+1, N+2, and beyond show 0% loss, it’s almost certainly ICMP rate limiting.

'''2. High Latency at a Specific Hop'''
3.|-- slow-link.net 0.0% 10 150.3 155.2 148.1 165.8 5.4
This indicates a network bottleneck, a congested link, geographical distance (intercontinental hops), or slow routing equipment.

'''3. No Response (???)'''
4.|-- ??? 100.0% 10 0.0 0.0 0.0 0.0 0.0
The router at this hop is configured not to respond to ICMP or traceroute packets, or a firewall is blocking diagnostic packets. This is not necessarily a problem if later hops respond normally.

'''4. High Jitter (StDev)'''
6.|-- unstable.net 0.0% 10 35.3 52.8 28.1 95.2 24.7
A high StDev (24.7ms in this example) indicates inconsistent latency caused by network congestion or instability. This is particularly noticeable in real-time applications such as VoIP, gaming, and video calls.

'''5. Sudden Latency Spike'''
1.|-- 192.168.1.1 0.0% 10 1.2 1.5 1.0 2.3 0.4
2.|-- 10.0.0.1 0.0% 10 8.5 9.2 7.8 12.1 1.3
3.|-- problematic-hop.net 0.0% 10 180.5 185.2 178.1 195.8 6.4
4.|-- next-hop.net 0.0% 10 182.3 187.8 180.5 198.2 6.8
Hop 3 introduces approximately 170ms of additional latency (jumping from 9ms to 180ms). Everything from hop 3 onward carries that penalty.

=== Advanced Usage ===

==== Report Mode with Different Output Formats ====
$ mtr --report --csv google.com > network-report.csv # CSV for logging

$ mtr --report --json google.com # JSON for parsing

$ mtr --report --xml google.com # XML format

$ mtr --report-wide google.com # wide report, no abbreviations

==== Protocol Selection ====
$ mtr google.com # ICMP (default)

$ mtr --udp google.com # UDP

$ mtr --tcp google.com # TCP

$ sudo mtr --tcp --port 443 google.com # specific TCP port

$ sudo mtr --tcp --port 22 remote-server.com # default TCP port

==== Timing and Duration ====
$ mtr --interval 0.5 google.com # interval between pings (default 1s)

$ mtr --report-cycles 100 google.com # extended test, 100 cycles

$ mtr google.com # continuous monitoring (Ctrl+C to stop)

$ mtr --report-cycles 10 google.com # quick 10-cycle report

==== Advanced Options ====
$ mtr --aslookup google.com # show Autonomous System (AS) numbers

$ mtr --max-ttl 20 google.com # set maximum number of hops

$ mtr --psize 1000 google.com # set packet size

$ mtr --show-ips google.com # show both IP and hostname

$ mtr --address 192.168.1.100 google.com # specify source address

$ mtr -4 google.com # IPv4 only

$ mtr -6 google.com # IPv6 only

=== Interactive Mode Commands ===
When running MTR in interactive mode (mtr hostname), the following keys are available:

'''h''' — Display help.

'''d''' — Toggle display mode, cycling through different views.

'''n''' — Toggle between hostnames and IP addresses.

'''r''' — Reset all statistics.

'''p''' — Pause/unpause the display.

'''q''' — Quit MTR.

'''u''' — Switch between ICMP, UDP, and TCP modes.

'''y''' — Switch between IPv4 and IPv6.

'''o''' — Toggle field display options.

'''j''' — Toggle latency display.

=== Practical Troubleshooting Scenarios ===

==== Scenario 1: Diagnosing a Slow Website ====
$ mtr --report-cycles 100 --no-dns example.com
Look for high average latency at specific hops, packet loss at the destination, and high StDev values indicating jitter.

==== Scenario 2: Testing if a Firewall Blocks SSH ====
$ sudo mtr --tcp --port 22 --report-cycles 50 remote-server.com
If the last hop shows 100% loss but earlier hops are fine, port 22 may be filtered. Run a standard ICMP test for comparison.

==== Scenario 3: ISP Performance Issues ====
$ mtr --report-cycles 100 8.8.8.8

$ mtr --report-cycles 100 1.1.1.1
If issues appear in the first three or four hops, the problem is likely within your ISP’s network. If issues appear later, the problem lies beyond your ISP.

==== Scenario 4: VPN Troubleshooting ====
$ mtr --report-cycles 50 --no-dns google.com > before-vpn.txt # before VPN

$ mtr --report-cycles 50 --no-dns google.com > after-vpn.txt # after VPN

$ diff before-vpn.txt after-vpn.txt # compare the two

==== Scenario 5: Gaming/Streaming Performance ====
$ mtr --report-cycles 200 game-server.com
For real-time applications, look for low average latency (under 50ms for gaming), low StDev (under 5ms preferred), and zero packet loss.

==== Scenario 6: Intermittent Connectivity ====
$ mtr --report-cycles 500 --interval 1 target.com > long-test.txt # long-running test

$ mtr target.com # real-time monitoring

=== Continuous Monitoring ===

==== Log Network Performance Over Time ====
while true; do
timestamp=$(date +%Y%m%d-%H%M%S)
mtr --report --report-cycles 50 google.com > "mtr-$timestamp.txt"
sleep 3600
done

==== Monitor Multiple Destinations ====
#!/bin/bash
echo "=== MTR Report $(date) ===" > daily-network-report.txt
echo "" >> daily-network-report.txt

echo "Google DNS:" >> daily-network-report.txt
mtr --report --report-cycles 50 --no-dns 8.8.8.8 >> daily-network-report.txt
echo "" >> daily-network-report.txt

echo "Cloudflare DNS:" >> daily-network-report.txt
mtr --report --report-cycles 50 --no-dns 1.1.1.1 >> daily-network-report.txt
echo "" >> daily-network-report.txt

echo "Your Server:" >> daily-network-report.txt
mtr --report --report-cycles 50 your-server.com >> daily-network-report.txt

=== Useful Aliases for .bashrc ===
Quick network path analysis
alias mtrreport='mtr --report --report-cycles 50 --no-dns'

Monitor connection to Google DNS
alias netcheck='mtr --report-cycles 20 8.8.8.8'

Extended network test
alias mtrlong='mtr --report-cycles 100'

TCP port 443 test (HTTPS)
alias mtrhttps='sudo mtr --tcp --port 443 --report-cycles 30'

Quick comparison of major DNS providers
alias dnstest='echo "Google:" && mtr --report-cycles 20 8.8.8.8 && echo -e "\nCloudflare:" && mtr --report-cycles 20 1.1.1.1'
After adding to ~/.bashrc, reload with:
source ~/.bashrc

=== Troubleshooting Tips ===
'''Permission issues with TCP mode:'''
Use sudo for TCP on privileged ports
$ sudo mtr --tcp --port 443 example.com

Or set capabilities (one-time setup)
$ sudo setcap cap_net_raw+ep /usr/bin/mtr-packet
'''False positives:''' Packet loss at an intermediate hop but not at the destination is almost always ICMP rate limiting at that router, not a real problem. Example:
3.|-- router.isp.net 20.0% 50 15.3 16.1 14.2 19.8 1.8
4.|-- next-hop.net 0.0% 50 18.5 19.2 17.8 22.1 1.3
5.|-- destination.com 0.0% 50 25.4 26.8 24.1 32.5 2.4
Hop 3 shows 20% loss, but hops 4 and 5 show 0%. Traffic is flowing normally.

'''DNS resolution delays:''' If MTR is slow to start, skip DNS resolution with mtr --no-dns target.com and resolve addresses afterward with host 203.0.113.1 if needed.

'''Comparing results:'''
$ mtr --report --report-cycles 50 example.com > test1.txt
$ sleep 60
$ mtr --report --report-cycles 50 example.com > test2.txt
$ diff test1.txt test2.txt

=== When to Use MTR vs Other Tools ===
'''MTR''' — Best for continuous monitoring, identifying problem hops, and gathering detailed statistics. Requires installation via apt.

'''ping''' — Best for a quick connectivity test or simple latency check. Only tests the endpoint, not the path.

'''traceroute''' — Best for one-time path discovery. Provides no continuous monitoring.

'''ss / netstat''' — Best for checking local connection status. Doesn’t test remote paths.

=== Best Practices ===
Run at least 50–100 cycles for accurate statistics. Use --no-dns for faster results and to avoid DNS resolution issues during testing. Test at different times of day, since network conditions vary. If one protocol shows issues, compare ICMP, UDP, and TCP modes. Save reports with timestamps for trend analysis. Use 8.8.8.8 or 1.1.1.1 as known-good hosts to verify your local network first. Let MTR run for at least 30–60 seconds before drawing conclusions.

=== Reading Between the Lines ===
'''Good network health:'''
HOST: hostname Loss% Snt Last Avg Best Wrst StDev
1.|-- 192.168.1.1 0.0% 50 1.1 1.2 0.9 2.1 0.2
2.|-- 10.0.0.1 0.0% 50 8.2 8.5 7.5 10.2 0.5
3.|-- isp-gateway.net 0.0% 50 15.1 15.5 14.0 18.3 0.8
4.|-- google.com 0.0% 50 24.8 25.2 23.5 28.1 1.1
No packet loss, consistent latency, low jitter throughout.

'''Problematic network:'''
HOST: hostname Loss% Snt Last Avg Best Wrst StDev
1.|-- 192.168.1.1 0.0% 50 1.2 1.5 1.0 2.3 0.3
2.|-- 10.0.0.1 5.0% 50 45.3 52.8 8.1 245.2 45.7
3.|-- ??? 100.0% 50 0.0 0.0 0.0 0.0 0.0
4.|-- destination.com 15.0% 50 95.4 125.8 48.2 385.5 78.2
Packet loss at hop 2 and the destination, high jitter, and a very high worst-case latency all point to a genuine problem at or near hop 2.

=== Summary ===
MTR is your friend when it comes to diagnostics. Use --report-cycles 50 or higher for reliable data. Watch for packet loss at the destination—intermediate losses may be false positives. High StDev indicates an unstable connection. High average latency points to slow links. Use --no-dns for faster results. Compare ICMP, UDP, and TCP modes if issues appear. Test at different times of day for a complete picture.
== Package Management Issues ==

=== Fix Broken Packages ===
$ sudo apt update # update package lists
$ sudo apt --fix-broken install # fix broken dependencies
$ sudo dpkg --configure -a # reconfigure packages (no output = nothing to do)
$ sudo apt clean # clean package cache
$ sudo apt autoclean
$ sudo apt autoremove # remove unused packages

=== Handle Held or Locked Packages ===

# Fix broken dependencies
sudo apt --fix-broken install

# Reconfigure packages
sudo dpkg --configure -a
# No output means there is nothing to reconfigure

# Clean package cache
sudo apt clean
sudo apt autoclean

# Remove unused packages
sudo apt autoremove

=== Handle Held or Locked Packages ===
# If apt is locked, find the process holding the lock
sudo lsof /var/lib/dpkg/lock-frontend

# Remove the lock (use with caution—only if no package manager is running)
sudo rm /var/lib/dpkg/lock-frontend
sudo rm /var/lib/apt/lists/lock

# Reconfigure dpkg
sudo dpkg --configure -a
Disk & Filesystem Issues

=== Check Disk Health ===
# Check disk space
df -h

# Check inode usage
df -i

# View disk I/O statistics
# iostat is part of the sysstat package, which also includes:
# mpstat (CPU statistics), sar (system activity reporter),
# pidstat (process statistics), and cifsiostat (CIFS statistics)
sudo apt install sysstat

iostat -x 1

# Show stats in MB instead of KB
iostat -xm 2

# Monitor a specific device
iostat -x sda 1

# Check for disk errors in dmesg
sudo dmesg | grep -i "error\|fail"

# SMART disk health (requires smartmontools)
sudo apt install smartmontools
sudo smartctl -a /dev/sda

=== Repair Filesystem ===
# Unmount the partition first
sudo umount /dev/sdXN

# Run filesystem check
sudo fsck /dev/sdXN

# For ext4 specifically
sudo e2fsck -f /dev/sdXN
Performance Issues

=== Identify Resource Hogs ===
# CPU usage by process
top -o %CPU

# Memory usage by process
top -o %MEM

# Disk usage by directory
du -sh /* | sort -h

# Find large files
find / -type f -size +100M 2>/dev/null

# Check running processes sorted by memory use
ps aux --sort=-%mem | head -20

=== System Temperature Monitoring ===
# Install lm-sensors if not already present
sudo apt install lm-sensors
sudo sensors-detect

# View temperatures
sensors

# Real-time temperature monitoring
watch -n 2 sensors
See section 12 for a shell alias that makes temperature monitoring more convenient.

Service & Application Errors

=== Debug Service Problems ===
# Check service status
sudo systemctl status service-name

# View service logs
sudo journalctl -u service-name

# Restart a service
sudo systemctl restart service-name

# Enable service at boot
sudo systemctl enable service-name

# View recent service failures
journalctl -p err -b

=== Application Crash Investigation ===
# Check for core dumps
ls -lh /var/crash/

# View available application logs
ls /var/log/

# Follow syslog for application errors in real-time
sudo tail -f /var/log/syslog
Permission & Access Issues

=== Fix Common Permission Problems ===
# Check file ownership
ls -l /path/to/file

# Change ownership (user:group)
sudo chown user:group /path/to/file

# Change permissions
sudo chmod 644 /path/to/file

# Recursively fix permissions on a directory
sudo chown -R user:group /path/to/directory

=== User & Authentication Issues ===
# Check user information
id username

# View user login history
last -a

# Check failed login attempts
sudo journalctl | grep "authentication failure"

# Reset user password
sudo passwd username
Hardware Issues

=== Identify Hardware ===
# List all hardware (install if needed: sudo apt install lshw)
sudo lshw -short

# PCI devices
lspci -v

# USB devices
lsusb -v

# CPU information
lscpu

# Memory information
sudo dmidecode --type memory

=== Check Hardware Errors ===
# Kernel ring buffer (hardware messages)
dmesg | less
# Press q to quit

# Search for specific hardware issues
dmesg | grep -i "error\|fail\|warn"

# Check for USB issues
dmesg | grep -i usb
Quick Diagnostic Commands

=== System Information at a Glance ===
# Uptime and load average
uptime

# Kernel version
uname -r

# Debian version
cat /etc/debian_version

# Full system summary (install if needed: sudo apt install inxi)
sudo inxi -Fxz

=== Emergency Toolkit ===
# Create a consolidated diagnostic report
sudo journalctl -b > ~/system-report.txt
dmesg >> ~/system-report.txt
systemctl --failed >> ~/system-report.txt
df -h >> ~/system-report.txt
free -h >> ~/system-report.txt

# Watch logs in real-time
sudo journalctl -f

# Monitor system resources continuously
watch -n 1 'free -h && df -h'
Useful Aliases & Shortcuts
Add these to your ~/.bashrc for quick access to common troubleshooting commands:
# Monitor system temperatures in real-time
alias temps="watch -n 2 'for i in /sys/class/hwmon/hwmon*/; do echo -n \"\$(cat \${i}name): \"; cat \${i}temp*_input 2>/dev/null | while read temp; do echo \"scale=1; \$temp/1000\" | bc; done | tr \"\n\" \" \"; echo \"°C\"; done'"

# Quick system status
alias sysstat='echo "=== CPU ===" && uptime && echo -e "\n=== Memory ===" && free -h && echo -e "\n=== Disk ===" && df -h && echo -e "\n=== Top Processes ===" && ps aux --sort=-%mem | head -10'

# View last boot logs
alias lastboot='journalctl -b -1'

# Check failed services
alias failedservices='systemctl --failed'

# Monitor logs in real-time
alias watchlog='sudo journalctl -f'

# Quick network status
alias netstatus='ip addr show && echo -e "\n=== Routes ===" && ip route show'
After adding these, reload your shell configuration:
source ~/.bashrc
Tips for Effective Troubleshooting

'''Check logs first.''' journalctl and dmesg are your best starting point for almost any problem.

'''Change one thing at a time.''' It’s tempting to make several changes at once, but doing so makes it impossible to know what actually fixed—or broke—the problem.

'''Document your changes.''' Keep notes on what you’ve tried and what the results were. You’ll thank yourself later.

'''Use exact error messages.''' Copy error messages verbatim and search for them alongside “Debian” and the version number—for example, “Debian 13” or “Debian 13.1” for point releases.

'''Check what changed recently.''' Did you install something, update packages, or change the kernel before the problem appeared? Start there.

'''Back up before making major changes.''' This goes without saying, but it’s easy to skip when you’re in a hurry.

'''Use verbose mode.''' Adding -v or -vv flags to commands often reveals what would otherwise be silent failures.

'''Use the community.''' The Debian forums, Reddit, Stack Exchange, and the Debian mailing lists are all active and well-indexed. Someone has usually seen your problem before.

''This work was contributed by <dfn>distro-nix</dfn> to the Debian User Forums on 2025-10-27.''

Debian Forums Cache:About

2026-05-15T13:20:35Z

Distro-nix: Added disclaimer

== About Debian Forums Cache: ==

'''About Debian Forums Cache'''

"Debian Forums Cache exists because good technical work deserves a permanent home."

The Debian forums hold a lot of useful knowledge—tested procedures, hard-won solutions, and practical guides built from real experience. The Archives allows the authors to have cited works and permanent links to their works, something the forums cannot offer.

Forum threads age, and when the forums are down or gone, that knowledge is gone with them.

This cache means content is attributed, versioned, and stable—available whether the forums are there or not. If you wrote something worth keeping, it gets a permanent link. If you are looking for something or sharing something, even in your own writing, you can cite what you find in a readable and professional format.

We're not an open wiki. Submissions go through a review process, which keeps the quality high and the spam out. That accountability runs both ways: authors stand behind what they publish, and readers can trust that what they're reading has been verified by their peers, and is known to work with their version.

With that being said, please enjoy the best work of the Debian User Forum community.

'''A note on accuracy'''

Technical content ages. Debian moves forward, and a procedure that worked on Trixie may need adjustment by the time Forky is stable. Every entry in the Cache carries a publication date and the Debian version it was tested against—use those as your guide. If something doesn't work as described, check the version, check the date, and if you've found a genuine error, the CacheHandlers want to know.

The Cache is maintained by community volunteers. Content is peer-reviewed but not formally audited. Use your judgment, test in a safe environment before applying anything critical to production, and as always—have a backup.

'''Disclaimer'''

The papers are provided by the Cache on behalf of the authors, and are provided without any express or implied warranty. This includes, but is not limited to, the implied warranties of merchantability and fitness for a particular purpose. Readers assume all risk relating to the quality of the papers when applying their contents. Should the paper contain incorrect information, readers will be responsible for the cost of all necessary servicing, repairs or corrections. Unless required by applicable law or agreed in writing, no member of the Debian Forum or Cache site will be liable to you for any damages arising from your use of the paper. This includes general, special, incidental or consequential damages, such as loss of or damage to data, or losses sustained by you or third parties, or failure to follow the instructions supplied with the paper. This applies even if they have been advised of the possibility of such damages.

Debian First Aid Kit

2026-05-14T21:15:20Z

Distro-nix: /* Check System Resources */ Adding helpers to each input

'''Author:''' distro-nix '''Forum Profile:''' <dfn><nowiki>https://forums.debian.net/memberlist.php?mode=viewprofile&u=239099</nowiki></dfn> '''Debian Version:''' Debian 13 (Trixie) '''Kernel Versions Tested:''' 6.12.73+deb13-amd64 and 6.16.3+deb13-amd64 (x86_64) '''Created:''' 2025-10-27 '''Last Updated:''' 2026-03-11 '''ID:''' 544201 '''License:''' CC BY-SA 4.0

''This guide is preserved in its original form. Questions and feedback welcome via the author’s <dfn>Debian Forums profile</dfn>.''

== Abstract ==
The Debian First Aid Kit is a practical reference for diagnosing and resolving common system problems on Debian GNU/Linux. It covers commonly occuring categories of failure: system freezes and crashes, boot problems, network issues, package management, disk and filesystem errors, performance degradation, service and application failures, permission and access problems, hardware identification and errors, and quick diagnostics. Each section gives the relevant commands so you can understand what they do and why, rather than a bare list to copy and paste blindly. A dedicated section covers MTR (Matt’s Traceroute) in depth, including output interpretation, practical troubleshooting, and how to monitor continuously. The guide also includes a set of shell aliases for faster access to common diagnostic tasks. All commands are verified on Debian 13 (Trixie) running kernel 6.12.73+deb13-amd64 and 6.16.3+deb13-amd64. This document is intended for users with basic command-line familiarity who need a reliable, consolidated reference when something goes wrong.

'''Keywords:''' Debian, troubleshooting, system administration, MTR, network diagnostics, package management, journalctl, filesystem repair
----

== Tested Commands Disclaimer ==
All commands in this document have been verified on Debian 13 (Trixie) with the kernel versions listed above. Results may vary on other releases or hardware configurations. Always test in a safe environment before applying any procedure to a production system.

== Conventions Used in This Document ==
Command examples throughout this guide follow standard Unix prompt conventions. A $ prefix indicates a command run as a regular user. Where elevated privileges are required, sudo is shown explicitly and # with a brief explanation if needed:
$ journalctl -b -1 # regular user
$ sudo apt install mtr # requires elevation via sudo
In recovery shell contexts, where you are operating directly as root, the prompt is #.

== System Freezes & Crashes ==

=== Check System Logs ===

=== Common Freeze Causes to Look For ===

* '''Kernel panics:''' Search for “kernel panic” or “Oops”
* '''Out of Memory (OOM):''' Search for “Out of memory” or “oom-killer”
* '''Hardware errors:''' Look for “MCE” (Machine Check Exception) or “hardware error”
* '''Driver issues:''' Check for module/driver failures
* '''Overheating:''' Check system temperatures

=== Check System Resources ===
$ free -h # memory usage

$ df -h # disk space

$ htop # interactive resource monitor

$ top # alternative resource monitor

$ sudo apt install btop # btop is a more detailed alternative

$ sudo dmesg | grep -i error # check for disk errors

=== Access Recovery Mode ===
Reboot and hold Shift to access the GRUB menu (timing depends on your GRUB configuration). Select “Advanced options”, then choose recovery mode. From the recovery menu, select “root” for a root shell.

=== Common Boot Fixes ===
$ sudo fsck /dev/sdXN # repair filesystem — identify device with lsblk first

$ sudo grub-install /dev/sdX # reinstall GRUB bootloader

$ sudo update-grub # regenerate GRUB configuration

$ cat /etc/fstab # check fstab for mount errors

The following are permanent errors caused by incomplete or buggy ACPI tables in the BIOS. They are harmless and can be safely ignored if they appear on your system:
[0.686554] ACPI Error: No handler for Region [ECRM] (00000000201accc4) [EmbeddedControl] (20250404/evregion-131)
[0.686577] ACPI Error: Region EmbeddedControl (ID=3) has no handler (20250404/exfldio-261)
[0.686594] ACPI Error: Aborting method \_SB.GPIO._EVT due to previous error (AE_NOT_EXIST) (20250404/psparse-529)

== Boot Problems ==

=== Check Boot Process ===
$ systemd-analyze blame # boot time by unit

$ systemd-analyze blame # boot time by unit

$ systemctl --failed # units that failed during boot

$ systemctl status <service-name> # e.g. systemctl status NetworkManager.service

=== Access Recovery Mode ===
Reboot and hold Shift to access the GRUB menu (timing depends on your GRUB configuration). Select “Advanced options”, then choose recovery mode. From the recovery menu, select “root” for a root shell.

=== Common Boot Fixes ===
$ sudo fsck /dev/sdXN # repair filesystem — identify device with lsblk first

$ sudo grub-install /dev/sdX # reinstall GRUB bootloader

$ sudo update-grub # regenerate GRUB configuration

$ cat /etc/fstab # check fstab for mount errors

== Network Issues ==

=== Diagnose Network Connection ===
$ ip addr show # check network interfaces

$ ping -c 4 8.8.8.8 # test IPv4 connectivity

$ ping -c 6 2a00:1450:4007:809::200e # test IPv6 connectivity

$ nslookup google.com # check DNS resolution

$ ip route show # view routing table

$ ss -tuln # check active connections

=== Restart Network Service ===
$ sudo systemctl restart NetworkManager # for systems using NetworkManager

$ sudo systemctl restart networking # for systems using networking service

$ sudo ip link set eth0 down # bring interface down

$ sudo ip link set eth0 up # bring interface back up

== MTR (Matt’s Traceroute) ==
MTR combines the functionality of ping and traceroute into a single real-time network diagnostic tool. It continuously monitors the path between your system and a destination, providing detailed statistics about latency and packet loss at each hop. When you need to demonstrate a network problem that is outside your control—to your hosting provider, for example—MTR gives you the evidence you need.

=== Installation ===
$ sudo apt install mtr

=== Basic Usage ===
$ mtr google.com # interactive mode

$ mtr --report google.com # report mode (10 cycles then exit)

$ mtr --report-cycles 50 google.com # specify number of cycles

$ mtr --tcp google.com # use TCP instead of ICMP

$ mtr --udp google.com # use UDP

$ mtr --no-dns google.com # skip DNS resolution (faster)

$ mtr --show-ips google.com # show both hostnames and IPs

== Understanding MTR Output ==

==== Sample Output ====
HOST: hostname Loss% Snt Last Avg Best Wrst StDev
1.|-- 192.168.1.1 0.0% 10 1.2 1.5 1.0 2.3 0.4
2.|-- 10.0.0.1 0.0% 10 8.5 9.2 7.8 12.1 1.3
3.|-- isp-gateway.net 0.0% 10 15.3 16.1 14.2 19.8 1.8
4.|-- ??? 100.0% 10 0.0 0.0 0.0 0.0 0.0
5.|-- google.com 0.0% 10 25.4 26.8 24.1 32.5 2.4

=== Column Meanings ===
'''HOST''' — Hostname or IP address of each hop in the route.

'''Loss%''' — Percentage of packets lost at this hop.

'''Snt''' — Number of packets sent to this hop.

'''Last''' — Latency of the most recent packet, in milliseconds.

'''Avg''' — Average latency across all packets sent, in milliseconds.

'''Best''' — Lowest latency recorded, in milliseconds.

'''Wrst''' — Highest latency recorded, in milliseconds.

'''StDev''' — Standard deviation of latency measurements—lower values indicate a more stable connection.

=== Interpreting Results ===
A healthy network shows 0% loss on all hops, stable latency with low StDev values, a gradual latency increase as hop count increases, and consistent response times.

==== Problem Indicators ====
'''1. High Packet Loss at a Specific Hop'''
5.|-- problem-router.net 25.0% 10 45.3 48.2 42.1 65.8 8.4
If loss continues to the destination, there is a real problem at that router. If loss appears only at that hop but not beyond, the router is likely rate-limiting ICMP responses—a false positive, not a real problem. The rule of thumb: if packet loss appears at hop N but hops N+1, N+2, and beyond show 0% loss, it’s almost certainly ICMP rate limiting.

'''2. High Latency at a Specific Hop'''
3.|-- slow-link.net 0.0% 10 150.3 155.2 148.1 165.8 5.4
This indicates a network bottleneck, a congested link, geographical distance (intercontinental hops), or slow routing equipment.

'''3. No Response (???)'''
4.|-- ??? 100.0% 10 0.0 0.0 0.0 0.0 0.0
The router at this hop is configured not to respond to ICMP or traceroute packets, or a firewall is blocking diagnostic packets. This is not necessarily a problem if later hops respond normally.

'''4. High Jitter (StDev)'''
6.|-- unstable.net 0.0% 10 35.3 52.8 28.1 95.2 24.7
A high StDev (24.7ms in this example) indicates inconsistent latency caused by network congestion or instability. This is particularly noticeable in real-time applications such as VoIP, gaming, and video calls.

'''5. Sudden Latency Spike'''
1.|-- 192.168.1.1 0.0% 10 1.2 1.5 1.0 2.3 0.4
2.|-- 10.0.0.1 0.0% 10 8.5 9.2 7.8 12.1 1.3
3.|-- problematic-hop.net 0.0% 10 180.5 185.2 178.1 195.8 6.4
4.|-- next-hop.net 0.0% 10 182.3 187.8 180.5 198.2 6.8
Hop 3 introduces approximately 170ms of additional latency (jumping from 9ms to 180ms). Everything from hop 3 onward carries that penalty.

=== Advanced Usage ===

==== Report Mode with Different Output Formats ====
$ mtr --report --csv google.com > network-report.csv # CSV for logging

$ mtr --report --json google.com # JSON for parsing

$ mtr --report --xml google.com # XML format

$ mtr --report-wide google.com # wide report, no abbreviations

==== Protocol Selection ====
$ mtr google.com # ICMP (default)

$ mtr --udp google.com # UDP

$ mtr --tcp google.com # TCP

$ sudo mtr --tcp --port 443 google.com # specific TCP port

$ sudo mtr --tcp --port 22 remote-server.com # default TCP port

==== Timing and Duration ====
$ mtr --interval 0.5 google.com # interval between pings (default 1s)

$ mtr --report-cycles 100 google.com # extended test, 100 cycles

$ mtr google.com # continuous monitoring (Ctrl+C to stop)

$ mtr --report-cycles 10 google.com # quick 10-cycle report

==== Advanced Options ====
$ mtr --aslookup google.com # show Autonomous System (AS) numbers

$ mtr --max-ttl 20 google.com # set maximum number of hops

$ mtr --psize 1000 google.com # set packet size

$ mtr --show-ips google.com # show both IP and hostname

$ mtr --address 192.168.1.100 google.com # specify source address

$ mtr -4 google.com # IPv4 only

$ mtr -6 google.com # IPv6 only

=== Interactive Mode Commands ===
When running MTR in interactive mode (mtr hostname), the following keys are available:

'''h''' — Display help.

'''d''' — Toggle display mode, cycling through different views.

'''n''' — Toggle between hostnames and IP addresses.

'''r''' — Reset all statistics.

'''p''' — Pause/unpause the display.

'''q''' — Quit MTR.

'''u''' — Switch between ICMP, UDP, and TCP modes.

'''y''' — Switch between IPv4 and IPv6.

'''o''' — Toggle field display options.

'''j''' — Toggle latency display.

=== Practical Troubleshooting Scenarios ===

==== Scenario 1: Diagnosing a Slow Website ====
$ mtr --report-cycles 100 --no-dns example.com
Look for high average latency at specific hops, packet loss at the destination, and high StDev values indicating jitter.

==== Scenario 2: Testing if a Firewall Blocks SSH ====
$ sudo mtr --tcp --port 22 --report-cycles 50 remote-server.com
If the last hop shows 100% loss but earlier hops are fine, port 22 may be filtered. Run a standard ICMP test for comparison.

==== Scenario 3: ISP Performance Issues ====
$ mtr --report-cycles 100 8.8.8.8

$ mtr --report-cycles 100 1.1.1.1
If issues appear in the first three or four hops, the problem is likely within your ISP’s network. If issues appear later, the problem lies beyond your ISP.

==== Scenario 4: VPN Troubleshooting ====
$ mtr --report-cycles 50 --no-dns google.com > before-vpn.txt # before VPN

$ mtr --report-cycles 50 --no-dns google.com > after-vpn.txt # after VPN

$ diff before-vpn.txt after-vpn.txt # compare the two

==== Scenario 5: Gaming/Streaming Performance ====
$ mtr --report-cycles 200 game-server.com
For real-time applications, look for low average latency (under 50ms for gaming), low StDev (under 5ms preferred), and zero packet loss.

==== Scenario 6: Intermittent Connectivity ====
$ mtr --report-cycles 500 --interval 1 target.com > long-test.txt # long-running test

$ mtr target.com # real-time monitoring

=== Continuous Monitoring ===

==== Log Network Performance Over Time ====
while true; do
timestamp=$(date +%Y%m%d-%H%M%S)
mtr --report --report-cycles 50 google.com > "mtr-$timestamp.txt"
sleep 3600
done

==== Monitor Multiple Destinations ====
#!/bin/bash
echo "=== MTR Report $(date) ===" > daily-network-report.txt
echo "" >> daily-network-report.txt

echo "Google DNS:" >> daily-network-report.txt
mtr --report --report-cycles 50 --no-dns 8.8.8.8 >> daily-network-report.txt
echo "" >> daily-network-report.txt

echo "Cloudflare DNS:" >> daily-network-report.txt
mtr --report --report-cycles 50 --no-dns 1.1.1.1 >> daily-network-report.txt
echo "" >> daily-network-report.txt

echo "Your Server:" >> daily-network-report.txt
mtr --report --report-cycles 50 your-server.com >> daily-network-report.txt

=== Useful Aliases for .bashrc ===
# Quick network path analysis
alias mtrreport='mtr --report --report-cycles 50 --no-dns'

# Monitor connection to Google DNS
alias netcheck='mtr --report-cycles 20 8.8.8.8'

# Extended network test
alias mtrlong='mtr --report-cycles 100'

# TCP port 443 test (HTTPS)
alias mtrhttps='sudo mtr --tcp --port 443 --report-cycles 30'

# Quick comparison of major DNS providers
alias dnstest='echo "Google:" && mtr --report-cycles 20 8.8.8.8 && echo -e "\nCloudflare:" && mtr --report-cycles 20 1.1.1.1'
After adding to ~/.bashrc, reload with:
source ~/.bashrc

=== Troubleshooting Tips ===
'''Permission issues with TCP mode:'''
# Use sudo for TCP on privileged ports
sudo mtr --tcp --port 443 example.com

# Or set capabilities (one-time setup)
sudo setcap cap_net_raw+ep /usr/bin/mtr-packet
'''False positives:''' Packet loss at an intermediate hop but not at the destination is almost always ICMP rate limiting at that router, not a real problem. Example:
3.|-- router.isp.net 20.0% 50 15.3 16.1 14.2 19.8 1.8
4.|-- next-hop.net 0.0% 50 18.5 19.2 17.8 22.1 1.3
5.|-- destination.com 0.0% 50 25.4 26.8 24.1 32.5 2.4
Hop 3 shows 20% loss, but hops 4 and 5 show 0%. Traffic is flowing normally.

'''DNS resolution delays:''' If MTR is slow to start, skip DNS resolution with mtr --no-dns target.com and resolve addresses afterward with host 203.0.113.1 if needed.

'''Comparing results:'''
mtr --report --report-cycles 50 example.com > test1.txt
sleep 60
mtr --report --report-cycles 50 example.com > test2.txt
diff test1.txt test2.txt

=== When to Use MTR vs Other Tools ===
'''MTR''' — Best for continuous monitoring, identifying problem hops, and gathering detailed statistics. Requires installation via apt.

'''ping''' — Best for a quick connectivity test or simple latency check. Only tests the endpoint, not the path.

'''traceroute''' — Best for one-time path discovery. Provides no continuous monitoring.

'''ss / netstat''' — Best for checking local connection status. Doesn’t test remote paths.

=== Best Practices ===
Run at least 50–100 cycles for accurate statistics. Use --no-dns for faster results and to avoid DNS resolution issues during testing. Test at different times of day, since network conditions vary. If one protocol shows issues, compare ICMP, UDP, and TCP modes. Save reports with timestamps for trend analysis. Use 8.8.8.8 or 1.1.1.1 as known-good hosts to verify your local network first. Let MTR run for at least 30–60 seconds before drawing conclusions.

=== Reading Between the Lines ===
'''Good network health:'''
HOST: hostname Loss% Snt Last Avg Best Wrst StDev
1.|-- 192.168.1.1 0.0% 50 1.1 1.2 0.9 2.1 0.2
2.|-- 10.0.0.1 0.0% 50 8.2 8.5 7.5 10.2 0.5
3.|-- isp-gateway.net 0.0% 50 15.1 15.5 14.0 18.3 0.8
4.|-- google.com 0.0% 50 24.8 25.2 23.5 28.1 1.1
No packet loss, consistent latency, low jitter throughout.

'''Problematic network:'''
HOST: hostname Loss% Snt Last Avg Best Wrst StDev
1.|-- 192.168.1.1 0.0% 50 1.2 1.5 1.0 2.3 0.3
2.|-- 10.0.0.1 5.0% 50 45.3 52.8 8.1 245.2 45.7
3.|-- ??? 100.0% 50 0.0 0.0 0.0 0.0 0.0
4.|-- destination.com 15.0% 50 95.4 125.8 48.2 385.5 78.2
Packet loss at hop 2 and the destination, high jitter, and a very high worst-case latency all point to a genuine problem at or near hop 2.

=== Summary ===
MTR is your Swiss Army knife for network diagnostics. Use --report-cycles 50 or higher for reliable data. Watch for packet loss at the destination—intermediate losses may be false positives. High StDev indicates an unstable connection. High average latency points to slow links. Use --no-dns for faster results. Compare ICMP, UDP, and TCP modes if issues appear. Test at different times of day for a complete picture.
----

== 5. Package Management Issues ==

=== Fix Broken Packages ===
# Update package lists
sudo apt update

# Fix broken dependencies
sudo apt --fix-broken install

# Reconfigure packages
sudo dpkg --configure -a
# No output means there is nothing to reconfigure

# Clean package cache
sudo apt clean
sudo apt autoclean

# Remove unused packages
sudo apt autoremove

=== Handle Held or Locked Packages ===
# If apt is locked, find the process holding the lock
sudo lsof /var/lib/dpkg/lock-frontend

# Remove the lock (use with caution—only if no package manager is running)
sudo rm /var/lib/dpkg/lock-frontend
sudo rm /var/lib/apt/lists/lock

# Reconfigure dpkg
sudo dpkg --configure -a
Disk & Filesystem Issues

=== Check Disk Health ===
# Check disk space
df -h

# Check inode usage
df -i

# View disk I/O statistics
# iostat is part of the sysstat package, which also includes:
# mpstat (CPU statistics), sar (system activity reporter),
# pidstat (process statistics), and cifsiostat (CIFS statistics)
sudo apt install sysstat

iostat -x 1

# Show stats in MB instead of KB
iostat -xm 2

# Monitor a specific device
iostat -x sda 1

# Check for disk errors in dmesg
sudo dmesg | grep -i "error\|fail"

# SMART disk health (requires smartmontools)
sudo apt install smartmontools
sudo smartctl -a /dev/sda

=== Repair Filesystem ===
# Unmount the partition first
sudo umount /dev/sdXN

# Run filesystem check
sudo fsck /dev/sdXN

# For ext4 specifically
sudo e2fsck -f /dev/sdXN
Performance Issues

=== Identify Resource Hogs ===
# CPU usage by process
top -o %CPU

# Memory usage by process
top -o %MEM

# Disk usage by directory
du -sh /* | sort -h

# Find large files
find / -type f -size +100M 2>/dev/null

# Check running processes sorted by memory use
ps aux --sort=-%mem | head -20

=== System Temperature Monitoring ===
# Install lm-sensors if not already present
sudo apt install lm-sensors
sudo sensors-detect

# View temperatures
sensors

# Real-time temperature monitoring
watch -n 2 sensors
See section 12 for a shell alias that makes temperature monitoring more convenient.

Service & Application Errors

=== Debug Service Problems ===
# Check service status
sudo systemctl status service-name

# View service logs
sudo journalctl -u service-name

# Restart a service
sudo systemctl restart service-name

# Enable service at boot
sudo systemctl enable service-name

# View recent service failures
journalctl -p err -b

=== Application Crash Investigation ===
# Check for core dumps
ls -lh /var/crash/

# View available application logs
ls /var/log/

# Follow syslog for application errors in real-time
sudo tail -f /var/log/syslog
Permission & Access Issues

=== Fix Common Permission Problems ===
# Check file ownership
ls -l /path/to/file

# Change ownership (user:group)
sudo chown user:group /path/to/file

# Change permissions
sudo chmod 644 /path/to/file

# Recursively fix permissions on a directory
sudo chown -R user:group /path/to/directory

=== User & Authentication Issues ===
# Check user information
id username

# View user login history
last -a

# Check failed login attempts
sudo journalctl | grep "authentication failure"

# Reset user password
sudo passwd username
Hardware Issues

=== Identify Hardware ===
# List all hardware (install if needed: sudo apt install lshw)
sudo lshw -short

# PCI devices
lspci -v

# USB devices
lsusb -v

# CPU information
lscpu

# Memory information
sudo dmidecode --type memory

=== Check Hardware Errors ===
# Kernel ring buffer (hardware messages)
dmesg | less
# Press q to quit

# Search for specific hardware issues
dmesg | grep -i "error\|fail\|warn"

# Check for USB issues
dmesg | grep -i usb
Quick Diagnostic Commands

=== System Information at a Glance ===
# Uptime and load average
uptime

# Kernel version
uname -r

# Debian version
cat /etc/debian_version

# Full system summary (install if needed: sudo apt install inxi)
sudo inxi -Fxz

=== Emergency Toolkit ===
# Create a consolidated diagnostic report
sudo journalctl -b > ~/system-report.txt
dmesg >> ~/system-report.txt
systemctl --failed >> ~/system-report.txt
df -h >> ~/system-report.txt
free -h >> ~/system-report.txt

# Watch logs in real-time
sudo journalctl -f

# Monitor system resources continuously
watch -n 1 'free -h && df -h'
Useful Aliases & Shortcuts
Add these to your ~/.bashrc for quick access to common troubleshooting commands:
# Monitor system temperatures in real-time
alias temps="watch -n 2 'for i in /sys/class/hwmon/hwmon*/; do echo -n \"\$(cat \${i}name): \"; cat \${i}temp*_input 2>/dev/null | while read temp; do echo \"scale=1; \$temp/1000\" | bc; done | tr \"\n\" \" \"; echo \"°C\"; done'"

# Quick system status
alias sysstat='echo "=== CPU ===" && uptime && echo -e "\n=== Memory ===" && free -h && echo -e "\n=== Disk ===" && df -h && echo -e "\n=== Top Processes ===" && ps aux --sort=-%mem | head -10'

# View last boot logs
alias lastboot='journalctl -b -1'

# Check failed services
alias failedservices='systemctl --failed'

# Monitor logs in real-time
alias watchlog='sudo journalctl -f'

# Quick network status
alias netstatus='ip addr show && echo -e "\n=== Routes ===" && ip route show'
After adding these, reload your shell configuration:
source ~/.bashrc
Tips for Effective Troubleshooting

'''Check logs first.''' journalctl and dmesg are your best starting point for almost any problem.

'''Change one thing at a time.''' It’s tempting to make several changes at once, but doing so makes it impossible to know what actually fixed—or broke—the problem.

'''Document your changes.''' Keep notes on what you’ve tried and what the results were. You’ll thank yourself later.

'''Use exact error messages.''' Copy error messages verbatim and search for them alongside “Debian” and the version number—for example, “Debian 13” or “Debian 13.1” for point releases.

'''Check what changed recently.''' Did you install something, update packages, or change the kernel before the problem appeared? Start there.

'''Back up before making major changes.''' This goes without saying, but it’s easy to skip when you’re in a hurry.

'''Use verbose mode.''' Adding -v or -vv flags to commands often reveals what would otherwise be silent failures.

'''Use the community.''' The Debian forums, Reddit, Stack Exchange, and the Debian mailing lists are all active and well-indexed. Someone has usually seen your problem before.

''This work was contributed by <dfn>distro-nix</dfn> to the Debian User Forums on 2025-10-27.''

Debian First Aid Kit

2026-05-14T20:47:57Z

Distro-nix: /* The following are permanent errors caused by incomplete or buggy ACPI tables in the BIOS. They are harmless and can be safely ignored if they appear on your system: */

'''Author:''' distro-nix '''Forum Profile:''' <dfn><nowiki>https://forums.debian.net/memberlist.php?mode=viewprofile&u=239099</nowiki></dfn> '''Debian Version:''' Debian 13 (Trixie) '''Kernel Versions Tested:''' 6.12.73+deb13-amd64 and 6.16.3+deb13-amd64 (x86_64) '''Created:''' 2025-10-27 '''Last Updated:''' 2026-03-11 '''ID:''' 544201 '''License:''' CC BY-SA 4.0

''This guide is preserved in its original form. Questions and feedback welcome via the author’s <dfn>Debian Forums profile</dfn>.''

== Abstract ==
The Debian First Aid Kit is a practical reference for diagnosing and resolving common system problems on Debian GNU/Linux. It covers commonly occuring categories of failure: system freezes and crashes, boot problems, network issues, package management, disk and filesystem errors, performance degradation, service and application failures, permission and access problems, hardware identification and errors, and quick diagnostics. Each section gives the relevant commands so you can understand what they do and why, rather than a bare list to copy and paste blindly. A dedicated section covers MTR (Matt’s Traceroute) in depth, including output interpretation, practical troubleshooting, and how to monitor continuously. The guide also includes a set of shell aliases for faster access to common diagnostic tasks. All commands are verified on Debian 13 (Trixie) running kernel 6.12.73+deb13-amd64 and 6.16.3+deb13-amd64. This document is intended for users with basic command-line familiarity who need a reliable, consolidated reference when something goes wrong.

'''Keywords:''' Debian, troubleshooting, system administration, MTR, network diagnostics, package management, journalctl, filesystem repair
----

== Tested Commands Disclaimer ==
All commands in this document have been verified on Debian 13 (Trixie) with the kernel versions listed above. Results may vary on other releases or hardware configurations. Always test in a safe environment before applying any procedure to a production system.

== Conventions Used in This Document ==
Command examples throughout this guide follow standard Unix prompt conventions. A $ prefix indicates a command run as a regular user. Where elevated privileges are required, sudo is shown explicitly and # with a brief explanation if needed:
$ journalctl -b -1 # regular user
$ sudo apt install mtr # requires elevation via sudo
In recovery shell contexts, where you are operating directly as root, the prompt is #.

== System Freezes & Crashes ==

=== Check System Logs ===

=== Common Freeze Causes to Look For ===

* '''Kernel panics:''' Search for “kernel panic” or “Oops”
* '''Out of Memory (OOM):''' Search for “Out of memory” or “oom-killer”
* '''Hardware errors:''' Look for “MCE” (Machine Check Exception) or “hardware error”
* '''Driver issues:''' Check for module/driver failures
* '''Overheating:''' Check system temperatures

=== Check System Resources ===
$ free -h # memory usage

$ df -h # disk space

$ htop # interactive resource monitor

$ top # alternative resource monitor

$ sudo apt install btop # btop is a more detailed alternative

$ sudo dmesg | grep -i error # check for disk errors

=== Access Recovery Mode ===
Reboot and hold Shift to access the GRUB menu (timing depends on your GRUB configuration). Select “Advanced options”, then choose recovery mode. From the recovery menu, select “root” for a root shell.

=== Common Boot Fixes ===
$ sudo fsck /dev/sdXN # repair filesystem — identify device with lsblk first

$ sudo grub-install /dev/sdX # reinstall GRUB bootloader

$ sudo update-grub # regenerate GRUB configuration

$ cat /etc/fstab # check fstab for mount errors

The following are permanent errors caused by incomplete or buggy ACPI tables in the BIOS. They are harmless and can be safely ignored if they appear on your system:
[0.686554] ACPI Error: No handler for Region [ECRM] (00000000201accc4) [EmbeddedControl] (20250404/evregion-131)
[0.686577] ACPI Error: Region EmbeddedControl (ID=3) has no handler (20250404/exfldio-261)
[0.686594] ACPI Error: Aborting method \_SB.GPIO._EVT due to previous error (AE_NOT_EXIST) (20250404/psparse-529)

== Boot Problems ==

=== Check Boot Process ===
$ systemd-analyze blame # boot time by unit

$ systemd-analyze blame # boot time by unit

$ systemctl --failed # units that failed during boot

$ systemctl status <service-name> # e.g. systemctl status NetworkManager.service

=== Access Recovery Mode ===
Reboot and hold Shift to access the GRUB menu (timing depends on your GRUB configuration). Select “Advanced options”, then choose recovery mode. From the recovery menu, select “root” for a root shell.

=== Common Boot Fixes ===
Repair filesystem errors — first identify the device with lsblk
sudo fsck /dev/sdXN

# Reinstall GRUB bootloader
sudo grub-install /dev/sdX
sudo update-grub

# Check fstab for mount errors
cat /etc/fstab

== Network Issues ==

=== Diagnose Network Connection ===
# Check network interfaces
ip addr show

# Test IPv4 connectivity
ping -c 4 8.8.8.8

# Test IPv6 connectivity
ping -c 6 2a00:1450:4007:809::200e

# Check DNS resolution
nslookup google.com

# View routing table
ip route show

# Check active connections
ss -tuln

=== Restart Network Service ===
# For systems with NetworkManager
sudo systemctl restart NetworkManager

# For systems with networking service
sudo systemctl restart networking

# Bring interface down and up
sudo ip link set eth0 down
sudo ip link set eth0 up

== MTR (Matt’s Traceroute) ==
MTR combines the functionality of ping and traceroute into a single real-time network diagnostic tool. It continuously monitors the path between your system and a destination, providing detailed statistics about latency and packet loss at each hop. When you need to demonstrate a network problem that is outside your control—to your hosting provider, for example—MTR gives you the evidence to do so.

=== Installation ===
sudo apt install mtr

=== Basic Usage ===
# Basic MTR (interactive mode)
mtr google.com

# Report mode (run 10 cycles and exit)
mtr --report google.com

# Specify number of pings
mtr --report-cycles 50 google.com

# Use TCP instead of ICMP
mtr --tcp google.com

# Use UDP
mtr --udp google.com

# No DNS resolution (faster, shows IPs only)
mtr --no-dns google.com

# Show both hostnames and IPs
mtr --show-ips google.com

== Understanding MTR Output ==

==== Sample Output ====
HOST: hostname Loss% Snt Last Avg Best Wrst StDev
1.|-- 192.168.1.1 0.0% 10 1.2 1.5 1.0 2.3 0.4
2.|-- 10.0.0.1 0.0% 10 8.5 9.2 7.8 12.1 1.3
3.|-- isp-gateway.net 0.0% 10 15.3 16.1 14.2 19.8 1.8
4.|-- ??? 100.0% 10 0.0 0.0 0.0 0.0 0.0
5.|-- google.com 0.0% 10 25.4 26.8 24.1 32.5 2.4

=== Column Meanings ===
'''HOST''' — Hostname or IP address of each hop in the route.

'''Loss%''' — Percentage of packets lost at this hop.

'''Snt''' — Number of packets sent to this hop.

'''Last''' — Latency of the most recent packet, in milliseconds.

'''Avg''' — Average latency across all packets sent, in milliseconds.

'''Best''' — Lowest latency recorded, in milliseconds.

'''Wrst''' — Highest latency recorded, in milliseconds.

'''StDev''' — Standard deviation of latency measurements—lower values indicate a more stable connection.

=== Interpreting Results ===
A healthy network shows 0% loss on all hops, stable latency with low StDev values, a gradual latency increase as hop count increases, and consistent response times.

==== Problem Indicators ====
'''1. High Packet Loss at a Specific Hop'''
5.|-- problem-router.net 25.0% 10 45.3 48.2 42.1 65.8 8.4
If loss continues to the destination, there is a real problem at that router. If loss appears only at that hop but not beyond, the router is likely rate-limiting ICMP responses—a false positive, not a real problem. The rule of thumb: if packet loss appears at hop N but hops N+1, N+2, and beyond show 0% loss, it’s almost certainly ICMP rate limiting.

'''2. High Latency at a Specific Hop'''
3.|-- slow-link.net 0.0% 10 150.3 155.2 148.1 165.8 5.4
This indicates a network bottleneck, a congested link, geographical distance (intercontinental hops), or slow routing equipment.

'''3. No Response (???)'''
4.|-- ??? 100.0% 10 0.0 0.0 0.0 0.0 0.0
The router at this hop is configured not to respond to ICMP or traceroute packets, or a firewall is blocking diagnostic packets. This is not necessarily a problem if later hops respond normally.

'''4. High Jitter (StDev)'''
6.|-- unstable.net 0.0% 10 35.3 52.8 28.1 95.2 24.7
A high StDev (24.7ms in this example) indicates inconsistent latency caused by network congestion or instability. This is particularly noticeable in real-time applications such as VoIP, gaming, and video calls.

'''5. Sudden Latency Spike'''
1.|-- 192.168.1.1 0.0% 10 1.2 1.5 1.0 2.3 0.4
2.|-- 10.0.0.1 0.0% 10 8.5 9.2 7.8 12.1 1.3
3.|-- problematic-hop.net 0.0% 10 180.5 185.2 178.1 195.8 6.4
4.|-- next-hop.net 0.0% 10 182.3 187.8 180.5 198.2 6.8
Hop 3 introduces approximately 170ms of additional latency (jumping from 9ms to 180ms). Everything from hop 3 onward carries that penalty.

=== Advanced Usage ===

==== Report Mode with Different Output Formats ====
# CSV format for logging and analysis
mtr --report --csv google.com > network-report.csv

# JSON output for parsing
mtr --report --json google.com

# XML format
mtr --report --xml google.com

# Wide report (no abbreviations)
mtr --report-wide google.com

==== Protocol Selection ====
# Use ICMP (default)
mtr google.com

# Use UDP
mtr --udp google.com

# Use TCP
mtr --tcp google.com

# Test a specific TCP port
sudo mtr --tcp --port 443 google.com
sudo mtr --tcp --port 22 remote-server.com

==== Timing and Duration ====
# Specify interval between pings (default 1 second)
mtr --interval 0.5 google.com

# Extended test with 100 cycles
mtr --report-cycles 100 google.com

# Continuous monitoring (Ctrl+C to stop)
mtr google.com

# Quick 10-cycle report
mtr --report-cycles 10 google.com

==== Advanced Options ====
# Show Autonomous System (AS) numbers
mtr --aslookup google.com

# Set maximum number of hops
mtr --max-ttl 20 google.com

# Set packet size
mtr --psize 1000 google.com

# Show both IP and hostname
mtr --show-ips google.com

# Specify source address (useful with multiple network interfaces)
mtr --address 192.168.1.100 google.com

# IPv4 only
mtr -4 google.com

# IPv6 only
mtr -6 google.com

=== Interactive Mode Commands ===
When running MTR in interactive mode (mtr hostname), the following keys are available:

'''h''' — Display help.

'''d''' — Toggle display mode, cycling through different views.

'''n''' — Toggle between hostnames and IP addresses.

'''r''' — Reset all statistics.

'''p''' — Pause/unpause the display.

'''q''' — Quit MTR.

'''u''' — Switch between ICMP, UDP, and TCP modes.

'''y''' — Switch between IPv4 and IPv6.

'''o''' — Toggle field display options.

'''j''' — Toggle latency display.

=== Practical Troubleshooting Scenarios ===

==== Scenario 1: Diagnosing a Slow Website ====
mtr --report-cycles 100 --no-dns example.com
Look for high average latency at specific hops, packet loss at the destination, and high StDev values indicating jitter.

==== Scenario 2: Testing if a Firewall Blocks SSH ====
sudo mtr --tcp --port 22 --report-cycles 50 remote-server.com
If the last hop shows 100% loss but earlier hops are fine, port 22 may be filtered. Run a standard ICMP test for comparison.

==== Scenario 3: ISP Performance Issues ====
mtr --report-cycles 100 8.8.8.8
mtr --report-cycles 100 1.1.1.1
If issues appear in the first three or four hops, the problem is likely within your ISP’s network. If issues appear later, the problem lies beyond your ISP.

==== Scenario 4: VPN Troubleshooting ====
# Test before connecting to VPN
mtr --report-cycles 50 --no-dns google.com > before-vpn.txt

# Test after connecting to VPN
mtr --report-cycles 50 --no-dns google.com > after-vpn.txt

# Compare the two
diff before-vpn.txt after-vpn.txt

==== Scenario 5: Gaming/Streaming Performance ====
mtr --report-cycles 200 game-server.com
For real-time applications, look for low average latency (under 50ms for gaming), low StDev (under 5ms preferred), and zero packet loss.

==== Scenario 6: Intermittent Connectivity ====
# Long-running test to catch intermittent issues
mtr --report-cycles 500 --interval 1 target.com > long-test.txt

# Or monitor in real-time for several minutes
mtr target.com

=== Continuous Monitoring ===

==== Log Network Performance Over Time ====
while true; do
timestamp=$(date +%Y%m%d-%H%M%S)
mtr --report --report-cycles 50 google.com > "mtr-$timestamp.txt"
sleep 3600
done

==== Monitor Multiple Destinations ====
#!/bin/bash
echo "=== MTR Report $(date) ===" > daily-network-report.txt
echo "" >> daily-network-report.txt

echo "Google DNS:" >> daily-network-report.txt
mtr --report --report-cycles 50 --no-dns 8.8.8.8 >> daily-network-report.txt
echo "" >> daily-network-report.txt

echo "Cloudflare DNS:" >> daily-network-report.txt
mtr --report --report-cycles 50 --no-dns 1.1.1.1 >> daily-network-report.txt
echo "" >> daily-network-report.txt

echo "Your Server:" >> daily-network-report.txt
mtr --report --report-cycles 50 your-server.com >> daily-network-report.txt

=== Useful Aliases for .bashrc ===
# Quick network path analysis
alias mtrreport='mtr --report --report-cycles 50 --no-dns'

# Monitor connection to Google DNS
alias netcheck='mtr --report-cycles 20 8.8.8.8'

# Extended network test
alias mtrlong='mtr --report-cycles 100'

# TCP port 443 test (HTTPS)
alias mtrhttps='sudo mtr --tcp --port 443 --report-cycles 30'

# Quick comparison of major DNS providers
alias dnstest='echo "Google:" && mtr --report-cycles 20 8.8.8.8 && echo -e "\nCloudflare:" && mtr --report-cycles 20 1.1.1.1'
After adding to ~/.bashrc, reload with:
source ~/.bashrc

=== Troubleshooting Tips ===
'''Permission issues with TCP mode:'''
# Use sudo for TCP on privileged ports
sudo mtr --tcp --port 443 example.com

# Or set capabilities (one-time setup)
sudo setcap cap_net_raw+ep /usr/bin/mtr-packet
'''False positives:''' Packet loss at an intermediate hop but not at the destination is almost always ICMP rate limiting at that router, not a real problem. Example:
3.|-- router.isp.net 20.0% 50 15.3 16.1 14.2 19.8 1.8
4.|-- next-hop.net 0.0% 50 18.5 19.2 17.8 22.1 1.3
5.|-- destination.com 0.0% 50 25.4 26.8 24.1 32.5 2.4
Hop 3 shows 20% loss, but hops 4 and 5 show 0%. Traffic is flowing normally.

'''DNS resolution delays:''' If MTR is slow to start, skip DNS resolution with mtr --no-dns target.com and resolve addresses afterward with host 203.0.113.1 if needed.

'''Comparing results:'''
mtr --report --report-cycles 50 example.com > test1.txt
sleep 60
mtr --report --report-cycles 50 example.com > test2.txt
diff test1.txt test2.txt

=== When to Use MTR vs Other Tools ===
'''MTR''' — Best for continuous monitoring, identifying problem hops, and gathering detailed statistics. Requires installation via apt.

'''ping''' — Best for a quick connectivity test or simple latency check. Only tests the endpoint, not the path.

'''traceroute''' — Best for one-time path discovery. Provides no continuous monitoring.

'''ss / netstat''' — Best for checking local connection status. Doesn’t test remote paths.

=== Best Practices ===
Run at least 50–100 cycles for accurate statistics. Use --no-dns for faster results and to avoid DNS resolution issues during testing. Test at different times of day, since network conditions vary. If one protocol shows issues, compare ICMP, UDP, and TCP modes. Save reports with timestamps for trend analysis. Use 8.8.8.8 or 1.1.1.1 as known-good hosts to verify your local network first. Let MTR run for at least 30–60 seconds before drawing conclusions.

=== Reading Between the Lines ===
'''Good network health:'''
HOST: hostname Loss% Snt Last Avg Best Wrst StDev
1.|-- 192.168.1.1 0.0% 50 1.1 1.2 0.9 2.1 0.2
2.|-- 10.0.0.1 0.0% 50 8.2 8.5 7.5 10.2 0.5
3.|-- isp-gateway.net 0.0% 50 15.1 15.5 14.0 18.3 0.8
4.|-- google.com 0.0% 50 24.8 25.2 23.5 28.1 1.1
No packet loss, consistent latency, low jitter throughout.

'''Problematic network:'''
HOST: hostname Loss% Snt Last Avg Best Wrst StDev
1.|-- 192.168.1.1 0.0% 50 1.2 1.5 1.0 2.3 0.3
2.|-- 10.0.0.1 5.0% 50 45.3 52.8 8.1 245.2 45.7
3.|-- ??? 100.0% 50 0.0 0.0 0.0 0.0 0.0
4.|-- destination.com 15.0% 50 95.4 125.8 48.2 385.5 78.2
Packet loss at hop 2 and the destination, high jitter, and a very high worst-case latency all point to a genuine problem at or near hop 2.

=== Summary ===
MTR is your Swiss Army knife for network diagnostics. Use --report-cycles 50 or higher for reliable data. Watch for packet loss at the destination—intermediate losses may be false positives. High StDev indicates an unstable connection. High average latency points to slow links. Use --no-dns for faster results. Compare ICMP, UDP, and TCP modes if issues appear. Test at different times of day for a complete picture.
----

== 5. Package Management Issues ==

=== Fix Broken Packages ===
# Update package lists
sudo apt update

# Fix broken dependencies
sudo apt --fix-broken install

# Reconfigure packages
sudo dpkg --configure -a
# No output means there is nothing to reconfigure

# Clean package cache
sudo apt clean
sudo apt autoclean

# Remove unused packages
sudo apt autoremove

=== Handle Held or Locked Packages ===
# If apt is locked, find the process holding the lock
sudo lsof /var/lib/dpkg/lock-frontend

# Remove the lock (use with caution—only if no package manager is running)
sudo rm /var/lib/dpkg/lock-frontend
sudo rm /var/lib/apt/lists/lock

# Reconfigure dpkg
sudo dpkg --configure -a
Disk & Filesystem Issues

=== Check Disk Health ===
# Check disk space
df -h

# Check inode usage
df -i

# View disk I/O statistics
# iostat is part of the sysstat package, which also includes:
# mpstat (CPU statistics), sar (system activity reporter),
# pidstat (process statistics), and cifsiostat (CIFS statistics)
sudo apt install sysstat

iostat -x 1

# Show stats in MB instead of KB
iostat -xm 2

# Monitor a specific device
iostat -x sda 1

# Check for disk errors in dmesg
sudo dmesg | grep -i "error\|fail"

# SMART disk health (requires smartmontools)
sudo apt install smartmontools
sudo smartctl -a /dev/sda

=== Repair Filesystem ===
# Unmount the partition first
sudo umount /dev/sdXN

# Run filesystem check
sudo fsck /dev/sdXN

# For ext4 specifically
sudo e2fsck -f /dev/sdXN
Performance Issues

=== Identify Resource Hogs ===
# CPU usage by process
top -o %CPU

# Memory usage by process
top -o %MEM

# Disk usage by directory
du -sh /* | sort -h

# Find large files
find / -type f -size +100M 2>/dev/null

# Check running processes sorted by memory use
ps aux --sort=-%mem | head -20

=== System Temperature Monitoring ===
# Install lm-sensors if not already present
sudo apt install lm-sensors
sudo sensors-detect

# View temperatures
sensors

# Real-time temperature monitoring
watch -n 2 sensors
See section 12 for a shell alias that makes temperature monitoring more convenient.

Service & Application Errors

=== Debug Service Problems ===
# Check service status
sudo systemctl status service-name

# View service logs
sudo journalctl -u service-name

# Restart a service
sudo systemctl restart service-name

# Enable service at boot
sudo systemctl enable service-name

# View recent service failures
journalctl -p err -b

=== Application Crash Investigation ===
# Check for core dumps
ls -lh /var/crash/

# View available application logs
ls /var/log/

# Follow syslog for application errors in real-time
sudo tail -f /var/log/syslog
Permission & Access Issues

=== Fix Common Permission Problems ===
# Check file ownership
ls -l /path/to/file

# Change ownership (user:group)
sudo chown user:group /path/to/file

# Change permissions
sudo chmod 644 /path/to/file

# Recursively fix permissions on a directory
sudo chown -R user:group /path/to/directory

=== User & Authentication Issues ===
# Check user information
id username

# View user login history
last -a

# Check failed login attempts
sudo journalctl | grep "authentication failure"

# Reset user password
sudo passwd username
Hardware Issues

=== Identify Hardware ===
# List all hardware (install if needed: sudo apt install lshw)
sudo lshw -short

# PCI devices
lspci -v

# USB devices
lsusb -v

# CPU information
lscpu

# Memory information
sudo dmidecode --type memory

=== Check Hardware Errors ===
# Kernel ring buffer (hardware messages)
dmesg | less
# Press q to quit

# Search for specific hardware issues
dmesg | grep -i "error\|fail\|warn"

# Check for USB issues
dmesg | grep -i usb
Quick Diagnostic Commands

=== System Information at a Glance ===
# Uptime and load average
uptime

# Kernel version
uname -r

# Debian version
cat /etc/debian_version

# Full system summary (install if needed: sudo apt install inxi)
sudo inxi -Fxz

=== Emergency Toolkit ===
# Create a consolidated diagnostic report
sudo journalctl -b > ~/system-report.txt
dmesg >> ~/system-report.txt
systemctl --failed >> ~/system-report.txt
df -h >> ~/system-report.txt
free -h >> ~/system-report.txt

# Watch logs in real-time
sudo journalctl -f

# Monitor system resources continuously
watch -n 1 'free -h && df -h'
Useful Aliases & Shortcuts
Add these to your ~/.bashrc for quick access to common troubleshooting commands:
# Monitor system temperatures in real-time
alias temps="watch -n 2 'for i in /sys/class/hwmon/hwmon*/; do echo -n \"\$(cat \${i}name): \"; cat \${i}temp*_input 2>/dev/null | while read temp; do echo \"scale=1; \$temp/1000\" | bc; done | tr \"\n\" \" \"; echo \"°C\"; done'"

# Quick system status
alias sysstat='echo "=== CPU ===" && uptime && echo -e "\n=== Memory ===" && free -h && echo -e "\n=== Disk ===" && df -h && echo -e "\n=== Top Processes ===" && ps aux --sort=-%mem | head -10'

# View last boot logs
alias lastboot='journalctl -b -1'

# Check failed services
alias failedservices='systemctl --failed'

# Monitor logs in real-time
alias watchlog='sudo journalctl -f'

# Quick network status
alias netstatus='ip addr show && echo -e "\n=== Routes ===" && ip route show'
After adding these, reload your shell configuration:
source ~/.bashrc
Tips for Effective Troubleshooting

'''Check logs first.''' journalctl and dmesg are your best starting point for almost any problem.

'''Change one thing at a time.''' It’s tempting to make several changes at once, but doing so makes it impossible to know what actually fixed—or broke—the problem.

'''Document your changes.''' Keep notes on what you’ve tried and what the results were. You’ll thank yourself later.

'''Use exact error messages.''' Copy error messages verbatim and search for them alongside “Debian” and the version number—for example, “Debian 13” or “Debian 13.1” for point releases.

'''Check what changed recently.''' Did you install something, update packages, or change the kernel before the problem appeared? Start there.

'''Back up before making major changes.''' This goes without saying, but it’s easy to skip when you’re in a hurry.

'''Use verbose mode.''' Adding -v or -vv flags to commands often reveals what would otherwise be silent failures.

'''Use the community.''' The Debian forums, Reddit, Stack Exchange, and the Debian mailing lists are all active and well-indexed. Someone has usually seen your problem before.

''This work was contributed by <dfn>distro-nix</dfn> to the Debian User Forums on 2025-10-27.''

Debian First Aid Kit

2026-05-14T20:45:49Z

Distro-nix:

'''Author:''' distro-nix '''Forum Profile:''' <dfn><nowiki>https://forums.debian.net/memberlist.php?mode=viewprofile&u=239099</nowiki></dfn> '''Debian Version:''' Debian 13 (Trixie) '''Kernel Versions Tested:''' 6.12.73+deb13-amd64 and 6.16.3+deb13-amd64 (x86_64) '''Created:''' 2025-10-27 '''Last Updated:''' 2026-03-11 '''ID:''' 544201 '''License:''' CC BY-SA 4.0

''This guide is preserved in its original form. Questions and feedback welcome via the author’s <dfn>Debian Forums profile</dfn>.''

== Abstract ==
The Debian First Aid Kit is a practical reference for diagnosing and resolving common system problems on Debian GNU/Linux. It covers commonly occuring categories of failure: system freezes and crashes, boot problems, network issues, package management, disk and filesystem errors, performance degradation, service and application failures, permission and access problems, hardware identification and errors, and quick diagnostics. Each section gives the relevant commands so you can understand what they do and why, rather than a bare list to copy and paste blindly. A dedicated section covers MTR (Matt’s Traceroute) in depth, including output interpretation, practical troubleshooting, and how to monitor continuously. The guide also includes a set of shell aliases for faster access to common diagnostic tasks. All commands are verified on Debian 13 (Trixie) running kernel 6.12.73+deb13-amd64 and 6.16.3+deb13-amd64. This document is intended for users with basic command-line familiarity who need a reliable, consolidated reference when something goes wrong.

'''Keywords:''' Debian, troubleshooting, system administration, MTR, network diagnostics, package management, journalctl, filesystem repair
----

== Tested Commands Disclaimer ==
All commands in this document have been verified on Debian 13 (Trixie) with the kernel versions listed above. Results may vary on other releases or hardware configurations. Always test in a safe environment before applying any procedure to a production system.

== Conventions Used in This Document ==
Command examples throughout this guide follow standard Unix prompt conventions. A $ prefix indicates a command run as a regular user. Where elevated privileges are required, sudo is shown explicitly and # with a brief explanation if needed:
$ journalctl -b -1 # regular user
$ sudo apt install mtr # requires elevation via sudo
In recovery shell contexts, where you are operating directly as root, the prompt is #.

== System Freezes & Crashes ==

=== Check System Logs ===

=== Common Freeze Causes to Look For ===

* '''Kernel panics:''' Search for “kernel panic” or “Oops”
* '''Out of Memory (OOM):''' Search for “Out of memory” or “oom-killer”
* '''Hardware errors:''' Look for “MCE” (Machine Check Exception) or “hardware error”
* '''Driver issues:''' Check for module/driver failures
* '''Overheating:''' Check system temperatures

=== Check System Resources ===
$ free -h # memory usage

$ df -h # disk space

$ htop # interactive resource monitor

$ top # alternative resource monitor

$ sudo apt install btop # btop is a more detailed alternative

$ sudo dmesg | grep -i error # check for disk errors

=== Access Recovery Mode ===
Reboot and hold Shift to access the GRUB menu (timing depends on your GRUB configuration). Select “Advanced options”, then choose recovery mode. From the recovery menu, select “root” for a root shell.

=== Common Boot Fixes ===
$ sudo fsck /dev/sdXN # repair filesystem — identify device with lsblk first

$ sudo grub-install /dev/sdX # reinstall GRUB bootloader

$ sudo update-grub # regenerate GRUB configuration

$ cat /etc/fstab # check fstab for mount errors

=== The following are permanent errors caused by incomplete or buggy ACPI tables in the BIOS. They are harmless and can be safely ignored if they appear on your system: ===
[0.686554] ACPI Error: No handler for Region [ECRM] (00000000201accc4) [EmbeddedControl] (20250404/evregion-131)
[0.686577] ACPI Error: Region EmbeddedControl (ID=3) has no handler (20250404/exfldio-261)
[0.686594] ACPI Error: Aborting method \_SB.GPIO._EVT due to previous error (AE_NOT_EXIST) (20250404/psparse-529)

== Boot Problems ==

=== Check Boot Process ===
$ systemd-analyze blame # boot time by unit

$ systemd-analyze blame # boot time by unit

$ systemctl --failed # units that failed during boot

$ systemctl status <service-name> # e.g. systemctl status NetworkManager.service

=== Access Recovery Mode ===
Reboot and hold Shift to access the GRUB menu (timing depends on your GRUB configuration). Select “Advanced options”, then choose recovery mode. From the recovery menu, select “root” for a root shell.

=== Common Boot Fixes ===
Repair filesystem errors — first identify the device with lsblk
sudo fsck /dev/sdXN

# Reinstall GRUB bootloader
sudo grub-install /dev/sdX
sudo update-grub

# Check fstab for mount errors
cat /etc/fstab

== Network Issues ==

=== Diagnose Network Connection ===
# Check network interfaces
ip addr show

# Test IPv4 connectivity
ping -c 4 8.8.8.8

# Test IPv6 connectivity
ping -c 6 2a00:1450:4007:809::200e

# Check DNS resolution
nslookup google.com

# View routing table
ip route show

# Check active connections
ss -tuln

=== Restart Network Service ===
# For systems with NetworkManager
sudo systemctl restart NetworkManager

# For systems with networking service
sudo systemctl restart networking

# Bring interface down and up
sudo ip link set eth0 down
sudo ip link set eth0 up

== MTR (Matt’s Traceroute) ==
MTR combines the functionality of ping and traceroute into a single real-time network diagnostic tool. It continuously monitors the path between your system and a destination, providing detailed statistics about latency and packet loss at each hop. When you need to demonstrate a network problem that is outside your control—to your hosting provider, for example—MTR gives you the evidence to do so.

=== Installation ===
sudo apt install mtr

=== Basic Usage ===
# Basic MTR (interactive mode)
mtr google.com

# Report mode (run 10 cycles and exit)
mtr --report google.com

# Specify number of pings
mtr --report-cycles 50 google.com

# Use TCP instead of ICMP
mtr --tcp google.com

# Use UDP
mtr --udp google.com

# No DNS resolution (faster, shows IPs only)
mtr --no-dns google.com

# Show both hostnames and IPs
mtr --show-ips google.com

== Understanding MTR Output ==

==== Sample Output ====
HOST: hostname Loss% Snt Last Avg Best Wrst StDev
1.|-- 192.168.1.1 0.0% 10 1.2 1.5 1.0 2.3 0.4
2.|-- 10.0.0.1 0.0% 10 8.5 9.2 7.8 12.1 1.3
3.|-- isp-gateway.net 0.0% 10 15.3 16.1 14.2 19.8 1.8
4.|-- ??? 100.0% 10 0.0 0.0 0.0 0.0 0.0
5.|-- google.com 0.0% 10 25.4 26.8 24.1 32.5 2.4

=== Column Meanings ===
'''HOST''' — Hostname or IP address of each hop in the route.

'''Loss%''' — Percentage of packets lost at this hop.

'''Snt''' — Number of packets sent to this hop.

'''Last''' — Latency of the most recent packet, in milliseconds.

'''Avg''' — Average latency across all packets sent, in milliseconds.

'''Best''' — Lowest latency recorded, in milliseconds.

'''Wrst''' — Highest latency recorded, in milliseconds.

'''StDev''' — Standard deviation of latency measurements—lower values indicate a more stable connection.

=== Interpreting Results ===
A healthy network shows 0% loss on all hops, stable latency with low StDev values, a gradual latency increase as hop count increases, and consistent response times.

==== Problem Indicators ====
'''1. High Packet Loss at a Specific Hop'''
5.|-- problem-router.net 25.0% 10 45.3 48.2 42.1 65.8 8.4
If loss continues to the destination, there is a real problem at that router. If loss appears only at that hop but not beyond, the router is likely rate-limiting ICMP responses—a false positive, not a real problem. The rule of thumb: if packet loss appears at hop N but hops N+1, N+2, and beyond show 0% loss, it’s almost certainly ICMP rate limiting.

'''2. High Latency at a Specific Hop'''
3.|-- slow-link.net 0.0% 10 150.3 155.2 148.1 165.8 5.4
This indicates a network bottleneck, a congested link, geographical distance (intercontinental hops), or slow routing equipment.

'''3. No Response (???)'''
4.|-- ??? 100.0% 10 0.0 0.0 0.0 0.0 0.0
The router at this hop is configured not to respond to ICMP or traceroute packets, or a firewall is blocking diagnostic packets. This is not necessarily a problem if later hops respond normally.

'''4. High Jitter (StDev)'''
6.|-- unstable.net 0.0% 10 35.3 52.8 28.1 95.2 24.7
A high StDev (24.7ms in this example) indicates inconsistent latency caused by network congestion or instability. This is particularly noticeable in real-time applications such as VoIP, gaming, and video calls.

'''5. Sudden Latency Spike'''
1.|-- 192.168.1.1 0.0% 10 1.2 1.5 1.0 2.3 0.4
2.|-- 10.0.0.1 0.0% 10 8.5 9.2 7.8 12.1 1.3
3.|-- problematic-hop.net 0.0% 10 180.5 185.2 178.1 195.8 6.4
4.|-- next-hop.net 0.0% 10 182.3 187.8 180.5 198.2 6.8
Hop 3 introduces approximately 170ms of additional latency (jumping from 9ms to 180ms). Everything from hop 3 onward carries that penalty.

=== Advanced Usage ===

==== Report Mode with Different Output Formats ====
# CSV format for logging and analysis
mtr --report --csv google.com > network-report.csv

# JSON output for parsing
mtr --report --json google.com

# XML format
mtr --report --xml google.com

# Wide report (no abbreviations)
mtr --report-wide google.com

==== Protocol Selection ====
# Use ICMP (default)
mtr google.com

# Use UDP
mtr --udp google.com

# Use TCP
mtr --tcp google.com

# Test a specific TCP port
sudo mtr --tcp --port 443 google.com
sudo mtr --tcp --port 22 remote-server.com

==== Timing and Duration ====
# Specify interval between pings (default 1 second)
mtr --interval 0.5 google.com

# Extended test with 100 cycles
mtr --report-cycles 100 google.com

# Continuous monitoring (Ctrl+C to stop)
mtr google.com

# Quick 10-cycle report
mtr --report-cycles 10 google.com

==== Advanced Options ====
# Show Autonomous System (AS) numbers
mtr --aslookup google.com

# Set maximum number of hops
mtr --max-ttl 20 google.com

# Set packet size
mtr --psize 1000 google.com

# Show both IP and hostname
mtr --show-ips google.com

# Specify source address (useful with multiple network interfaces)
mtr --address 192.168.1.100 google.com

# IPv4 only
mtr -4 google.com

# IPv6 only
mtr -6 google.com

=== Interactive Mode Commands ===
When running MTR in interactive mode (mtr hostname), the following keys are available:

'''h''' — Display help.

'''d''' — Toggle display mode, cycling through different views.

'''n''' — Toggle between hostnames and IP addresses.

'''r''' — Reset all statistics.

'''p''' — Pause/unpause the display.

'''q''' — Quit MTR.

'''u''' — Switch between ICMP, UDP, and TCP modes.

'''y''' — Switch between IPv4 and IPv6.

'''o''' — Toggle field display options.

'''j''' — Toggle latency display.

=== Practical Troubleshooting Scenarios ===

==== Scenario 1: Diagnosing a Slow Website ====
mtr --report-cycles 100 --no-dns example.com
Look for high average latency at specific hops, packet loss at the destination, and high StDev values indicating jitter.

==== Scenario 2: Testing if a Firewall Blocks SSH ====
sudo mtr --tcp --port 22 --report-cycles 50 remote-server.com
If the last hop shows 100% loss but earlier hops are fine, port 22 may be filtered. Run a standard ICMP test for comparison.

==== Scenario 3: ISP Performance Issues ====
mtr --report-cycles 100 8.8.8.8
mtr --report-cycles 100 1.1.1.1
If issues appear in the first three or four hops, the problem is likely within your ISP’s network. If issues appear later, the problem lies beyond your ISP.

==== Scenario 4: VPN Troubleshooting ====
# Test before connecting to VPN
mtr --report-cycles 50 --no-dns google.com > before-vpn.txt

# Test after connecting to VPN
mtr --report-cycles 50 --no-dns google.com > after-vpn.txt

# Compare the two
diff before-vpn.txt after-vpn.txt

==== Scenario 5: Gaming/Streaming Performance ====
mtr --report-cycles 200 game-server.com
For real-time applications, look for low average latency (under 50ms for gaming), low StDev (under 5ms preferred), and zero packet loss.

==== Scenario 6: Intermittent Connectivity ====
# Long-running test to catch intermittent issues
mtr --report-cycles 500 --interval 1 target.com > long-test.txt

# Or monitor in real-time for several minutes
mtr target.com

=== Continuous Monitoring ===

==== Log Network Performance Over Time ====
while true; do
timestamp=$(date +%Y%m%d-%H%M%S)
mtr --report --report-cycles 50 google.com > "mtr-$timestamp.txt"
sleep 3600
done

==== Monitor Multiple Destinations ====
#!/bin/bash
echo "=== MTR Report $(date) ===" > daily-network-report.txt
echo "" >> daily-network-report.txt

echo "Google DNS:" >> daily-network-report.txt
mtr --report --report-cycles 50 --no-dns 8.8.8.8 >> daily-network-report.txt
echo "" >> daily-network-report.txt

echo "Cloudflare DNS:" >> daily-network-report.txt
mtr --report --report-cycles 50 --no-dns 1.1.1.1 >> daily-network-report.txt
echo "" >> daily-network-report.txt

echo "Your Server:" >> daily-network-report.txt
mtr --report --report-cycles 50 your-server.com >> daily-network-report.txt

=== Useful Aliases for .bashrc ===
# Quick network path analysis
alias mtrreport='mtr --report --report-cycles 50 --no-dns'

# Monitor connection to Google DNS
alias netcheck='mtr --report-cycles 20 8.8.8.8'

# Extended network test
alias mtrlong='mtr --report-cycles 100'

# TCP port 443 test (HTTPS)
alias mtrhttps='sudo mtr --tcp --port 443 --report-cycles 30'

# Quick comparison of major DNS providers
alias dnstest='echo "Google:" && mtr --report-cycles 20 8.8.8.8 && echo -e "\nCloudflare:" && mtr --report-cycles 20 1.1.1.1'
After adding to ~/.bashrc, reload with:
source ~/.bashrc

=== Troubleshooting Tips ===
'''Permission issues with TCP mode:'''
# Use sudo for TCP on privileged ports
sudo mtr --tcp --port 443 example.com

# Or set capabilities (one-time setup)
sudo setcap cap_net_raw+ep /usr/bin/mtr-packet
'''False positives:''' Packet loss at an intermediate hop but not at the destination is almost always ICMP rate limiting at that router, not a real problem. Example:
3.|-- router.isp.net 20.0% 50 15.3 16.1 14.2 19.8 1.8
4.|-- next-hop.net 0.0% 50 18.5 19.2 17.8 22.1 1.3
5.|-- destination.com 0.0% 50 25.4 26.8 24.1 32.5 2.4
Hop 3 shows 20% loss, but hops 4 and 5 show 0%. Traffic is flowing normally.

'''DNS resolution delays:''' If MTR is slow to start, skip DNS resolution with mtr --no-dns target.com and resolve addresses afterward with host 203.0.113.1 if needed.

'''Comparing results:'''
mtr --report --report-cycles 50 example.com > test1.txt
sleep 60
mtr --report --report-cycles 50 example.com > test2.txt
diff test1.txt test2.txt

=== When to Use MTR vs Other Tools ===
'''MTR''' — Best for continuous monitoring, identifying problem hops, and gathering detailed statistics. Requires installation via apt.

'''ping''' — Best for a quick connectivity test or simple latency check. Only tests the endpoint, not the path.

'''traceroute''' — Best for one-time path discovery. Provides no continuous monitoring.

'''ss / netstat''' — Best for checking local connection status. Doesn’t test remote paths.

=== Best Practices ===
Run at least 50–100 cycles for accurate statistics. Use --no-dns for faster results and to avoid DNS resolution issues during testing. Test at different times of day, since network conditions vary. If one protocol shows issues, compare ICMP, UDP, and TCP modes. Save reports with timestamps for trend analysis. Use 8.8.8.8 or 1.1.1.1 as known-good hosts to verify your local network first. Let MTR run for at least 30–60 seconds before drawing conclusions.

=== Reading Between the Lines ===
'''Good network health:'''
HOST: hostname Loss% Snt Last Avg Best Wrst StDev
1.|-- 192.168.1.1 0.0% 50 1.1 1.2 0.9 2.1 0.2
2.|-- 10.0.0.1 0.0% 50 8.2 8.5 7.5 10.2 0.5
3.|-- isp-gateway.net 0.0% 50 15.1 15.5 14.0 18.3 0.8
4.|-- google.com 0.0% 50 24.8 25.2 23.5 28.1 1.1
No packet loss, consistent latency, low jitter throughout.

'''Problematic network:'''
HOST: hostname Loss% Snt Last Avg Best Wrst StDev
1.|-- 192.168.1.1 0.0% 50 1.2 1.5 1.0 2.3 0.3
2.|-- 10.0.0.1 5.0% 50 45.3 52.8 8.1 245.2 45.7
3.|-- ??? 100.0% 50 0.0 0.0 0.0 0.0 0.0
4.|-- destination.com 15.0% 50 95.4 125.8 48.2 385.5 78.2
Packet loss at hop 2 and the destination, high jitter, and a very high worst-case latency all point to a genuine problem at or near hop 2.

=== Summary ===
MTR is your Swiss Army knife for network diagnostics. Use --report-cycles 50 or higher for reliable data. Watch for packet loss at the destination—intermediate losses may be false positives. High StDev indicates an unstable connection. High average latency points to slow links. Use --no-dns for faster results. Compare ICMP, UDP, and TCP modes if issues appear. Test at different times of day for a complete picture.
----

== 5. Package Management Issues ==

=== Fix Broken Packages ===
# Update package lists
sudo apt update

# Fix broken dependencies
sudo apt --fix-broken install

# Reconfigure packages
sudo dpkg --configure -a
# No output means there is nothing to reconfigure

# Clean package cache
sudo apt clean
sudo apt autoclean

# Remove unused packages
sudo apt autoremove

=== Handle Held or Locked Packages ===
# If apt is locked, find the process holding the lock
sudo lsof /var/lib/dpkg/lock-frontend

# Remove the lock (use with caution—only if no package manager is running)
sudo rm /var/lib/dpkg/lock-frontend
sudo rm /var/lib/apt/lists/lock

# Reconfigure dpkg
sudo dpkg --configure -a
Disk & Filesystem Issues

=== Check Disk Health ===
# Check disk space
df -h

# Check inode usage
df -i

# View disk I/O statistics
# iostat is part of the sysstat package, which also includes:
# mpstat (CPU statistics), sar (system activity reporter),
# pidstat (process statistics), and cifsiostat (CIFS statistics)
sudo apt install sysstat

iostat -x 1

# Show stats in MB instead of KB
iostat -xm 2

# Monitor a specific device
iostat -x sda 1

# Check for disk errors in dmesg
sudo dmesg | grep -i "error\|fail"

# SMART disk health (requires smartmontools)
sudo apt install smartmontools
sudo smartctl -a /dev/sda

=== Repair Filesystem ===
# Unmount the partition first
sudo umount /dev/sdXN

# Run filesystem check
sudo fsck /dev/sdXN

# For ext4 specifically
sudo e2fsck -f /dev/sdXN
Performance Issues

=== Identify Resource Hogs ===
# CPU usage by process
top -o %CPU

# Memory usage by process
top -o %MEM

# Disk usage by directory
du -sh /* | sort -h

# Find large files
find / -type f -size +100M 2>/dev/null

# Check running processes sorted by memory use
ps aux --sort=-%mem | head -20

=== System Temperature Monitoring ===
# Install lm-sensors if not already present
sudo apt install lm-sensors
sudo sensors-detect

# View temperatures
sensors

# Real-time temperature monitoring
watch -n 2 sensors
See section 12 for a shell alias that makes temperature monitoring more convenient.

Service & Application Errors

=== Debug Service Problems ===
# Check service status
sudo systemctl status service-name

# View service logs
sudo journalctl -u service-name

# Restart a service
sudo systemctl restart service-name

# Enable service at boot
sudo systemctl enable service-name

# View recent service failures
journalctl -p err -b

=== Application Crash Investigation ===
# Check for core dumps
ls -lh /var/crash/

# View available application logs
ls /var/log/

# Follow syslog for application errors in real-time
sudo tail -f /var/log/syslog
Permission & Access Issues

=== Fix Common Permission Problems ===
# Check file ownership
ls -l /path/to/file

# Change ownership (user:group)
sudo chown user:group /path/to/file

# Change permissions
sudo chmod 644 /path/to/file

# Recursively fix permissions on a directory
sudo chown -R user:group /path/to/directory

=== User & Authentication Issues ===
# Check user information
id username

# View user login history
last -a

# Check failed login attempts
sudo journalctl | grep "authentication failure"

# Reset user password
sudo passwd username
Hardware Issues

=== Identify Hardware ===
# List all hardware (install if needed: sudo apt install lshw)
sudo lshw -short

# PCI devices
lspci -v

# USB devices
lsusb -v

# CPU information
lscpu

# Memory information
sudo dmidecode --type memory

=== Check Hardware Errors ===
# Kernel ring buffer (hardware messages)
dmesg | less
# Press q to quit

# Search for specific hardware issues
dmesg | grep -i "error\|fail\|warn"

# Check for USB issues
dmesg | grep -i usb
Quick Diagnostic Commands

=== System Information at a Glance ===
# Uptime and load average
uptime

# Kernel version
uname -r

# Debian version
cat /etc/debian_version

# Full system summary (install if needed: sudo apt install inxi)
sudo inxi -Fxz

=== Emergency Toolkit ===
# Create a consolidated diagnostic report
sudo journalctl -b > ~/system-report.txt
dmesg >> ~/system-report.txt
systemctl --failed >> ~/system-report.txt
df -h >> ~/system-report.txt
free -h >> ~/system-report.txt

# Watch logs in real-time
sudo journalctl -f

# Monitor system resources continuously
watch -n 1 'free -h && df -h'
Useful Aliases & Shortcuts
Add these to your ~/.bashrc for quick access to common troubleshooting commands:
# Monitor system temperatures in real-time
alias temps="watch -n 2 'for i in /sys/class/hwmon/hwmon*/; do echo -n \"\$(cat \${i}name): \"; cat \${i}temp*_input 2>/dev/null | while read temp; do echo \"scale=1; \$temp/1000\" | bc; done | tr \"\n\" \" \"; echo \"°C\"; done'"

# Quick system status
alias sysstat='echo "=== CPU ===" && uptime && echo -e "\n=== Memory ===" && free -h && echo -e "\n=== Disk ===" && df -h && echo -e "\n=== Top Processes ===" && ps aux --sort=-%mem | head -10'

# View last boot logs
alias lastboot='journalctl -b -1'

# Check failed services
alias failedservices='systemctl --failed'

# Monitor logs in real-time
alias watchlog='sudo journalctl -f'

# Quick network status
alias netstatus='ip addr show && echo -e "\n=== Routes ===" && ip route show'
After adding these, reload your shell configuration:
source ~/.bashrc
Tips for Effective Troubleshooting

'''Check logs first.''' journalctl and dmesg are your best starting point for almost any problem.

'''Change one thing at a time.''' It’s tempting to make several changes at once, but doing so makes it impossible to know what actually fixed—or broke—the problem.

'''Document your changes.''' Keep notes on what you’ve tried and what the results were. You’ll thank yourself later.

'''Use exact error messages.''' Copy error messages verbatim and search for them alongside “Debian” and the version number—for example, “Debian 13” or “Debian 13.1” for point releases.

'''Check what changed recently.''' Did you install something, update packages, or change the kernel before the problem appeared? Start there.

'''Back up before making major changes.''' This goes without saying, but it’s easy to skip when you’re in a hurry.

'''Use verbose mode.''' Adding -v or -vv flags to commands often reveals what would otherwise be silent failures.

'''Use the community.''' The Debian forums, Reddit, Stack Exchange, and the Debian mailing lists are all active and well-indexed. Someone has usually seen your problem before.

''This work was contributed by <dfn>distro-nix</dfn> to the Debian User Forums on 2025-10-27.''

Debian First Aid Kit

2026-05-14T20:13:18Z

Distro-nix: Pasting preformatted code blocks

'''Author:''' distro-nix '''Forum Profile:''' <dfn><nowiki>https://forums.debian.net/memberlist.php?mode=viewprofile&u=239099</nowiki></dfn> '''Debian Version:''' Debian 13 (Trixie) '''Kernel Versions Tested:''' 6.12.73+deb13-amd64 and 6.16.3+deb13-amd64 (x86_64) '''Created:''' 2025-10-27 '''Last Updated:''' 2026-03-11 '''ID:''' 544000.5 '''License:''' CC BY-SA 4.0

''This guide is preserved in its original form. Questions and feedback welcome via the author’s <dfn>Debian Forums profile</dfn>.''

== Abstract ==
The Debian First Aid Kit is a practical reference for diagnosing and resolving common system problems on Debian GNU/Linux. It covers commonly occuring categories of failure: system freezes and crashes, boot problems, network issues, package management, disk and filesystem errors, performance degradation, service and application failures, permission and access problems, hardware identification and errors, and quick diagnostics. Each section gives the relevant commands so you can understand what they do and why, rather than a bare list to copy and paste blindly. A dedicated section covers MTR (Matt’s Traceroute) in depth, including output interpretation, practical troubleshooting, and how to monitor continuously. The guide also includes a set of shell aliases for faster access to common diagnostic tasks. All commands are verified on Debian 13 (Trixie) running kernel 6.12.73+deb13-amd64 and 6.16.3+deb13-amd64. This document is intended for users with basic command-line familiarity who need a reliable, consolidated reference when something goes wrong.

'''Keywords:''' Debian, troubleshooting, system administration, MTR, network diagnostics, package management, journalctl, filesystem repair
----

== Tested Commands Disclaimer ==
All commands in this document have been verified on Debian 13 (Trixie) with the kernel versions listed above. Results may vary on other releases or hardware configurations. Always test in a safe environment before applying any procedure to a production system.

== Conventions Used in This Document ==
Command examples throughout this guide follow standard Unix prompt conventions. A $ prefix indicates a command run as a regular user. Where elevated privileges are required, sudo is shown explicitly and # with a brief explanation if needed:
$ journalctl -b -1 # regular user
$ sudo apt install mtr # requires elevation via sudo
In recovery shell contexts, where you are operating directly as root, the prompt is #.

== System Freezes & Crashes ==

=== Check System Logs ===

=== Common Freeze Causes to Look For ===

* '''Kernel panics:''' Search for “kernel panic” or “Oops”
* '''Out of Memory (OOM):''' Search for “Out of memory” or “oom-killer”
* '''Hardware errors:''' Look for “MCE” (Machine Check Exception) or “hardware error”
* '''Driver issues:''' Check for module/driver failures
* '''Overheating:''' Check system temperatures

=== Check System Resources ===
$ free -h # memory usage

$ df -h # disk space

$ htop # interactive resource monitor

$ top # alternative resource monitor

$ sudo apt install btop # btop is a more detailed alternative

$ sudo dmesg | grep -i error # check for disk errors

=== Access Recovery Mode ===
Reboot and hold Shift to access the GRUB menu (timing depends on your GRUB configuration). Select “Advanced options”, then choose recovery mode. From the recovery menu, select “root” for a root shell.

=== Common Boot Fixes ===
$ sudo fsck /dev/sdXN # repair filesystem — identify device with lsblk first

$ sudo grub-install /dev/sdX # reinstall GRUB bootloader

$ sudo update-grub # regenerate GRUB configuration

$ cat /etc/fstab # check fstab for mount errors

=== The following are permanent errors caused by incomplete or buggy ACPI tables in the BIOS. They are harmless and can be safely ignored if they appear on your system: ===
[0.686554] ACPI Error: No handler for Region [ECRM] (00000000201accc4) [EmbeddedControl] (20250404/evregion-131)
[0.686577] ACPI Error: Region EmbeddedControl (ID=3) has no handler (20250404/exfldio-261)
[0.686594] ACPI Error: Aborting method \_SB.GPIO._EVT due to previous error (AE_NOT_EXIST) (20250404/psparse-529)

== Boot Problems ==

=== Check Boot Process ===
$ systemd-analyze blame # boot time by unit

$ systemd-analyze blame # boot time by unit

$ systemctl --failed # units that failed during boot

$ systemctl status <service-name> # e.g. systemctl status NetworkManager.service

=== Access Recovery Mode ===
Reboot and hold Shift to access the GRUB menu (timing depends on your GRUB configuration). Select “Advanced options”, then choose recovery mode. From the recovery menu, select “root” for a root shell.

=== Common Boot Fixes ===
Repair filesystem errors — first identify the device with lsblk
sudo fsck /dev/sdXN

# Reinstall GRUB bootloader
sudo grub-install /dev/sdX
sudo update-grub

# Check fstab for mount errors
cat /etc/fstab

== Network Issues ==

=== Diagnose Network Connection ===
# Check network interfaces
ip addr show

# Test IPv4 connectivity
ping -c 4 8.8.8.8

# Test IPv6 connectivity
ping -c 6 2a00:1450:4007:809::200e

# Check DNS resolution
nslookup google.com

# View routing table
ip route show

# Check active connections
ss -tuln

=== Restart Network Service ===
# For systems with NetworkManager
sudo systemctl restart NetworkManager

# For systems with networking service
sudo systemctl restart networking

# Bring interface down and up
sudo ip link set eth0 down
sudo ip link set eth0 up

== MTR (Matt’s Traceroute) ==
MTR combines the functionality of ping and traceroute into a single real-time network diagnostic tool. It continuously monitors the path between your system and a destination, providing detailed statistics about latency and packet loss at each hop. When you need to demonstrate a network problem that is outside your control—to your hosting provider, for example—MTR gives you the evidence to do so.

=== Installation ===
sudo apt install mtr

=== Basic Usage ===
# Basic MTR (interactive mode)
mtr google.com

# Report mode (run 10 cycles and exit)
mtr --report google.com

# Specify number of pings
mtr --report-cycles 50 google.com

# Use TCP instead of ICMP
mtr --tcp google.com

# Use UDP
mtr --udp google.com

# No DNS resolution (faster, shows IPs only)
mtr --no-dns google.com

# Show both hostnames and IPs
mtr --show-ips google.com

== Understanding MTR Output ==

==== Sample Output ====
HOST: hostname Loss% Snt Last Avg Best Wrst StDev
1.|-- 192.168.1.1 0.0% 10 1.2 1.5 1.0 2.3 0.4
2.|-- 10.0.0.1 0.0% 10 8.5 9.2 7.8 12.1 1.3
3.|-- isp-gateway.net 0.0% 10 15.3 16.1 14.2 19.8 1.8
4.|-- ??? 100.0% 10 0.0 0.0 0.0 0.0 0.0
5.|-- google.com 0.0% 10 25.4 26.8 24.1 32.5 2.4

=== Column Meanings ===
'''HOST''' — Hostname or IP address of each hop in the route.

'''Loss%''' — Percentage of packets lost at this hop.

'''Snt''' — Number of packets sent to this hop.

'''Last''' — Latency of the most recent packet, in milliseconds.

'''Avg''' — Average latency across all packets sent, in milliseconds.

'''Best''' — Lowest latency recorded, in milliseconds.

'''Wrst''' — Highest latency recorded, in milliseconds.

'''StDev''' — Standard deviation of latency measurements—lower values indicate a more stable connection.

=== Interpreting Results ===
A healthy network shows 0% loss on all hops, stable latency with low StDev values, a gradual latency increase as hop count increases, and consistent response times.

==== Problem Indicators ====
'''1. High Packet Loss at a Specific Hop'''
5.|-- problem-router.net 25.0% 10 45.3 48.2 42.1 65.8 8.4
If loss continues to the destination, there is a real problem at that router. If loss appears only at that hop but not beyond, the router is likely rate-limiting ICMP responses—a false positive, not a real problem. The rule of thumb: if packet loss appears at hop N but hops N+1, N+2, and beyond show 0% loss, it’s almost certainly ICMP rate limiting.

'''2. High Latency at a Specific Hop'''
3.|-- slow-link.net 0.0% 10 150.3 155.2 148.1 165.8 5.4
This indicates a network bottleneck, a congested link, geographical distance (intercontinental hops), or slow routing equipment.

'''3. No Response (???)'''
4.|-- ??? 100.0% 10 0.0 0.0 0.0 0.0 0.0
The router at this hop is configured not to respond to ICMP or traceroute packets, or a firewall is blocking diagnostic packets. This is not necessarily a problem if later hops respond normally.

'''4. High Jitter (StDev)'''
6.|-- unstable.net 0.0% 10 35.3 52.8 28.1 95.2 24.7
A high StDev (24.7ms in this example) indicates inconsistent latency caused by network congestion or instability. This is particularly noticeable in real-time applications such as VoIP, gaming, and video calls.

'''5. Sudden Latency Spike'''
1.|-- 192.168.1.1 0.0% 10 1.2 1.5 1.0 2.3 0.4
2.|-- 10.0.0.1 0.0% 10 8.5 9.2 7.8 12.1 1.3
3.|-- problematic-hop.net 0.0% 10 180.5 185.2 178.1 195.8 6.4
4.|-- next-hop.net 0.0% 10 182.3 187.8 180.5 198.2 6.8
Hop 3 introduces approximately 170ms of additional latency (jumping from 9ms to 180ms). Everything from hop 3 onward carries that penalty.

=== Advanced Usage ===

==== Report Mode with Different Output Formats ====
# CSV format for logging and analysis
mtr --report --csv google.com > network-report.csv

# JSON output for parsing
mtr --report --json google.com

# XML format
mtr --report --xml google.com

# Wide report (no abbreviations)
mtr --report-wide google.com

==== Protocol Selection ====
# Use ICMP (default)
mtr google.com

# Use UDP
mtr --udp google.com

# Use TCP
mtr --tcp google.com

# Test a specific TCP port
sudo mtr --tcp --port 443 google.com
sudo mtr --tcp --port 22 remote-server.com

==== Timing and Duration ====
# Specify interval between pings (default 1 second)
mtr --interval 0.5 google.com

# Extended test with 100 cycles
mtr --report-cycles 100 google.com

# Continuous monitoring (Ctrl+C to stop)
mtr google.com

# Quick 10-cycle report
mtr --report-cycles 10 google.com

==== Advanced Options ====
# Show Autonomous System (AS) numbers
mtr --aslookup google.com

# Set maximum number of hops
mtr --max-ttl 20 google.com

# Set packet size
mtr --psize 1000 google.com

# Show both IP and hostname
mtr --show-ips google.com

# Specify source address (useful with multiple network interfaces)
mtr --address 192.168.1.100 google.com

# IPv4 only
mtr -4 google.com

# IPv6 only
mtr -6 google.com

=== Interactive Mode Commands ===
When running MTR in interactive mode (mtr hostname), the following keys are available:

'''h''' — Display help.

'''d''' — Toggle display mode, cycling through different views.

'''n''' — Toggle between hostnames and IP addresses.

'''r''' — Reset all statistics.

'''p''' — Pause/unpause the display.

'''q''' — Quit MTR.

'''u''' — Switch between ICMP, UDP, and TCP modes.

'''y''' — Switch between IPv4 and IPv6.

'''o''' — Toggle field display options.

'''j''' — Toggle latency display.

=== Practical Troubleshooting Scenarios ===

==== Scenario 1: Diagnosing a Slow Website ====
mtr --report-cycles 100 --no-dns example.com
Look for high average latency at specific hops, packet loss at the destination, and high StDev values indicating jitter.

==== Scenario 2: Testing if a Firewall Blocks SSH ====
sudo mtr --tcp --port 22 --report-cycles 50 remote-server.com
If the last hop shows 100% loss but earlier hops are fine, port 22 may be filtered. Run a standard ICMP test for comparison.

==== Scenario 3: ISP Performance Issues ====
mtr --report-cycles 100 8.8.8.8
mtr --report-cycles 100 1.1.1.1
If issues appear in the first three or four hops, the problem is likely within your ISP’s network. If issues appear later, the problem lies beyond your ISP.

==== Scenario 4: VPN Troubleshooting ====
# Test before connecting to VPN
mtr --report-cycles 50 --no-dns google.com > before-vpn.txt

# Test after connecting to VPN
mtr --report-cycles 50 --no-dns google.com > after-vpn.txt

# Compare the two
diff before-vpn.txt after-vpn.txt

==== Scenario 5: Gaming/Streaming Performance ====
mtr --report-cycles 200 game-server.com
For real-time applications, look for low average latency (under 50ms for gaming), low StDev (under 5ms preferred), and zero packet loss.

==== Scenario 6: Intermittent Connectivity ====
# Long-running test to catch intermittent issues
mtr --report-cycles 500 --interval 1 target.com > long-test.txt

# Or monitor in real-time for several minutes
mtr target.com

=== Continuous Monitoring ===

==== Log Network Performance Over Time ====
while true; do
timestamp=$(date +%Y%m%d-%H%M%S)
mtr --report --report-cycles 50 google.com > "mtr-$timestamp.txt"
sleep 3600
done

==== Monitor Multiple Destinations ====
#!/bin/bash
echo "=== MTR Report $(date) ===" > daily-network-report.txt
echo "" >> daily-network-report.txt

echo "Google DNS:" >> daily-network-report.txt
mtr --report --report-cycles 50 --no-dns 8.8.8.8 >> daily-network-report.txt
echo "" >> daily-network-report.txt

echo "Cloudflare DNS:" >> daily-network-report.txt
mtr --report --report-cycles 50 --no-dns 1.1.1.1 >> daily-network-report.txt
echo "" >> daily-network-report.txt

echo "Your Server:" >> daily-network-report.txt
mtr --report --report-cycles 50 your-server.com >> daily-network-report.txt

=== Useful Aliases for .bashrc ===
# Quick network path analysis
alias mtrreport='mtr --report --report-cycles 50 --no-dns'

# Monitor connection to Google DNS
alias netcheck='mtr --report-cycles 20 8.8.8.8'

# Extended network test
alias mtrlong='mtr --report-cycles 100'

# TCP port 443 test (HTTPS)
alias mtrhttps='sudo mtr --tcp --port 443 --report-cycles 30'

# Quick comparison of major DNS providers
alias dnstest='echo "Google:" && mtr --report-cycles 20 8.8.8.8 && echo -e "\nCloudflare:" && mtr --report-cycles 20 1.1.1.1'
After adding to ~/.bashrc, reload with:
source ~/.bashrc

=== Troubleshooting Tips ===
'''Permission issues with TCP mode:'''
# Use sudo for TCP on privileged ports
sudo mtr --tcp --port 443 example.com

# Or set capabilities (one-time setup)
sudo setcap cap_net_raw+ep /usr/bin/mtr-packet
'''False positives:''' Packet loss at an intermediate hop but not at the destination is almost always ICMP rate limiting at that router, not a real problem. Example:
3.|-- router.isp.net 20.0% 50 15.3 16.1 14.2 19.8 1.8
4.|-- next-hop.net 0.0% 50 18.5 19.2 17.8 22.1 1.3
5.|-- destination.com 0.0% 50 25.4 26.8 24.1 32.5 2.4
Hop 3 shows 20% loss, but hops 4 and 5 show 0%. Traffic is flowing normally.

'''DNS resolution delays:''' If MTR is slow to start, skip DNS resolution with mtr --no-dns target.com and resolve addresses afterward with host 203.0.113.1 if needed.

'''Comparing results:'''
mtr --report --report-cycles 50 example.com > test1.txt
sleep 60
mtr --report --report-cycles 50 example.com > test2.txt
diff test1.txt test2.txt

=== When to Use MTR vs Other Tools ===
'''MTR''' — Best for continuous monitoring, identifying problem hops, and gathering detailed statistics. Requires installation via apt.

'''ping''' — Best for a quick connectivity test or simple latency check. Only tests the endpoint, not the path.

'''traceroute''' — Best for one-time path discovery. Provides no continuous monitoring.

'''ss / netstat''' — Best for checking local connection status. Doesn’t test remote paths.

=== Best Practices ===
Run at least 50–100 cycles for accurate statistics. Use --no-dns for faster results and to avoid DNS resolution issues during testing. Test at different times of day, since network conditions vary. If one protocol shows issues, compare ICMP, UDP, and TCP modes. Save reports with timestamps for trend analysis. Use 8.8.8.8 or 1.1.1.1 as known-good hosts to verify your local network first. Let MTR run for at least 30–60 seconds before drawing conclusions.

=== Reading Between the Lines ===
'''Good network health:'''
HOST: hostname Loss% Snt Last Avg Best Wrst StDev
1.|-- 192.168.1.1 0.0% 50 1.1 1.2 0.9 2.1 0.2
2.|-- 10.0.0.1 0.0% 50 8.2 8.5 7.5 10.2 0.5
3.|-- isp-gateway.net 0.0% 50 15.1 15.5 14.0 18.3 0.8
4.|-- google.com 0.0% 50 24.8 25.2 23.5 28.1 1.1
No packet loss, consistent latency, low jitter throughout.

'''Problematic network:'''
HOST: hostname Loss% Snt Last Avg Best Wrst StDev
1.|-- 192.168.1.1 0.0% 50 1.2 1.5 1.0 2.3 0.3
2.|-- 10.0.0.1 5.0% 50 45.3 52.8 8.1 245.2 45.7
3.|-- ??? 100.0% 50 0.0 0.0 0.0 0.0 0.0
4.|-- destination.com 15.0% 50 95.4 125.8 48.2 385.5 78.2
Packet loss at hop 2 and the destination, high jitter, and a very high worst-case latency all point to a genuine problem at or near hop 2.

=== Summary ===
MTR is your Swiss Army knife for network diagnostics. Use --report-cycles 50 or higher for reliable data. Watch for packet loss at the destination—intermediate losses may be false positives. High StDev indicates an unstable connection. High average latency points to slow links. Use --no-dns for faster results. Compare ICMP, UDP, and TCP modes if issues appear. Test at different times of day for a complete picture.
----

== 5. Package Management Issues ==

=== Fix Broken Packages ===
# Update package lists
sudo apt update

# Fix broken dependencies
sudo apt --fix-broken install

# Reconfigure packages
sudo dpkg --configure -a
# No output means there is nothing to reconfigure

# Clean package cache
sudo apt clean
sudo apt autoclean

# Remove unused packages
sudo apt autoremove

=== Handle Held or Locked Packages ===
# If apt is locked, find the process holding the lock
sudo lsof /var/lib/dpkg/lock-frontend

# Remove the lock (use with caution—only if no package manager is running)
sudo rm /var/lib/dpkg/lock-frontend
sudo rm /var/lib/apt/lists/lock

# Reconfigure dpkg
sudo dpkg --configure -a
Disk & Filesystem Issues

=== Check Disk Health ===
# Check disk space
df -h

# Check inode usage
df -i

# View disk I/O statistics
# iostat is part of the sysstat package, which also includes:
# mpstat (CPU statistics), sar (system activity reporter),
# pidstat (process statistics), and cifsiostat (CIFS statistics)
sudo apt install sysstat

iostat -x 1

# Show stats in MB instead of KB
iostat -xm 2

# Monitor a specific device
iostat -x sda 1

# Check for disk errors in dmesg
sudo dmesg | grep -i "error\|fail"

# SMART disk health (requires smartmontools)
sudo apt install smartmontools
sudo smartctl -a /dev/sda

=== Repair Filesystem ===
# Unmount the partition first
sudo umount /dev/sdXN

# Run filesystem check
sudo fsck /dev/sdXN

# For ext4 specifically
sudo e2fsck -f /dev/sdXN
Performance Issues

=== Identify Resource Hogs ===
# CPU usage by process
top -o %CPU

# Memory usage by process
top -o %MEM

# Disk usage by directory
du -sh /* | sort -h

# Find large files
find / -type f -size +100M 2>/dev/null

# Check running processes sorted by memory use
ps aux --sort=-%mem | head -20

=== System Temperature Monitoring ===
# Install lm-sensors if not already present
sudo apt install lm-sensors
sudo sensors-detect

# View temperatures
sensors

# Real-time temperature monitoring
watch -n 2 sensors
See section 12 for a shell alias that makes temperature monitoring more convenient.

Service & Application Errors

=== Debug Service Problems ===
# Check service status
sudo systemctl status service-name

# View service logs
sudo journalctl -u service-name

# Restart a service
sudo systemctl restart service-name

# Enable service at boot
sudo systemctl enable service-name

# View recent service failures
journalctl -p err -b

=== Application Crash Investigation ===
# Check for core dumps
ls -lh /var/crash/

# View available application logs
ls /var/log/

# Follow syslog for application errors in real-time
sudo tail -f /var/log/syslog
Permission & Access Issues

=== Fix Common Permission Problems ===
# Check file ownership
ls -l /path/to/file

# Change ownership (user:group)
sudo chown user:group /path/to/file

# Change permissions
sudo chmod 644 /path/to/file

# Recursively fix permissions on a directory
sudo chown -R user:group /path/to/directory

=== User & Authentication Issues ===
# Check user information
id username

# View user login history
last -a

# Check failed login attempts
sudo journalctl | grep "authentication failure"

# Reset user password
sudo passwd username
Hardware Issues

=== Identify Hardware ===
# List all hardware (install if needed: sudo apt install lshw)
sudo lshw -short

# PCI devices
lspci -v

# USB devices
lsusb -v

# CPU information
lscpu

# Memory information
sudo dmidecode --type memory

=== Check Hardware Errors ===
# Kernel ring buffer (hardware messages)
dmesg | less
# Press q to quit

# Search for specific hardware issues
dmesg | grep -i "error\|fail\|warn"

# Check for USB issues
dmesg | grep -i usb
Quick Diagnostic Commands

=== System Information at a Glance ===
# Uptime and load average
uptime

# Kernel version
uname -r

# Debian version
cat /etc/debian_version

# Full system summary (install if needed: sudo apt install inxi)
sudo inxi -Fxz

=== Emergency Toolkit ===
# Create a consolidated diagnostic report
sudo journalctl -b > ~/system-report.txt
dmesg >> ~/system-report.txt
systemctl --failed >> ~/system-report.txt
df -h >> ~/system-report.txt
free -h >> ~/system-report.txt

# Watch logs in real-time
sudo journalctl -f

# Monitor system resources continuously
watch -n 1 'free -h && df -h'
Useful Aliases & Shortcuts
Add these to your ~/.bashrc for quick access to common troubleshooting commands:
# Monitor system temperatures in real-time
alias temps="watch -n 2 'for i in /sys/class/hwmon/hwmon*/; do echo -n \"\$(cat \${i}name): \"; cat \${i}temp*_input 2>/dev/null | while read temp; do echo \"scale=1; \$temp/1000\" | bc; done | tr \"\n\" \" \"; echo \"°C\"; done'"

# Quick system status
alias sysstat='echo "=== CPU ===" && uptime && echo -e "\n=== Memory ===" && free -h && echo -e "\n=== Disk ===" && df -h && echo -e "\n=== Top Processes ===" && ps aux --sort=-%mem | head -10'

# View last boot logs
alias lastboot='journalctl -b -1'

# Check failed services
alias failedservices='systemctl --failed'

# Monitor logs in real-time
alias watchlog='sudo journalctl -f'

# Quick network status
alias netstatus='ip addr show && echo -e "\n=== Routes ===" && ip route show'
After adding these, reload your shell configuration:
source ~/.bashrc
Tips for Effective Troubleshooting

'''Check logs first.''' journalctl and dmesg are your best starting point for almost any problem.

'''Change one thing at a time.''' It’s tempting to make several changes at once, but doing so makes it impossible to know what actually fixed—or broke—the problem.

'''Document your changes.''' Keep notes on what you’ve tried and what the results were. You’ll thank yourself later.

'''Use exact error messages.''' Copy error messages verbatim and search for them alongside “Debian” and the version number—for example, “Debian 13” or “Debian 13.1” for point releases.

'''Check what changed recently.''' Did you install something, update packages, or change the kernel before the problem appeared? Start there.

'''Back up before making major changes.''' This goes without saying, but it’s easy to skip when you’re in a hurry.

'''Use verbose mode.''' Adding -v or -vv flags to commands often reveals what would otherwise be silent failures.

'''Use the community.''' The Debian forums, Reddit, Stack Exchange, and the Debian mailing lists are all active and well-indexed. Someone has usually seen your problem before.

''This work was contributed by <dfn>distro-nix</dfn> to the Debian User Forums on 2025-10-27.''

Debian First Aid Kit

2026-05-14T19:02:31Z

Distro-nix: Pasting updated sections

Debian First Aid Kit

2026-05-14T18:22:23Z

Distro-nix: Repasting new sections

Debian First Aid Kit

2026-05-14T17:05:04Z

Distro-nix: /* Debian First Aid Kit */ Meeting new submission standards

Debian Forums Cache:About

2026-05-14T14:40:18Z

Distro-nix: /* About Debian Forums Cache: */ Added note on accuracy

Debian Live USB with Persistence

2026-04-10T03:21:00Z

Distro-nix: /* Citations */

All commands are verified on Debian 13.4 (Trixie) / 6.12.74+deb13+1-amd64

Created: 2026-04-04

Updated: 2026-04-09 - added references and restructured heading order;

ID: 005146.11

'''''A Debian Live USB with persistence boots a full Debian environment from a USB drive while retaining changes across sessions. Files you create, settings you change, and packages you install survive reboot—stored on a dedicated partition rather than the volatile live environment. Nothing is written to the host machine’s internal disk.'''''

'''''A standard live USB is stateless: each boot starts from the same ISO image. The persistence partition changes that. From a single drive you get your shell configuration, documents, utilities, browser profile, and scripts, available on any machine you boot it from.'''''

== What You’ll Need ==
A USB drive of at least 8 GB—the minimum is 4 GB, but that leaves little room for persistence. A 32 GB drive is a practical working size. Download the Debian Live ISO from the official Debian website: <nowiki>https://www.debian.org/CD/live</nowiki>

The tools required—dd, fdisk, and mkfs.ext4—are standard on Debian. On a minimal live environment, fdisk and mkfs.ext4 may not be present. Install them before starting:

<code>$ sudo apt install fdisk e2fsprogs</code>

These packages are available from the live session’s apt cache without adding sources. They don’t persist to the USB unless persistence is already active.

== Identifying Your USB Device ==
Plug in the USB drive and run:

<code>$ lsblk</code>

Your USB device will appear as /dev/sdb, /dev/sdc, or similar, depending on what else is attached. Confirm it by cross-referencing the reported size with the known capacity of the drive.

Two placeholders are used throughout this document. /dev/sdX refers to the device itself—substitute your actual device identifier everywhere it appears. /dev/sdXN refers to the persistence partition—N is the partition number you’ll confirm from lsblk output after you've created it.

Use the device path, not a partition path: /dev/sdb is correct; /dev/sdb1 is not.

== Boot Entry Selection ==
The firmware boot menu on many systems presents two entries for a USB device: one labeled with the device name alone, and one prefixed with UEFI:. On the hardware used for this document, selecting the UEFI:-prefixed entry after adding a persistence partition consistently produced a bare grub> prompt rather than the Debian live menu. Selecting the plain device entry booted correctly every time.

'''Throughout this procedure, select the plain device entry—not the one prefixed with UEFI''':.

This behavior was observed on specific hardware. Whether it reproduces on all firmware is not established here. The mechanism and a possible explanation are in <ref>https://uefi.org/specs/UEFI/2.10/03_Boot_Manager.html</ref> If your firmware presents only a UEFI boot path, recovery from within a booted live session may be possible by correcting the GRUB prefix, but that procedure is outside the scope of this document.

= Procedure =

=== Clean the Drive ===
If the USB has been used before, you may have residual partition signatures or bootloader data in the early sectors, and they can interfere with the ISO write or the subsequent boot. Clear them before starting:

<code>$ sudo wipefs -a /dev/sdX</code>

<code>$ sudo dd if=/dev/zero of=/dev/sdX bs=1M count=200</code>

wipefs removes filesystem and partition table signatures. The dd command zeros the first 200 MB, covering the MBR, GPT header, and any EFI data in the early sectors.

=== Write the ISO ===
Before running dd, confirm that sync resolves to the system binary and not a shell alias:

<code>$ type sync</code>

The output should be sync is /usr/bin/sync. If it returns an alias definition instead, unalias it with unalias sync, or substitute /usr/bin/sync explicitly at the end of the dd command.

Write the ISO:

<code>$ sudo dd if=/path/to/debian-live-13.4.0-amd64-kde.iso of=/dev/sdX bs=4M status=progress && sync</code>

status=progress shows the running transfer rate. sync flushes the write cache on completion—don’t remove the drive before it returns. The write will take a few minutes depending on the drive’s speed.

=== Verify the Base Boot ===
Before adding the persistence partition, confirm the ISO write produced a bootable drive. Boot from the USB, selecting the plain '''(non-UEFI:) device entry.''' If it reaches the Debian live menu and boots cleanly, you have a confirmed working baseline. If it doesn’t boot at this stage, don’t continue—go back to [[Debian Live USB with Persistence#Procedure|4.1 Clean the Drive]] and continue from there.

== Create the Persistence Partition ==
The ISO write leaves unallocated space at the end of the drive. The persistence partition goes there. The Debian live ISO uses an isohybrid layout—a hybrid MBR/GPT partition table that supports both BIOS and UEFI boot paths. Adding a partition requires an explicit start sector; without it, fdisk’s default placement can conflict with the hybrid structure in ways that affect the boot. The failure modes of parted and gdisk with this layout are documented in <ref>https://manpages.debian.org/trixie/live-boot-doc/persistence.conf.5.en.html</ref>

Launch fdisk:

<code>$ sudo fdisk /dev/sdX</code>

Print the current partition table:

Command (m for help): <code>p</code>

Two partitions will be present: the main ISO partition (type 0 or Empty—this is expected for an isohybrid layout) and a small EFI partition of approximately 3.2 MB. The p output shows the end sector of the last partition in the End column. The persistence partition must start at the next sector—that value plus one. On the test hardware with a Debian 13.4 live ISO written to the drive, the last partition ended at sector 8175999 and the persistence partition started at sector 8176000. This figure will vary with the ISO version and release; read your own p output and don’t use the test value without verifying it matches.

Create the new partition:

Command (m for help): <code>n</code>

When fdisk prompts for the first sector, enter the start sector you identified from the p output—do not accept the default. Accept the default for the last sector to use the remaining space on the drive.

If fdisk asks whether to remove an existing ext4 signature, answer <code>Y</code>. This clears a remnant from a previous attempt.

Write the changes and exit:

Command (m for help): <code>w</code>

After fdisk writes the new partition table, the running kernel may not immediately recognize the change. partprobe signals the kernel to re-read the partition table on the device without requiring a reboot:

<code>$ sudo partprobe /dev/sdX</code>

Once it completes, confirm the new partition is visible:

<code>$ lsblk /dev/sdX</code>

Your new partition will appear beneath the device in the output. Note its full identifier—for example, sdb3 or sdc4—as you’ll need it in the next two steps. If it doesn’t appear immediately, wait a moment and run lsblk again before proceeding.

== Format the Partition ==
Format the new partition as ext4 with the label persistence. The Debian live system identifies the persistence partition by this label at boot:

<code>$ sudo mkfs.ext4 -L persistence /dev/sdXN</code>

== Create the Persistence Configuration ==
Mount the partition:

<code>$ sudo mount /dev/sdXN /mnt</code>

Create the configuration file:

<code>$ echo "/ union" | sudo tee /mnt/persistence.conf</code>

The / union directive instructs the live system to mount the entire filesystem as a union overlay. Changes made during a session are written to the persistence partition rather than discarded on shutdown.

Unmount the partition:

<code>$ sudo umount /mnt</code>

== Boot with Persistence ==
Boot from the USB, selecting the plain (non-UEFI:) device entry in the firmware boot menu. At the Debian live GRUB menu, press Tab to edit the selected boot entry. On the line beginning with linux, append persistence after boot=live components. Press Enter to boot.

The first boot with persistence active will be slower than a standard live session—the system is running entirely from the USB drive, and the persistence overlay adds a layer of I/O. This is normal.

== Verify Persistence ==
After booting, confirm the persistence partition is mounted:

<code>$ lsblk -f</code>

The output should show sdXN mounted at /run/live/persistence/sdXN.

Create a test file:

<code>$ touch ~/persistence-test.txt</code>

Reboot with persistence enabled. If the file is present after reboot, persistence is working.

== What Didn’t Work, and Why ==

=== parted ===
parted misreported the capacity of the test USB drive: a 32 GB device was reported as approximately 125 GB. Determining where the free space began or how large to make the persistence partition was not possible from that output. parted was not used further. The broader behavior—libparted-based tools handling the isohybrid format unreliably—is documented on the Debian User mailing list [9].

=== gdisk / sgdisk ===
gdisk and sgdisk are GPT-aware tools and would seem the logical choice for a drive with a hybrid partition table. On the test USB, sgdisk produced:<blockquote>Found invalid GPT and valid MBR; converting MBR to GPT format in memory.

Warning! Secondary partition table overlaps the last partition by 33 blocks!

Could not create partition 3 from 14580 to 8177663

Non-GPT disk; not saving changes. Use -g to override.

Error encountered; not saving changes.</blockquote>The isohybrid layout places the secondary GPT header in a location that overlaps the last partition. sgdisk identifies this as invalid and refuses to proceed. Using -g to override would corrupt the hybrid layout and break the boot. sgdisk can’t be used here.

cp was also tested as an alternative to dd for writing the ISO, on the theory that it might handle the isohybrid structure differently. It produced an identical partition layout. It’s not an alternative—it’s a slower write with the same result.

== The UEFI grub> Prompt ==
On the test hardware, adding a persistence partition and then selecting the UEFI:-prefixed boot entry consistently dropped to a bare grub> prompt. Running ls from that prompt showed the internal drive’s partitions; the USB’s live partition was unreachable from there.

What was observed: the USB’s own GRUB appeared briefly—visible as a rapid flash on screen with a “no such…” error—then failed, after which the firmware handed control to the internal drive’s GRUB, which presented its own grub> prompt.

The UEFI specification states that firmware may remove boot options it considers invalid [7]; the ArchWiki documents firmware removing entries for drives it detects as changed as a known occurrence [8]. After adding the persistence partition, the USB’s EFI entry—pointing to EFI/boot/bootx64.efi—may no longer have been treated as valid by the firmware, causing it to fall through to the next available bootloader on the internal drive. Whether modifying the partition table on a removable device is sufficient to trigger this behavior, and under what firmware conditions, wasn’t reproducible in a controlled way during testing.

Selecting the plain (non-UEFI:) device entry uses the legacy MBR boot path, which loads the USB’s own GRUB directly and has no dependency on the UEFI NVRAM or EFI boot entries. On the test hardware, this path was unaffected by partition table changes.

== Anonymity ==
A Debian Live USB with persistence, used correctly, writes nothing to the host machine’s internal disk. It doesn’t modify the installed OS, create logs on the host filesystem, or leave traces in the usual locations.

It doesn’t provide anonymity. Boot events may be logged by firmware or enterprise management systems. Network activity is visible to routers, ISPs, and network administrators regardless of which OS is running. Mounting the internal disk during a live session makes its contents accessible to the live environment, and any writes to it are permanent. A live USB is a portable, self-contained Debian environment—it isn’t a privacy tool by design.

== A Note on Carrying Data ==
The persistence partition is a practical place to keep working data alongside the system state—documents, scripts, configuration files, anything you’d normally keep in ~/Documents or ~/bin. It travels with the drive.

Installed packages persist across sessions but may have hardware-specific dependencies. For general desktop use this isn’t usually an issue, but it’s worth keeping in mind if you’re installing drivers or kernel modules that assume specific hardware.

== Citations ==
Syslinux Project. “Isohybrid.” Syslinux Wiki.

[2] Linux Command Library. “isohybrid(1) man page.” https://linuxcommandlibrary.com/man/isohybrid

[3] Smith, Roderick W. “Hybrid MBRs.” GPT fdisk documentation. https://www.rodsbooks.com/gdisk/hybrid.html

[4] Debian Manpages. “persistence.conf(5).” live-boot-doc, Trixie. https://manpages.debian.org/trixie/live-boot-doc/persistence.conf.5.en.html

[5] Debian Live Team. “Customizing Run-Time Behaviours.” Debian Live Manual. https://live-team.pages.debian.net/live-manual/html/live-manual/customizing-run-time-behaviours.en.html

[6] Debian Manpages. “live-boot(7).” live-boot-doc. https://manpages.debian.org/unstable/live-boot-doc/live-boot.7.en.html

[7] UEFI Forum. “Boot Manager.” UEFI Specification 2.10, §3. https://uefi.org/specs/UEFI/2.10/03_Boot_Manager.html

[8] ArchWiki. “Unified Extensible Firmware Interface.” https://wiki.archlinux.org/title/Unified_Extensible_Firmware_Interface

[9] Debian User Mailing List. “Error message from GParted, means what?” November 2016. https://lists.debian.org/debian-user/2016/11/msg01017.html

<references />

''Contributed by [https://forums.debian.net/search.php?keywords=distro-nix distro-nix] on Debian User Forums on 2026-04-06.'' Comments, corrections, suggestions or resources are welcome.

Debian Live USB with Persistence

2026-04-10T03:20:32Z

Distro-nix: /* Citations */ Added blank line above footer for separation.

All commands are verified on Debian 13.4 (Trixie) / 6.12.74+deb13+1-amd64

Created: 2026-04-04

Updated: 2026-04-09 - added references and restructured heading order;

ID: 005146.11

'''''A Debian Live USB with persistence boots a full Debian environment from a USB drive while retaining changes across sessions. Files you create, settings you change, and packages you install survive reboot—stored on a dedicated partition rather than the volatile live environment. Nothing is written to the host machine’s internal disk.'''''

'''''A standard live USB is stateless: each boot starts from the same ISO image. The persistence partition changes that. From a single drive you get your shell configuration, documents, utilities, browser profile, and scripts, available on any machine you boot it from.'''''

== What You’ll Need ==
A USB drive of at least 8 GB—the minimum is 4 GB, but that leaves little room for persistence. A 32 GB drive is a practical working size. Download the Debian Live ISO from the official Debian website: <nowiki>https://www.debian.org/CD/live</nowiki>

The tools required—dd, fdisk, and mkfs.ext4—are standard on Debian. On a minimal live environment, fdisk and mkfs.ext4 may not be present. Install them before starting:

<code>$ sudo apt install fdisk e2fsprogs</code>

These packages are available from the live session’s apt cache without adding sources. They don’t persist to the USB unless persistence is already active.

== Identifying Your USB Device ==
Plug in the USB drive and run:

<code>$ lsblk</code>

Your USB device will appear as /dev/sdb, /dev/sdc, or similar, depending on what else is attached. Confirm it by cross-referencing the reported size with the known capacity of the drive.

Two placeholders are used throughout this document. /dev/sdX refers to the device itself—substitute your actual device identifier everywhere it appears. /dev/sdXN refers to the persistence partition—N is the partition number you’ll confirm from lsblk output after you've created it.

Use the device path, not a partition path: /dev/sdb is correct; /dev/sdb1 is not.

== Boot Entry Selection ==
The firmware boot menu on many systems presents two entries for a USB device: one labeled with the device name alone, and one prefixed with UEFI:. On the hardware used for this document, selecting the UEFI:-prefixed entry after adding a persistence partition consistently produced a bare grub> prompt rather than the Debian live menu. Selecting the plain device entry booted correctly every time.

'''Throughout this procedure, select the plain device entry—not the one prefixed with UEFI''':.

This behavior was observed on specific hardware. Whether it reproduces on all firmware is not established here. The mechanism and a possible explanation are in <ref>https://uefi.org/specs/UEFI/2.10/03_Boot_Manager.html</ref> If your firmware presents only a UEFI boot path, recovery from within a booted live session may be possible by correcting the GRUB prefix, but that procedure is outside the scope of this document.

= Procedure =

=== Clean the Drive ===
If the USB has been used before, you may have residual partition signatures or bootloader data in the early sectors, and they can interfere with the ISO write or the subsequent boot. Clear them before starting:

<code>$ sudo wipefs -a /dev/sdX</code>

<code>$ sudo dd if=/dev/zero of=/dev/sdX bs=1M count=200</code>

wipefs removes filesystem and partition table signatures. The dd command zeros the first 200 MB, covering the MBR, GPT header, and any EFI data in the early sectors.

=== Write the ISO ===
Before running dd, confirm that sync resolves to the system binary and not a shell alias:

<code>$ type sync</code>

The output should be sync is /usr/bin/sync. If it returns an alias definition instead, unalias it with unalias sync, or substitute /usr/bin/sync explicitly at the end of the dd command.

Write the ISO:

<code>$ sudo dd if=/path/to/debian-live-13.4.0-amd64-kde.iso of=/dev/sdX bs=4M status=progress && sync</code>

status=progress shows the running transfer rate. sync flushes the write cache on completion—don’t remove the drive before it returns. The write will take a few minutes depending on the drive’s speed.

=== Verify the Base Boot ===
Before adding the persistence partition, confirm the ISO write produced a bootable drive. Boot from the USB, selecting the plain '''(non-UEFI:) device entry.''' If it reaches the Debian live menu and boots cleanly, you have a confirmed working baseline. If it doesn’t boot at this stage, don’t continue—go back to [[Debian Live USB with Persistence#Procedure|4.1 Clean the Drive]] and continue from there.

== Create the Persistence Partition ==
The ISO write leaves unallocated space at the end of the drive. The persistence partition goes there. The Debian live ISO uses an isohybrid layout—a hybrid MBR/GPT partition table that supports both BIOS and UEFI boot paths. Adding a partition requires an explicit start sector; without it, fdisk’s default placement can conflict with the hybrid structure in ways that affect the boot. The failure modes of parted and gdisk with this layout are documented in <ref>https://manpages.debian.org/trixie/live-boot-doc/persistence.conf.5.en.html</ref>

Launch fdisk:

<code>$ sudo fdisk /dev/sdX</code>

Print the current partition table:

Command (m for help): <code>p</code>

Two partitions will be present: the main ISO partition (type 0 or Empty—this is expected for an isohybrid layout) and a small EFI partition of approximately 3.2 MB. The p output shows the end sector of the last partition in the End column. The persistence partition must start at the next sector—that value plus one. On the test hardware with a Debian 13.4 live ISO written to the drive, the last partition ended at sector 8175999 and the persistence partition started at sector 8176000. This figure will vary with the ISO version and release; read your own p output and don’t use the test value without verifying it matches.

Create the new partition:

Command (m for help): <code>n</code>

When fdisk prompts for the first sector, enter the start sector you identified from the p output—do not accept the default. Accept the default for the last sector to use the remaining space on the drive.

If fdisk asks whether to remove an existing ext4 signature, answer <code>Y</code>. This clears a remnant from a previous attempt.

Write the changes and exit:

Command (m for help): <code>w</code>

After fdisk writes the new partition table, the running kernel may not immediately recognize the change. partprobe signals the kernel to re-read the partition table on the device without requiring a reboot:

<code>$ sudo partprobe /dev/sdX</code>

Once it completes, confirm the new partition is visible:

<code>$ lsblk /dev/sdX</code>

Your new partition will appear beneath the device in the output. Note its full identifier—for example, sdb3 or sdc4—as you’ll need it in the next two steps. If it doesn’t appear immediately, wait a moment and run lsblk again before proceeding.

== Format the Partition ==
Format the new partition as ext4 with the label persistence. The Debian live system identifies the persistence partition by this label at boot:

<code>$ sudo mkfs.ext4 -L persistence /dev/sdXN</code>

== Create the Persistence Configuration ==
Mount the partition:

<code>$ sudo mount /dev/sdXN /mnt</code>

Create the configuration file:

<code>$ echo "/ union" | sudo tee /mnt/persistence.conf</code>

The / union directive instructs the live system to mount the entire filesystem as a union overlay. Changes made during a session are written to the persistence partition rather than discarded on shutdown.

Unmount the partition:

<code>$ sudo umount /mnt</code>

== Boot with Persistence ==
Boot from the USB, selecting the plain (non-UEFI:) device entry in the firmware boot menu. At the Debian live GRUB menu, press Tab to edit the selected boot entry. On the line beginning with linux, append persistence after boot=live components. Press Enter to boot.

The first boot with persistence active will be slower than a standard live session—the system is running entirely from the USB drive, and the persistence overlay adds a layer of I/O. This is normal.

== Verify Persistence ==
After booting, confirm the persistence partition is mounted:

<code>$ lsblk -f</code>

The output should show sdXN mounted at /run/live/persistence/sdXN.

Create a test file:

<code>$ touch ~/persistence-test.txt</code>

Reboot with persistence enabled. If the file is present after reboot, persistence is working.

== What Didn’t Work, and Why ==

=== parted ===
parted misreported the capacity of the test USB drive: a 32 GB device was reported as approximately 125 GB. Determining where the free space began or how large to make the persistence partition was not possible from that output. parted was not used further. The broader behavior—libparted-based tools handling the isohybrid format unreliably—is documented on the Debian User mailing list [9].

=== gdisk / sgdisk ===
gdisk and sgdisk are GPT-aware tools and would seem the logical choice for a drive with a hybrid partition table. On the test USB, sgdisk produced:<blockquote>Found invalid GPT and valid MBR; converting MBR to GPT format in memory.

Warning! Secondary partition table overlaps the last partition by 33 blocks!

Could not create partition 3 from 14580 to 8177663

Non-GPT disk; not saving changes. Use -g to override.

Error encountered; not saving changes.</blockquote>The isohybrid layout places the secondary GPT header in a location that overlaps the last partition. sgdisk identifies this as invalid and refuses to proceed. Using -g to override would corrupt the hybrid layout and break the boot. sgdisk can’t be used here.

cp was also tested as an alternative to dd for writing the ISO, on the theory that it might handle the isohybrid structure differently. It produced an identical partition layout. It’s not an alternative—it’s a slower write with the same result.

== The UEFI grub> Prompt ==
On the test hardware, adding a persistence partition and then selecting the UEFI:-prefixed boot entry consistently dropped to a bare grub> prompt. Running ls from that prompt showed the internal drive’s partitions; the USB’s live partition was unreachable from there.

What was observed: the USB’s own GRUB appeared briefly—visible as a rapid flash on screen with a “no such…” error—then failed, after which the firmware handed control to the internal drive’s GRUB, which presented its own grub> prompt.

The UEFI specification states that firmware may remove boot options it considers invalid [7]; the ArchWiki documents firmware removing entries for drives it detects as changed as a known occurrence [8]. After adding the persistence partition, the USB’s EFI entry—pointing to EFI/boot/bootx64.efi—may no longer have been treated as valid by the firmware, causing it to fall through to the next available bootloader on the internal drive. Whether modifying the partition table on a removable device is sufficient to trigger this behavior, and under what firmware conditions, wasn’t reproducible in a controlled way during testing.

Selecting the plain (non-UEFI:) device entry uses the legacy MBR boot path, which loads the USB’s own GRUB directly and has no dependency on the UEFI NVRAM or EFI boot entries. On the test hardware, this path was unaffected by partition table changes.

== Anonymity ==
A Debian Live USB with persistence, used correctly, writes nothing to the host machine’s internal disk. It doesn’t modify the installed OS, create logs on the host filesystem, or leave traces in the usual locations.

It doesn’t provide anonymity. Boot events may be logged by firmware or enterprise management systems. Network activity is visible to routers, ISPs, and network administrators regardless of which OS is running. Mounting the internal disk during a live session makes its contents accessible to the live environment, and any writes to it are permanent. A live USB is a portable, self-contained Debian environment—it isn’t a privacy tool by design.

== A Note on Carrying Data ==
The persistence partition is a practical place to keep working data alongside the system state—documents, scripts, configuration files, anything you’d normally keep in ~/Documents or ~/bin. It travels with the drive.

Installed packages persist across sessions but may have hardware-specific dependencies. For general desktop use this isn’t usually an issue, but it’s worth keeping in mind if you’re installing drivers or kernel modules that assume specific hardware.

== Citations ==
Syslinux Project. “Isohybrid.” Syslinux Wiki.

[2] Linux Command Library. “isohybrid(1) man page.” https://linuxcommandlibrary.com/man/isohybrid

[3] Smith, Roderick W. “Hybrid MBRs.” GPT fdisk documentation. https://www.rodsbooks.com/gdisk/hybrid.html

[4] Debian Manpages. “persistence.conf(5).” live-boot-doc, Trixie. https://manpages.debian.org/trixie/live-boot-doc/persistence.conf.5.en.html

[5] Debian Live Team. “Customizing Run-Time Behaviours.” Debian Live Manual. https://live-team.pages.debian.net/live-manual/html/live-manual/customizing-run-time-behaviours.en.html

[6] Debian Manpages. “live-boot(7).” live-boot-doc. https://manpages.debian.org/unstable/live-boot-doc/live-boot.7.en.html

[7] UEFI Forum. “Boot Manager.” UEFI Specification 2.10, §3. https://uefi.org/specs/UEFI/2.10/03_Boot_Manager.html

[8] ArchWiki. “Unified Extensible Firmware Interface.” https://wiki.archlinux.org/title/Unified_Extensible_Firmware_Interface

[9] Debian User Mailing List. “Error message from GParted, means what?” November 2016. https://lists.debian.org/debian-user/2016/11/msg01017.html

<references />
''Contributed by [https://forums.debian.net/search.php?keywords=distro-nix distro-nix] on Debian User Forums on 2026-04-06.'' Comments, corrections, suggestions or resources are welcome.

''Contributed by [https://forums.debian.net/search.php?keywords=distro-nix distro-nix] on Debian User Forums on 2026-04-06.'' Comments, corrections, suggestions or resources are welcome.

Debian Live USB with Persistence

2026-04-10T03:15:27Z

Distro-nix: /* Verify the Base Boot */

All commands are verified on Debian 13.4 (Trixie) / 6.12.74+deb13+1-amd64

Created: 2026-04-04

Updated: 2026-04-09 - added references and restructured heading order;

ID: 005146.11

'''''A Debian Live USB with persistence boots a full Debian environment from a USB drive while retaining changes across sessions. Files you create, settings you change, and packages you install survive reboot—stored on a dedicated partition rather than the volatile live environment. Nothing is written to the host machine’s internal disk.'''''

'''''A standard live USB is stateless: each boot starts from the same ISO image. The persistence partition changes that. From a single drive you get your shell configuration, documents, utilities, browser profile, and scripts, available on any machine you boot it from.'''''

== What You’ll Need ==
A USB drive of at least 8 GB—the minimum is 4 GB, but that leaves little room for persistence. A 32 GB drive is a practical working size. Download the Debian Live ISO from the official Debian website: <nowiki>https://www.debian.org/CD/live</nowiki>

The tools required—dd, fdisk, and mkfs.ext4—are standard on Debian. On a minimal live environment, fdisk and mkfs.ext4 may not be present. Install them before starting:

<code>$ sudo apt install fdisk e2fsprogs</code>

These packages are available from the live session’s apt cache without adding sources. They don’t persist to the USB unless persistence is already active.

== Identifying Your USB Device ==
Plug in the USB drive and run:

<code>$ lsblk</code>

Your USB device will appear as /dev/sdb, /dev/sdc, or similar, depending on what else is attached. Confirm it by cross-referencing the reported size with the known capacity of the drive.

Two placeholders are used throughout this document. /dev/sdX refers to the device itself—substitute your actual device identifier everywhere it appears. /dev/sdXN refers to the persistence partition—N is the partition number you’ll confirm from lsblk output after you've created it.

Use the device path, not a partition path: /dev/sdb is correct; /dev/sdb1 is not.

== Boot Entry Selection ==
The firmware boot menu on many systems presents two entries for a USB device: one labeled with the device name alone, and one prefixed with UEFI:. On the hardware used for this document, selecting the UEFI:-prefixed entry after adding a persistence partition consistently produced a bare grub> prompt rather than the Debian live menu. Selecting the plain device entry booted correctly every time.

'''Throughout this procedure, select the plain device entry—not the one prefixed with UEFI''':.

This behavior was observed on specific hardware. Whether it reproduces on all firmware is not established here. The mechanism and a possible explanation are in <ref>https://uefi.org/specs/UEFI/2.10/03_Boot_Manager.html</ref> If your firmware presents only a UEFI boot path, recovery from within a booted live session may be possible by correcting the GRUB prefix, but that procedure is outside the scope of this document.

= Procedure =

=== Clean the Drive ===
If the USB has been used before, you may have residual partition signatures or bootloader data in the early sectors, and they can interfere with the ISO write or the subsequent boot. Clear them before starting:

<code>$ sudo wipefs -a /dev/sdX</code>

<code>$ sudo dd if=/dev/zero of=/dev/sdX bs=1M count=200</code>

wipefs removes filesystem and partition table signatures. The dd command zeros the first 200 MB, covering the MBR, GPT header, and any EFI data in the early sectors.

=== Write the ISO ===
Before running dd, confirm that sync resolves to the system binary and not a shell alias:

<code>$ type sync</code>

The output should be sync is /usr/bin/sync. If it returns an alias definition instead, unalias it with unalias sync, or substitute /usr/bin/sync explicitly at the end of the dd command.

Write the ISO:

<code>$ sudo dd if=/path/to/debian-live-13.4.0-amd64-kde.iso of=/dev/sdX bs=4M status=progress && sync</code>

status=progress shows the running transfer rate. sync flushes the write cache on completion—don’t remove the drive before it returns. The write will take a few minutes depending on the drive’s speed.

=== Verify the Base Boot ===
Before adding the persistence partition, confirm the ISO write produced a bootable drive. Boot from the USB, selecting the plain '''(non-UEFI:) device entry.''' If it reaches the Debian live menu and boots cleanly, you have a confirmed working baseline. If it doesn’t boot at this stage, don’t continue—go back to [[Debian Live USB with Persistence#Procedure|4.1 Clean the Drive]] and continue from there.

== Create the Persistence Partition ==
The ISO write leaves unallocated space at the end of the drive. The persistence partition goes there. The Debian live ISO uses an isohybrid layout—a hybrid MBR/GPT partition table that supports both BIOS and UEFI boot paths. Adding a partition requires an explicit start sector; without it, fdisk’s default placement can conflict with the hybrid structure in ways that affect the boot. The failure modes of parted and gdisk with this layout are documented in <ref>https://manpages.debian.org/trixie/live-boot-doc/persistence.conf.5.en.html</ref>

Launch fdisk:

<code>$ sudo fdisk /dev/sdX</code>

Print the current partition table:

Command (m for help): <code>p</code>

Two partitions will be present: the main ISO partition (type 0 or Empty—this is expected for an isohybrid layout) and a small EFI partition of approximately 3.2 MB. The p output shows the end sector of the last partition in the End column. The persistence partition must start at the next sector—that value plus one. On the test hardware with a Debian 13.4 live ISO written to the drive, the last partition ended at sector 8175999 and the persistence partition started at sector 8176000. This figure will vary with the ISO version and release; read your own p output and don’t use the test value without verifying it matches.

Create the new partition:

Command (m for help): <code>n</code>

When fdisk prompts for the first sector, enter the start sector you identified from the p output—do not accept the default. Accept the default for the last sector to use the remaining space on the drive.

If fdisk asks whether to remove an existing ext4 signature, answer <code>Y</code>. This clears a remnant from a previous attempt.

Write the changes and exit:

Command (m for help): <code>w</code>

After fdisk writes the new partition table, the running kernel may not immediately recognize the change. partprobe signals the kernel to re-read the partition table on the device without requiring a reboot:

<code>$ sudo partprobe /dev/sdX</code>

Once it completes, confirm the new partition is visible:

<code>$ lsblk /dev/sdX</code>

Your new partition will appear beneath the device in the output. Note its full identifier—for example, sdb3 or sdc4—as you’ll need it in the next two steps. If it doesn’t appear immediately, wait a moment and run lsblk again before proceeding.

== Format the Partition ==
Format the new partition as ext4 with the label persistence. The Debian live system identifies the persistence partition by this label at boot:

<code>$ sudo mkfs.ext4 -L persistence /dev/sdXN</code>

== Create the Persistence Configuration ==
Mount the partition:

<code>$ sudo mount /dev/sdXN /mnt</code>

Create the configuration file:

<code>$ echo "/ union" | sudo tee /mnt/persistence.conf</code>

The / union directive instructs the live system to mount the entire filesystem as a union overlay. Changes made during a session are written to the persistence partition rather than discarded on shutdown.

Unmount the partition:

<code>$ sudo umount /mnt</code>

== Boot with Persistence ==
Boot from the USB, selecting the plain (non-UEFI:) device entry in the firmware boot menu. At the Debian live GRUB menu, press Tab to edit the selected boot entry. On the line beginning with linux, append persistence after boot=live components. Press Enter to boot.

The first boot with persistence active will be slower than a standard live session—the system is running entirely from the USB drive, and the persistence overlay adds a layer of I/O. This is normal.

== Verify Persistence ==
After booting, confirm the persistence partition is mounted:

<code>$ lsblk -f</code>

The output should show sdXN mounted at /run/live/persistence/sdXN.

Create a test file:

<code>$ touch ~/persistence-test.txt</code>

Reboot with persistence enabled. If the file is present after reboot, persistence is working.

== What Didn’t Work, and Why ==

=== parted ===
parted misreported the capacity of the test USB drive: a 32 GB device was reported as approximately 125 GB. Determining where the free space began or how large to make the persistence partition was not possible from that output. parted was not used further. The broader behavior—libparted-based tools handling the isohybrid format unreliably—is documented on the Debian User mailing list [9].

=== gdisk / sgdisk ===
gdisk and sgdisk are GPT-aware tools and would seem the logical choice for a drive with a hybrid partition table. On the test USB, sgdisk produced:<blockquote>Found invalid GPT and valid MBR; converting MBR to GPT format in memory.

Warning! Secondary partition table overlaps the last partition by 33 blocks!

Could not create partition 3 from 14580 to 8177663

Non-GPT disk; not saving changes. Use -g to override.

Error encountered; not saving changes.</blockquote>The isohybrid layout places the secondary GPT header in a location that overlaps the last partition. sgdisk identifies this as invalid and refuses to proceed. Using -g to override would corrupt the hybrid layout and break the boot. sgdisk can’t be used here.

cp was also tested as an alternative to dd for writing the ISO, on the theory that it might handle the isohybrid structure differently. It produced an identical partition layout. It’s not an alternative—it’s a slower write with the same result.

== The UEFI grub> Prompt ==
On the test hardware, adding a persistence partition and then selecting the UEFI:-prefixed boot entry consistently dropped to a bare grub> prompt. Running ls from that prompt showed the internal drive’s partitions; the USB’s live partition was unreachable from there.

What was observed: the USB’s own GRUB appeared briefly—visible as a rapid flash on screen with a “no such…” error—then failed, after which the firmware handed control to the internal drive’s GRUB, which presented its own grub> prompt.

The UEFI specification states that firmware may remove boot options it considers invalid [7]; the ArchWiki documents firmware removing entries for drives it detects as changed as a known occurrence [8]. After adding the persistence partition, the USB’s EFI entry—pointing to EFI/boot/bootx64.efi—may no longer have been treated as valid by the firmware, causing it to fall through to the next available bootloader on the internal drive. Whether modifying the partition table on a removable device is sufficient to trigger this behavior, and under what firmware conditions, wasn’t reproducible in a controlled way during testing.

Selecting the plain (non-UEFI:) device entry uses the legacy MBR boot path, which loads the USB’s own GRUB directly and has no dependency on the UEFI NVRAM or EFI boot entries. On the test hardware, this path was unaffected by partition table changes.

== Anonymity ==
A Debian Live USB with persistence, used correctly, writes nothing to the host machine’s internal disk. It doesn’t modify the installed OS, create logs on the host filesystem, or leave traces in the usual locations.

It doesn’t provide anonymity. Boot events may be logged by firmware or enterprise management systems. Network activity is visible to routers, ISPs, and network administrators regardless of which OS is running. Mounting the internal disk during a live session makes its contents accessible to the live environment, and any writes to it are permanent. A live USB is a portable, self-contained Debian environment—it isn’t a privacy tool by design.

== A Note on Carrying Data ==
The persistence partition is a practical place to keep working data alongside the system state—documents, scripts, configuration files, anything you’d normally keep in ~/Documents or ~/bin. It travels with the drive.

Installed packages persist across sessions but may have hardware-specific dependencies. For general desktop use this isn’t usually an issue, but it’s worth keeping in mind if you’re installing drivers or kernel modules that assume specific hardware.

== Citations ==
Syslinux Project. “Isohybrid.” Syslinux Wiki.

[2] Linux Command Library. “isohybrid(1) man page.” https://linuxcommandlibrary.com/man/isohybrid

[3] Smith, Roderick W. “Hybrid MBRs.” GPT fdisk documentation. https://www.rodsbooks.com/gdisk/hybrid.html

[4] Debian Manpages. “persistence.conf(5).” live-boot-doc, Trixie. https://manpages.debian.org/trixie/live-boot-doc/persistence.conf.5.en.html

[5] Debian Live Team. “Customizing Run-Time Behaviours.” Debian Live Manual. https://live-team.pages.debian.net/live-manual/html/live-manual/customizing-run-time-behaviours.en.html

[6] Debian Manpages. “live-boot(7).” live-boot-doc. https://manpages.debian.org/unstable/live-boot-doc/live-boot.7.en.html

[7] UEFI Forum. “Boot Manager.” UEFI Specification 2.10, §3. https://uefi.org/specs/UEFI/2.10/03_Boot_Manager.html

[8] ArchWiki. “Unified Extensible Firmware Interface.” https://wiki.archlinux.org/title/Unified_Extensible_Firmware_Interface

[9] Debian User Mailing List. “Error message from GParted, means what?” November 2016. https://lists.debian.org/debian-user/2016/11/msg01017.html

<references />
''Contributed by [https://forums.debian.net/search.php?keywords=distro-nix distro-nix] on Debian User Forums on 2026-04-06.'' Comments, corrections, suggestions or resources are welcome.

Debian Live USB with Persistence

2026-04-10T03:13:07Z

Distro-nix: /* Identifying Your USB Device */ 4.1 Clean the Drive

All commands are verified on Debian 13.4 (Trixie) / 6.12.74+deb13+1-amd64

Created: 2026-04-04

Updated: 2026-04-09 - added references and restructured heading order;

ID: 005146.11

'''''A Debian Live USB with persistence boots a full Debian environment from a USB drive while retaining changes across sessions. Files you create, settings you change, and packages you install survive reboot—stored on a dedicated partition rather than the volatile live environment. Nothing is written to the host machine’s internal disk.'''''

'''''A standard live USB is stateless: each boot starts from the same ISO image. The persistence partition changes that. From a single drive you get your shell configuration, documents, utilities, browser profile, and scripts, available on any machine you boot it from.'''''

== What You’ll Need ==
A USB drive of at least 8 GB—the minimum is 4 GB, but that leaves little room for persistence. A 32 GB drive is a practical working size. Download the Debian Live ISO from the official Debian website: <nowiki>https://www.debian.org/CD/live</nowiki>

The tools required—dd, fdisk, and mkfs.ext4—are standard on Debian. On a minimal live environment, fdisk and mkfs.ext4 may not be present. Install them before starting:

<code>$ sudo apt install fdisk e2fsprogs</code>

These packages are available from the live session’s apt cache without adding sources. They don’t persist to the USB unless persistence is already active.

== Identifying Your USB Device ==
Plug in the USB drive and run:

<code>$ lsblk</code>

Your USB device will appear as /dev/sdb, /dev/sdc, or similar, depending on what else is attached. Confirm it by cross-referencing the reported size with the known capacity of the drive.

Two placeholders are used throughout this document. /dev/sdX refers to the device itself—substitute your actual device identifier everywhere it appears. /dev/sdXN refers to the persistence partition—N is the partition number you’ll confirm from lsblk output after you've created it.

Use the device path, not a partition path: /dev/sdb is correct; /dev/sdb1 is not.

== Boot Entry Selection ==
The firmware boot menu on many systems presents two entries for a USB device: one labeled with the device name alone, and one prefixed with UEFI:. On the hardware used for this document, selecting the UEFI:-prefixed entry after adding a persistence partition consistently produced a bare grub> prompt rather than the Debian live menu. Selecting the plain device entry booted correctly every time.

'''Throughout this procedure, select the plain device entry—not the one prefixed with UEFI''':.

This behavior was observed on specific hardware. Whether it reproduces on all firmware is not established here. The mechanism and a possible explanation are in <ref>https://uefi.org/specs/UEFI/2.10/03_Boot_Manager.html</ref> If your firmware presents only a UEFI boot path, recovery from within a booted live session may be possible by correcting the GRUB prefix, but that procedure is outside the scope of this document.

= Procedure =

=== Clean the Drive ===
If the USB has been used before, you may have residual partition signatures or bootloader data in the early sectors, and they can interfere with the ISO write or the subsequent boot. Clear them before starting:

<code>$ sudo wipefs -a /dev/sdX</code>

<code>$ sudo dd if=/dev/zero of=/dev/sdX bs=1M count=200</code>

wipefs removes filesystem and partition table signatures. The dd command zeros the first 200 MB, covering the MBR, GPT header, and any EFI data in the early sectors.

=== Write the ISO ===
Before running dd, confirm that sync resolves to the system binary and not a shell alias:

<code>$ type sync</code>

The output should be sync is /usr/bin/sync. If it returns an alias definition instead, unalias it with unalias sync, or substitute /usr/bin/sync explicitly at the end of the dd command.

Write the ISO:

<code>$ sudo dd if=/path/to/debian-live-13.4.0-amd64-kde.iso of=/dev/sdX bs=4M status=progress && sync</code>

status=progress shows the running transfer rate. sync flushes the write cache on completion—don’t remove the drive before it returns. The write will take a few minutes depending on the drive’s speed.

=== Verify the Base Boot ===
Before adding the persistence partition, confirm the ISO write produced a bootable drive. Boot from the USB, selecting the plain '''(non-UEFI:) device entry.''' If it reaches the Debian live menu and boots cleanly, you have a confirmed working baseline. If it doesn’t boot at this stage, don’t continue—return to [[Debian Live USB with Persistence#Procedure|4.1 Clean the Drive]]

== Create the Persistence Partition ==
The ISO write leaves unallocated space at the end of the drive. The persistence partition goes there. The Debian live ISO uses an isohybrid layout—a hybrid MBR/GPT partition table that supports both BIOS and UEFI boot paths. Adding a partition requires an explicit start sector; without it, fdisk’s default placement can conflict with the hybrid structure in ways that affect the boot. The failure modes of parted and gdisk with this layout are documented in <ref>https://manpages.debian.org/trixie/live-boot-doc/persistence.conf.5.en.html</ref>

Launch fdisk:

<code>$ sudo fdisk /dev/sdX</code>

Print the current partition table:

Command (m for help): <code>p</code>

Two partitions will be present: the main ISO partition (type 0 or Empty—this is expected for an isohybrid layout) and a small EFI partition of approximately 3.2 MB. The p output shows the end sector of the last partition in the End column. The persistence partition must start at the next sector—that value plus one. On the test hardware with a Debian 13.4 live ISO written to the drive, the last partition ended at sector 8175999 and the persistence partition started at sector 8176000. This figure will vary with the ISO version and release; read your own p output and don’t use the test value without verifying it matches.

Create the new partition:

Command (m for help): <code>n</code>

When fdisk prompts for the first sector, enter the start sector you identified from the p output—do not accept the default. Accept the default for the last sector to use the remaining space on the drive.

If fdisk asks whether to remove an existing ext4 signature, answer <code>Y</code>. This clears a remnant from a previous attempt.

Write the changes and exit:

Command (m for help): <code>w</code>

After fdisk writes the new partition table, the running kernel may not immediately recognize the change. partprobe signals the kernel to re-read the partition table on the device without requiring a reboot:

<code>$ sudo partprobe /dev/sdX</code>

Once it completes, confirm the new partition is visible:

<code>$ lsblk /dev/sdX</code>

Your new partition will appear beneath the device in the output. Note its full identifier—for example, sdb3 or sdc4—as you’ll need it in the next two steps. If it doesn’t appear immediately, wait a moment and run lsblk again before proceeding.

== Format the Partition ==
Format the new partition as ext4 with the label persistence. The Debian live system identifies the persistence partition by this label at boot:

<code>$ sudo mkfs.ext4 -L persistence /dev/sdXN</code>

== Create the Persistence Configuration ==
Mount the partition:

<code>$ sudo mount /dev/sdXN /mnt</code>

Create the configuration file:

<code>$ echo "/ union" | sudo tee /mnt/persistence.conf</code>

The / union directive instructs the live system to mount the entire filesystem as a union overlay. Changes made during a session are written to the persistence partition rather than discarded on shutdown.

Unmount the partition:

<code>$ sudo umount /mnt</code>

== Boot with Persistence ==
Boot from the USB, selecting the plain (non-UEFI:) device entry in the firmware boot menu. At the Debian live GRUB menu, press Tab to edit the selected boot entry. On the line beginning with linux, append persistence after boot=live components. Press Enter to boot.

The first boot with persistence active will be slower than a standard live session—the system is running entirely from the USB drive, and the persistence overlay adds a layer of I/O. This is normal.

== Verify Persistence ==
After booting, confirm the persistence partition is mounted:

<code>$ lsblk -f</code>

The output should show sdXN mounted at /run/live/persistence/sdXN.

Create a test file:

<code>$ touch ~/persistence-test.txt</code>

Reboot with persistence enabled. If the file is present after reboot, persistence is working.

== What Didn’t Work, and Why ==

=== parted ===
parted misreported the capacity of the test USB drive: a 32 GB device was reported as approximately 125 GB. Determining where the free space began or how large to make the persistence partition was not possible from that output. parted was not used further. The broader behavior—libparted-based tools handling the isohybrid format unreliably—is documented on the Debian User mailing list [9].

=== gdisk / sgdisk ===
gdisk and sgdisk are GPT-aware tools and would seem the logical choice for a drive with a hybrid partition table. On the test USB, sgdisk produced:<blockquote>Found invalid GPT and valid MBR; converting MBR to GPT format in memory.

Warning! Secondary partition table overlaps the last partition by 33 blocks!

Could not create partition 3 from 14580 to 8177663

Non-GPT disk; not saving changes. Use -g to override.

Error encountered; not saving changes.</blockquote>The isohybrid layout places the secondary GPT header in a location that overlaps the last partition. sgdisk identifies this as invalid and refuses to proceed. Using -g to override would corrupt the hybrid layout and break the boot. sgdisk can’t be used here.

cp was also tested as an alternative to dd for writing the ISO, on the theory that it might handle the isohybrid structure differently. It produced an identical partition layout. It’s not an alternative—it’s a slower write with the same result.

== The UEFI grub> Prompt ==
On the test hardware, adding a persistence partition and then selecting the UEFI:-prefixed boot entry consistently dropped to a bare grub> prompt. Running ls from that prompt showed the internal drive’s partitions; the USB’s live partition was unreachable from there.

What was observed: the USB’s own GRUB appeared briefly—visible as a rapid flash on screen with a “no such…” error—then failed, after which the firmware handed control to the internal drive’s GRUB, which presented its own grub> prompt.

The UEFI specification states that firmware may remove boot options it considers invalid [7]; the ArchWiki documents firmware removing entries for drives it detects as changed as a known occurrence [8]. After adding the persistence partition, the USB’s EFI entry—pointing to EFI/boot/bootx64.efi—may no longer have been treated as valid by the firmware, causing it to fall through to the next available bootloader on the internal drive. Whether modifying the partition table on a removable device is sufficient to trigger this behavior, and under what firmware conditions, wasn’t reproducible in a controlled way during testing.

Selecting the plain (non-UEFI:) device entry uses the legacy MBR boot path, which loads the USB’s own GRUB directly and has no dependency on the UEFI NVRAM or EFI boot entries. On the test hardware, this path was unaffected by partition table changes.

== Anonymity ==
A Debian Live USB with persistence, used correctly, writes nothing to the host machine’s internal disk. It doesn’t modify the installed OS, create logs on the host filesystem, or leave traces in the usual locations.

It doesn’t provide anonymity. Boot events may be logged by firmware or enterprise management systems. Network activity is visible to routers, ISPs, and network administrators regardless of which OS is running. Mounting the internal disk during a live session makes its contents accessible to the live environment, and any writes to it are permanent. A live USB is a portable, self-contained Debian environment—it isn’t a privacy tool by design.

== A Note on Carrying Data ==
The persistence partition is a practical place to keep working data alongside the system state—documents, scripts, configuration files, anything you’d normally keep in ~/Documents or ~/bin. It travels with the drive.

Installed packages persist across sessions but may have hardware-specific dependencies. For general desktop use this isn’t usually an issue, but it’s worth keeping in mind if you’re installing drivers or kernel modules that assume specific hardware.

== Citations ==
Syslinux Project. “Isohybrid.” Syslinux Wiki.

[2] Linux Command Library. “isohybrid(1) man page.” https://linuxcommandlibrary.com/man/isohybrid

[3] Smith, Roderick W. “Hybrid MBRs.” GPT fdisk documentation. https://www.rodsbooks.com/gdisk/hybrid.html

[4] Debian Manpages. “persistence.conf(5).” live-boot-doc, Trixie. https://manpages.debian.org/trixie/live-boot-doc/persistence.conf.5.en.html

[5] Debian Live Team. “Customizing Run-Time Behaviours.” Debian Live Manual. https://live-team.pages.debian.net/live-manual/html/live-manual/customizing-run-time-behaviours.en.html

[6] Debian Manpages. “live-boot(7).” live-boot-doc. https://manpages.debian.org/unstable/live-boot-doc/live-boot.7.en.html

[7] UEFI Forum. “Boot Manager.” UEFI Specification 2.10, §3. https://uefi.org/specs/UEFI/2.10/03_Boot_Manager.html

[8] ArchWiki. “Unified Extensible Firmware Interface.” https://wiki.archlinux.org/title/Unified_Extensible_Firmware_Interface

[9] Debian User Mailing List. “Error message from GParted, means what?” November 2016. https://lists.debian.org/debian-user/2016/11/msg01017.html

<references />
''Contributed by [https://forums.debian.net/search.php?keywords=distro-nix distro-nix] on Debian User Forums on 2026-04-06.'' Comments, corrections, suggestions or resources are welcome.

Debian Live USB with Persistence

2026-04-10T02:58:23Z

Distro-nix: /* Verify the Base Boot */ Return to 4

All commands are verified on Debian 13.4 (Trixie) / 6.12.74+deb13+1-amd64

Created: 2026-04-04

Updated: 2026-04-09 - added references and restructured heading order;

ID: 005146.11

'''''A Debian Live USB with persistence boots a full Debian environment from a USB drive while retaining changes across sessions. Files you create, settings you change, and packages you install survive reboot—stored on a dedicated partition rather than the volatile live environment. Nothing is written to the host machine’s internal disk.'''''

'''''A standard live USB is stateless: each boot starts from the same ISO image. The persistence partition changes that. From a single drive you get your shell configuration, documents, utilities, browser profile, and scripts, available on any machine you boot it from.'''''

== What You’ll Need ==
A USB drive of at least 8 GB—the minimum is 4 GB, but that leaves little room for persistence. A 32 GB drive is a practical working size. Download the Debian Live ISO from the official Debian website: <nowiki>https://www.debian.org/CD/live</nowiki>

The tools required—dd, fdisk, and mkfs.ext4—are standard on Debian. On a minimal live environment, fdisk and mkfs.ext4 may not be present. Install them before starting:

<code>$ sudo apt install fdisk e2fsprogs</code>

These packages are available from the live session’s apt cache without adding sources. They don’t persist to the USB unless persistence is already active.

== Identifying Your USB Device ==
Plug in the USB drive and run:

<code>$ lsblk</code>

Your USB device will appear as /dev/sdb, /dev/sdc, or similar, depending on what else is attached. Confirm it by cross-referencing the reported size with the known capacity of the drive.

Two placeholders are used throughout this document. /dev/sdX refers to the device itself—substitute your actual device identifier everywhere it appears. /dev/sdXN refers to the persistence partition—N is the partition number you’ll confirm from lsblk output after you've created it.

Use the device path, not a partition path: /dev/sdb is correct; /dev/sdb1 is not.

== Boot Entry Selection ==
The firmware boot menu on many systems presents two entries for a USB device: one labeled with the device name alone, and one prefixed with UEFI:. On the hardware used for this document, selecting the UEFI:-prefixed entry after adding a persistence partition consistently produced a bare grub> prompt rather than the Debian live menu. Selecting the plain device entry booted correctly every time.

Throughout this procedure, select the plain device entry—not the one prefixed with UEFI:.

This behavior was observed on specific hardware. Whether it reproduces on all firmware is not established here. The mechanism and a possible explanation are in <ref>https://uefi.org/specs/UEFI/2.10/03_Boot_Manager.html</ref> If your firmware presents only a UEFI boot path, recovery from within a booted live session may be possible by correcting the GRUB prefix, but that procedure is outside the scope of this document.

= Procedure =

=== Clean the Drive ===
If the USB has been used before, residual partition signatures or bootloader data in the early sectors can interfere with the ISO write or the subsequent boot. Clear them before starting:

<code>$ sudo wipefs -a /dev/sdX</code>

<code>$ sudo dd if=/dev/zero of=/dev/sdX bs=1M count=200</code>

wipefs removes filesystem and partition table signatures. The dd command zeros the first 200 MB, covering the MBR, GPT header, and any EFI data in the early sectors.

=== Write the ISO ===
Before running dd, confirm that sync resolves to the system binary and not a shell alias:

<code>$ type sync</code>

The output should be sync is /usr/bin/sync. If it returns an alias definition instead, unalias it with unalias sync, or substitute /usr/bin/sync explicitly at the end of the dd command.

Write the ISO:

<code>$ sudo dd if=/path/to/debian-live-13.4.0-amd64-kde.iso of=/dev/sdX bs=4M status=progress && sync</code>

status=progress shows the running transfer rate. sync flushes the write cache on completion—don’t remove the drive before it returns. The write will take a few minutes depending on the drive’s speed.

=== Verify the Base Boot ===
Before adding the persistence partition, confirm the ISO write produced a bootable drive. Boot from the USB, selecting the plain (non-UEFI:) device entry. If it reaches the Debian live menu and boots cleanly, you have a confirmed working baseline. If it doesn’t boot at this stage, don’t continue—return to [[Debian Live USB with Persistence#Procedure|4]]

== Create the Persistence Partition ==
The ISO write leaves unallocated space at the end of the drive. The persistence partition goes there. The Debian live ISO uses an isohybrid layout—a hybrid MBR/GPT partition table that supports both BIOS and UEFI boot paths. Adding a partition requires an explicit start sector; without it, fdisk’s default placement can conflict with the hybrid structure in ways that affect the boot. The failure modes of parted and gdisk with this layout are documented in §4.9

Launch fdisk:

<code>$ sudo fdisk /dev/sdX</code>

Print the current partition table:

Command (m for help): <code>p</code>

Two partitions will be present: the main ISO partition (type 0 or Empty—this is expected for an isohybrid layout) and a small EFI partition of approximately 3.2 MB. The p output shows the end sector of the last partition in the End column. The persistence partition must start at the next sector—that value plus one. On the test hardware with a Debian 13.4 live ISO written to the drive, the last partition ended at sector 8175999 and the persistence partition started at sector 8176000. This figure will vary with the ISO version and release; read your own p output and don’t use the test value without verifying it matches.

Create the new partition:

Command (m for help): <code>n</code>

When fdisk prompts for the first sector, enter the start sector you identified from the p output—do not accept the default. Accept the default for the last sector to use the remaining space on the drive.

If fdisk asks whether to remove an existing ext4 signature, answer <code>Y</code>. This clears a remnant from a previous attempt.

Write the changes and exit:

Command (m for help): <code>w</code>

After fdisk writes the new partition table, the running kernel may not immediately recognize the change. partprobe signals the kernel to re-read the partition table on the device without requiring a reboot:

<code>$ sudo partprobe /dev/sdX</code>

Once it completes, confirm the new partition is visible:

<code>$ lsblk /dev/sdX</code>

Your new partition will appear beneath the device in the output. Note its full identifier—for example, sdb3 or sdc4—as you’ll need it in the next two steps. If it doesn’t appear immediately, wait a moment and run lsblk again before proceeding.

== Format the Partition ==
Format the new partition as ext4 with the label persistence. The Debian live system identifies the persistence partition by this label at boot:

<code>$ sudo mkfs.ext4 -L persistence /dev/sdXN</code>

== Create the Persistence Configuration ==
Mount the partition:

<code>$ sudo mount /dev/sdXN /mnt</code>

Create the configuration file:

<code>$ echo "/ union" | sudo tee /mnt/persistence.conf</code>

The / union directive instructs the live system to mount the entire filesystem as a union overlay. Changes made during a session are written to the persistence partition rather than discarded on shutdown.

Unmount the partition:

<code>$ sudo umount /mnt</code>

== Boot with Persistence ==
Boot from the USB, selecting the plain (non-UEFI:) device entry in the firmware boot menu. At the Debian live GRUB menu, press Tab to edit the selected boot entry. On the line beginning with linux, append persistence after boot=live components. Press Enter to boot.

The first boot with persistence active will be slower than a standard live session—the system is running entirely from the USB drive, and the persistence overlay adds a layer of I/O. This is normal.

== Verify Persistence ==
After booting, confirm the persistence partition is mounted:

<code>$ lsblk -f</code>

The output should show sdXN mounted at /run/live/persistence/sdXN.

Create a test file:

<code>$ touch ~/persistence-test.txt</code>

Reboot with persistence enabled. If the file is present after reboot, persistence is working.

== What Didn’t Work, and Why ==

=== parted ===
parted misreported the capacity of the test USB drive: a 32 GB device was reported as approximately 125 GB. Determining where the free space began or how large to make the persistence partition was not possible from that output. parted was not used further. The broader behavior—libparted-based tools handling the isohybrid format unreliably—is documented on the Debian User mailing list [9].

=== gdisk / sgdisk ===
gdisk and sgdisk are GPT-aware tools and would seem the logical choice for a drive with a hybrid partition table. On the test USB, sgdisk produced:<blockquote>Found invalid GPT and valid MBR; converting MBR to GPT format in memory.

Warning! Secondary partition table overlaps the last partition by 33 blocks!

Could not create partition 3 from 14580 to 8177663

Non-GPT disk; not saving changes. Use -g to override.

Error encountered; not saving changes.</blockquote>The isohybrid layout places the secondary GPT header in a location that overlaps the last partition. sgdisk identifies this as invalid and refuses to proceed. Using -g to override would corrupt the hybrid layout and break the boot. sgdisk can’t be used here.

cp was also tested as an alternative to dd for writing the ISO, on the theory that it might handle the isohybrid structure differently. It produced an identical partition layout. It’s not an alternative—it’s a slower write with the same result.

== The UEFI grub> Prompt ==
On the test hardware, adding a persistence partition and then selecting the UEFI:-prefixed boot entry consistently dropped to a bare grub> prompt. Running ls from that prompt showed the internal drive’s partitions; the USB’s live partition was unreachable from there.

What was observed: the USB’s own GRUB appeared briefly—visible as a rapid flash on screen with a “no such…” error—then failed, after which the firmware handed control to the internal drive’s GRUB, which presented its own grub> prompt.

The UEFI specification states that firmware may remove boot options it considers invalid [7]; the ArchWiki documents firmware removing entries for drives it detects as changed as a known occurrence [8]. After adding the persistence partition, the USB’s EFI entry—pointing to EFI/boot/bootx64.efi—may no longer have been treated as valid by the firmware, causing it to fall through to the next available bootloader on the internal drive. Whether modifying the partition table on a removable device is sufficient to trigger this behavior, and under what firmware conditions, wasn’t reproducible in a controlled way during testing.

Selecting the plain (non-UEFI:) device entry uses the legacy MBR boot path, which loads the USB’s own GRUB directly and has no dependency on the UEFI NVRAM or EFI boot entries. On the test hardware, this path was unaffected by partition table changes.

== Anonymity ==
A Debian Live USB with persistence, used correctly, writes nothing to the host machine’s internal disk. It doesn’t modify the installed OS, create logs on the host filesystem, or leave traces in the usual locations.

It doesn’t provide anonymity. Boot events may be logged by firmware or enterprise management systems. Network activity is visible to routers, ISPs, and network administrators regardless of which OS is running. Mounting the internal disk during a live session makes its contents accessible to the live environment, and any writes to it are permanent. A live USB is a portable, self-contained Debian environment—it isn’t a privacy tool by design.

== A Note on Carrying Data ==
The persistence partition is a practical place to keep working data alongside the system state—documents, scripts, configuration files, anything you’d normally keep in ~/Documents or ~/bin. It travels with the drive.

Installed packages persist across sessions but may have hardware-specific dependencies. For general desktop use this isn’t usually an issue, but it’s worth keeping in mind if you’re installing drivers or kernel modules that assume specific hardware.

== Citations ==
Syslinux Project. “Isohybrid.” Syslinux Wiki.

[2] Linux Command Library. “isohybrid(1) man page.” https://linuxcommandlibrary.com/man/isohybrid

[3] Smith, Roderick W. “Hybrid MBRs.” GPT fdisk documentation. https://www.rodsbooks.com/gdisk/hybrid.html

[4] Debian Manpages. “persistence.conf(5).” live-boot-doc, Trixie. https://manpages.debian.org/trixie/live-boot-doc/persistence.conf.5.en.html

[5] Debian Live Team. “Customizing Run-Time Behaviours.” Debian Live Manual. https://live-team.pages.debian.net/live-manual/html/live-manual/customizing-run-time-behaviours.en.html

[6] Debian Manpages. “live-boot(7).” live-boot-doc. https://manpages.debian.org/unstable/live-boot-doc/live-boot.7.en.html

[7] UEFI Forum. “Boot Manager.” UEFI Specification 2.10, §3. https://uefi.org/specs/UEFI/2.10/03_Boot_Manager.html

[8] ArchWiki. “Unified Extensible Firmware Interface.” https://wiki.archlinux.org/title/Unified_Extensible_Firmware_Interface

[9] Debian User Mailing List. “Error message from GParted, means what?” November 2016. https://lists.debian.org/debian-user/2016/11/msg01017.html

<references />
''Contributed by [https://forums.debian.net/search.php?keywords=distro-nix distro-nix] on Debian User Forums on 2026-04-06.'' Comments, corrections, suggestions or resources are welcome.

Debian Live USB with Persistence

2026-04-10T02:52:56Z

Distro-nix: Moved footer position

All commands are verified on Debian 13.4 (Trixie) / 6.12.74+deb13+1-amd64

Created: 2026-04-04

Updated: 2026-04-09 - added references and restructured heading order;

ID: 005146.11

'''''A Debian Live USB with persistence boots a full Debian environment from a USB drive while retaining changes across sessions. Files you create, settings you change, and packages you install survive reboot—stored on a dedicated partition rather than the volatile live environment. Nothing is written to the host machine’s internal disk.'''''

'''''A standard live USB is stateless: each boot starts from the same ISO image. The persistence partition changes that. From a single drive you get your shell configuration, documents, utilities, browser profile, and scripts, available on any machine you boot it from.'''''

== What You’ll Need ==
A USB drive of at least 8 GB—the minimum is 4 GB, but that leaves little room for persistence. A 32 GB drive is a practical working size. Download the Debian Live ISO from the official Debian website: <nowiki>https://www.debian.org/CD/live</nowiki>

The tools required—dd, fdisk, and mkfs.ext4—are standard on Debian. On a minimal live environment, fdisk and mkfs.ext4 may not be present. Install them before starting:

<code>$ sudo apt install fdisk e2fsprogs</code>

These packages are available from the live session’s apt cache without adding sources. They don’t persist to the USB unless persistence is already active.

== Identifying Your USB Device ==
Plug in the USB drive and run:

<code>$ lsblk</code>

Your USB device will appear as /dev/sdb, /dev/sdc, or similar, depending on what else is attached. Confirm it by cross-referencing the reported size with the known capacity of the drive.

Two placeholders are used throughout this document. /dev/sdX refers to the device itself—substitute your actual device identifier everywhere it appears. /dev/sdXN refers to the persistence partition—N is the partition number you’ll confirm from lsblk output after you've created it.

Use the device path, not a partition path: /dev/sdb is correct; /dev/sdb1 is not.

== Boot Entry Selection ==
The firmware boot menu on many systems presents two entries for a USB device: one labeled with the device name alone, and one prefixed with UEFI:. On the hardware used for this document, selecting the UEFI:-prefixed entry after adding a persistence partition consistently produced a bare grub> prompt rather than the Debian live menu. Selecting the plain device entry booted correctly every time.

Throughout this procedure, select the plain device entry—not the one prefixed with UEFI:.

This behavior was observed on specific hardware. Whether it reproduces on all firmware is not established here. The mechanism and a possible explanation are in <ref>https://uefi.org/specs/UEFI/2.10/03_Boot_Manager.html</ref> If your firmware presents only a UEFI boot path, recovery from within a booted live session may be possible by correcting the GRUB prefix, but that procedure is outside the scope of this document.

= Procedure =

=== Clean the Drive ===
If the USB has been used before, residual partition signatures or bootloader data in the early sectors can interfere with the ISO write or the subsequent boot. Clear them before starting:

<code>$ sudo wipefs -a /dev/sdX</code>

<code>$ sudo dd if=/dev/zero of=/dev/sdX bs=1M count=200</code>

wipefs removes filesystem and partition table signatures. The dd command zeros the first 200 MB, covering the MBR, GPT header, and any EFI data in the early sectors.

=== Write the ISO ===
Before running dd, confirm that sync resolves to the system binary and not a shell alias:

<code>$ type sync</code>

The output should be sync is /usr/bin/sync. If it returns an alias definition instead, unalias it with unalias sync, or substitute /usr/bin/sync explicitly at the end of the dd command.

Write the ISO:

<code>$ sudo dd if=/path/to/debian-live-13.4.0-amd64-kde.iso of=/dev/sdX bs=4M status=progress && sync</code>

status=progress shows the running transfer rate. sync flushes the write cache on completion—don’t remove the drive before it returns. The write will take a few minutes depending on the drive’s speed.

=== Verify the Base Boot ===
Before adding the persistence partition, confirm the ISO write produced a bootable drive. Boot from the USB, selecting the plain (non-UEFI:) device entry. If it reaches the Debian live menu and boots cleanly, you have a confirmed working baseline. If it doesn’t boot at this stage, don’t continue—repeat [[Debian Live USB with Persistence#Procedure|4]]

== Create the Persistence Partition ==
The ISO write leaves unallocated space at the end of the drive. The persistence partition goes there. The Debian live ISO uses an isohybrid layout—a hybrid MBR/GPT partition table that supports both BIOS and UEFI boot paths. Adding a partition requires an explicit start sector; without it, fdisk’s default placement can conflict with the hybrid structure in ways that affect the boot. The failure modes of parted and gdisk with this layout are documented in §4.9

Launch fdisk:

<code>$ sudo fdisk /dev/sdX</code>

Print the current partition table:

Command (m for help): <code>p</code>

Two partitions will be present: the main ISO partition (type 0 or Empty—this is expected for an isohybrid layout) and a small EFI partition of approximately 3.2 MB. The p output shows the end sector of the last partition in the End column. The persistence partition must start at the next sector—that value plus one. On the test hardware with a Debian 13.4 live ISO written to the drive, the last partition ended at sector 8175999 and the persistence partition started at sector 8176000. This figure will vary with the ISO version and release; read your own p output and don’t use the test value without verifying it matches.

Create the new partition:

Command (m for help): <code>n</code>

When fdisk prompts for the first sector, enter the start sector you identified from the p output—do not accept the default. Accept the default for the last sector to use the remaining space on the drive.

If fdisk asks whether to remove an existing ext4 signature, answer <code>Y</code>. This clears a remnant from a previous attempt.

Write the changes and exit:

Command (m for help): <code>w</code>

After fdisk writes the new partition table, the running kernel may not immediately recognize the change. partprobe signals the kernel to re-read the partition table on the device without requiring a reboot:

<code>$ sudo partprobe /dev/sdX</code>

Once it completes, confirm the new partition is visible:

<code>$ lsblk /dev/sdX</code>

Your new partition will appear beneath the device in the output. Note its full identifier—for example, sdb3 or sdc4—as you’ll need it in the next two steps. If it doesn’t appear immediately, wait a moment and run lsblk again before proceeding.

== Format the Partition ==
Format the new partition as ext4 with the label persistence. The Debian live system identifies the persistence partition by this label at boot:

<code>$ sudo mkfs.ext4 -L persistence /dev/sdXN</code>

== Create the Persistence Configuration ==
Mount the partition:

<code>$ sudo mount /dev/sdXN /mnt</code>

Create the configuration file:

<code>$ echo "/ union" | sudo tee /mnt/persistence.conf</code>

The / union directive instructs the live system to mount the entire filesystem as a union overlay. Changes made during a session are written to the persistence partition rather than discarded on shutdown.

Unmount the partition:

<code>$ sudo umount /mnt</code>

== Boot with Persistence ==
Boot from the USB, selecting the plain (non-UEFI:) device entry in the firmware boot menu. At the Debian live GRUB menu, press Tab to edit the selected boot entry. On the line beginning with linux, append persistence after boot=live components. Press Enter to boot.

The first boot with persistence active will be slower than a standard live session—the system is running entirely from the USB drive, and the persistence overlay adds a layer of I/O. This is normal.

== Verify Persistence ==
After booting, confirm the persistence partition is mounted:

<code>$ lsblk -f</code>

The output should show sdXN mounted at /run/live/persistence/sdXN.

Create a test file:

<code>$ touch ~/persistence-test.txt</code>

Reboot with persistence enabled. If the file is present after reboot, persistence is working.

== What Didn’t Work, and Why ==

=== parted ===
parted misreported the capacity of the test USB drive: a 32 GB device was reported as approximately 125 GB. Determining where the free space began or how large to make the persistence partition was not possible from that output. parted was not used further. The broader behavior—libparted-based tools handling the isohybrid format unreliably—is documented on the Debian User mailing list [9].

=== gdisk / sgdisk ===
gdisk and sgdisk are GPT-aware tools and would seem the logical choice for a drive with a hybrid partition table. On the test USB, sgdisk produced:<blockquote>Found invalid GPT and valid MBR; converting MBR to GPT format in memory.

Warning! Secondary partition table overlaps the last partition by 33 blocks!

Could not create partition 3 from 14580 to 8177663

Non-GPT disk; not saving changes. Use -g to override.

Error encountered; not saving changes.</blockquote>The isohybrid layout places the secondary GPT header in a location that overlaps the last partition. sgdisk identifies this as invalid and refuses to proceed. Using -g to override would corrupt the hybrid layout and break the boot. sgdisk can’t be used here.

cp was also tested as an alternative to dd for writing the ISO, on the theory that it might handle the isohybrid structure differently. It produced an identical partition layout. It’s not an alternative—it’s a slower write with the same result.

== The UEFI grub> Prompt ==
On the test hardware, adding a persistence partition and then selecting the UEFI:-prefixed boot entry consistently dropped to a bare grub> prompt. Running ls from that prompt showed the internal drive’s partitions; the USB’s live partition was unreachable from there.

What was observed: the USB’s own GRUB appeared briefly—visible as a rapid flash on screen with a “no such…” error—then failed, after which the firmware handed control to the internal drive’s GRUB, which presented its own grub> prompt.

The UEFI specification states that firmware may remove boot options it considers invalid [7]; the ArchWiki documents firmware removing entries for drives it detects as changed as a known occurrence [8]. After adding the persistence partition, the USB’s EFI entry—pointing to EFI/boot/bootx64.efi—may no longer have been treated as valid by the firmware, causing it to fall through to the next available bootloader on the internal drive. Whether modifying the partition table on a removable device is sufficient to trigger this behavior, and under what firmware conditions, wasn’t reproducible in a controlled way during testing.

Selecting the plain (non-UEFI:) device entry uses the legacy MBR boot path, which loads the USB’s own GRUB directly and has no dependency on the UEFI NVRAM or EFI boot entries. On the test hardware, this path was unaffected by partition table changes.

== Anonymity ==
A Debian Live USB with persistence, used correctly, writes nothing to the host machine’s internal disk. It doesn’t modify the installed OS, create logs on the host filesystem, or leave traces in the usual locations.

It doesn’t provide anonymity. Boot events may be logged by firmware or enterprise management systems. Network activity is visible to routers, ISPs, and network administrators regardless of which OS is running. Mounting the internal disk during a live session makes its contents accessible to the live environment, and any writes to it are permanent. A live USB is a portable, self-contained Debian environment—it isn’t a privacy tool by design.

== A Note on Carrying Data ==
The persistence partition is a practical place to keep working data alongside the system state—documents, scripts, configuration files, anything you’d normally keep in ~/Documents or ~/bin. It travels with the drive.

Installed packages persist across sessions but may have hardware-specific dependencies. For general desktop use this isn’t usually an issue, but it’s worth keeping in mind if you’re installing drivers or kernel modules that assume specific hardware.

== Citations ==
Syslinux Project. “Isohybrid.” Syslinux Wiki.

[2] Linux Command Library. “isohybrid(1) man page.” https://linuxcommandlibrary.com/man/isohybrid

[3] Smith, Roderick W. “Hybrid MBRs.” GPT fdisk documentation. https://www.rodsbooks.com/gdisk/hybrid.html

[4] Debian Manpages. “persistence.conf(5).” live-boot-doc, Trixie. https://manpages.debian.org/trixie/live-boot-doc/persistence.conf.5.en.html

[5] Debian Live Team. “Customizing Run-Time Behaviours.” Debian Live Manual. https://live-team.pages.debian.net/live-manual/html/live-manual/customizing-run-time-behaviours.en.html

[6] Debian Manpages. “live-boot(7).” live-boot-doc. https://manpages.debian.org/unstable/live-boot-doc/live-boot.7.en.html

[7] UEFI Forum. “Boot Manager.” UEFI Specification 2.10, §3. https://uefi.org/specs/UEFI/2.10/03_Boot_Manager.html

[8] ArchWiki. “Unified Extensible Firmware Interface.” https://wiki.archlinux.org/title/Unified_Extensible_Firmware_Interface

[9] Debian User Mailing List. “Error message from GParted, means what?” November 2016. https://lists.debian.org/debian-user/2016/11/msg01017.html

<references />
''Contributed by [https://forums.debian.net/search.php?keywords=distro-nix distro-nix] on Debian User Forums on 2026-04-06.'' Comments, corrections, suggestions or resources are welcome.

Debian Live USB with Persistence

2026-04-10T02:48:41Z

Distro-nix: /* Verify the Base Boot */ Inserted link to 4

All commands are verified on Debian 13.4 (Trixie) / 6.12.74+deb13+1-amd64

Created: 2026-04-04

Updated: 2026-04-09 - added references and restructured heading order;

ID: 005146.11

'''''A Debian Live USB with persistence boots a full Debian environment from a USB drive while retaining changes across sessions. Files you create, settings you change, and packages you install survive reboot—stored on a dedicated partition rather than the volatile live environment. Nothing is written to the host machine’s internal disk.'''''

'''''A standard live USB is stateless: each boot starts from the same ISO image. The persistence partition changes that. From a single drive you get your shell configuration, documents, utilities, browser profile, and scripts, available on any machine you boot it from.'''''

== What You’ll Need ==
A USB drive of at least 8 GB—the minimum is 4 GB, but that leaves little room for persistence. A 32 GB drive is a practical working size. Download the Debian Live ISO from the official Debian website: <nowiki>https://www.debian.org/CD/live</nowiki>

The tools required—dd, fdisk, and mkfs.ext4—are standard on Debian. On a minimal live environment, fdisk and mkfs.ext4 may not be present. Install them before starting:

<code>$ sudo apt install fdisk e2fsprogs</code>

These packages are available from the live session’s apt cache without adding sources. They don’t persist to the USB unless persistence is already active.

== Identifying Your USB Device ==
Plug in the USB drive and run:

<code>$ lsblk</code>

Your USB device will appear as /dev/sdb, /dev/sdc, or similar, depending on what else is attached. Confirm it by cross-referencing the reported size with the known capacity of the drive.

Two placeholders are used throughout this document. /dev/sdX refers to the device itself—substitute your actual device identifier everywhere it appears. /dev/sdXN refers to the persistence partition—N is the partition number you’ll confirm from lsblk output after you've created it.

Use the device path, not a partition path: /dev/sdb is correct; /dev/sdb1 is not.

== Boot Entry Selection ==
The firmware boot menu on many systems presents two entries for a USB device: one labeled with the device name alone, and one prefixed with UEFI:. On the hardware used for this document, selecting the UEFI:-prefixed entry after adding a persistence partition consistently produced a bare grub> prompt rather than the Debian live menu. Selecting the plain device entry booted correctly every time.

Throughout this procedure, select the plain device entry—not the one prefixed with UEFI:.

This behavior was observed on specific hardware. Whether it reproduces on all firmware is not established here. The mechanism and a possible explanation are in <ref>https://uefi.org/specs/UEFI/2.10/03_Boot_Manager.html</ref> If your firmware presents only a UEFI boot path, recovery from within a booted live session may be possible by correcting the GRUB prefix, but that procedure is outside the scope of this document.

= Procedure =

=== Clean the Drive ===
If the USB has been used before, residual partition signatures or bootloader data in the early sectors can interfere with the ISO write or the subsequent boot. Clear them before starting:

<code>$ sudo wipefs -a /dev/sdX</code>

<code>$ sudo dd if=/dev/zero of=/dev/sdX bs=1M count=200</code>

wipefs removes filesystem and partition table signatures. The dd command zeros the first 200 MB, covering the MBR, GPT header, and any EFI data in the early sectors.

=== Write the ISO ===
Before running dd, confirm that sync resolves to the system binary and not a shell alias:

<code>$ type sync</code>

The output should be sync is /usr/bin/sync. If it returns an alias definition instead, unalias it with unalias sync, or substitute /usr/bin/sync explicitly at the end of the dd command.

Write the ISO:

<code>$ sudo dd if=/path/to/debian-live-13.4.0-amd64-kde.iso of=/dev/sdX bs=4M status=progress && sync</code>

status=progress shows the running transfer rate. sync flushes the write cache on completion—don’t remove the drive before it returns. The write will take a few minutes depending on the drive’s speed.

=== Verify the Base Boot ===
Before adding the persistence partition, confirm the ISO write produced a bootable drive. Boot from the USB, selecting the plain (non-UEFI:) device entry. If it reaches the Debian live menu and boots cleanly, you have a confirmed working baseline. If it doesn’t boot at this stage, don’t continue—repeat [[Debian Live USB with Persistence#Procedure|4]]

== Create the Persistence Partition ==
The ISO write leaves unallocated space at the end of the drive. The persistence partition goes there. The Debian live ISO uses an isohybrid layout—a hybrid MBR/GPT partition table that supports both BIOS and UEFI boot paths. Adding a partition requires an explicit start sector; without it, fdisk’s default placement can conflict with the hybrid structure in ways that affect the boot. The failure modes of parted and gdisk with this layout are documented in §4.9

Launch fdisk:

<code>$ sudo fdisk /dev/sdX</code>

Print the current partition table:

Command (m for help): <code>p</code>

Two partitions will be present: the main ISO partition (type 0 or Empty—this is expected for an isohybrid layout) and a small EFI partition of approximately 3.2 MB. The p output shows the end sector of the last partition in the End column. The persistence partition must start at the next sector—that value plus one. On the test hardware with a Debian 13.4 live ISO written to the drive, the last partition ended at sector 8175999 and the persistence partition started at sector 8176000. This figure will vary with the ISO version and release; read your own p output and don’t use the test value without verifying it matches.

Create the new partition:

Command (m for help): <code>n</code>

When fdisk prompts for the first sector, enter the start sector you identified from the p output—do not accept the default. Accept the default for the last sector to use the remaining space on the drive.

If fdisk asks whether to remove an existing ext4 signature, answer <code>Y</code>. This clears a remnant from a previous attempt.

Write the changes and exit:

Command (m for help): <code>w</code>

After fdisk writes the new partition table, the running kernel may not immediately recognize the change. partprobe signals the kernel to re-read the partition table on the device without requiring a reboot:

<code>$ sudo partprobe /dev/sdX</code>

Once it completes, confirm the new partition is visible:

<code>$ lsblk /dev/sdX</code>

Your new partition will appear beneath the device in the output. Note its full identifier—for example, sdb3 or sdc4—as you’ll need it in the next two steps. If it doesn’t appear immediately, wait a moment and run lsblk again before proceeding.

== Format the Partition ==
Format the new partition as ext4 with the label persistence. The Debian live system identifies the persistence partition by this label at boot:

<code>$ sudo mkfs.ext4 -L persistence /dev/sdXN</code>

== Create the Persistence Configuration ==
Mount the partition:

<code>$ sudo mount /dev/sdXN /mnt</code>

Create the configuration file:

<code>$ echo "/ union" | sudo tee /mnt/persistence.conf</code>

The / union directive instructs the live system to mount the entire filesystem as a union overlay. Changes made during a session are written to the persistence partition rather than discarded on shutdown.

Unmount the partition:

<code>$ sudo umount /mnt</code>

== Boot with Persistence ==
Boot from the USB, selecting the plain (non-UEFI:) device entry in the firmware boot menu. At the Debian live GRUB menu, press Tab to edit the selected boot entry. On the line beginning with linux, append persistence after boot=live components. Press Enter to boot.

The first boot with persistence active will be slower than a standard live session—the system is running entirely from the USB drive, and the persistence overlay adds a layer of I/O. This is normal.

== Verify Persistence ==
After booting, confirm the persistence partition is mounted:

<code>$ lsblk -f</code>

The output should show sdXN mounted at /run/live/persistence/sdXN.

Create a test file:

<code>$ touch ~/persistence-test.txt</code>

Reboot with persistence enabled. If the file is present after reboot, persistence is working.

== What Didn’t Work, and Why ==

=== parted ===
parted misreported the capacity of the test USB drive: a 32 GB device was reported as approximately 125 GB. Determining where the free space began or how large to make the persistence partition was not possible from that output. parted was not used further. The broader behavior—libparted-based tools handling the isohybrid format unreliably—is documented on the Debian User mailing list [9].

=== gdisk / sgdisk ===
gdisk and sgdisk are GPT-aware tools and would seem the logical choice for a drive with a hybrid partition table. On the test USB, sgdisk produced:<blockquote>Found invalid GPT and valid MBR; converting MBR to GPT format in memory.

Warning! Secondary partition table overlaps the last partition by 33 blocks!

Could not create partition 3 from 14580 to 8177663

Non-GPT disk; not saving changes. Use -g to override.

Error encountered; not saving changes.</blockquote>The isohybrid layout places the secondary GPT header in a location that overlaps the last partition. sgdisk identifies this as invalid and refuses to proceed. Using -g to override would corrupt the hybrid layout and break the boot. sgdisk can’t be used here.

cp was also tested as an alternative to dd for writing the ISO, on the theory that it might handle the isohybrid structure differently. It produced an identical partition layout. It’s not an alternative—it’s a slower write with the same result.

== The UEFI grub> Prompt ==
On the test hardware, adding a persistence partition and then selecting the UEFI:-prefixed boot entry consistently dropped to a bare grub> prompt. Running ls from that prompt showed the internal drive’s partitions; the USB’s live partition was unreachable from there.

What was observed: the USB’s own GRUB appeared briefly—visible as a rapid flash on screen with a “no such…” error—then failed, after which the firmware handed control to the internal drive’s GRUB, which presented its own grub> prompt.

The UEFI specification states that firmware may remove boot options it considers invalid [7]; the ArchWiki documents firmware removing entries for drives it detects as changed as a known occurrence [8]. After adding the persistence partition, the USB’s EFI entry—pointing to EFI/boot/bootx64.efi—may no longer have been treated as valid by the firmware, causing it to fall through to the next available bootloader on the internal drive. Whether modifying the partition table on a removable device is sufficient to trigger this behavior, and under what firmware conditions, wasn’t reproducible in a controlled way during testing.

Selecting the plain (non-UEFI:) device entry uses the legacy MBR boot path, which loads the USB’s own GRUB directly and has no dependency on the UEFI NVRAM or EFI boot entries. On the test hardware, this path was unaffected by partition table changes.

== Anonymity ==
A Debian Live USB with persistence, used correctly, writes nothing to the host machine’s internal disk. It doesn’t modify the installed OS, create logs on the host filesystem, or leave traces in the usual locations.

It doesn’t provide anonymity. Boot events may be logged by firmware or enterprise management systems. Network activity is visible to routers, ISPs, and network administrators regardless of which OS is running. Mounting the internal disk during a live session makes its contents accessible to the live environment, and any writes to it are permanent. A live USB is a portable, self-contained Debian environment—it isn’t a privacy tool by design.

== A Note on Carrying Data ==
The persistence partition is a practical place to keep working data alongside the system state—documents, scripts, configuration files, anything you’d normally keep in ~/Documents or ~/bin. It travels with the drive.

Installed packages persist across sessions but may have hardware-specific dependencies. For general desktop use this isn’t usually an issue, but it’s worth keeping in mind if you’re installing drivers or kernel modules that assume specific hardware.

== Citations ==
Syslinux Project. “Isohybrid.” Syslinux Wiki.

[2] Linux Command Library. “isohybrid(1) man page.” https://linuxcommandlibrary.com/man/isohybrid

[3] Smith, Roderick W. “Hybrid MBRs.” GPT fdisk documentation. https://www.rodsbooks.com/gdisk/hybrid.html

[4] Debian Manpages. “persistence.conf(5).” live-boot-doc, Trixie. https://manpages.debian.org/trixie/live-boot-doc/persistence.conf.5.en.html

[5] Debian Live Team. “Customizing Run-Time Behaviours.” Debian Live Manual. https://live-team.pages.debian.net/live-manual/html/live-manual/customizing-run-time-behaviours.en.html

[6] Debian Manpages. “live-boot(7).” live-boot-doc. https://manpages.debian.org/unstable/live-boot-doc/live-boot.7.en.html

[7] UEFI Forum. “Boot Manager.” UEFI Specification 2.10, §3. https://uefi.org/specs/UEFI/2.10/03_Boot_Manager.html

[8] ArchWiki. “Unified Extensible Firmware Interface.” https://wiki.archlinux.org/title/Unified_Extensible_Firmware_Interface

[9] Debian User Mailing List. “Error message from GParted, means what?” November 2016. https://lists.debian.org/debian-user/2016/11/msg01017.html

''Contributed by [https://forums.debian.net/search.php?keywords=distro-nix distro-nix] on Debian User Forums on 2026-04-06.'' Comments, corrections, suggestions or resources are welcome.

Debian Live USB with Persistence

2026-04-10T02:38:27Z

Distro-nix: Added citation

All commands are verified on Debian 13.4 (Trixie) / 6.12.74+deb13+1-amd64

Created: 2026-04-04

Updated: 2026-04-09 - added references and restructured heading order;

ID: 005146.11

'''''A Debian Live USB with persistence boots a full Debian environment from a USB drive while retaining changes across sessions. Files you create, settings you change, and packages you install survive reboot—stored on a dedicated partition rather than the volatile live environment. Nothing is written to the host machine’s internal disk.'''''

'''''A standard live USB is stateless: each boot starts from the same ISO image. The persistence partition changes that. From a single drive you get your shell configuration, documents, utilities, browser profile, and scripts, available on any machine you boot it from.'''''

== What You’ll Need ==
A USB drive of at least 8 GB—the minimum is 4 GB, but that leaves little room for persistence. A 32 GB drive is a practical working size. Download the Debian Live ISO from the official Debian website: <nowiki>https://www.debian.org/CD/live</nowiki>

The tools required—dd, fdisk, and mkfs.ext4—are standard on Debian. On a minimal live environment, fdisk and mkfs.ext4 may not be present. Install them before starting:

<code>$ sudo apt install fdisk e2fsprogs</code>

These packages are available from the live session’s apt cache without adding sources. They don’t persist to the USB unless persistence is already active.

== Identifying Your USB Device ==
Plug in the USB drive and run:

<code>$ lsblk</code>

Your USB device will appear as /dev/sdb, /dev/sdc, or similar, depending on what else is attached. Confirm it by cross-referencing the reported size with the known capacity of the drive.

Two placeholders are used throughout this document. /dev/sdX refers to the device itself—substitute your actual device identifier everywhere it appears. /dev/sdXN refers to the persistence partition—N is the partition number you’ll confirm from lsblk output after you've created it.

Use the device path, not a partition path: /dev/sdb is correct; /dev/sdb1 is not.

== Boot Entry Selection ==
The firmware boot menu on many systems presents two entries for a USB device: one labeled with the device name alone, and one prefixed with UEFI:. On the hardware used for this document, selecting the UEFI:-prefixed entry after adding a persistence partition consistently produced a bare grub> prompt rather than the Debian live menu. Selecting the plain device entry booted correctly every time.

Throughout this procedure, select the plain device entry—not the one prefixed with UEFI:.

This behavior was observed on specific hardware. Whether it reproduces on all firmware is not established here. The mechanism and a possible explanation are in <ref>https://uefi.org/specs/UEFI/2.10/03_Boot_Manager.html</ref> If your firmware presents only a UEFI boot path, recovery from within a booted live session may be possible by correcting the GRUB prefix, but that procedure is outside the scope of this document.

= Procedure =

=== Clean the Drive ===
If the USB has been used before, residual partition signatures or bootloader data in the early sectors can interfere with the ISO write or the subsequent boot. Clear them before starting:

<code>$ sudo wipefs -a /dev/sdX</code>

<code>$ sudo dd if=/dev/zero of=/dev/sdX bs=1M count=200</code>

wipefs removes filesystem and partition table signatures. The dd command zeros the first 200 MB, covering the MBR, GPT header, and any EFI data in the early sectors.

=== Write the ISO ===
Before running dd, confirm that sync resolves to the system binary and not a shell alias:

<code>$ type sync</code>

The output should be sync is /usr/bin/sync. If it returns an alias definition instead, unalias it with unalias sync, or substitute /usr/bin/sync explicitly at the end of the dd command.

Write the ISO:

<code>$ sudo dd if=/path/to/debian-live-13.4.0-amd64-kde.iso of=/dev/sdX bs=4M status=progress && sync</code>

status=progress shows the running transfer rate. sync flushes the write cache on completion—don’t remove the drive before it returns. The write will take a few minutes depending on the drive’s speed.

=== Verify the Base Boot ===
Before adding the persistence partition, confirm the ISO write produced a bootable drive. Boot from the USB, selecting the plain (non-UEFI:) device entry. If it reaches the Debian live menu and boots cleanly, you have a confirmed working baseline. If it doesn’t boot at this stage, don’t continue—repeat §4.1

== Create the Persistence Partition ==
The ISO write leaves unallocated space at the end of the drive. The persistence partition goes there. The Debian live ISO uses an isohybrid layout—a hybrid MBR/GPT partition table that supports both BIOS and UEFI boot paths. Adding a partition requires an explicit start sector; without it, fdisk’s default placement can conflict with the hybrid structure in ways that affect the boot. The failure modes of parted and gdisk with this layout are documented in §4.9

Launch fdisk:

<code>$ sudo fdisk /dev/sdX</code>

Print the current partition table:

Command (m for help): <code>p</code>

Two partitions will be present: the main ISO partition (type 0 or Empty—this is expected for an isohybrid layout) and a small EFI partition of approximately 3.2 MB. The p output shows the end sector of the last partition in the End column. The persistence partition must start at the next sector—that value plus one. On the test hardware with a Debian 13.4 live ISO written to the drive, the last partition ended at sector 8175999 and the persistence partition started at sector 8176000. This figure will vary with the ISO version and release; read your own p output and don’t use the test value without verifying it matches.

Create the new partition:

Command (m for help): <code>n</code>

When fdisk prompts for the first sector, enter the start sector you identified from the p output—do not accept the default. Accept the default for the last sector to use the remaining space on the drive.

If fdisk asks whether to remove an existing ext4 signature, answer <code>Y</code>. This clears a remnant from a previous attempt.

Write the changes and exit:

Command (m for help): <code>w</code>

After fdisk writes the new partition table, the running kernel may not immediately recognize the change. partprobe signals the kernel to re-read the partition table on the device without requiring a reboot:

<code>$ sudo partprobe /dev/sdX</code>

Once it completes, confirm the new partition is visible:

<code>$ lsblk /dev/sdX</code>

Your new partition will appear beneath the device in the output. Note its full identifier—for example, sdb3 or sdc4—as you’ll need it in the next two steps. If it doesn’t appear immediately, wait a moment and run lsblk again before proceeding.

== Format the Partition ==
Format the new partition as ext4 with the label persistence. The Debian live system identifies the persistence partition by this label at boot:

<code>$ sudo mkfs.ext4 -L persistence /dev/sdXN</code>

== Create the Persistence Configuration ==
Mount the partition:

<code>$ sudo mount /dev/sdXN /mnt</code>

Create the configuration file:

<code>$ echo "/ union" | sudo tee /mnt/persistence.conf</code>

The / union directive instructs the live system to mount the entire filesystem as a union overlay. Changes made during a session are written to the persistence partition rather than discarded on shutdown.

Unmount the partition:

<code>$ sudo umount /mnt</code>

== Boot with Persistence ==
Boot from the USB, selecting the plain (non-UEFI:) device entry in the firmware boot menu. At the Debian live GRUB menu, press Tab to edit the selected boot entry. On the line beginning with linux, append persistence after boot=live components. Press Enter to boot.

The first boot with persistence active will be slower than a standard live session—the system is running entirely from the USB drive, and the persistence overlay adds a layer of I/O. This is normal.

== Verify Persistence ==
After booting, confirm the persistence partition is mounted:

<code>$ lsblk -f</code>

The output should show sdXN mounted at /run/live/persistence/sdXN.

Create a test file:

<code>$ touch ~/persistence-test.txt</code>

Reboot with persistence enabled. If the file is present after reboot, persistence is working.

== What Didn’t Work, and Why ==

=== parted ===
parted misreported the capacity of the test USB drive: a 32 GB device was reported as approximately 125 GB. Determining where the free space began or how large to make the persistence partition was not possible from that output. parted was not used further. The broader behavior—libparted-based tools handling the isohybrid format unreliably—is documented on the Debian User mailing list [9].

=== gdisk / sgdisk ===
gdisk and sgdisk are GPT-aware tools and would seem the logical choice for a drive with a hybrid partition table. On the test USB, sgdisk produced:<blockquote>Found invalid GPT and valid MBR; converting MBR to GPT format in memory.

Warning! Secondary partition table overlaps the last partition by 33 blocks!

Could not create partition 3 from 14580 to 8177663

Non-GPT disk; not saving changes. Use -g to override.

Error encountered; not saving changes.</blockquote>The isohybrid layout places the secondary GPT header in a location that overlaps the last partition. sgdisk identifies this as invalid and refuses to proceed. Using -g to override would corrupt the hybrid layout and break the boot. sgdisk can’t be used here.

cp was also tested as an alternative to dd for writing the ISO, on the theory that it might handle the isohybrid structure differently. It produced an identical partition layout. It’s not an alternative—it’s a slower write with the same result.

== The UEFI grub> Prompt ==
On the test hardware, adding a persistence partition and then selecting the UEFI:-prefixed boot entry consistently dropped to a bare grub> prompt. Running ls from that prompt showed the internal drive’s partitions; the USB’s live partition was unreachable from there.

What was observed: the USB’s own GRUB appeared briefly—visible as a rapid flash on screen with a “no such…” error—then failed, after which the firmware handed control to the internal drive’s GRUB, which presented its own grub> prompt.

The UEFI specification states that firmware may remove boot options it considers invalid [7]; the ArchWiki documents firmware removing entries for drives it detects as changed as a known occurrence [8]. After adding the persistence partition, the USB’s EFI entry—pointing to EFI/boot/bootx64.efi—may no longer have been treated as valid by the firmware, causing it to fall through to the next available bootloader on the internal drive. Whether modifying the partition table on a removable device is sufficient to trigger this behavior, and under what firmware conditions, wasn’t reproducible in a controlled way during testing.

Selecting the plain (non-UEFI:) device entry uses the legacy MBR boot path, which loads the USB’s own GRUB directly and has no dependency on the UEFI NVRAM or EFI boot entries. On the test hardware, this path was unaffected by partition table changes.

== Anonymity ==
A Debian Live USB with persistence, used correctly, writes nothing to the host machine’s internal disk. It doesn’t modify the installed OS, create logs on the host filesystem, or leave traces in the usual locations.

It doesn’t provide anonymity. Boot events may be logged by firmware or enterprise management systems. Network activity is visible to routers, ISPs, and network administrators regardless of which OS is running. Mounting the internal disk during a live session makes its contents accessible to the live environment, and any writes to it are permanent. A live USB is a portable, self-contained Debian environment—it isn’t a privacy tool by design.

== A Note on Carrying Data ==
The persistence partition is a practical place to keep working data alongside the system state—documents, scripts, configuration files, anything you’d normally keep in ~/Documents or ~/bin. It travels with the drive.

Installed packages persist across sessions but may have hardware-specific dependencies. For general desktop use this isn’t usually an issue, but it’s worth keeping in mind if you’re installing drivers or kernel modules that assume specific hardware.

== Citations ==
Syslinux Project. “Isohybrid.” Syslinux Wiki.

[2] Linux Command Library. “isohybrid(1) man page.” https://linuxcommandlibrary.com/man/isohybrid

[3] Smith, Roderick W. “Hybrid MBRs.” GPT fdisk documentation. https://www.rodsbooks.com/gdisk/hybrid.html

[4] Debian Manpages. “persistence.conf(5).” live-boot-doc, Trixie. https://manpages.debian.org/trixie/live-boot-doc/persistence.conf.5.en.html

[5] Debian Live Team. “Customizing Run-Time Behaviours.” Debian Live Manual. https://live-team.pages.debian.net/live-manual/html/live-manual/customizing-run-time-behaviours.en.html

[6] Debian Manpages. “live-boot(7).” live-boot-doc. https://manpages.debian.org/unstable/live-boot-doc/live-boot.7.en.html

[7] UEFI Forum. “Boot Manager.” UEFI Specification 2.10, §3. https://uefi.org/specs/UEFI/2.10/03_Boot_Manager.html

[8] ArchWiki. “Unified Extensible Firmware Interface.” https://wiki.archlinux.org/title/Unified_Extensible_Firmware_Interface

[9] Debian User Mailing List. “Error message from GParted, means what?” November 2016. https://lists.debian.org/debian-user/2016/11/msg01017.html

''Contributed by [https://forums.debian.net/search.php?keywords=distro-nix distro-nix] on Debian User Forums on 2026-04-06.'' Comments, corrections, suggestions or resources are welcome.

Debian Live USB with Persistence

2026-04-10T02:30:45Z

Distro-nix:

All commands are verified on Debian 13.4 (Trixie) / 6.12.74+deb13+1-amd64

Created: 2026-04-04

Updated: 2026-04-09 - added references and restructured heading order;

ID: 005146.11

'''''A Debian Live USB with persistence boots a full Debian environment from a USB drive while retaining changes across sessions. Files you create, settings you change, and packages you install survive reboot—stored on a dedicated partition rather than the volatile live environment. Nothing is written to the host machine’s internal disk.'''''

'''''A standard live USB is stateless: each boot starts from the same ISO image. The persistence partition changes that. From a single drive you get your shell configuration, documents, utilities, browser profile, and scripts, available on any machine you boot it from.'''''

== What You’ll Need ==
A USB drive of at least 8 GB—the minimum is 4 GB, but that leaves little room for persistence. A 32 GB drive is a practical working size. Download the Debian Live ISO from the official Debian website: <nowiki>https://www.debian.org/CD/live</nowiki>

The tools required—dd, fdisk, and mkfs.ext4—are standard on Debian. On a minimal live environment, fdisk and mkfs.ext4 may not be present. Install them before starting:

<code>$ sudo apt install fdisk e2fsprogs</code>

These packages are available from the live session’s apt cache without adding sources. They don’t persist to the USB unless persistence is already active.

== Identifying Your USB Device ==
Plug in the USB drive and run:

<code>$ lsblk</code>

Your USB device will appear as /dev/sdb, /dev/sdc, or similar, depending on what else is attached. Confirm it by cross-referencing the reported size with the known capacity of the drive.

Two placeholders are used throughout this document. /dev/sdX refers to the device itself—substitute your actual device identifier everywhere it appears. /dev/sdXN refers to the persistence partition—N is the partition number you’ll confirm from lsblk output after you've created it.

Use the device path, not a partition path: /dev/sdb is correct; /dev/sdb1 is not.

== Boot Entry Selection ==
The firmware boot menu on many systems presents two entries for a USB device: one labeled with the device name alone, and one prefixed with UEFI:. On the hardware used for this document, selecting the UEFI:-prefixed entry after adding a persistence partition consistently produced a bare grub> prompt rather than the Debian live menu. Selecting the plain device entry booted correctly every time.

Throughout this procedure, select the plain device entry—not the one prefixed with UEFI:.

This behavior was observed on specific hardware. Whether it reproduces on all firmware is not established here. The mechanism and a possible explanation are in §5.3. If your firmware presents only a UEFI boot path, recovery from within a booted live session may be possible by correcting the GRUB prefix, but that procedure is outside the scope of this document.

= Procedure =

=== Clean the Drive ===
If the USB has been used before, residual partition signatures or bootloader data in the early sectors can interfere with the ISO write or the subsequent boot. Clear them before starting:

<code>$ sudo wipefs -a /dev/sdX</code>

<code>$ sudo dd if=/dev/zero of=/dev/sdX bs=1M count=200</code>

wipefs removes filesystem and partition table signatures. The dd command zeros the first 200 MB, covering the MBR, GPT header, and any EFI data in the early sectors.

=== Write the ISO ===
Before running dd, confirm that sync resolves to the system binary and not a shell alias:

<code>$ type sync</code>

The output should be sync is /usr/bin/sync. If it returns an alias definition instead, unalias it with unalias sync, or substitute /usr/bin/sync explicitly at the end of the dd command.

Write the ISO:

<code>$ sudo dd if=/path/to/debian-live-13.4.0-amd64-kde.iso of=/dev/sdX bs=4M status=progress && sync</code>

status=progress shows the running transfer rate. sync flushes the write cache on completion—don’t remove the drive before it returns. The write will take a few minutes depending on the drive’s speed.

=== Verify the Base Boot ===
Before adding the persistence partition, confirm the ISO write produced a bootable drive. Boot from the USB, selecting the plain (non-UEFI:) device entry. If it reaches the Debian live menu and boots cleanly, you have a confirmed working baseline. If it doesn’t boot at this stage, don’t continue—repeat §4.1

== Create the Persistence Partition ==
The ISO write leaves unallocated space at the end of the drive. The persistence partition goes there. The Debian live ISO uses an isohybrid layout—a hybrid MBR/GPT partition table that supports both BIOS and UEFI boot paths. Adding a partition requires an explicit start sector; without it, fdisk’s default placement can conflict with the hybrid structure in ways that affect the boot. The failure modes of parted and gdisk with this layout are documented in §4.9

Launch fdisk:

<code>$ sudo fdisk /dev/sdX</code>

Print the current partition table:

Command (m for help): <code>p</code>

Two partitions will be present: the main ISO partition (type 0 or Empty—this is expected for an isohybrid layout) and a small EFI partition of approximately 3.2 MB. The p output shows the end sector of the last partition in the End column. The persistence partition must start at the next sector—that value plus one. On the test hardware with a Debian 13.4 live ISO written to the drive, the last partition ended at sector 8175999 and the persistence partition started at sector 8176000. This figure will vary with the ISO version and release; read your own p output and don’t use the test value without verifying it matches.

Create the new partition:

Command (m for help): <code>n</code>

When fdisk prompts for the first sector, enter the start sector you identified from the p output—do not accept the default. Accept the default for the last sector to use the remaining space on the drive.

If fdisk asks whether to remove an existing ext4 signature, answer <code>Y</code>. This clears a remnant from a previous attempt.

Write the changes and exit:

Command (m for help): <code>w</code>

After fdisk writes the new partition table, the running kernel may not immediately recognize the change. partprobe signals the kernel to re-read the partition table on the device without requiring a reboot:

<code>$ sudo partprobe /dev/sdX</code>

Once it completes, confirm the new partition is visible:

<code>$ lsblk /dev/sdX</code>

Your new partition will appear beneath the device in the output. Note its full identifier—for example, sdb3 or sdc4—as you’ll need it in the next two steps. If it doesn’t appear immediately, wait a moment and run lsblk again before proceeding.

== Format the Partition ==
Format the new partition as ext4 with the label persistence. The Debian live system identifies the persistence partition by this label at boot:

<code>$ sudo mkfs.ext4 -L persistence /dev/sdXN</code>

== Create the Persistence Configuration ==
Mount the partition:

<code>$ sudo mount /dev/sdXN /mnt</code>

Create the configuration file:

<code>$ echo "/ union" | sudo tee /mnt/persistence.conf</code>

The / union directive instructs the live system to mount the entire filesystem as a union overlay. Changes made during a session are written to the persistence partition rather than discarded on shutdown.

Unmount the partition:

<code>$ sudo umount /mnt</code>

== Boot with Persistence ==
Boot from the USB, selecting the plain (non-UEFI:) device entry in the firmware boot menu. At the Debian live GRUB menu, press Tab to edit the selected boot entry. On the line beginning with linux, append persistence after boot=live components. Press Enter to boot.

The first boot with persistence active will be slower than a standard live session—the system is running entirely from the USB drive, and the persistence overlay adds a layer of I/O. This is normal.

== Verify Persistence ==
After booting, confirm the persistence partition is mounted:

<code>$ lsblk -f</code>

The output should show sdXN mounted at /run/live/persistence/sdXN.

Create a test file:

<code>$ touch ~/persistence-test.txt</code>

Reboot with persistence enabled. If the file is present after reboot, persistence is working.

== What Didn’t Work, and Why ==

=== parted ===
parted misreported the capacity of the test USB drive: a 32 GB device was reported as approximately 125 GB. Determining where the free space began or how large to make the persistence partition was not possible from that output. parted was not used further. The broader behavior—libparted-based tools handling the isohybrid format unreliably—is documented on the Debian User mailing list [9].

=== gdisk / sgdisk ===
gdisk and sgdisk are GPT-aware tools and would seem the logical choice for a drive with a hybrid partition table. On the test USB, sgdisk produced:<blockquote>Found invalid GPT and valid MBR; converting MBR to GPT format in memory.

Warning! Secondary partition table overlaps the last partition by 33 blocks!

Could not create partition 3 from 14580 to 8177663

Non-GPT disk; not saving changes. Use -g to override.

Error encountered; not saving changes.</blockquote>The isohybrid layout places the secondary GPT header in a location that overlaps the last partition. sgdisk identifies this as invalid and refuses to proceed. Using -g to override would corrupt the hybrid layout and break the boot. sgdisk can’t be used here.

cp was also tested as an alternative to dd for writing the ISO, on the theory that it might handle the isohybrid structure differently. It produced an identical partition layout. It’s not an alternative—it’s a slower write with the same result.

== The UEFI grub> Prompt ==
On the test hardware, adding a persistence partition and then selecting the UEFI:-prefixed boot entry consistently dropped to a bare grub> prompt. Running ls from that prompt showed the internal drive’s partitions; the USB’s live partition was unreachable from there.

What was observed: the USB’s own GRUB appeared briefly—visible as a rapid flash on screen with a “no such…” error—then failed, after which the firmware handed control to the internal drive’s GRUB, which presented its own grub> prompt.

The UEFI specification states that firmware may remove boot options it considers invalid [7]; the ArchWiki documents firmware removing entries for drives it detects as changed as a known occurrence [8]. After adding the persistence partition, the USB’s EFI entry—pointing to EFI/boot/bootx64.efi—may no longer have been treated as valid by the firmware, causing it to fall through to the next available bootloader on the internal drive. Whether modifying the partition table on a removable device is sufficient to trigger this behavior, and under what firmware conditions, wasn’t reproducible in a controlled way during testing.

Selecting the plain (non-UEFI:) device entry uses the legacy MBR boot path, which loads the USB’s own GRUB directly and has no dependency on the UEFI NVRAM or EFI boot entries. On the test hardware, this path was unaffected by partition table changes.

== Anonymity ==
A Debian Live USB with persistence, used correctly, writes nothing to the host machine’s internal disk. It doesn’t modify the installed OS, create logs on the host filesystem, or leave traces in the usual locations.

It doesn’t provide anonymity. Boot events may be logged by firmware or enterprise management systems. Network activity is visible to routers, ISPs, and network administrators regardless of which OS is running. Mounting the internal disk during a live session makes its contents accessible to the live environment, and any writes to it are permanent. A live USB is a portable, self-contained Debian environment—it isn’t a privacy tool by design.

== A Note on Carrying Data ==
The persistence partition is a practical place to keep working data alongside the system state—documents, scripts, configuration files, anything you’d normally keep in ~/Documents or ~/bin. It travels with the drive.

Installed packages persist across sessions but may have hardware-specific dependencies. For general desktop use this isn’t usually an issue, but it’s worth keeping in mind if you’re installing drivers or kernel modules that assume specific hardware.

== Citations ==
Syslinux Project. “Isohybrid.” Syslinux Wiki.

[2] Linux Command Library. “isohybrid(1) man page.” https://linuxcommandlibrary.com/man/isohybrid

[3] Smith, Roderick W. “Hybrid MBRs.” GPT fdisk documentation. https://www.rodsbooks.com/gdisk/hybrid.html

[4] Debian Manpages. “persistence.conf(5).” live-boot-doc, Trixie. https://manpages.debian.org/trixie/live-boot-doc/persistence.conf.5.en.html

[5] Debian Live Team. “Customizing Run-Time Behaviours.” Debian Live Manual. https://live-team.pages.debian.net/live-manual/html/live-manual/customizing-run-time-behaviours.en.html

[6] Debian Manpages. “live-boot(7).” live-boot-doc. https://manpages.debian.org/unstable/live-boot-doc/live-boot.7.en.html

[7] UEFI Forum. “Boot Manager.” UEFI Specification 2.10, §3. https://uefi.org/specs/UEFI/2.10/03_Boot_Manager.html

[8] ArchWiki. “Unified Extensible Firmware Interface.” https://wiki.archlinux.org/title/Unified_Extensible_Firmware_Interface

[9] Debian User Mailing List. “Error message from GParted, means what?” November 2016. https://lists.debian.org/debian-user/2016/11/msg01017.html

''Contributed by [https://forums.debian.net/search.php?keywords=distro-nix distro-nix] on Debian User Forums on 2026-04-06.'' Comments, corrections, suggestions or resources are welcome.

Debian Live USB with Persistence

2026-04-10T02:27:16Z

Distro-nix: /* Citations */ added new citation

All commands are verified on Debian 13.4 (Trixie) / 6.12.74+deb13+1-amd64

Created: 2026-04-04

Updated: 2026-04-09 - added references and restructured heading order;

ID: 005146.11

'''''A Debian Live USB with persistence boots a full Debian environment from a USB drive while retaining changes across sessions. Files you create, settings you change, and packages you install survive reboot—stored on a dedicated partition rather than the volatile live environment. Nothing is written to the host machine’s internal disk.'''''

'''''A standard live USB is stateless: each boot starts from the same ISO image. The persistence partition changes that. From a single drive you get your shell configuration, documents, utilities, browser profile, and scripts, available on any machine you boot it from.'''''

== What You’ll Need ==
A USB drive of at least 8 GB—the minimum is 4 GB, but that leaves little room for persistence. A 32 GB drive is a practical working size. Download the Debian Live ISO from the official Debian website: <nowiki>https://www.debian.org/CD/live</nowiki>

The tools required—dd, fdisk, and mkfs.ext4—are standard on Debian. On a minimal live environment, fdisk and mkfs.ext4 may not be present. Install them before starting:

<code>$ sudo apt install fdisk e2fsprogs</code>

These packages are available from the live session’s apt cache without adding sources. They don’t persist to the USB unless persistence is already active.

== Identifying Your USB Device ==
Plug in the USB drive and run:

<code>$ lsblk</code>

Your USB device will appear as /dev/sdb, /dev/sdc, or similar, depending on what else is attached. Confirm it by cross-referencing the reported size with the known capacity of the drive.

Two placeholders are used throughout this document. /dev/sdX refers to the device itself—substitute your actual device identifier everywhere it appears. /dev/sdXN refers to the persistence partition—N is the partition number you’ll confirm from lsblk output after you've created it.

Use the device path, not a partition path: /dev/sdb is correct; /dev/sdb1 is not.

== Boot Entry Selection ==
The firmware boot menu on many systems presents two entries for a USB device: one labeled with the device name alone, and one prefixed with UEFI:. On the hardware used for this document, selecting the UEFI:-prefixed entry after adding a persistence partition consistently produced a bare grub> prompt rather than the Debian live menu. Selecting the plain device entry booted correctly every time.

Throughout this procedure, select the plain device entry—not the one prefixed with UEFI:.

This behavior was observed on specific hardware. Whether it reproduces on all firmware is not established here. The mechanism and a possible explanation are in §5.3. If your firmware presents only a UEFI boot path, recovery from within a booted live session may be possible by correcting the GRUB prefix, but that procedure is outside the scope of this document.

= Procedure =

=== Clean the Drive ===
If the USB has been used before, residual partition signatures or bootloader data in the early sectors can interfere with the ISO write or the subsequent boot. Clear them before starting:

<code>$ sudo wipefs -a /dev/sdX</code>

<code>$ sudo dd if=/dev/zero of=/dev/sdX bs=1M count=200</code>

wipefs removes filesystem and partition table signatures. The dd command zeros the first 200 MB, covering the MBR, GPT header, and any EFI data in the early sectors.

=== Write the ISO ===
Before running dd, confirm that sync resolves to the system binary and not a shell alias:

<code>$ type sync</code>

The output should be sync is /usr/bin/sync. If it returns an alias definition instead, unalias it with unalias sync, or substitute /usr/bin/sync explicitly at the end of the dd command.

Write the ISO:

<code>$ sudo dd if=/path/to/debian-live-13.4.0-amd64-kde.iso of=/dev/sdX bs=4M status=progress && sync</code>

status=progress shows the running transfer rate. sync flushes the write cache on completion—don’t remove the drive before it returns. The write will take a few minutes depending on the drive’s speed.

=== Verify the Base Boot ===
Before adding the persistence partition, confirm the ISO write produced a bootable drive. Boot from the USB, selecting the plain (non-UEFI:) device entry. If it reaches the Debian live menu and boots cleanly, you have a confirmed working baseline. If it doesn’t boot at this stage, don’t continue—repeat §4.1

== Create the Persistence Partition ==
The ISO write leaves unallocated space at the end of the drive. The persistence partition goes there. The Debian live ISO uses an isohybrid layout—a hybrid MBR/GPT partition table that supports both BIOS and UEFI boot paths. Adding a partition requires an explicit start sector; without it, fdisk’s default placement can conflict with the hybrid structure in ways that affect the boot. The failure modes of parted and gdisk with this layout are documented in §4.9

Launch fdisk:

<code>$ sudo fdisk /dev/sdX</code>

Print the current partition table:

Command (m for help): <code>p</code>

Two partitions will be present: the main ISO partition (type 0 or Empty—this is expected for an isohybrid layout) and a small EFI partition of approximately 3.2 MB. The p output shows the end sector of the last partition in the End column. The persistence partition must start at the next sector—that value plus one. On the test hardware with a Debian 13.4 live ISO written to the drive, the last partition ended at sector 8175999 and the persistence partition started at sector 8176000. This figure will vary with the ISO version and release; read your own p output and don’t use the test value without verifying it matches.

Create the new partition:

Command (m for help): <code>n</code>

When fdisk prompts for the first sector, enter the start sector you identified from the p output—do not accept the default. Accept the default for the last sector to use the remaining space on the drive.

If fdisk asks whether to remove an existing ext4 signature, answer <code>Y</code>. This clears a remnant from a previous attempt.

Write the changes and exit:

Command (m for help): <code>w</code>

After fdisk writes the new partition table, the running kernel may not immediately recognize the change. partprobe signals the kernel to re-read the partition table on the device without requiring a reboot:

<code>$ sudo partprobe /dev/sdX</code>

Once it completes, confirm the new partition is visible:

<code>$ lsblk /dev/sdX</code>

Your new partition will appear beneath the device in the output. Note its full identifier—for example, sdb3 or sdc4—as you’ll need it in the next two steps. If it doesn’t appear immediately, wait a moment and run lsblk again before proceeding.

== Format the Partition ==
Format the new partition as ext4 with the label persistence. The Debian live system identifies the persistence partition by this label at boot:

<code>$ sudo mkfs.ext4 -L persistence /dev/sdXN</code>

== Create the Persistence Configuration ==
Mount the partition:

<code>$ sudo mount /dev/sdXN /mnt</code>

Create the configuration file:

<code>$ echo "/ union" | sudo tee /mnt/persistence.conf</code>

The / union directive instructs the live system to mount the entire filesystem as a union overlay. Changes made during a session are written to the persistence partition rather than discarded on shutdown.

Unmount the partition:

<code>$ sudo umount /mnt</code>

== Boot with Persistence ==
Boot from the USB, selecting the plain (non-UEFI:) device entry in the firmware boot menu. At the Debian live GRUB menu, press Tab to edit the selected boot entry. On the line beginning with linux, append persistence after boot=live components. Press Enter to boot.

The first boot with persistence active will be slower than a standard live session—the system is running entirely from the USB drive, and the persistence overlay adds a layer of I/O. This is normal.

== Verify Persistence ==
After booting, confirm the persistence partition is mounted:

<code>$ lsblk -f</code>

The output should show sdXN mounted at /run/live/persistence/sdXN.

Create a test file:

<code>$ touch ~/persistence-test.txt</code>

Reboot with persistence enabled. If the file is present after reboot, persistence is working.

== What Didn’t Work, and Why ==

=== parted ===
parted misreported the capacity of the test USB drive: a 32 GB device was reported as approximately 125 GB. Determining where the free space began or how large to make the persistence partition was not possible from that output. parted was not used further. The broader behavior—libparted-based tools handling the isohybrid format unreliably—is documented on the Debian User mailing list [9].

=== gdisk / sgdisk ===
gdisk and sgdisk are GPT-aware tools and would seem the logical choice for a drive with a hybrid partition table. On the test USB, sgdisk produced:<blockquote>Found invalid GPT and valid MBR; converting MBR to GPT format in memory.

Warning! Secondary partition table overlaps the last partition by 33 blocks!

Could not create partition 3 from 14580 to 8177663

Non-GPT disk; not saving changes. Use -g to override.

Error encountered; not saving changes.</blockquote>The isohybrid layout places the secondary GPT header in a location that overlaps the last partition. sgdisk identifies this as invalid and refuses to proceed. Using -g to override would corrupt the hybrid layout and break the boot. sgdisk can’t be used here.

cp was also tested as an alternative to dd for writing the ISO, on the theory that it might handle the isohybrid structure differently. It produced an identical partition layout. It’s not an alternative—it’s a slower write with the same result.

== The UEFI grub> Prompt ==
On the test hardware, adding a persistence partition and then selecting the UEFI:-prefixed boot entry consistently dropped to a bare grub> prompt. Running ls from that prompt showed the internal drive’s partitions; the USB’s live partition was unreachable from there.

What was observed: the USB’s own GRUB appeared briefly—visible as a rapid flash on screen with a “no such…” error—then failed, after which the firmware handed control to the internal drive’s GRUB, which presented its own grub> prompt.

The UEFI specification states that firmware may remove boot options it considers invalid [7]; the ArchWiki documents firmware removing entries for drives it detects as changed as a known occurrence [8]. After adding the persistence partition, the USB’s EFI entry—pointing to EFI/boot/bootx64.efi—may no longer have been treated as valid by the firmware, causing it to fall through to the next available bootloader on the internal drive. Whether modifying the partition table on a removable device is sufficient to trigger this behavior, and under what firmware conditions, wasn’t reproducible in a controlled way during testing.

Selecting the plain (non-UEFI:) device entry uses the legacy MBR boot path, which loads the USB’s own GRUB directly and has no dependency on the UEFI NVRAM or EFI boot entries. On the test hardware, this path was unaffected by partition table changes.

== Anonymity ==
A Debian Live USB with persistence, used correctly, writes nothing to the host machine’s internal disk. It doesn’t modify the installed OS, create logs on the host filesystem, or leave traces in the usual locations.

It doesn’t provide anonymity. Boot events may be logged by firmware or enterprise management systems. Network activity is visible to routers, ISPs, and network administrators regardless of which OS is running. Mounting the internal disk during a live session makes its contents accessible to the live environment, and any writes to it are permanent. A live USB is a portable, self-contained Debian environment—it isn’t a privacy tool by design.

== A Note on Carrying Data ==
The persistence partition is a practical place to keep working data alongside the system state—documents, scripts, configuration files, anything you’d normally keep in ~/Documents or ~/bin. It travels with the drive.

Installed packages persist across sessions but may have hardware-specific dependencies. For general desktop use this isn’t usually an issue, but it’s worth keeping in mind if you’re installing drivers or kernel modules that assume specific hardware.

== Citations ==
Syslinux Project. “Isohybrid.” Syslinux Wiki. <ref><nowiki>https://wiki.syslinux.org/wiki/index.php?title=Isohybrid</nowiki></ref>

[2] Linux Command Library. “isohybrid(1) man page.” https://linuxcommandlibrary.com/man/isohybrid

[3] Smith, Roderick W. “Hybrid MBRs.” GPT fdisk documentation. https://www.rodsbooks.com/gdisk/hybrid.html

[4] Debian Manpages. “persistence.conf(5).” live-boot-doc, Trixie. https://manpages.debian.org/trixie/live-boot-doc/persistence.conf.5.en.html

[5] Debian Live Team. “Customizing Run-Time Behaviours.” Debian Live Manual. https://live-team.pages.debian.net/live-manual/html/live-manual/customizing-run-time-behaviours.en.html

[6] Debian Manpages. “live-boot(7).” live-boot-doc. https://manpages.debian.org/unstable/live-boot-doc/live-boot.7.en.html

[7] UEFI Forum. “Boot Manager.” UEFI Specification 2.10, §3. https://uefi.org/specs/UEFI/2.10/03_Boot_Manager.html

[8] ArchWiki. “Unified Extensible Firmware Interface.” https://wiki.archlinux.org/title/Unified_Extensible_Firmware_Interface

[9] Debian User Mailing List. “Error message from GParted, means what?” November 2016. https://lists.debian.org/debian-user/2016/11/msg01017.html

''Contributed by [https://forums.debian.net/search.php?keywords=distro-nix distro-nix] on Debian User Forums on 2026-04-06.'' Comments, corrections, suggestions or resources are welcome.

Debian Live USB with Persistence

2026-04-10T02:17:39Z

Distro-nix: Replaced References heading with Citations

All commands are verified on Debian 13.4 (Trixie) / 6.12.74+deb13+1-amd64

Created: 2026-04-04

Updated: 2026-04-09 - added references and restructured heading order;

ID: 005146.11

'''''A Debian Live USB with persistence boots a full Debian environment from a USB drive while retaining changes across sessions. Files you create, settings you change, and packages you install survive reboot—stored on a dedicated partition rather than the volatile live environment. Nothing is written to the host machine’s internal disk.'''''

'''''A standard live USB is stateless: each boot starts from the same ISO image. The persistence partition changes that. From a single drive you get your shell configuration, documents, utilities, browser profile, and scripts, available on any machine you boot it from.'''''

== What You’ll Need ==
A USB drive of at least 8 GB—the minimum is 4 GB, but that leaves little room for persistence. A 32 GB drive is a practical working size. Download the Debian Live ISO from the official Debian website: <nowiki>https://www.debian.org/CD/live</nowiki>

The tools required—dd, fdisk, and mkfs.ext4—are standard on Debian. On a minimal live environment, fdisk and mkfs.ext4 may not be present. Install them before starting:

<code>$ sudo apt install fdisk e2fsprogs</code>

These packages are available from the live session’s apt cache without adding sources. They don’t persist to the USB unless persistence is already active.

== Identifying Your USB Device ==
Plug in the USB drive and run:

<code>$ lsblk</code>

Your USB device will appear as /dev/sdb, /dev/sdc, or similar, depending on what else is attached. Confirm it by cross-referencing the reported size with the known capacity of the drive.

Two placeholders are used throughout this document. /dev/sdX refers to the device itself—substitute your actual device identifier everywhere it appears. /dev/sdXN refers to the persistence partition—N is the partition number you’ll confirm from lsblk output after you've created it.

Use the device path, not a partition path: /dev/sdb is correct; /dev/sdb1 is not.

== Boot Entry Selection ==
The firmware boot menu on many systems presents two entries for a USB device: one labeled with the device name alone, and one prefixed with UEFI:. On the hardware used for this document, selecting the UEFI:-prefixed entry after adding a persistence partition consistently produced a bare grub> prompt rather than the Debian live menu. Selecting the plain device entry booted correctly every time.

Throughout this procedure, select the plain device entry—not the one prefixed with UEFI:.

This behavior was observed on specific hardware. Whether it reproduces on all firmware is not established here. The mechanism and a possible explanation are in §5.3. If your firmware presents only a UEFI boot path, recovery from within a booted live session may be possible by correcting the GRUB prefix, but that procedure is outside the scope of this document.

= Procedure =

=== Clean the Drive ===
If the USB has been used before, residual partition signatures or bootloader data in the early sectors can interfere with the ISO write or the subsequent boot. Clear them before starting:

<code>$ sudo wipefs -a /dev/sdX</code>

<code>$ sudo dd if=/dev/zero of=/dev/sdX bs=1M count=200</code>

wipefs removes filesystem and partition table signatures. The dd command zeros the first 200 MB, covering the MBR, GPT header, and any EFI data in the early sectors.

=== Write the ISO ===
Before running dd, confirm that sync resolves to the system binary and not a shell alias:

<code>$ type sync</code>

The output should be sync is /usr/bin/sync. If it returns an alias definition instead, unalias it with unalias sync, or substitute /usr/bin/sync explicitly at the end of the dd command.

Write the ISO:

<code>$ sudo dd if=/path/to/debian-live-13.4.0-amd64-kde.iso of=/dev/sdX bs=4M status=progress && sync</code>

status=progress shows the running transfer rate. sync flushes the write cache on completion—don’t remove the drive before it returns. The write will take a few minutes depending on the drive’s speed.

=== Verify the Base Boot ===
Before adding the persistence partition, confirm the ISO write produced a bootable drive. Boot from the USB, selecting the plain (non-UEFI:) device entry. If it reaches the Debian live menu and boots cleanly, you have a confirmed working baseline. If it doesn’t boot at this stage, don’t continue—repeat §4.1

== Create the Persistence Partition ==
The ISO write leaves unallocated space at the end of the drive. The persistence partition goes there. The Debian live ISO uses an isohybrid layout—a hybrid MBR/GPT partition table that supports both BIOS and UEFI boot paths. Adding a partition requires an explicit start sector; without it, fdisk’s default placement can conflict with the hybrid structure in ways that affect the boot. The failure modes of parted and gdisk with this layout are documented in §4.9

Launch fdisk:

<code>$ sudo fdisk /dev/sdX</code>

Print the current partition table:

Command (m for help): <code>p</code>

Two partitions will be present: the main ISO partition (type 0 or Empty—this is expected for an isohybrid layout) and a small EFI partition of approximately 3.2 MB. The p output shows the end sector of the last partition in the End column. The persistence partition must start at the next sector—that value plus one. On the test hardware with a Debian 13.4 live ISO written to the drive, the last partition ended at sector 8175999 and the persistence partition started at sector 8176000. This figure will vary with the ISO version and release; read your own p output and don’t use the test value without verifying it matches.

Create the new partition:

Command (m for help): <code>n</code>

When fdisk prompts for the first sector, enter the start sector you identified from the p output—do not accept the default. Accept the default for the last sector to use the remaining space on the drive.

If fdisk asks whether to remove an existing ext4 signature, answer <code>Y</code>. This clears a remnant from a previous attempt.

Write the changes and exit:

Command (m for help): <code>w</code>

After fdisk writes the new partition table, the running kernel may not immediately recognize the change. partprobe signals the kernel to re-read the partition table on the device without requiring a reboot:

<code>$ sudo partprobe /dev/sdX</code>

Once it completes, confirm the new partition is visible:

<code>$ lsblk /dev/sdX</code>

Your new partition will appear beneath the device in the output. Note its full identifier—for example, sdb3 or sdc4—as you’ll need it in the next two steps. If it doesn’t appear immediately, wait a moment and run lsblk again before proceeding.

== Format the Partition ==
Format the new partition as ext4 with the label persistence. The Debian live system identifies the persistence partition by this label at boot:

<code>$ sudo mkfs.ext4 -L persistence /dev/sdXN</code>

== Create the Persistence Configuration ==
Mount the partition:

<code>$ sudo mount /dev/sdXN /mnt</code>

Create the configuration file:

<code>$ echo "/ union" | sudo tee /mnt/persistence.conf</code>

The / union directive instructs the live system to mount the entire filesystem as a union overlay. Changes made during a session are written to the persistence partition rather than discarded on shutdown.

Unmount the partition:

<code>$ sudo umount /mnt</code>

== Boot with Persistence ==
Boot from the USB, selecting the plain (non-UEFI:) device entry in the firmware boot menu. At the Debian live GRUB menu, press Tab to edit the selected boot entry. On the line beginning with linux, append persistence after boot=live components. Press Enter to boot.

The first boot with persistence active will be slower than a standard live session—the system is running entirely from the USB drive, and the persistence overlay adds a layer of I/O. This is normal.

== Verify Persistence ==
After booting, confirm the persistence partition is mounted:

<code>$ lsblk -f</code>

The output should show sdXN mounted at /run/live/persistence/sdXN.

Create a test file:

<code>$ touch ~/persistence-test.txt</code>

Reboot with persistence enabled. If the file is present after reboot, persistence is working.

== What Didn’t Work, and Why ==

=== parted ===
parted misreported the capacity of the test USB drive: a 32 GB device was reported as approximately 125 GB. Determining where the free space began or how large to make the persistence partition was not possible from that output. parted was not used further. The broader behavior—libparted-based tools handling the isohybrid format unreliably—is documented on the Debian User mailing list [9].

=== gdisk / sgdisk ===
gdisk and sgdisk are GPT-aware tools and would seem the logical choice for a drive with a hybrid partition table. On the test USB, sgdisk produced:<blockquote>Found invalid GPT and valid MBR; converting MBR to GPT format in memory.

Warning! Secondary partition table overlaps the last partition by 33 blocks!

Could not create partition 3 from 14580 to 8177663

Non-GPT disk; not saving changes. Use -g to override.

Error encountered; not saving changes.</blockquote>The isohybrid layout places the secondary GPT header in a location that overlaps the last partition. sgdisk identifies this as invalid and refuses to proceed. Using -g to override would corrupt the hybrid layout and break the boot. sgdisk can’t be used here.

cp was also tested as an alternative to dd for writing the ISO, on the theory that it might handle the isohybrid structure differently. It produced an identical partition layout. It’s not an alternative—it’s a slower write with the same result.

== The UEFI grub> Prompt ==
On the test hardware, adding a persistence partition and then selecting the UEFI:-prefixed boot entry consistently dropped to a bare grub> prompt. Running ls from that prompt showed the internal drive’s partitions; the USB’s live partition was unreachable from there.

What was observed: the USB’s own GRUB appeared briefly—visible as a rapid flash on screen with a “no such…” error—then failed, after which the firmware handed control to the internal drive’s GRUB, which presented its own grub> prompt.

The UEFI specification states that firmware may remove boot options it considers invalid [7]; the ArchWiki documents firmware removing entries for drives it detects as changed as a known occurrence [8]. After adding the persistence partition, the USB’s EFI entry—pointing to EFI/boot/bootx64.efi—may no longer have been treated as valid by the firmware, causing it to fall through to the next available bootloader on the internal drive. Whether modifying the partition table on a removable device is sufficient to trigger this behavior, and under what firmware conditions, wasn’t reproducible in a controlled way during testing.

Selecting the plain (non-UEFI:) device entry uses the legacy MBR boot path, which loads the USB’s own GRUB directly and has no dependency on the UEFI NVRAM or EFI boot entries. On the test hardware, this path was unaffected by partition table changes.

== Anonymity ==
A Debian Live USB with persistence, used correctly, writes nothing to the host machine’s internal disk. It doesn’t modify the installed OS, create logs on the host filesystem, or leave traces in the usual locations.

It doesn’t provide anonymity. Boot events may be logged by firmware or enterprise management systems. Network activity is visible to routers, ISPs, and network administrators regardless of which OS is running. Mounting the internal disk during a live session makes its contents accessible to the live environment, and any writes to it are permanent. A live USB is a portable, self-contained Debian environment—it isn’t a privacy tool by design.

== A Note on Carrying Data ==
The persistence partition is a practical place to keep working data alongside the system state—documents, scripts, configuration files, anything you’d normally keep in ~/Documents or ~/bin. It travels with the drive.

Installed packages persist across sessions but may have hardware-specific dependencies. For general desktop use this isn’t usually an issue, but it’s worth keeping in mind if you’re installing drivers or kernel modules that assume specific hardware.

== Citations ==
[1] Syslinux Project. “Isohybrid.” Syslinux Wiki. https://wiki.syslinux.org/wiki/index.php?title=Isohybrid

[2] Linux Command Library. “isohybrid(1) man page.” https://linuxcommandlibrary.com/man/isohybrid

[3] Smith, Roderick W. “Hybrid MBRs.” GPT fdisk documentation. https://www.rodsbooks.com/gdisk/hybrid.html

[4] Debian Manpages. “persistence.conf(5).” live-boot-doc, Trixie. https://manpages.debian.org/trixie/live-boot-doc/persistence.conf.5.en.html

[5] Debian Live Team. “Customizing Run-Time Behaviours.” Debian Live Manual. https://live-team.pages.debian.net/live-manual/html/live-manual/customizing-run-time-behaviours.en.html

[6] Debian Manpages. “live-boot(7).” live-boot-doc. https://manpages.debian.org/unstable/live-boot-doc/live-boot.7.en.html

[7] UEFI Forum. “Boot Manager.” UEFI Specification 2.10, §3. https://uefi.org/specs/UEFI/2.10/03_Boot_Manager.html

[8] ArchWiki. “Unified Extensible Firmware Interface.” https://wiki.archlinux.org/title/Unified_Extensible_Firmware_Interface

[9] Debian User Mailing List. “Error message from GParted, means what?” November 2016. https://lists.debian.org/debian-user/2016/11/msg01017.html

''Contributed by [https://forums.debian.net/search.php?keywords=distro-nix distro-nix] on Debian User Forums on 2026-04-06.'' Comments, corrections, suggestions or resources are welcome.

Debian Live USB with Persistence

2026-04-10T01:18:12Z

Distro-nix: references checked

All commands are verified on Debian 13.4 (Trixie) / 6.12.74+deb13+1-amd64

Created: 2026-04-04

Updated: 2026-04-09 - added references and restructured heading order;

ID: 005146.11

'''''A Debian Live USB with persistence boots a full Debian environment from a USB drive while retaining changes across sessions. Files you create, settings you change, and packages you install survive reboot—stored on a dedicated partition rather than the volatile live environment. Nothing is written to the host machine’s internal disk.'''''

'''''A standard live USB is stateless: each boot starts from the same ISO image. The persistence partition changes that. From a single drive you get your shell configuration, documents, utilities, browser profile, and scripts, available on any machine you boot it from.'''''

== What You’ll Need ==
A USB drive of at least 8 GB—the minimum is 4 GB, but that leaves little room for persistence. A 32 GB drive is a practical working size. Download the Debian Live ISO from the official Debian website: <nowiki>https://www.debian.org/CD/live</nowiki>

The tools required—dd, fdisk, and mkfs.ext4—are standard on Debian. On a minimal live environment, fdisk and mkfs.ext4 may not be present. Install them before starting:

<code>$ sudo apt install fdisk e2fsprogs</code>

These packages are available from the live session’s apt cache without adding sources. They don’t persist to the USB unless persistence is already active.

== Identifying Your USB Device ==
Plug in the USB drive and run:

<code>$ lsblk</code>

Your USB device will appear as /dev/sdb, /dev/sdc, or similar, depending on what else is attached. Confirm it by cross-referencing the reported size with the known capacity of the drive.

Two placeholders are used throughout this document. /dev/sdX refers to the device itself—substitute your actual device identifier everywhere it appears. /dev/sdXN refers to the persistence partition—N is the partition number you’ll confirm from lsblk output after you've created it.

Use the device path, not a partition path: /dev/sdb is correct; /dev/sdb1 is not.

== Boot Entry Selection ==
The firmware boot menu on many systems presents two entries for a USB device: one labeled with the device name alone, and one prefixed with UEFI:. On the hardware used for this document, selecting the UEFI:-prefixed entry after adding a persistence partition consistently produced a bare grub> prompt rather than the Debian live menu. Selecting the plain device entry booted correctly every time.

Throughout this procedure, select the plain device entry—not the one prefixed with UEFI:.

This behavior was observed on specific hardware. Whether it reproduces on all firmware is not established here. The mechanism and a possible explanation are in §5.3. If your firmware presents only a UEFI boot path, recovery from within a booted live session may be possible by correcting the GRUB prefix, but that procedure is outside the scope of this document.

= Procedure =

=== Clean the Drive ===
If the USB has been used before, residual partition signatures or bootloader data in the early sectors can interfere with the ISO write or the subsequent boot. Clear them before starting:

<code>$ sudo wipefs -a /dev/sdX</code>

<code>$ sudo dd if=/dev/zero of=/dev/sdX bs=1M count=200</code>

wipefs removes filesystem and partition table signatures. The dd command zeros the first 200 MB, covering the MBR, GPT header, and any EFI data in the early sectors.

=== Write the ISO ===
Before running dd, confirm that sync resolves to the system binary and not a shell alias:

<code>$ type sync</code>

The output should be sync is /usr/bin/sync. If it returns an alias definition instead, unalias it with unalias sync, or substitute /usr/bin/sync explicitly at the end of the dd command.

Write the ISO:

<code>$ sudo dd if=/path/to/debian-live-13.4.0-amd64-kde.iso of=/dev/sdX bs=4M status=progress && sync</code>

status=progress shows the running transfer rate. sync flushes the write cache on completion—don’t remove the drive before it returns. The write will take a few minutes depending on the drive’s speed.

=== Verify the Base Boot ===
Before adding the persistence partition, confirm the ISO write produced a bootable drive. Boot from the USB, selecting the plain (non-UEFI:) device entry. If it reaches the Debian live menu and boots cleanly, you have a confirmed working baseline. If it doesn’t boot at this stage, don’t continue—repeat §4.1

== Create the Persistence Partition ==
The ISO write leaves unallocated space at the end of the drive. The persistence partition goes there. The Debian live ISO uses an isohybrid layout—a hybrid MBR/GPT partition table that supports both BIOS and UEFI boot paths. Adding a partition requires an explicit start sector; without it, fdisk’s default placement can conflict with the hybrid structure in ways that affect the boot. The failure modes of parted and gdisk with this layout are documented in §4.9

Launch fdisk:

<code>$ sudo fdisk /dev/sdX</code>

Print the current partition table:

Command (m for help): <code>p</code>

Two partitions will be present: the main ISO partition (type 0 or Empty—this is expected for an isohybrid layout) and a small EFI partition of approximately 3.2 MB. The p output shows the end sector of the last partition in the End column. The persistence partition must start at the next sector—that value plus one. On the test hardware with a Debian 13.4 live ISO written to the drive, the last partition ended at sector 8175999 and the persistence partition started at sector 8176000. This figure will vary with the ISO version and release; read your own p output and don’t use the test value without verifying it matches.

Create the new partition:

Command (m for help): <code>n</code>

When fdisk prompts for the first sector, enter the start sector you identified from the p output—do not accept the default. Accept the default for the last sector to use the remaining space on the drive.

If fdisk asks whether to remove an existing ext4 signature, answer <code>Y</code>. This clears a remnant from a previous attempt.

Write the changes and exit:

Command (m for help): <code>w</code>

After fdisk writes the new partition table, the running kernel may not immediately recognize the change. partprobe signals the kernel to re-read the partition table on the device without requiring a reboot:

<code>$ sudo partprobe /dev/sdX</code>

Once it completes, confirm the new partition is visible:

<code>$ lsblk /dev/sdX</code>

Your new partition will appear beneath the device in the output. Note its full identifier—for example, sdb3 or sdc4—as you’ll need it in the next two steps. If it doesn’t appear immediately, wait a moment and run lsblk again before proceeding.

== Format the Partition ==
Format the new partition as ext4 with the label persistence. The Debian live system identifies the persistence partition by this label at boot:

<code>$ sudo mkfs.ext4 -L persistence /dev/sdXN</code>

== Create the Persistence Configuration ==
Mount the partition:

<code>$ sudo mount /dev/sdXN /mnt</code>

Create the configuration file:

<code>$ echo "/ union" | sudo tee /mnt/persistence.conf</code>

The / union directive instructs the live system to mount the entire filesystem as a union overlay. Changes made during a session are written to the persistence partition rather than discarded on shutdown.

Unmount the partition:

<code>$ sudo umount /mnt</code>

== Boot with Persistence ==
Boot from the USB, selecting the plain (non-UEFI:) device entry in the firmware boot menu. At the Debian live GRUB menu, press Tab to edit the selected boot entry. On the line beginning with linux, append persistence after boot=live components. Press Enter to boot.

The first boot with persistence active will be slower than a standard live session—the system is running entirely from the USB drive, and the persistence overlay adds a layer of I/O. This is normal.

== Verify Persistence ==
After booting, confirm the persistence partition is mounted:

<code>$ lsblk -f</code>

The output should show sdXN mounted at /run/live/persistence/sdXN.

Create a test file:

<code>$ touch ~/persistence-test.txt</code>

Reboot with persistence enabled. If the file is present after reboot, persistence is working.

== What Didn’t Work, and Why ==

=== parted ===
parted misreported the capacity of the test USB drive: a 32 GB device was reported as approximately 125 GB. Determining where the free space began or how large to make the persistence partition was not possible from that output. parted was not used further. The broader behavior—libparted-based tools handling the isohybrid format unreliably—is documented on the Debian User mailing list [9].

=== gdisk / sgdisk ===
gdisk and sgdisk are GPT-aware tools and would seem the logical choice for a drive with a hybrid partition table. On the test USB, sgdisk produced:<blockquote>Found invalid GPT and valid MBR; converting MBR to GPT format in memory.

Warning! Secondary partition table overlaps the last partition by 33 blocks!

Could not create partition 3 from 14580 to 8177663

Non-GPT disk; not saving changes. Use -g to override.

Error encountered; not saving changes.</blockquote>The isohybrid layout places the secondary GPT header in a location that overlaps the last partition. sgdisk identifies this as invalid and refuses to proceed. Using -g to override would corrupt the hybrid layout and break the boot. sgdisk can’t be used here.

cp was also tested as an alternative to dd for writing the ISO, on the theory that it might handle the isohybrid structure differently. It produced an identical partition layout. It’s not an alternative—it’s a slower write with the same result.

== The UEFI grub> Prompt ==
On the test hardware, adding a persistence partition and then selecting the UEFI:-prefixed boot entry consistently dropped to a bare grub> prompt. Running ls from that prompt showed the internal drive’s partitions; the USB’s live partition was unreachable from there.

What was observed: the USB’s own GRUB appeared briefly—visible as a rapid flash on screen with a “no such…” error—then failed, after which the firmware handed control to the internal drive’s GRUB, which presented its own grub> prompt.

The UEFI specification states that firmware may remove boot options it considers invalid [7]; the ArchWiki documents firmware removing entries for drives it detects as changed as a known occurrence [8]. After adding the persistence partition, the USB’s EFI entry—pointing to EFI/boot/bootx64.efi—may no longer have been treated as valid by the firmware, causing it to fall through to the next available bootloader on the internal drive. Whether modifying the partition table on a removable device is sufficient to trigger this behavior, and under what firmware conditions, wasn’t reproducible in a controlled way during testing.

Selecting the plain (non-UEFI:) device entry uses the legacy MBR boot path, which loads the USB’s own GRUB directly and has no dependency on the UEFI NVRAM or EFI boot entries. On the test hardware, this path was unaffected by partition table changes.

== Anonymity ==
A Debian Live USB with persistence, used correctly, writes nothing to the host machine’s internal disk. It doesn’t modify the installed OS, create logs on the host filesystem, or leave traces in the usual locations.

It doesn’t provide anonymity. Boot events may be logged by firmware or enterprise management systems. Network activity is visible to routers, ISPs, and network administrators regardless of which OS is running. Mounting the internal disk during a live session makes its contents accessible to the live environment, and any writes to it are permanent. A live USB is a portable, self-contained Debian environment—it isn’t a privacy tool by design.

== A Note on Carrying Data ==
The persistence partition is a practical place to keep working data alongside the system state—documents, scripts, configuration files, anything you’d normally keep in ~/Documents or ~/bin. It travels with the drive.

Installed packages persist across sessions but may have hardware-specific dependencies. For general desktop use this isn’t usually an issue, but it’s worth keeping in mind if you’re installing drivers or kernel modules that assume specific hardware.

== References ==
[1] Syslinux Project. “Isohybrid.” Syslinux Wiki. https://wiki.syslinux.org/wiki/index.php?title=Isohybrid

[2] Linux Command Library. “isohybrid(1) man page.” https://linuxcommandlibrary.com/man/isohybrid

[3] Smith, Roderick W. “Hybrid MBRs.” GPT fdisk documentation. https://www.rodsbooks.com/gdisk/hybrid.html

[4] Debian Manpages. “persistence.conf(5).” live-boot-doc, Trixie. https://manpages.debian.org/trixie/live-boot-doc/persistence.conf.5.en.html

[5] Debian Live Team. “Customizing Run-Time Behaviours.” Debian Live Manual. https://live-team.pages.debian.net/live-manual/html/live-manual/customizing-run-time-behaviours.en.html

[6] Debian Manpages. “live-boot(7).” live-boot-doc. https://manpages.debian.org/unstable/live-boot-doc/live-boot.7.en.html

[7] UEFI Forum. “Boot Manager.” UEFI Specification 2.10, §3. https://uefi.org/specs/UEFI/2.10/03_Boot_Manager.html

[8] ArchWiki. “Unified Extensible Firmware Interface.” https://wiki.archlinux.org/title/Unified_Extensible_Firmware_Interface

[9] Debian User Mailing List. “Error message from GParted, means what?” November 2016. https://lists.debian.org/debian-user/2016/11/msg01017.html

''Contributed by [https://forums.debian.net/search.php?keywords=distro-nix distro-nix] on Debian User Forums on 2026-04-06.'' Comments, corrections, suggestions or resources are welcome.

Debian Live USB with Persistence

2026-04-09T21:46:52Z

Distro-nix: hyperlinks

All commands are verified on Debian 13.4 (Trixie) / 6.12.74+deb13+1-amd64

Created: 2026-04-04

Updated: 2026-04-09 - added references and restructured heading order;

ID: 005146.11

'''''A Debian Live USB with persistence boots a full Debian environment from a USB drive while retaining changes across sessions. Files you create, settings you change, and packages you install survive reboot—stored on a dedicated partition rather than the volatile live environment. Nothing is written to the host machine’s internal disk.'''''

'''''A standard live USB is stateless: each boot starts from the same ISO image. The persistence partition changes that. From a single drive you get your shell configuration, documents, utilities, browser profile, and scripts, available on any machine you boot it from.'''''

== What You’ll Need ==
A USB drive of at least 8 GB—the minimum is 4 GB, but that leaves little room for persistence. A 32 GB drive is a practical working size. Download the Debian Live ISO from the official Debian website: <nowiki>https://www.debian.org/CD/live</nowiki>

The tools required—dd, fdisk, and mkfs.ext4—are standard on Debian. On a minimal live environment, fdisk and mkfs.ext4 may not be present. Install them before starting:

<code>$ sudo apt install fdisk e2fsprogs</code>

These packages are available from the live session’s apt cache without adding sources. They don’t persist to the USB unless persistence is already active.

== Identifying Your USB Device ==
Plug in the USB drive and run:

<code>$ lsblk</code>

Your USB device will appear as /dev/sdb, /dev/sdc, or similar, depending on what else is attached. Confirm it by cross-referencing the reported size with the known capacity of the drive.

Two placeholders are used throughout this document. /dev/sdX refers to the device itself—substitute your actual device identifier everywhere it appears. /dev/sdXN refers to the persistence partition—N is the partition number you’ll confirm from lsblk output after creating it in §4.

Use the device path, not a partition path: /dev/sdb is correct; /dev/sdb1 is not.

== Boot Entry Selection ==
The firmware boot menu on many systems presents two entries for a USB device: one labeled with the device name alone, and one prefixed with UEFI:. On the hardware used for this document, selecting the UEFI:-prefixed entry after adding a persistence partition consistently produced a bare grub> prompt rather than the Debian live menu. Selecting the plain device entry booted correctly every time.

Throughout this procedure, select the plain device entry—not the one prefixed with UEFI:.

This behavior was observed on specific hardware. Whether it reproduces on all firmware is not established here. The mechanism and a possible explanation are in §5.3. If your firmware presents only a UEFI boot path, recovery from within a booted live session may be possible by correcting the GRUB prefix, but that procedure is outside the scope of this document.

= Procedure =

=== Clean the Drive ===
If the USB has been used before, residual partition signatures or bootloader data in the early sectors can interfere with the ISO write or the subsequent boot. Clear them before starting:

<code>$ sudo wipefs -a /dev/sdX</code>

<code>$ sudo dd if=/dev/zero of=/dev/sdX bs=1M count=200</code>

wipefs removes filesystem and partition table signatures. The dd command zeros the first 200 MB, covering the MBR, GPT header, and any EFI data in the early sectors.

=== Write the ISO ===
Before running dd, confirm that sync resolves to the system binary and not a shell alias:

<code>$ type sync</code>

The output should be sync is /usr/bin/sync. If it returns an alias definition instead, unalias it with unalias sync, or substitute /usr/bin/sync explicitly at the end of the dd command.

Write the ISO:

<code>$ sudo dd if=/path/to/debian-live-13.4.0-amd64-kde.iso of=/dev/sdX bs=4M status=progress && sync</code>

status=progress shows the running transfer rate. sync flushes the write cache on completion—don’t remove the drive before it returns. The write will take a few minutes depending on the drive’s speed.

=== Verify the Base Boot ===
Before adding the persistence partition, confirm the ISO write produced a bootable drive. Boot from the USB, selecting the plain (non-UEFI:) device entry. If it reaches the Debian live menu and boots cleanly, you have a confirmed working baseline. If it doesn’t boot at this stage, don’t continue—repeat §4.

== Create the Persistence Partition ==
The ISO write leaves unallocated space at the end of the drive. The persistence partition goes there. The Debian live ISO uses an isohybrid layout—a hybrid MBR/GPT partition table that supports both BIOS and UEFI boot paths. Adding a partition requires an explicit start sector; without it, fdisk’s default placement can conflict with the hybrid structure in ways that affect the boot. The failure modes of parted and gdisk with this layout are documented in §5.

Launch fdisk:

<code>$ sudo fdisk /dev/sdX</code>

Print the current partition table:

Command (m for help): <code>p</code>

Two partitions will be present: the main ISO partition (type 0 or Empty—this is expected for an isohybrid layout) and a small EFI partition of approximately 3.2 MB. The p output shows the end sector of the last partition in the End column. The persistence partition must start at the next sector—that value plus one. On the test hardware with a Debian 13.4 live ISO written to the drive, the last partition ended at sector 8175999 and the persistence partition started at sector 8176000. This figure will vary with the ISO version and release; read your own p output and don’t use the test value without verifying it matches.

Create the new partition:

Command (m for help): <code>n</code>

When fdisk prompts for the first sector, enter the start sector you identified from the p output—do not accept the default. Accept the default for the last sector to use the remaining space on the drive.

If fdisk asks whether to remove an existing ext4 signature, answer <code>Y</code>. This clears a remnant from a previous attempt.

Write the changes and exit:

Command (m for help): <code>w</code>

After fdisk writes the new partition table, the running kernel may not immediately recognize the change. partprobe signals the kernel to re-read the partition table on the device without requiring a reboot:

<code>$ sudo partprobe /dev/sdX</code>

Once it completes, confirm the new partition is visible:

<code>$ lsblk /dev/sdX</code>

Your new partition will appear beneath the device in the output. Note its full identifier—for example, sdb3 or sdc4—as you’ll need it in the next two steps. If it doesn’t appear immediately, wait a moment and run lsblk again before proceeding.

== Format the Partition ==
Format the new partition as ext4 with the label persistence. The Debian live system identifies the persistence partition by this label at boot:

<code>$ sudo mkfs.ext4 -L persistence /dev/sdXN</code>

== Create the Persistence Configuration ==
Mount the partition:

<code>$ sudo mount /dev/sdXN /mnt</code>

Create the configuration file:

<code>$ echo "/ union" | sudo tee /mnt/persistence.conf</code>

The / union directive instructs the live system to mount the entire filesystem as a union overlay. Changes made during a session are written to the persistence partition rather than discarded on shutdown.

Unmount the partition:

<code>$ sudo umount /mnt</code>

== Boot with Persistence ==
Boot from the USB, selecting the plain (non-UEFI:) device entry in the firmware boot menu. At the Debian live GRUB menu, press Tab to edit the selected boot entry. On the line beginning with linux, append persistence after boot=live components. Press Enter to boot.

The first boot with persistence active will be slower than a standard live session—the system is running entirely from the USB drive, and the persistence overlay adds a layer of I/O. This is normal.

== Verify Persistence ==
After booting, confirm the persistence partition is mounted:

<code>$ lsblk -f</code>

The output should show sdXN mounted at /run/live/persistence/sdXN.

Create a test file:

<code>$ touch ~/persistence-test.txt</code>

Reboot with persistence enabled. If the file is present after reboot, persistence is working.

== What Didn’t Work, and Why ==

=== parted ===
parted misreported the capacity of the test USB drive: a 32 GB device was reported as approximately 125 GB. Determining where the free space began or how large to make the persistence partition was not possible from that output. parted was not used further. The broader behavior—libparted-based tools handling the isohybrid format unreliably—is documented on the Debian User mailing list [9].

=== gdisk / sgdisk ===
gdisk and sgdisk are GPT-aware tools and would seem the logical choice for a drive with a hybrid partition table. On the test USB, sgdisk produced:<blockquote>Found invalid GPT and valid MBR; converting MBR to GPT format in memory.

Warning! Secondary partition table overlaps the last partition by 33 blocks!

Could not create partition 3 from 14580 to 8177663

Non-GPT disk; not saving changes. Use -g to override.

Error encountered; not saving changes.</blockquote>The isohybrid layout places the secondary GPT header in a location that overlaps the last partition. sgdisk identifies this as invalid and refuses to proceed. Using -g to override would corrupt the hybrid layout and break the boot. sgdisk can’t be used here.

cp was also tested as an alternative to dd for writing the ISO, on the theory that it might handle the isohybrid structure differently. It produced an identical partition layout. It’s not an alternative—it’s a slower write with the same result.

== The UEFI grub> Prompt ==
On the test hardware, adding a persistence partition and then selecting the UEFI:-prefixed boot entry consistently dropped to a bare grub> prompt. Running ls from that prompt showed the internal drive’s partitions; the USB’s live partition was unreachable from there.

What was observed: the USB’s own GRUB appeared briefly—visible as a rapid flash on screen with a “no such…” error—then failed, after which the firmware handed control to the internal drive’s GRUB, which presented its own grub> prompt.

The UEFI specification states that firmware may remove boot options it considers invalid [7]; the ArchWiki documents firmware removing entries for drives it detects as changed as a known occurrence [8]. After adding the persistence partition, the USB’s EFI entry—pointing to EFI/boot/bootx64.efi—may no longer have been treated as valid by the firmware, causing it to fall through to the next available bootloader on the internal drive. Whether modifying the partition table on a removable device is sufficient to trigger this behavior, and under what firmware conditions, wasn’t reproducible in a controlled way during testing.

Selecting the plain (non-UEFI:) device entry uses the legacy MBR boot path, which loads the USB’s own GRUB directly and has no dependency on the UEFI NVRAM or EFI boot entries. On the test hardware, this path was unaffected by partition table changes.

== Anonymity ==
A Debian Live USB with persistence, used correctly, writes nothing to the host machine’s internal disk. It doesn’t modify the installed OS, create logs on the host filesystem, or leave traces in the usual locations.

It doesn’t provide anonymity. Boot events may be logged by firmware or enterprise management systems. Network activity is visible to routers, ISPs, and network administrators regardless of which OS is running. Mounting the internal disk during a live session makes its contents accessible to the live environment, and any writes to it are permanent. A live USB is a portable, self-contained Debian environment—it isn’t a privacy tool by design.

== A Note on Carrying Data ==
The persistence partition is a practical place to keep working data alongside the system state—documents, scripts, configuration files, anything you’d normally keep in ~/Documents or ~/bin. It travels with the drive.

Installed packages persist across sessions but may have hardware-specific dependencies. For general desktop use this isn’t usually an issue, but it’s worth keeping in mind if you’re installing drivers or kernel modules that assume specific hardware.

== References ==
[1] Syslinux Project. “Isohybrid.” Syslinux Wiki. https://wiki.syslinux.org/wiki/index.php?title=Isohybrid

[2] Linux Command Library. “isohybrid(1) man page.” https://linuxcommandlibrary.com/man/isohybrid

[3] Smith, Roderick W. “Hybrid MBRs.” GPT fdisk documentation. https://www.rodsbooks.com/gdisk/hybrid.html

[4] Debian Manpages. “persistence.conf(5).” live-boot-doc, Trixie. https://manpages.debian.org/trixie/live-boot-doc/persistence.conf.5.en.html

[5] Debian Live Team. “Customizing Run-Time Behaviours.” Debian Live Manual. https://live-team.pages.debian.net/live-manual/html/live-manual/customizing-run-time-behaviours.en.html

[6] Debian Manpages. “live-boot(7).” live-boot-doc. https://manpages.debian.org/unstable/live-boot-doc/live-boot.7.en.html

[7] UEFI Forum. “Boot Manager.” UEFI Specification 2.10, §3. https://uefi.org/specs/UEFI/2.10/03_Boot_Manager.html

[8] ArchWiki. “Unified Extensible Firmware Interface.” https://wiki.archlinux.org/title/Unified_Extensible_Firmware_Interface

[9] Debian User Mailing List. “Error message from GParted, means what?” November 2016. https://lists.debian.org/debian-user/2016/11/msg01017.html

''Contributed by [https://forums.debian.net/search.php?keywords=distro-nix distro-nix] on Debian User Forums on 2026-04-06.'' Comments, corrections, suggestions or resources are welcome.

Debian Live USB with Persistence

2026-04-09T19:20:25Z

Distro-nix: minor edits

All commands are verified on Debian 13.4 (Trixie) / 6.12.74+deb13+1-amd64

Created: 2026-04-04

Updated: 2026-04-09 - added references and restructured heading order;

ID: 005146.11

'''''A Debian Live USB with persistence boots a full Debian environment from a USB drive while retaining changes across sessions. Files you create, settings you change, and packages you install survive reboot—stored on a dedicated partition rather than the volatile live environment. Nothing is written to the host machine’s internal disk.'''''

'''''A standard live USB is stateless: each boot starts from the same ISO image. The persistence partition changes that. From a single drive you get your shell configuration, documents, utilities, browser profile, and scripts, available on any machine you boot it from.'''''

== What You’ll Need ==
A USB drive of at least 8 GB—the minimum is 4 GB, but that leaves little room for persistence. A 32 GB drive is a practical working size. Download the Debian Live ISO from the official Debian website: <nowiki>https://www.debian.org/CD/live</nowiki>

The tools required—dd, fdisk, and mkfs.ext4—are standard on Debian. On a minimal live environment, fdisk and mkfs.ext4 may not be present. Install them before starting:

<code>$ sudo apt install fdisk e2fsprogs</code>

These packages are available from the live session’s apt cache without adding sources. They don’t persist to the USB unless persistence is already active.

== Identifying Your USB Device ==
Plug in the USB drive and run:

<code>$ lsblk</code>

Your USB device will appear as /dev/sdb, /dev/sdc, or similar, depending on what else is attached. Confirm it by cross-referencing the reported size with the known capacity of the drive.

Two placeholders are used throughout this document. /dev/sdX refers to the device itself—substitute your actual device identifier everywhere it appears. /dev/sdXN refers to the persistence partition—N is the partition number you’ll confirm from lsblk output after creating it in §4.

Use the device path, not a partition path: /dev/sdb is correct; /dev/sdb1 is not.

== Boot Entry Selection ==
The firmware boot menu on many systems presents two entries for a USB device: one labeled with the device name alone, and one prefixed with UEFI:. On the hardware used for this document, selecting the UEFI:-prefixed entry after adding a persistence partition consistently produced a bare grub> prompt rather than the Debian live menu. Selecting the plain device entry booted correctly every time.

Throughout this procedure, select the plain device entry—not the one prefixed with UEFI:.

This behavior was observed on specific hardware. Whether it reproduces on all firmware is not established here. The mechanism and a possible explanation are in §5.3. If your firmware presents only a UEFI boot path, recovery from within a booted live session may be possible by correcting the GRUB prefix, but that procedure is outside the scope of this document.

= Procedure =

=== Clean the Drive ===
If the USB has been used before, residual partition signatures or bootloader data in the early sectors can interfere with the ISO write or the subsequent boot. Clear them before starting:

<code>$ sudo wipefs -a /dev/sdX</code>

<code>$ sudo dd if=/dev/zero of=/dev/sdX bs=1M count=200</code>

wipefs removes filesystem and partition table signatures. The dd command zeros the first 200 MB, covering the MBR, GPT header, and any EFI data in the early sectors.

=== Write the ISO ===
Before running dd, confirm that sync resolves to the system binary and not a shell alias:

<code>$ type sync</code>

The output should be sync is /usr/bin/sync. If it returns an alias definition instead, unalias it with unalias sync, or substitute /usr/bin/sync explicitly at the end of the dd command.

Write the ISO:

<code>$ sudo dd if=/path/to/debian-live-13.4.0-amd64-kde.iso of=/dev/sdX bs=4M status=progress && sync</code>

status=progress shows the running transfer rate. sync flushes the write cache on completion—don’t remove the drive before it returns. The write will take a few minutes depending on the drive’s speed.

=== Verify the Base Boot ===
Before adding the persistence partition, confirm the ISO write produced a bootable drive. Boot from the USB, selecting the plain (non-UEFI:) device entry. If it reaches the Debian live menu and boots cleanly, you have a confirmed working baseline. If it doesn’t boot at this stage, don’t continue—repeat §4.

== Create the Persistence Partition ==
The ISO write leaves unallocated space at the end of the drive. The persistence partition goes there. The Debian live ISO uses an isohybrid layout—a hybrid MBR/GPT partition table that supports both BIOS and UEFI boot paths. Adding a partition requires an explicit start sector; without it, fdisk’s default placement can conflict with the hybrid structure in ways that affect the boot. The failure modes of parted and gdisk with this layout are documented in §5.

Launch fdisk:

<code>$ sudo fdisk /dev/sdX</code>

Print the current partition table:

Command (m for help): <code>p</code>

Two partitions will be present: the main ISO partition (type 0 or Empty—this is expected for an isohybrid layout) and a small EFI partition of approximately 3.2 MB. The p output shows the end sector of the last partition in the End column. The persistence partition must start at the next sector—that value plus one. On the test hardware with a Debian 13.4 live ISO written to the drive, the last partition ended at sector 8175999 and the persistence partition started at sector 8176000. This figure will vary with the ISO version and release; read your own p output and don’t use the test value without verifying it matches.

Create the new partition:

Command (m for help): <code>n</code>

When fdisk prompts for the first sector, enter the start sector you identified from the p output—do not accept the default. Accept the default for the last sector to use the remaining space on the drive.

If fdisk asks whether to remove an existing ext4 signature, answer <code>Y</code>. This clears a remnant from a previous attempt.

Write the changes and exit:

Command (m for help): <code>w</code>

After fdisk writes the new partition table, the running kernel may not immediately recognize the change. partprobe signals the kernel to re-read the partition table on the device without requiring a reboot:

<code>$ sudo partprobe /dev/sdX</code>

Once it completes, confirm the new partition is visible:

<code>$ lsblk /dev/sdX</code>

Your new partition will appear beneath the device in the output. Note its full identifier—for example, sdb3 or sdc4—as you’ll need it in the next two steps. If it doesn’t appear immediately, wait a moment and run lsblk again before proceeding.

== Format the Partition ==
Format the new partition as ext4 with the label persistence. The Debian live system identifies the persistence partition by this label at boot:

<code>$ sudo mkfs.ext4 -L persistence /dev/sdXN</code>

== Create the Persistence Configuration ==
Mount the partition:

<code>$ sudo mount /dev/sdXN /mnt</code>

Create the configuration file:

<code>$ echo "/ union" | sudo tee /mnt/persistence.conf</code>

The / union directive instructs the live system to mount the entire filesystem as a union overlay. Changes made during a session are written to the persistence partition rather than discarded on shutdown.

Unmount the partition:

<code>$ sudo umount /mnt</code>

== Boot with Persistence ==
Boot from the USB, selecting the plain (non-UEFI:) device entry in the firmware boot menu. At the Debian live GRUB menu, press Tab to edit the selected boot entry. On the line beginning with linux, append persistence after boot=live components. Press Enter to boot.

The first boot with persistence active will be slower than a standard live session—the system is running entirely from the USB drive, and the persistence overlay adds a layer of I/O. This is normal.

== Verify Persistence ==
After booting, confirm the persistence partition is mounted:

<code>$ lsblk -f</code>

The output should show sdXN mounted at /run/live/persistence/sdXN.

Create a test file:

<code>$ touch ~/persistence-test.txt</code>

Reboot with persistence enabled. If the file is present after reboot, persistence is working.

== What Didn’t Work, and Why ==

=== parted ===
parted misreported the capacity of the test USB drive: a 32 GB device was reported as approximately 125 GB. Determining where the free space began or how large to make the persistence partition was not possible from that output. parted was not used further. The broader behavior—libparted-based tools handling the isohybrid format unreliably—is documented on the Debian User mailing list [9].

=== gdisk / sgdisk ===
gdisk and sgdisk are GPT-aware tools and would seem the logical choice for a drive with a hybrid partition table. On the test USB, sgdisk produced:<blockquote>Found invalid GPT and valid MBR; converting MBR to GPT format in memory.

Warning! Secondary partition table overlaps the last partition by 33 blocks!

Could not create partition 3 from 14580 to 8177663

Non-GPT disk; not saving changes. Use -g to override.

Error encountered; not saving changes.</blockquote>The isohybrid layout places the secondary GPT header in a location that overlaps the last partition. sgdisk identifies this as invalid and refuses to proceed. Using -g to override would corrupt the hybrid layout and break the boot. sgdisk can’t be used here.

cp was also tested as an alternative to dd for writing the ISO, on the theory that it might handle the isohybrid structure differently. It produced an identical partition layout. It’s not an alternative—it’s a slower write with the same result.

== The UEFI grub> Prompt ==
On the test hardware, adding a persistence partition and then selecting the UEFI:-prefixed boot entry consistently dropped to a bare grub> prompt. Running ls from that prompt showed the internal drive’s partitions; the USB’s live partition was unreachable from there.

What was observed: the USB’s own GRUB appeared briefly—visible as a rapid flash on screen with a “no such…” error—then failed, after which the firmware handed control to the internal drive’s GRUB, which presented its own grub> prompt.

The UEFI specification states that firmware may remove boot options it considers invalid [7]; the ArchWiki documents firmware removing entries for drives it detects as changed as a known occurrence [8]. After adding the persistence partition, the USB’s EFI entry—pointing to EFI/boot/bootx64.efi—may no longer have been treated as valid by the firmware, causing it to fall through to the next available bootloader on the internal drive. Whether modifying the partition table on a removable device is sufficient to trigger this behavior, and under what firmware conditions, wasn’t reproducible in a controlled way during testing.

Selecting the plain (non-UEFI:) device entry uses the legacy MBR boot path, which loads the USB’s own GRUB directly and has no dependency on the UEFI NVRAM or EFI boot entries. On the test hardware, this path was unaffected by partition table changes.

== Anonymity ==
A Debian Live USB with persistence, used correctly, writes nothing to the host machine’s internal disk. It doesn’t modify the installed OS, create logs on the host filesystem, or leave traces in the usual locations.

It doesn’t provide anonymity. Boot events may be logged by firmware or enterprise management systems. Network activity is visible to routers, ISPs, and network administrators regardless of which OS is running. Mounting the internal disk during a live session makes its contents accessible to the live environment, and any writes to it are permanent. A live USB is a portable, self-contained Debian environment—it isn’t a privacy tool by design.

== A Note on Carrying Data ==
The persistence partition is a practical place to keep working data alongside the system state—documents, scripts, configuration files, anything you’d normally keep in ~/Documents or ~/bin. It travels with the drive.

Installed packages persist across sessions but may have hardware-specific dependencies. For general desktop use this isn’t usually an issue, but it’s worth keeping in mind if you’re installing drivers or kernel modules that assume specific hardware.

== References ==
[1] Syslinux Project. “Isohybrid.” Syslinux Wiki. <nowiki>https://wiki.syslinux.org/wiki/index.php?title=Isohybrid</nowiki>

[2] Linux Command Library. “isohybrid(1) man page.” <nowiki>https://linuxcommandlibrary.com/man/isohybrid</nowiki>

[3] Smith, Roderick W. “Hybrid MBRs.” GPT fdisk documentation. <nowiki>https://www.rodsbooks.com/gdisk/hybrid.html</nowiki>

[4] Debian Manpages. “persistence.conf(5).” live-boot-doc, Trixie. <nowiki>https://manpages.debian.org/trixie/live-boot-doc/persistence.conf.5.en.html</nowiki>

[5] Debian Live Team. “Customizing Run-Time Behaviours.” Debian Live Manual. <nowiki>https://live-team.pages.debian.net/live-manual/html/live-manual/customizing-run-time-behaviours.en.html</nowiki>

[6] Debian Manpages. “live-boot(7).” live-boot-doc. <nowiki>https://manpages.debian.org/unstable/live-boot-doc/live-boot.7.en.html</nowiki>

[7] UEFI Forum. “Boot Manager.” UEFI Specification 2.10, §3. <nowiki>https://uefi.org/specs/UEFI/2.10/03_Boot_Manager.html</nowiki>

[8] ArchWiki. “Unified Extensible Firmware Interface.” <nowiki>https://wiki.archlinux.org/title/Unified_Extensible_Firmware_Interface</nowiki>

[9] Debian User Mailing List. “Error message from GParted, means what?” November 2016. <nowiki>https://lists.debian.org/debian-user/2016/11/msg01017.html</nowiki>

''Contributed by distro-nix on Debian User Forums on 2026-04-06.'' Comments, corrections, suggestions or resources are welcome.

Debian Live USB with Persistence

2026-04-09T19:13:46Z

Distro-nix: proff reading

All commands are verified on Debian 13.4 (Trixie) / 6.12.74+deb13+1-amd64

Created: 2026-04-04

Updated: 2026-04-09 - added references and restructured heading order;

ID: 005146.11

'''''A Debian Live USB with persistence boots a full Debian environment from a USB drive while retaining changes across sessions. Files you create, settings you change, and packages you install survive reboot—stored on a dedicated partition rather than the volatile live environment. Nothing is written to the host machine’s internal disk.'''''

'''''A standard live USB is stateless: each boot starts from the same ISO image. The persistence partition changes that. From a single drive you get your shell configuration, documents, utilities, browser profile, and scripts, available on any machine you boot it from.'''''

== What You’ll Need ==
A USB drive of at least 8 GB—the minimum is 4 GB, but that leaves little room for persistence. A 32 GB drive is a practical working size. Download the Debian Live ISO from the official Debian website: <nowiki>https://www.debian.org/CD/live</nowiki>

The tools required—dd, fdisk, and mkfs.ext4—are standard on Debian. On a minimal live environment, fdisk and mkfs.ext4 may not be present. Install them before starting:

<code>$ sudo apt install fdisk e2fsprogs</code>

These packages are available from the live session’s apt cache without adding sources. They don’t persist to the USB unless persistence is already active.

== Identifying Your USB Device ==
Plug in the USB drive and run:

<code>$ lsblk</code>

Your USB device will appear as /dev/sdb, /dev/sdc, or similar, depending on what else is attached. Confirm it by cross-referencing the reported size with the known capacity of the drive.

Two placeholders are used throughout this document. /dev/sdX refers to the device itself—substitute your actual device identifier everywhere it appears. /dev/sdXN refers to the persistence partition—N is the partition number you’ll confirm from lsblk output after creating it in §4.

Use the device path, not a partition path: /dev/sdb is correct; /dev/sdb1 is not.

== Boot Entry Selection ==
The firmware boot menu on many systems presents two entries for a USB device: one labeled with the device name alone, and one prefixed with UEFI:. On the hardware used for this document, selecting the UEFI:-prefixed entry after adding a persistence partition consistently produced a bare grub> prompt rather than the Debian live menu. Selecting the plain device entry booted correctly every time.

Throughout this procedure, select the plain device entry—not the one prefixed with UEFI:.

This behavior was observed on specific hardware. Whether it reproduces on all firmware is not established here. The mechanism and a possible explanation are in §5.3. If your firmware presents only a UEFI boot path, recovery from within a booted live session may be possible by correcting the GRUB prefix, but that procedure is outside the scope of this document.

== Procedure ==

=== Clean the Drive ===
If the USB has been used before, residual partition signatures or bootloader data in the early sectors can interfere with the ISO write or the subsequent boot. Clear them before starting:

<code>$ sudo wipefs -a /dev/sdX</code>

<code>$ sudo dd if=/dev/zero of=/dev/sdX bs=1M count=200</code>

wipefs removes filesystem and partition table signatures. The dd command zeros the first 200 MB, covering the MBR, GPT header, and any EFI data in the early sectors.

== Write the ISO ==
Before running dd, confirm that sync resolves to the system binary and not a shell alias:

<code>$ type sync</code>

The output should be sync is /usr/bin/sync. If it returns an alias definition instead, unalias it with unalias sync, or substitute /usr/bin/sync explicitly at the end of the dd command.

Write the ISO:

<code>$ sudo dd if=/path/to/debian-live-13.4.0-amd64-kde.iso of=/dev/sdX bs=4M status=progress && sync</code>

status=progress shows the running transfer rate. sync flushes the write cache on completion—don’t remove the drive before it returns. The write will take a few minutes depending on the drive’s speed.

== Verify the Base Boot ==
Before adding the persistence partition, confirm the ISO write produced a bootable drive. Boot from the USB, selecting the plain (non-UEFI:) device entry. If it reaches the Debian live menu and boots cleanly, you have a confirmed working baseline. If it doesn’t boot at this stage, don’t continue—repeat §4.

== Create the Persistence Partition ==
The ISO write leaves unallocated space at the end of the drive. The persistence partition goes there. The Debian live ISO uses an isohybrid layout—a hybrid MBR/GPT partition table that supports both BIOS and UEFI boot paths. Adding a partition requires an explicit start sector; without it, fdisk’s default placement can conflict with the hybrid structure in ways that affect the boot. The failure modes of parted and gdisk with this layout are documented in §5.

Launch fdisk:

<code>$ sudo fdisk /dev/sdX</code>

Print the current partition table:

Command (m for help): <code>p</code>

Two partitions will be present: the main ISO partition (type 0 or Empty—this is expected for an isohybrid layout) and a small EFI partition of approximately 3.2 MB. The p output shows the end sector of the last partition in the End column. The persistence partition must start at the next sector—that value plus one. On the test hardware with a Debian 13.4 live ISO written to the drive, the last partition ended at sector 8175999 and the persistence partition started at sector 8176000. This figure will vary with the ISO version and release; read your own p output and don’t use the test value without verifying it matches.

Create the new partition:

Command (m for help): <code>n</code>

When fdisk prompts for the first sector, enter the start sector you identified from the p output—do not accept the default. Accept the default for the last sector to use the remaining space on the drive.

If fdisk asks whether to remove an existing ext4 signature, answer <code>Y</code>. This clears a remnant from a previous attempt.

Write the changes and exit:

Command (m for help): <code>w</code>

After fdisk writes the new partition table, the running kernel may not immediately recognize the change. partprobe signals the kernel to re-read the partition table on the device without requiring a reboot:

<code>$ sudo partprobe /dev/sdX</code>

Once it completes, confirm the new partition is visible:

<code>$ lsblk /dev/sdX</code>

Your new partition will appear beneath the device in the output. Note its full identifier—for example, sdb3 or sdc4—as you’ll need it in the next two steps. If it doesn’t appear immediately, wait a moment and run lsblk again before proceeding.

== Format the Partition ==
Format the new partition as ext4 with the label persistence. The Debian live system identifies the persistence partition by this label at boot:

<code>$ sudo mkfs.ext4 -L persistence /dev/sdXN</code>

== Create the Persistence Configuration ==
Mount the partition:

<code>$ sudo mount /dev/sdXN /mnt</code>

Create the configuration file:

<code>$ echo "/ union" | sudo tee /mnt/persistence.conf</code>

The / union directive instructs the live system to mount the entire filesystem as a union overlay. Changes made during a session are written to the persistence partition rather than discarded on shutdown.

Unmount the partition:

<code>$ sudo umount /mnt</code>

== Boot with Persistence ==
Boot from the USB, selecting the plain (non-UEFI:) device entry in the firmware boot menu. At the Debian live GRUB menu, press Tab to edit the selected boot entry. On the line beginning with linux, append persistence after boot=live components. Press Enter to boot.

The first boot with persistence active will be slower than a standard live session—the system is running entirely from the USB drive, and the persistence overlay adds a layer of I/O. This is normal.

== Verify Persistence ==
After booting, confirm the persistence partition is mounted:

<code>$ lsblk -f</code>

The output should show sdXN mounted at /run/live/persistence/sdXN.

Create a test file:

<code>$ touch ~/persistence-test.txt</code>

Reboot with persistence enabled. If the file is present after reboot, persistence is working.

== What Didn’t Work, and Why ==

=== parted ===
parted misreported the capacity of the test USB drive: a 32 GB device was reported as approximately 125 GB. Determining where the free space began or how large to make the persistence partition was not possible from that output. parted was not used further. The broader behavior—libparted-based tools handling the isohybrid format unreliably—is documented on the Debian User mailing list [9].

=== gdisk / sgdisk ===
gdisk and sgdisk are GPT-aware tools and would seem the logical choice for a drive with a hybrid partition table. On the test USB, sgdisk produced:<blockquote>Found invalid GPT and valid MBR; converting MBR to GPT format in memory.

Warning! Secondary partition table overlaps the last partition by 33 blocks!

Could not create partition 3 from 14580 to 8177663

Non-GPT disk; not saving changes. Use -g to override.

Error encountered; not saving changes.</blockquote>The isohybrid layout places the secondary GPT header in a location that overlaps the last partition. sgdisk identifies this as invalid and refuses to proceed. Using -g to override would corrupt the hybrid layout and break the boot. sgdisk can’t be used here.

cp was also tested as an alternative to dd for writing the ISO, on the theory that it might handle the isohybrid structure differently. It produced an identical partition layout. It’s not an alternative—it’s a slower write with the same result.

== The UEFI grub> Prompt ==
On the test hardware, adding a persistence partition and then selecting the UEFI:-prefixed boot entry consistently dropped to a bare grub> prompt. Running ls from that prompt showed the internal drive’s partitions; the USB’s live partition was unreachable from there.

What was observed: the USB’s own GRUB appeared briefly—visible as a rapid flash on screen with a “no such…” error—then failed, after which the firmware handed control to the internal drive’s GRUB, which presented its own grub> prompt.

The UEFI specification states that firmware may remove boot options it considers invalid [7]; the ArchWiki documents firmware removing entries for drives it detects as changed as a known occurrence [8]. After adding the persistence partition, the USB’s EFI entry—pointing to EFI/boot/bootx64.efi—may no longer have been treated as valid by the firmware, causing it to fall through to the next available bootloader on the internal drive. Whether modifying the partition table on a removable device is sufficient to trigger this behavior, and under what firmware conditions, wasn’t reproducible in a controlled way during testing.

Selecting the plain (non-UEFI:) device entry uses the legacy MBR boot path, which loads the USB’s own GRUB directly and has no dependency on the UEFI NVRAM or EFI boot entries. On the test hardware, this path was unaffected by partition table changes.

== Anonymity ==
A Debian Live USB with persistence, used correctly, writes nothing to the host machine’s internal disk. It doesn’t modify the installed OS, create logs on the host filesystem, or leave traces in the usual locations.

It doesn’t provide anonymity. Boot events may be logged by firmware or enterprise management systems. Network activity is visible to routers, ISPs, and network administrators regardless of which OS is running. Mounting the internal disk during a live session makes its contents accessible to the live environment, and any writes to it are permanent. A live USB is a portable, self-contained Debian environment—it isn’t a privacy tool by design.

== A Note on Carrying Data ==
The persistence partition is a practical place to keep working data alongside the system state—documents, scripts, configuration files, anything you’d normally keep in ~/Documents or ~/bin. It travels with the drive.

Installed packages persist across sessions but may have hardware-specific dependencies. For general desktop use this isn’t usually an issue, but it’s worth keeping in mind if you’re installing drivers or kernel modules that assume specific hardware.

== References ==
[1] Syslinux Project. “Isohybrid.” Syslinux Wiki. <nowiki>https://wiki.syslinux.org/wiki/index.php?title=Isohybrid</nowiki>

[2] Linux Command Library. “isohybrid(1) man page.” <nowiki>https://linuxcommandlibrary.com/man/isohybrid</nowiki>

[3] Smith, Roderick W. “Hybrid MBRs.” GPT fdisk documentation. <nowiki>https://www.rodsbooks.com/gdisk/hybrid.html</nowiki>

[4] Debian Manpages. “persistence.conf(5).” live-boot-doc, Trixie. <nowiki>https://manpages.debian.org/trixie/live-boot-doc/persistence.conf.5.en.html</nowiki>

[5] Debian Live Team. “Customizing Run-Time Behaviours.” Debian Live Manual. <nowiki>https://live-team.pages.debian.net/live-manual/html/live-manual/customizing-run-time-behaviours.en.html</nowiki>

[6] Debian Manpages. “live-boot(7).” live-boot-doc. <nowiki>https://manpages.debian.org/unstable/live-boot-doc/live-boot.7.en.html</nowiki>

[7] UEFI Forum. “Boot Manager.” UEFI Specification 2.10, §3. <nowiki>https://uefi.org/specs/UEFI/2.10/03_Boot_Manager.html</nowiki>

[8] ArchWiki. “Unified Extensible Firmware Interface.” <nowiki>https://wiki.archlinux.org/title/Unified_Extensible_Firmware_Interface</nowiki>

[9] Debian User Mailing List. “Error message from GParted, means what?” November 2016. <nowiki>https://lists.debian.org/debian-user/2016/11/msg01017.html</nowiki>

''Contributed by distro-nix on Debian User Forums on 2026-04-06.'' Comments, corrections, suggestions or resources are welcome.

Debian Live USB with Persistence

2026-04-09T19:11:07Z

Distro-nix: Added references and restructured heading order

All commands are verified on Debian 13.4 (Trixie) / 6.12.74+deb13+1-amd64

Created: 2026-04-04

Updated: 2026-04-09 - added references and restructured heading order;

ID: 005146.11

'''''A Debian Live USB with persistence boots a full Debian environment from a USB drive while retaining changes across sessions. Files you create, settings you change, and packages you install survive reboot—stored on a dedicated partition rather than the volatile live environment. Nothing is written to the host machine’s internal disk.'''''

'''''A standard live USB is stateless: each boot starts from the same ISO image. The persistence partition changes that. From a single drive you get your shell configuration, documents, utilities, browser profile, and scripts, available on any machine you boot it from.'''''

== What You’ll Need ==
A USB drive of at least 8 GB—the minimum is 4 GB, but that leaves little room for persistence. A 32 GB drive is a practical working size. Download the Debian Live ISO from the official Debian website: <nowiki>https://www.debian.org/CD/live</nowiki>

The tools required—dd, fdisk, and mkfs.ext4—are standard on Debian. On a minimal live environment, fdisk and mkfs.ext4 may not be present. Install them before starting:

<code>$ sudo apt install fdisk e2fsprogs</code>

These packages are available from the live session’s apt cache without adding sources. They don’t persist to the USB unless persistence is already active.

== Identifying Your USB Device ==
Plug in the USB drive and run:

<code>$ lsblk</code>

Your USB device will appear as /dev/sdb, /dev/sdc, or similar, depending on what else is attached. Confirm it by cross-referencing the reported size with the known capacity of the drive.

Two placeholders are used throughout this document. /dev/sdX refers to the device itself—substitute your actual device identifier everywhere it appears. /dev/sdXN refers to the persistence partition—N is the partition number you’ll confirm from lsblk output after creating it in §4.4.

Use the device path, not a partition path: /dev/sdb is correct; /dev/sdb1 is not.

== Boot Entry Selection ==
The firmware boot menu on many systems presents two entries for a USB device: one labeled with the device name alone, and one prefixed with UEFI:. On the hardware used for this document, selecting the UEFI:-prefixed entry after adding a persistence partition consistently produced a bare grub> prompt rather than the Debian live menu. Selecting the plain device entry booted correctly every time.

Throughout this procedure, select the plain device entry—not the one prefixed with UEFI:.

This behavior was observed on specific hardware. Whether it reproduces on all firmware is not established here. The mechanism and a possible explanation are in §5.3. If your firmware presents only a UEFI boot path, recovery from within a booted live session may be possible by correcting the GRUB prefix, but that procedure is outside the scope of this document.

== Procedure ==

=== Clean the Drive ===
If the USB has been used before, residual partition signatures or bootloader data in the early sectors can interfere with the ISO write or the subsequent boot. Clear them before starting:

<code>$ sudo wipefs -a /dev/sdX</code>

<code>$ sudo dd if=/dev/zero of=/dev/sdX bs=1M count=200</code>

wipefs removes filesystem and partition table signatures. The dd command zeros the first 200 MB, covering the MBR, GPT header, and any EFI data in the early sectors.

== Write the ISO ==
Before running dd, confirm that sync resolves to the system binary and not a shell alias:

<code>$ type sync</code>

The output should be sync is /usr/bin/sync. If it returns an alias definition instead, unalias it with unalias sync, or substitute /usr/bin/sync explicitly at the end of the dd command.

Write the ISO:

<code>$ sudo dd if=/path/to/debian-live-13.4.0-amd64-kde.iso of=/dev/sdX bs=4M status=progress && sync</code>

status=progress shows the running transfer rate. sync flushes the write cache on completion—don’t remove the drive before it returns. The write will take a few minutes depending on the drive’s speed.

== Verify the Base Boot ==
Before adding the persistence partition, confirm the ISO write produced a bootable drive. Boot from the USB, selecting the plain (non-UEFI:) device entry. If it reaches the Debian live menu and boots cleanly, you have a confirmed working baseline. If it doesn’t boot at this stage, don’t continue—repeat §4.1 and §4.2.

== Create the Persistence Partition ==
The ISO write leaves unallocated space at the end of the drive. The persistence partition goes there. The Debian live ISO uses an isohybrid layout—a hybrid MBR/GPT partition table that supports both BIOS and UEFI boot paths. Adding a partition requires an explicit start sector; without it, fdisk’s default placement can conflict with the hybrid structure in ways that affect the boot. The failure modes of parted and gdisk with this layout are documented in §5.

Launch fdisk:

<code>$ sudo fdisk /dev/sdX</code>

Print the current partition table:

Command (m for help): <code>p</code>

Two partitions will be present: the main ISO partition (type 0 or Empty—this is expected for an isohybrid layout) and a small EFI partition of approximately 3.2 MB. The p output shows the end sector of the last partition in the End column. The persistence partition must start at the next sector—that value plus one. On the test hardware with a Debian 13.4 live ISO written to the drive, the last partition ended at sector 8175999 and the persistence partition started at sector 8176000. This figure will vary with the ISO version and release; read your own p output and don’t use the test value without verifying it matches.

Create the new partition:

Command (m for help): <code>n</code>

When fdisk prompts for the first sector, enter the start sector you identified from the p output—do not accept the default. Accept the default for the last sector to use the remaining space on the drive.

If fdisk asks whether to remove an existing ext4 signature, answer <code>Y</code>. This clears a remnant from a previous attempt.

Write the changes and exit:

Command (m for help): <code>w</code>

After fdisk writes the new partition table, the running kernel may not immediately recognize the change. partprobe signals the kernel to re-read the partition table on the device without requiring a reboot:

<code>$ sudo partprobe /dev/sdX</code>

Once it completes, confirm the new partition is visible:

<code>$ lsblk /dev/sdX</code>

Your new partition will appear beneath the device in the output. Note its full identifier—for example, sdb3 or sdc4—as you’ll need it in the next two steps. If it doesn’t appear immediately, wait a moment and run lsblk again before proceeding.

== Format the Partition ==
Format the new partition as ext4 with the label persistence. The Debian live system identifies the persistence partition by this label at boot:

<code>$ sudo mkfs.ext4 -L persistence /dev/sdXN</code>

== Create the Persistence Configuration ==
Mount the partition:

<code>$ sudo mount /dev/sdXN /mnt</code>

Create the configuration file:

<code>$ echo "/ union" | sudo tee /mnt/persistence.conf</code>

The / union directive instructs the live system to mount the entire filesystem as a union overlay. Changes made during a session are written to the persistence partition rather than discarded on shutdown.

Unmount the partition:

<code>$ sudo umount /mnt</code>

== Boot with Persistence ==
Boot from the USB, selecting the plain (non-UEFI:) device entry in the firmware boot menu. At the Debian live GRUB menu, press Tab to edit the selected boot entry. On the line beginning with linux, append persistence after boot=live components. Press Enter to boot.

The first boot with persistence active will be slower than a standard live session—the system is running entirely from the USB drive, and the persistence overlay adds a layer of I/O. This is normal.

== Verify Persistence ==
After booting, confirm the persistence partition is mounted:

<code>$ lsblk -f</code>

The output should show sdXN mounted at /run/live/persistence/sdXN.

Create a test file:

<code>$ touch ~/persistence-test.txt</code>

Reboot with persistence enabled. If the file is present after reboot, persistence is working.

== What Didn’t Work, and Why ==

=== parted ===
parted misreported the capacity of the test USB drive: a 32 GB device was reported as approximately 125 GB. Determining where the free space began or how large to make the persistence partition was not possible from that output. parted was not used further. The broader behavior—libparted-based tools handling the isohybrid format unreliably—is documented on the Debian User mailing list [9].

=== gdisk / sgdisk ===
gdisk and sgdisk are GPT-aware tools and would seem the logical choice for a drive with a hybrid partition table. On the test USB, sgdisk produced:<blockquote>Found invalid GPT and valid MBR; converting MBR to GPT format in memory.

Warning! Secondary partition table overlaps the last partition by 33 blocks!

Could not create partition 3 from 14580 to 8177663

Non-GPT disk; not saving changes. Use -g to override.

Error encountered; not saving changes.</blockquote>The isohybrid layout places the secondary GPT header in a location that overlaps the last partition. sgdisk identifies this as invalid and refuses to proceed. Using -g to override would corrupt the hybrid layout and break the boot. sgdisk can’t be used here.

cp was also tested as an alternative to dd for writing the ISO, on the theory that it might handle the isohybrid structure differently. It produced an identical partition layout. It’s not an alternative—it’s a slower write with the same result.

== The UEFI grub> Prompt ==
On the test hardware, adding a persistence partition and then selecting the UEFI:-prefixed boot entry consistently dropped to a bare grub> prompt. Running ls from that prompt showed the internal drive’s partitions; the USB’s live partition was unreachable from there.

What was observed: the USB’s own GRUB appeared briefly—visible as a rapid flash on screen with a “no such…” error—then failed, after which the firmware handed control to the internal drive’s GRUB, which presented its own grub> prompt.

The UEFI specification states that firmware may remove boot options it considers invalid [7]; the ArchWiki documents firmware removing entries for drives it detects as changed as a known occurrence [8]. After adding the persistence partition, the USB’s EFI entry—pointing to EFI/boot/bootx64.efi—may no longer have been treated as valid by the firmware, causing it to fall through to the next available bootloader on the internal drive. Whether modifying the partition table on a removable device is sufficient to trigger this behavior, and under what firmware conditions, wasn’t reproducible in a controlled way during testing.

Selecting the plain (non-UEFI:) device entry uses the legacy MBR boot path, which loads the USB’s own GRUB directly and has no dependency on the UEFI NVRAM or EFI boot entries. On the test hardware, this path was unaffected by partition table changes.

== Anonymity ==
A Debian Live USB with persistence, used correctly, writes nothing to the host machine’s internal disk. It doesn’t modify the installed OS, create logs on the host filesystem, or leave traces in the usual locations.

It doesn’t provide anonymity. Boot events may be logged by firmware or enterprise management systems. Network activity is visible to routers, ISPs, and network administrators regardless of which OS is running. Mounting the internal disk during a live session makes its contents accessible to the live environment, and any writes to it are permanent. A live USB is a portable, self-contained Debian environment—it isn’t a privacy tool by design.

7. A Note on Carrying Data

The persistence partition is a practical place to keep working data alongside the system state—documents, scripts, configuration files, anything you’d normally keep in ~/Documents or ~/bin. It travels with the drive.

Installed packages persist across sessions but may have hardware-specific dependencies. For general desktop use this isn’t usually an issue, but it’s worth keeping in mind if you’re installing drivers or kernel modules that assume specific hardware.

== References ==
[1] Syslinux Project. “Isohybrid.” Syslinux Wiki. <nowiki>https://wiki.syslinux.org/wiki/index.php?title=Isohybrid</nowiki>

[2] Linux Command Library. “isohybrid(1) man page.” <nowiki>https://linuxcommandlibrary.com/man/isohybrid</nowiki>

[3] Smith, Roderick W. “Hybrid MBRs.” GPT fdisk documentation. <nowiki>https://www.rodsbooks.com/gdisk/hybrid.html</nowiki>

[4] Debian Manpages. “persistence.conf(5).” live-boot-doc, Trixie. <nowiki>https://manpages.debian.org/trixie/live-boot-doc/persistence.conf.5.en.html</nowiki>

[5] Debian Live Team. “Customizing Run-Time Behaviours.” Debian Live Manual. <nowiki>https://live-team.pages.debian.net/live-manual/html/live-manual/customizing-run-time-behaviours.en.html</nowiki>

[6] Debian Manpages. “live-boot(7).” live-boot-doc. <nowiki>https://manpages.debian.org/unstable/live-boot-doc/live-boot.7.en.html</nowiki>

[7] UEFI Forum. “Boot Manager.” UEFI Specification 2.10, §3. <nowiki>https://uefi.org/specs/UEFI/2.10/03_Boot_Manager.html</nowiki>

[8] ArchWiki. “Unified Extensible Firmware Interface.” <nowiki>https://wiki.archlinux.org/title/Unified_Extensible_Firmware_Interface</nowiki>

[9] Debian User Mailing List. “Error message from GParted, means what?” November 2016. <nowiki>https://lists.debian.org/debian-user/2016/11/msg01017.html</nowiki>

''Contributed by distro-nix on Debian User Forums on 2026-04-06.'' Comments, corrections, suggestions or resources are welcome.

Debian Live USB with Persistence

2026-04-08T14:15:42Z

Distro-nix:

All commands are verified on Debian 13.4 (Trixie) / 6.12.74+deb13+1-amd64

Created : 2026-04-04

Last Updated : 2026-04-07

ID: 005146.10

A persistent Debian live USB is one of those tools that can come in handy once you have it. Boot it on your own machine, a colleague's laptop, or a borrowed computer. It has your environment, your files, your settings, exactly where you left them. Nothing is written to the host machine, nothing left behind when you pull out the USB.

The persistence layer is what makes all the difference: a standard live USB is stateless—helpful for recovery and installation, but each session starts fresh. By adding a persistence partition, you have a portable Debian installation: your shell configuration, your documents, your utilities, your scripts, your browser profile, all carried around in your pocket.

This is also a practical solution when you're working across multiple machines but don't want to maintain separate installations on each. It's particularly useful for sysadmins who move between systems regularly, or anyone who needs a known-good Debian environment available on demand without depending on the host.

== What you'll need ==
A USB drive of at least 8 GB—the minimum requirement is 4 GB, but that leaves you very little room for the persistence partition. The more you plan to store, the larger the drive should be. A 32 GB drive gives you a nice working space.

Download the latest Debian Live ISO from the official Debian website at <nowiki>https://www.debian.org/CD/live</nowiki>. Choose the release and desktop environment that suits you, or the minimal image if you want to keep it lean.

you'll be working entirely in the terminal on a GNU/Linux system. The tools you need: '''dd, fdisk, and mkfs.ext4'''—are all standard on Debian. On a minimal live environment, fdisk and mkfs.ext4 may not be present. Check first:

<code>$ sudo apt install fdisk e2fsprogs</code>

They are small, standard packages and will be in the live session's apt cache without needing to add sources. Neither persists to the USB unless you're already booted with persistence active, so install them in the same session you're doing the setup.

A word about dd before you start

'''dd''' has the nickname of "disk destroyer" because whatever is in the of= (output file) if you mistype is gone. dd writes directly to block devices with no undo—confirm your device identifier before running the command.

Plug in the USB drive, open the terminal, and run:

<code>$ lsblk</code>

Look for your USB device in the output—it will appear as /dev/sdb, /dev/sdc, or similar depending on what else you have attached.

Cross-reference the size to confirm you have the right device. In the commands that follow, /dev/sdX is a placeholder—replace it with your actual device identifier every time.

'''Use the device itself (e.g. /dev/sdb), not a partition such as /dev/sdb1. '''

Another trap worth knowing: if your shell has a sync alias defined—from an old backup script, for example—the && sync at the end of the dd command will silently call that alias instead of /usr/bin/sync, leaving the write cache unflushed. You'd never know from the output. Before running dd, confirm you're getting the real thing:

<code>$ type sync</code>

It should return sync is /usr/bin/sync. If it returns an alias definition instead, either unalias it with:

<code>$ unalias sync</code>

or call the binary directly with '''/usr/bin/sync''' at the end of your '''dd''' command.

== A note on UEFI ==
I found out the long, hard way that UEFI firmware causes persistent problems with live USB booting, particularly after the partition table on the USB has been modified. The symptom is consistent: the machine drops to a '''grub> prompt on boot, and the internal disk's filesystem is visible while the USB's live partition is''' unreachable. On some firmware implementations, adding a partition to the USB invalidates its EFI boot entry, causing the firmware to fall back to the internal drive's bootloader—even when you explicitly select the USB in the boot menu.

This isn't a problem with the USB, the ISO, or the persistence setup. It is a firmware behavior that seems to affect a wide range of hardware.

During testing, both parted and '''gdisk/sgdisk were''' used. parted misreported the USB capacity. sgdisk detected an invalid GPT with an overlapping secondary partition table and refused to proceed. '''fdisk with an explicit partition start sector is the only tool that works cleanly with the Debian live ISO's isohybrid layout.'''

'''When selecting your USB drive from the firmware boot menu, always choose the plain device entry—not the one prefixed with "UEFI:".'''

'''Most firmware boot menus show two entries for a USB device—one labelled simply with the device name (legacy MBR boot) and one prefixed with "UEFI:".''' Most people choose the UEFI entry because they have modern hardware. But it is not the case here.

When you think of "Legacy boot" in this context, it doesn't mean out of date or inferior. It means a different bootloader handoff mechanism—one that isn't broken by partition table changes. The Debian Live GRUB it loads is identical in any case. You're not giving anything up by choosing it over UEFI.

Select the plain device entry. Persistence works correctly and reliably via this path. The first persistence-enabled boot will be noticeably slower than usual since the system is running entirely from the USB key—and that’s normal. A USB 3.0 drive was used in the physical testing for this paper.

If your machine has no legacy boot option and UEFI is the only available path, the problem can potentially be resolved from within a booted live session by correcting the GRUB prefix—but this is an advanced recovery procedure, not the standard setup path.

== What went wrong, and why ==
I wish this was a straightforward tutorial, but it didn't go as smoothly as I'd hoped. The pitfalls and troubleshooting can serve as useful reference. This section documents the full troubleshooting process that led to finding a working procedure. If you've already hit '''grub> and given up, welcome to the club. If you're wondering why the obvious approaches don't work, read this first—it might save you several hours.'''

=== '''The symptom many seem to run into''' ===
The most common failure mode is this: you write the ISO, verify it boots, add the persistence partition, reboot—and instead of the Debian live menu you get a bare grub> prompt. Nothing you type seems to help. Running ls shows your internal disk's filesystem, but the USB's live partition is nowhere to be found.

This is the point where you conclude that persistence is broken, that the USB stick is faulty, or that the procedure is wrong. That's not where the problem lies.

=== '''The host OS red herring''' ===
The first machine used for testing had Ubuntu Server 24.04 installed on the internal SSD. Ubuntu's GRUB had registered itself as the primary EFI bootloader in the UEFI NVRAM. When the USB was selected in the boot menu, the firmware handed control to Ubuntu's GRUB rather than the USB's own EFI bootloader—even when the USB was explicitly chosen.

From the '''grub>''' prompt, running :

'''grub> ls hd0,gpt2/''' contained Ubuntu's /boot partition with vmlinuz-6.8.0-106-generic, and '''hd0,gpt3''' contained Ubuntu's root filesystem. The USB's live partition was accessible on '''hd2''', but Ubuntu's GRUB could not read the iso9660 filesystem on it—the module reported "unknown filesystem" on every attempt, even after loading iso9660.mod. The live kernel and initrd were completely unreachable from that prompt.

The theory was that installing Debian 13 on the test machine would fix the problem. It resolved the host OS interference, but not the '''grub>''' problem. The only thing "persistent" about this setup was that after adding the persistence partition it persistently went back to '''grub>''' showing that the Ubuntu machine was a red herring obscuring the real problem.

=== The partitioning tool failures ===
The Debian live ISO uses an isohybrid layout—a hybrid partition table that is valid as both an MBR and a GPT. This is why a single ISO can boot on both legacy BIOS and UEFI systems. It's also what makes adding a persistence partition not so straightforward.

=== Three tools I tried before getting something to work ===
'''parted''' was the first try. It misreported the USB capacity entirely by reporting my 32GB thumbdrive reported 125GB, making it impossible to determine where the free space began or how large the persistence partition would be. Next...

'''gdisk/sgdisk''' was the next candidate, specifically because it is ''GPT-aware'' and should in theory handle a hybrid partition table better than fdisk. The output told a different story:

<blockquote> ''Found invalid GPT and valid MBR; converting MBR to GPT format in memory.''

''Warning! Secondary partition table overlaps the last partition by 33 blocks!''

''Could not create partition 3 from 14580 to 8177663''

''Non-GPT disk; not saving changes. Use -g to override.''

''Error encountered; not saving changes.''

</blockquote> The isohybrid layout places the secondary GPT partition table in a location that overlaps the last partition. sgdisk correctly identifies this as invalid and refuses to go any further. Using -g to override would corrupt the hybrid layout and break the boot. sgdisk cannot be used here.

'''cp''' was a last resort alternative to '''dd''' for writing the ISO, based on the theory that it might handle the isohybrid structure differently. It produced an identical partition layout to dd. cp is not a solution—it's just a slower version of the same write operation.

fdisk with its default n command also caused problems on earlier attempts—it prompted to remove an existing ext4 signature from a previous attempt and placed the partition in a way that triggered the UEFI firmware issue. The fix was to specify the start sector explicitly as 8176000, the sector immediately following the end of the ISO partition. This places the persistence partition cleanly in the unallocated space without touching any part of the isohybrid structure.

=== Why the grub> prompt had me stumped ===
After adding the persistence partition with fdisk and rebooting, the machine consistently dropped to '''grub>'''. The assumption at this point was that the USB's own GRUB was loading but failing to find its configuration file—which is what the prompt suggests.

In reality, the USB's GRUB was loading briefly (visible as three rapid flashes on screen with a "no such..." error), then failing, and the firmware was falling back to the internal Debian installation's GRUB, which then presented its own grub> prompt. The ls output confirmed this—the internal drive's EFI partition was visible as hd0,gpt1, not the USB.

The USB's GRUB fails after partition table modification because many UEFI firmware implementations invalidate or reorder EFI boot entries when they detect a change to the partition table on a device. The USB's EFI entry—pointing to EFI/boot/bootx64.efi was effectively orphaned. The firmware no longer saw it as a valid boot option and fell through to the next entry, which was the internal drive. At least that’s my theory.

== The solution: legacy boot ==
The boot menu on the test machine showed two entries for the USB:<blockquote>Choose this → '''''SanDisk Cruzer Glide (29812MB)'''''

''Not this → UEFI: SanDisk Cruzer Glide (29812MB)''</blockquote>The UEFI entry is the one that breaks after partition table changes. The plain entry uses the legacy MBR boot path, which is not affected by EFI boot entry invalidation. Selecting the legacy entry boots the USB's own GRUB directly via the MBR, which has zero dependency on the UEFI NVRAM and isn't affected by partition table changes.

Persistence works correctly and reliably via this path. The procedure itself—fdisk with an explicit sector, mkfs.ext4, persistence.conf—is valid. The only required change from most published guides is which boot entry you select in the firmware menu.

== Clean the drive ==
If the USB has been used before, old partition signatures or bootloader data in the early sectors can cause problems—stale EFI entries or a leftover GRUB installation can interfere with the boot process even after a fresh ISO write. Clear them out first:

<code>$ sudo wipefs -a /dev/sdX </code>

<code>$ sudo dd if=/dev/zero of=/dev/sdX bs=1M count=200</code>

'''wipefs''' removes filesystem and partition table signatures. The '''dd''' zeros the first 200 MB, which covers the MBR, GPT header, and any EFI data that might otherwise survive a simple reformat. Then proceed with the ISO write.

== Write the ISO to the USB drive ==
<code>$ sudo dd if=/path/to/debian-live-13.4.0-amd64-kde.iso of=/dev/sdX bs=4M status=progress && sync</code>

The status=progress flag gives you a running transfer rate, otherwise you can't see it is working. The sync at the end flushes any remaining data from the write cache—don't pull the drive before it completes.

This may take a few minutes depending on the speed of your drive, so be patient and don't be tempted to abort the process even if it appears to stall. When it finishes, the USB is bootable but stateless. The next steps add the persistence layer.

Before continuing, verify the USB boots. Select the legacy (non-UEFI) entry for the USB in your firmware boot menu. This confirms the ISO write is clean and gives you a known-good baseline before you make any further changes. If it doesn't boot at this stage, stop and repeat Steps 1 and 2—don't proceed to the persistence steps until you have a confirmed working boot.

== Create the persistence partition ==
Finally, I tried '''fdisk'''. The ISO write leaves unallocated space at the end of the drive, so fdisk creates a new partition there for persistence. The Debian live ISO uses an isohybrid layout—a hybrid MBR/GPT partition table—and the partition has to be created with an explicit start sector to avoid corrupting that layout. Other partitioning tools (parted, gdisk) do not handle this correctly.

Launch fdisk:

<code>$ sudo fdisk /dev/sdX</code>

Once inside '''fdisk''', print the current partition layout:

Command (m for help): '''p'''

You'll see two existing partitions—the main ISO partition (type 0 or Empty, which is normal for isohybrid) and a small 3.2 MB EFI partition.

The persistence partition must start at the next sector after the last partition ends—on a Debian 13.4 live ISO this is sector 8176000, immediately after the ISO partition which ends at 8175999.

Create the new partition:

Command (m for help): '''n'''

When prompted for the first sector, enter '''8176000''' rather than accepting the default.

Accept the default for the last sector to use the remaining space.

If '''fdisk''' asks whether to remove an existing ext4 signature, answer '''Y'''—this is a remnant from a previous attempt and should be cleared.

Write the changes and exit:

Command (m for help): '''w'''

If the system does not immediately recognize the new partition, run:

<code>$ sudo partprobe</code>

== Format the persistence partition ==
Format the new partition as ext4 with the label '''persistence—the label is not''' a suggestion, the Debian live system looks for it by name at boot:

<code>$ sudo mkfs.ext4 -L persistence /dev/sdX3</code>

Replace sdX3 with your actual partition identifier (for example, /dev/sdb3).

== Create the persistence configuration file ==
Mount the partition:

<code>$ sudo mount /dev/sdX3 /mnt</code>

Create the configuration file that tells the live system what to persist:

<code>$ echo "/ union" | sudo tee /mnt/persistence.conf</code>

The '''/ union''' directive mounts the entire filesystem as a union overlay—changes you make during a session are written to the persistence partition rather than lost when you shut down.

Unmount the partition cleanly:

<code>$ sudo umount /mnt</code>

== Boot with persistence ==
Boot your system from the USB drive, selecting the legacy '''(non-UEFI)''' entry in the firmware boot menu. At the Debian live GRUB menu, press the Tab key to edit the boot entry. Find the line beginning with linux and add persistence after boot=live components. Press Enter to boot into the live system.

From this point on, any files you create, settings you change, or packages you install will survive across reboots—stored on the persistence partition rather than in the volatile live environment. The first boot with persistence active will be slower than a standard live session since the system is running entirely from the USB.

== Verify persistence is working ==
After booting, check that the persistence partition is mounted:

<code>$ lsblk -f</code>

You should see sdX3 mounted at /run/live/persistence/sdX3.

Then create a test file:

<code>$ touch ~/persistence-test.txt</code>

Reboot again with the persistence option enabled. If the file is there, persistence is working.

Anonymity

This is not a stealth tool. When used correctly, a Debian Live USB with persistence doesn't modify the host system—no files or logs are written to the internal disk unless you explicitly mount and use it. However, it does not provide anonymity or invisibility:

* Boot events may be logged by firmware or enterprise systems
* USB usage may be detectable
* Network activity is visible to routers, ISPs, and organisations
* User actions can leave traces if internal drives are mounted

'''A note on carrying data files'''

The persistence partition isn't just for system state. It's a practical place to keep working data—documents, scripts, utilities, configuration files—that you need available no matter which system you boot from. Think of it as your portable home directory. Anything you'd normally keep in /Documents or /bin travels with the drive.

If you're working across machines with different hardware, keep in mind that installed packages persist but may have hardware-specific dependencies. For most desktop use this isn't an issue, but it's worth being aware of if you're installing drivers or kernel modules.

''This work was contributed by distro-nix on Debian User Forums on 2026-04-06.'' Comments, suggestions or resources are welcome.

Debian Live USB with Persistence

2026-04-08T14:04:03Z

Distro-nix:

All commands are verified on Debian 13.4 (Trixie) / 6.12.74+deb13+1-amd64

Created : 2026-04-04

Last Updated : 2026-04-07

ID: 005146.10

<blockquote>
=== A persistent Debian live USB is one of those tools that can come in handy once you have it. Boot it on your own machine, a colleague's laptop, or a borrowed computer. It has your environment, your files, your settings, exactly where you left them. Nothing is written to the host machine, nothing left behind when you pull out the USB. ===
</blockquote>The persistence layer is what makes all the difference: a standard live USB is stateless—helpful for recovery and installation, but each session starts fresh. By adding a persistence partition, you have a portable Debian installation: your shell configuration, your documents, your utilities, your scripts, your browser profile, all carried around in your pocket.

This is also a practical solution when you're working across multiple machines but don't want to maintain separate installations on each. It's particularly useful for sysadmins who move between systems regularly, or anyone who needs a known-good Debian environment available on demand without depending on the host.

== What you'll need ==
A USB drive of at least 8 GB—the minimum requirement is 4 GB, but that leaves you very little room for the persistence partition. The more you plan to store, the larger the drive should be. A 32 GB drive gives you a nice working space.

Download the latest Debian Live ISO from the official Debian website at <nowiki>https://www.debian.org/CD/live</nowiki>. Choose the release and desktop environment that suits you, or the minimal image if you want to keep it lean.

you'll be working entirely in the terminal on a GNU/Linux system. The tools you need: '''dd, fdisk, and mkfs.ext4'''—are all standard on Debian. On a minimal live environment, fdisk and mkfs.ext4 may not be present. Check first:

<code>$ sudo apt install fdisk e2fsprogs</code>

They are small, standard packages and will be in the live session's apt cache without needing to add sources. Neither persists to the USB unless you're already booted with persistence active, so install them in the same session you're doing the setup.

A word about dd before you start

'''dd''' has the nickname of "disk destroyer" because whatever is in the of= (output file) if you mistype is gone. dd writes directly to block devices with no undo—confirm your device identifier before running the command.

Plug in the USB drive, open the terminal, and run:

<code>$ lsblk</code>

Look for your USB device in the output—it will appear as /dev/sdb, /dev/sdc, or similar depending on what else you have attached.

Cross-reference the size to confirm you have the right device. In the commands that follow, /dev/sdX is a placeholder—replace it with your actual device identifier every time.

'''Use the device itself (e.g. /dev/sdb), not a partition such as /dev/sdb1. '''

Another trap worth knowing: if your shell has a sync alias defined—from an old backup script, for example—the && sync at the end of the dd command will silently call that alias instead of /usr/bin/sync, leaving the write cache unflushed. You'd never know from the output. Before running dd, confirm you're getting the real thing:

<code>$ type sync</code>

It should return sync is /usr/bin/sync. If it returns an alias definition instead, either unalias it with:

<code>$ unalias sync</code>

or call the binary directly with '''/usr/bin/sync''' at the end of your '''dd''' command.

== A note on UEFI ==
I found out the long, hard way that UEFI firmware causes persistent problems with live USB booting, particularly after the partition table on the USB has been modified. The symptom is consistent: the machine drops to a '''grub> prompt on boot, and the internal disk's filesystem is visible while the USB's live partition is''' unreachable. On some firmware implementations, adding a partition to the USB invalidates its EFI boot entry, causing the firmware to fall back to the internal drive's bootloader—even when you explicitly select the USB in the boot menu.

This isn't a problem with the USB, the ISO, or the persistence setup. It is a firmware behavior that seems to affect a wide range of hardware.

During testing, both parted and '''gdisk/sgdisk were''' used. parted misreported the USB capacity. sgdisk detected an invalid GPT with an overlapping secondary partition table and refused to proceed. '''fdisk with an explicit partition start sector is the only tool that works cleanly with the Debian live ISO's isohybrid layout.'''

'''When selecting your USB drive from the firmware boot menu, always choose the plain device entry—not the one prefixed with "UEFI:".'''

'''Most firmware boot menus show two entries for a USB device—one labelled simply with the device name (legacy MBR boot) and one prefixed with "UEFI:".''' Most people choose the UEFI entry because they have modern hardware. But it is not the case here.

When you think of "Legacy boot" in this context, it doesn't mean out of date or inferior. It means a different bootloader handoff mechanism—one that isn't broken by partition table changes. The Debian Live GRUB it loads is identical in any case. You're not giving anything up by choosing it over UEFI.

Select the plain device entry. Persistence works correctly and reliably via this path. The first persistence-enabled boot will be noticeably slower than usual since the system is running entirely from the USB key—and that’s normal. A USB 3.0 drive was used in the physical testing for this paper.

If your machine has no legacy boot option and UEFI is the only available path, the problem can potentially be resolved from within a booted live session by correcting the GRUB prefix—but this is an advanced recovery procedure, not the standard setup path.

== What went wrong, and why ==
I wish this was a straightforward tutorial, but it didn't go as smoothly as I'd hoped. The pitfalls and troubleshooting can serve as useful reference. This section documents the full troubleshooting process that led to finding a working procedure. If you've already hit '''grub> and given up, welcome to the club. If you're wondering why the obvious approaches don't work, read this first—it might save you several hours.'''

=== '''The symptom many seem to run into''' ===
The most common failure mode is this: you write the ISO, verify it boots, add the persistence partition, reboot—and instead of the Debian live menu you get a bare grub> prompt. Nothing you type seems to help. Running ls shows your internal disk's filesystem, but the USB's live partition is nowhere to be found.

This is the point where you conclude that persistence is broken, that the USB stick is faulty, or that the procedure is wrong. That's not where the problem lies.

=== '''The host OS red herring''' ===
The first machine used for testing had Ubuntu Server 24.04 installed on the internal SSD. Ubuntu's GRUB had registered itself as the primary EFI bootloader in the UEFI NVRAM. When the USB was selected in the boot menu, the firmware handed control to Ubuntu's GRUB rather than the USB's own EFI bootloader—even when the USB was explicitly chosen.

From the '''grub>''' prompt, running :

'''grub> ls hd0,gpt2/''' contained Ubuntu's /boot partition with vmlinuz-6.8.0-106-generic, and '''hd0,gpt3''' contained Ubuntu's root filesystem. The USB's live partition was accessible on '''hd2''', but Ubuntu's GRUB could not read the iso9660 filesystem on it—the module reported "unknown filesystem" on every attempt, even after loading iso9660.mod. The live kernel and initrd were completely unreachable from that prompt.

The theory was that installing Debian 13 on the test machine would fix the problem. It resolved the host OS interference, but not the '''grub>''' problem. The only thing "persistent" about this setup was that after adding the persistence partition it persistently went back to '''grub>''' showing that the Ubuntu machine was a red herring obscuring the real problem.

=== The partitioning tool failures ===
The Debian live ISO uses an isohybrid layout—a hybrid partition table that is valid as both an MBR and a GPT. This is why a single ISO can boot on both legacy BIOS and UEFI systems. It's also what makes adding a persistence partition not so straightforward.

=== Three tools I tried before getting something to work ===
parted was the first try. It misreported the USB capacity entirely by reporting my 32GB ad 125GB, making it impossible to determine where the free space began or how large the persistence partition would be. Next...

'''gdisk/sgdisk''' was the next candidate, specifically because it is ''GPT-aware'' and should in theory handle a hybrid partition table better than fdisk. The output told a different story:

<blockquote> ''Found invalid GPT and valid MBR; converting MBR to GPT format in memory.''

''Warning! Secondary partition table overlaps the last partition by 33 blocks!''

''Could not create partition 3 from 14580 to 8177663''

''Non-GPT disk; not saving changes. Use -g to override.''

''Error encountered; not saving changes.''

</blockquote> The isohybrid layout places the secondary GPT partition table in a location that overlaps the last partition. sgdisk correctly identifies this as invalid and refuses to go any further. Using -g to override would corrupt the hybrid layout and break the boot. sgdisk cannot be used here.

'''cp''' was a last resort alternative to '''dd''' for writing the ISO, based on the theory that it might handle the isohybrid structure differently. It produced an identical partition layout to dd. cp is not a solution—it's just a slower version of the same write operation.

fdisk with its default n command also caused problems on earlier attempts—it prompted to remove an existing ext4 signature from a previous attempt and placed the partition in a way that triggered the UEFI firmware issue. The fix was to specify the start sector explicitly as 8176000, the sector immediately following the end of the ISO partition. This places the persistence partition cleanly in the unallocated space without touching any part of the isohybrid structure.

=== Why the grub> prompt had me stumped ===
After adding the persistence partition with fdisk and rebooting, the machine consistently dropped to grub>. The assumption at this point was that the USB's own GRUB was loading but failing to find its configuration file—which is what the prompt suggests.

In reality, the USB's GRUB was loading briefly (visible as three rapid flashes on screen with a "no such..." error), then failing, and the firmware was falling back to the internal Debian installation's GRUB, which then presented its own grub> prompt. The ls output confirmed this—the internal drive's EFI partition was visible as hd0,gpt1, not the USB.

The USB's GRUB fails after partition table modification because many UEFI firmware implementations invalidate or reorder EFI boot entries when they detect a change to the partition table on a device. The USB's EFI entry—pointing to EFI/boot/bootx64.efi was effectively orphaned. The firmware no longer saw it as a valid boot option and fell through to the next entry, which was the internal drive. At least that’s my theory.

== The solution: legacy boot ==
The boot menu on the test machine showed two entries for the USB:<blockquote>Choose this → '''''SanDisk Cruzer Glide (29812MB)'''''

''Not this → UEFI: SanDisk Cruzer Glide (29812MB)''</blockquote>The UEFI entry is the one that breaks after partition table changes. The plain entry uses the legacy MBR boot path, which is not affected by EFI boot entry invalidation. Selecting the legacy entry boots the USB's own GRUB directly via the MBR, which has zero dependency on the UEFI NVRAM and isn't affected by partition table changes.

Persistence works correctly and reliably via this path. The procedure itself—fdisk with an explicit sector, mkfs.ext4, persistence.conf—is valid. The only required change from most published guides is which boot entry you select in the firmware menu.

== Clean the drive ==
If the USB has been used before, old partition signatures or bootloader data in the early sectors can cause problems—stale EFI entries or a leftover GRUB installation can interfere with the boot process even after a fresh ISO write. Clear them out first:

<code>$ sudo wipefs -a /dev/sdX </code>

<code>$ sudo dd if=/dev/zero of=/dev/sdX bs=1M count=200</code>

'''wipefs''' removes filesystem and partition table signatures. The '''dd''' zeros the first 200 MB, which covers the MBR, GPT header, and any EFI data that might otherwise survive a simple reformat. Then proceed with the ISO write.

== Write the ISO to the USB drive ==
<code>$ sudo dd if=/path/to/debian-live-13.4.0-amd64-kde.iso of=/dev/sdX bs=4M status=progress && sync</code>

The status=progress flag gives you a running transfer rate, otherwise you can't see it is working. The sync at the end flushes any remaining data from the write cache—don't pull the drive before it completes.

This may take a few minutes depending on the speed of your drive, so be patient and don't be tempted to abort the process even if it appears to stall. When it finishes, the USB is bootable but stateless. The next steps add the persistence layer.

Before continuing, verify the USB boots. Select the legacy (non-UEFI) entry for the USB in your firmware boot menu. This confirms the ISO write is clean and gives you a known-good baseline before you make any further changes. If it doesn't boot at this stage, stop and repeat Steps 1 and 2—don't proceed to the persistence steps until you have a confirmed working boot.

== Create the persistence partition ==
The ISO write leaves unallocated space at the end of the drive. You'll use fdisk to create a new partition there for persistence. The Debian live ISO uses an isohybrid layout—a hybrid MBR/GPT partition table—and the partition must be created with an explicit start sector to avoid corrupting that layout. Other partitioning tools (parted, gdisk) do not handle this correctly.

Launch fdisk:

<code>$ sudo fdisk /dev/sdX</code>

Once inside '''fdisk''', print the current partition layout:

Command (m for help): '''p'''

You'll see two existing partitions—the main ISO partition (type 0 or Empty, which is normal for isohybrid) and a small 3.2 MB EFI partition.

The persistence partition must start at the next sector after the last partition ends—on a Debian 13.4 live ISO this is sector 8176000, immediately after the ISO partition which ends at 8175999.

Create the new partition:

Command (m for help): '''n'''

When prompted for the first sector, enter '''8176000''' rather than accepting the default.

Accept the default for the last sector to use the remaining space.

If '''fdisk''' asks whether to remove an existing ext4 signature, answer '''Y'''—this is a remnant from a previous attempt and should be cleared.

Write the changes and exit:

Command (m for help): '''w'''

If the system does not immediately recognize the new partition, run:

<code>$ sudo partprobe</code>

== Format the persistence partition ==
Format the new partition as ext4 with the label '''persistence—the label is not''' a suggestion, the Debian live system looks for it by name at boot:

<code>$ sudo mkfs.ext4 -L persistence /dev/sdX3</code>

Replace sdX3 with your actual partition identifier (for example, /dev/sdb3).

== Create the persistence configuration file ==
Mount the partition:

<code>$ sudo mount /dev/sdX3 /mnt</code>

Create the configuration file that tells the live system what to persist:

<code>$ echo "/ union" | sudo tee /mnt/persistence.conf</code>

The '''/ union''' directive mounts the entire filesystem as a union overlay—changes you make during a session are written to the persistence partition rather than lost when you shut down.

Unmount the partition cleanly:

<code>$ sudo umount /mnt</code>

== Boot with persistence ==
Boot your system from the USB drive, selecting the legacy '''(non-UEFI)''' entry in the firmware boot menu. At the Debian live GRUB menu, press the Tab key to edit the boot entry. Find the line beginning with linux and add persistence after boot=live components. Press Enter to boot into the live system.

From this point on, any files you create, settings you change, or packages you install will survive across reboots—stored on the persistence partition rather than in the volatile live environment. The first boot with persistence active will be slower than a standard live session since the system is running entirely from the USB.

== Verify persistence is working ==
After booting, check that the persistence partition is mounted:

<code>$ lsblk -f</code>

You should see sdX3 mounted at /run/live/persistence/sdX3.

Then create a test file:

<code>$ touch ~/persistence-test.txt</code>

Reboot again with the persistence option enabled. If the file is there, persistence is working.

Anonymity

This is not a stealth tool. When used correctly, a Debian Live USB with persistence doesn't modify the host system—no files or logs are written to the internal disk unless you explicitly mount and use it. However, it does not provide anonymity or invisibility:

* Boot events may be logged by firmware or enterprise systems
* USB usage may be detectable
* Network activity is visible to routers, ISPs, and organisations
* User actions can leave traces if internal drives are mounted

'''A note on carrying data files'''

The persistence partition isn't just for system state. It's a practical place to keep working data—documents, scripts, utilities, configuration files—that you need available no matter which system you boot from. Think of it as your portable home directory. Anything you'd normally keep in /Documents or /bin travels with the drive.

If you're working across machines with different hardware, keep in mind that installed packages persist but may have hardware-specific dependencies. For most desktop use this isn't an issue, but it's worth being aware of if you're installing drivers or kernel modules.

''This work was contributed by distro-nix on Debian User Forums on 2026-04-06.'' Comments, suggestions or resources are welcome.

Debian Live USB with Persistence

2026-04-08T02:42:51Z

Distro-nix: removed step numbers

All commands are verified on Debian 13.4 (Trixie) / 6.12.74+deb13+1-amd64

Created : 2026-04-04

Last Updated : 2026-04-07

ID: 005146.10

=== A persistent Debian live USB is one of those tools that can come in handy once you have it. Boot it on your own machine, a colleague's laptop, or a borrowed computer. It has your environment, your files, your settings, exactly where you left them. Nothing is written to the host machine, nothing left behind when you pull out the USB. ===

The persistence layer is what makes all the difference: a standard live USB is stateless—helpful for recovery and installation, but each session starts fresh. By adding a persistence partition, you have a portable Debian installation: your shell configuration, your documents, your utilities, your scripts, your browser profile, all carried around in your pocket.

This is also a practical solution when you're working across multiple machines but don't want to maintain separate installations on each. It's particularly useful for sysadmins who move between systems regularly, or anyone who needs a known-good Debian environment available on demand without depending on the host.

== What you'll need ==
A USB drive of at least 8 GB—the minimum requirement is 4 GB, but that leaves you very little room for the persistence partition. The more you plan to store, the larger the drive should be. A 32 GB drive gives you a nice working space.

Download the latest Debian Live ISO from the official Debian website at <nowiki>https://www.debian.org/CD/live</nowiki>. Choose the release and desktop environment that suits you, or the minimal image if you want to keep it lean.

you'll be working entirely in the terminal on a GNU/Linux system. The tools you need: '''dd, fdisk, and mkfs.ext4'''—are all standard on Debian. On a minimal live environment, fdisk and mkfs.ext4 may not be present. Check first:

<code>$ sudo apt install fdisk e2fsprogs</code>

They are small, standard packages and will be in the live session's apt cache without needing to add sources. Neither persists to the USB unless you're already booted with persistence active, so install them in the same session you're doing the setup.

A word about dd before you start

'''dd''' has the nickname of "disk destroyer" because whatever is in the of= (output file) if you mistype is gone. dd writes directly to block devices with no undo—confirm your device identifier before running the command.

Plug in the USB drive, open the terminal, and run:

<code>$ lsblk</code>

Look for your USB device in the output—it will appear as /dev/sdb, /dev/sdc, or similar depending on what else you have attached.

Cross-reference the size to confirm you have the right device. In the commands that follow, /dev/sdX is a placeholder—replace it with your actual device identifier every time.

'''Use the device itself (e.g. /dev/sdb), not a partition such as /dev/sdb1. '''

Another trap worth knowing: if your shell has a sync alias defined—from an old backup script, for example—the && sync at the end of the dd command will silently call that alias instead of /usr/bin/sync, leaving the write cache unflushed. You'd never know from the output. Before running dd, confirm you're getting the real thing:

<code>$ type sync</code>

It should return sync is /usr/bin/sync. If it returns an alias definition instead, either unalias it with:

<code>$ unalias sync</code>

or call the binary directly with '''/usr/bin/sync''' at the end of your '''dd''' command.

== A note on UEFI ==
I found out the long, hard way that UEFI firmware causes persistent problems with live USB booting, particularly after the partition table on the USB has been modified. The symptom is consistent: the machine drops to a '''grub> prompt on boot, and the internal disk's filesystem is visible while the USB's live partition is''' unreachable. On some firmware implementations, adding a partition to the USB invalidates its EFI boot entry, causing the firmware to fall back to the internal drive's bootloader—even when you explicitly select the USB in the boot menu.

This isn't a problem with the USB, the ISO, or the persistence setup. It is a firmware behavior that seems to affect a wide range of hardware.

During testing, both parted and '''gdisk/sgdisk were''' used. parted misreported the USB capacity. sgdisk detected an invalid GPT with an overlapping secondary partition table and refused to proceed. '''fdisk with an explicit partition start sector is the only tool that works cleanly with the Debian live ISO's isohybrid layout.'''

'''When selecting your USB drive from the firmware boot menu, always choose the plain device entry—not the one prefixed with "UEFI:".'''

'''Most firmware boot menus show two entries for a USB device—one labelled simply with the device name (legacy MBR boot) and one prefixed with "UEFI:".''' Most people choose the UEFI entry because they have modern hardware. But it is not the case here.

When you think of "Legacy boot" in this context, it doesn't mean out of date or inferior. It means a different bootloader handoff mechanism—one that isn't broken by partition table changes. The Debian Live GRUB it loads is identical in any case. You're not giving anything up by choosing it over UEFI.

Select the plain device entry. Persistence works correctly and reliably via this path. The first persistence-enabled boot will be noticeably slower than usual since the system is running entirely from the USB key—and that’s normal. A USB 3.0 drive was used in the physical testing for this paper.

If your machine has no legacy boot option and UEFI is the only available path, the problem can potentially be resolved from within a booted live session by correcting the GRUB prefix—but this is an advanced recovery procedure, not the standard setup path.

== What went wrong, and why ==
I wish this was a straightforward tutorial, but it didn't go as smoothly as I'd hoped. The pitfalls and troubleshooting can serve as useful reference. This section documents the full troubleshooting process that led to finding a working procedure. If you've already hit '''grub> and given up, welcome to the club. If you're wondering why the obvious approaches don't work, read this first—it might save you several hours.'''

=== '''The symptom many seem to run into''' ===
The most common failure mode is this: you write the ISO, verify it boots, add the persistence partition, reboot—and instead of the Debian live menu you get a bare grub> prompt. Nothing you type seems to help. Running ls shows your internal disk's filesystem, but the USB's live partition is nowhere to be found.

This is the point where you conclude that persistence is broken, that the USB stick is faulty, or that the procedure is wrong. That's not where the problem lies.

=== '''The host OS red herring''' ===
The first machine used for testing had Ubuntu Server 24.04 installed on the internal SSD. Ubuntu's GRUB had registered itself as the primary EFI bootloader in the UEFI NVRAM. When the USB was selected in the boot menu, the firmware handed control to Ubuntu's GRUB rather than the USB's own EFI bootloader—even when the USB was explicitly chosen.

From the '''grub>''' prompt, running :

'''grub> ls hd0,gpt2/''' contained Ubuntu's /boot partition with vmlinuz-6.8.0-106-generic, and '''hd0,gpt3''' contained Ubuntu's root filesystem. The USB's live partition was accessible on '''hd2''', but Ubuntu's GRUB could not read the iso9660 filesystem on it—the module reported "unknown filesystem" on every attempt, even after loading iso9660.mod. The live kernel and initrd were completely unreachable from that prompt.

The theory was that installing Debian 13 on the test machine would fix the problem. It resolved the host OS interference, but not the '''grub>''' problem. The only thing "persistent" about this setup was that after adding the persistence partition it persistently went back to '''grub>''' showing that the Ubuntu machine was a red herring obscuring the real problem.

=== The partitioning tool failures ===
The Debian live ISO uses an isohybrid layout—a hybrid partition table that is valid as both an MBR and a GPT. This is why a single ISO can boot on both legacy BIOS and UEFI systems. It's also what makes adding a persistence partition not so straightforward.

=== Three tools I tried before getting something to work ===
parted was the first try. It misreported the USB capacity entirely by reporting my 32GB ad 125GB, making it impossible to determine where the free space began or how large the persistence partition would be. Next...

'''gdisk/sgdisk''' was the next candidate, specifically because it is ''GPT-aware'' and should in theory handle a hybrid partition table better than fdisk. The output told a different story:

<blockquote> ''Found invalid GPT and valid MBR; converting MBR to GPT format in memory.''

''Warning! Secondary partition table overlaps the last partition by 33 blocks!''

''Could not create partition 3 from 14580 to 8177663''

''Non-GPT disk; not saving changes. Use -g to override.''

''Error encountered; not saving changes.''

</blockquote> The isohybrid layout places the secondary GPT partition table in a location that overlaps the last partition. sgdisk correctly identifies this as invalid and refuses to go any further. Using -g to override would corrupt the hybrid layout and break the boot. sgdisk cannot be used here.

'''cp''' was a last resort alternative to '''dd''' for writing the ISO, based on the theory that it might handle the isohybrid structure differently. It produced an identical partition layout to dd. cp is not a solution—it's just a slower version of the same write operation.

fdisk with its default n command also caused problems on earlier attempts—it prompted to remove an existing ext4 signature from a previous attempt and placed the partition in a way that triggered the UEFI firmware issue. The fix was to specify the start sector explicitly as 8176000, the sector immediately following the end of the ISO partition. This places the persistence partition cleanly in the unallocated space without touching any part of the isohybrid structure.

=== Why the grub> prompt had me stumped ===
After adding the persistence partition with fdisk and rebooting, the machine consistently dropped to grub>. The assumption at this point was that the USB's own GRUB was loading but failing to find its configuration file—which is what the prompt suggests.

In reality, the USB's GRUB was loading briefly (visible as three rapid flashes on screen with a "no such..." error), then failing, and the firmware was falling back to the internal Debian installation's GRUB, which then presented its own grub> prompt. The ls output confirmed this—the internal drive's EFI partition was visible as hd0,gpt1, not the USB.

The USB's GRUB fails after partition table modification because many UEFI firmware implementations invalidate or reorder EFI boot entries when they detect a change to the partition table on a device. The USB's EFI entry—pointing to EFI/boot/bootx64.efi was effectively orphaned. The firmware no longer saw it as a valid boot option and fell through to the next entry, which was the internal drive. At least that’s my theory.

== The solution: legacy boot ==
The boot menu on the test machine showed two entries for the USB:<blockquote>Choose this → '''''SanDisk Cruzer Glide (29812MB)'''''

''Not this → UEFI: SanDisk Cruzer Glide (29812MB)''</blockquote>The UEFI entry is the one that breaks after partition table changes. The plain entry uses the legacy MBR boot path, which is not affected by EFI boot entry invalidation. Selecting the legacy entry boots the USB's own GRUB directly via the MBR, which has zero dependency on the UEFI NVRAM and isn't affected by partition table changes.

Persistence works correctly and reliably via this path. The procedure itself—fdisk with an explicit sector, mkfs.ext4, persistence.conf—is valid. The only required change from most published guides is which boot entry you select in the firmware menu.

== Clean the drive ==
If the USB has been used before, old partition signatures or bootloader data in the early sectors can cause problems—stale EFI entries or a leftover GRUB installation can interfere with the boot process even after a fresh ISO write. Clear them out first:

<code>$ sudo wipefs -a /dev/sdX </code>

<code>$ sudo dd if=/dev/zero of=/dev/sdX bs=1M count=200</code>

'''wipefs''' removes filesystem and partition table signatures. The '''dd''' zeros the first 200 MB, which covers the MBR, GPT header, and any EFI data that might otherwise survive a simple reformat. Then proceed with the ISO write.

== Write the ISO to the USB drive ==
<code>$ sudo dd if=/path/to/debian-live-13.4.0-amd64-kde.iso of=/dev/sdX bs=4M status=progress && sync</code>

The status=progress flag gives you a running transfer rate, otherwise you can't see it is working. The sync at the end flushes any remaining data from the write cache—don't pull the drive before it completes.

This may take a few minutes depending on the speed of your drive, so be patient and don't be tempted to abort the process even if it appears to stall. When it finishes, the USB is bootable but stateless. The next steps add the persistence layer.

Before continuing, verify the USB boots. Select the legacy (non-UEFI) entry for the USB in your firmware boot menu. This confirms the ISO write is clean and gives you a known-good baseline before you make any further changes. If it doesn't boot at this stage, stop and repeat Steps 1 and 2—don't proceed to the persistence steps until you have a confirmed working boot.

== Create the persistence partition ==
The ISO write leaves unallocated space at the end of the drive. You'll use fdisk to create a new partition there for persistence. The Debian live ISO uses an isohybrid layout—a hybrid MBR/GPT partition table—and the partition must be created with an explicit start sector to avoid corrupting that layout. Other partitioning tools (parted, gdisk) do not handle this correctly.

Launch fdisk:

<code>$ sudo fdisk /dev/sdX</code>

Once inside '''fdisk''', print the current partition layout:

Command (m for help): '''p'''

You'll see two existing partitions—the main ISO partition (type 0 or Empty, which is normal for isohybrid) and a small 3.2 MB EFI partition.

The persistence partition must start at the next sector after the last partition ends—on a Debian 13.4 live ISO this is sector 8176000, immediately after the ISO partition which ends at 8175999.

Create the new partition:

Command (m for help): '''n'''

When prompted for the first sector, enter '''8176000''' rather than accepting the default.

Accept the default for the last sector to use the remaining space.

If '''fdisk''' asks whether to remove an existing ext4 signature, answer '''Y'''—this is a remnant from a previous attempt and should be cleared.

Write the changes and exit:

Command (m for help): '''w'''

If the system does not immediately recognize the new partition, run:

<code>$ sudo partprobe</code>

== Format the persistence partition ==
Format the new partition as ext4 with the label '''persistence—the label is not''' a suggestion, the Debian live system looks for it by name at boot:

<code>$ sudo mkfs.ext4 -L persistence /dev/sdX3</code>

Replace sdX3 with your actual partition identifier (for example, /dev/sdb3).

== Create the persistence configuration file ==
Mount the partition:

<code>$ sudo mount /dev/sdX3 /mnt</code>

Create the configuration file that tells the live system what to persist:

<code>$ echo "/ union" | sudo tee /mnt/persistence.conf</code>

The '''/ union''' directive mounts the entire filesystem as a union overlay—changes you make during a session are written to the persistence partition rather than lost when you shut down.

Unmount the partition cleanly:

<code>$ sudo umount /mnt</code>

== Boot with persistence ==
Boot your system from the USB drive, selecting the legacy '''(non-UEFI)''' entry in the firmware boot menu. At the Debian live GRUB menu, press the Tab key to edit the boot entry. Find the line beginning with linux and add persistence after boot=live components. Press Enter to boot into the live system.

From this point on, any files you create, settings you change, or packages you install will survive across reboots—stored on the persistence partition rather than in the volatile live environment. The first boot with persistence active will be slower than a standard live session since the system is running entirely from the USB.

== Verify persistence is working ==
After booting, check that the persistence partition is mounted:

<code>$ lsblk -f</code>

You should see sdX3 mounted at /run/live/persistence/sdX3.

Then create a test file:

<code>$ touch ~/persistence-test.txt</code>

Reboot again with the persistence option enabled. If the file is there, persistence is working.

Anonymity

This is not a stealth tool. When used correctly, a Debian Live USB with persistence doesn't modify the host system—no files or logs are written to the internal disk unless you explicitly mount and use it. However, it does not provide anonymity or invisibility:

* Boot events may be logged by firmware or enterprise systems
* USB usage may be detectable
* Network activity is visible to routers, ISPs, and organisations
* User actions can leave traces if internal drives are mounted

'''A note on carrying data files'''

The persistence partition isn't just for system state. It's a practical place to keep working data—documents, scripts, utilities, configuration files—that you need available no matter which system you boot from. Think of it as your portable home directory. Anything you'd normally keep in /Documents or /bin travels with the drive.

If you're working across machines with different hardware, keep in mind that installed packages persist but may have hardware-specific dependencies. For most desktop use this isn't an issue, but it's worth being aware of if you're installing drivers or kernel modules.

''This work was contributed by distro-nix on Debian User Forums on 2026-04-06.'' Comments, suggestions or resources are welcome.

Debian Live USB with Persistence

2026-04-08T02:40:28Z

Distro-nix: minir typos and headings

All commands are verified on Debian 13.4 (Trixie) / 6.12.74+deb13+1-amd64

Created : 2026-04-04

Last Updated : 2026-04-07

ID: 005146.10

=== A persistent Debian live USB is one of those tools that can come in handy once you have it. Boot it on your own machine, a colleague's laptop, or a borrowed computer. It has your environment, your files, your settings, exactly where you left them. Nothing is written to the host machine, nothing left behind when you pull out the USB. ===

The persistence layer is what makes all the difference: a standard live USB is stateless—helpful for recovery and installation, but each session starts fresh. By adding a persistence partition, you have a portable Debian installation: your shell configuration, your documents, your utilities, your scripts, your browser profile, all carried around in your pocket.

This is also a practical solution when you're working across multiple machines but don't want to maintain separate installations on each. It's particularly useful for sysadmins who move between systems regularly, or anyone who needs a known-good Debian environment available on demand without depending on the host.

== What you'll need ==
A USB drive of at least 8 GB—the minimum requirement is 4 GB, but that leaves you very little room for the persistence partition. The more you plan to store, the larger the drive should be. A 32 GB drive gives you a nice working space.

Download the latest Debian Live ISO from the official Debian website at <nowiki>https://www.debian.org/CD/live</nowiki>. Choose the release and desktop environment that suits you, or the minimal image if you want to keep it lean.

you'll be working entirely in the terminal on a GNU/Linux system. The tools you need: '''dd, fdisk, and mkfs.ext4'''—are all standard on Debian. On a minimal live environment, fdisk and mkfs.ext4 may not be present. Check first:

<code>$ sudo apt install fdisk e2fsprogs</code>

They are small, standard packages and will be in the live session's apt cache without needing to add sources. Neither persists to the USB unless you're already booted with persistence active, so install them in the same session you're doing the setup.

A word about dd before you start

'''dd''' has the nickname of "disk destroyer" because whatever is in the of= (output file) if you mistype is gone. dd writes directly to block devices with no undo—confirm your device identifier before running the command.

Plug in the USB drive, open the terminal, and run:

<code>$ lsblk</code>

Look for your USB device in the output—it will appear as /dev/sdb, /dev/sdc, or similar depending on what else you have attached.

Cross-reference the size to confirm you have the right device. In the commands that follow, /dev/sdX is a placeholder—replace it with your actual device identifier every time.

'''Use the device itself (e.g. /dev/sdb), not a partition such as /dev/sdb1. '''

Another trap worth knowing: if your shell has a sync alias defined—from an old backup script, for example—the && sync at the end of the dd command will silently call that alias instead of /usr/bin/sync, leaving the write cache unflushed. You'd never know from the output. Before running dd, confirm you're getting the real thing:

<code>$ type sync</code>

It should return sync is /usr/bin/sync. If it returns an alias definition instead, either unalias it with:

<code>$ unalias sync</code>

or call the binary directly with '''/usr/bin/sync''' at the end of your '''dd''' command.

== A note on UEFI ==
I found out the long, hard way that UEFI firmware causes persistent problems with live USB booting, particularly after the partition table on the USB has been modified. The symptom is consistent: the machine drops to a '''grub> prompt on boot, and the internal disk's filesystem is visible while the USB's live partition is''' unreachable. On some firmware implementations, adding a partition to the USB invalidates its EFI boot entry, causing the firmware to fall back to the internal drive's bootloader—even when you explicitly select the USB in the boot menu.

This isn't a problem with the USB, the ISO, or the persistence setup. It is a firmware behavior that seems to affect a wide range of hardware.

During testing, both parted and '''gdisk/sgdisk were''' used. parted misreported the USB capacity. sgdisk detected an invalid GPT with an overlapping secondary partition table and refused to proceed. '''fdisk with an explicit partition start sector is the only tool that works cleanly with the Debian live ISO's isohybrid layout.'''

'''When selecting your USB drive from the firmware boot menu, always choose the plain device entry—not the one prefixed with "UEFI:".'''

'''Most firmware boot menus show two entries for a USB device—one labelled simply with the device name (legacy MBR boot) and one prefixed with "UEFI:".''' Most people choose the UEFI entry because they have modern hardware. But it is not the case here.

When you think of "Legacy boot" in this context, it doesn't mean out of date or inferior. It means a different bootloader handoff mechanism—one that isn't broken by partition table changes. The Debian Live GRUB it loads is identical in any case. You're not giving anything up by choosing it over UEFI.

Select the plain device entry. Persistence works correctly and reliably via this path. The first persistence-enabled boot will be noticeably slower than usual since the system is running entirely from the USB key—and that’s normal. A USB 3.0 drive was used in the physical testing for this paper.

If your machine has no legacy boot option and UEFI is the only available path, the problem can potentially be resolved from within a booted live session by correcting the GRUB prefix—but this is an advanced recovery procedure, not the standard setup path.

== What went wrong, and why ==
I wish this was a straightforward tutorial, but it didn't go as smoothly as I'd hoped. The pitfalls and troubleshooting can serve as useful reference. This section documents the full troubleshooting process that led to finding a working procedure. If you've already hit '''grub> and given up, welcome to the club. If you're wondering why the obvious approaches don't work, read this first—it might save you several hours.'''

=== '''The symptom many seem to run into''' ===
The most common failure mode is this: you write the ISO, verify it boots, add the persistence partition, reboot—and instead of the Debian live menu you get a bare grub> prompt. Nothing you type seems to help. Running ls shows your internal disk's filesystem, but the USB's live partition is nowhere to be found.

This is the point where you conclude that persistence is broken, that the USB stick is faulty, or that the procedure is wrong. That's not where the problem lies.

=== '''The host OS red herring''' ===
The first machine used for testing had Ubuntu Server 24.04 installed on the internal SSD. Ubuntu's GRUB had registered itself as the primary EFI bootloader in the UEFI NVRAM. When the USB was selected in the boot menu, the firmware handed control to Ubuntu's GRUB rather than the USB's own EFI bootloader—even when the USB was explicitly chosen.

From the '''grub>''' prompt, running :

'''grub> ls hd0,gpt2/''' contained Ubuntu's /boot partition with vmlinuz-6.8.0-106-generic, and '''hd0,gpt3''' contained Ubuntu's root filesystem. The USB's live partition was accessible on '''hd2''', but Ubuntu's GRUB could not read the iso9660 filesystem on it—the module reported "unknown filesystem" on every attempt, even after loading iso9660.mod. The live kernel and initrd were completely unreachable from that prompt.

The theory was that installing Debian 13 on the test machine would fix the problem. It resolved the host OS interference, but not the '''grub>''' problem. The only thing "persistent" about this setup was that after adding the persistence partition it persistently went back to '''grub>''' showing that the Ubuntu machine was a red herring obscuring the real problem.

=== The partitioning tool failures ===
The Debian live ISO uses an isohybrid layout—a hybrid partition table that is valid as both an MBR and a GPT. This is why a single ISO can boot on both legacy BIOS and UEFI systems. It's also what makes adding a persistence partition not so straightforward.

=== Three tools I tried before getting something to work ===
parted was the first try. It misreported the USB capacity entirely by reporting my 32GB ad 125GB, making it impossible to determine where the free space began or how large the persistence partition would be. Next...

'''gdisk/sgdisk''' was the next candidate, specifically because it is ''GPT-aware'' and should in theory handle a hybrid partition table better than fdisk. The output told a different story:

<blockquote> ''Found invalid GPT and valid MBR; converting MBR to GPT format in memory.''

''Warning! Secondary partition table overlaps the last partition by 33 blocks!''

''Could not create partition 3 from 14580 to 8177663''

''Non-GPT disk; not saving changes. Use -g to override.''

''Error encountered; not saving changes.''

</blockquote> The isohybrid layout places the secondary GPT partition table in a location that overlaps the last partition. sgdisk correctly identifies this as invalid and refuses to go any further. Using -g to override would corrupt the hybrid layout and break the boot. sgdisk cannot be used here.

'''cp''' was a last resort alternative to '''dd''' for writing the ISO, based on the theory that it might handle the isohybrid structure differently. It produced an identical partition layout to dd. cp is not a solution—it's just a slower version of the same write operation.

fdisk with its default n command also caused problems on earlier attempts—it prompted to remove an existing ext4 signature from a previous attempt and placed the partition in a way that triggered the UEFI firmware issue. The fix was to specify the start sector explicitly as 8176000, the sector immediately following the end of the ISO partition. This places the persistence partition cleanly in the unallocated space without touching any part of the isohybrid structure.

=== Why the grub> prompt had me stumped ===
After adding the persistence partition with fdisk and rebooting, the machine consistently dropped to grub>. The assumption at this point was that the USB's own GRUB was loading but failing to find its configuration file—which is what the prompt suggests.

In reality, the USB's GRUB was loading briefly (visible as three rapid flashes on screen with a "no such..." error), then failing, and the firmware was falling back to the internal Debian installation's GRUB, which then presented its own grub> prompt. The ls output confirmed this—the internal drive's EFI partition was visible as hd0,gpt1, not the USB.

The USB's GRUB fails after partition table modification because many UEFI firmware implementations invalidate or reorder EFI boot entries when they detect a change to the partition table on a device. The USB's EFI entry—pointing to EFI/boot/bootx64.efi was effectively orphaned. The firmware no longer saw it as a valid boot option and fell through to the next entry, which was the internal drive. At least that’s my theory.

== The solution: legacy boot ==
The boot menu on the test machine showed two entries for the USB:<blockquote>Choose this → '''''SanDisk Cruzer Glide (29812MB)'''''

''Not this → UEFI: SanDisk Cruzer Glide (29812MB)''</blockquote>The UEFI entry is the one that breaks after partition table changes. The plain entry uses the legacy MBR boot path, which is not affected by EFI boot entry invalidation. Selecting the legacy entry boots the USB's own GRUB directly via the MBR, which has zero dependency on the UEFI NVRAM and isn't affected by partition table changes.

Persistence works correctly and reliably via this path. The procedure itself—fdisk with an explicit sector, mkfs.ext4, persistence.conf—is valid. The only required change from most published guides is which boot entry you select in the firmware menu.

== Step 1—Clean the drive ==
If the USB has been used before, old partition signatures or bootloader data in the early sectors can cause problems—stale EFI entries or a leftover GRUB installation can interfere with the boot process even after a fresh ISO write. Clear them out first:

<code>$ sudo wipefs -a /dev/sdX </code>

<code>$ sudo dd if=/dev/zero of=/dev/sdX bs=1M count=200</code>

'''wipefs''' removes filesystem and partition table signatures. The '''dd''' zeros the first 200 MB, which covers the MBR, GPT header, and any EFI data that might otherwise survive a simple reformat. Then proceed with the ISO write.

== Step 2—Write the ISO to the USB drive ==
<code>$ sudo dd if=/path/to/debian-live-13.4.0-amd64-kde.iso of=/dev/sdX bs=4M status=progress && sync</code>

The status=progress flag gives you a running transfer rate, otherwise you can't see it is working. The sync at the end flushes any remaining data from the write cache—don't pull the drive before it completes.

This may take a few minutes depending on the speed of your drive, so be patient and don't be tempted to abort the process even if it appears to stall. When it finishes, the USB is bootable but stateless. The next steps add the persistence layer.

Before continuing, verify the USB boots. Select the legacy (non-UEFI) entry for the USB in your firmware boot menu. This confirms the ISO write is clean and gives you a known-good baseline before you make any further changes. If it doesn't boot at this stage, stop and repeat Steps 1 and 2—don't proceed to the persistence steps until you have a confirmed working boot.

== Step 3—Create the persistence partition ==
The ISO write leaves unallocated space at the end of the drive. You'll use fdisk to create a new partition there for persistence. The Debian live ISO uses an isohybrid layout—a hybrid MBR/GPT partition table—and the partition must be created with an explicit start sector to avoid corrupting that layout. Other partitioning tools (parted, gdisk) do not handle this correctly.

Launch fdisk:

<code>$ sudo fdisk /dev/sdX</code>

Once inside '''fdisk''', print the current partition layout:

Command (m for help): '''p'''

You'll see two existing partitions—the main ISO partition (type 0 or Empty, which is normal for isohybrid) and a small 3.2 MB EFI partition.

The persistence partition must start at the next sector after the last partition ends—on a Debian 13.4 live ISO this is sector 8176000, immediately after the ISO partition which ends at 8175999.

Create the new partition:

Command (m for help): '''n'''

When prompted for the first sector, enter '''8176000''' rather than accepting the default.

Accept the default for the last sector to use the remaining space.

If '''fdisk''' asks whether to remove an existing ext4 signature, answer '''Y'''—this is a remnant from a previous attempt and should be cleared.

Write the changes and exit:

Command (m for help): '''w'''

If the system does not immediately recognize the new partition, run:

<code>$ sudo partprobe</code>

== Step 4—Format the persistence partition ==
Format the new partition as ext4 with the label '''persistence—the label is not''' a suggestion, the Debian live system looks for it by name at boot:

<code>$ sudo mkfs.ext4 -L persistence /dev/sdX3</code>

Replace sdX3 with your actual partition identifier (for example, /dev/sdb3).

== Step 5—Create the persistence configuration file ==
Mount the partition:

<code>$ sudo mount /dev/sdX3 /mnt</code>

Create the configuration file that tells the live system what to persist:

<code>$ echo "/ union" | sudo tee /mnt/persistence.conf</code>

The '''/ union''' directive mounts the entire filesystem as a union overlay—changes you make during a session are written to the persistence partition rather than lost when you shut down.

Unmount the partition cleanly:

<code>$ sudo umount /mnt</code>

== Step 6—Boot with persistence ==
Boot your system from the USB drive, selecting the legacy '''(non-UEFI)''' entry in the firmware boot menu. At the Debian live GRUB menu, press the Tab key to edit the boot entry. Find the line beginning with linux and add persistence after boot=live components. Press Enter to boot into the live system.

From this point on, any files you create, settings you change, or packages you install will survive across reboots—stored on the persistence partition rather than in the volatile live environment. The first boot with persistence active will be slower than a standard live session since the system is running entirely from the USB.

== Step 7—Verify persistence is working ==
After booting, check that the persistence partition is mounted:

<code>$ lsblk -f</code>

You should see sdX3 mounted at /run/live/persistence/sdX3.

Then create a test file:

<code>$ touch ~/persistence-test.txt</code>

Reboot again with the persistence option enabled. If the file is there, persistence is working.

Anonymity

This is not a stealth tool. When used correctly, a Debian Live USB with persistence doesn't modify the host system—no files or logs are written to the internal disk unless you explicitly mount and use it. However, it does not provide anonymity or invisibility:

* Boot events may be logged by firmware or enterprise systems
* USB usage may be detectable
* Network activity is visible to routers, ISPs, and organisations
* User actions can leave traces if internal drives are mounted

'''A note on carrying data files'''

The persistence partition isn't just for system state. It's a practical place to keep working data—documents, scripts, utilities, configuration files—that you need available no matter which system you boot from. Think of it as your portable home directory. Anything you'd normally keep in /Documents or /bin travels with the drive.

If you're working across machines with different hardware, keep in mind that installed packages persist but may have hardware-specific dependencies. For most desktop use this isn't an issue, but it's worth being aware of if you're installing drivers or kernel modules.

''This work was contributed by distro-nix on Debian User Forums on 2026-04-06.'' Comments, suggestions or resources are welcome.

Debian Live USB with Persistence

2026-04-08T02:25:01Z

Distro-nix: Rewrite for new tools

All commands are verified on Debian 13.4 (Trixie) / 6.12.74+deb13+1-amd64

Created : 2026-04-04

Last Updated : 2026-04-07

ID: 005146.10

=== A persistent Debian live USB is one of those tools that can come in handy once you have it. Boot it on your own machine, a colleague's laptop, or a borrowed computer. It has your environment, your files, your settings, exactly where you left them. Nothing is written to the host machine, nothing left behind when you pull out the USB. ===

The persistence layer is what makes all the difference: a standard live USB is stateless—helpful for recovery and installation, but each session starts fresh. By adding a persistence partition, you have a portable Debian installation: your shell configuration, your documents, your utilities, your scripts, your browser profile, all carried around in your pocket.

This is also a practical solution when you're working across multiple machines but don't want to maintain separate installations on each. It's particularly useful for sysadmins who move between systems regularly, or anyone who needs a known-good Debian environment available on demand without depending on the host.

What you'll need

A USB drive of at least 8 GB—the minimum requirement is 4 GB, but that leaves you very little room for the persistence partition. The more you plan to store, the larger the drive should be. A 32 GB drive gives you a nice working space.

Download the latest Debian Live ISO from the official Debian website at <nowiki>https://www.debian.org/CD/live</nowiki>. Choose the release and desktop environment that suits you, or the minimal image if you want to keep it lean.

you'll be working entirely in the terminal on a GNU/Linux system. The tools you need: '''dd, fdisk, and mkfs.ext4'''—are all standard on Debian. On a minimal live environment, fdisk and mkfs.ext4 may not be present. Check first:

<code>$ sudo apt install fdisk e2fsprogs</code>

They are small, standard packages and will be in the live session's apt cache without needing to add sources. Neither persists to the USB unless you're already booted with persistence active, so install them in the same session you're doing the setup.

A word about dd before you start

'''dd''' has the nickname of "disk destroyer" because whatever is in the of= (output file) if you mistype is gone. dd writes directly to block devices with no undo—confirm your device identifier before running the command.

Plug in the USB drive, open the terminal, and run:

<code>$ lsblk</code>

Look for your USB device in the output—it will appear as /dev/sdb, /dev/sdc, or similar depending on what else you have attached.

Cross-reference the size to confirm you have the right device. In the commands that follow, /dev/sdX is a placeholder—replace it with your actual device identifier every time.

'''Use the device itself (e.g. /dev/sdb), not a partition such as /dev/sdb1. '''

Another trap worth knowing: if your shell has a sync alias defined—from an old backup script, for example—the && sync at the end of the dd command will silently call that alias instead of /usr/bin/sync, leaving the write cache unflushed. You'd never know from the output. Before running dd, confirm you're getting the real thing:

<code>$ type sync</code>

It should return sync is /usr/bin/sync. If it returns an alias definition instead, either unalias it with:

<code>$ unalias sync</code>

or call the binary directly with '''/usr/bin/sync''' at the end of your '''dd''' command.

== A note on UEFI ==
I found out the long, hard way that UEFI firmware causes persistent problems with live USB booting, particularly after the partition table on the USB has been modified. The symptom is consistent: the machine drops to a '''grub> prompt on boot, and the internal disk's filesystem is visible while the USB's live partition is''' unreachable. On some firmware implementations, adding a partition to the USB invalidates its EFI boot entry, causing the firmware to fall back to the internal drive's bootloader—even when you explicitly select the USB in the boot menu.

This isn't a problem with the USB, the ISO, or the persistence setup. It is a firmware behavior that seems to affect a wide range of hardware.

During testing, both parted and '''gdisk/sgdisk were''' used. parted misreported the USB capacity. sgdisk detected an invalid GPT with an overlapping secondary partition table and refused to proceed. '''fdisk with an explicit partition start sector is the only tool that works cleanly with the Debian live ISO's isohybrid layout.'''

'''When selecting your USB drive from the firmware boot menu, always choose the plain device entry—not the one prefixed with "UEFI:".'''

'''Most firmware boot menus show two entries for a USB device—one labelled simply with the device name (legacy MBR boot) and one prefixed with "UEFI:".''' Most people choose the UEFI entry because they have modern hardware. But it is not the case here.

When you think of "Legacy boot" in this context, it doesn't mean out of date or inferior. It means a different bootloader handoff mechanism—one that isn't broken by partition table changes. The Debian Live GRUB it loads is identical in any case. You're not giving anything up by choosing it over UEFI.

Select the plain device entry. Persistence works correctly and reliably via this path. The first persistence-enabled boot will be noticeably slower than usual since the system is running entirely from the USB key—and that’s normal. A USB 3.0 drive was used in the physical testing for this paper.

If your machine has no legacy boot option and UEFI is the only available path, the problem can potentially be resolved from within a booted live session by correcting the GRUB prefix—but this is an advanced recovery procedure, not the standard setup path.

What went wrong, and why—a diagnostic record

I wish this was a straightforward tutorial, but it didn't go as smoothly as I'd hoped. The pitfalls and troubleshooting can serve as useful reference. This section documents the full troubleshooting process that led to finding a working procedure. If you've already hit '''grub> and given up, welcome to the club. If you're wondering why the obvious approaches don't work, read this first—it might save you several hours.'''

=== '''The symptom everyone hits''' ===
The most common failure mode is this: you write the ISO, verify it boots, add the persistence partition, reboot—and instead of the Debian live menu you get a bare grub> prompt. Nothing you type seems to help. Running ls shows your internal disk's filesystem, but the USB's live partition is nowhere to be found.

This is the point where you conclude that persistence is broken, that the USB stick is faulty, or that the procedure is wrong. That's not where the problem lies.

=== '''The host OS red herring''' ===
The first machine used for testing had Ubuntu Server 24.04 installed on the internal SSD. Ubuntu's GRUB had registered itself as the primary EFI bootloader in the UEFI NVRAM. When the USB was selected in the boot menu, the firmware handed control to Ubuntu's GRUB rather than the USB's own EFI bootloader—even when the USB was explicitly chosen.

From the '''grub>''' prompt, running :

'''grub> ls hd0,gpt2/''' contained Ubuntu's /boot partition with vmlinuz-6.8.0-106-generic, and '''hd0,gpt3''' contained Ubuntu's root filesystem. The USB's live partition was accessible on '''hd2''', but Ubuntu's GRUB could not read the iso9660 filesystem on it—the module reported "unknown filesystem" on every attempt, even after loading iso9660.mod. The live kernel and initrd were completely unreachable from that prompt.

The theory was that installing Debian 13 on the test machine would fix the problem. It resolved the host OS interference, but not the '''grub>''' problem. The only thing "persistent" about this setup was that after adding the persistence partition it persistently went back to '''grub>''' showing that the Ubuntu machine was a red herring obscuring the real problem.

The partitioning tool failures

The Debian live ISO uses an isohybrid layout—a hybrid partition table that is valid as both an MBR and a GPT. This is why a single ISO can boot on both legacy BIOS and UEFI systems. It's also what makes adding a persistence partition not so straightforward.

Three tools were tried before getting something that worked.

parted was the first try. It misreported the USB capacity entirely by reporting my 32GB ad 125GB, making it impossible to determine where the free space began or how large the persistence partition would be. Next...

'''gdisk/sgdisk''' was the next candidate, specifically because it is ''GPT-aware'' and should in theory handle a hybrid partition table better than fdisk. The output told a different story:

<blockquote> ''Found invalid GPT and valid MBR; converting MBR to GPT format in memory.''

''Warning! Secondary partition table overlaps the last partition by 33 blocks!''

''Could not create partition 3 from 14580 to 8177663''

''Non-GPT disk; not saving changes. Use -g to override.''

''Error encountered; not saving changes.''

</blockquote> The isohybrid layout places the secondary GPT partition table in a location that overlaps the last partition. sgdisk correctly identifies this as invalid and refuses to go any further. Using -g to override would corrupt the hybrid layout and break the boot. sgdisk cannot be used here.

'''cp''' was tried as a last resort alternative to '''dd''' for writing the ISO, based on the theory that it might handle the isohybrid structure differently. It produced an identical partition layout to dd. cp is not a solution—it's just a slower version of the same write operation.

fdisk with its default n command also caused problems on earlier attempts—it prompted to remove an existing ext4 signature from a previous attempt and placed the partition in a way that triggered the UEFI firmware issue. The fix was to specify the start sector explicitly as 8176000, the sector immediately following the end of the ISO partition. This places the persistence partition cleanly in the unallocated space without touching any part of the isohybrid structure.

Why the grub> prompt had me stumped

After adding the persistence partition with fdisk and rebooting, the machine consistently dropped to grub>. The assumption at this point was that the USB's own GRUB was loading but failing to find its configuration file—which is what the prompt suggests.

In reality, the USB's GRUB was loading briefly (visible as three rapid flashes on screen with a "no such..." error), then failing, and the firmware was falling back to the internal Debian installation's GRUB, which then presented its own grub> prompt. The ls output confirmed this—the internal drive's EFI partition was visible as hd0,gpt1, not the USB.

The USB's GRUB fails after partition table modification because many UEFI firmware implementations invalidate or reorder EFI boot entries when they detect a change to the partition table on a device. The USB's EFI entry—pointing to EFI/boot/bootx64.efi was effectively orphaned. The firmware no longer saw it as a valid boot option and fell through to the next entry, which was the internal drive. At least that’s my theory.

The solution: legacy boot

The boot menu on the test machine showed two entries for the USB:

Choose this → '''''SanDisk Cruzer Glide (29812MB)'''''

''Not this → UEFI: SanDisk Cruzer Glide (29812MB)''

The UEFI entry is the one that breaks after partition table changes. The plain entry uses the legacy MBR boot path, which is not affected by EFI boot entry invalidation. Selecting the legacy entry boots the USB's own GRUB directly via the MBR, which has zero dependency on the UEFI NVRAM and isn't affected by partition table changes.

Persistence works correctly and reliably via this path. The procedure itself—fdisk with an explicit sector, mkfs.ext4, persistence.conf—is valid. The only required change from most published guides is which boot entry you select in the firmware menu.

== Step 1—Clean the drive ==
If the USB has been used before, old partition signatures or bootloader data in the early sectors can cause problems—stale EFI entries or a leftover GRUB installation can interfere with the boot process even after a fresh ISO write. Clear them out first:

<code>$ sudo wipefs -a /dev/sdX </code>

<code>$ sudo dd if=/dev/zero of=/dev/sdX bs=1M count=200</code>

'''wipefs''' removes filesystem and partition table signatures. The '''dd''' zeros the first 200 MB, which covers the MBR, GPT header, and any EFI data that might otherwise survive a simple reformat. Then proceed with the ISO write.

== Step 2—Write the ISO to the USB drive ==
<code>$ sudo dd if=/path/to/debian-live-13.4.0-amd64-kde.iso of=/dev/sdX bs=4M status=progress && sync</code>

The status=progress flag gives you a running transfer rate, otherwise you can't see it is working. The sync at the end flushes any remaining data from the write cache—don't pull the drive before it completes.

This may take a few minutes depending on the speed of your drive, so be patient and don't be tempted to abort the process even if it appears to stall. When it finishes, the USB is bootable but stateless. The next steps add the persistence layer.

Before continuing, verify the USB boots. Select the legacy (non-UEFI) entry for the USB in your firmware boot menu. This confirms the ISO write is clean and gives you a known-good baseline before you make any further changes. If it doesn't boot at this stage, stop and repeat Steps 1 and 2—don't proceed to the persistence steps until you have a confirmed working boot.

== Step 3—Create the persistence partition ==
The ISO write leaves unallocated space at the end of the drive. You'll use fdisk to create a new partition there for persistence. The Debian live ISO uses an isohybrid layout—a hybrid MBR/GPT partition table—and the partition must be created with an explicit start sector to avoid corrupting that layout. Other partitioning tools (parted, gdisk) do not handle this correctly.

Launch fdisk:

<code>$ sudo fdisk /dev/sdX</code>

Once inside '''fdisk''', print the current partition layout:

Command (m for help): '''p'''

You'll see two existing partitions—the main ISO partition (type 0 or Empty, which is normal for isohybrid) and a small 3.2 MB EFI partition.

The persistence partition must start at the next sector after the last partition ends—on a Debian 13.4 live ISO this is sector 8176000, immediately after the ISO partition which ends at 8175999.

Create the new partition:

Command (m for help): '''n'''

When prompted for the first sector, enter '''8176000''' rather than accepting the default.

Accept the default for the last sector to use the remaining space.

If '''fdisk''' asks whether to remove an existing ext4 signature, answer '''Y'''—this is a remnant from a previous attempt and should be cleared.

Write the changes and exit:

Command (m for help): '''w'''

If the system does not immediately recognize the new partition, run:

<code>$ sudo partprobe</code>

== Step 4—Format the persistence partition ==
Format the new partition as ext4 with the label '''persistence—the label is not''' a suggestion, the Debian live system looks for it by name at boot:

<code>$ sudo mkfs.ext4 -L persistence /dev/sdX3</code>

Replace sdX3 with your actual partition identifier (for example, /dev/sdb3).

== Step 5—Create the persistence configuration file ==
Mount the partition:

<code>$ sudo mount /dev/sdX3 /mnt</code>

Create the configuration file that tells the live system what to persist:

<code>$ echo "/ union" | sudo tee /mnt/persistence.conf</code>

The '''/ union''' directive mounts the entire filesystem as a union overlay—changes you make during a session are written to the persistence partition rather than lost when you shut down.

Unmount the partition cleanly:

<code>$ sudo umount /mnt</code>

== Step 6—Boot with persistence ==
Boot your system from the USB drive, selecting the legacy '''(non-UEFI)''' entry in the firmware boot menu. At the Debian live GRUB menu, press the Tab key to edit the boot entry. Find the line beginning with linux and add persistence after boot=live components. Press Enter to boot into the live system.

From this point on, any files you create, settings you change, or packages you install will survive across reboots—stored on the persistence partition rather than in the volatile live environment. The first boot with persistence active will be slower than a standard live session since the system is running entirely from the USB.

== Step 7—Verify persistence is working ==
After booting, check that the persistence partition is mounted:

<code>$ lsblk -f</code>

You should see sdX3 mounted at /run/live/persistence/sdX3.

Then create a test file:

<code>$ touch ~/persistence-test.txt</code>

Reboot again with the persistence option enabled. If the file is there, persistence is working.

Anonymity

This is not a stealth tool. When used correctly, a Debian Live USB with persistence doesn't modify the host system—no files or logs are written to the internal disk unless you explicitly mount and use it. However, it does not provide anonymity or invisibility:

* Boot events may be logged by firmware or enterprise systems
* USB usage may be detectable
* Network activity is visible to routers, ISPs, and organisations
* User actions can leave traces if internal drives are mounted

'''A note on carrying data files'''

The persistence partition isn't just for system state. It's a practical place to keep working data—documents, scripts, utilities, configuration files—that you need available no matter which system you boot from. Think of it as your portable home directory. Anything you'd normally keep in /Documents or /bin travels with the drive.

If you're working across machines with different hardware, keep in mind that installed packages persist but may have hardware-specific dependencies. For most desktop use this isn't an issue, but it's worth being aware of if you're installing drivers or kernel modules.

''This work was contributed by distro-nix on Debian User Forums on 2026-04-06.'' Comments, suggestions or resources are welcome.

Debian Live USB with Persistence

2026-04-06T22:09:04Z

Distro-nix: changed typo creation date

All commands are verified on Debian 13.4 (Trixie) / 6.12.74+deb13+1-amd64

Created : 2026-04-04

Last Updated : 2026-04-06

ID: 005146.8

''A persistent Debian live USB can come in handy once you have prepared it. Boot it on your own machine, a colleague’s laptop, or a borrowed computer. It has your environment, your files, your settings, exactly where you left them.''

The persistence layer is what makes all the difference: a standard live USB is stateless—helpful for recovery and installation, but each session starts fresh. By adding a persistence partition, you have a portable Debian installation: your shell configuration, your documents, your utilities, your scripts, your browser profile, all carried around in your pocket.

This is also a practical solution when you’re working across multiple machines but don’t want to maintain separate installations on each. It’s particularly useful for sysadmins who move between systems regularly, or anyone who needs a known-good Debian environment available on demand without depending on the host.

== ''What you’ll need'' ==
A USB drive of at least 8 GB—the minimum requirement is 4GB, but that leaves you very little room for the persistence partition. The more you plan to store, the larger the drive should be. A 32 GB drive gives you more comfortable working space.

Download the latest Debian Live ISO from the official Debian website at <nowiki>https://www.debian.org/CD/live</nowiki> Choose the release and desktop environment that suits you, or the standard image if you prefer a lean setup.

Verify the authenticity of the image <nowiki>https://www.debian.org/CD/verify</nowiki>

You’ll be working entirely in the terminal on a GNU/Linux system. The tools you need:

* dd
* parted
* mkfs.ext4

These are standard utilities in Debian systems *.

dd is part of GNU coreutils package

mkfs.ext4 is part of the e2fsprogs package.*

<nowiki>*</nowiki> usually present on full Debian installations, but on a standard live environment it may not be present.
$ sudo apt install e2fsprogs parted
These are small, standard packages available in the Debian repositories. In a live session, they don’t persist unless you have persistence already enabled, so install them in the same session you do the setup.

== A word about dd before you start ==
The dd package has earned the nickname "disk destroyer" because it destroys whatever you specify as the output (of=) without warning if you make a mistake. dd writes directly to block devices with no undo. Double-check your device identifier carefully before running any command.

Plug in the USB drive and open the terminal and run:
$ lsblk
Look for your USB device in the output—it will appear as /dev/sdb, /dev/sdc, or similar depending on what else is attached. Check that you have the right device.

'''Important:''' use the device itself (e.g., /dev/sdb), not a partition such as /dev/sdb1.

In the commands that follow, /dev/sdX is a placeholder—replace it with your actual device identifier every time.

== Step 1—Write the ISO to the USB drive ==
$ sudo dd if=/path/to/debian-live-13.4.0-amd64-kde.iso of=/dev/sdX bs=4M status=progress && sync
The status=progress flag gives you a running transfer rate, otherwise you can’t see it is working. The sync at the end flushes any remaining data from the write cache—don’t pull the drive before it completes.

This will take a few minutes depending on the drive speed. When it finishes, the USB is bootable but stateless. The next steps add the persistence layer.

== Step 2—Create the persistence partition ==
After writing the ISO to the USB drive, only a portion of the disk is used. The remaining space is left unallocated. You’ll create a new partition in that space for ''persistence''.

Make sure the USB device isn’t mounted, then start parted:
$ sudo parted /dev/sdX
Inside, parted display the current layout:
(parted) print free
You should see existing partitions followed by a region labeled '''Free Space'''.

=== Example layout ===
Your output may look similar to this:

Number Start End Size File system Flags

1 1049kB 3500MB 3499MB fat32 boot, esp

2 3500MB 3600MB 100MB

 3600MB 32.1GB ~28GB Free Space

The important part is the '''free space at the end''' of the disk. This is where the persistence partition will go.

=== Create the new partition ===
Instead of calculating start and end points manually, let parted use the available space:
(parted) mkpart primary ext4 100%
This creates a new partition using all remaining free space.

Set the persistence flag (replace N with the partition number shown in print):

(parted) set N persistence on
(parted) quit
Note: The persistence flag is not strictly required in all cases, but setting it won’t harm it, and may improve compatibility.

If the system does not immediately recognize the new partition, run:

sudo partprobe

== Step 3—Format the persistence partition ==
Format the new partition as ext4 with the label persistence—the Debian live system looks for it by name at boot:
$ sudo mkfs.ext4 -L persistence /dev/sdXN
Replace sdXN with your actual partition (for example, /dev/sdb3 ).

== Step 4—Create the persistence configuration file ==
Mount the new partition:
$ sudo mount /dev/sdXN /mnt
Create the configuration file that tells the live system what to persist:
$ echo "/ union" | sudo tee /mnt/persistence.conf
This tells the system to use the entire filesystem as a writable overlay, preserving changes across reboots.

Unmount the partition cleanly:
$ sudo umount /mnt

== Step 5—Boot with persistence ==
Boot your system from the USB drive.

At the Debian live boot menu:

* Press Tab (on BIOS systems), or
* Press e (on UEFI/GRUB systems)

Find the line beginning with linux and add:
persistence
to the end of that line. Then boot the system.

== Step 6—Verify persistence is working ==
After booting, create a test file:
$ touch ~/persistence-test
Reboot again with the persistence option enabled. If the file is there, persistence is working. Good job!

This is NOT a stealth tool. When used correctly, a Debian Live USB Persistent setup doesn’t modify the host system. No files or logs are written to the internal disk unless you explicitly mount and use it.

It does not provide anonymity or invisibility:

* '''Boot events may be logged (firmware / enterprise systems)'''
* '''USB usage may be detectable'''
* '''Network activity is visible to routers, ISPs, and organizations'''
* '''User actions can leave traces if internal drives are mounted'''

The persistence partition is a convenient place to store working data such as documents, scripts, and configuration files. Think of it as a portable home directory stored in your portable USB drive.

If you’re working across machines with different hardware, most software will work without any issues. However, certain hardware-specific packages such as drivers or kernel modules) may not work on different systems.

---

''This work was contributed by distro-nix on Debian User Forums on 2026-04-06.'' Comments, suggestions or resources are welcome.

Debian Live USB with Persistence

2026-04-06T22:00:47Z

Distro-nix:

All commands are verified on Debian 13.4 (Trixie) / 6.12.74+deb13+1-amd64

Created : 2026-05-04

Last Updated : 2026-05-04

ID: 005146.8

''A persistent Debian live USB can come in handy once you have prepared it. Boot it on your own machine, a colleague’s laptop, or a borrowed computer. It has your environment, your files, your settings, exactly where you left them.''

The persistence layer is what makes all the difference: a standard live USB is stateless—helpful for recovery and installation, but each session starts fresh. By adding a persistence partition, you have a portable Debian installation: your shell configuration, your documents, your utilities, your scripts, your browser profile, all carried around in your pocket.

This is also a practical solution when you’re working across multiple machines but don’t want to maintain separate installations on each. It’s particularly useful for sysadmins who move between systems regularly, or anyone who needs a known-good Debian environment available on demand without depending on the host.

== ''What you’ll need'' ==
A USB drive of at least 8 GB—the minimum requirement is 4GB, but that leaves you very little room for the persistence partition. The more you plan to store, the larger the drive should be. A 32 GB drive gives you more comfortable working space.

Download the latest Debian Live ISO from the official Debian website at <nowiki>https://www.debian.org/CD/live</nowiki> Choose the release and desktop environment that suits you, or the standard image if you prefer a lean setup.

Verify the authenticity of the image <nowiki>https://www.debian.org/CD/verify</nowiki>

You’ll be working entirely in the terminal on a GNU/Linux system. The tools you need:

* dd
* parted
* mkfs.ext4

These are standard utilities in Debian systems *.

dd is part of GNU coreutils package

mkfs.ext4 is part of the e2fsprogs package.*

<nowiki>*</nowiki> usually present on full Debian installations, but on a standard live environment it may not be present.
$ sudo apt install e2fsprogs parted
These are small, standard packages available in the Debian repositories. In a live session, they don’t persist unless you have persistence already enabled, so install them in the same session you do the setup.

== A word about dd before you start ==
The dd package has earned the nickname "disk destroyer" because it destroys whatever you specify as the output (of=) without warning if you make a mistake. dd writes directly to block devices with no undo. Double-check your device identifier carefully before running any command.

Plug in the USB drive and open the terminal and run:
$ lsblk
Look for your USB device in the output—it will appear as /dev/sdb, /dev/sdc, or similar depending on what else is attached. Check that you have the right device.

'''Important:''' use the device itself (e.g., /dev/sdb), not a partition such as /dev/sdb1.

In the commands that follow, /dev/sdX is a placeholder—replace it with your actual device identifier every time.

== Step 1—Write the ISO to the USB drive ==
$ sudo dd if=/path/to/debian-live-13.4.0-amd64-kde.iso of=/dev/sdX bs=4M status=progress && sync
The status=progress flag gives you a running transfer rate, otherwise you can’t see it is working. The sync at the end flushes any remaining data from the write cache—don’t pull the drive before it completes.

This will take a few minutes depending on the drive speed. When it finishes, the USB is bootable but stateless. The next steps add the persistence layer.

== Step 2—Create the persistence partition ==
After writing the ISO to the USB drive, only a portion of the disk is used. The remaining space is left unallocated. You’ll create a new partition in that space for ''persistence''.

Make sure the USB device isn’t mounted, then start parted:
$ sudo parted /dev/sdX
Inside, parted display the current layout:
(parted) print free
You should see existing partitions followed by a region labeled '''Free Space'''.

=== Example layout ===
Your output may look similar to this:

Number Start End Size File system Flags

1 1049kB 3500MB 3499MB fat32 boot, esp

2 3500MB 3600MB 100MB

 3600MB 32.1GB ~28GB Free Space

The important part is the '''free space at the end''' of the disk. This is where the persistence partition will go.

=== Create the new partition ===
Instead of calculating start and end points manually, let parted use the available space:
(parted) mkpart primary ext4 100%
This creates a new partition using all remaining free space.

Set the persistence flag (replace N with the partition number shown in print):

(parted) set N persistence on
(parted) quit
Note: The persistence flag is not strictly required in all cases, but setting it won’t harm it, and may improve compatibility.

If the system does not immediately recognize the new partition, run:

sudo partprobe

== Step 3—Format the persistence partition ==
Format the new partition as ext4 with the label persistence—the Debian live system looks for it by name at boot:
$ sudo mkfs.ext4 -L persistence /dev/sdXN
Replace sdXN with your actual partition (for example, /dev/sdb3 ).

== Step 4—Create the persistence configuration file ==
Mount the new partition:
$ sudo mount /dev/sdXN /mnt
Create the configuration file that tells the live system what to persist:
$ echo "/ union" | sudo tee /mnt/persistence.conf
This tells the system to use the entire filesystem as a writable overlay, preserving changes across reboots.

Unmount the partition cleanly:
$ sudo umount /mnt

== Step 5—Boot with persistence ==
Boot your system from the USB drive.

At the Debian live boot menu:

* Press Tab (on BIOS systems), or
* Press e (on UEFI/GRUB systems)

Find the line beginning with linux and add:
persistence
to the end of that line. Then boot the system.

== Step 6—Verify persistence is working ==
After booting, create a test file:
$ touch ~/persistence-test
Reboot again with the persistence option enabled. If the file is there, persistence is working. Good job!

This is NOT a stealth tool. When used correctly, a Debian Live USB Persistent setup doesn’t modify the host system. No files or logs are written to the internal disk unless you explicitly mount and use it.

It does not provide anonymity or invisibility:

* '''Boot events may be logged (firmware / enterprise systems)'''
* '''USB usage may be detectable'''
* '''Network activity is visible to routers, ISPs, and organizations'''
* '''User actions can leave traces if internal drives are mounted'''

The persistence partition is a convenient place to store working data such as documents, scripts, and configuration files. Think of it as a portable home directory stored in your portable USB drive.

If you’re working across machines with different hardware, most software will work without any issues. However, certain hardware-specific packages such as drivers or kernel modules) may not work on different systems.

---

''This work was contributed by distro-nix on Debian User Forums on 2026-04-05.'' Comments, suggestions or resources are welcome.

Debian Live USB with Persistence

2026-04-06T20:06:42Z

Distro-nix: /* A word about dd before you start */

All commands are verified on Debian 13.4 (Trixie) / 6.12.74+deb13+1-amd64

Created : 2026-05-04

Last Updated : 2026-04-05

ID: 005146.7

''A persistent Debian live USB can come in handy once you have prepared it. Boot it on your own machine, a colleague’s laptop, or a borrowed computer. It has your environment, your files, your settings, exactly where you left them.''

The persistence layer is what makes all the difference: a standard live USB is stateless—helpful for recovery and installation, but each session starts fresh. By adding a persistence partition, you have a portable Debian installation: your shell configuration, your documents, your utilities, your scripts, your browser profile, all carried around in your pocket.

This is also a practical solution when you’re working across multiple machines but don’t want to maintain separate installations on each. It’s particularly useful for sysadmins who move between systems regularly, or anyone who needs a known-good Debian environment available on demand without depending on the host.

== ''What you’ll need'' ==
A USB drive of at least 8 GB—the minimum requirement is 4GB, but that leaves you very little room for the persistence partition. The more you plan to store, the larger the drive should be. A 32 GB drive gives you more comfortable working space.

Download the latest Debian Live ISO from the official Debian website at <nowiki>https://www.debian.org/CD/live</nowiki> Choose the release and desktop environment that suits you, or the standard image if you prefer a lean setup.

Verify the authenticity of the image <nowiki>https://www.debian.org/CD/verify</nowiki>

You’ll be working entirely in the terminal on a GNU/Linux system. The tools you need:

* dd
* parted
* mkfs.ext4

These are standard utilities in Debian systems *.

dd is part of GNU coreutils package

mkfs.ext4 is part of the e2fsprogs package.*

<nowiki>*</nowiki> usually present on full Debian installations, but on a standard live environment it may not be present.
$ sudo apt install e2fsprogs parted
These are small, standard packages available in the Debian repositories. In a live session, they don’t persist unless you have persistence already enabled, so install them in the same session you do the setup.

== A word about dd before you start ==
The dd package has earned the nickname "disk destroyer" because it destroys whatever you specify as the output (of=) without warning if you make a mistake. dd writes directly to block devices with no undo. Double-check your device identifier carefully before running any command.

Plug in the USB drive and open the terminal and run:
$ lsblk
Look for your USB device in the output—it will appear as /dev/sdb, /dev/sdc, or similar depending on what else is attached. Check that you have the right device.

'''Important:''' use the device itself (e.g., /dev/sdb), not a partition such as /dev/sdb1.

In the commands that follow, /dev/sdX is a placeholder—replace it with your actual device identifier every time.

== Step 1—Write the ISO to the USB drive ==
$ sudo dd if=/path/to/debian-live-13.4.0-amd64-kde.iso of=/dev/sdX bs=4M status=progress && sync
The status=progress flag gives you a running transfer rate, otherwise you can’t see it is working. The sync at the end flushes any remaining data from the write cache—don’t pull the drive before it completes.

This will take a few minutes depending on the drive speed. When it finishes, the USB is bootable but stateless. The next steps add the persistence layer.

== Step 2—Create the persistence partition ==
After writing the ISO to the USB drive, only a portion of the disk is used. The remaining space is left unallocated. You’ll create a new partition in that space for ''persistence''.

Make sure the USB device isn’t mounted, then start parted:
$ sudo parted /dev/sdX
Inside, parted display the current layout:
(parted) print free
You should see existing partitions followed by a region labeled '''Free Space'''.

=== Example layout ===
Your output may look similar to this:

Number Start End Size File system Flags

1 1049kB 3500MB 3499MB fat32 boot, esp

2 3500MB 3600MB 100MB

 3600MB 32.1GB ~28GB Free Space

The important part is the '''free space at the end''' of the disk. This is where the persistence partition will go.

=== Create the new partition ===
Instead of calculating start and end points manually, let parted use the available space:
(parted) mkpart primary ext4 100%
This creates a new partition using all remaining free space.

Set the persistence flag (replace N with the partition number shown in print):

(parted) set N persistence on
(parted) quit
Note: The persistence flag is not strictly required in all cases, but setting it won’t harm it, and may improve compatibility.

If the system does not immediately recognize the new partition, run:

sudo partprobe

== Step 3—Format the persistence partition ==
Format the new partition as ext4 with the label persistence—the Debian live system looks for it by name at boot:
$ sudo mkfs.ext4 -L persistence /dev/sdXN
Replace sdXN with your actual partition (for example, /dev/sdb3 ).

== Step 4—Create the persistence configuration file ==
Mount the new partition:
$ sudo mount /dev/sdXN /mnt
Create the configuration file that tells the live system what to persist:
$ echo "/ union" | sudo tee /mnt/persistence.conf
This tells the system to use the entire filesystem as a writable overlay, preserving changes across reboots.

Unmount the partition cleanly:
$ sudo umount /mnt

== Step 5—Boot with persistence ==
Boot your system from the USB drive.

At the Debian live boot menu:

* Press Tab (on BIOS systems), or
* Press e (on UEFI/GRUB systems)

Find the line beginning with linux and add:
persistence
to the end of that line. Then boot the system.

== Step 6—Verify persistence is working ==
After booting, create a test file:
$ touch ~/persistence-test
Reboot again with the persistence option enabled. If the file is there, persistence is working. Good job!

This is NOT a stealth tool. When used correctly, a Debian Live USB Persistent setup doesn’t modify the host system. No files or logs are written to the internal disk unless you explicitly mount and use it.

It does not provide anonymity or invisibility:

* '''Boot events may be logged (firmware / enterprise systems)'''
* '''USB usage may be detectable'''
* '''Network activity is visible to routers, ISPs, and organizations'''
* '''User actions can leave traces if internal drives are mounted'''

The persistence partition is a convenient place to store working data such as documents, scripts, and configuration files. Think of it as a portable home directory stored in your portable USB drive.

If you’re working across machines with different hardware, most software will work without any issues. However, certain hardware-specific packages ‘such as drivers or kernel modules) may not work on different systems.

---

''This work was contributed by distro-nix on Debian User Forums on 2026-05-05.'' Comments, suggestions or resources are welcome.

Debian Live USB with Persistence

2026-04-06T19:56:21Z

Distro-nix:

All commands are verified on Debian 13.4 (Trixie) / 6.12.74+deb13+1-amd64

Created : 2026-05-04

Last Updated : 2026-04-05

ID: 005146.7

''A persistent Debian live USB can come in handy once you have prepared it. Boot it on your own machine, a colleague’s laptop, or a borrowed computer. It has your environment, your files, your settings, exactly where you left them.''

The persistence layer is what makes all the difference: a standard live USB is stateless—helpful for recovery and installation, but each session starts fresh. By adding a persistence partition, you have a portable Debian installation: your shell configuration, your documents, your utilities, your scripts, your browser profile, all carried around in your pocket.

This is also a practical solution when you’re working across multiple machines but don’t want to maintain separate installations on each. It’s particularly useful for sysadmins who move between systems regularly, or anyone who needs a known-good Debian environment available on demand without depending on the host.

== ''What you’ll need'' ==
A USB drive of at least 8 GB—the minimum requirement is 4GB, but that leaves you very little room for the persistence partition. The more you plan to store, the larger the drive should be. A 32 GB drive gives you more comfortable working space.

Download the latest Debian Live ISO from the official Debian website at <nowiki>https://www.debian.org/CD/live</nowiki> Choose the release and desktop environment that suits you, or the standard image if you prefer a lean setup.

Verify the authenticity of the image <nowiki>https://www.debian.org/CD/verify</nowiki>

You’ll be working entirely in the terminal on a GNU/Linux system. The tools you need:

* dd
* parted
* mkfs.ext4

These are standard utilities in Debian systems *.

dd is part of GNU coreutils package

mkfs.ext4 is part of the e2fsprogs package.*

<nowiki>*</nowiki> usually present on full Debian installations, but on a standard live environment it may not be present.

$ sudo apt install e2fsprogs parted

These are small, standard packages available in the Debian repositories. In a live session, they don’t persist unless you have persistence already enabled, so install them in the same session you do the setup.

== A word about dd before you start ==
The dd package has earned the nickname "disk destroyer" because it destroys whatever you specify as the output (of=) without warning if you make a mistake. dd writes directly to block devices with no undo. Double-check your device identifier carefully before running any command.

Plug in the USB drive and open the terminal and run:

$ lsblk

Look for your USB device in the output—it will appear as /dev/sdb, /dev/sdc, or similar depending on what else is attached. Check that you have the right device.

'''Important:''' use the device itself (e.g., /dev/sdb), not a partition such as /dev/sdb1.

In the commands that follow, /dev/sdX is a placeholder—replace it with your actual device identifier every time.

Step 1—Write the ISO to the USB drive

$ sudo dd if=/path/to/debian-live-13.4.0-amd64-kde.iso of=/dev/sdX bs=4M status=progress && sync

The status=progress flag gives you a running transfer rate, otherwise you can’t see it is working. The sync at the end flushes any remaining data from the write cache—don’t pull the drive before it completes.

This will take a few minutes depending on the drive speed. When it finishes, the USB is bootable but stateless. The next steps add the persistence layer.

Step 2—Create the persistence partition

After writing the ISO to the USB drive, only a portion of the disk is used. The remaining space is left unallocated. You’ll create a new partition in that space for ''persistence''.

Make sure the USB device isn’t mounted, then start parted:

$ sudo parted /dev/sdX

Inside, parted display the current layout:

(parted) print free

You should see existing partitions followed by a region labeled '''Free Space'''.

=== Example layout ===
Your output may look similar to this:

Number Start End Size File system Flags

1 1049kB 3500MB 3499MB fat32 boot, esp

2 3500MB 3600MB 100MB

 3600MB 32.1GB ~28GB Free Space

The important part is the '''free space at the end''' of the disk. This is where the persistence partition will go.

=== Create the new partition ===
Instead of calculating start and end points manually, let parted use the available space:

(parted) mkpart primary ext4 100%

This creates a new partition using all remaining free space.

Set the persistence flag (replace N with the partition number shown in print):

(parted) set N persistence on

(parted) quit

Note: The persistence flag is not strictly required in all cases, but setting it won’t harm it, and may improve compatibility.

If the system does not immediately recognize the new partition, run:

sudo partprobe

Step 3—Format the persistence partition

Format the new partition as ext4 with the label persistence—the Debian live system looks for it by name at boot:

$ sudo mkfs.ext4 -L persistence /dev/sdXN

Replace sdXN with your actual partition (for example, /dev/sdb3 ).

Step 4—Create the persistence configuration file

Mount the new partition:

$ sudo mount /dev/sdXN /mnt

Create the configuration file that tells the live system what to persist:

$ echo "/ union" | sudo tee /mnt/persistence.conf

This tells the system to use the entire filesystem as a writable overlay, preserving changes across reboots.

Unmount the partition cleanly:

$ sudo umount /mnt

Step 5—Boot with persistence

Boot your system from the USB drive.

At the Debian live boot menu:

* Press Tab (on BIOS systems), or
* Press e (on UEFI/GRUB systems)

Find the line beginning with linux and add:

persistence

to the end of that line. Then boot the system.

== Step 6—Verify persistence is working ==
After booting, create a test file:

$ touch ~/persistence-test

Reboot again with the persistence option enabled. If the file is there, persistence is working. Good job!

This is NOT a stealth tool. When used correctly, a Debian Live USB Persistent setup doesn’t modify the host system. No files or logs are written to the internal disk unless you explicitly mount and use it.

It does not provide anonymity or invisibility:

* '''Boot events may be logged (firmware / enterprise systems)'''
* '''USB usage may be detectable'''
* '''Network activity is visible to routers, ISPs, and organizations'''
* '''User actions can leave traces if internal drives are mounted'''

The persistence partition is a convenient place to store working data such as documents, scripts, and configuration files. Think of it as a portable home directory stored in your portable USB drive.

If you’re working across machines with different hardware, most software will work without any issues. However, certain hardware-specific packages ‘such as drivers or kernel modules) may not work on different systems.

---

''This work was contributed by distro-nix on Debian User Forums on 2026-05-05.'' Comments, suggestions or resources are welcome.

Debian Live USB with Persistence

2026-04-06T19:50:48Z

Distro-nix: Created page with "* '''Debian Live USB with Persistence''' All commands are verified on Debian 13.4 (Trixie) / 6.12.74+deb13+1-amd64 Created : 2026-05-04 Last Updated : 2026-04-05 ID: 005146.7 ''A persistent Debian live USB can come in handy once you have prepared it. Boot it on your own machine, a colleague’s laptop, or a borrowed computer. It has your environment, your files, your settings, exactly where you left them.'' The persistence layer is what makes all the difference: a..."

* '''Debian Live USB with Persistence'''

All commands are verified on Debian 13.4 (Trixie) / 6.12.74+deb13+1-amd64

Created : 2026-05-04

Last Updated : 2026-04-05

ID: 005146.7

''A persistent Debian live USB can come in handy once you have prepared it. Boot it on your own machine, a colleague’s laptop, or a borrowed computer. It has your environment, your files, your settings, exactly where you left them.''

The persistence layer is what makes all the difference: a standard live USB is stateless—helpful for recovery and installation, but each session starts fresh. By adding a persistence partition, you have a portable Debian installation: your shell configuration, your documents, your utilities, your scripts, your browser profile, all carried around in your pocket.

This is also a practical solution when you’re working across multiple machines but don’t want to maintain separate installations on each. It’s particularly useful for sysadmins who move between systems regularly, or anyone who needs a known-good Debian environment available on demand without depending on the host.

== ''What you’ll need'' ==
A USB drive of at least 8 GB—the minimum requirement is 4GB, but that leaves you very little room for the persistence partition. The more you plan to store, the larger the drive should be. A 32 GB drive gives you more comfortable working space.

Download the latest Debian Live ISO from the official Debian website at <nowiki>https://www.debian.org/CD/live</nowiki> Choose the release and desktop environment that suits you, or the standard image if you prefer a lean setup.

Verify the authenticity of the image <nowiki>https://www.debian.org/CD/verify</nowiki>

You’ll be working entirely in the terminal on a GNU/Linux system. The tools you need:

* dd
* parted
* mkfs.ext4

These are standard utilities in Debian systems *.

dd is part of GNU coreutils package

mkfs.ext4 is part of the e2fsprogs package.*

<nowiki>*</nowiki> usually present on full Debian installations, but on a standard live environment it may not be present.

$ sudo apt install e2fsprogs parted

These are small, standard packages available in the Debian repositories. In a live session, they don’t persist unless you have persistence already enabled, so install them in the same session you do the setup.

== A word about dd before you start ==
The dd package has earned the nickname "disk destroyer" because it destroys whatever you specify as the output (of=) without warning if you make a mistake. dd writes directly to block devices with no undo. Double-check your device identifier carefully before running any command.

Plug in the USB drive and open the terminal and run:

$ lsblk

Look for your USB device in the output—it will appear as /dev/sdb, /dev/sdc, or similar depending on what else is attached. Check that you have the right device.

'''Important:''' use the device itself (e.g., /dev/sdb), not a partition such as /dev/sdb1.

In the commands that follow, /dev/sdX is a placeholder—replace it with your actual device identifier every time.

Step 1—Write the ISO to the USB drive

$ sudo dd if=/path/to/debian-live-13.4.0-amd64-kde.iso of=/dev/sdX bs=4M status=progress && sync

The status=progress flag gives you a running transfer rate, otherwise you can’t see it is working. The sync at the end flushes any remaining data from the write cache—don’t pull the drive before it completes.

This will take a few minutes depending on the drive speed. When it finishes, the USB is bootable but stateless. The next steps add the persistence layer.

Step 2—Create the persistence partition

After writing the ISO to the USB drive, only a portion of the disk is used. The remaining space is left unallocated. You’ll create a new partition in that space for ''persistence''.

Make sure the USB device isn’t mounted, then start parted:

$ sudo parted /dev/sdX

Inside, parted display the current layout:

(parted) print free

You should see existing partitions followed by a region labeled '''Free Space'''.

=== Example layout ===
Your output may look similar to this:

Number Start End Size File system Flags

1 1049kB 3500MB 3499MB fat32 boot, esp

2 3500MB 3600MB 100MB

 3600MB 32.1GB ~28GB Free Space

The important part is the '''free space at the end''' of the disk. This is where the persistence partition will go.

=== Create the new partition ===
Instead of calculating start and end points manually, let parted use the available space:

(parted) mkpart primary ext4 100%

This creates a new partition using all remaining free space.

Set the persistence flag (replace N with the partition number shown in print):

(parted) set N persistence on

(parted) quit

Note: The persistence flag is not strictly required in all cases, but setting it won’t harm it, and may improve compatibility.

If the system does not immediately recognize the new partition, run:

sudo partprobe

Step 3—Format the persistence partition

Format the new partition as ext4 with the label persistence—the Debian live system looks for it by name at boot:

$ sudo mkfs.ext4 -L persistence /dev/sdXN

Replace sdXN with your actual partition (for example, /dev/sdb3 ).

Step 4—Create the persistence configuration file

Mount the new partition:

$ sudo mount /dev/sdXN /mnt

Create the configuration file that tells the live system what to persist:

$ echo "/ union" | sudo tee /mnt/persistence.conf

This tells the system to use the entire filesystem as a writable overlay, preserving changes across reboots.

Unmount the partition cleanly:

$ sudo umount /mnt

Step 5—Boot with persistence

Boot your system from the USB drive.

At the Debian live boot menu:

* Press Tab (on BIOS systems), or
* Press e (on UEFI/GRUB systems)

Find the line beginning with linux and add:

persistence

to the end of that line. Then boot the system.

== Step 6—Verify persistence is working ==
After booting, create a test file:

$ touch ~/persistence-test

Reboot again with the persistence option enabled. If the file is there, persistence is working. Good job!

This is NOT a stealth tool. When used correctly, a Debian Live USB Persistent setup doesn’t modify the host system. No files or logs are written to the internal disk unless you explicitly mount and use it.

It does not provide anonymity or invisibility:

* '''Boot events may be logged (firmware / enterprise systems)'''
* '''USB usage may be detectable'''
* '''Network activity is visible to routers, ISPs, and organizations'''
* '''User actions can leave traces if internal drives are mounted'''

The persistence partition is a convenient place to store working data such as documents, scripts, and configuration files. Think of it as a portable home directory stored in your portable USB drive.

If you’re working across machines with different hardware, most software will work without any issues. However, certain hardware-specific packages ‘such as drivers or kernel modules) may not work on different systems.

''This work was contributed by distro-nix on Debian User Forums on 2026-05-05.'' Comments, suggestions or resources are welcome.

Main Page

2026-04-06T19:21:08Z

Distro-nix: /* HowTo (guides) */

= Welcome to the Debian User Forum Archives. =

This wiki is dedicated specific member solutions from the [https://forums.debian.net Debian User Forums] and not to be confused with the larger man style documentation driven official [https://wiki.debian.org Debian Project Wiki].

Our works here are citable formats of some of our best threads, member contributions, guides, and information that can provide assistance on or offline.

== About ==
About:
* [[Archive Debian Forums:About|Archive Debian Forums]]
* [[Debian User Forums]]
* [[The Debian Project]]
* [[Forum jokes, lingo, and memorable quotes]]

== Guides (reference) ==

* [[Root, Sudo, and SU|Root, sudo, su, and su -]] (forum archive)
* [[Debian First Aid Kit]] (full paper)
* [[Fonts and Themes in Cinnamon Desktop]] (full paper)
* [[To install Firefox from Mozilla repo|Installing Firefox from the Mozilla repository]] (forum archive)
* [[Security Hardening for Debian Users: Protecting Against Targeted Attacks]] (full paper)

== HowTo (guides) ==
* [[MTR Network Diagnostics]] (full paper)
* [[WiFi Signal Strength and Sharing]] (forum archive)
* [[REFInd USB Drive (for emergency boot)]] (forum archive)
* [[Trixie iwd]] (forum archive)
* [[Debian Live USB with Persistence]] (full paper)

== Licensing and Use ==
Licensing:
* [[Publish using CC International licencing and your Forum name]].
* [[Archive AI Use]]

File:Debian USB Persistent Setup-005146.odt

2026-04-06T18:52:44Z

Distro-nix: A persistent Debian live USB. By adding a persistence partition, you have a portable Debian installation: your shell configuration, your documents, your utilities, your scripts, your browser profile, all carried around in your pocket.

== Summary ==
A persistent Debian live USB. By adding a persistence partition, you have a portable Debian installation: your shell configuration, your documents, your utilities, your scripts, your browser profile, all carried around in your pocket.

MTR Network Diagnostics

2026-03-28T23:50:00Z

Distro-nix: /* Permission Note */

= MTR: Network Path Diagnostics =

MTR combines the functionality of ping and traceroute into a single tool that monitors every hop along a network path continuously, producing statistical output that identifies where latency and packet loss actually occur. When basic connectivity tools confirm a problem exists but don’t locate it, MTR provides the data needed to isolate it.

=== Installation ===
<code>sudo apt install mtr</code>

MTR is available in the Debian main repository and requires no additional sources. It is maintained across all current Debian releases and installs without external dependencies. The package name is mtr, which includes both the command-line interface and mtr-packet, the helper binary responsible for sending and receiving network packets.

=== Basic Usage ===
MTR operates in two modes: interactive, which displays a live updating view of the network path, and report mode, which runs a fixed number of cycles and prints a summary.

* Interactive mode
<code>mtr example.com</code>

* Report mode — 50 cycles, no DNS resolution
<code>mtr --report --report-cycles 50 --no-dns example.com</code>

* TCP mode — useful when ICMP is filtered
<code>sudo mtr --tcp --port 443 example.com</code>

* Save report to file
<code>mtr --report --report-cycles 100 --no-dns example.com > mtr-report.txt</code>

{| class="wikitable"
|+ Output Example
|-
! Host !! hostname !! Loss% !! Snt !! Last !! Ave !! Best !! Wrst !! StDEV
|-
| 1. || 192.168.1.1 || 0.0% || 50 || 1.2 || 1.5 || 1.0 || 2.3 || 0.4
|-
| 2. || 10.0.0.1 || 0.0% || 50 || 8.5 || 9.2 || 7.8 || 12.1 || 1.3
|-
| 3. || isp-gateway.net || 0.0% || 50 || 15.3 || 16.1 || 14.2 || 19.8 || 1.8
|-
| 4. || ??? || 100.0% || 50 || 0.0 || 0.0 || 0.0 || 0.0 || 0.0
|-
| 5. || destination.com || 0.0%|| 50|| 25.4|| 26.8|| 24.1|| 32.5|| 2.4
|-
|}
Each row represents one hop. The columns report packet loss percentage, packets sent, and latency values in milliseconds — last, average, best, worst, and standard deviation. Standard deviation measures consistency: a low value indicates stable latency, a high value indicates jitter.

==== Interpreting Results ====

'''Packet loss at an intermediate hop that does not persist to subsequent hops''' is typically not a network problem. Many routers deprioritise or rate-limit ICMP responses while forwarding traffic normally. If hop 4 shows 100% loss but hop 5 responds cleanly, traffic is flowing through hop 4 — it simply isn’t generating diagnostic replies.

'''Packet loss that begins at a hop and continues through to the destination''' indicates a genuine problem at or beyond that hop.

'''A sudden increase in latency at a specific hop that persists to all subsequent hops''' points to a congested or slow link at that point in the path.

'''High standard deviation''' indicates unstable latency, which affects real-time applications such as VoIP and video conferencing even when average latency appears acceptable.

'''A large latency increase with stable standard deviation and no packet loss''' reflects geographic distance or a slower interconnect, not a fault.

'''Problems confined to the first hop''' — your local router — and absent from all subsequent hops point to a local network issue: wireless interference, a misconfigured interface, or a problem with the router itself.

==== Practical Scenarios ====
===== Isolating a problem for an ISP or hosting provider =====

Run an extended report against a known external address and save the output:
<code>mtr --report-cycles 100 --no-dns 8.8.8.8 > isp-report.txt</code>

The report shows exactly where in the path packet loss or latency appears, which hop belongs to which network, and whether the problem is within the provider’s infrastructure or beyond it.

===== Testing when ICMP is filtered =====
Some paths block ICMP but pass TCP traffic. Testing with TCP against a specific port confirms whether a service is reachable end-to-end:

<code>sudo mtr --tcp --port 22 --report-cycles 50 remote-server.com
</code>

<code>sudo mtr --tcp --port 443 --report-cycles 50 remote-server.com
</code>
===== Diagnosing jitter for real-time applications =====
Run a longer test and focus on the StDev column:

<code>mtr --report-cycles 200 game-server.com</code>

Values above 5ms in the StDev column at any hop that persists to the destination will affect latency-sensitive applications.

==== Catching intermittent problems ====
Intermittent issues require longer observation windows than a standard report provides:

<code>mtr --report-cycles 500 --interval 1 target.com > long-test.txt
</code>

===== Protocol Selection =====

MTR supports ICMP (default), UDP, and TCP. When one protocol appears to show loss or filtering, testing with another confirms whether the issue is protocol-specific or affects all traffic:

''Icmp''
mtr example.com

''UDP''
mtr --udp example.com

''TCP on port 80''
sudo mtr --tcp --port 80 example.com

===== Output Formats =====
MTR can produce machine-readable output for logging or automated analysis:

<code>
mtr --report --csv example.com

mtr --report --json example.com

mtr --report --xml example.com
</code>

===== Additional Options =====
* Show both hostname and IP address
<code>mtr --show-ips example.com</code>

* Display Autonomous System numbers
<code>mtr --aslookup example.com</code>

* Limit maximum hops
<code>mtr --max-ttl 20 example.com</code>

* Specify source address on systems with multiple interfaces
<code>mtr --address 192.168.1.100 example.com</code>

* Force IPv4 or IPv6
<code>
mtr -4 example.com

mtr -6 example.com
</code>

===== Interactive Mode Key Bindings =====
In interactive mode, the following keys are available:
{| class="wikitable"
|+ Available keys
|-
! Key !! Action
|-
| d || Toggle display mode
|-
| n || Toggle between hostnames and IP addresses
|-
| r || Reset statistics
|-
| p || Pause and resume
|-
| u || Cycle between ICMP, UDP, and TCP
|-
| y || Toggle between IPv4 and IPv6
|-
| q || Quit
|}

===== Useful Aliases =====
These aliases cover the most common diagnostic tasks. Add them to ~/.bashrc and reload with source ~/.bashrc:

* Standard report — 50 cycles, no DNS
<code>alias mtrreport='mtr --report --report-cycles 50 --no-dns'</code>

* Extended report — 100 cycles
<code>alias mtrlong='mtr --report --report-cycles 100 --no-dns'</code>

* HTTPS path test
<code>alias mtrhttps='sudo mtr --tcp --port 443 --report-cycles 50'
</code>

===== Permission Note =====
TCP mode on privileged ports requires either sudo or the cap_net_raw capability set on the mtr-packet binary:

<code>sudo setcap cap_net_raw+ep /usr/bin/mtr-packet</code>

After setting the capability, TCP tests run without sudo.

Remember: If in doubt, search for the specific error message along with "Debian" and the version number. e.g. Debian 13 or point release if needed, Debian 13,1

This work was contributed by distro-nix on Debian User Forums on 2025-10-27 23:38:22

Further resources are available at https://www.debianfirstaid.org
[[index.php?title=Category:HowTo]]
[[index.php?title=Category:Networking]]
[[index.php?title=Category:Full Paper]]

Fonts and Themes in Cinnamon Desktop

2026-03-28T23:45:53Z

Distro-nix: /* Additional Resources */

== Fonts and Themes in Cinnamon Desktop ==

* Verified on Debian 13.1 (Trixie) / 6.12.43+deb13-amd64 x64_64
* Created : 28/10/2025 21:09:21
* Last Updated : 27/10/2025 23:42:45

A comprehensive reference for managing console fonts, GUI fonts, and theme customization in Debian with Cinnamon DE.


== Table of Contents ==

<ol style="list-style-type: decimal;">
<li><blockquote>[https://claude.ai/chat/6986fa74-5020-47a2-8736-f9eaf474bfc0#understanding-font-types Understanding Font Types]</blockquote></li>
<li><blockquote>[https://claude.ai/chat/6986fa74-5020-47a2-8736-f9eaf474bfc0#console-fonts-tty Console Fonts (TTY)]</blockquote></li>
<li><blockquote>[https://claude.ai/chat/6986fa74-5020-47a2-8736-f9eaf474bfc0#gui-fonts GUI Fonts]</blockquote></li>
<li><blockquote>[https://claude.ai/chat/6986fa74-5020-47a2-8736-f9eaf474bfc0#cinnamon-system-fonts Cinnamon System Fonts]</blockquote></li>
<li><blockquote>[https://claude.ai/chat/6986fa74-5020-47a2-8736-f9eaf474bfc0#theme-customization-with-css Theme Customization with CSS]</blockquote></li>
<li><blockquote>[https://claude.ai/chat/6986fa74-5020-47a2-8736-f9eaf474bfc0#accessibility-and-scaling Accessibility and Scaling]</blockquote></li>
<li><blockquote>[https://claude.ai/chat/6986fa74-5020-47a2-8736-f9eaf474bfc0#troubleshooting Troubleshooting]</blockquote></li></ol>


== Understanding Font Types ==

Linux uses different font systems depending on where text appears:


=== Console Fonts (TTY) ===

<ul>
<li><blockquote>'''Format''': PSF (PC Screen Font) - .psf or .psf.gz</blockquote></li>
<li><blockquote>'''Location''': /usr/share/consolefonts/</blockquote></li>
<li><blockquote>'''Used by''': Text-based virtual consoles (Ctrl+Alt+F2 through F6)</blockquote></li>
<li><blockquote>'''Loaded''': Before graphical system starts</blockquote></li>
<li><blockquote>'''Limitations''': No anti-aliasing, fixed character grid</blockquote></li></ul>


=== GUI Fonts ===

<ul>
<li><blockquote>'''Formats''': TrueType (.ttf), OpenType (.otf)</blockquote></li>
<li><blockquote>'''Location''': /usr/share/fonts/truetype/, /usr/share/fonts/opentype/</blockquote></li>
<li><blockquote>'''Used by''': All graphical applications</blockquote></li>
<li><blockquote>'''Features''': Anti-aliasing, scalability, subpixel rendering</blockquote></li></ul>


=== Why Two Systems? ===

The Linux console operates at a lower level than the graphical interface. It needs to display text even if the GUI fails to load, so it uses simpler font formats that don't require graphics libraries.


== Console Fonts (TTY) ==


=== Installing Console Fonts ===

Install Terminus (popular monospace console font):

sudo apt update

sudo apt install console-setup xfonts-terminus


=== Configuring Console Fonts ===


==== Method 1: Interactive Configuration ====

sudo dpkg-reconfigure console-setup

Follow the prompts:

<ol style="list-style-type: decimal;">
<li><blockquote>'''Encoding''': UTF-8</blockquote></li>
<li><blockquote>'''Character set''': Guess optimal character set</blockquote></li>
<li><blockquote>'''Font''': Terminus</blockquote></li>
<li><blockquote>'''Font size''': Choose from available sizes</blockquote></li></ol>

Available Terminus sizes: 8x14, 8x16, 10x20, 12x24, 14x28, 16x32

Recommendation: 16x32 for good readability


==== Method 2: Direct Configuration File Editing ====

Edit /etc/default/console-setup:

sudo nano /etc/default/console-setup

Add or modify these lines:

FONTFACE="Terminus"

FONTSIZE="16x32"

Other useful settings you might find:

CHARMAP="UTF-8"

CODESET="guess"


=== Applying Console Font Changes ===

The changes take effect on boot, but to apply immediately on all consoles:

sudo setupcon

'''Note''': This command only works when run from an actual TTY console, not from a terminal emulator in the GUI. If you get "not on the console" error, that's normal - the configuration is saved and will apply on next boot or when you switch to a TTY.


=== Testing Console Fonts ===

<ol style="list-style-type: decimal;">
<li><blockquote>Switch to virtual console: Ctrl + Alt + F2</blockquote></li>
<li><blockquote>Log in with your credentials</blockquote></li>
<li><blockquote>Check the font - you should see Terminus</blockquote></li>
<li><blockquote>Return to GUI: Ctrl + Alt + F1 or Ctrl + Alt + F7</blockquote></li></ol>


=== Viewing Available Console Fonts ===

List all available console fonts:

ls /usr/share/consolefonts/

Filter for Terminus fonts:

ls /usr/share/consolefonts/ | grep -i ter


=== Temporarily Testing Console Fonts ===

To test a font before making it permanent:

setfont /usr/share/consolefonts/Ter16x32n.psf.gz

'''Important''': setfont only works when run from a TTY console (Ctrl+Alt+F2), not from GUI terminals.

To check current console font (from TTY):

setfont -v


== GUI Fonts ==


=== Installing GUI Fonts ===


==== System-wide Installation ====

Install Terminus for graphical applications:

sudo apt install fonts-terminus

Install other popular fonts:

# Programming fonts

sudo apt install fonts-firacode fonts-hack fonts-jetbrains-mono

# General purpose fonts

sudo apt install fonts-liberation fonts-dejavu fonts-noto

# More options

sudo apt install fonts-ubuntu fonts-roboto

After installing, log out and back in, or rebuild font cache:

fc-cache -fv


==== User-only Installation ====

To install fonts just for your user account:

<ol style="list-style-type: decimal;">
<li><blockquote>Create fonts directory:</blockquote></li></ol>

mkdir -p ~/.local/share/fonts

<ol start="2" style="list-style-type: decimal;">
<li><blockquote>Copy font files (.ttf or .otf) to this directory:</blockquote></li></ol>

cp /path/to/your/font.ttf ~/.local/share/fonts/

<ol start="3" style="list-style-type: decimal;">
<li><blockquote>Rebuild font cache:</blockquote></li></ol>

fc-cache -fv


=== Font Locations ===

'''System-wide fonts''':

/usr/share/fonts/truetype/

/usr/share/fonts/opentype/

/usr/share/fonts/X11/

'''User-specific fonts''':

~/.local/share/fonts/


=== Listing Available Fonts ===

List all fonts available to GUI applications:

fc-list

Search for specific font:

fc-list | grep -i terminus

fc-list | grep -i dejavu

List fonts with details:

fc-list : family style file


=== Configuring Terminal Emulator Fonts ===


==== GNOME Terminal (default in Cinnamon) ====

<ol style="list-style-type: decimal;">
<li><blockquote>Open GNOME Terminal</blockquote></li>
<li><blockquote>Menu → Preferences</blockquote></li>
<li><blockquote>Select your profile</blockquote></li>
<li><blockquote>Go to '''Text''' tab</blockquote></li>
<li><blockquote>Uncheck "Use the system fixed-width font"</blockquote></li>
<li><blockquote>Click font button and select Terminus (or other font)</blockquote></li>
<li><blockquote>Choose size (12, 14, or 16 recommended)</blockquote></li></ol>


==== Guake (Drop-down Terminal) ====

Install GUI version of Terminus first:

sudo apt install fonts-terminus

guake --quit

guake &

Configure via GUI:

<ol style="list-style-type: decimal;">
<li><blockquote>Open Guake (F12)</blockquote></li>
<li><blockquote>Right-click → Preferences</blockquote></li>
<li><blockquote>Go to '''Appearance''' tab</blockquote></li>
<li><blockquote>Uncheck "Use the system fixed width font"</blockquote></li>
<li><blockquote>Select Terminus and size</blockquote></li></ol>

Configure via command line:

gsettings set org.guake.style.font use-system-font false

gsettings set org.guake.style.font font-name 'Terminus 12'


== Cinnamon System Fonts ==


=== Quick Font Configuration ===

Open '''System Settings''':

cinnamon-settings

Navigate to: '''Font Selection'''

You'll see these options:

<ul>
<li><blockquote>'''Default font''': Used for menus, buttons, dialogs (recommended: 10-12pt)</blockquote></li>
<li><blockquote>'''Desktop font''': Used for desktop icon labels (recommended: 10-11pt)</blockquote></li>
<li><blockquote>'''Document font''': Used in document viewers (recommended: 11pt)</blockquote></li>
<li><blockquote>'''Monospace font''': Used in terminals and code editors (recommended: Terminus 12pt)</blockquote></li>
<li><blockquote>'''Window title font''': Used in window title bars (recommended: 10-11pt bold)</blockquote></li></ul>


=== Recommended Configuration for Better Readability ===

Default font: Sans 11 or 12

Desktop font: Sans 10 or 11

Document font: Sans 11

Monospace font: Terminus 12 or 14

Window title font: Sans Bold 10 or 11


=== Using Command Line to Change Fonts ===

View current settings:

gsettings get org.cinnamon.desktop.interface font-name

gsettings get org.gnome.desktop.interface monospace-font-name

Change default font:

gsettings set org.cinnamon.desktop.interface font-name 'Sans 12'

Change monospace font:

gsettings set org.gnome.desktop.interface monospace-font-name 'Terminus 12'


=== Font Recommendations by Use Case ===

'''For general readability''':

<ul>
<li><blockquote>Default: DejaVu Sans 11-12</blockquote></li>
<li><blockquote>Monospace: Terminus 12 or DejaVu Sans Mono 11</blockquote></li></ul>

'''For programming''':

<ul>
<li><blockquote>Monospace: Fira Code 11, JetBrains Mono 11, or Hack 11</blockquote></li></ul>

'''For visually impaired users''':

<ul>
<li><blockquote>Default: Sans 13-14</blockquote></li>
<li><blockquote>Monospace: Terminus 14-16</blockquote></li>
<li><blockquote>Consider UI scaling (see Accessibility section)</blockquote></li></ul>


== Theme Customization with CSS ==

Cinnamon themes use CSS for styling. You can customize any theme to change fonts, colors, spacing, and more.


=== Finding Your Current Theme ===

Check which theme you're using:

gsettings get org.cinnamon.theme name

List available themes:

ls /usr/share/themes/

ls ~/.themes/

View your theme in System Settings: '''System Settings → Themes'''


=== Creating a Custom Theme Copy ===

Always work on a copy to avoid breaking your system:

# Copy system theme to your user directory

cp -r /usr/share/themes/YOUR-THEME-NAME ~/.themes/

# Example:

cp -r /usr/share/themes/Mint-Y ~/.themes/

cp -r /usr/share/themes/Green-Submarine ~/.themes/


=== Theme Structure ===

A typical Cinnamon theme:

~/.themes/YOUR-THEME-NAME/

├── cinnamon/

│ ├── cinnamon.css # Main styling file

│ ├── thumbnail.png

│ └── assets/ # Images, icons

├── gtk-3.0/ # GTK3 application styling

├── gtk-2.0/ # GTK2 application styling

├── metacity-1/ # Window decorations

└── index.theme # Theme metadata

The file you'll edit most: cinnamon/cinnamon.css


=== Editing Theme CSS ===

Open the CSS file:

nano ~/.themes/YOUR-THEME-NAME/cinnamon/cinnamon.css


==== Common CSS Selectors ====

/* Overall stage - affects most UI elements */

stage {

font-size: 10pt;

font-family: sans-serif;

}

/* Main menu */

.menu {

font-size: 10pt;

background-color: rgba(48, 48, 48, 0.95);

border-radius: 8px;

}

/* Menu application buttons */

.menu-application-button-label {

font-size: 9pt;

padding-left: 10px;

}

/* Panel (taskbar) */

.panel {

font-size: 9pt;

background-color: #2b2b2b;

height: 32px;

}

/* Panel applet labels */

.panel-button {

font-size: 9pt;

}

/* Notifications */

.notification {

font-size: 10pt;

}

/* Window list buttons */

.window-list-item-box {

font-size: 9pt;

}

/* Tooltips */

.tooltip {

font-size: 9pt;

}


=== Example: Increasing Menu Font Size ===

Find the menu section and modify:

.menu {

font-size: 14pt; /* Increase from default 9-10pt */

background-color: rgba(48, 48, 48, 0.95);

border-radius: 8px;

}

.menu-application-button-label {

font-size: 13pt; /* Increase menu item text */

padding-left: 10px;

}

/* Also increase category labels */

.menu-category-button-label {

font-size: 13pt;

}


=== Example: Changing Font Family ===

stage {

font-family: "DejaVu Sans", sans-serif;

font-size: 11pt;

}

/* Use monospace for specific elements */

.some-element {

font-family: "Terminus", "DejaVu Sans Mono", monospace;

}


=== Applying CSS Changes ===

After editing the CSS file:

<ol style="list-style-type: decimal;">
<li><blockquote>Save the file: Ctrl+O then Enter (in nano)</blockquote></li>
<li><blockquote>Exit: Ctrl+X (in nano)</blockquote></li>
<li><blockquote>Restart Cinnamon: Press Alt+F2, type r, press Enter</blockquote></li></ol>

Or reload from command line:

# Ensure theme is set

gsettings set org.cinnamon.theme name 'YOUR-THEME-NAME'

# Restart Cinnamon

cinnamon --replace &


=== CSS Tips and Tricks ===

'''Finding the right selector''': Use Cinnamon's Looking Glass debugger

<ol style="list-style-type: decimal;">
<li><blockquote>Press Alt+F2</blockquote></li>
<li><blockquote>Type lg and press Enter</blockquote></li>
<li><blockquote>Go to '''Picker''' tab</blockquote></li>
<li><blockquote>Click on UI elements to see their CSS classes</blockquote></li></ol>

'''Testing changes quickly''': Keep the CSS file open in one workspace, test in another, and use Alt+F2 → r to reload.

'''Backup before editing''':

cp ~/.themes/YOUR-THEME-NAME/cinnamon/cinnamon.css ~/.themes/YOUR-THEME-NAME/cinnamon/cinnamon.css.backup

'''Restore backup if needed''':

cp ~/.themes/YOUR-THEME-NAME/cinnamon/cinnamon.css.backup ~/.themes/YOUR-THEME-NAME/cinnamon/cinnamon.css


=== Common CSS Properties ===

/* Fonts */

font-family: "Font Name", fallback;

font-size: 10pt; /* or 12px, 1.2em */

font-weight: bold; /* or normal, 600 */

font-style: italic;

/* Colors */

color: #ffffff;

background-color: rgba(48, 48, 48, 0.95);

border-color: #444444;

/* Spacing */

padding: 10px;

margin: 5px;

padding-left: 10px;

/* Borders */

border: 1px solid #444444;

border-radius: 8px;

/* Sizing */

width: 300px;

height: 40px;

min-width: 200px;


== Accessibility and Scaling ==


=== UI Scaling (Recommended for Visual Impairment) ===


==== Via System Settings ====

'''System Settings → Display → UI Scale'''

Options typically: 100%, 125%, 150%, 200%

Start with 125% and adjust as needed.


==== Via Command Line ====

# Check current scaling

gsettings get org.cinnamon.desktop.interface scaling-factor

# Set scaling (1 = 100%, 2 = 200%)

gsettings set org.cinnamon.desktop.interface scaling-factor 2

'''Note''': Fractional scaling (1.25, 1.5) may not be available on all systems.


=== Text Scaling Only ===

If you don't want to scale everything, increase text size:

# Increase text scaling factor

gsettings set org.gnome.desktop.interface text-scaling-factor 1.25

Values: 1.0 = 100%, 1.25 = 125%, 1.5 = 150%


=== High Contrast Themes ===

'''System Settings → Themes'''

Look for high-contrast themes:

<ul>
<li><blockquote>HighContrast (if available)</blockquote></li>
<li><blockquote>HighContrastInverse</blockquote></li></ul>

Or install:

sudo apt install gnome-themes-extra


=== Large Cursor ===

'''System Settings → Mouse and Touchpad → Cursor'''

Select a larger cursor size (32px or 48px).

Or via command line:

gsettings set org.cinnamon.desktop.interface cursor-size 32


=== Desktop Zoom (Magnifier) ===

'''System Settings → Accessibility → Zoom'''

Enable desktop magnification. Typical shortcut: Alt + Super + 8


=== Font Smoothing ===

Ensure font smoothing is enabled for better readability:

gsettings set org.cinnamon.settings-daemon.plugins.xsettings antialiasing 'rgba'

gsettings set org.cinnamon.settings-daemon.plugins.xsettings hinting 'slight'

Options for antialiasing: none, grayscale, rgba Options for hinting: none, slight, medium, full

Recommended: rgba with slight hinting


== Troubleshooting ==


=== Console Font Issues ===

'''Problem''': setupcon says "not on the console"

'''Solution''': This is normal when running from GUI. The configuration is saved. Test by switching to TTY with Ctrl+Alt+F2.

'''Problem''': Font changes don't persist after reboot

'''Solution''': Verify /etc/default/console-setup has correct settings. Run sudo dpkg-reconfigure console-setup again.

'''Problem''': setfont gives "couldn't get file descriptor" error

'''Solution''': setfont only works from actual TTY console, not GUI terminal emulators.


=== GUI Font Issues ===

'''Problem''': Installed font doesn't appear in applications

'''Solution''':

# Rebuild font cache

fc-cache -fv

# Log out and back in

# Or restart the application

'''Problem''': Font looks pixelated or ugly

'''Solution''': Check antialiasing settings:

gsettings set org.cinnamon.settings-daemon.plugins.xsettings antialiasing 'rgba'

'''Problem''': Terminus not showing in Guake

'''Solution''': Install the TrueType version:

sudo apt install fonts-terminus

guake --quit

guake &


=== Theme CSS Issues ===

'''Problem''': CSS changes don't take effect

'''Solution''':

<ol style="list-style-type: decimal;">
<li><blockquote>Save the CSS file</blockquote></li>
<li><blockquote>Restart Cinnamon: Alt+F2 → r</blockquote></li>
<li><blockquote>Verify you're editing the active theme's CSS</blockquote></li>
<li><blockquote>Check for CSS syntax errors</blockquote></li></ol>

'''Problem''': Cinnamon crashes after CSS edit

'''Solution''': Boot to TTY (Ctrl+Alt+F2), restore backup:

cp ~/.themes/YOUR-THEME/cinnamon/cinnamon.css.backup ~/.themes/YOUR-THEME/cinnamon/cinnamon.css

Then switch back to GUI and restart Cinnamon.

'''Problem''': Can't find the right CSS selector

'''Solution''': Use Looking Glass:

<ol style="list-style-type: decimal;">
<li><blockquote>Alt+F2 → type lg → Enter</blockquote></li>
<li><blockquote>Go to Picker tab</blockquote></li>
<li><blockquote>Click UI element to see its classes</blockquote></li></ol>


=== Scaling Issues ===

'''Problem''': UI scaling makes everything blurry

'''Solution''': Some applications don't handle scaling well. Try:

<ul>
<li><blockquote>Integer scaling only (100%, 200%) instead of fractional</blockquote></li>
<li><blockquote>Increase font sizes instead of UI scaling</blockquote></li>
<li><blockquote>Update graphics drivers</blockquote></li></ul>

'''Problem''': Some applications ignore scaling

'''Solution''': Set scaling per-application (for X11 apps):

GDK_SCALE=2 application-name


== Quick Reference Commands ==


=== Console Fonts ===

# Configure interactively

sudo dpkg-reconfigure console-setup

# Apply changes

sudo setupcon

# List console fonts

ls /usr/share/consolefonts/

# Test font (from TTY only)

setfont /usr/share/consolefonts/Ter16x32n.psf.gz


=== GUI Fonts ===

# Install fonts

sudo apt install fonts-terminus fonts-firacode

# Rebuild font cache

fc-cache -fv

# List all fonts

fc-list

# Search for specific font

fc-list | grep -i terminus


=== Cinnamon Settings ===

# Open settings

cinnamon-settings

# Check current theme

gsettings get org.cinnamon.theme name

# Set theme

gsettings set org.cinnamon.theme name 'YOUR-THEME'

# UI scaling

gsettings set org.cinnamon.desktop.interface scaling-factor 2

# Restart Cinnamon

Alt+F2 → r


=== Theme Editing ===

# Copy theme

cp -r /usr/share/themes/THEME-NAME ~/.themes/

# Edit CSS

nano ~/.themes/THEME-NAME/cinnamon/cinnamon.css

# Backup CSS

cp ~/.themes/THEME-NAME/cinnamon/cinnamon.css{,.backup}

# Restore backup

cp ~/.themes/THEME-NAME/cinnamon/cinnamon.css{.backup,}


== Additional Resources ==

'''Official Cinnamon Documentation''': https://cinnamon-spices.linuxmint.com/

'''Cinnamon GitHub''': https://github.com/linuxmint/cinnamon

'''Theme Development Guide''': Available in Cinnamon documentation

'''Font Configuration''': man fonts-conf or man fc-cache

'''Console Setup''': man console-setup or man setupcon

'''Last Updated''': October 2025 
'''Compatible with''': Debian 13, Linux Mint 21+, any distribution using Cinnamon DE

=== Credits & Comments ===
This work was contributed by distro-nix on Debian User Forums on 2026-03-28 19:43:50

I welcome comments, suggestions or resources : dev@divsmart.com . or visit https://www.debianfirstaid.org for more resources.

[[index.php?title=Category:Fonts]]
[[index.php?title=Category:Themes]]
[[index.php?title=Category:Desktop]]
[[index.php?title=Category:Full Paper]]

Debian First Aid Kit

2026-03-12T02:39:20Z

Distro-nix: /* Debian First Aid Kit */

= Debian First Aid Kit =
'''All commands are verified on Debian 13.1 (Trixie) / 6.12.73+deb13-amd64 and'''

'''6.16.3+deb13-amd64 x64_64'''

Created : 2025-10-27 15:54:21

Last Updated : 2026-03-11 22:18:41 '''ID : 544000.5'''

This guide is preserved in its original form. For updated content and the forthcoming Debian book, visit www.debianfirstaid.org .

== Table of Contents ==

# Issues
# Package Management Issues
# Disk & Filesystem Issues
# Performance Issues
# Service & Application Errors
# Permission & Access Issues
# Hardware IssuesSystem Freezes & Crashes
# Boot Problems
# Network
# Quick Diagnostic Commands
# Useful Aliases & Shortcuts
# Tips for Effective Troubleshooting

----

== 1. System Freezes & Crashes ==

=== Check System Logs ===
<code># View logs from previous boot (after freeze/crash)</code>
<code>journalctl -b -1</code>

<code># List all available boots</code>
<code>journalctl --list-boots</code>

<code># Show only kernel messages from previous boot</code>
<code>journalctl -b -1 -k</code>

<code># Show errors and critical messages only</code>
<code>journalctl -b -1 -p err</code>

<code># Save logs to file for analysis</code>
<code>journalctl -b -1 > ~/crash-log.txt</code>

=== Common Freeze Causes to Look For ===

* '''Kernel panics''': Search for "kernel panic" or "Oops"
* '''Out of Memory (OOM)''': Search for "Out of memory" or "oom-killer"
* '''Hardware errors''': Look for "MCE" (Machine Check Exception) or "hardware error"
* '''Driver issues''': Check for module/driver failures
* '''Overheating''': Check system temperatures

=== Check System Resources ===
<code># View memory usage</code>
<code>free -h</code>

<code># Check disk space</code>
<code>df -h</code>

<code># Monitor system resources in real-time</code>
<code>htop</code>
<code># or</code>
<code>top</code>
(I prefer btop for better presentation)
You would need to install it. sudo apt install btop

<code># Check for disk errors</code>
<code>sudo dmesg | grep -i error</code>
<code>These are permanent errors due to incomplete/buggy ACPI tables in the BIOS, but they are harmless :</code>
<code>0.686554] ACPI Error: No handler for Region [ECRM] (00000000201accc4) [EmbeddedControl] (20250404/evregion-131)</code>
<code>0.686577] ACPI Error: Region EmbeddedControl (ID=3) has no handler (20250404/exfldio-261)</code>
<code>0.686594] ACPI Error: Aborting method \_SB.GPIO._EVT due to previous error (AE_NOT_EXIST) (20250404/psparse-529)</code>

== 2. Boot Problems ==

=== Check Boot Process ===
<code># View systemd boot analysis</code>
<code>systemd-analyze blame</code>

<code># See what failed during boot</code>
<code>systemctl --failed</code>

<code># Check specific service status</code>
<code>systemctl status <service-name> e.g NetworkManager.service</code>

=== Access Recovery Mode ===

# Reboot and hold <code>Shift</code> to access GRUB menu (depending on your grub timing settings)
# Select "Advanced options"
# Choose recovery mode
# Select "root" for root shell access

=== Common Boot Fixes ===
<code># Repair filesystem errors</code>
<code>Once you identify a device with lsblk</code>
<code>sudo fsck /dev/sdXN</code>

<code># Reinstall GRUB bootloader</code>
<code>sudo grub-install /dev/sdX</code>
<code>sudo update-grub</code>

<code># Check fstab for mount errors</code>
<code>cat /etc/fstab</code>

== 3. Network Issues ==

=== Diagnose Network Connection ===
<code># Check network interfaces</code>
<code>ip addr show</code>

<code># Test connectivity</code>
<code>ping -c 4 8.8.8.8</code>
<code>or</code>
<code>ping -c 6 2a00:1450:4007:809::200e</code>

<code># Check DNS resolution</code>
<code>nslookup google.com</code>

<code># View routing table</code>
<code>ip route show</code>

<code># Check active connections</code>
<code>ss -tuln</code>

=== Restart Network Service ===
<code># For systems with NetworkManager</code>
<code>sudo systemctl restart NetworkManager</code>

<code># For systems with networking service</code>
<code>sudo systemctl restart networking</code>

<code># Bring interface down and up</code>
<code>sudo ip link set eth0 down</code>
<code>sudo ip link set eth0 up</code>

<code>If you need to prove to your server host something that is beyond your control, you can always get out the big guns with MTR.</code>

= <code>MTR (It’s Traceroute on Steroids)</code> =

== What is MTR? ==
MTR combines the functionality of <code>ping</code> and <code>traceroute</code> into a single real-time network diagnostic tool. It continuously monitors the path between your system and a destination, providing detailed statistics about latency and packet loss at each hop.

== Installation ==
<code>sudo apt install mtr</code>

== Basic Usage ==
<code># Basic MTR (interactive mode)</code>
<code>mtr google.com</code>

<code># Report mode (run 10 cycles and exit)</code>
<code>mtr --report google.com</code>

<code># Specify number of pings</code>
<code>mtr --report-cycles 50 google.com</code>

<code># Use TCP instead of ICMP</code>
<code>mtr --tcp google.com</code>

<code># Use UDP</code>
<code>mtr --udp google.com</code>

<code># No DNS resolution (faster, shows IPs only)</code>
<code>mtr --no-dns google.com</code>

<code># Show both hostnames and IPs</code>
<code>mtr --show-ips google.com</code>

== Understanding MTR Output ==

=== Sample Output ===
<code>HOST: hostname Loss% Snt Last Avg Best Wrst StDev</code>
<code>1.|-- 192.168.1.1 0.0% 10 1.2 1.5 1.0 2.3 0.4</code>
<code>2.|-- 10.0.0.1 0.0% 10 8.5 9.2 7.8 12.1 1.3</code>
<code>3.|-- isp-gateway.net 0.0% 10 15.3 16.1 14.2 19.8 1.8</code>
<code>4.|-- ??? 100.0% 10 0.0 0.0 0.0 0.0 0.0</code>
<code>5.|-- google.com 0.0% 10 25.4 26.8 24.1 32.5 2.4</code>

=== Column Meanings ===

* '''HOST''': Hostname or IP address of each hop in the route
* '''Loss%''': Percentage of packets lost at this hop
* '''Snt''': Number of packets sent to this hop
* '''Last''': Latency of the most recent packet (milliseconds)
* '''Avg''': Average latency across all packets (milliseconds)
* '''Best''': Lowest latency recorded (milliseconds)
* '''Wrst''': Highest latency recorded (milliseconds)
* '''StDev''': Standard deviation - measures latency consistency (lower is better)

== Interpreting Results ==

=== Healthy Network ===

* '''Loss% = 0%''' on all hops
* '''Stable latency''' (low StDev values)
* '''Gradual latency increase''' as hop count increases
* '''Consistent response times'''

=== Problem Indicators ===

==== 1. High Packet Loss at Specific Hop ====
<code>5.|-- problem-router.net 25.0% 10 45.3 48.2 42.1 65.8 8.4</code>
'''Analysis:'''

* If loss continues to destination: Real problem at this router
* If loss only at this hop but NOT beyond: Router may be rate-limiting ICMP (false positive, not a real problem)

'''Rule of thumb:''' If packet loss appears at hop N but hops N+1, N+2, etc. show 0% loss, it's usually just ICMP rate limiting.

==== 2. High Latency at Specific Hop ====
<code>3.|-- slow-link.net 0.0% 10 150.3 155.2 148.1 165.8 5.4</code>
'''Indicates:'''

* Network bottleneck or congested link
* Geographical distance (intercontinental hops)
* Slow routing equipment

==== 3. No Response (???) ====
<code>4.|-- ??? 100.0% 10 0.0 0.0 0.0 0.0 0.0</code>
'''Possible causes:'''

* Router configured to not respond to ICMP/traceroute packets
* Firewall blocking diagnostic packets
* '''Not necessarily a problem''' if later hops respond normally

==== 4. High Jitter (StDev) ====
<code>6.|-- unstable.net 0.0% 10 35.3 52.8 28.1 95.2 24.7</code>
'''Indicates:'''

* Inconsistent latency (high StDev of 24.7ms)
* Network congestion or instability
* Poor for real-time applications (VoIP, gaming, video calls)

==== 5. Sudden Latency Spike ====
<code>1.|-- 192.168.1.1 0.0% 10 1.2 1.5 1.0 2.3 0.4</code>
<code>2.|-- 10.0.0.1 0.0% 10 8.5 9.2 7.8 12.1 1.3</code>
<code>3.|-- problematic-hop.net 0.0% 10 180.5 185.2 178.1 195.8 6.4</code>
<code>4.|-- next-hop.net 0.0% 10 182.3 187.8 180.5 198.2 6.8</code>
'''Problem identified:''' Hop 3 introduces ~170ms of latency (jump from 9ms to 180ms)

== Advanced Usage ==

=== Report Mode with Different Output Formats ===
<code># CSV format for logging and analysis</code>
<code>mtr --report --csv google.com > network-report.csv</code>

<code># JSON output for parsing</code>
<code>mtr --report --json google.com</code>

<code># XML format</code>
<code>mtr --report --xml google.com</code>

<code># Wide report (no abbreviations)</code>
<code>mtr --report-wide google.com</code>

=== Protocol Selection ===
<code># Use ICMP (default, requires no special permissions)</code>
<code>mtr google.com</code>

<code># Use UDP (alternative to ICMP)</code>
<code>mtr --udp google.com</code>

<code># Use TCP (useful for firewall testing)</code>
<code>mtr --tcp google.com</code>

<code># Test specific TCP port</code>
<code>sudo mtr --tcp --port 443 google.com</code>
<code>sudo mtr --tcp --port 22 remote-server.com</code>

=== Timing and Duration ===
<code># Specify interval between pings (default 1 second)</code>
<code>mtr --interval 0.5 google.com</code>

<code># Extended test with 100 cycles</code>
<code>mtr --report-cycles 100 google.com</code>

<code># Continuous monitoring (Ctrl+C to stop)</code>
<code>mtr google.com</code>

<code># Quick 10-cycle report</code>
<code>mtr --report-cycles 10 google.com</code>

=== Advanced Options ===
<code># Show Autonomous System (AS) numbers</code>
<code>mtr --aslookup google.com</code>

<code># Set maximum number of hops</code>
<code>mtr --max-ttl 20 google.com</code>

<code># Set packet size</code>
<code>mtr --psize 1000 google.com</code>

<code># Show both IP and hostname</code>
<code>mtr --show-ips google.com</code>

<code># Specify source address (multiple network interfaces)</code>
<code>mtr --address 192.168.1.100 google.com</code>

<code># IPv4 only</code>
<code>mtr -4 google.com</code>

<code># IPv6 only</code>
<code>mtr -6 google.com</code>

== Interactive Mode Commands ==
When running MTR in interactive mode (just <code>mtr hostname</code>), use these keys:
{| class="wikitable"
!Key
!Function
|-
|'''h'''
|Display help
|-
|'''d'''
|Toggle display mode (cycle through different views)
|-
|'''n'''
|Toggle between hostnames and IP addresses
|-
|'''r'''
|Reset all statistics
|-
|'''p'''
|Pause/unpause the display
|-
|'''q'''
|Quit MTR
|-
|'''u'''
|Switch between ICMP, UDP, and TCP modes
|-
|'''y'''
|Switch between IPv4 and IPv6
|-
|'''o'''
|Toggle field display options
|-
|'''j'''
|Toggle latency display
|}

== Practical Troubleshooting Scenarios ==

=== Scenario 1: Diagnosing Slow Website ===
<code># Run extended test to get accurate statistics</code>
<code>mtr --report-cycles 100 --no-dns example.com</code>

<code># Look for:</code>
<code># - High average latency at specific hops</code>
<code># - Packet loss at destination</code>
<code># - High StDev values (jitter)</code>

=== Scenario 2: Testing if Firewall Blocks SSH ===
<code># Test SSH port (22) connectivity</code>
<code>sudo mtr --tcp --port 22 --report-cycles 50 remote-server.com</code>

<code># If last hop shows 100% loss but earlier hops are fine:</code>
<code># - Port 22 might be filtered</code>
<code># - Try standard ICMP test for comparison</code>

=== Scenario 3: ISP Performance Issues ===
<code># Test path to reliable external server</code>
<code>mtr --report-cycles 100 8.8.8.8</code>

<code># Compare with another DNS server</code>
<code>mtr --report-cycles 100 1.1.1.1</code>

<code># If issues appear in first 3-4 hops: likely ISP problem</code>
<code># If issues appear later: problem is beyond your ISP</code>

=== Scenario 4: VPN Troubleshooting ===
<code># Test before connecting to VPN</code>
<code>mtr --report-cycles 50 --no-dns google.com > before-vpn.txt</code>

<code># Test after connecting to VPN</code>
<code>mtr --report-cycles 50 --no-dns google.com > after-vpn.txt</code>

<code># Compare the two files to see VPN impact</code>
<code>diff before-vpn.txt after-vpn.txt</code>

=== Scenario 5: Gaming/Streaming Performance ===
<code># Test for jitter (important for real-time applications)</code>
<code>mtr --report-cycles 200 game-server.com</code>

<code># Look for:</code>
<code># - Low average latency (< 50ms for gaming)</code>
<code># - Low StDev (< 5ms preferred)</code>
<code># - Zero packet loss</code>

=== Scenario 6: Intermittent Connectivity ===
<code># Long-running test to catch intermittent issues</code>
<code>mtr --report-cycles 500 --interval 1 target.com > long-test.txt</code>

<code># Monitor in real-time for several minutes</code>
<code>mtr target.com</code>
<code># Watch for sudden spikes in Loss% or latency</code>

== Continuous Monitoring ==

=== Log Network Performance Over Time ===
<code># Create timestamped reports every hour</code>
<code>while true; do</code>
<code>timestamp=$(date +%Y%m%d-%H%M%S)</code>
<code>mtr --report --report-cycles 50 google.com > "mtr-$timestamp.txt"</code>
<code>sleep 3600</code>
<code>done</code>

=== Monitor Multiple Destinations ===
<code># Create a simple monitoring script</code>
<code>#!/bin/bash</code>
<code>echo "=== MTR Report $(date) ===" > daily-network-report.txt</code>
<code>echo "" >> daily-network-report.txt</code>

<code>echo "Google DNS:" >> daily-network-report.txt</code>
<code>mtr --report --report-cycles 50 --no-dns 8.8.8.8 >> daily-network-report.txt</code>
<code>echo "" >> daily-network-report.txt</code>

<code>echo "Cloudflare DNS:" >> daily-network-report.txt</code>
<code>mtr --report --report-cycles 50 --no-dns 1.1.1.1 >> daily-network-report.txt</code>
<code>echo "" >> daily-network-report.txt</code>

<code>echo "Your Server:" >> daily-network-report.txt</code>
<code>mtr --report --report-cycles 50 your-server.com >> daily-network-report.txt</code>

== Useful Aliases for .bashrc ==
<code># Quick network path analysis</code>
<code>alias mtrreport='mtr --report --report-cycles 50 --no-dns'</code>

<code># Monitor connection to Google DNS</code>
<code>alias netcheck='mtr --report-cycles 20 8.8.8.8'</code>

<code># Extended network test</code>
<code>alias mtrlong='mtr --report-cycles 100'</code>

<code># TCP port 443 test (HTTPS)</code>
<code>alias mtrhttps='sudo mtr --tcp --port 443 --report-cycles 30'</code>

<code># Quick comparison of major DNS providers</code>
<code>alias dnstest='echo "Google:" && mtr --report-cycles 20 8.8.8.8 && echo -e "\nCloudflare:" && mtr --report-cycles 20 1.1.1.1'</code>
After adding to <code>~/.bashrc</code>:
<code>source ~/.bashrc</code>

== Troubleshooting Tips ==

=== 1. Permission Issues ===
If you get permission errors with TCP mode:
<code># Use sudo for TCP on privileged ports</code>
<code>sudo mtr --tcp --port 443 example.com</code>

<code># Or set capabilities (one-time setup)</code>
<code>sudo setcap cap_net_raw+ep /usr/bin/mtr-packet</code>

=== 2. False Positives ===
'''Common false positive:''' Packet loss at intermediate hops but NOT at the destination.

Example:
<code>3.|-- router.isp.net 20.0% 50 15.3 16.1 14.2 19.8 1.8</code>
<code>4.|-- next-hop.net 0.0% 50 18.5 19.2 17.8 22.1 1.3</code>
<code>5.|-- destination.com 0.0% 50 25.4 26.8 24.1 32.5 2.4</code>
'''This is OK!''' Hop 3 shows 20% loss, but hops 4 and 5 show 0% loss. The router at hop 3 is rate-limiting ICMP responses, but actual traffic flows normally.

=== 3. DNS Resolution Delays ===
If MTR seems slow to start:
<code># Skip DNS resolution for faster results</code>
<code>mtr --no-dns target.com</code>

<code># Resolve names afterward if needed</code>
<code>host 203.0.113.1</code>

=== 4. Comparing Results ===
<code># Run multiple tests and compare</code>
<code>mtr --report --report-cycles 50 example.com > test1.txt</code>
<code>sleep 60</code>
<code>mtr --report --report-cycles 50 example.com > test2.txt</code>
<code>diff test1.txt test2.txt</code>

== When to Use MTR vs Other Tools ==
{| class="wikitable"
!Tool
!Best For
!Limitations
|-
|'''MTR'''
|Continuous monitoring, identifying problem hops, detailed statistics
|Requires installation
|-
|'''ping'''
|Quick connectivity test, simple latency check
|Only tests endpoint
|-
|'''traceroute'''
|One-time path discovery
|No continuous monitoring
|-
|'''ss/netstat'''
|Local connection status
|Doesn't test remote paths
|}

== Best Practices ==

# '''Run enough cycles''': Use at least 50-100 cycles for accurate statistics
# '''Use --no-dns''': Faster and avoids DNS resolution issues during testing
# '''Check multiple times''': Network conditions vary; test at different times
# '''Compare protocols''': Try ICMP, UDP, and TCP if one shows issues
# '''Document findings''': Save reports with timestamps for trend analysis
# '''Test known-good hosts''': Use 8.8.8.8 or 1.1.1.1 to verify your network first
# '''Be patient''': Let MTR run for at least 30-60 seconds before drawing conclusions

== Reading Between the Lines ==

=== Good Network Health Example ===
<code>HOST: hostname Loss% Snt Last Avg Best Wrst StDev</code>
<code>1.|-- 192.168.1.1 0.0% 50 1.1 1.2 0.9 2.1 0.2</code>
<code>2.|-- 10.0.0.1 0.0% 50 8.2 8.5 7.5 10.2 0.5</code>
<code>3.|-- isp-gateway.net 0.0% 50 15.1 15.5 14.0 18.3 0.8</code>
<code>4.|-- google.com 0.0% 50 24.8 25.2 23.5 28.1 1.1</code>
✅ No packet loss, consistent latency, low jitter

=== Problem Network Example ===
<code>HOST: hostname Loss% Snt Last Avg Best Wrst StDev</code>
<code>1.|-- 192.168.1.1 0.0% 50 1.2 1.5 1.0 2.3 0.3</code>
<code>2.|-- 10.0.0.1 5.0% 50 45.3 52.8 8.1 245.2 45.7</code>
<code>3.|-- ??? 100.0% 50 0.0 0.0 0.0 0.0 0.0</code>
<code>4.|-- destination.com 15.0% 50 95.4 125.8 48.2 385.5 78.2</code>
❌ Packet loss at hop 2 and destination, high jitter, very high worst-case latency

== Summary ==
MTR is your Swiss Army knife for network diagnostics. Key takeaways:

* Use <code>--report-cycles 50+</code> for reliable data
* Watch for packet loss at the '''destination''' (intermediate losses may be false positives)
* High '''StDev''' indicates unstable connection
* High '''Avg''' latency shows slow links
* Use <code>--no-dns</code> for faster results
* Compare '''ICMP''', '''UDP''', and '''TCP''' modes if issues appear
* Test at different times of day for comprehensive analysis

== 4. Package Management Issues ==

=== Fix Broken Packages ===
<code># Update package lists</code>
<code>sudo apt update</code>

<code># Fix broken dependencies</code>
<code>sudo apt --fix-broken install</code>

<code># Reconfigure packages</code>
<code>sudo dpkg --configure -a</code>
<code>(if no output, there is nothing to do)</code>

<code># Clean package cache</code>
<code>sudo apt clean</code>
<code>sudo apt autoclean</code>

<code># Remove unused packages</code>
<code>sudo apt autoremove</code>

=== Handle Held or Locked Packages ===
<code># If apt is locked, find the process</code>
<code>sudo lsof /var/lib/dpkg/lock-frontend</code>

<code># Force remove lock (use carefully)</code>
<code>sudo rm /var/lib/dpkg/lock-frontend</code>
<code>sudo rm /var/lib/apt/lists/lock</code>

<code># Reconfigure dpkg</code>
<code>sudo dpkg --configure -a</code>

== 5. Disk & Filesystem Issues ==

=== Check Disk Health ===
<code># Check disk space</code>
<code>df -h</code>

<code># Check inode usage</code>
<code>df -i</code>

<code># View disk I/O statistics</code>
<code>iostat -x 1</code>
<code>(Make sure you have sysstat which includes useful performance monitoring tools other than iostat - disk I/O statistics</code>

* <code>mpstat</code> - CPU statistics

* <code>sar</code> - system activity reporter

* <code>pidstat</code> - process statistics

* <code>cifsiostat</code> - CIFS statistics

<code>''# Show stats in MB instead of KB'' iostat -xm 2</code>

<code>''# Monitor specific device'' iostat -x sda 1</code>
<code># Check for disk errors in dmesg</code>
<code>sudo dmesg | grep -i "error\|fail"</code>

<code># SMART disk health (if smartmontools installed)</code>
<code>sudo smartctl -a /dev/sda</code>

=== Repair Filesystem ===
<code># Unmount the partition first</code>
<code>sudo umount /dev/sdXN</code>

<code># Run filesystem check</code>
<code>sudo fsck /dev/sdXN</code>

<code># For ext4 specifically</code>
<code>sudo e2fsck -f /dev/sdXN</code>

== 6. Performance Issues ==

=== Identify Resource Hogs ===
<code># CPU usage by process</code>
<code>top -o %CPU</code>

<code># Memory usage by process</code>
<code>top -o %MEM</code>

<code># Disk usage by directory</code>
<code>du -sh /* | sort -h</code>

<code># Find large files</code>
<code>find / -type f -size +100M 2>/dev/null</code>

<code># Check running processes</code>
<code>ps aux --sort=-%mem | head -20</code>

=== System Temperature Monitoring ===
<code># Install sensors (if not installed)</code>
<code>sudo apt install lm-sensors</code>
<code>sudo sensors-detect</code>

<code># View temperatures</code>
<code>sensors</code>

<code># Real-time temperature monitoring</code>
<code>watch -n 2 sensors</code>
<code>''I have it as an alias in ~/.bashrc''</code>
<code>''Go to 11. Useful Aliases & Shortcuts''</code>

== 7. Service & Application Errors ==

=== Debug Service Problems ===
<code># Check service status</code>
<code>sudo systemctl status service-name</code>

<code># View service logs</code>
<code>sudo journalctl -u service-name</code>

<code># Restart a service</code>
<code>sudo systemctl restart service-name</code>

<code># Enable service at boot</code>
<code>sudo systemctl enable service-name</code>

<code># View recent service failures</code>
<code>journalctl -p err -b</code>

=== Application Crash Investigation ===
<code># Check for core dumps</code>
<code>ls -lh /var/crash/</code>

<code># View application-specific logs</code>
<code>ls /var/log/</code>

<code># Check syslog for application errors</code>
<code>sudo tail -f /var/log/syslog</code>

== 8. Permission & Access Issues ==

=== Fix Common Permission Problems ===
<code># Check file ownership</code>
<code>ls -l /path/to/file</code>

<code># Change ownership</code>
<code>sudo chown michael:michael /path/to/file</code>
user:group

<code># Change permissions</code>
<code>sudo chmod 644 /path/to/file</code>

<code># Recursively fix permissions</code>
<code>sudo chown -R user:group /path/to/directory</code>

=== User & Authentication Issues ===
<code># Check user information</code>
<code>id username</code>

<code># View user login history</code>
<code>last -a</code>

<code># Check failed login attempts</code>
<code>sudo journalctl | grep "authentication failure"</code>

<code># Reset user password</code>
<code>sudo passwd username</code>

== 9. Hardware Issues ==

=== Identify Hardware ===
<code># List all hardware</code>
<code>sudo lshw -short</code>
(May not be installed by default)
sudo apt install lshw

<code># PCI devices</code>
<code>lspci -v</code>

<code># USB devices</code>
<code>lsusb -v</code>

<code># CPU information</code>
<code>lscpu</code>

<code># Memory information</code>
<code>sudo dmidecode --type memory</code>

=== Check Hardware Errors ===
<code># Kernel ring buffer (hardware messages)</code>
<code>dmesg | less</code>
<code>(If no output, good, no errors)</code>
<code>q to quit</code>

<code># Search for specific hardware issues</code>
<code>dmesg | grep -i "error\|fail\|warn"</code>

<code># Check for USB issues</code>
<code>dmesg | grep -i usb</code>

== 10. Quick Diagnostic Commands ==

=== System Information at a Glance ===
<code># Uptime and load average</code>
<code>uptime</code>

<code># Kernel version</code>
<code>uname -r</code>

<code># Debian version</code>
<code>cat /etc/debian_version</code>

<code># System summary</code>
<code>sudo inxi -Fxz</code>

=== Emergency Toolkit ===
<code># Create a diagnostic report</code>
<code>sudo journalctl -b > ~/system-report.txt</code>
<code>dmesg >> ~/system-report.txt</code>
<code>systemctl --failed >> ~/system-report.txt</code>
<code>df -h >> ~/system-report.txt</code>
<code>free -h >> ~/system-report.txt</code>

<code># Watch logs in real-time</code>
<code>sudo journalctl -f</code>

<code># Monitor system resources continuously</code>
<code>watch -n 1 'free -h && df -h'</code>

== 11. Useful Aliases & Shortcuts ==
Add these to your <code>~/.bashrc</code> for quick access to common troubleshooting commands:
<code># Monitor system temperatures in real-time</code>
<code>alias temps="watch -n 2 'for i in /sys/class/hwmon/hwmon*/; do echo -n \"\$(cat \${i}name): \"; cat \${i}temp*_input 2>/dev/null | while read temp; do echo \"scale=1; \$temp/1000\" | bc; done | tr \"\n\" \" \"; echo \"°C\"; done'"</code>
<code>''or run the watch command in the shell without the opening and closing double quotes.''</code>

<code># Quick system status</code>
<code>alias sysstat='echo "=== CPU ===" && uptime && echo -e "\n=== Memory ===" && free -h && echo -e "\n=== Disk ===" && df -h && echo -e "\n=== Top Processes ===" && ps aux --sort=-%mem | head -10'</code>
<code>(It’s a messy mayout, but I’m terrible with awk. Feel free to improve the layoput for me)</code>

<code># View last boot logs</code>
<code>alias lastboot='journalctl -b -1'</code>

<code># Check failed services</code>
<code>alias failedservices='systemctl --failed'</code>

<code># Monitor logs in real-time</code>
<code>alias watchlog='sudo journalctl -f'</code>

<code># Quick network status</code>
<code>alias netstat='ip addr show && echo -e "\n=== Routes ===" && ip route show'</code>
After adding these, run:
<code>source ~/.bashrc</code>
----

== Tips for Your Troubleshooting ==

# '''First check logs''': <code>journalctl</code> and <code>dmesg</code> are your best friends
# '''Work through the sections''': Change one thing at a time
# '''Document changes''': Keep notes on what you've tried
# '''Search for error messages''': Copy exact error messages into search engines or AI
# '''Check recent changes''': What you did before it happened? Install something, update packages, kernel?
# '''Make backups''': Before major changes, backup important data
# '''Use verbose mode''': Add <code>-v</code> or <code>-vv</code> flags to commands for more detail
# '''Check forums''': Debian forum, Reddit, Stack Exchange, and mailing lists

----'''Remember''': If in doubt, search for the specific error message along with "Debian" and the version number. e.g. Debian 13 or point release if needed, Debian 13.1

This work was contributed by distro-nix on Debian User Forums on 2025-10-27 23:38:22

I welcome comments, suggestions or resources : dev@divsmart.com .

[[index.php?title=Category:Troubleshooting]]
[[index.php?title=Category:Guides]]
[[index.php?title=Category:Administration]]
[[index.php?title=Category:Guides]]
[[index.php?title=Category:Full Paper]]

Debian First Aid Kit

2026-03-10T11:27:00Z

Distro-nix: /* Debian First Aid Kit */

= Debian First Aid Kit =
'''All commands are verified on Debian 13.1 (Trixie) / 6.16.3+deb13-amd64 x64_64'''

Created : 2025-10-27 15:54:21

Last Updated : 2026-03-10 07:10:53 '''ID : 544000.4'''

== Table of Contents ==

# Issues
# Package Management Issues
# Disk & Filesystem Issues
# Performance Issues
# Service & Application Errors
# Permission & Access Issues
# Hardware IssuesSystem Freezes & Crashes
# Boot Problems
# Network
# Quick Diagnostic Commands
# Useful Aliases & Shortcuts
# Tips for Effective Troubleshooting

----

== 1. System Freezes & Crashes ==

=== Check System Logs ===
<code># View logs from previous boot (after freeze/crash)</code>
<code>journalctl -b -1</code>

<code># List all available boots</code>
<code>journalctl --list-boots</code>

<code># Show only kernel messages from previous boot</code>
<code>journalctl -b -1 -k</code>

<code># Show errors and critical messages only</code>
<code>journalctl -b -1 -p err</code>

<code># Save logs to file for analysis</code>
<code>journalctl -b -1 > ~/crash-log.txt</code>

=== Common Freeze Causes to Look For ===

* '''Kernel panics''': Search for "kernel panic" or "Oops"
* '''Out of Memory (OOM)''': Search for "Out of memory" or "oom-killer"
* '''Hardware errors''': Look for "MCE" (Machine Check Exception) or "hardware error"
* '''Driver issues''': Check for module/driver failures
* '''Overheating''': Check system temperatures

=== Check System Resources ===
<code># View memory usage</code>
<code>free -h</code>

<code># Check disk space</code>
<code>df -h</code>

<code># Monitor system resources in real-time</code>
<code>htop</code>
<code># or</code>
<code>top</code>
(I prefer btop for better presentation)
You would need to install it. sudo apt install btop

<code># Check for disk errors</code>
<code>sudo dmesg | grep -i error</code>
<code>These are permanent errors due to incomplete/buggy ACPI tables in the BIOS, but they are harmless :</code>
<code>0.686554] ACPI Error: No handler for Region [ECRM] (00000000201accc4) [EmbeddedControl] (20250404/evregion-131)</code>
<code>0.686577] ACPI Error: Region EmbeddedControl (ID=3) has no handler (20250404/exfldio-261)</code>
<code>0.686594] ACPI Error: Aborting method \_SB.GPIO._EVT due to previous error (AE_NOT_EXIST) (20250404/psparse-529)</code>

== 2. Boot Problems ==

=== Check Boot Process ===
<code># View systemd boot analysis</code>
<code>systemd-analyze blame</code>

<code># See what failed during boot</code>
<code>systemctl --failed</code>

<code># Check specific service status</code>
<code>systemctl status <service-name> e.g NetworkManager.service</code>

=== Access Recovery Mode ===

# Reboot and hold <code>Shift</code> to access GRUB menu (depending on your grub timing settings)
# Select "Advanced options"
# Choose recovery mode
# Select "root" for root shell access

=== Common Boot Fixes ===
<code># Repair filesystem errors</code>
<code>Once you identify a device with lsblk</code>
<code>sudo fsck /dev/sdXN</code>

<code># Reinstall GRUB bootloader</code>
<code>sudo grub-install /dev/sdX</code>
<code>sudo update-grub</code>

<code># Check fstab for mount errors</code>
<code>cat /etc/fstab</code>

== 3. Network Issues ==

=== Diagnose Network Connection ===
<code># Check network interfaces</code>
<code>ip addr show</code>

<code># Test connectivity</code>
<code>ping -c 4 8.8.8.8</code>
<code>or</code>
<code>ping -c 6 2a00:1450:4007:809::200e</code>

<code># Check DNS resolution</code>
<code>nslookup google.com</code>

<code># View routing table</code>
<code>ip route show</code>

<code># Check active connections</code>
<code>ss -tuln</code>

=== Restart Network Service ===
<code># For systems with NetworkManager</code>
<code>sudo systemctl restart NetworkManager</code>

<code># For systems with networking service</code>
<code>sudo systemctl restart networking</code>

<code># Bring interface down and up</code>
<code>sudo ip link set eth0 down</code>
<code>sudo ip link set eth0 up</code>

<code>If you need to prove to your server host something that is beyond your control, you can always get out the big guns with MTR.</code>

= <code>MTR (It’s Traceroute on Steroids)</code> =

== What is MTR? ==
MTR combines the functionality of <code>ping</code> and <code>traceroute</code> into a single real-time network diagnostic tool. It continuously monitors the path between your system and a destination, providing detailed statistics about latency and packet loss at each hop.

== Installation ==
<code>sudo apt install mtr</code>

== Basic Usage ==
<code># Basic MTR (interactive mode)</code>
<code>mtr google.com</code>

<code># Report mode (run 10 cycles and exit)</code>
<code>mtr --report google.com</code>

<code># Specify number of pings</code>
<code>mtr --report-cycles 50 google.com</code>

<code># Use TCP instead of ICMP</code>
<code>mtr --tcp google.com</code>

<code># Use UDP</code>
<code>mtr --udp google.com</code>

<code># No DNS resolution (faster, shows IPs only)</code>
<code>mtr --no-dns google.com</code>

<code># Show both hostnames and IPs</code>
<code>mtr --show-ips google.com</code>

== Understanding MTR Output ==

=== Sample Output ===
<code>HOST: hostname Loss% Snt Last Avg Best Wrst StDev</code>
<code>1.|-- 192.168.1.1 0.0% 10 1.2 1.5 1.0 2.3 0.4</code>
<code>2.|-- 10.0.0.1 0.0% 10 8.5 9.2 7.8 12.1 1.3</code>
<code>3.|-- isp-gateway.net 0.0% 10 15.3 16.1 14.2 19.8 1.8</code>
<code>4.|-- ??? 100.0% 10 0.0 0.0 0.0 0.0 0.0</code>
<code>5.|-- google.com 0.0% 10 25.4 26.8 24.1 32.5 2.4</code>

=== Column Meanings ===

* '''HOST''': Hostname or IP address of each hop in the route
* '''Loss%''': Percentage of packets lost at this hop
* '''Snt''': Number of packets sent to this hop
* '''Last''': Latency of the most recent packet (milliseconds)
* '''Avg''': Average latency across all packets (milliseconds)
* '''Best''': Lowest latency recorded (milliseconds)
* '''Wrst''': Highest latency recorded (milliseconds)
* '''StDev''': Standard deviation - measures latency consistency (lower is better)

== Interpreting Results ==

=== Healthy Network ===

* '''Loss% = 0%''' on all hops
* '''Stable latency''' (low StDev values)
* '''Gradual latency increase''' as hop count increases
* '''Consistent response times'''

=== Problem Indicators ===

==== 1. High Packet Loss at Specific Hop ====
<code>5.|-- problem-router.net 25.0% 10 45.3 48.2 42.1 65.8 8.4</code>
'''Analysis:'''

* If loss continues to destination: Real problem at this router
* If loss only at this hop but NOT beyond: Router may be rate-limiting ICMP (false positive, not a real problem)

'''Rule of thumb:''' If packet loss appears at hop N but hops N+1, N+2, etc. show 0% loss, it's usually just ICMP rate limiting.

==== 2. High Latency at Specific Hop ====
<code>3.|-- slow-link.net 0.0% 10 150.3 155.2 148.1 165.8 5.4</code>
'''Indicates:'''

* Network bottleneck or congested link
* Geographical distance (intercontinental hops)
* Slow routing equipment

==== 3. No Response (???) ====
<code>4.|-- ??? 100.0% 10 0.0 0.0 0.0 0.0 0.0</code>
'''Possible causes:'''

* Router configured to not respond to ICMP/traceroute packets
* Firewall blocking diagnostic packets
* '''Not necessarily a problem''' if later hops respond normally

==== 4. High Jitter (StDev) ====
<code>6.|-- unstable.net 0.0% 10 35.3 52.8 28.1 95.2 24.7</code>
'''Indicates:'''

* Inconsistent latency (high StDev of 24.7ms)
* Network congestion or instability
* Poor for real-time applications (VoIP, gaming, video calls)

==== 5. Sudden Latency Spike ====
<code>1.|-- 192.168.1.1 0.0% 10 1.2 1.5 1.0 2.3 0.4</code>
<code>2.|-- 10.0.0.1 0.0% 10 8.5 9.2 7.8 12.1 1.3</code>
<code>3.|-- problematic-hop.net 0.0% 10 180.5 185.2 178.1 195.8 6.4</code>
<code>4.|-- next-hop.net 0.0% 10 182.3 187.8 180.5 198.2 6.8</code>
'''Problem identified:''' Hop 3 introduces ~170ms of latency (jump from 9ms to 180ms)

== Advanced Usage ==

=== Report Mode with Different Output Formats ===
<code># CSV format for logging and analysis</code>
<code>mtr --report --csv google.com > network-report.csv</code>

<code># JSON output for parsing</code>
<code>mtr --report --json google.com</code>

<code># XML format</code>
<code>mtr --report --xml google.com</code>

<code># Wide report (no abbreviations)</code>
<code>mtr --report-wide google.com</code>

=== Protocol Selection ===
<code># Use ICMP (default, requires no special permissions)</code>
<code>mtr google.com</code>

<code># Use UDP (alternative to ICMP)</code>
<code>mtr --udp google.com</code>

<code># Use TCP (useful for firewall testing)</code>
<code>mtr --tcp google.com</code>

<code># Test specific TCP port</code>
<code>sudo mtr --tcp --port 443 google.com</code>
<code>sudo mtr --tcp --port 22 remote-server.com</code>

=== Timing and Duration ===
<code># Specify interval between pings (default 1 second)</code>
<code>mtr --interval 0.5 google.com</code>

<code># Extended test with 100 cycles</code>
<code>mtr --report-cycles 100 google.com</code>

<code># Continuous monitoring (Ctrl+C to stop)</code>
<code>mtr google.com</code>

<code># Quick 10-cycle report</code>
<code>mtr --report-cycles 10 google.com</code>

=== Advanced Options ===
<code># Show Autonomous System (AS) numbers</code>
<code>mtr --aslookup google.com</code>

<code># Set maximum number of hops</code>
<code>mtr --max-ttl 20 google.com</code>

<code># Set packet size</code>
<code>mtr --psize 1000 google.com</code>

<code># Show both IP and hostname</code>
<code>mtr --show-ips google.com</code>

<code># Specify source address (multiple network interfaces)</code>
<code>mtr --address 192.168.1.100 google.com</code>

<code># IPv4 only</code>
<code>mtr -4 google.com</code>

<code># IPv6 only</code>
<code>mtr -6 google.com</code>

== Interactive Mode Commands ==
When running MTR in interactive mode (just <code>mtr hostname</code>), use these keys:
{| class="wikitable"
!Key
!Function
|-
|'''h'''
|Display help
|-
|'''d'''
|Toggle display mode (cycle through different views)
|-
|'''n'''
|Toggle between hostnames and IP addresses
|-
|'''r'''
|Reset all statistics
|-
|'''p'''
|Pause/unpause the display
|-
|'''q'''
|Quit MTR
|-
|'''u'''
|Switch between ICMP, UDP, and TCP modes
|-
|'''y'''
|Switch between IPv4 and IPv6
|-
|'''o'''
|Toggle field display options
|-
|'''j'''
|Toggle latency display
|}

== Practical Troubleshooting Scenarios ==

=== Scenario 1: Diagnosing Slow Website ===
<code># Run extended test to get accurate statistics</code>
<code>mtr --report-cycles 100 --no-dns example.com</code>

<code># Look for:</code>
<code># - High average latency at specific hops</code>
<code># - Packet loss at destination</code>
<code># - High StDev values (jitter)</code>

=== Scenario 2: Testing if Firewall Blocks SSH ===
<code># Test SSH port (22) connectivity</code>
<code>sudo mtr --tcp --port 22 --report-cycles 50 remote-server.com</code>

<code># If last hop shows 100% loss but earlier hops are fine:</code>
<code># - Port 22 might be filtered</code>
<code># - Try standard ICMP test for comparison</code>

=== Scenario 3: ISP Performance Issues ===
<code># Test path to reliable external server</code>
<code>mtr --report-cycles 100 8.8.8.8</code>

<code># Compare with another DNS server</code>
<code>mtr --report-cycles 100 1.1.1.1</code>

<code># If issues appear in first 3-4 hops: likely ISP problem</code>
<code># If issues appear later: problem is beyond your ISP</code>

=== Scenario 4: VPN Troubleshooting ===
<code># Test before connecting to VPN</code>
<code>mtr --report-cycles 50 --no-dns google.com > before-vpn.txt</code>

<code># Test after connecting to VPN</code>
<code>mtr --report-cycles 50 --no-dns google.com > after-vpn.txt</code>

<code># Compare the two files to see VPN impact</code>
<code>diff before-vpn.txt after-vpn.txt</code>

=== Scenario 5: Gaming/Streaming Performance ===
<code># Test for jitter (important for real-time applications)</code>
<code>mtr --report-cycles 200 game-server.com</code>

<code># Look for:</code>
<code># - Low average latency (< 50ms for gaming)</code>
<code># - Low StDev (< 5ms preferred)</code>
<code># - Zero packet loss</code>

=== Scenario 6: Intermittent Connectivity ===
<code># Long-running test to catch intermittent issues</code>
<code>mtr --report-cycles 500 --interval 1 target.com > long-test.txt</code>

<code># Monitor in real-time for several minutes</code>
<code>mtr target.com</code>
<code># Watch for sudden spikes in Loss% or latency</code>

== Continuous Monitoring ==

=== Log Network Performance Over Time ===
<code># Create timestamped reports every hour</code>
<code>while true; do</code>
<code>timestamp=$(date +%Y%m%d-%H%M%S)</code>
<code>mtr --report --report-cycles 50 google.com > "mtr-$timestamp.txt"</code>
<code>sleep 3600</code>
<code>done</code>

=== Monitor Multiple Destinations ===
<code># Create a simple monitoring script</code>
<code>#!/bin/bash</code>
<code>echo "=== MTR Report $(date) ===" > daily-network-report.txt</code>
<code>echo "" >> daily-network-report.txt</code>

<code>echo "Google DNS:" >> daily-network-report.txt</code>
<code>mtr --report --report-cycles 50 --no-dns 8.8.8.8 >> daily-network-report.txt</code>
<code>echo "" >> daily-network-report.txt</code>

<code>echo "Cloudflare DNS:" >> daily-network-report.txt</code>
<code>mtr --report --report-cycles 50 --no-dns 1.1.1.1 >> daily-network-report.txt</code>
<code>echo "" >> daily-network-report.txt</code>

<code>echo "Your Server:" >> daily-network-report.txt</code>
<code>mtr --report --report-cycles 50 your-server.com >> daily-network-report.txt</code>

== Useful Aliases for .bashrc ==
<code># Quick network path analysis</code>
<code>alias mtrreport='mtr --report --report-cycles 50 --no-dns'</code>

<code># Monitor connection to Google DNS</code>
<code>alias netcheck='mtr --report-cycles 20 8.8.8.8'</code>

<code># Extended network test</code>
<code>alias mtrlong='mtr --report-cycles 100'</code>

<code># TCP port 443 test (HTTPS)</code>
<code>alias mtrhttps='sudo mtr --tcp --port 443 --report-cycles 30'</code>

<code># Quick comparison of major DNS providers</code>
<code>alias dnstest='echo "Google:" && mtr --report-cycles 20 8.8.8.8 && echo -e "\nCloudflare:" && mtr --report-cycles 20 1.1.1.1'</code>
After adding to <code>~/.bashrc</code>:
<code>source ~/.bashrc</code>

== Troubleshooting Tips ==

=== 1. Permission Issues ===
If you get permission errors with TCP mode:
<code># Use sudo for TCP on privileged ports</code>
<code>sudo mtr --tcp --port 443 example.com</code>

<code># Or set capabilities (one-time setup)</code>
<code>sudo setcap cap_net_raw+ep /usr/bin/mtr-packet</code>

=== 2. False Positives ===
'''Common false positive:''' Packet loss at intermediate hops but NOT at the destination.

Example:
<code>3.|-- router.isp.net 20.0% 50 15.3 16.1 14.2 19.8 1.8</code>
<code>4.|-- next-hop.net 0.0% 50 18.5 19.2 17.8 22.1 1.3</code>
<code>5.|-- destination.com 0.0% 50 25.4 26.8 24.1 32.5 2.4</code>
'''This is OK!''' Hop 3 shows 20% loss, but hops 4 and 5 show 0% loss. The router at hop 3 is rate-limiting ICMP responses, but actual traffic flows normally.

=== 3. DNS Resolution Delays ===
If MTR seems slow to start:
<code># Skip DNS resolution for faster results</code>
<code>mtr --no-dns target.com</code>

<code># Resolve names afterward if needed</code>
<code>host 203.0.113.1</code>

=== 4. Comparing Results ===
<code># Run multiple tests and compare</code>
<code>mtr --report --report-cycles 50 example.com > test1.txt</code>
<code>sleep 60</code>
<code>mtr --report --report-cycles 50 example.com > test2.txt</code>
<code>diff test1.txt test2.txt</code>

== When to Use MTR vs Other Tools ==
{| class="wikitable"
!Tool
!Best For
!Limitations
|-
|'''MTR'''
|Continuous monitoring, identifying problem hops, detailed statistics
|Requires installation
|-
|'''ping'''
|Quick connectivity test, simple latency check
|Only tests endpoint
|-
|'''traceroute'''
|One-time path discovery
|No continuous monitoring
|-
|'''ss/netstat'''
|Local connection status
|Doesn't test remote paths
|}

== Best Practices ==

# '''Run enough cycles''': Use at least 50-100 cycles for accurate statistics
# '''Use --no-dns''': Faster and avoids DNS resolution issues during testing
# '''Check multiple times''': Network conditions vary; test at different times
# '''Compare protocols''': Try ICMP, UDP, and TCP if one shows issues
# '''Document findings''': Save reports with timestamps for trend analysis
# '''Test known-good hosts''': Use 8.8.8.8 or 1.1.1.1 to verify your network first
# '''Be patient''': Let MTR run for at least 30-60 seconds before drawing conclusions

== Reading Between the Lines ==

=== Good Network Health Example ===
<code>HOST: hostname Loss% Snt Last Avg Best Wrst StDev</code>
<code>1.|-- 192.168.1.1 0.0% 50 1.1 1.2 0.9 2.1 0.2</code>
<code>2.|-- 10.0.0.1 0.0% 50 8.2 8.5 7.5 10.2 0.5</code>
<code>3.|-- isp-gateway.net 0.0% 50 15.1 15.5 14.0 18.3 0.8</code>
<code>4.|-- google.com 0.0% 50 24.8 25.2 23.5 28.1 1.1</code>
✅ No packet loss, consistent latency, low jitter

=== Problem Network Example ===
<code>HOST: hostname Loss% Snt Last Avg Best Wrst StDev</code>
<code>1.|-- 192.168.1.1 0.0% 50 1.2 1.5 1.0 2.3 0.3</code>
<code>2.|-- 10.0.0.1 5.0% 50 45.3 52.8 8.1 245.2 45.7</code>
<code>3.|-- ??? 100.0% 50 0.0 0.0 0.0 0.0 0.0</code>
<code>4.|-- destination.com 15.0% 50 95.4 125.8 48.2 385.5 78.2</code>
❌ Packet loss at hop 2 and destination, high jitter, very high worst-case latency

== Summary ==
MTR is your Swiss Army knife for network diagnostics. Key takeaways:

* Use <code>--report-cycles 50+</code> for reliable data
* Watch for packet loss at the '''destination''' (intermediate losses may be false positives)
* High '''StDev''' indicates unstable connection
* High '''Avg''' latency shows slow links
* Use <code>--no-dns</code> for faster results
* Compare '''ICMP''', '''UDP''', and '''TCP''' modes if issues appear
* Test at different times of day for comprehensive analysis

== 4. Package Management Issues ==

=== Fix Broken Packages ===
<code># Update package lists</code>
<code>sudo apt update</code>

<code># Fix broken dependencies</code>
<code>sudo apt --fix-broken install</code>

<code># Reconfigure packages</code>
<code>sudo dpkg --configure -a</code>
<code>(if no output, there is nothing to do)</code>

<code># Clean package cache</code>
<code>sudo apt clean</code>
<code>sudo apt autoclean</code>

<code># Remove unused packages</code>
<code>sudo apt autoremove</code>

=== Handle Held or Locked Packages ===
<code># If apt is locked, find the process</code>
<code>sudo lsof /var/lib/dpkg/lock-frontend</code>

<code># Force remove lock (use carefully)</code>
<code>sudo rm /var/lib/dpkg/lock-frontend</code>
<code>sudo rm /var/lib/apt/lists/lock</code>

<code># Reconfigure dpkg</code>
<code>sudo dpkg --configure -a</code>

== 5. Disk & Filesystem Issues ==

=== Check Disk Health ===
<code># Check disk space</code>
<code>df -h</code>

<code># Check inode usage</code>
<code>df -i</code>

<code># View disk I/O statistics</code>
<code>iostat -x 1</code>
<code>(Make sure you have sysstat which includes useful performance monitoring tools other than iostat - disk I/O statistics</code>

* <code>mpstat</code> - CPU statistics

* <code>sar</code> - system activity reporter

* <code>pidstat</code> - process statistics

* <code>cifsiostat</code> - CIFS statistics

<code>''# Show stats in MB instead of KB'' iostat -xm 2</code>

<code>''# Monitor specific device'' iostat -x sda 1</code>
<code># Check for disk errors in dmesg</code>
<code>sudo dmesg | grep -i "error\|fail"</code>

<code># SMART disk health (if smartmontools installed)</code>
<code>sudo smartctl -a /dev/sda</code>

=== Repair Filesystem ===
<code># Unmount the partition first</code>
<code>sudo umount /dev/sdXN</code>

<code># Run filesystem check</code>
<code>sudo fsck /dev/sdXN</code>

<code># For ext4 specifically</code>
<code>sudo e2fsck -f /dev/sdXN</code>

== 6. Performance Issues ==

=== Identify Resource Hogs ===
<code># CPU usage by process</code>
<code>top -o %CPU</code>

<code># Memory usage by process</code>
<code>top -o %MEM</code>

<code># Disk usage by directory</code>
<code>du -sh /* | sort -h</code>

<code># Find large files</code>
<code>find / -type f -size +100M 2>/dev/null</code>

<code># Check running processes</code>
<code>ps aux --sort=-%mem | head -20</code>

=== System Temperature Monitoring ===
<code># Install sensors (if not installed)</code>
<code>sudo apt install lm-sensors</code>
<code>sudo sensors-detect</code>

<code># View temperatures</code>
<code>sensors</code>

<code># Real-time temperature monitoring</code>
<code>watch -n 2 sensors</code>
<code>''I have it as an alias in ~/.bashrc''</code>
<code>''Go to 11. Useful Aliases & Shortcuts''</code>

== 7. Service & Application Errors ==

=== Debug Service Problems ===
<code># Check service status</code>
<code>sudo systemctl status service-name</code>

<code># View service logs</code>
<code>sudo journalctl -u service-name</code>

<code># Restart a service</code>
<code>sudo systemctl restart service-name</code>

<code># Enable service at boot</code>
<code>sudo systemctl enable service-name</code>

<code># View recent service failures</code>
<code>journalctl -p err -b</code>

=== Application Crash Investigation ===
<code># Check for core dumps</code>
<code>ls -lh /var/crash/</code>

<code># View application-specific logs</code>
<code>ls /var/log/</code>

<code># Check syslog for application errors</code>
<code>sudo tail -f /var/log/syslog</code>

== 8. Permission & Access Issues ==

=== Fix Common Permission Problems ===
<code># Check file ownership</code>
<code>ls -l /path/to/file</code>

<code># Change ownership</code>
<code>sudo chown michael:michael /path/to/file</code>
user:group

<code># Change permissions</code>
<code>sudo chmod 644 /path/to/file</code>

<code># Recursively fix permissions</code>
<code>sudo chown -R user:group /path/to/directory</code>

=== User & Authentication Issues ===
<code># Check user information</code>
<code>id username</code>

<code># View user login history</code>
<code>last -a</code>

<code># Check failed login attempts</code>
<code>sudo journalctl | grep "authentication failure"</code>

<code># Reset user password</code>
<code>sudo passwd username</code>

== 9. Hardware Issues ==

=== Identify Hardware ===
<code># List all hardware</code>
<code>sudo lshw -short</code>
(May not be installed by default)
sudo apt install lshw

<code># PCI devices</code>
<code>lspci -v</code>

<code># USB devices</code>
<code>lspci -v</code>

<code># CPU information</code>
<code>lscpu</code>

<code># Memory information</code>
<code>sudo dmidecode --type memory</code>

=== Check Hardware Errors ===
<code># Kernel ring buffer (hardware messages)</code>
<code>dmesg | less</code>
<code>(If no output, good, no errors)</code>
<code>q to quit</code>

<code># Search for specific hardware issues</code>
<code>dmesg | grep -i "error\|fail\|warn"</code>

<code># Check for USB issues</code>
<code>dmesg | grep -i usb</code>

== 10. Quick Diagnostic Commands ==

=== System Information at a Glance ===
<code># Uptime and load average</code>
<code>uptime</code>

<code># Kernel version</code>
<code>uname -r</code>

<code># Debian version</code>
<code>cat /etc/debian_version</code>

<code># System summary</code>
<code>sudo inxi -Fxz</code>

=== Emergency Toolkit ===
<code># Create a diagnostic report</code>
<code>sudo journalctl -b > ~/system-report.txt</code>
<code>dmesg >> ~/system-report.txt</code>
<code>systemctl --failed >> ~/system-report.txt</code>
<code>df -h >> ~/system-report.txt</code>
<code>free -h >> ~/system-report.txt</code>

<code># Watch logs in real-time</code>
<code>sudo journalctl -f</code>

<code># Monitor system resources continuously</code>
<code>watch -n 1 'free -h && df -h'</code>

== 11. Useful Aliases & Shortcuts ==
Add these to your <code>~/.bashrc</code> for quick access to common troubleshooting commands:
<code># Monitor system temperatures in real-time</code>
<code>alias temps="watch -n 2 'for i in /sys/class/hwmon/hwmon*/; do echo -n \"\$(cat \${i}name): \"; cat \${i}temp*_input 2>/dev/null | while read temp; do echo \"scale=1; \$temp/1000\" | bc; done | tr \"\n\" \" \"; echo \"°C\"; done'"</code>
<code>''or run the watch command in the shell without the opening and closing double quotes.''</code>

<code># Quick system status</code>
<code>alias sysstat='echo "=== CPU ===" && uptime && echo -e "\n=== Memory ===" && free -h && echo -e "\n=== Disk ===" && df -h && echo -e "\n=== Top Processes ===" && ps aux --sort=-%mem | head -10'</code>
<code>(It’s a messy mayout, but I’m terrible with awk. Feel free to improve the layoput for me)</code>

<code># View last boot logs</code>
<code>alias lastboot='journalctl -b -1'</code>

<code># Check failed services</code>
<code>alias failedservices='systemctl --failed'</code>

<code># Monitor logs in real-time</code>
<code>alias watchlog='sudo journalctl -f'</code>

<code># Quick network status</code>
<code>alias netstat='ip addr show && echo -e "\n=== Routes ===" && ip route show'</code>
After adding these, run:
<code>source ~/.bashrc</code>
----

== Tips for Your Troubleshooting ==

# '''First check logs''': <code>journalctl</code> and <code>dmesg</code> are your best friends
# '''Work through the sections''': Change one thing at a time
# '''Document changes''': Keep notes on what you've tried
# '''Search for error messages''': Copy exact error messages into search engines or AI
# '''Check recent changes''': What you did before it happened? Install something, update packages, kernel?
# '''Make backups''': Before major changes, backup important data
# '''Use verbose mode''': Add <code>-v</code> or <code>-vv</code> flags to commands for more detail
# '''Check forums''': Debian forum, Reddit, Stack Exchange, and mailing lists

----'''Remember''': If in doubt, search for the specific error message along with "Debian" and the version number. e.g. Debian 13 or point release if needed, Debian 13,1

This work was contributed by distro-nix on Debian User Forums on 2025-10-27 23:38:22

I welcome comments, suggestions or resources : dev@divsmart.com .

[[index.php?title=Category:Troubleshooting]]
[[index.php?title=Category:Guides]]