Skip to main content
Notes by Peter Galonza(Пётр Галонза)
Toggle Dark/Light/Auto mode Toggle Dark/Light/Auto mode Toggle Dark/Light/Auto mode Back to homepage

Proccess

Background

Running some program in background

nohup <program_name> > <program_name>.out 2> <program_name>.err < /dev/null & echo -n "$!" > pid.file &

Pipes

Create named pipe

mkfifo <name of pipe>
mknod <name of pipe> p

Write in pipe

echo <> > <pipe path>

Read from pipe

tail -f <pipe path>

Remove named pipe

unlink <pipe path>

Process Discovery

Look up process by name

pgrep <process name>

Show process tree

pstree
pstree -p <PID>

Show information about a specific PID in real time

top -p <PID>
top -p <PID> -c

Batch mode (single snapshot)

top -b -n 1

Process information pseudo-filesystem

/proc

Symlink to the current process

ls /proc/self

Root directory of the current process

ls /proc/self/root

PS

All processes

ps -ef
ps aux

Show RAM and CPU columns

ps -eo pid,cmd,%cpu,%mem

Sort by RAM

ps aux --sort=%mem

Sort by CPU

ps aux --sort=%cpu

Process tree with ASCII art

ps -ef --forest

Get PIDs of a process by name

pgrep <process name>
ps -ef | grep -v grep | grep <process name> | awk '{ print $2 }'

Print environment variables of a process

ps ewww

TOP field reference

Field Description
us, user Time running un-niced user processes
sy, system Time running kernel processes
ni, nice Time running niced user processes
id, idle Time spent in the kernel idle handler
wa, IO-wait Time waiting for I/O completion
hi Time servicing hardware interrupts
si Time servicing software interrupts
st Time stolen from this VM by the hypervisor
Column Description
PID Process ID
USER Username of the process owner
PR Priority (lower value = higher priority, range -20 to 20)
NI Nice value
VIRT Virtual memory size (KiB)
RES Resident (physical) memory size (KiB)
SHR Shared memory size (KiB)
S Status: D (uninterruptible sleep), R (running), S (sleeping), T (traced/stopped), Z (zombie)
%CPU CPU usage (can exceed 100% on multi-core)
%MEM Memory usage (RES / total RAM)
TIME+ Total CPU time since start
COMMAND Command name or command line (-c for full line)

Process Control

Kill process by name

pkill <process name>
killall <process name>

Kill all processes of a user

killall -u <user name>

Show what process is using a device or mount point

fuser -m /mnt

Set or retrieve CPU affinity

taskset -pc <core> <PID>

Batch scheduling policy

chrt -b -p 0 <PID>

Set affinity to a NUMA node

numactl --cpunodebind=<NUMA node> --membind=<NUMA node> <command>

Run with CPU affinity and I/O weight limits

systemd-run --scope -p CPUAffinity=<core> -p IOWeight=<weight> -- <command>

Show limits of a process

cat /proc/<PID>/limits

Show shell limits of the current user

ulimit -a

Change process resource limits

prlimit --pid <PID> --nofile=<soft>:<hard>

Show kernel parameters

sysctl -a

Apply changes from /etc/sysctl.conf

sysctl -p

Priority

Start with a niceness value

nice -n <value> <command>

Change niceness of a running process

renice -n <value> -p <PID>

Set I/O scheduling class and priority

ionice -c <1-3> -n <0-7> <command>
ionice -c <1-3> -n <0-7> -p <PID>

Namespaces & Cgroups

Run a program in new namespaces

unshare <parameters> <program>

Join an existing namespace

nsenter --target <PID> <parameters> <program>

List namespace objects

lsns

Show namespaces of a process

ls -l /proc/<PID>/ns/

Show user/group ID mappings

cat /proc/<PID>/uid_map
cat /proc/<PID>/gid_map

Show cgroups of a process

cat /proc/<PID>/cgroup

Hide processes from other users (add to /etc/fstab)

proc /proc hidepid=<0-2>,gid=<group> ...

Open Files & Sockets

Show maximum number of open files system-wide

cat /proc/sys/fs/file-max

Show allocated, free, and max file descriptors

cat /proc/sys/fs/file-nr

Show open files of a process

ls -l /proc/<PID>/fd/
lsof -p <PID>

Show processes of a user

lsof -u <user name>

Show open files by command name

lsof -c <command>

Show what process is using a port

lsof -i :<port number>

Show sockets of a process

lsof -i -p <PID>
lsof -i -a -p <PID>

Show open files in a directory

lsof +D <path to directory>

Find a socket by its inode number

grep <socket number> /proc/net/tcp

Count total open files

lsof | wc -l

Show deleted files still held open

lsof -nP | grep '(deleted)'
lsof -nP +L1

Read stdout/stderr of a running process

cat /proc/<PID>/fd/1
cat /proc/<PID>/fd/2
cat /proc/<PID>/fd/1 > /tmp/stdout.log
cat /proc/<PID>/fd/2 > /tmp/stderr.log

Truncate a file by path

: > <path to file>

Truncate an open file descriptor

: > /proc/<PID>/fd/<fd number>

Inotify limits

Maximum inotify instances per user

cat /proc/sys/fs/inotify/max_user_instances

Maximum watches per inotify instance

cat /proc/sys/fs/inotify/max_user_watches

Maximum queued events

cat /proc/sys/fs/inotify/max_queued_events

Tracing & Debugging

Strace

Trace syscalls of a new process

strace -f -tt -s <number of symbols> -o <log file> <application>

Attach to a running process by name

pgrep <application> | awk '{print "-p " $1}' | xargs strace -f -tt -s <number of symbols> -o <log file>

Bpftrace

Trace file open operations

bpftrace -e 'tracepoint:syscalls:sys_enter_openat { printf("%s %s\n", comm, str(args->filename)); }'

Trace process execution

bpftrace -e 'tracepoint:syscalls:sys_enter_execve { printf("%s\n", comm); }'

Trace signals sent to processes

bpftrace -e 'tracepoint:syscalls:sys_enter_kill { printf("%s -> PID %d, SIG %d\n", comm, args->pid, args->sig); }'

Count syscalls per process (live histogram)

bpftrace -e 'tracepoint:raw_syscalls:sys_enter { @[comm] = count(); }'

Trace TCP connect attempts

bpftrace -e 'kfunc:tcp_connect { printf("%s -> %s:%d\n", comm, ntop2(args->sk->__sk_common.skc_daddr), args->sk->__sk_common.skc_dport); }'

GDB

Attach to a process or open a core dump

gdb <program>
gdb <program> <core dump>
gdb -p <PID>

Performance Analysis

Collect performance metrics for a process

perf stat -d -p <PID> -- sleep 10

Monitor process statistics (context switches, CPU, memory)

pidstat -wstu -p <PID> 1

Pressure Stall Information (CPU, IO, memory, IRQ)

tail -f /proc/pressure/cpu
tail -f /proc/pressure/io
tail -f /proc/pressure/memory
tail -f /proc/pressure/irq

Missing files or libraries

Find missing libraries using strace

strace -eopen <application name>

Find missing libraries using LD_DEBUG

LD_DEBUG=files <application name>

Find interpreter

readelf -a /usr/bin/<application name> | grep interp

Debug library loading

LD_DEBUG=all /lib64/ld-linux-x86-64.so.2 <application name>

Load one core

yes > /dev/null &
perl -e 'while(1){}'