Process Management in Linux

This article will guide you through essential process management techniques in Linux, covering `systemd`, `supervisord`, process priorities, and `cgroups`. Effective process management is crucial for maintaining server stability, performance, and security.

Prerequisites

• Basic understanding of the Linux command line.
• Root or sudo privileges on your Linux server.
• A running Linux server (e.g., Ubuntu, CentOS, Debian).
• Familiarity with text editors like `nano` or `vim`.

Understanding Processes in Linux

A process is an instance of a running program. Each process has a unique Process ID (PID), which is used by the operating system to manage and track it. Understanding how processes are managed is fundamental to server administration.

Key Concepts

• **PID (Process ID):** A unique number assigned to each running process.
• **Parent Process:** A process that creates another process (child process).
• **Child Process:** A process created by another process.
• **Daemon:** A background process that runs without direct user interaction, often providing services.

Managing Processes with systemd

`systemd` is the modern init system and service manager for most Linux distributions. It's responsible for starting, stopping, and managing services (daemons) at boot time and throughout the system's lifecycle.

Creating a systemd Service File

To manage a custom application as a service, you'll create a `.service` unit file.

1. **Create the service file:** ```bash sudo nano /etc/systemd/system/my_app.service ```

2. **Add the following content:** ```ini [Unit] Description=My Custom Application Service After=network.target

[Service] ExecStart=/usr/local/bin/my_app --config /etc/my_app/config.conf Restart=on-failure User=my_app_user Group=my_app_group WorkingDirectory=/opt/my_app

[Install] WantedBy=multi-user.target ``` * **`[Unit]` Section:** * `Description`: A human-readable description of the service. * `After`: Specifies that this service should start after the `network.target` is reached, ensuring network connectivity is available. * **`[Service]` Section:** * `ExecStart`: The command to execute to start the application. Ensure the path to your application is correct. * `Restart=on-failure`: Configures `systemd` to automatically restart the service if it exits with a non-zero status code (indicating an error). * `User`/`Group`: Specifies the user and group under which the service will run. This is a critical security practice to limit the service's privileges. * `WorkingDirectory`: The directory where the application will be executed. * **`[Install]` Section:** * `WantedBy=multi-user.target`: Defines the target that this service should be enabled for. `multi-user.target` is the standard runlevel for a non-graphical, multi-user system.

3. **Reload systemd to recognize the new service:** ```bash sudo systemctl daemon-reload ``` **Why this step is important:** `daemon-reload` tells `systemd` to rescan its unit files and load any new or modified configurations.

4. **Enable the service to start on boot:** ```bash sudo systemctl enable my_app.service ``` **Why this step is important:** This creates a symbolic link from the `multi-user.target.wants` directory to your service file, ensuring it's started automatically when the system boots into that target.

5. **Start the service:** ```bash sudo systemctl start my_app.service ```

6. **Check the service status:** ```bash sudo systemctl status my_app.service ``` **Expected Output Snippet:** ``` ● my_app.service - My Custom Application Service Loaded: loaded (/etc/systemd/system/my_app.service; enabled; vendor preset: enabled) Active: active (running) since Tue 2023-10-27 10:00:00 UTC; 1min ago Main PID: 12345 (my_app) Tasks: 1 (limit: 4915) Memory: 5.0M CPU: 100ms CGroup: /system.slice/my_app.service └─12345 /usr/local/bin/my_app --config /etc/my_app/config.conf ```

7. **Stop the service:** ```bash sudo systemctl stop my_app.service ```

8. **Restart the service:** ```bash sudo systemctl restart my_app.service ```

Troubleshooting systemd Services

• **Service not starting:** Check `sudo systemctl status my_app.service` for error messages. Examine application logs, which can often be found in `/var/log/` or specified within the application's configuration.
• **Permission denied:** Ensure the `User` and `Group` specified in the service file have the necessary permissions to access the application executable, configuration files, and any directories it needs to write to.
• **Application crashes immediately:** Verify the `ExecStart` command is correct and that the application runs successfully when executed manually from the command line as the specified user.

Managing Processes with supervisord

`supervisord` is a process control system that allows you to monitor and control a number of processes on UNIX-like operating systems. It's particularly useful for managing applications that are not designed to run as system daemons or when you need more fine-grained control over child processes.

Installing and Configuring supervisord

1. **Install supervisord:** * **Debian/Ubuntu:** ```bash sudo apt update sudo apt install supervisor ``` * **CentOS/RHEL:** ```bash sudo yum install epel-release sudo yum install supervisor ```

2. **Create a configuration file for your application:** ```bash sudo nano /etc/supervisor/conf.d/my_app.conf ```

3. **Add the following content:** ```ini [program:my_app] command=/usr/local/bin/my_app --config /etc/my_app/config.conf directory=/opt/my_app user=my_app_user autostart=true autorestart=true stderr_logfile=/var/log/supervisor/my_app.err.log stdout_logfile=/var/log/supervisor/my_app.out.log ``` * **`[program:my_app]`:** Defines a program named `my_app`. * `command`: The command to run the application. * `directory`: The working directory for the command. * `user`: The user to run the command as. * `autostart=true`: Automatically start this program when `supervisord` starts. * `autorestart=true`: Automatically restart the program if it exits. * `stderr_logfile`/`stdout_logfile`: Paths for standard error and standard output logs.

4. **Create the log directory if it doesn't exist:** ```bash sudo mkdir -p /var/log/supervisor sudo chown my_app_user:my_app_group /var/log/supervisor # Adjust ownership if needed ```

5. **Update supervisord's configuration:** ```bash sudo supervisorctl reread sudo supervisorctl update ``` **Why these steps are important:** `reread` tells `supervisord` to re-examine its configuration files, and `update` applies the changes, starting or stopping programs as defined in the new configuration.

6. **Check the status of your application:** ```bash sudo supervisorctl status my_app ``` **Expected Output Snippet:** ``` my_app RUNNING PID 12346, uptime 0:01:30 ```

7. **To stop, start, or restart:** ```bash sudo supervisorctl stop my_app sudo supervisorctl start my_app sudo supervisorctl restart my_app ```

Troubleshooting supervisord

• **Program not starting:** Use `sudo supervisorctl status` to see if it's running. Check `sudo supervisorctl tail my_app stderr` and `sudo supervisorctl tail my_app stdout` for error messages from the application.
• **Permission errors:** Ensure the `user` specified in the `.conf` file has read/write permissions to log files and any directories the application needs.
• **Configuration changes not applied:** Always run `sudo supervisorctl reread` and `sudo supervisorctl update` after modifying `.conf` files.

Process Priorities (Nice Values)

Process priority determines how much CPU time a process receives. Linux uses "nice" values to control this. A lower nice value means a higher priority (more CPU time), and a higher nice value means a lower priority (less CPU time). The range is typically -20 (highest priority) to +19 (lowest priority).

Adjusting Process Priority

• **To run a command with a lower priority (higher nice value):**
```bash nice -n 10 your_command ``` This will run `your_command` with a nice value of 10.
• **To run a command with a higher priority (lower nice value):**
```bash sudo renice -n -5 -p ``` This command increases the priority of an already running process with PID `` (requires root privileges for negative nice values).
• **To check the nice value of a running process:**
```bash ps -eo pid,ni,comm ``` The `NI` column shows the nice value.
• *Why this matters:** You can use nice values to ensure critical processes get enough CPU resources while less important background tasks don't monopolize the CPU, improving overall system responsiveness.

Troubleshooting Priority Issues

• **Cannot set negative nice value:** You need root privileges (`sudo`) to increase a process's priority (decrease its nice value).
• **Process still slow:** While priority affects CPU allocation, it doesn't solve other performance bottlenecks like I/O wait or insufficient RAM. Use tools like `top`, `htop`, and `iostat` to diagnose other issues.

Control Groups (cgroups)

Control Groups (cgroups) are a Linux kernel feature that allows you to allocate, limit, and prioritize system resources (CPU, memory, I/O, network) for a collection of processes. `systemd` heavily utilizes cgroups for resource management.

Understanding cgroups

• **Hierarchical Structure:** Cgroups are organized in a hierarchy, allowing for complex resource allocation policies.
• **Controllers:** Different controllers manage specific resources (e.g., `cpu`, `memory`, `blkio`).
• **Systemd Integration:** `systemd` automatically places services it manages into their own cgroups, making it easier to manage resource limits for individual services.

Basic cgroup Management (via systemd)

While direct manipulation of cgroups can be complex, `systemd` provides a user-friendly way to set resource limits for services.

1. **Edit the systemd service file** (e.g., `/etc/systemd/system/my_app.service`).

2. **Add resource control directives to the `[Service]` section:** ```ini [Unit] Description=My Custom Application Service After=network.target

[Service] ExecStart=/usr/local/bin/my_app --config /etc/my_app/config.conf Restart=on-failure User=my_app_user Group=my_app_group WorkingDirectory=/opt/my_app

# CPU and Memory Limits CPUQuota=50% # Limit CPU usage to 50% of one core MemoryMax=256M # Limit memory usage to 256MB MemorySwapMax=0 # Disable swap usage for this service

[Install] WantedBy=multi-user.target ``` * `CPUQuota`: Limits the CPU time the service can use. `50%` means it can use at most half of one CPU core. You can also use `CPUShares` for relative weighting. * `MemoryMax`: Sets a hard limit on memory usage. * `MemorySwapMax`: Controls swap usage. `0` disables it.

3. **Reload systemd and restart the service:** ```bash sudo systemctl daemon-reload sudo systemctl restart my_app.service ```

4. **Monitor resource usage:** ```bash systemd-cgtop ``` This command provides a real-time view of resource usage by cgroups, similar to `top` but for cgroups.

• *Why this matters:** Cgroups are essential for preventing runaway processes from consuming all system resources, ensuring stability and fair resource distribution, especially in multi-tenant environments or when running multiple demanding applications.

Troubleshooting cgroups

• **Service failing to start with resource limits:** Ensure the limits are reasonable for your application. A limit set too low might prevent the application from even starting. Check `journalctl -u my_app.service` for specific error messages.
• **Application behaving erratically:** If an application is hitting its memory limit, it might be killed by the OOM (Out-Of-Memory) killer. Check system logs (`/var/log/syslog` or `journalctl`) for OOM killer messages.
• **`systemd-cgtop` shows unexpected usage:** Verify that the correct service is associated with the cgroup you are monitoring. Ensure no other processes are inadvertently grouped with your service.

Conclusion

Category:Optimization Category:System Administration Category:Linux