Applies to openSUSE Leap 42.3

10 The systemd Daemon

The program systemd is the process with process ID 1. It is responsible for initializing the system in the required way. systemd is started directly by the kernel and resists signal 9, which normally terminates processes. All other programs are either started directly by systemd or by one of its child processes.

Starting with openSUSE Leap 42.1 systemd is a replacement for the popular System V init daemon. systemd is fully compatible with System V init (by supporting init scripts). One of the main advantages of systemd is that it considerably speeds up boot time by aggressively paralleling service starts. Furthermore, systemd only starts a service when it is really needed. Daemons are not started unconditionally at boot time, but rather when being required for the first time. systemd also supports Kernel Control Groups (cgroups), snapshotting and restoring the system state and more. See http://www.freedesktop.org/wiki/Software/systemd/ for details.

10.1 The systemd Concept

This section will go into detail about the concept behind systemd.

10.1.1 What Is systemd

systemd is a system and session manager for Linux, compatible with System V and LSB init scripts. The main features are:

  • provides aggressive parallelization capabilities

  • uses socket and D-Bus activation for starting services

  • offers on-demand starting of daemons

  • keeps track of processes using Linux cgroups

  • supports snapshotting and restoring of the system state

  • maintains mount and automount points

  • implements an elaborate transactional dependency-based service control logic

10.1.2 Unit File

A unit configuration file encodes information about a service, a socket, a device, a mount point, an automount point, a swap file or partition, a start-up target, a watched file system path, a timer controlled and supervised by systemd, a temporary system state snapshot, a resource management slice or a group of externally created processes. Unit file is a generic term used by systemd for the following:

  • Service.  Information about a process (for example running a daemon); file ends with .service

  • Targets.  Used for grouping units and as synchronization points during start-up; file ends with .target

  • Sockets.  Information about an IPC or network socket or a file system FIFO, for socket-based activation (like inetd); file ends with .socket

  • Path.  Used to trigger other units (for example running a service when files change); file ends with .path

  • Timer.  Information about a timer controlled, for timer-based activation; file ends with .timer

  • Mount point.  Usually auto-generated by the fstab generator; file ends with .mount

  • Automount point.  Information about a file system automount point; file ends with .automount

  • Swap.  Information about a swap device or file for memory paging; file ends with .swap

  • Device.  Information about a device unit as exposed in the sysfs/udev(7) device tree; file ends with .device

  • Scope / Slice.  A concept for hierarchically managing resources of a group of processes; file ends with .scope/.slice

For more information about systemd.unit see http://www.freedesktop.org/software/systemd/man/systemd.unit.html

10.2 Basic Usage

The System V init system uses several commands to handle services—the init scripts, insserv, telinit and others. systemd makes it easier to manage services, since there is only one command to memorize for the majority of service-handling tasks: systemctl. It uses the command plus subcommand notation like git or zypper:

systemctl GENERAL OPTIONS SUBCOMMAND SUBCOMMAND OPTIONS

See man 1 systemctl for a complete manual.

Tip
Tip: Terminal Output and Bash Completion

If the output goes to a terminal (and not to a pipe or a file, for example) systemd commands send long output to a pager by default. Use the --no-pager option to turn off paging mode.

systemd also supports bash-completion, allowing you to enter the first letters of a subcommand and then press →| to automatically complete it. This feature is only available in the bash shell and requires the installation of the package bash-completion.

10.2.1 Managing Services in a Running System

Subcommands for managing services are the same as for managing a service with System V init (start, stop, ...). The general syntax for service management commands is as follows:

systemd
systemctl reload|restart|start|status|stop|... MY_SERVICE(S)
System V init
rcMY_SERVICE(S) reload|restart|start|status|stop|...

systemd allows you to manage several services in one go. Instead of executing init scripts one after the other as with System V init, execute a command like the following:

systemctl start MY_1ST_SERVICE MY_2ND_SERVICE

If you want to list all services available on the system:

systemctl list-unit-files --type=service

The following table lists the most important service management commands for systemd and System V init:

Table 10.1: Service Management Commands

Task

systemd Command

System V init Command

Starting. 

start
start

Stopping. 

stop
stop

Restarting.  Shuts down services and starts them afterward. If a service is not yet running it will be started.

restart
restart

Restarting conditionally.  Restarts services if they are currently running. Does nothing for services that are not running.

try-restart
try-restart

Reloading.  Tells services to reload their configuration files without interrupting operation. Use case: Tell Apache to reload a modified httpd.conf configuration file. Note that not all services support reloading.

reload
reload

Reloading or restarting.  Reloads services if reloading is supported, otherwise restarts them. If a service is not yet running it will be started.

reload-or-restart
n/a

Reloading or restarting conditionally.  Reloads services if reloading is supported, otherwise restarts them if currently running. Does nothing for services that are not running.

reload-or-try-restart
n/a

Getting detailed status information.  Lists information about the status of services. The systemd command shows details such as description, executable, status, cgroup, and messages last issued by a service (see Section 10.6.8, “Debugging Services”). The level of details displayed with the System V init differs from service to service.

status
status

Getting short status information.  Shows whether services are active or not.

is-active
status

10.2.2 Permanently Enabling/Disabling Services

The service management commands mentioned in the previous section let you manipulate services for the current session. systemd also lets you permanently enable or disable services, so they are automatically started when requested or are always unavailable. You can either do this by using YaST, or on the command line.

10.2.2.1 Enabling/Disabling Services on the Command Line

The following table lists enabling and disabling commands for systemd and System V init:

Important
Important: Service Start

When enabling a service on the command line, it is not started automatically. It is scheduled to be started with the next system start-up or runlevel/target change. To immediately start a service after having enabled it, explicitly run systemctl start MY_SERVICE or rc MY_SERVICE start.

Table 10.2: Commands for Enabling and Disabling Services

Task

systemd Command

System V init Command

Enabling. 

systemctl enable MY_SERVICE(S)

insserv MY_SERVICE(S), chkconfig -a MY_SERVICE(S)

Disabling. 

systemctl disable MY_SERVICE(S).service

insserv -r MY_SERVICE(S), chkconfig -d MY_SERVICE(S)

Checking.  Shows whether a service is enabled or not.

systemctl is-enabled MY_SERVICE

chkconfig MY_SERVICE

Re-enabling.  Similar to restarting a service, this command first disables and then enables a service. Useful to re-enable a service with its defaults.

systemctl reenable MY_SERVICE

n/a

Masking.  After disabling a service, it can still be started manually. To completely disable a service, you need to mask it. Use with care.

systemctl mask MY_SERVICE

n/a

Unmasking.  A service that has been masked can only be used again after it has been unmasked.

systemctl unmask MY_SERVICE

n/a

10.3 System Start and Target Management

The entire process of starting the system and shutting it down is maintained by systemd. From this point of view, the kernel can be considered a background process to maintain all other processes and adjust CPU time and hardware access according to requests from other programs.

10.3.1 Targets Compared to Runlevels

With System V init the system was booted into a so-called Runlevel. A runlevel defines how the system is started and what services are available in the running system. Runlevels are numbered; the most commonly known ones are 0 (shutting down the system), 3 (multiuser with network) and 5 (multiuser with network and display manager).

systemd introduces a new concept by using so-called target units. However, it remains fully compatible with the runlevel concept. Target units are named rather than numbered and serve specific purposes. For example, the targets local-fs.target and swap.target mount local file systems and swap spaces.

The target graphical.target provides a multiuser system with network and display manager capabilities and is equivalent to runlevel 5. Complex targets, such as graphical.target act as meta targets by combining a subset of other targets. Since systemd makes it easy to create custom targets by combining existing targets, it offers great flexibility.

The following list shows the most important systemd target units. For a full list refer to man 7 systemd.special.

Selected systemd Target Units
default.target

The target that is booted by default. Not a real target, but rather a symbolic link to another target like graphic.target. Can be permanently changed via YaST (see Section 10.4, “Managing Services with YaST”). To change it for a session, use the kernel parameter systemd.unit=MY_TARGET.target at the boot prompt.

emergency.target

Starts an emergency shell on the console. Only use it at the boot prompt as systemd.unit=emergency.target.

graphical.target

Starts a system with network, multiuser support and a display manager.

halt.target

Shuts down the system.

mail-transfer-agent.target

Starts all services necessary for sending and receiving mails.

multi-user.target

Starts a multiuser system with network.

reboot.target

Reboots the system.

rescue.target

Starts a single-user system without network.

To remain compatible with the System V init runlevel system, systemd provides special targets named runlevelX.target mapping the corresponding runlevels numbered X.

If you want to know the current target, use the command: systemctl get-default

Table 10.3: System V Runlevels and systemd Target Units

System V runlevel

systemd target

Purpose

0

runlevel0.target, halt.target, poweroff.target

System shutdown

1, S

runlevel1.target, rescue.target,

Single-user mode

2

runlevel2.target, multi-user.target,

Local multiuser without remote network

3

runlevel3.target, multi-user.target,

Full multiuser with network

4

runlevel4.target

Unused/User-defined

5

runlevel5.target, graphical.target,

Full multiuser with network and display manager

6

runlevel6.target, reboot.target,

System reboot

Important
Important: systemd Ignores /etc/inittab

The runlevels in a System V init system are configured in /etc/inittab. systemd does not use this configuration. Refer to Section 10.5.3, “Creating Custom Targets” for instructions on how to create your own bootable target.

10.3.1.1 Commands to Change Targets

Use the following commands to operate with target units:

Task

systemd Command

System V init Command

Change the current target/runlevel

systemctl isolate MY_TARGET.target

telinit X

Change to the default target/runlevel

systemctl default

n/a

Get the current target/runlevel

systemctl list-units --type=target

With systemd there is usually more than one active target. The command lists all currently active targets.

who -r

or

runlevel

persistently change the default runlevel

Use the Services Manager or run the following command:

ln -sf /usr/lib/systemd/system/ MY_TARGET.target /etc/systemd/system/default.target

Use the Services Manager or change the line

id: X:initdefault:

in /etc/inittab

Change the default runlevel for the current boot process

Enter the following option at the boot prompt

systemd.unit= MY_TARGET.target

Enter the desired runlevel number at the boot prompt.

Show a target's/runlevel's dependencies

systemctl show -p "Requires" MY_TARGET.target

systemctl show -p "Wants" MY_TARGET.target

Requires lists the hard dependencies (the ones that must be resolved), whereas Wants lists the soft dependencies (the ones that get resolved if possible).

n/a

10.3.2 Debugging System Start-Up

systemd offers the means to analyze the system start-up process. You can conveniently review the list of all services and their status (rather than having to parse /varlog/). systemd also allows you to scan the start-up procedure to find out how much time each service start-up consumes.

10.3.2.1 Review Start-Up of Services

To review the complete list of services that have been started since booting the system, enter the command systemctl. It lists all active services like shown below (shortened). To get more information on a specific service, use systemctl status MY_SERVICE.

Example 10.1: List Active Services
root # systemctl
UNIT                        LOAD   ACTIVE SUB       JOB DESCRIPTION
[...]
iscsi.service               loaded active exited    Login and scanning of iSC+
kmod-static-nodes.service   loaded active exited    Create list of required s+
libvirtd.service            loaded active running   Virtualization daemon
nscd.service                loaded active running   Name Service Cache Daemon
ntpd.service                loaded active running   NTP Server Daemon
polkit.service              loaded active running   Authorization Manager
postfix.service             loaded active running   Postfix Mail Transport Ag+
rc-local.service            loaded active exited    /etc/init.d/boot.local Co+
rsyslog.service             loaded active running   System Logging Service
[...]
LOAD   = Reflects whether the unit definition was properly loaded.
ACTIVE = The high-level unit activation state, i.e. generalization of SUB.
SUB    = The low-level unit activation state, values depend on unit type.

161 loaded units listed. Pass --all to see loaded but inactive units, too.
To show all installed unit files use 'systemctl list-unit-files'.

To restrict the output to services that failed to start, use the --failed option:

Example 10.2: List Failed Services
root # systemctl --failed
UNIT                   LOAD   ACTIVE SUB    JOB DESCRIPTION
apache2.service        loaded failed failed     apache
NetworkManager.service loaded failed failed     Network Manager
plymouth-start.service loaded failed failed     Show Plymouth Boot Screen

[...]

10.3.2.2 Debug Start-Up Time

To debug system start-up time, systemd offers the systemd-analyze command. It shows the total start-up time, a list of services ordered by start-up time and can also generate an SVG graphic showing the time services took to start in relation to the other services.

Listing the System Start-Up Time
root # systemd-analyze
Startup finished in 2666ms (kernel) + 21961ms (userspace) = 24628ms
Listing the Services Start-Up Time
root # systemd-analyze blame
  6472ms systemd-modules-load.service
  5833ms remount-rootfs.service
  4597ms network.service
  4254ms systemd-vconsole-setup.service
  4096ms postfix.service
  2998ms xdm.service
  2483ms localnet.service
  2470ms SuSEfirewall2_init.service
  2189ms avahi-daemon.service
  2120ms systemd-logind.service
  1210ms xinetd.service
  1080ms ntp.service
[...]
    75ms fbset.service
    72ms purge-kernels.service
    47ms dev-vda1.swap
    38ms bluez-coldplug.service
    35ms splash_early.service
Services Start-Up Time Graphics
root # systemd-analyze plot > jupiter.example.com-startup.svg

10.3.2.3 Review the Complete Start-Up Process

The above-mentioned commands let you review the services that started and the time it took to start them. If you need to know more details, you can tell systemd to verbosely log the complete start-up procedure by entering the following parameters at the boot prompt:

systemd.log_level=debug systemd.log_target=kmsg

Now systemd writes its log messages into the kernel ring buffer. View that buffer with dmesg:

dmesg -T | less

10.3.3 System V Compatibility

systemd is compatible with System V, allowing you to still use existing System V init scripts. However, there is at least one known issue where a System V init script does not work with systemd out of the box: starting a service as a different user via su or sudo in init scripts will result in a failure of the script, producing an Access denied error.

When changing the user with su or sudo, a PAM session is started. This session will be terminated after the init script is finished. As a consequence, the service that has been started by the init script will also be terminated. To work around this error, proceed as follows:

  1. Create a service file wrapper with the same name as the init script plus the file name extension .service:

    [Unit]
    Description=DESCRIPTION
    After=network.target
    
    [Service]
    User=USER
    Type=forking1
    PIDFile=PATH TO PID FILE1
    ExecStart=PATH TO INIT SCRIPT start
    ExecStop=PATH TO INIT SCRIPT stop
    ExecStopPost=/usr/bin/rm -f PATH TO PID FILE1
    
    [Install]
    WantedBy=multi-user.target2

    Replace all values written in UPPERCASE LETTERS with appropriate values.

    1

    Optional—only use if the init script starts a daemon.

    2

    multi-user.target also starts the init script when booting into graphical.target. If it should only be started when booting into the display manager, user graphical.target here.

  2. Start the daemon with systemctl start APPLICATION.

10.4 Managing Services with YaST

Basic service management can also be done with the YaST Services Manager module. It supports starting, stopping, enabling and disabling services. It also lets you show a service's status and change the default target. Start the YaST module with YaST › System › Services Manager.

Services Manager
Figure 10.1: Services Manager
Changing the Default System Target

To change the target the system boots into, choose a target from the Default System Target drop-down box. The most often used targets are Graphical Interface (starting a graphical login screen) and Multi-User (starting the system in command line mode).

Starting or Stopping a Service

Select a service from the table. The Active column shows whether it is currently running (Active) or not (Inactive). Toggle its status by choosing Start/Stop.

Starting or stopping a service changes its status for the currently running session. To change its status throughout a reboot, you need to enable or disable it.

Enabling or Disabling a Service

Select a service from the table. The Enabled column shows whether it is currently Enabled or Disabled. Toggle its status by choosing Enable/Disable.

By enabling or disabling a service you configure whether it is started during booting (Enabled) or not (Disabled). This setting will not affect the current session. To change its status in the current session, you need to start or stop it.

View a Status Messages

To view the status message of a service, select it from the list and choose Show Details. The output you will see is identical to the one generated by the command systemctl -l status MY_SERVICE.

Warning
Warning: Faulty Runlevel Settings May Damage Your System

Faulty runlevel settings may make your system unusable. Before applying your changes, make absolutely sure that you know their consequences.

10.5 Customization of systemd

The following sections contain some examples for systemd customization.

Warning
Warning: Avoiding Overwritten Customization

Always do systemd customization in /etc/systemd/, never in /usr/lib/systemd/. Otherwise your changes will be overwritten by the next update of systemd.

10.5.1 Customizing Service Files

The systemd service files are located in /usr/lib/systemd/system. If you want to customize them, proceed as follows:

  1. Copy the files you want to modify from /usr/lib/systemd/system to /etc/systemd/system. Keep the file names identical to the original ones.

  2. Modify the copies in /etc/systemd/system according to your needs.

  3. For an overview of your configuration changes, use the systemd-delta command. It can compare and identify configuration files that override other configuration files. For details, refer to the systemd-delta man page.

The modified files in /etc/systemd will take precedence over the original files in /usr/lib/systemd/system, provided that their file name is the same.

10.5.2 Creating Drop-in Files

If you only want to add a few lines to a configuration file or modify a small part of it, you can use so-called drop-in files. Drop-in files let you extend the configuration of unit files without having to edit or override the unit files themselves.

For example, to change one value for the FOOBAR service located in /usr/lib/systemd/system/FOOBAR.SERVICE, proceed as follows:

  1. Create a directory called /etc/systemd/system/MY_SERVICE.service.d/.

    Note the .d suffix. The directory must otherwise be named like the service that you want to patch with the drop-in file.

  2. In that directory, create a file WHATEVERMODIFICATION.conf.

    Make sure it only contains the line with the value that you want to modify.

  3. Save your changes to the file. It will be used as an extension of the original file.

10.5.3 Creating Custom Targets

On System V init SUSE systems, runlevel 4 is unused to allow administrators to create their own runlevel configuration. systemd allows you to create any number of custom targets. It is suggested to start by adapting an existing target such as graphical.target.

  1. Copy the configuration file /usr/lib/systemd/system/graphical.target to /etc/systemd/system/MY_TARGET.target and adjust it according to your needs.

  2. The configuration file copied in the previous step already covers the required (hard) dependencies for the target. To also cover the wanted (soft) dependencies, create a directory /etc/systemd/system/MY_TARGET.target.wants.

  3. For each wanted service, create a symbolic link from /usr/lib/systemd/system into /etc/systemd/system/MY_TARGET.target.wants.

  4. Once you have finished setting up the target, reload the systemd configuration to make the new target available:

    systemctl daemon-reload

10.6 Advanced Usage

The following sections cover advanced topics for system administrators. For even more advanced systemd documentation, refer to Lennart Pöttering's series about systemd for administrators at http://0pointer.de/blog/projects.

10.6.1 Cleaning Temporary Directories

systemd supports cleaning temporary directories regularly. The configuration from the previous system version is automatically migrated and active. tmpfiles.d—which is responsible for managing temporary files—reads its configuration from /etc/tmpfiles.d/*.conf , /run/tmpfiles.d/*.conf, and /usr/lib/tmpfiles.d/*.conf files. Configuration placed in /etc/tmpfiles.d/*.conf overrides related configurations from the other two directories (/usr/lib/tmpfiles.d/*.conf is where packages store their configuration files).

The configuration format is one line per path containing action and path, and optionally mode, ownership, age and argument fields, depending on the action. The following example unlinks the X11 lock files:

Type Path               Mode UID  GID  Age Argument
r    /tmp/.X[0-9]*-lock

To get the status the tmpfile timer:

systemctl status systemd-tmpfiles-clean.timer
systemd-tmpfiles-clean.timer - Daily Cleanup of Temporary Directories
 Loaded: loaded (/usr/lib/systemd/system/systemd-tmpfiles-clean.timer; static)
 Active: active (waiting) since Tue 2014-09-09 15:30:36 CEST; 1 weeks 6 days ago
   Docs: man:tmpfiles.d(5)
         man:systemd-tmpfiles(8)

Sep 09 15:30:36 jupiter systemd[1]: Starting Daily Cleanup of Temporary Directories.
Sep 09 15:30:36 jupiter systemd[1]: Started Daily Cleanup of Temporary Directories.

For more information on temporary files handling, see man 5 tmpfiles.d.

10.6.2 System Log

Section 10.6.8, “Debugging Services” explains how to view log messages for a given service. However, displaying log messages is not restricted to service logs. You can also access and query the complete log messages written by systemd—the so-called Journal. Use the command systemd-journalctl to display the complete log messages starting with the oldest entries. Refer to man 1 systemd-journalctl for options such as applying filters or changing the output format.

10.6.3 Snapshots

You can save the current state of systemd to a named snapshot and later revert to it with the isolate subcommand. This is useful when testing services or custom targets, because it allows you to return to a defined state at any time. A snapshot is only available in the current session and will automatically be deleted on reboot. A snapshot name must end in .snapshot.

Create a Snapshot
systemctl snapshot MY_SNAPSHOT.snapshot
Delete a Snapshot
systemctl delete MY_SNAPSHOT.snapshot
View a Snapshot
systemctl show MY_SNAPSHOT.snapshot
Activate a Snapshot
systemctl isolate MY_SNAPSHOT.snapshot

10.6.4 Loading Kernel Modules

With systemd, kernel modules can automatically be loaded at boot time via a configuration file in /etc/modules-load.d. The file should be named MODULE.conf and have the following content:

# load module MODULE at boot time
MODULE

In case a package installs a configuration file for loading a kernel module, the file gets installed to /usr/lib/modules-load.d. If two configuration files with the same name exist, the one in /etc/modules-load.d tales precedence.

For more information, see the modules-load.d(5) man page.

10.6.5 Performing Actions before Loading a Service

With System V init actions that need to be performed before loading a service, needed to be specified in /etc/init.d/before.local . This procedure is no longer supported with systemd. If you need to do actions before starting services, do the following:

Loading Kernel Modules

Create a drop-in file in /etc/modules-load.d directory (see man modules-load.d for the syntax)

Creating Files or Directories, Cleaning-up Directories, Changing Ownership

Create a drop-in file in /etc/tmpfiles.d (see man tmpfiles.d for the syntax)

Other Tasks

Create a system service file, for example /etc/systemd/system/before.service, from the following template:

[Unit]
Before=NAME OF THE SERVICE YOU WANT THIS SERVICE TO BE STARTED BEFORE
[Service]
Type=oneshot
RemainAfterExit=true
ExecStart=YOUR_COMMAND
# beware, executable is run directly, not through a shell, check the man pages
# systemd.service and systemd.unit for full syntax
[Install]
# target in which to start the service
WantedBy=multi-user.target
#WantedBy=graphical.target

When the service file is created, you should run the following commands (as root):

systemctl daemon-reload
systemctl enable before

Every time you modify the service file, you need to run:

systemctl daemon-reload

10.6.6 Kernel Control Groups (cgroups)

On a traditional System V init system it is not always possible to clearly assign a process to the service that spawned it. Some services, such as Apache, spawn a lot of third-party processes (for example CGI or Java processes), which themselves spawn more processes. This makes a clear assignment difficult or even impossible. Additionally, a service may not terminate correctly, leaving some children alive.

systemd solves this problem by placing each service into its own cgroup. cgroups are a kernel feature that allows aggregating processes and all their children into hierarchical organized groups. systemd names each cgroup after its service. Since a non-privileged process is not allowed to leave its cgroup, this provides an effective way to label all processes spawned by a service with the name of the service.

To list all processes belonging to a service, use the command systemd-cgls. The result will look like the following (shortened) example:

Example 10.3: List all Processes Belonging to a Service
root # systemd-cgls --no-pager
├─1 /usr/lib/systemd/systemd --switched-root --system --deserialize 20
├─user.slice
│ └─user-1000.slice
│   ├─session-102.scope
│   │ ├─12426 gdm-session-worker [pam/gdm-password]
│   │ ├─15831 gdm-session-worker [pam/gdm-password]
│   │ ├─15839 gdm-session-worker [pam/gdm-password]
│   │ ├─15858 /usr/lib/gnome-terminal-server

[...]

└─system.slice
  ├─systemd-hostnamed.service
  │ └─17616 /usr/lib/systemd/systemd-hostnamed
  ├─cron.service
  │ └─1689 /usr/sbin/cron -n
  ├─ntpd.service
  │ └─1328 /usr/sbin/ntpd -p /var/run/ntp/ntpd.pid -g -u ntp:ntp -c /etc/ntp.conf
  ├─postfix.service
  │ ├─ 1676 /usr/lib/postfix/master -w
  │ ├─ 1679 qmgr -l -t fifo -u
  │ └─15590 pickup -l -t fifo -u
  ├─sshd.service
  │ └─1436 /usr/sbin/sshd -D

[...]

See Book “System Analysis and Tuning Guide”, Chapter 9 “Kernel Control Groups” for more information about cgroups.

10.6.7 Terminating Services (Sending Signals)

As explained in Section 10.6.6, “Kernel Control Groups (cgroups)”, it is not always possible to assign a process to its parent service process in a System V init system. This makes it difficult to terminate a service and all of its children. Child processes that have not been terminated will remain as zombie processes.

systemd's concept of confining each service into a cgroup makes it possible to clearly identify all child processes of a service and therefore allows you to send a signal to each of these processes. Use systemctl kill to send signals to services. For a list of available signals refer to man 7 signals.

Sending SIGTERM to a Service

SIGTERM is the default signal that is sent.

systemctl kill MY_SERVICE
Sending SIGNAL to a Service

Use the -s option to specify the signal that should be sent.

systemctl kill -s SIGNAL MY_SERVICE
Selecting Processes

By default the kill command sends the signal to all processes of the specified cgroup. You can restrict it to the control or the main process. The latter is for example useful to force a service to reload its configuration by sending SIGHUP:

systemctl kill -s SIGHUP --kill-who=main MY_SERVICE
Warning
Warning: Terminating or Restarting the D-Bus Service is Not Supported

The D-Bus service is the message bus for communication between systemd clients and the systemd manager that is running as pid 1. Even though dbus is a standalone daemon, it is an integral part of the init infrastructure.

Terminating dbus or restarting it in the running system is similar to an attempt to terminate or restart pid 1. It will break systemd client-server communication and make most systemd functions unusable.

Therefore, terminating or restarting dbus is neither recommended nor supported.

10.6.8 Debugging Services

By default, systemd is not overly verbose. If a service was started successfully, no output will be produced. In case of a failure, a short error message will be displayed. However, systemctl status provides means to debug start-up and operation of a service.

systemd comes with its own logging mechanism (The Journal) that logs system messages. This allows you to display the service messages together with status messages. The status command works similar to tail and can also display the log messages in different formats, making it a powerful debugging tool.

Show Service Start-Up Failure

Whenever a service fails to start, use systemctl status MY_SERVICE to get a detailed error message:

root # systemctl start apache2
Job failed. See system journal and 'systemctl status' for details.
root # systemctl status apache2
   Loaded: loaded (/usr/lib/systemd/system/apache2.service; disabled)
   Active: failed (Result: exit-code) since Mon, 04 Jun 2012 16:52:26 +0200; 29s ago
   Process: 3088 ExecStart=/usr/sbin/start_apache2 -D SYSTEMD -k start (code=exited, status=1/FAILURE)
   CGroup: name=systemd:/system/apache2.service

Jun 04 16:52:26 g144 start_apache2[3088]: httpd2-prefork: Syntax error on line
205 of /etc/apache2/httpd.conf: Syntax error on li...alHost>
Show Last N Service Messages

The default behavior of the status subcommand is to display the last ten messages a service issued. To change the number of messages to show, use the --lines=N parameter:

systemctl status ntp
systemctl --lines=20 status ntp
Show Service Messages in Append Mode

To display a live stream of service messages, use the --follow option, which works like tail -f:

systemctl --follow status ntp
Messages Output Format

The --output=MODE parameter allows you to change the output format of service messages. The most important modes available are:

short

The default format. Shows the log messages with a human readable time stamp.

verbose

Full output with all fields.

cat

Terse output without time stamps.

10.7 More Information

For more information on systemd refer to the following online resources:

Homepage

http://www.freedesktop.org/wiki/Software/systemd

systemd for Administrators

Lennart Pöttering, one of the systemd authors, has written a series of blog entries (13 at the time of writing this chapter). Find them at http://0pointer.de/blog/projects.