Jump to contentJump to page navigation: previous page [access key p]/next page [access key n]
Applies to openSUSE Leap 15.1

18 Using systemd-coredump to Debug Application Crashes

systemd-coredump collects and displays kernel core dumps, for analyzing application crashes. When a process crashes (or all processes belonging to an application), its default is to log the core dump to the systemd journal, including a backtrace if possible, and to store the core dump in a file in /var/lib/systemd/coredump. You also have the option to examine the dump file with other tools such as gdb or crash (see Section 17.8, “Analyzing the Crash Dump”). There is an option to not store core dumps, but to log only to the journal, which may be useful to minimize the collection and storage of sensitive information.

18.1 Use and Configuration

systemd-coredump is enabled and ready to run by default. The default configuration is in /etc/systemd/coredump.conf:

[Coredump]
#Storage=external
#Compress=yes
#ProcessSizeMax=2G
#ExternalSizeMax=2G
#JournalSizeMax=767M
#MaxUse=
#KeepFree=

The following example shows how to use Vim for simple testing, by creating a segfault to generate journal entries and a core dump.

Procedure 18.1: Creating a core dump with Vim
  1. Enable the debuginfo-pool and debuginfo-update repositories

  2. Install vim-debuginfo

  3. Launch vim testfile and type a few characters

  4. Get the PID and generate a segfault:

    tux > ps ax | grep vim
    2345 pts/3    S+     0:00 vim testfile               
                     
    root # kill -s SIGSEGV 2345

    Vim will emit error messages:

    Vim: Caught deadly signal SEGV
    Vim: Finished.
    Segmentation fault (core dumped)
  5. List your core dumps, then examine them:

    root # coredumpctl
    TIME                        PID  UID  GID SIG PRESENT EXE
    Wed 2019-11-12 11:56:47 PST 2345 1000 100 11  *       /bin/vim
    
    root # coredumpctl info
    PID: 2345 (vim)
    UID: 0 (root)
    GID: 0 (root)
    Signal: 11 (SEGV)
    Timestamp: Wed 2019-11-12 11:58:05 PST
    Command Line: vim testfile
    Executable: /bin/vim
    Control Group: /user.slice/user-1000.slice/session-1.scope
        Unit: session-1.scope
        Slice: user-1000.slice
        Session: 1
        Owner UID: 1000 (tux)
        Boot ID: b5c251b86ab34674a2222cef102c0c88
        Machine ID: b43c44a64696799b985cafd95dc1b698
        Hostname: linux-uoch
        Coredump: /var/lib/systemd/coredump/core.vim.0.b5c251b86ab34674a2222cef102
        Message: Process 2345 (vim) of user 0 dumped core.
                    
             Stack trace of thread 2345:
             #0  0x00007f21dd87e2a7 kill (libc.so.6)
             #1  0x000000000050cb35 may_core_dump (vim)
             #2  0x00007f21ddbfec70 __restore_rt (libpthread.so.0)
             #3  0x00007f21dd92ea33 __select (libc.so.6)
             #4  0x000000000050b4e3 RealWaitForChar (vim)
             #5  0x000000000050b86b mch_inchar (vim)
    [...]

When you have multiple core dumps, coredumpctl info displays all of them. Filter them by PID, COMM (command), or EXE (full path to the executable), for example, all core dumps for Vim:

root # coredumpctl info /bin/vim

See a single core dump by PID:

root # coredumpctl info 2345

Output the selected core to gdb:

root # coredumpctl gdb 2345

The asterisk in the PRESENT column indicates that a stored core dump is present. If the field is empty there is no stored core dump, and coredumpctl retrieves crash information from the journal. You may control this behavior in /etc/systemd/coredump.conf with the Storage option:

  • Storage=none, core dumps are logged in the journal, but not stored. This is useful to minimize collecting and storing sensitive information, for example for General Data Protection Regulation (GDPR) compliance.

  • Storage=external, cores are stored in /var/lib/systemd/coredump

  • Storage=journal, cores are stored in the systemd journal

A new instance of systemd-coredump is invoked for every core dump, so configuration changes are applied with the next core dump, and there is no need to restart any services.

Core dumps are not preserved after a system restart. You may save them permanently with coredumpctl. The following example filters by the PID and stores the core in vim.dump:

root # coredumpctl -o vim.dump dump 2345

See man systemd-coredump, man coredumpctl, and man coredump.conf for complete command and option listings.

Print this page