Jump to contentJump to page navigation: previous page [access key p]/next page [access key n]
System Analysis and Tuning Guide › Basics › General notes on system tuning
ContentsContents
System Analysis and Tuning Guide
  1. Preface
  2. I Basics
    1. 1 General notes on system tuning
  3. II System monitoring
    1. 2 System monitoring utilities
    2. 3 System log files
  4. III Kernel monitoring
    1. 4 SystemTap—filtering and analyzing system data
    2. 5 Kernel probes
    3. 6 Hardware-based performance monitoring with Perf
    4. 7 OProfile—system-wide profiler
    5. 8 Dynamic debug—kernel debugging messages
  5. IV Resource management
    1. 9 General system resource management
    2. 10 Kernel control groups
    3. 11 Automatic Non-Uniform Memory Access (NUMA) balancing
    4. 12 Power management
  6. V Kernel tuning
    1. 13 Tuning I/O performance
    2. 14 Tuning the task scheduler
    3. 15 Tuning the memory management subsystem
    4. 16 Tuning the network
  7. VI Handling system dumps
    1. 17 Tracing tools
    2. 18 Kexec and Kdump
    3. 19 Using systemd-coredump to debug application crashes
  8. VII Synchronized clocks with Precision Time Protocol
    1. 20 Precision Time Protocol
  9. A GNU licenses
Navigation←→
System Analysis and Tuning Guide › Basics › General notes on system tuning
Top
←→
Applies to openSUSE Leap 15.6

1 General notes on system tuning Edit source

Abstract

This manual discusses how to find the reasons for performance problems and provides means to solve these problems. Before you start tuning your system, you should make sure you have ruled out common problems and have found the cause for the problem. You should also have a detailed plan on how to tune the system, because applying random tuning tips does not help and could make things worse.

1.1 Be sure what problem to solve
1.2 Rule out common problems
1.3 Finding the bottleneck
1.4 Step-by-step tuning
Procedure 1.1: General approach when tuning a system

  1. Specify the problem that needs to be solved.

  2. In case the degradation is new, identify any recent changes to the system.

  3. Identify why the issue is considered a performance problem.

  4. Specify a metric that can be used to analyze performance. This metric could for example be latency, throughput, the maximum number of users that are simultaneously logged in, or the maximum number of active users.

  5. Measure current performance using the metric from the previous step.

  6. Identify the subsystems where the application is spending the most time.

    1. Monitor the system and/or the application.

    2. Analyze the data, categorize where time is being spent.

  8. Tune the subsystem identified in the previous step.

  9. Remeasure the current performance without monitoring using the same metric as before.

  10. If performance is still not acceptable, start over with Step 3.

1.1 Be sure what problem to solve Edit source

Before starting to tuning a system, try to describe the problem as exactly as possible. A statement like “The system is slow!” is not a helpful problem description. For example, it could make a difference whether the system speed needs to be generally improved, or only at peak times.

Furthermore, make sure you can apply a measurement to your problem, otherwise you cannot verify if the tuning was a success or not. You should always be able to compare “before” and “after”. Which metrics to use depends on the scenario or application you are looking into. For example, relevant Web server metrics could be expressed in terms of the following:

Latency

The time to deliver a page

Throughput

Number of pages served per second or megabytes transferred per second

Active users

The maximum number of users that can download pages while still receiving pages within an acceptable latency

1.2 Rule out common problems Edit source

A performance problem often is caused by network or hardware problems, bugs or configuration issues. Make sure to rule out problems such as the ones listed below before attempting to tune your system:

  • Check the output of the systemd journal (see Book “Reference”, Chapter 11 “journalctl: query the systemd journal”) for unusual entries.

  • Check (using top or ps) whether a certain process misbehaves by eating up unusual amounts of CPU time or memory.

  • Check for network problems by inspecting /proc/net/dev.

  • In case of I/O problems with physical disks, make sure it is not caused by hardware problems (check the disk with the smartmontools) or by a full disk.

  • Ensure that background jobs are scheduled to be carried out in times the server load is low. Those jobs should also run with low priority (set via nice).

  • If the machine runs several services using the same resources, consider moving services to another server.

  • Last, make sure your software is up-to-date.

1.3 Finding the bottleneck Edit source

Finding the bottleneck is the hardest part when tuning a system. openSUSE Leap offers many tools to help you with this task. See Part II, “System monitoring” for detailed information on general system monitoring applications and log file analysis. If the problem requires a long-time in-depth analysis, the Linux kernel offers means to perform such analysis. See Part III, “Kernel monitoring” for coverage.

Once you have collected the data, it needs to be analyzed. First, inspect if the server's hardware (memory, CPU, bus) and its I/O capacities (disk, network) are sufficient. If these basic conditions are met, the system can benefit from tuning.

1.4 Step-by-step tuning Edit source

Make sure to carefully plan the tuning itself. It is of vital importance to only do one step at a time. Only by doing so can you measure whether the change made an improvement or even had a negative impact. Each tuning activity should be measured over a sufficient time period to ensure you can do an analysis based on significant data. If you cannot measure a positive effect, do not make the change permanent. Chances are that it can have a negative effect in the future.

→Part II System monitoring←Part I Basics
Print this page

© 2024 SUSE