Stress testing - ArchWiki

Jump to content

# Stress testing

From ArchWiki

Related articles

- Benchmarking

Stress testing is the process of running various work loads on a computer to assess its stability. This is often used to reliably check the stability of overclocked/undervolted hardware and monitor the thermal behavior of the system (e.g. maximum temperatures, throttling, noise levels). There are several programs available to stress test various parts of the system such as CPU, GPU, RAM, and storage, using different types of work loads.

## Stress testing tasks

The table below lists some stress testing software based on the kind of test and the overall intensity of the work load. It is important to stress test using mixed loads to verify stability under many use cases.

Warning Before proceeding, it is highly recommended that users have some means to monitor the system temperatures. See Sensors.

Work load | Tested hardware1 | Task | Description
--- | --- | --- | ---
Light2
CPU, storage | Updating patches | Custom script Refreshing hundreds of kernel patches in the OpenWRT project. See #Updating patches for OpenWRT.
CPU, storage | Writing a disk image | See #Writing to an image file.
RAM | Memory stressing | See #Memtest86+.
Realistic3
CPU, RAM, storage | Compilation | Parallel compilation is a good way to stress test the CPU. See #GCC.
CPU, RAM | Video encoding | ffmpeg, x264, handbrake-cli, etc. can be used to encode video. See #Video encoding.
CPU, GPU, RAM | Cryptocurrencies mining | xmrig is a CPU/GPU mine which uses different cryptocurrency mining algorithms to generate load on the GPU for stability and temperature testing. See Benchmarking#xmrig.
GPU | 3D rendering | unigine-heavenAUR is a GPU benchmark that runs in a loop. It is a decent stress test for GPUs. See Benchmarking#Unigine Engine.
Synthetic4 | CPU, RAM, storage | Synthetic stressing | stress is a simple CPU, memory, I/O, and disk workload generator implemented in C. See #stress.
CPU, RAM | Prime numbers calculation | mprimeAUR factors large numbers and is an excellent way to stress CPU and memory. See #MPrime.
CPU | Algebra calculation | linpackAUR - Linpack makes use of the BLAS (Basic Linear Algebra Subprograms) libraries for performing basic vector and matrix operations and is an excellent way to stress CPUs for stability. See #Linpack.
CPU | Pi decimals calculation | systesterAUR Systester is a multithreaded piece of software capable of deriving values of pi out to 128,000,000 decimal places. It has built in check for system stability. See #Systester.
RAM | Memory stressing | stressapptestAUR is a memory interface test.
CPU | Various | strainAUR A multi-function stress-testing utility. Written in Rust.

- 1 The main target of the test, virtually all testing will also involve the CPU and RAM to some extent.
- 2 Light tests do not push the components very hard (in terms of power/heat limits). These tests are still useful to test how the hardware behaves in lower power levels (P states), in particular for undervolted systems.
- 3 Realistic tests are based on real world workloads.
- 4 Synthetic tests are explicitly designed to torture the hardware as much as possible and may not be representative of real-world workloads.

Tip To ensure the stability of a system, it is recommended to run such tests for a long period of time, from a few hours to a few days, in different temperature conditions. If the room temperature can for example vary significantly between winter and summer time, this is something to be considered.

### Updating patches for OpenWRT

A good stability test of a low load workload is to run through updating the patch sets in the OpenWRT project. Follow these steps.

Note Several dependencies including git and curl are needed, most others should be provided by the dependencies of the base-devel meta package.

    $ git clone --depth 1 https://github.com/openwrt/openwrt.git
    $ cd openwrt
    $ mkdir -p staging_dir/host/bin
    $ cp /usr/bin/sed ./staging_dir/host/bin
    $ curl -Os https://raw.githubusercontent.com/KanjiMonster/maintainer-tools/master/update_kernel.sh
    $ chmod +x update_kernel.sh
    $ ./update_kernel.sh -v -u 6.6

### stress

stress performs a loop that calculates the square root of a random number in order to stress the CPU. It can run simultaneously several workers to load all the cores of a CPU for example. It can also generate memory, I/O or disk workload depending on the parameters passed. The FAQ provides examples and explanations.

To spawn 4 workers working on calculating a square root, use the command:

    $ stress --cpu 4

### s-tui

s-tui is terminal-based CPU stress and monitoring utility. It can both monitor CPU usage and temperatures and can also stress the CPU. It is a one stop shop tool where you can see all the information needed in one single screen, with a terminal graphical interface.

### MPrime

MPrime (also known as Prime95 in its Windows and MacOS implementation) is recognised universally as one de facto measure of system stability. MPrime under torture test mode will perform a series of very CPU intensive calculations and compare the values it gets to known good values.

The Linux implementation is called mprimeAUR.

To run mprime, simply open a shell and type "mprime":

    $ mprime

Note If using CPU frequency scaling, sometimes users need to manually set the processor to run with its highest multiplier because mprime uses a nice value that does not always trip the step-up in multiplier.

When the software loads, simply answer 'N' to the first question to begin the torture testing:

         Main Menu

     1.  Test/Primenet
     2.  Test/Workers
     3.  Test/Status
     4.  Test/Continue
     5.  Test/Exit
     6.  Advanced/Test
     7.  Advanced/Time
     8.  Advanced/P-1
     9.  Advanced/ECM
    10.  Advanced/Manual Communication
    11.  Advanced/Unreserve Exponent
    12.  Advanced/Quit Gimps
    13.  Options/CPU
    14.  Options/Resource Limits
    15.  Options/Preferences
    16.  Options/Torture Test
    17.  Options/Benchmark
    18.  Help/About
    19.  Help/About PrimeNet Server

There are several options for the torture test (menu option 16).

1. Smallest FFTs to test the CPU L1/L2 (high CPU stress)
2. Small FFTs to test the CPU L1/L2/L3 (maximum CPU stress)
3. Large FFTs to test the memory controller and RAM (medium CPU stress)
4. Blend is the default and constitutes a hybrid mode which stresses the CPU and RAM.

The "Customize settings (N)" prompt allows you to customize settings such as FFT size range and memory use. The default for Large and Blend is to use about 90% of the current amount of available memory, and to alternate between in-memory FFT and in-place FFT.

Errors will be reported should they occur both to stdout and to `~/results.txt` for review later. Many do not consider a system as 'stable' unless it can run the Large FFTs without error for a 24 hour period. (The stdout is more useful as it includes information on which worker produced the error, so consider using `tee` or `script` to save the stdout. By default, each core is assigned one worker, so weak cores can be easily identified.)

Example `~/results.txt`; note that the two runs from 26-June indicate a hardware failure. In this case, due to insufficient vcore to the CPU:

    [Sun Jun 26 20:10:35 2011]
    FATAL ERROR: Rounding was 0.5, expected less than 0.4
    Hardware failure detected, consult stress.txt file.
    FATAL ERROR: Rounding was 0.5, expected less than 0.4
    Hardware failure detected, consult stress.txt file.
    [Sat Aug 20 10:50:45 2011]
    Self-test 480K passed!
    Self-test 480K passed!
    [Sat Aug 20 11:06:02 2011]
    Self-test 128K passed!
    Self-test 128K passed!
    [Sat Aug 20 11:22:10 2011]
    Self-test 560K passed!
    Self-test 560K passed!
    ...

Note Users suspecting bad memory or memory controllers should try the blend tor large tests first as the small FFT test uses very little memory.

### Linpack

linpackAUR makes use of the BLAS (Basic Linear Algebra Subprograms) libraries for performing basic vector and matrix operations. It is an excellent way to stress CPUs for stability (only Intel CPUs are supported). After installation, users should copy `/usr/share/linpack/linpack.conf` to `~/.config/linpack.conf` and adjust it according to the amount of memory on the system.

### Systester

SystesterAUR (aka SuperPi for Windows) is available in both CLI and GUI version. It tests system stability by calculating up to 128 millions of Pi digits and includes error checking. Note that one can select from two different calculation algorithms: Quadratic Convergence of Borwein and Gauss-Legendre. The latter being the same method that the popular SuperPi for Windows uses.

A CLI example using 8 threads is given:

    $ systester-cli -gausslg 64M -threads 8

### Memtest86+

Use MemTest86 (proprietary) or Memtest86+ (GPL) to test your memory (RAM).

- The GPL version is available on the Arch Linux install image. It can be installed:
  - for EFI systems with memtest86+-efi,
  - or for BIOS systems with memtest86+.
- The proprietary version does not support BIOS. Install it with memtest86-efiAUR.
- After installation, its users can update GRUB; it will auto-detect the package and allow users to boot directly to it.
- Users of systemd-boot must manually configure a boot loader entry.

Tip

- A reliable source of the version history is the History of MemTest86 section in memtest86.com, in particular the section "2002 - 2004" and following. Notice the proprietary MemTest86 from version 5 through 7 claims to support both BIOS and UEFI, but they simply bundle old and new versions.
- Allowing tests to run for at least 10 cycles without errors is usually sufficient.

### Writing to an image file

A good stability test under a low load workload is using `dd` to format an image. This can be a physical disk or a loop mounted image. The script below uses mounted image and cycles through each core one-by-one. Note that you should adjust the variables in the top of script to match your system. By default the script will run the command just once per core. It can be easily customised to run on known-weak cores rather than scanning all core 0 through n by altering the for loop. Run the script as root.

    format-test.sh

    #!/bin/bash

    # define the path to store the image, recommended to be a tmpfs mounted location to avoid read/writes
    img=/scratch/image.img

    # define the mount point
    mnt=/mnt/loop

    # size of time arg to pass to truncate, make sure you select something less than the free memory on the system
    # see truncate --help for available options
    size=40G

    # defaults to 1 less than the number of virtual cores, manually redefine if desired
    max=$(($(nproc) - 1))

    if [[ ! -f $img ]]; then
      truncate -s $size $img
      mkfs.ext4 $img
      [[ -d $mnt ]] || mkdir -p $mnt
      if ! mountpoint -q $mnt; then
        mount -o loop $img $mnt || exit 1
      fi
    fi

    for i in $(eval echo "{0..$max}"); do
      echo "using core $i of $max"
      taskset -c "$i" time dd if=/dev/zero of=$mnt/zerofill status=progress
    done

    umount $mnt
    rm $img

### GCC

Parallel compilation using GCC (or other compilers) will generate a heavy load on the CPU and memory. To avoid I/O bottlenecking, compile on an SSD or in a tmpfs.

A good example would be compiling the kernel: see Kernel/Arch build system for detailed instructions, run `makepkg -sf MAKEFLAGS="-j$(nproc)"` at Kernel/Arch build system#Compiling.

### Video encoding

Most video encoders are highly parallel and are designed to use most of a CPU's capabilities. The example below will encode noise using x265, and discard the result. This will heavily load the CPU.

    ffmpeg -y -f rawvideo -video_size 1920x1080 -pixel_format yuv420p -framerate 60 -i /dev/urandom -c:v libx265 -preset placebo -f matroska /dev/null

## Discovering errors

Some stressing applications like #MPrime, #Linpack, or systester have built in consistency checks to discover errors due to non-matching results. This is more sensitive than looking for kernel errors and should be preferred to it.

A more general method for measuring hardware instabilities can be found in the kernel itself, through hardware error detection or outright failures. Most mistakes in data transfer or calculation do not trigger a failure dramatic enough to be reported as a machine check error.

### Using journalctl manually searching

Simply filter the journal to identify machine check errors:

    # journalctl -k --grep=mce

### Using rasdaemon

Alternatively, build rasdaemonAUR and start the included service. Rasdaemon keeps an sqlite database of errors which can be queried with the included `ras-mc-ctl` binary. Example:

    # ras-mc-ctl --errors
    No Memory errors.
    No PCIe AER errors.
    No ARM processor errors.
    No Extlog errors.
    No devlink errors.
    MCE events:
    1 2025-06-23 08:24:48 -0400 error: Corrected error, no action required., CPU 2, bank Load Store Unit (bank=0), mcg mcgstatus=0, mci CECC, mca DC data error type 2. Ext Err Code: 13 Memory Error 'mem-tx: evict, tx: data, level: L1', mcgcap=0x00000107, status=0x9c204000000d0175, addr=0x185dd56627, misc=0xd01a000100000000, walltime=0x68594791, cpu=0x00000002, cpuid=0x00b40f40, apicid=0x00000004, microcode=0x0b404032

### Tracking errors back to cores

When optimizing settings, it becomes critical to understand stability on a per-core basis which is the context many BIOS overclocking sections use. A physical core can contain multiple logical CPUs.

Consider the following error from the journal on a Ryzen 9950X (16-physical cores/32-CPUs):

    [Hardware Error]: Corrected error, no action required.
    [Hardware Error]: CPU:5 (1a:44:0) MC0_STATUS[-|CE|MiscV|AddrV|-|-|SyndV|CECC|-|-|-]: 0x9c204000000d0175
    [Hardware Error]: Error Addr: 0x000000185d960d07
    [Hardware Error]: IPID: 0x000000b000000000, Syndrome: 0x0000000b1a173405
    [Hardware Error]: Load Store Unit Ext. Error Code: 13
    [Hardware Error]: cache level: L1, tx: DATA, mem-tx: EV

The mapping of logical to physical processing units (PUs) can be obtained with lstopo-no-graphics from hwloc:

    $ lstopo-no-graphics --only PU

    PU L#0 (P#0)
    PU L#1 (P#10)
    PU L#2 (P#1)
    PU L#3 (P#11)
    PU L#4 (P#2)
    PU L#5 (P#12)
    PU L#6 (P#3)
    PU L#7 (P#13)
    PU L#8 (P#4)
    PU L#9 (P#14)
    PU L#10 (P#5)
    PU L#11 (P#15)
    PU L#12 (P#6)
    PU L#13 (P#16)
    PU L#14 (P#7)
    PU L#15 (P#17)
    PU L#16 (P#8)
    PU L#17 (P#18)
    PU L#18 (P#9)
    PU L#19 (P#19)

Category:

- CPU

Stress testing
Add topic