Correct a test failure in netboot_test and file_filter_test caused by an
issue cleaning up the tmp directory created for netboot. Netboot creates
a subdirectory in the tmp folder that causes the rmdir to fail - so
cleanup the subdirectory first.
Fixes: 1d59f39b5f1b (tests/util/grub-shell: Remove the work directory on successful run and debug is not on)
Signed-off-by: Andrew Hamilton <adhamilt@gmail.com>
Tested-by: Leo Sandoval <lsandova@redhat.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
This removes a lot of empty grub-shell working directories in the TMPDIR directory.
Signed-off-by: Thomas Schmitt <scdbackup@gmx.net>
Signed-off-by: Glenn Washburn <development@efficientek.com>
Tested-by: Thomas Schmitt <scdbackup@gmx.net>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
Fix a regression where qemuopts was mistakenly defaulted to the empty
string. This prevents the sending of arbitrary QEMU options to tests,
which may be desirable for overriding the machine type. There was a
concern that allowing the tester to accept arbitrary options would add
headaches for another developer trying to diagnose why a test failed on
the testers machine because he could not be sure if any additional
options were passed to make the test fail. However, all the options are
recorded in the run.sh generated script, so this concern is unwarranted.
Fixes: 6d729ced70 (tests/util/grub-shell: Add $GRUB_QEMU_OPTS to run.sh to easily see unofficial QEMU arguments)
Signed-off-by: Glenn Washburn <development@efficientek.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
For the tpm2_key_protector module, the TCG2 command submission function
is the only difference between a QEMU instance and grub-emu. To test
TPM2 key unsealing with a QEMU instance, it requires an extra OS image
to invoke grub-protect to seal the LUKS key, rather than a simple
grub-shell rescue CD image. On the other hand, grub-emu can share the
emulated TPM2 device with the host, so that we can seal the LUKS key on
host and test key unsealing with grub-emu.
This test script firstly creates a simple LUKS image to be loaded as a
loopback device in grub-emu. Then an emulated TPM2 device is created by
"swtpm chardev" and PCR 0 and 1 are extended.
There are several test cases in the script to test various settings. Each
test case uses grub-protect or tpm2-tools to seal the LUKS password
with PCR 0 and PCR 1. Then grub-emu is launched to load the LUKS image,
try to mount the image with tpm2_key_protector_init and cryptomount, and
verify the result.
Based on the idea from Michael Chang.
Cc: Michael Chang <mchang@suse.com>
Cc: Stefan Berger <stefanb@linux.ibm.com>
Cc: Glenn Washburn <development@efficientek.com>
Signed-off-by: Gary Lin <glin@suse.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
Reviewed-by: Stefan Berger <stefanb@linux.ibm.com>
Tested-by: Stefan Berger <stefanb@linux.ibm.com>
First look for firmware files in the source directory and then, if not
found, look for them in locations where Debian installs them. Prefer to
use the unified firmware file and, if not found, use the pflash firmware
files split in to code and variables. By looking for files in the source
directory first, system firmware files can be overridden and it can be
ensured that the tests can be run regardless of the distro or where the
system firmware files are stored. If no firmware files are found, print
an error message and exit with error.
If a firmware VARS file is found, use it with snapshot mode enabled, which
makes the VARS writable to the virtual machine, but does not write back
the changes to the file. This allows using the readonly system VARS file
without copying it or using it in readonly mode, which causes the ARM
machine to fail. This also gives tests effectively their own ephemeral VARS
file that can be written to without causing side-effects for other tests.
Signed-off-by: Glenn Washburn <development@efficientek.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
According to the OVMF whitepaper [1]:
IMPORTANT: Never pass OVMF.fd to qemu with the -bios option. That option
maps the firmware image as ROM into the guest's address space, and forces
OVMF to emulate non-volatile variables with a fallback driver that is
bound to have insufficient and confusing semantics.
Use the pflash interface instead. Currently the unified firmware file is
used, which contains both firmware code and variable sections. By enabling
snapshot on the pflash device, the firmware can be loaded in such a way
that variables can be written to without writing to the backing file.
Since pflash does no searching for firmware paths that are not absolute,
unlike the -bios option, also make firmware paths absolute. Additionally,
update the previous firmware paths or file names that did not correspond to
ones installed by Debian.
Use the q35 machine, instead of the default i440fx, for i386-efi because
the default machine type does not emulate a flash device, which is now
needed to load the firmware.
[1] http://www.linux-kvm.org/downloads/lersek/ovmf-whitepaper-c770f8c.txt
Signed-off-by: Glenn Washburn <development@efficientek.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
Allow using GDB to debug a failing QEMU test. This output does not cause
issues for tests because it happens before the trim line, and so will be
ignored.
Signed-off-by: Glenn Washburn <development@efficientek.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
In certain firmwares, e.g. OVMF, the RNG protocol is not enabled unless
there is an RNG device. When not enabled, GRUB fails to initialize the
stack guard with random bytes. For testing, this is not a big issue, but
there have been bugs found in the initialization. So turn this on for EFI
platforms to catch any regressions.
Signed-off-by: Glenn Washburn <development@efficientek.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
While here, move "-qemu=*" case to be next to the "--qemu-opts=*" case.
This causes no change in logic, but is more logically located.
Signed-off-by: Glenn Washburn <development@efficientek.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
This allows an invocation of grub-shell to set the value of debug regardless
of the global default environment variable GRUB_SHELL_DEFAULT_DEBUG.
Signed-off-by: Glenn Washburn <development@efficientek.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
Since this is fairly verbose output, do not enable first level of debug
is turned on.
Signed-off-by: Glenn Washburn <development@efficientek.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
I ran the test suite on a 3A5000 desktop, a LoongArch architecture machine,
using Archlinux for LoongArch distro, see https://github.com/loongarchlinux.
Some software versions are:
* linux 6.3.0-rc4
* gcc 13.0.1 20230312
* binutils 2.40
* qemu 7.2.0
The test results of running "make check" with qemu 7.2 are as follows:
=================================
GRUB 2.11: ./test-suite.log
=================================
# TOTAL: 85
# PASS: 73
# SKIP: 8
# XFAIL: 0
# FAIL: 2
# XPASS: 0
# ERROR: 2
.. contents:: :depth: 2
ERROR: f2fs_test
================
mount: /tmp/grub-fs-tester.20230418175640563815408.f2fs.UDs/f2fs_rw: unknown filesystem type 'f2fs'.
dmesg(1) may have more information after failed mount system call.
MOUNT FAILED.
ERROR f2fs_test (exit status: 99)
FAIL: hfs_test
==============
recode: Request `utf8..macroman' is erroneous
mkfs.hfs: name required with -v option
FAIL hfs_test (exit status: 1)
ERROR: zfs_test
===============
zpool not installed; cannot test zfs.
ERROR zfs_test (exit status: 99)
SKIP: pata_test
===============
SKIP pata_test (exit status: 77)
SKIP: ahci_test
===============
SKIP ahci_test (exit status: 77)
SKIP: uhci_test
===============
SKIP uhci_test (exit status: 77)
SKIP: ohci_test
===============
SKIP ohci_test (exit status: 77)
SKIP: ehci_test
===============
SKIP ehci_test (exit status: 77)
SKIP: fddboot_test
==================
SKIP fddboot_test (exit status: 77)
SKIP: netboot_test
==================
SKIP netboot_test (exit status: 77)
SKIP: pseries_test
==================
SKIP pseries_test (exit status: 77)
FAIL: grub_func_test
====================
WARNING: Image format was not specified for '/tmp/grub-shell.HeTAD8Ty3U/grub.iso' and probing guessed raw.
Automatically detecting the format is dangerous for raw images, write operations on block 0 will be restricted.
Specify the 'raw' format explicitly to remove the restrictions.
Functional test failure: shift_test:
...
gfxterm_menu_640x480xi16:3 failed: 0xce34981e vs 0xd9f04953
tests/video_checksum.c:checksum:615: assert failed: 0 Checksum
gfxterm_menu_640x480xi16:2 failed: 0xa8fb749d vs 0xbf3fa5d0
tests/video_checksum.c:checksum:615: assert failed: 0 Checksum
gfxterm_menu_640x480xi16:1 failed: 0xce34981e vs 0xd9f04953
gfxterm_menu: FAIL
...
videotest_checksum:
videotest_checksum: PASS
exfctest:
exfctest: PASS
TEST FAILURE
FAIL grub_func_test (exit status: 1)
We got 2 errors:
* f2fs_test
The kernel uses 16k pages, causing failures when loading the f2fs kernel module,
see https://github.com/torvalds/linux/blob/master/fs/f2fs/super.c#L4670
This error can be ignored.
* zfs_test
zfs does not support the LoongArch architecture and is not compatible with the
6.3 kernel.
This error can be ignored.
We got 2 failures:
* hfs_test
I use recode 3.7.14-1 on Archlinux, running `recode -l` gives no output `MacRoman`,
so we get this error.
On Linux systems that support LoongArch, there is currently no need to use HFS,
so this failure can be ignored.
* grub_func_test
I don't know the reason for this failure. I guess it may be related to qemu's edk2.
In the previous review, I was told that the failure here is the expected behavior.
So, we can ignore this failure.
Signed-off-by: Xiaotian Wu <wuxiaotian@loongson.cn>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
When re-running a failed test, even the non-standard grub-shell QEMU
arguments should be preserved in the run.sh to more precisely replay
the failed test run.
Signed-off-by: Glenn Washburn <development@efficientek.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
Now it becomes trivial to re-run a test from the output in its working
directory. This also makes it easy to send a reproducible failing test to
the mailing list. This has allowed a refactor so that the duplicated code
to call QEMU has be condensed (e.g. the use of timeout and file descriptor
redirection). The run.sh script will pass any arguments given to QEMU.
This allows QEMU to be easily started in a state ready for GDB to be
attached.
Signed-off-by: Glenn Washburn <development@efficientek.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
This allows turning on shell tracing for grub-shell and grub-fs-tester
when its not practical or not possible to use command line arguments
(e.g. from "make check"). Turn on tracing when the envvar is an integer
greater than 1, since these can generate a lot of output. Since this
change uses the environment variables to set the default value for debug
in grub-shell, this allows enabling grub-shell's debug mode which will
preserve various generated output files that are helpful for debugging
tests.
Signed-off-by: Glenn Washburn <development@efficientek.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
This allows test case scripts to use the appropriate halt command for
the built architecture to end execution early. Otherwise, test case
scripts have no way to know the appropriate mechanism for halting the
test case early.
Signed-off-by: Glenn Washburn <development@efficientek.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
When turning on shell tracing the trim line will be output before we
actually want to start the trim. However, in this case the trim line never
starts from the beginning of the line. So start trimming from the correct
line by matching from the beginning of the line.
Signed-off-by: Glenn Washburn <development@efficientek.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
This will be useful for tests that have unwanted output from setup. This is
not documented because its only intended to be internal at the moment. Also,
--no-trim is allowed to explicitly turn off trim.
Signed-off-by: Glenn Washburn <development@efficientek.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
This keeps the generated files to aid in diagnosing the source of the failure.
Signed-off-by: Glenn Washburn <development@efficientek.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
This allows us to test if unexpected output in test scripts is because of
a bug in GRUB, because there was an error in QEMU, or QEMU was killed due
to a timeout.
Signed-off-by: Glenn Washburn <development@efficientek.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
This allows seeing full QEMU output of grub-shell, which can be invaluable
when debugging failing tests.
Signed-off-by: Glenn Washburn <development@efficientek.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
When running tests there are many invocations of grub-shell, and because
the output files are all random names in the same tmp directory, it
becomes more work to figure out which files went with which grub-shell
invocations. So all generated files from one invocation of grub-shell
are put into a randomly named directory, so as not to collide with other
grub-shell invocations. And now that the generated files can be put in
a location where they will not get stepped on, and they can be named
sensible names.
Signed-off-by: Glenn Washburn <development@efficientek.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
The previous behavior ignored an error and the output from grub-mkrescue.
This made it difficult to discover that grub-mkrescue was the reason that
tests which rely on grub-shell were failing. Even after discovering
grub-mkrescue was the culprit, there was no output to indicate why it was
failing. It turns out that grub-mkrescue is a thin wrapper around xorriso.
So if you do not have xorriso installed it will fail with an error message
about not being able to find xorriso.
This change will allow grub-mkrescue output to be written to stderr, only
if grub-mkrescue fails. If grub-mkrescue succeeds, there will be no output
from grub-mkrescue so as not to interfere with the functioning of tests.
This change should have no effect on the running of tests or other uses of
grub-shell as it only modifies the error path.
Also, if grub-mkrescue fails, the script exits early. Since grub-shell
needs the ISO image created by grub-mkresue to boot the QEMU instance,
a failure here should be considered fatal.
Signed-off-by: Glenn Washburn <development@efficientek.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
By using shell variable that are set once by the expansion of an autoconf
variable, the resulting shell script is more easily moved and modified
from the build/install directory it was generated for. The resulting
script is more readable as well.
Signed-off-by: Glenn Washburn <development@efficientek.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
This breaks the tests on pseries - just restrict it to x86 platforms
that don't specify an EFI.
Signed-off-by: Daniel Axtens <dja@axtens.net>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
A recent refactoring of CUDA command code has exposed a bug in OpenBIOS [1]
which was causing system powerdown and system reset to fail, thus causing
the QEMU instance to hang. This in turn caused the grub-shell command to
timeout causing it to return an error code when the test actually completed
successfully.
Since it could be a while before the patch fixing this issue in OpenBIOS
filters down to the average distro, switch to PMU to allow powerdowns and
reboots to work as expected.
[1] https://gitlab.com/qemu-project/qemu/-/issues/624
Signed-off-by: Glenn Washburn <development@efficientek.com>
Signed-off-by: Daniel Axtens <dja@axtens.net>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
SeaBIOS 1.11.0 added support for VGA emulation over a serial port, which
interferes with grub-shell. Turn it off.
Signed-off-by: Colin Watson <cjwatson@ubuntu.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
Copying the themes directory in grub-shell isn't
parallel-test-friendly and breaks on the second test when the source
directory is read-only (as in "make distcheck"). Instead, add a
hidden --themes-directory option to grub-mkrescue et al, and use it
in grub-shell.
