I ran the test suite on a 3A5000 desktop, a LoongArch architecture machine,
using Archlinux for LoongArch distro, see https://github.com/loongarchlinux.
Some software versions are:
* linux 6.3.0-rc4
* gcc 13.0.1 20230312
* binutils 2.40
* qemu 7.2.0
The test results of running "make check" with qemu 7.2 are as follows:
=================================
GRUB 2.11: ./test-suite.log
=================================
# TOTAL: 85
# PASS: 73
# SKIP: 8
# XFAIL: 0
# FAIL: 2
# XPASS: 0
# ERROR: 2
.. contents:: :depth: 2
ERROR: f2fs_test
================
mount: /tmp/grub-fs-tester.20230418175640563815408.f2fs.UDs/f2fs_rw: unknown filesystem type 'f2fs'.
dmesg(1) may have more information after failed mount system call.
MOUNT FAILED.
ERROR f2fs_test (exit status: 99)
FAIL: hfs_test
==============
recode: Request `utf8..macroman' is erroneous
mkfs.hfs: name required with -v option
FAIL hfs_test (exit status: 1)
ERROR: zfs_test
===============
zpool not installed; cannot test zfs.
ERROR zfs_test (exit status: 99)
SKIP: pata_test
===============
SKIP pata_test (exit status: 77)
SKIP: ahci_test
===============
SKIP ahci_test (exit status: 77)
SKIP: uhci_test
===============
SKIP uhci_test (exit status: 77)
SKIP: ohci_test
===============
SKIP ohci_test (exit status: 77)
SKIP: ehci_test
===============
SKIP ehci_test (exit status: 77)
SKIP: fddboot_test
==================
SKIP fddboot_test (exit status: 77)
SKIP: netboot_test
==================
SKIP netboot_test (exit status: 77)
SKIP: pseries_test
==================
SKIP pseries_test (exit status: 77)
FAIL: grub_func_test
====================
WARNING: Image format was not specified for '/tmp/grub-shell.HeTAD8Ty3U/grub.iso' and probing guessed raw.
Automatically detecting the format is dangerous for raw images, write operations on block 0 will be restricted.
Specify the 'raw' format explicitly to remove the restrictions.
Functional test failure: shift_test:
...
gfxterm_menu_640x480xi16:3 failed: 0xce34981e vs 0xd9f04953
tests/video_checksum.c:checksum:615: assert failed: 0 Checksum
gfxterm_menu_640x480xi16:2 failed: 0xa8fb749d vs 0xbf3fa5d0
tests/video_checksum.c:checksum:615: assert failed: 0 Checksum
gfxterm_menu_640x480xi16:1 failed: 0xce34981e vs 0xd9f04953
gfxterm_menu: FAIL
...
videotest_checksum:
videotest_checksum: PASS
exfctest:
exfctest: PASS
TEST FAILURE
FAIL grub_func_test (exit status: 1)
We got 2 errors:
* f2fs_test
The kernel uses 16k pages, causing failures when loading the f2fs kernel module,
see https://github.com/torvalds/linux/blob/master/fs/f2fs/super.c#L4670
This error can be ignored.
* zfs_test
zfs does not support the LoongArch architecture and is not compatible with the
6.3 kernel.
This error can be ignored.
We got 2 failures:
* hfs_test
I use recode 3.7.14-1 on Archlinux, running `recode -l` gives no output `MacRoman`,
so we get this error.
On Linux systems that support LoongArch, there is currently no need to use HFS,
so this failure can be ignored.
* grub_func_test
I don't know the reason for this failure. I guess it may be related to qemu's edk2.
In the previous review, I was told that the failure here is the expected behavior.
So, we can ignore this failure.
Signed-off-by: Xiaotian Wu <wuxiaotian@loongson.cn>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
When re-running a failed test, even the non-standard grub-shell QEMU
arguments should be preserved in the run.sh to more precisely replay
the failed test run.
Signed-off-by: Glenn Washburn <development@efficientek.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
Now it becomes trivial to re-run a test from the output in its working
directory. This also makes it easy to send a reproducible failing test to
the mailing list. This has allowed a refactor so that the duplicated code
to call QEMU has be condensed (e.g. the use of timeout and file descriptor
redirection). The run.sh script will pass any arguments given to QEMU.
This allows QEMU to be easily started in a state ready for GDB to be
attached.
Signed-off-by: Glenn Washburn <development@efficientek.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
This allows turning on shell tracing for grub-shell and grub-fs-tester
when its not practical or not possible to use command line arguments
(e.g. from "make check"). Turn on tracing when the envvar is an integer
greater than 1, since these can generate a lot of output. Since this
change uses the environment variables to set the default value for debug
in grub-shell, this allows enabling grub-shell's debug mode which will
preserve various generated output files that are helpful for debugging
tests.
Signed-off-by: Glenn Washburn <development@efficientek.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
This allows test case scripts to use the appropriate halt command for
the built architecture to end execution early. Otherwise, test case
scripts have no way to know the appropriate mechanism for halting the
test case early.
Signed-off-by: Glenn Washburn <development@efficientek.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
When turning on shell tracing the trim line will be output before we
actually want to start the trim. However, in this case the trim line never
starts from the beginning of the line. So start trimming from the correct
line by matching from the beginning of the line.
Signed-off-by: Glenn Washburn <development@efficientek.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
This will be useful for tests that have unwanted output from setup. This is
not documented because its only intended to be internal at the moment. Also,
--no-trim is allowed to explicitly turn off trim.
Signed-off-by: Glenn Washburn <development@efficientek.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
This keeps the generated files to aid in diagnosing the source of the failure.
Signed-off-by: Glenn Washburn <development@efficientek.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
This allows us to test if unexpected output in test scripts is because of
a bug in GRUB, because there was an error in QEMU, or QEMU was killed due
to a timeout.
Signed-off-by: Glenn Washburn <development@efficientek.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
This allows seeing full QEMU output of grub-shell, which can be invaluable
when debugging failing tests.
Signed-off-by: Glenn Washburn <development@efficientek.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
When running tests there are many invocations of grub-shell, and because
the output files are all random names in the same tmp directory, it
becomes more work to figure out which files went with which grub-shell
invocations. So all generated files from one invocation of grub-shell
are put into a randomly named directory, so as not to collide with other
grub-shell invocations. And now that the generated files can be put in
a location where they will not get stepped on, and they can be named
sensible names.
Signed-off-by: Glenn Washburn <development@efficientek.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
The previous behavior ignored an error and the output from grub-mkrescue.
This made it difficult to discover that grub-mkrescue was the reason that
tests which rely on grub-shell were failing. Even after discovering
grub-mkrescue was the culprit, there was no output to indicate why it was
failing. It turns out that grub-mkrescue is a thin wrapper around xorriso.
So if you do not have xorriso installed it will fail with an error message
about not being able to find xorriso.
This change will allow grub-mkrescue output to be written to stderr, only
if grub-mkrescue fails. If grub-mkrescue succeeds, there will be no output
from grub-mkrescue so as not to interfere with the functioning of tests.
This change should have no effect on the running of tests or other uses of
grub-shell as it only modifies the error path.
Also, if grub-mkrescue fails, the script exits early. Since grub-shell
needs the ISO image created by grub-mkresue to boot the QEMU instance,
a failure here should be considered fatal.
Signed-off-by: Glenn Washburn <development@efficientek.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
By using shell variable that are set once by the expansion of an autoconf
variable, the resulting shell script is more easily moved and modified
from the build/install directory it was generated for. The resulting
script is more readable as well.
Signed-off-by: Glenn Washburn <development@efficientek.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
This breaks the tests on pseries - just restrict it to x86 platforms
that don't specify an EFI.
Signed-off-by: Daniel Axtens <dja@axtens.net>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
A recent refactoring of CUDA command code has exposed a bug in OpenBIOS [1]
which was causing system powerdown and system reset to fail, thus causing
the QEMU instance to hang. This in turn caused the grub-shell command to
timeout causing it to return an error code when the test actually completed
successfully.
Since it could be a while before the patch fixing this issue in OpenBIOS
filters down to the average distro, switch to PMU to allow powerdowns and
reboots to work as expected.
[1] https://gitlab.com/qemu-project/qemu/-/issues/624
Signed-off-by: Glenn Washburn <development@efficientek.com>
Signed-off-by: Daniel Axtens <dja@axtens.net>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
SeaBIOS 1.11.0 added support for VGA emulation over a serial port, which
interferes with grub-shell. Turn it off.
Signed-off-by: Colin Watson <cjwatson@ubuntu.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
Copying the themes directory in grub-shell isn't
parallel-test-friendly and breaks on the second test when the source
directory is read-only (as in "make distcheck"). Instead, add a
hidden --themes-directory option to grub-mkrescue et al, and use it
in grub-shell.
the function of these files exceeds what can be sanely handled in shell
in posix-comaptible way. Also writing it in C extends the functionality
to non-UNIX-like OS and minimal environments.
