As explained in commit a21618c8a (tests: Test aborts due to missing
requirements should be marked as error instead of skipped) and in the
Automake manual[1], skipped tests are tests that should not be run, e.g.
running the ohci test on the powerpc-ieee1275 as there are no native ohci
drivers for that platform. Test that fail for reasons other than there is
a bug in GRUB code that is causing the test to fail are hard errors.
Commonly this is because the test is run in an improperly configured
environment, like required programs are missing. If a hard error condition
is identified with a SKIP return code, the person running the tests can not
know without investigating every skip if a SKIP in the tests was because
the test does not apply to the target being tested or because the user had
a misconfigured environment that was causing the test not to run. By
ensuring that a test is skipped only when it should not run, the person
running the test can be sure that there is no need to investigate why the
test was skipped.
This reverts commit bf13fed5f (tests: Skip tests if required tools are not available).
[1] https://www.gnu.org/software/automake/manual/automake.html#Generalities-about-Testing
Signed-off-by: Glenn Washburn <development@efficientek.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
In this case it does not hurt to increase bash execution verbosity so
we can get more insight in case of issues.
Signed-off-by: Leo Sandoval <lsandova@redhat.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
There is no reason to fail a test if the required testing tool is not
present on the system, so skip the test instead of failing it.
Signed-off-by: Leo Sandoval <lsandova@redhat.com>
Reviewed-by: Andrew Hamilton <adhamilt@gmail.com>
Reviewed-by: Neal Gompa <ngompa13@gmail.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
Many tests abort due to not being root or missing tools, for instance mkfs
commands for file system tests. The tests are exited with code 77, which
means they were skipped. A skipped test is a test that should not be run,
e.g. a test specific to ARM64 should not be run on an x86 build. These aborts
are actually a hard error, code 99. That means that the test could not be
completed, but not because what was supposed to be tested failed, e.g. in
these cases where a missing tool prevents the running of a test.
Signed-off-by: Glenn Washburn <development@efficientek.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
