When working on memory, it's nice to be able to test your work.
Add a memtest module. When compiled with --enable-mm-debug, it exposes
3 commands:
* lsmem - print all allocations and free space in all regions
* lsfreemem - print free space in all regions
* stress_big_allocs - stress test large allocations:
- how much memory can we allocate in one chunk?
- how many 1MB chunks can we allocate?
- check that gap-filling works with a 1MB aligned 900kB alloc + a
100kB alloc.
Signed-off-by: Daniel Axtens <dja@axtens.net>
Signed-off-by: Stefan Berger <stefanb@linux.ibm.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
Tested-by: Nageswara R Sastry <rnsastry@linux.ibm.com>
Reviewed-by: Robbie Harwood <rharwood@redhat.com>
As a legacy support, if the vector 5 is not implemented, Power Hypervisor will
consider the max CPUs as 64 instead 256 currently supported during
client-architecture-support negotiation.
This patch implements the vector 5 and set the MAX CPUs to 256 while setting the
others values to 0 (default).
Signed-off-by: Diego Domingos <diegodo@linux.vnet.ibm.com>
Acked-by: Daniel Axtens <dja@axtens.net>
Signed-off-by: Stefan Berger <stefanb@linux.ibm.com>
Signed-off-by: Avnish Chouhan <avnish@linux.vnet.ibm.com>
Tested-by: Nageswara R Sastry <rnsastry@linux.ibm.com>
Reviewed-by: Robbie Harwood <rharwood@redhat.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
On powerpc-ieee1275, we are running out of memory trying to verify
anything. This is because:
- we have to load an entire file into memory to verify it. This is
difficult to change with appended signatures.
- We only have 32MB of heap.
- Distro kernels are now often around 30MB.
So we want to be able to claim more memory from OpenFirmware for our heap
at runtime.
There are some complications:
- The grub mm code isn't the only thing that will make claims on
memory from OpenFirmware:
* PFW/SLOF will have claimed some for their own use.
* The ieee1275 loader will try to find other bits of memory that we
haven't claimed to place the kernel and initrd when we go to boot.
* Once we load Linux, it will also try to claim memory. It claims
memory without any reference to /memory/available, it just starts
at min(top of RMO, 768MB) and works down. So we need to avoid this
area. See arch/powerpc/kernel/prom_init.c as of v5.11.
- The smallest amount of memory a ppc64 KVM guest can have is 256MB.
It doesn't work with distro kernels but can work with custom kernels.
We should maintain support for that. (ppc32 can boot with even less,
and we shouldn't break that either.)
- Even if a VM has more memory, the memory OpenFirmware makes available
as Real Memory Area can be restricted. Even with our CAS work, an LPAR
on a PowerVM box is likely to have only 512MB available to OpenFirmware
even if it has many gigabytes of memory allocated.
What should we do?
We don't know in advance how big the kernel and initrd are going to be,
which makes figuring out how much memory we can take a bit tricky.
To figure out how much memory we should leave unused, I looked at:
- an Ubuntu 20.04.1 ppc64le pseries KVM guest:
vmlinux: ~30MB
initrd: ~50MB
- a RHEL8.2 ppc64le pseries KVM guest:
vmlinux: ~30MB
initrd: ~30MB
So to give us a little wriggle room, I think we want to leave at least
128MB for the loader to put vmlinux and initrd in memory and leave Linux
with space to satisfy its early allocations.
Allow other space to be allocated at runtime.
Tested-by: Stefan Berger <stefanb@linux.ibm.com>
Signed-off-by: Daniel Axtens <dja@axtens.net>
Tested-by: Nageswara R Sastry <rnsastry@linux.ibm.com>
Reviewed-by: Robbie Harwood <rharwood@redhat.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
This was apparently "required by some firmware": commit dc9468500919
(2007-02-12 Hollis Blanchard <hollis@penguinppc.org>).
It's not clear what firmware that was, and what platform from 14 years ago
which exhibited the bug then is still both in use and buggy now.
It doesn't cause issues on qemu (mac99 or pseries) or under PFW for Power8.
I don't have access to old Mac hardware, but if anyone feels especially
strongly we can put it under some feature flag. I really want to disable
it under pseries because it will mess with region merging.
Signed-off-by: Daniel Axtens <dja@axtens.net>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
Tested-by: Nageswara R Sastry <rnsastry@linux.ibm.com>
Reviewed-by: Robbie Harwood <rharwood@redhat.com>
On PowerVM, the first time we boot a Linux partition, we may only get
256MB of real memory area, even if the partition has more memory.
This isn't enough to reliably verify a kernel. Fortunately, the Power
Architecture Platform Reference (PAPR) defines a method we can call to ask
for more memory: the broad and powerful ibm,client-architecture-support
(CAS) method.
CAS can do an enormous amount of things on a PAPR platform: as well as
asking for memory, you can set the supported processor level, the interrupt
controller, hash vs radix mmu, and so on.
If:
- we are running under what we think is PowerVM (compatible property of /
begins with "IBM"), and
- the full amount of RMA is less than 512MB (as determined by the reg
property of /memory)
then call CAS as follows: (refer to the Linux on Power Architecture
Reference, LoPAR, which is public, at B.5.2.3):
- Use the "any" PVR value and supply 2 option vectors.
- Set option vector 1 (PowerPC Server Processor Architecture Level)
to "ignore".
- Set option vector 2 with default or Linux-like options, including a
min-rma-size of 512MB.
- Set option vector 3 to request Floating Point, VMX and Decimal Floating
point, but don't abort the boot if we can't get them.
- Set option vector 4 to request a minimum VP percentage to 1%, which is
what Linux requests, and is below the default of 10%. Without this,
some systems with very large or very small configurations fail to boot.
This will cause a CAS reboot and the partition will restart with 512MB
of RMA. Importantly, grub will notice the 512MB and not call CAS again.
Notes about the choices of parameters:
- A partition can be configured with only 256MB of memory, which would
mean this request couldn't be satisfied, but PFW refuses to load with
only 256MB of memory, so it's a bit moot. SLOF will run fine with 256MB,
but we will never call CAS under qemu/SLOF because /compatible won't
begin with "IBM".)
- unspecified CAS vectors take on default values. Some of these values
might restrict the ability of certain hardware configurations to boot.
This is why we need to specify the VP percentage in vector 4, which is
in turn why we need to specify vector 3.
Finally, we should have enough memory to verify a kernel, and we will
reach Linux. One of the first things Linux does while still running under
OpenFirmware is to call CAS with a much fuller set of options (including
asking for 512MB of memory). Linux includes a much more restrictive set of
PVR values and processor support levels, and this CAS invocation will likely
induce another reboot. On this reboot grub will again notice the higher RMA,
and not call CAS. We will get to Linux again, Linux will call CAS again, but
because the values are now set for Linux this will not induce another CAS
reboot and we will finally boot all the way to userspace.
On all subsequent boots, everything will be configured with 512MB of RMA,
so there will be no further CAS reboots from grub. (phyp is super sticky
with the RMA size - it persists even on cold boots. So if you've ever booted
Linux in a partition, you'll probably never have grub call CAS. It'll only
ever fire the first time a partition loads grub, or if you deliberately lower
the amount of memory your partition has below 512MB.)
Signed-off-by: Daniel Axtens <dja@axtens.net>
Signed-off-by: Stefan Berger <stefanb@linux.ibm.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
Tested-by: Nageswara R Sastry <rnsastry@linux.ibm.com>
Reviewed-by: Robbie Harwood <rharwood@redhat.com>
Found during a test build on Debian/hurd-i386 with --disable-werror enabled:
In file included from grub-core/osdep/getroot.c:12:
grub-core/osdep/hurd/getroot.c: In function ‘grub_util_find_hurd_root_device’:
grub-core/osdep/hurd/getroot.c:126:13: error: unused variable ‘next’ [-Werror=unused-variable]
126 | char *next;
| ^~~~
grub-core/osdep/hurd/getroot.c:125:14: error: unused variable ‘size’ [-Werror=unused-variable]
125 | size_t size;
| ^~~~
Fixes: e981b0a24 (osdep/hurd/getroot: Use "part:" qualifier)
Signed-off-by: John Paul Adrian Glaubitz <glaubitz@physik.fu-berlin.de>
Reviewed-by: Samuel Thibault <samuel.thibault@ens-lyon.org>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
This prevents load_all_modules from failing when called before any
modules have been loaded. Failures in GDB user-defined functions cause
any function which called them to also fail.
Signed-off-by: Glenn Washburn <development@efficientek.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
GDB logging is redirected to write .segments.tmp, which means that GDB
will wrap lines longer than what it thinks is the screen width
(typically 80 characters). When wrapping does occur it causes gmodule.pl
to misbehave. So disable line wrapping by using GDB's "with" command so
that its guaranteed to return the width to the previous value upon
command completion.
Also disable command tracing when dumping the module sections because that
output will go to .segments.tmp and thus cause gmodule.pl to misbehave.
Signed-off-by: Glenn Washburn <development@efficientek.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
An error in any GDB command causes it to immediately abort with an error,
this includes any command that calls that command. This leads to an issue
in dump_module_sections where an error causes the command to exit without
turning off file redirection. The user then ends up with a GDB command
line where commands output nothing to the console.
Instead do the work of dump_module_sections in the command
dump_module_sections_helper and run the command using GDB's pipe command
which does the redirection and undoes the redirection when it finishes
regardless of any errors in the command.
Also, remove .segments.tmp file prior to loading modules in case one was
left from a previous run.
Signed-off-by: Glenn Washburn <development@efficientek.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
On EFI platforms where EFI calls do not require a wrapper (notably i386-efi
and arm64-efi), the func argument needs to be wrapped in parenthesis to
allow valid syntax when func is an expression which evaluates to a function
pointer. On EFI platforms that do need a wrapper, this was never an issue
because func is passed to the C function wrapper as an argument and thus
does not need parenthesis to be evaluated.
Signed-off-by: Glenn Washburn <development@efficientek.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
The "addr" is used to request the memory with specific ranges but the real
loadable address come from the relocator. Thus, print the final retrieved
addresses, virtual and physical, for initrd.
On the occasion migrate to PRIxGRUB_ADDR and PRIxGRUB_SIZE format specifiers.
Signed-off-by: Jeremy Szu <jeremy.szu@canonical.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
The functional test requires unicode.pf2 to run successfully, so
explicitly have the test return ERROR when its not found.
Signed-off-by: Glenn Washburn <development@efficientek.com>
Tested-by: Thomas Schmitt <scdbackup@gmx.net>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
Tests should be SKIP'd only when they do not apply to a particular target.
Hard errors are for when the test should run but can not be setup properly.
Signed-off-by: Glenn Washburn <development@efficientek.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
These are not added to grub-fs-tester because they are not generated and
none of the filesystem tests are run on these ISOs. The test is to run the
command "ls /" on the ISO, and a failure is determined if the command
times out, has non-zero return value or has any output.
Signed-off-by: Glenn Washburn <development@efficientek.com>
Tested-by: Thomas Schmitt <scdbackup@gmx.net>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
The disk sector size provided by sysfs file system considers the sector
size of 512 irrespective of disk sector size, thus causing the read by
the GRUB to an incorrect offset from what was originally intended.
Considering the 512 sector size of sysfs data the actual sector needs to
be modified corresponding to disk sector size.
Signed-off-by: Mukesh Kumar Chaurasiya <mchauras@linux.vnet.ibm.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
By using a shell variable that is set once by the expansion of an autoconf
variable, the resulting script is more readable.
Signed-off-by: Glenn Washburn <development@efficientek.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
In the function send_dhcp_packet(), added an error check for the return
value of grub_netbuff_push().
Fixes: CID 404614
Signed-off-by: Alec Brown <alec.r.brown@oracle.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
We do a lot of math about heap growth in hot path of grub_memalign().
However, the result is only used if out of memory is encountered, which
is seldom.
This patch moves these calculations away from hot path. These
calculations are now only done if out of memory is encountered. This
change can also help compiler to optimize integer overflow checks away.
Signed-off-by: Zhang Boyang <zhangboyang.id@gmail.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
When grub_memalign() encounters out-of-memory, it will try
grub_mm_add_region_fn() to request more memory from system firmware.
However, it doesn't preallocate memory space for future allocation
requests. In extreme cases, it requires one call to
grub_mm_add_region_fn() for each memory allocation request. This can
be very slow.
This patch introduces GRUB_MM_HEAP_GROW_EXTRA, the minimal heap growth
granularity. The new region size is now set to the bigger one of its
original value and GRUB_MM_HEAP_GROW_EXTRA. Thus, it will result in some
memory space preallocated if current allocations request is small.
The value of GRUB_MM_HEAP_GROW_EXTRA is set to 1MB. If this value is
smaller, the cost of small memory allocations will be higher. If this
value is larger, more memory will be wasted and it might cause
out-of-memory on machines with small amount of RAM.
Signed-off-by: Zhang Boyang <zhangboyang.id@gmail.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
When grub_memalign() encounters out-of-memory, it will try
grub_mm_add_region_fn() to request more memory from system firmware.
However, the size passed to it doesn't take region management overhead
into account. Adding a memory area of "size" bytes may result in a heap
region of less than "size" bytes really available. Thus, the new region
may not be adequate for current allocation request, confusing
out-of-memory handling code.
This patch introduces GRUB_MM_MGMT_OVERHEAD to address the region
management overhead (e.g. metadata, padding). The value of this new
constant must be large enough to make sure grub_memalign(align, size)
always succeeds after a successful call to
grub_mm_init_region(addr, size + align + GRUB_MM_MGMT_OVERHEAD),
for any given addr and size (assuming no integer overflow).
The size passed to grub_mm_add_region_fn() is now correctly adjusted,
thus if grub_mm_add_region_fn() succeeded, current allocation request
can always succeed.
Signed-off-by: Zhang Boyang <zhangboyang.id@gmail.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
When re-running a failed test, even the non-standard grub-shell QEMU
arguments should be preserved in the run.sh to more precisely replay
the failed test run.
Signed-off-by: Glenn Washburn <development@efficientek.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
Now it becomes trivial to re-run a test from the output in its working
directory. This also makes it easy to send a reproducible failing test to
the mailing list. This has allowed a refactor so that the duplicated code
to call QEMU has be condensed (e.g. the use of timeout and file descriptor
redirection). The run.sh script will pass any arguments given to QEMU.
This allows QEMU to be easily started in a state ready for GDB to be
attached.
Signed-off-by: Glenn Washburn <development@efficientek.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
This allows turning on shell tracing for grub-shell and grub-fs-tester
when its not practical or not possible to use command line arguments
(e.g. from "make check"). Turn on tracing when the envvar is an integer
greater than 1, since these can generate a lot of output. Since this
change uses the environment variables to set the default value for debug
in grub-shell, this allows enabling grub-shell's debug mode which will
preserve various generated output files that are helpful for debugging
tests.
Signed-off-by: Glenn Washburn <development@efficientek.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
If processing of a SUSP CE entry leads to a continuation area which
begins by entry CE or ST, then these entries were skipped without
interpretation. In case of CE this would lead to premature end of
processing the SUSP entries of the file. In case of ST this could
cause following non-SUSP bytes to be interpreted as SUSP entries.
Signed-off-by: Thomas Schmitt <scdbackup@gmx.net>
Tested-by: Lidong Chen <lidong.chen@oracle.com>
Reviewed-by: Thomas Schmitt <scdbackup@gmx.net>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
An SL entry consists of the entry info and the component area.
The entry info should take up 5 bytes instead of sizeof(*entry).
The area after the first 5 bytes is the component area. It is
incorrect to use the sizeof(*entry) to check the entry boundary.
Signed-off-by: Lidong Chen <lidong.chen@oracle.com>
Reviewed-by: Thomas Schmitt <scdbackup@gmx.net>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
Added a check for the SP entry data boundary before reading it.
Signed-off-by: Lidong Chen <lidong.chen@oracle.com>
Reviewed-by: Thomas Schmitt <scdbackup@gmx.net>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
In the code, the for loop advanced the entry pointer to the next entry before
checking if the next entry is within the system use area boundary. Another
issue in the code was that there is no check for the size of system use area.
For a corrupted system, the size of system use area can be less than the size
of minimum SUSP entry size (4 bytes). These can cause buffer overrun. The fixes
added the checks to ensure the read is valid and within the boundary.
Signed-off-by: Lidong Chen <lidong.chen@oracle.com>
Reviewed-by: Thomas Schmitt <scdbackup@gmx.net>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
There is no check for the end of block when reading
directory extents. It resulted in read_node() always
read from the same offset in the while loop, thus
caused infinite loop. The fix added a check for the
end of the block and ensure the read is within directory
boundary.
Signed-off-by: Lidong Chen <lidong.chen@oracle.com>
Reviewed-by: Thomas Schmitt <scdbackup@gmx.net>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
The logical sector size used by LUKS1 is 512 bytes and LUKS2 uses 512 to
4069 bytes. The default password used is "pass", but can be overridden
by setting the PASS environment variable. The device mapper name is set
to the name of the temp directory so that its easy to correlate device
mapper name with a particular test run. Also since this name is unique
per test run, multiple simultaneous test runs are allowed.
Note that cryptsetup is passing the --disable-locks parameter to allow
cryptsetup run successfully when /run/lock/cryptsetup is not accessible.
Since the device mapper name is unique per test run, there is no need to
worry about locking the device to serialize access.
Signed-off-by: Pierre-Louis Bonicoli <pierre-louis.bonicoli@libregerbil.fr>
Tested-by: Glenn Washburn <development@efficientek.com>
Signed-off-by: Glenn Washburn <development@efficientek.com>
Reviewed-by: Patrick Steinhardt <ps@pks.im>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
This lets a LUKS2 cryptodisk have its cipher and hash filled out,
otherwise they wouldn't be initialized if cheat mounted.
Signed-off-by: Josselin Poiret <dev@jpoiret.xyz>
Tested-by: Glenn Washburn <development@efficientek.com>
Reviewed-by: Patrick Steinhardt <ps@pks.im>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
Changes UUID comparisons so that LUKS1 and LUKS2 are both recognized
as being LUKS cryptodisks.
Signed-off-by: Josselin Poiret <dev@jpoiret.xyz>
Tested-by: Glenn Washburn <development@efficientek.com>
Reviewed-by: Patrick Steinhardt <ps@pks.im>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
When using grub-probe with cryptodisk, the mapped block device from the host
is used directly instead of decrypting the source device in GRUB code.
In that case, the sector size and count of the host device needs to be used.
This is especially important when using LUKS2, which does not assign
total_sectors and log_sector_size when scanning, but only later when the
segments in the JSON area are evaluated. With an unset log_sector_size,
grub_device_open() complains.
This fixes grub-probe failing with
"error: sector sizes of 1 bytes aren't supported yet.".
Signed-off-by: Fabian Vogt <fvogt@suse.de>
Reviewed-by: Patrick Steinhardt <ps@pks.im>
Tested-by: Glenn Washburn <development@efficientek.com>
Reviewed-by: Glenn Washburn <development@efficientek.com>
Reviewed-by: Patrick Steinhardt <ps@pks.im>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
Oops. You're allowed to have up to n = NAT_JOURNAL_ENTRIES entries
_inclusive_, because the loop below uses i < n, not i <= n. D'oh.
Fixes: 4bd9877f6216 (fs/f2fs: Do not read past the end of nat journal entries)
Reported-by: программист нект <programmer11180@programist.ru>
Tested-by: программист нект <programmer11180@programist.ru>
Signed-off-by: Daniel Axtens <dja@axtens.net>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
When building .img files, a .interp section from the .image files will
sometimes be copied into the .img file. This additional section pushes
the .img file beyond the 512-byte limit and causes grub-install to fail
to run for i386-pc platforms.
Signed-off-by: Nicholas Vinson <nvinson234@gmail.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
The grub_cmd_cryptomount make check test performs some functional testing
of cryptomount and by extension the underlying cryptodisk infrastructure.
A utility test script named grub-shell-luks-tester is created to handle the
complexities of the testing, making it simpler to add new test cases in
grub_cmd_cryptomount.
Signed-off-by: Glenn Washburn <development@efficientek.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
This allows test case scripts to use the appropriate halt command for
the built architecture to end execution early. Otherwise, test case
scripts have no way to know the appropriate mechanism for halting the
test case early.
Signed-off-by: Glenn Washburn <development@efficientek.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
When turning on shell tracing the trim line will be output before we
actually want to start the trim. However, in this case the trim line never
starts from the beginning of the line. So start trimming from the correct
line by matching from the beginning of the line.
Signed-off-by: Glenn Washburn <development@efficientek.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
This will be useful for tests that have unwanted output from setup. This is
not documented because its only intended to be internal at the moment. Also,
--no-trim is allowed to explicitly turn off trim.
Signed-off-by: Glenn Washburn <development@efficientek.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
This keeps the generated files to aid in diagnosing the source of the failure.
Signed-off-by: Glenn Washburn <development@efficientek.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
This allows us to test if unexpected output in test scripts is because of
a bug in GRUB, because there was an error in QEMU, or QEMU was killed due
to a timeout.
Signed-off-by: Glenn Washburn <development@efficientek.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
The "transparent" parameter to grub_gzio_open() was removed in 2010, fc2ef1172c
(* grub-core/io/gzio.c (grub_gzio_open): Removed "transparent" parameter.)
Signed-off-by: Glenn Washburn <development@efficientek.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
Commit f5759a878 (normal/help: Add paging instructions to normal and help
prompts) changed the output of the help command, which broke the help
test. This change allows the test to pass.
On the occasion do s/outpu/output/.
Signed-off-by: Glenn Washburn <development@efficientek.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
