JSON strings require certain characters to be encoded, either by using
a single reverse solidus character "\" for a set of popular characters,
or by using a Unicode representation of "\uXXXXX". The jsmn library
doesn't handle unescaping for us, so we must implement this functionality
for ourselves.
Add a new function grub_json_unescape() that takes a potentially
escaped JSON string as input and returns a new unescaped string.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Reviewed-by: Daniel Kiper <dkiper@net-space.pl>
In the case of an error grub_initrd_load() uses argv[] to print the
filename that caused the error. It is also possible to obtain the
filename from the file handles and there is no need to duplicate that
information in argv[], so let's drop it.
Signed-off-by: Nikita Ermakov <arei@altlinux.org>
Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
In grub-core/loader/i386/bsdXX.c and grub-core/loader/multiboot_elfxx.c, error
conditionals are simplified to statements such as "if (err)". Even though the
assumption that non-zero values give errors is correct, it would be clearer and
more consistent to compare these conditionals to GRUB_ERR_NONE.
Signed-off-by: Alec Brown <alec.r.brown@oracle.com>
Reviewed-by: Darren Kenny <darren.kenny@oracle.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
In util/grub-module-verifierXX.c, the function get_shnum() returns the variable
shnum, which is of the type Elf_Word. In the function, shnum can be obtained by
the e_shnum member of an Elf_Ehdr or the sh_size member of an Elf_Shdr. The
sh_size member can either be grub_uint32_t or grub_uint64_t, depending on the
architecture, but Elf_Word is only grub_uint32_t. To account for when sh_size is
grub_uint64_t, we can set shnum to have type Elf_Shnum and have get_shnum()
return an Elf_Shnum.
Signed-off-by: Alec Brown <alec.r.brown@oracle.com>
Reviewed-by: Darren Kenny <darren.kenny@oracle.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
In bsdXX.c and multiboot_elfxx.c, e_phnum is used to obtain the number of
program header table entries, but it wasn't being checked if the value was
there.
According to the elf(5) manual page,
"If the number of entries in the program header table is larger than or equal to
PN_XNUM (0xffff), this member holds PN_XNUM (0xffff) and the real number of
entries in the program header table is held in the sh_info member of the
initial entry in section header table. Otherwise, the sh_info member of the
initial entry contains the value zero."
Since this check wasn't being made, grub_elfXX_get_phnum() is being added to
elfXX.c to make this check and use e_phnum if it doesn't have PN_XNUM as a
value, else use sh_info. We also need to make sure e_phnum isn't greater than
PN_XNUM and sh_info isn't less than PN_XNUM.
Note that even though elf.c and elfXX.c are located in grub-core/kern, they are
compiled as modules and don't need the EXPORT_FUNC() macro to define the functions
in elf.h.
Also, changed casts of phnum to match variables being set as well as dropped
casts when unnecessary.
Signed-off-by: Alec Brown <alec.r.brown@oracle.com>
Reviewed-by: Darren Kenny <darren.kenny@oracle.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
In multiboot_elfxx.c, e_shstrndx is used to obtain the section header table
index of the section name string table, but it wasn't being checked if the value
was there.
According to the elf(5) manual page,
"If the index of section name string table section is larger than or equal to
SHN_LORESERVE (0xff00), this member holds SHN_XINDEX (0xffff) and the real
index of the section name string table section is held in the sh_link member of
the initial entry in section header table. Otherwise, the sh_link member of the
initial entry in section header table contains the value zero."
Since this check wasn't being made, grub_elfXX_get_shstrndx() is being added to
elfXX.c to make this check and use e_shstrndx if it doesn't have SHN_XINDEX as a
value, else use sh_link. We also need to make sure e_shstrndx isn't greater than
or equal to SHN_LORESERVE and sh_link isn't less than SHN_LORESERVE.
Note that even though elf.c and elfXX.c are located in grub-core/kern, they are
compiled as modules and don't need the EXPORT_FUNC() macro to define the functions
in elf.h.
Signed-off-by: Alec Brown <alec.r.brown@oracle.com>
Reviewed-by: Darren Kenny <darren.kenny@oracle.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
In bsdXX.c and multiboot_elfxx.c, e_shnum is used to obtain the number of
section header table entries, but it wasn't being checked if the value was
there.
According to the elf(5) manual page,
"If the number of entries in the section header table is larger than or equal to
SHN_LORESERVE (0xff00), e_shnum holds the value zero and the real number of
entries in the section header table is held in the sh_size member of the initial
entry in section header table. Otherwise, the sh_size member of the initial
entry in the section header table holds the value zero."
Since this check wasn't being made, grub_elfXX_get_shnum() is being added to
elfXX.c to make this check and use whichever member doesn't have a value of
zero. If both are zero, then we must return an error. We also need to make sure
that e_shnum doesn't have a value greater than or equal to SHN_LORESERVE and
sh_size isn't less than SHN_LORESERVE.
In order to get this function to work, the type ElfXX_Shnum is being added where
Elf32_Shnum defines Elf32_Word and Elf64_Shnum defines Elf64_Xword. This new
type is needed because if shnum obtains a value from sh_size, sh_size could be
of type El32_Word for Elf32_Shdr structures or Elf64_Xword for Elf64_Shdr
structures.
Note that even though elf.c and elfXX.c are located in grub-core/kern, they are
compiled as modules and don't need the EXPORT_FUNC() macro to define the functions
in elf.h.
For a few smaller changes, changed casts of shnum to match variables being set
as well as dropped casts when unnecessary and fixed spacing errors in bsdXX.c.
Also, shnum is an unsigned integer and is compared to int i in multiboot_elfxx.c,
it should be unsigned to match shnum.
Signed-off-by: Alec Brown <alec.r.brown@oracle.com>
Reviewed-by: Darren Kenny <darren.kenny@oracle.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
The linux_xen template orders the "early" initrd file(s) _first_
(i.e., before the "real" initrd files) and that seems reasonable,
as microcode updates usually come first.
However, this usually breaks Linux boot with initrd under Xen
because Xen assumes the real initrd is the first multiboot[2]
module after the kernel, passing its address over to Linux in
Xen's start_info struct.
So, if a microcode-only initrd (i.e., without init/userspace)
is found by grub-mkconfig, it ends up considered as a normal
initrd by the Linux kernel, which cannot do anything with it
(as it has no other files) and panic()s unable to mount root
if it depends on a initrd to do that (e.g., root=UUID=...).
...
Well, since Xen doesn't actually use the provided microcode
by default / unless the 'ucode=<module number|scan>' option
is enabled, this isn't used in the general case (and breaks).
Additionally, if an user enables the 'ucode=' option, that
either specifies which module is to be used for microcode,
or scans all modules (regardless of being first) for that.
Thus, for Xen:
- it is *not required* to have microcode first,
- but it is *required* to have real initrd first
So, fix it by ordering the real initrd before early initrd(s).
After:
# touch /boot/xen /boot/microcode.cpio
# grub-mkconfig 2>/dev/null | grep -P '^\t(multiboot|module)'
multiboot /boot/xen ...
module /boot/vmlinuz-5.4.0-122-generic ...
module --nounzip /boot/initrd.img-5.4.0-122-generic
module --nounzip /boot/microcode.cpio
...
Corner case specific to Xen implementation details:
It is actually _possible_ to have a microcode initrd first,
but that requires a non-default option (so can't rely on it),
and it turns out to be inconsistent with its counterpart
(really shouldn't rely on it, as it may get confusing; below).
'ucode=1' does manually specify the first module is microcode
_AND_ clears its bit in the module bitmap. The next module is
now the 'new first', and gets passed to Linux as initrd. Good.
'ucode=scan' checks all modules for microcode, but does _NOT_
clear a bit if it finds one (reasonable, as it can find that
prepended in a "real" initrd anyway, which needs to be used).
The first module still gets passed to Linux as initrd. Bad.
Fixes: e86f6aafb8de (grub-mkconfig/20_linux_xen: Support multiple early initrd images)
Signed-off-by: Mauricio Faria de Oliveira <mfo@canonical.com>
Acked-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
The linux_xen template can put multiple initrd files in the
same multiboot[2] module[2] command, which is against specs.
This causes ONLY the _first_ initrd file to be loaded; other
files just have filenames in a "cmdline" string of the first
initrd file and are NOT loaded.
Fix this by inserting a module[2] command per initrd file.
Before:
# touch /boot/xen /boot/microcode.cpio
# grub-mkconfig 2>/dev/null | grep -P '^\t(multiboot|module)'
multiboot /boot/xen ...
module /boot/vmlinuz-5.4.0-122-generic ...
module --nounzip /boot/microcode.cpio /boot/initrd.img-5.4.0-122-generic
After:
# touch /boot/xen /boot/microcode.cpio
# grub-mkconfig 2>/dev/null | grep -P '^\t(multiboot|module)'
multiboot /boot/xen ...
module /boot/vmlinuz-5.4.0-122-generic ...
module --nounzip /boot/microcode.cpio
module --nounzip /boot/initrd.img-5.4.0-122-generic
Cause:
The code was copied from the linux template, which is *apparently*
equivalent.. but its initrd command grub_cmd_initrd() *supports*
multiple files (see grub_initrd_init()), while module/module2 in
grub_cmd_module() *does not* (see grub_multiboot[2]_add_module()).
See commit e86f6aafb8de (grub-mkconfig/20_linux_xen: Support multiple early initrd images):
'This is basically a copy of a698240d "grub-mkconfig/10_linux:
Support multiple early initrd images" ...'
Specs:
Both multiboot and multiboot2 specifications mention support for
'multiple boot modules' (struct/tag used for kernel/initrd files):
"Boot loaders don’t have to support multiple boot modules,
but they are strongly encouraged to" [1,2]
However, there is a 1:1 relationship between boot modules and files,
more or less clearly; note the usage of singular/plural "module(s)".
(Multiboot2, clearly: "One tag appears per module".)
Multiboot [1]:
"the ‘mods’ fields indicate ... what boot modules
were loaded ..., and where they can be found.
‘mods_count’ contains the number of modules loaded"
"The first two fields contain the start and end addresses
of the boot module itself."
Multiboot2 [2]:
"This tag indicates ... what boot module was loaded ...,
and where it can be found."
"The ‘mod_start’ and ‘mod_end’ contain the start and end
physical addresses of the boot module itself."
"One tag appears per module.
This tag type may appear multiple times."
And both clearly mention the 'string' field of a boot module,
which is to be used by the operating system, not boot loader:
"The ‘string’ field provides an arbitrary string to be
associated with that particular boot module ...
its exact use is specific to the operating system."
Links:
[1] https://www.gnu.org/software/grub/manual/multiboot/multiboot.html
3.3 Boot information format
[2] https://www.gnu.org/software/grub/manual/multiboot2/multiboot.html
3.6.6 Modules
Fixes: e86f6aafb8de (grub-mkconfig/20_linux_xen: Support multiple early initrd images)
Signed-off-by: Mauricio Faria de Oliveira <mfo@canonical.com>
Acked-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
Note this cast was fixed in grub_strcasecmp() in commit ce41ab7aab
(* grub-core/kern/misc.c (grub_strcmp): Use unsigned comparison as per
common usage and preffered in several parts of code.), but this commit
omitted fixing it in grub_strncasecmp().
Signed-off-by: Glenn Washburn <development@efficientek.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
The previous behavior ignored an error and the output from grub-mkrescue.
This made it difficult to discover that grub-mkrescue was the reason that
tests which rely on grub-shell were failing. Even after discovering
grub-mkrescue was the culprit, there was no output to indicate why it was
failing. It turns out that grub-mkrescue is a thin wrapper around xorriso.
So if you do not have xorriso installed it will fail with an error message
about not being able to find xorriso.
This change will allow grub-mkrescue output to be written to stderr, only
if grub-mkrescue fails. If grub-mkrescue succeeds, there will be no output
from grub-mkrescue so as not to interfere with the functioning of tests.
This change should have no effect on the running of tests or other uses of
grub-shell as it only modifies the error path.
Also, if grub-mkrescue fails, the script exits early. Since grub-shell
needs the ISO image created by grub-mkresue to boot the QEMU instance,
a failure here should be considered fatal.
Signed-off-by: Glenn Washburn <development@efficientek.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
The "ARM\x64" magic number in the file header identifies an image as one
that implements the bare metal boot protocol, allowing the loader to
simply move the file to a suitably aligned address in memory, with
sufficient headroom for the trailing .bss segment (the required memory
size is described in the header as well).
Note of this matters for GRUB, as it only supports EFI boot. EFI does
not care about this magic number, and nor should GRUB: this prevents us
from booting other PE linux images, such as the generic EFI zboot
decompressor, which is a pure PE/COFF image, and does not implement the
bare metal boot protocol.
So drop the magic number check.
Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
Because grub_util_mkdir() is implemented to not return a value on any
platform, grub_instal_mkdir_p() can test for success by confirming that
the directory requested exists after attempting to create it, otherwise
it should fail with an error and exit.
While fixing this, a flaw in the logic was shown, where the first match
of the path separator, which almost always was the first character in
the path (e.g. /boot/grub2) would result in creating a directory with an
empty name (i.e. ""). To avoid that, it should skip the handling of the
path separator where p is pointing to the first character.
Signed-off-by: Darren Kenny <darren.kenny@oracle.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
Coverity signaled 2 issues where the return value of grub_util_mkdir()
was not being tested.
The Windows variant of this code defines the function as having no
return value (void), but the UNIX variants all are mapped using a macro
to the libc mkdir() function, which returns an int value.
To be consistent, the mapping should cast to void to for these too.
Fixes: CID 73583
Fixes: CID 73617
Signed-off-by: Darren Kenny <darren.kenny@oracle.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
Update the read hook to take into account encrypted volumes on a partition.
GRUB disk read hooks supply an absolute sector number at which the read is
started from. If the encrypted volume is in a partition, the sector number
given to the read hook will be offset by the number of the sector at the
start of the partition. The read hook then needs to subtract the partition
start from the supplied sector to get the correct start sector for the read
into the detached header file.
Reported-by: brutser <brutser@perso.be>
Signed-off-by: Glenn Washburn <development@efficientek.com>
Tested-by: brutser <brutser@perso.be>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
By using shell variable that are set once by the expansion of an autoconf
variable, the resulting shell script is more easily moved and modified
from the build/install directory it was generated for. The resulting
script is more readable as well.
Signed-off-by: Glenn Washburn <development@efficientek.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
On my laptop running at 2.4GHz, if I run a VM where tsc calibration
using pmtimer will fail presuming a broken pmtimer, it takes ~51 seconds
to do so (as measured with the stopwatch on my phone), with a tsc delta
of 0x1cd1c85300, or around 125 billion cycles.
If instead of trying to wait for 5-200ms to show up on the pmtimer, we
try to wait for 5-200us, it decides it's broken in ~0x2626aa0 TSCs, aka
~2.4 million cycles, or more or less instantly.
Additionally, this reading the pmtimer was returning 0xffffffff anyway,
and that's obviously an invalid return. I've added a check for that and
0 so we don't bother waiting for the test if what we're seeing is dead
pins with no response at all.
If "debug" includes "pmtimer", you will see one of the following three
outcomes. If pmtimer gives all 0 or all 1 bits, you will see:
pmtimer: 0xffffff bad_reads: 1
pmtimer: 0xffffff bad_reads: 2
pmtimer: 0xffffff bad_reads: 3
pmtimer: 0xffffff bad_reads: 4
pmtimer: 0xffffff bad_reads: 5
pmtimer: 0xffffff bad_reads: 6
pmtimer: 0xffffff bad_reads: 7
pmtimer: 0xffffff bad_reads: 8
pmtimer: 0xffffff bad_reads: 9
pmtimer: 0xffffff bad_reads: 10
timer is broken; giving up.
This outcome was tested using qemu+kvm with UEFI (OVMF) firmware and
these options: -machine pc-q35-2.10 -cpu Broadwell-noTSX
If pmtimer gives any other bit patterns but is not actually marching
forward fast enough to use for clock calibration, you will see:
pmtimer delta is 0x0 (1904 iterations)
tsc delta is implausible: 0x2626aa0
This outcome was tested using GRUB patched to not ignore bad reads using
qemu+kvm with UEFI (OVMF) firmware, and these options:
-machine pc-q35-2.10 -cpu Broadwell-noTSX
If pmtimer actually works, you'll see something like:
pmtimer delta is 0xdff
tsc delta is 0x278756
This outcome was tested using qemu+kvm with UEFI (OVMF) firmware, and
these options: -machine pc-i440fx-2.4 -cpu Broadwell-noTSX
I've also tested this outcome on a real Intel Xeon E3-1275v3 on an Intel
Server Board S1200V3RPS using the SDV.RP.B8 "Release" build here:
https://www.intel.com/content/www/us/en/download/674448/firmware-update-for-the-intel-server-board-s1200rp-uefi-development-kit-release-vb8.html
Signed-off-by: Peter Jones <pjones@redhat.com>
Signed-off-by: Javier Martinez Canillas <javierm@redhat.com>
Signed-off-by: Robbie Harwood <rharwood@redhat.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
luks2_get_keyslot() can fail for a variety of reasons that do not necessarily
mean the next keyslot should not be tried (e.g. a new kdf type). So always
try the next slot. This will make GRUB more resilient to non-spec json data
that 3rd party systems may add. We do not care if some of the keyslots are
unusable, only if there is at least one that is.
Signed-off-by: Glenn Washburn <development@efficientek.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
This command is meant to behave similarly to the "mode" command of the EFI
Shell application. In addition to allowing mode selection by giving the
number of columns and rows as arguments, the command allows specifying the
mode number to select the mode. Also supported are the arguments "min" and
"max", which set the mode to the minimum and maximum mode respectively as
calculated by the columns * rows of that mode.
Signed-off-by: Glenn Washburn <development@efficientek.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
When filesystem detection fails, all that's currently debug-logged is
a series of messages like:
grub-core/kern/fs.c:56:fs: Detecting ntfs...
grub-core/kern/fs.c:76:fs: ntfs detection failed.
repeated for each filesystem. Any messages provided to grub_error() by
the filesystem are lost, and one has to break out gdb to figure out what
went wrong.
With this change, one instead sees:
grub-core/kern/fs.c:56:fs: Detecting fat...
grub-core/osdep/hostdisk.c:357:hostdisk: reusing open device
`/path/to/device'
grub-core/kern/fs.c:77:fs: error: invalid modification timestamp for /.
grub-core/kern/fs.c:79:fs: fat detection failed.
in the debug prints.
Signed-off-by: Robbie Harwood <rharwood@redhat.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
The way the code is written the tofree variable would never be passed to
the free_subchunk() function uninitialized. Coverity cannot determine
this and flags the situation as "Using uninitialized value...". The fix
is just to initialize the local struct.
Fixes: CID 314016
Signed-off-by: Ross Philipson <ross.philipson@oracle.com>
Reviewed-by: Darren Kenny <darren.kenny@oracle.com>
Tested-by: Alec Brown <alec.r.brown@oracle.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
The EFI_CC_MEASUREMENT_PROTOCOL abstracts the measurement for virtual firmware
in confidential computing environment. It is similar to the EFI_TCG2_PROTOCOL.
It was proposed by Intel and ARM and approved by UEFI organization.
It is defined in Intel GHCI specification: https://cdrdv2.intel.com/v1/dl/getContent/726790 .
The EDKII header file is available at https://github.com/tianocore/edk2/blob/master/MdePkg/Include/Protocol/CcMeasurement.h .
Signed-off-by: Lu Ken <ken.lu@intel.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
The event description is a string, so using grub_strcpy() is cleaner than
using grub_memcpy().
Signed-off-by: Lu Ken <ken.lu@intel.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
1. Use macro GRUB_ERR_NONE instead of hard code 0.
2. Keep lowercase of the first char for the status string of log event.
Signed-off-by: Lu Ken <ken.lu@intel.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
Introduce ERROR_PLATFORM_NOT_SUPPORT_SSP environment variable to treat
the "--enable-stack-protector is only supported on EFI platforms" message
as a warning instead of an error. If ERROR_PLATFORM_NOT_SUPPORT_SSP is
set to "no" (case-insensitive), then the message will be printed as
a warning. Otherwise, it prints as an error. The default behavior is to
print the message as an error.
For any wrapper build script that has some variation of:
for p in SELECTED_GRUB_PLATFORMS; do \
configure --enable-stack-protector \
--with-platform${P} ... || die; \
done
make
The GRUB will fail to build if SELECTED_GRUB_PLATFORMS contains a platform
that does not support SSP.
Such wrapper scripts need to work-around this issue by modifying the
above for-loop, so it conditionally passes --enable-stack-protector to
configure for the proper GRUB platform(s).
However, if the above example is modified to have to conditionally pass
in --enable-stack-protector, its behavior is effectively the same as the
proposed change. Additionally, The list of SSP supported platforms is
now in 2 places. One in the configure script and one in the build wrapper
script. If the second list is not properly maintained it could mistakenly
disable SSP for a platform that later gained support for it.
Signed-off-by: Nicholas Vinson <nvinson234@gmail.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
With gsub substitutions the offsets should be validated against the
number of glyphs in a font face and the memory allocated for the gsub
substitution data.
Both the number of glyphs and the last address in the allocated data are
passed in to process_cursive(), where the number of glyphs validates the end
of the range.
Enabling memory allocation validation uses two macros, one to simply check the
address against the allocated space, and the other to check that the number of
items of a given size doesn't extend outside of the allocated space.
Fixes: CID 73770
Fixes: CID 314040
Signed-off-by: Darren Kenny <darren.kenny@oracle.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
There are no users left of version_find_latest(), version_test_gt(), and
version_test_numeric(). Remove those unused helper functions. Using
those helper functions is what caused the quadratic sorting performance
issues in the first place, so removing them is a net win.
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Reviewed-by: Robbie Harwood <rharwood@redhat.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
The current implementation of the 10_kfreebsd script implements its menu
items sorting in bash with a quadratic algorithm, calling "sed", "sort",
"head", and "grep" to compare versions between individual lines, which
is annoyingly slow for kernel developers who can easily end up with
50-100 kernels in their boot partition.
This fix is ported from the 10_linux script, which has a similar
quadratic code pattern.
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Cc: debian-bsd@lists.debian.org
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
The current implementation of the 10_hurd script implements its menu
items sorting in bash with a quadratic algorithm, calling "sed", "sort",
"head", and "grep" to compare versions between individual lines, which
is annoyingly slow for kernel developers who can easily end up with
50-100 kernels in their boot partition.
This fix is ported from the 10_linux script, which has a similar
quadratic code pattern.
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Cc: Samuel Thibault <samuel.thibault@ens-lyon.org>
Tested-by: Samuel Thibault <samuel.thibault@ens-lyon.org>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
The current implementation of the 20_linux_xen script implements its
menu items sorting in bash with a quadratic algorithm, calling "sed",
"sort", "head", and "grep" to compare versions between individual lines,
which is annoyingly slow for kernel developers who can easily end up
with 50-100 kernels in their boot partition.
This fix is ported from the 10_linux script, which has a similar
quadratic code pattern.
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Cc: xen-devel@lists.xenproject.org
Tested-by: Jason Andryuk <jandryuk@gmail.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
The current implementation of the 10_linux script implements its menu
items sorting in bash with a quadratic algorithm, calling "sed", "sort",
"head", and "grep" to compare versions between individual lines, which
is annoyingly slow for kernel developers who can easily end up with
50-100 kernels in /boot.
As an example, on a Intel(R) Core(TM) i7-8650U CPU @ 1.90GHz, running:
/usr/sbin/grub-mkconfig > /dev/null
With 44 kernels in /boot, this command takes 10-15 seconds to complete.
After this fix, the same command runs in 5 seconds.
With 116 kernels in /boot, this command takes 40 seconds to complete.
After this fix, the same command runs in 8 seconds.
For reference, the quadratic algorithm here is:
while [ "x$list" != "x" ] ; do <--- outer loop
linux=`version_find_latest $list`
version_find_latest()
for i in "$@" ; do <--- inner loop
version_test_gt()
fork+exec sed
version_test_numeric()
version_sort
fork+exec sort
fork+exec head -n 1
fork+exec grep
list=`echo $list | tr ' ' '\n' | fgrep -vx "$linux" | tr '\n' ' '`
tr
fgrep
tr
So all commands executed under version_test_gt() are executed
O(n^2) times where n is the number of kernel images in /boot.
Here is the improved algorithm proposed:
- Prepare a list with all the relevant information for ordering by a single
sort(1) execution. This is done by renaming ".old" suffixes by " 1" and
by suffixing all other files with " 2", thus making sure the ".old" entries
will follow the non-old entries in reverse-sorted-order.
- Call version_reverse_sort on the list (sort -r -V): A single execution of
sort(1). For instance, GNU coreutils' sort will reverse-sort the list in
O(n*log(n)) with a merge sort.
- Replace the " 1" suffixes by ".old", and remove the " 2" suffixes.
- Iterate on the reverse-sorted list to output each menu entry item.
Therefore, the algorithm proposed has O(n*log(n)) complexity with GNU
coreutils' sort compared to the prior O(n^2) complexity. Moreover, the
constant time required for each list entry is much less because sorting
is done within a single execution of sort(1) rather than requiring
O(n^2) executions of sed(1), sort(1), head(1), and grep(1) in
sub-shells.
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Reviewed-by: Robbie Harwood <rharwood@redhat.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
Using the disk read hook mechanism, setup a read hook on the source disk
which will read from the given header file during the scan and recovery
cryptodisk backend functions. Disk read hooks are executed after the data
has been read from the disk. This is okay, because the read hook is given
the read buffer before its sent back to the caller. In this case, the hook
can then overwrite the data read from the disk device with data from the
header file sent in as the read hook data. This is transparent to the
read caller. Since the callers of this function have just opened the
source disk, there are no current read hooks, so there's no need to
save/restore them nor consider if they should be called or not.
This hook assumes that the header is at the start of the volume, which
is not the case for some formats (e.g. GELI). So GELI will return an
error if a detached header is specified. It also can only be used
with formats where the detached header file can be written to the
first blocks of the volume and the volume could still be unlocked.
So the header file can not be formatted differently from the on-disk
header. If these assumpts are not met, detached header file processing
must be specially handled in the cryptodisk backend module.
The hook will be called potentially many times by a backend. This is fine
because of the assumptions mentioned and the read hook reads from absolute
offsets and is stateless.
Also add a --header (short -H) option to cryptomount which takes a file
argument.
Signed-off-by: Glenn Washburn <development@efficientek.com>
Reviewed-by: Patrick Steinhardt <ps@pks.im>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
It will be desirable in the future to allow having the read hook modify the
data passed back from a read function call on a disk or file. This adds that
infrastructure and has no impact on code flow for existing uses of the read
hook. Also changed is that now when the read hook callback is called it can
also indicate what error code should be sent back to the read caller.
Signed-off-by: Glenn Washburn <development@efficientek.com>
Reviewed-by: Patrick Steinhardt <ps@pks.im>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
Document the variables net_<interface>_clientid, net_<interface>_clientuuid,
lockdown, and shim_lock in the list of special environment variables.
Signed-off-by: Glenn Washburn <development@efficientek.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
Adjust the interface of grub_efi_mm_add_regions() to take a set of
GRUB_MM_ADD_REGION_* flags, which most notably is currently only the
GRUB_MM_ADD_REGION_CONSECUTIVE flag. This allows us to set the function
up as callback for the memory subsystem and have it call out to us in
case there's not enough pages available in the current heap.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Daniel Axtens <dja@axtens.net>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
Tested-by: Patrick Steinhardt <ps@pks.im>
The function add_memory_regions() is currently only called on system
initialization to allocate a fixed amount of pages. As such, it didn't
need to return any errors: in case it failed, we cannot proceed anyway.
This will change with the upcoming support for requesting more memory
from the firmware at runtime, where it doesn't make sense anymore to
fail hard.
Refactor the function to return an error to prepare for this. Note that
this does not change the behaviour when initializing the memory system
because grub_efi_mm_init() knows to call grub_fatal() in case
grub_efi_mm_add_regions() returns an error.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Daniel Axtens <dja@axtens.net>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
Tested-by: Patrick Steinhardt <ps@pks.im>
In preparation of support for runtime-allocating additional memory
region, this patch extracts the function to retrieve the EFI memory
map and add a subset of it to GRUB's own memory regions.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Daniel Axtens <dja@axtens.net>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
Tested-by: Patrick Steinhardt <ps@pks.im>
When initializing the EFI memory subsystem, we will by default request
a quarter of the available memory, bounded by a minimum/maximum value.
Given that we're about to extend the EFI memory system to dynamically
request additional pages from the firmware as required, this scaling of
requested memory based on available memory will not make a lot of sense
anymore.
Remove this logic as a preparatory patch such that we'll instead defer
to the runtime memory allocator. Note that ideally, we'd want to change
this after dynamic requesting of pages has been implemented for the EFI
platform. But because we'll need to split up initialization of the
memory subsystem and the request of pages from the firmware, we'd have
to duplicate quite some logic at first only to remove it afterwards
again. This seems quite pointless, so we instead have patches slightly
out of order.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Daniel Axtens <dja@axtens.net>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
Tested-by: Patrick Steinhardt <ps@pks.im>
Currently, all platforms will set up their heap on initialization of the
platform code. While this works mostly fine, it poses some limitations
on memory management on us. Most notably, allocating big chunks of
memory in the gigabyte range would require us to pre-request this many
bytes from the firmware and add it to the heap from the beginning on
some platforms like EFI. As this isn't needed for most configurations,
it is inefficient and may even negatively impact some usecases when,
e.g., chainloading. Nonetheless, allocating big chunks of memory is
required sometimes, where one example is the upcoming support for the
Argon2 key derival function in LUKS2.
In order to avoid pre-allocating big chunks of memory, this commit
implements a runtime mechanism to add more pages to the system. When
a given allocation cannot be currently satisfied, we'll call a given
callback set up by the platform's own memory management subsystem,
asking it to add a memory area with at least "n" bytes. If this
succeeds, we retry searching for a valid memory region, which should
now succeed.
If this fails, we try asking for "n" bytes, possibly spread across
multiple regions, in hopes that region merging means that we end up
with enough memory for things to work out.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Daniel Axtens <dja@axtens.net>
Tested-by: Stefan Berger <stefanb@linux.ibm.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
Tested-by: Patrick Steinhardt <ps@pks.im>
In grub_memalign(), there's a commented section which would allow for
unloading of unneeded modules in case where there is not enough free
memory available to satisfy a request. Given that this code is never
compiled in, let's remove it together with grub_dl_unload_unneeded().
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Daniel Axtens <dja@axtens.net>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
Tested-by: Patrick Steinhardt <ps@pks.im>
This is handy for debugging. Enable with "set debug=regions".
Signed-off-by: Daniel Axtens <dja@axtens.net>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
Tested-by: Patrick Steinhardt <ps@pks.im>
On x86_64-efi (at least) regions seem to be added from top down. The mm
code will merge a new region with an existing region that comes
immediately before the new region. This allows larger allocations to be
satisfied that would otherwise be the case.
On powerpc-ieee1275, however, regions are added from bottom up. So if
we add 3x 32MB regions, we can still only satisfy a 32MB allocation,
rather than the 96MB allocation we might otherwise be able to satisfy.
* Define 'post_size' as being bytes lost to the end of an allocation
due to being given weird sizes from firmware that are not multiples
of GRUB_MM_ALIGN.
* Allow merging of regions immediately _after_ existing regions, not
just before. As with the other approach, we create an allocated
block to represent the new space and the pass it to grub_free() to
get the metadata right.
Signed-off-by: Daniel Axtens <dja@axtens.net>
Tested-by: Stefan Berger <stefanb@linux.ibm.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
Tested-by: Patrick Steinhardt <ps@pks.im>
grub_mm_region_init() does:
h = (grub_mm_header_t) (r + 1);
where h is a grub_mm_header_t and r is a grub_mm_region_t.
Cells are supposed to be GRUB_MM_ALIGN aligned, but while grub_mm_dump
ensures this vs the region header, grub_mm_region_init() does not.
It's better to be explicit than implicit here: rather than changing
grub_mm_region_init() to ALIGN_UP(), require that the struct is
explicitly a multiple of the header size.
Signed-off-by: Daniel Axtens <dja@axtens.net>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
Tested-by: Patrick Steinhardt <ps@pks.im>
