DSPy a system for AI to Write Prompts and Do Fine Tuning
Implements BIOS emulation support for BHyVe: A BSD Hypervisor
1. Implements BIOS emulation support for BHyVe: A BSD Hypervisor
Abstract
Current BHyVe only supports FreeBSD/amd64 as a GuestOS.
One of the reason why BHyVe cannot support other OSes is lack of BIOS
support.
My project is implementing BIOS emulator on BHyVe, to remove these
limitations.
What is BHyVe?
BHyVe is new project to implement a hypervisor witch will integrate in FreeBSD.
The concept is similar to Linux KVM, it provides hypervisor driver to
unmodified BSD kernel running on baremetal machine.
With the driver, the kernel become a hypervisor, able to run GuestOS just like
normal process on the kernel.
Both hypervisors are designed for VT-x and AMD-V, hardware assisted
virtualization, unlike Xen s paravirtualization and VMware s binary translation1.
The kernel module only provides a feature to switch CPU modes between Host
mode and Guest mode2, almost all device emulation is performed in userland
process(Figure 1).
Linux KVM uses modified QEMU3 as the userland part.
It s good way to support large coverage of Guest OSes, because QEMU is highly
developed emulator, many people already confirmed to run variety of OSes on it.
KVM could support almost same features what QEMU has, and it just worked
fine.
BHyVe s approach is different.
BHyVe implements minimum set of device support which required to run
FreeBSD guest, from scratch.
In the result, we could have completely GPL-free, BSD licensed, well coded
hypervisor, but it only supports FreeBSD/amd64 as a Guest OS.
One of the reason why BHyVe cannot support other OSes is lack of BIOS
support.
1 Xen and VMware had provided hypervisors which use software technique to
virtualize non VT-x x86 CPUs. KVM and BHyVe doesn t take care such old CPUs,
designed for using the feature at first.
2 In Intel VT-x, it called VMX root mode and VMX non-root mode .
3 Original QEMU has full emulation of x86 CPU, but on KVM we want to use VT-x
hardware assisted virtualization instead of CPU emulation.
So they replace CPU emulation code to KVM driver call.
2. BHyVe loads and executes FreeBSD kernel directly using custom OS loader runs
on Host OS, instead of boot up from disk image.
With this method, we need to implement OS loader for each OSes, and currently
we don t have any loader other than FreeBSD.
Also, it doesn t support some OSes which calls BIOS function while running.
So I started the project to implementing BIOS emulator on BHyVe, to remove
these limitations.
/usr/sbin/bhyve
BSD
kernel
vmm.ko
ioctl(VM_RUN)
guest
kernel
VMExit
VMEntry
user
program
Figure 1. BHyVe overview
BIOS on real hardware
BIOS call is implemented as software interrupt handler on real mode(Figure 2).
CPU executes initialization code on BIOS ROM at the beginning of startup
machine, the code initialize real mode interrupt vector, to handle number of
software interrupts which are reserved for BIOS call(Figure 3).
And BIOS call is not only for legacy OSes like MS-DOS, almost all boot loaders
for mordan OSes are using BIOS call to access disks, display and keyboard.
Software interrupt(INTx)
CPU reads interrupt vector
Execute BIOS call handler
IO
Hardware
int 13h
Figure 2. BIOS call mechanism on real hardware
3. Interrupt vector
lowmem
Video RAM, etc
ROM BIOS
highmem
0000:0000
0000:0400
A000:0000
F000:0000
FFFF:0000
FFFF:000F
①Fetch interrupt handler address
②Jump to the handler address
③Handler accesses HW by IO instruction
Figure 3. Memory map on real hardware
BIOS on Linux KVM
On Linux KVM, QEMU loads Real BIOS(called SeaBIOS) on guest memory area.
Real BIOS means it also can install on some real motherboard, has full
functionality just like other proprietary BIOSes which installed on real hardware.
As I mentioned SeaBIOS support real motherboard, but usually used on
hypervisors and emulators.
KVM version of SeaBIOS s BIOS call handler accesses hardware by IO
instruction, and that behavior is basically same as BIOS for real hardware.
The difference is how the hardware access handled.
On KVM, the hardware access will trapped by KVM hypervisor driver, and QEMU
emulates hardware device, then KVM hypervisor driver resume a guest
environment(Figure 4).
In this implementation, KVM and QEMU does nothing about BIOS call, just
implements hardware device emulation and load real BIOS on guest memory
space(Figure 5).
4. Software interrupt(INTx)
CPU reads interrupt vector
Execute BIOS call handler
QEMU HW
Emulation
IO TrapSeaBIOS preforms IO
to virtual HW
QEMU emulates HW IO
HyperVisor
Guest
int 13h
Figure 4. BIOS call mechanism on KVM
Interrupt vector
lowmem
Video RAM, etc
SeaBIOS
highmem
0000:0000
0000:0400
A000:0000
F000:0000
FFFF:0000
FFFF:000F
①Fetch interrupt handler address
②Jump to the handler address
③Handler accesses HW by IO instr
QEMU emulates the IO
Figure 5. Memory map on KVM
How can we implement BIOS on BHyVe?
Port SeaBIOS on BHyVe and implement hardware emulation is an option, and it s
best way to improve compatibility of legacy code, but SeaBIOS is GPL d
software, it s not comfortable to bring in FreeBSD code tree.
And there s no implementation non-GPL real BIOS.
Instead, there s BSD licensed DOS Emulator called doscmd .
It s the software to run old DOS application on FreeBSD, the mechanism is
described as follows:
• Map page to 0x0
• Load DOS program
• run DOS program by entering v8086 mode
5. • DOS program calls BIOS call or DOS API by invoke INTx instruction
• DOS Emulator traps software interrupt, emulate BIOS call or DOS API
• Resuming v8086 mode
So it has BIOS emulation code which runs on FreeBSD protected mode
program.
Then, we decided to port BIOS emulation code to BHyVe and trap BIOS call
which executed on guest environment, instead of bring in real BIOS.
Trapping BIOS call
VT-x has functionality to trap various event on guest mode, it can be done by
changing VT-x configuration structure called VMCS.
And BHyVe kernel module can notify these events by IOCTL return.
So we need to trap BIOS call by changing configuration on VMCS, but what
event can be used for trap BIOS call?
Looks like trapping software interrupt is the easiest way, but let s think carefully
about protected mode.
On protected mode, it uses different interrupt vector called IDT, even userland
program calls INT 13H, BIOS INT 13H service won t called.
Maybe we can detect mode change between real mode/protected mode, and
enable/disable software interrupt trapping, but it s bit complicated.
Pseudo BIOS
Instead of implement complicated mode change detection, we decided to use
VMCALL instruction.
VMCALL instruction is the instruction which causes unconditional VMExit4.
And we made really simple pseudo BIOS for it.
Every software interrupt handler for BIOS call is like this:
VMCALL
IRET
BHyVe make interrupt vector when initialize guest memory area for BIOS call
handler, set pointer of the BIOS call handler described above.
In this way, it doesn t take care about mode changes anymore.
Conclusion
We decided to choose different approach than Linux KVM, to getting BIOS
support on BHyVe.
Linux KVM uses real BIOS and perform hardware emulation, BHyVe emulates
BIOS call it self(Figure 6).
4 Trap by hypervisor
