1. OSb: OSv on BitVisor
(2014/11/21 BitVisor Summit 3)
Yushi Omote
University of Tsukuba
Takahiro Shinagawa
The University of Tokyo
Monday, November 24, 14
2. BitVisor
App App App
BitVisor
Hardware
OS
http://www.justis.as-1.co.jp
Monday, November 24, 14
7. OSv on BitVisor
BitVisor
Hardware
App
OSv
Monday, November 24, 14
8. OSv on BitVisor
BitVisor
Hardware
App
OSv
OSb
Monday, November 24, 14
9. OSv on BitVisor
BitVisor
Hardware
App
OSv
What the fuck
no Virtio !
OSb
Physical interface
sucks !
What the hell is
PRO/1000 !
Monday, November 24, 14
10. OSv on BitVisor
BitVisor
Hardware
App
OSv
What the fuck
no Virtio !
OSb
Physical interface
sucks !
What the hell is
PRO/1000 !
You have no
choice.
Virtio
Monday, November 24, 14
12. OSv Code Reading
• Boot Process
• Some Drivers
- MBR
- Read local disk with INT13/42.
- Load command line, boot loader, OSv kernel.
- Get memory map with INT15/E820.
- Setup segment descriptors/page tables.
- Switch to 64-bit mode.
- premain()
- main()
...
- Serial / VGA output.
- SATA.
- Virtio NIC/BLK/RNG/SCSI.
- ACPI.
- APIC.
...
OSv may run on
physical machine.
(=BitVisor)
Monday, November 24, 14
13. Only one thing to fix
drivers/acpi.cc:
// Copy the root table list to dynamic memory
if (!is_bitvisor()) {
status = AcpiReallocateRootTable();
if (ACPI_FAILURE(status)) {
acpi_e("AcpiReallocateRootTable failed: %sn",
AcpiFormatException(status));
return;
}
}
Triple fault
Skip on BitVisor
Monday, November 24, 14
15. OSb
On BitVisor,
Whenever updating OSv images,
you update the entire Physical Disk.
http://a.fsdn.com/sd/articles/14/11/12/1946208-1.jpg http://www.dreamstime.com
Troublesome...
(especially for development)
Monday, November 24, 14
25. Performance?
26.00
19.50
13.00
6.50
0
OSb Linux (Host, Bare-metal)
Elapsed Time (sec)
for (12-billion times) val++;
Monday, November 24, 14
26. Virtio NIC
BitVisor
Hardware
App
OSv
No PRO/1000...
Virtio NIC please...
PRO/1000...
Monday, November 24, 14
27. Virtio NIC
BitVisor
Hardware
App
OSv
Virtio NIC
No PRO/1000...
Virtio NIC please...
Implementing Virtio
PRO/1000...
Monday, November 24, 14
28. How to implement?
BitVisor
Hardware
App
OSv
Virtio NIC
Finally, Our
Virtualization?
Monday, November 24, 14
29. How to implement?
BitVisor
Hardware
App
OSv
Virtio NIC
Finally, Our
Virtualization?
No,
BitVisor’s Way!
Monday, November 24, 14
30. Para Pass-through
Virtio
BitVisor
Hardware
Intercept only interesting I/Os!
Virtio Ring
PRO/1000 Ring
App
OSv
MSI-X etc.
PCI Functions
Pass-through!
Monday, November 24, 14
32. Faking PCI BARs
BAR0: IO space
for Virtio = PRO/1000 IO space
Everything else
Pass-through
(e.g. MSI-X)
Monday, November 24, 14
33. Virtio NIC Operations
• Virtio Ring
• Packet Transmission
• Packet Reception
Monday, November 24, 14
34. Virtio Ring
Guest selects rings:
* Transmission Ring
* Reception Ring
BAR0
(IO)
BitVisor emulates
these registers
Memory space
Virtio Ring (Available + Used)
Req. Req. Req.
Monday, November 24, 14
35. Packet Transmission
OSv
(Driver)
BitVisor
(Virtio NIC)
Transmission Ring
Req.
PRO/1000
Req.
Req.
Req. Req. Req.
Transmission Ring
Notify!
(PIO)
Req.
Req.
Reuse
VMExit
Monday, November 24, 14
36. Packet Reception
OSv
(Driver)
BitVisor
(Virtio NIC)
Reception Ring
Req.
PRO/1000
Req.
Req.
Req. Req. Req.
Reception Ring
Interrupt
Req.
Req.
Reuse
VMExit
Monday, November 24, 14
37. Netperf TCP_STREAM
1000.00
750.00
500.00
250.00
0
941.39
816.78
OSb Linux (Host, Bare-metal)
Throughput (Mbps)
Monday, November 24, 14
43. Still Under
Optimization
• TODOs
• Interrupt Moderation
• Zero Copy/VMDq
• Advanced PRO/1000 Descriptor
• TCP Offloading
Monday, November 24, 14
44. Summary &
Future Work
• Summary
• Running OSv on BitVisor
• Virtio NIC + Optimization
• Future work
• Further Optimization & Evaluation
• Other Virtio Devices (BLK, RNG...)
Monday, November 24, 14
45. Thank you !
http://www.root.ne.jp/nishide/shs/
Monday, November 24, 14