iPXE trivia #1: Teaching new cloudy hotness old INT 13h tricks
I love iPXE. Its fun, simple and easily rewarding, so I'm going to use it as a target of affection in trying to churn out more of these writing exercises. So here's is my first iPXE trivia post.
One of the more common uses of iPXE is to quickly boot a server into a "bootable" ISO over a network.
This is relatively well understood, with two commons paths:
- Boot into the ISO with memdisk as your kernel and the ISO as your initial RAM disk / filesystem.
- Boot into the ISO with sanboot
The simplicity of this kind of old-school functionality is one of the attributes that makes iPXE so fun to work with. "It can teach old dogs new tricks" to use a common phrase.
Sometimes it's easy to forget just how silly some of the old trick are, and revisiting them can sometimes help re-appreciate the context of how we approach the new tricks.
Basic Input/Output System and disk access
When an "traditional" x86 computer "boots", one of the major checkpoints it should eventually get to is loading it's BIOS.
When you strip away the modern context smog for the word BIOS, it's really just a very simple, pre-defined and communicated group of groups of functions, usually loaded from a ROM. These groups of functions served their purpose so well, they were built on and used for all sorts of purposes and schemes well beyond their original intent, and became "enshrined" in x86 computing, still being leveraged today.
INT13h (also sometimes referenced as 0x13 or int 13 is a group of functions specifically purposed around attached storage, at the time presuming the constructs of floppy or hdd storage. In order for a BIOS to load it's next boot stage from attached storage, it will do it through an INT13h function.
In the world of PC's with floppy drives and IDE attached hard disks, the functions themselves and their usage are generally within the realm of sanity. The loss of sanity and use of these functions for crazy purposes starts when you try to start doing things other than interacting with floppy and IDE drives, like the kinds of devices modern computers have, CD-ROMs, NICs, NVMe drives etc.
El-Torito, AKA booting from CD-ROM
Just for example, one of my favorite INT13h insanities is El-Torito(PDF). El-Torrito is a community agreed (for decades more like disagreed) upon specification that allows x86 PCs to boot from a CD-ROM (note not just ISO, but CD-ROM).
Booting from CD-ROM is not really a task the original group of group of functions called a BIOS was tooled to solve, but by this point (early 90s), it was already too late to try to significantly extend the base of functions. The devices we called "PCs" had become so structural to us that this functionality was needed without having to go and massively redefine the worlds understanding of how to "boot a PC". So smart folks had to figure out how to do what they wanted with what they had, which is generally a good starting place for a recipe of fun insanity.
Through some significant abuse of both INT19 (after POST OS load hooks) and INT13 (storage hooks), and an agreed upon hook of the two, El-Torito enables the ISO format of a CD-ROM can be accessed like it was a floppy to load a POST OS environment, namely the kernel and filesystem written as data inside the ISO filesystem.
When booting from CD-ROM, you are essentially abusing the same mechanic that allowed a computer to boot from floppy, just from an unexpected path.
It should be noted booting from CD-ROM is dependent on many more clever hacks and abuses, not just INT19/13h tricks. But that gets to my shower-thought:
When I think of the likely hundreds of millions of "Operating System" installs that have been completed by a "PC" "installing from CD-ROM', and that all of that dependent human workflow, is dependent on some very clever but egregious abuse of early computing primitives, I tend to find that pretty neat.
Insanities at cloudscale
Ok so booting from ISO is a neat insanity trick, but how about doing that at "cloudscale"?
iPXE has became a common tool in the operators "how to boot computers over a network" toolbelt. Orchestration / Virtualization ecosystems like OpenStack (plus Ironic), VMware, QEMU have all done fun things with iPXE. Cloud platforms like Equinix Metal and others enable their BYO-OS features through it. It's also just a great systems operator tool, I've become a big fan, hence this post.
One of iPXE's handiest tricks, booting a server over a network into into an ISO, builds on the El-Torito insanity. Using the iPXE sanboot feature, a computer can be booted into an ISO file, with iPXE handling all the plumbing. That feature loads an ISO file over a network, and then uses the El-Torito trick to load into it.
When it goes wrong, you cloud install environment reads a lot like an error message from the mid 1990s
So when you as an operator spin up a 40x node OpenShift cluster on Equinix Metal(shameless self plug), your cloud delivered, multi-thousand core application environment on-demand, it can be heartening to keep in mind that all that coolness comes from hundreds of thousands of clever but easily forgotten little hacks that have paid for themselves in ways that are difficult to describe.