My problem: I’m splitting my firmware into a “boot loader and OS” part and an “application” part, in order to OTA-flash the application separately. The application calls some bootloader functions, thus I’d like to teach “pio run” that it needs to link the boot loader before linking the corresponding app.
Is there a way to specify that? Ideally, using a custom extra script, because I’m already using that to build (part of) the application’s linker command line.
Good question. I haven’t yet seen such a of this dynamic linking between two images. So you would e.g. build a bootloader the resulting .elf file would place a function abc() at a pretty random address, e.g., 0x80000ab, and on a application rebuild you would want a call to that function go directly to 0x80000ab?
Forum posts like Share code between bootloader and application and Dynamic Linking of librairies without and OS suggest that creating a jump- or function table at a constant place in flash would be an option. The table would then contain the actual addresses of the functions with a convention that e.g. the first index would describe function f_1, etc.
In the ESP-IDF project for the ESP32 project I’ve also seen that the linker script just exposes addresses of functions, which are e.g. stored in its factory-programmed, unchangable mask-ROM, so that these calls can be resolved.
With a header declaration like e.g.
that can then be resolved.
With a little bit of an extra script, such linker script directives can also be surely generated by running the bootloader elf file through the toolchain-specific nm tool to view the symbols and its addresses, then extract the addresses for the needed functions and generate the PROVIDE(func = addr); text.
But that has also the significant drawback that an application would be linked against one specific bootloader version and the bootloader is not interchangable. If bootloader v1 placed function abc() at one address and then app v1 was linked against that, as soon as another bootloader version would put the function at a different address while the app is kept the same, the app would crash, since it points to the old address. Same for the way if an app was linked with a newer bootloader but being placed in flash where an older bootloader is.
Thus the “jump table” approach would be prefferable I guess since an application would be invariant against internally changing addresses in the bootloader, as long as the correct address is always placed in the jump table of known constant address. Enforcing that an object like that jumptable is placed at a predictable address can also be done with the linker script in the bootloader I suppose. On modern operating systems, the PLT (procedure linkage table) and GOT (global offset table) (https://www.technovelty.org/linux/plt-and-got-the-key-to-code-sharing-and-dynamic-libraries.html) are a little fancier versions of a simple jump table, so one can have a look at how those work too.
There may also be a way where the bootloader is compiled to an .so file that is dynamically linked against (without the bootloader code being placed in the application as result) with some link flags, I’m unsure how that would concretely work in this instance though.
I did think long and hard about using a jump table or some other “structured” approach.
Unfortunately it’s not that easy. Consider for instance a serial interface. The boot loader uses one, which pulls in a heap of support code from the system library. Now the application comes along and wants to use another serial interface, thus requiring access to that support code. I cannot add the serial’s internal functions to the jump table because I don’t even know their names, which means that the app will contain a duplicate code. Worse, I might end up with duplicate data structures, causing inconsistencies.
My boot loader isn’t exactly small, it occupies almost half the flash on the smaller chips I use (can’t go above that, the boot loader must be able to re-flash itself) because I’m implementing a multi-wire communication protocol, so there’s also debug code and whatnot in there. Adding all that to a jump table takes time and effort I’d rather spend improving the code.
I already deal with versioning: the app header contains a checksum of the bootloader.
Why would internal support functions need to be added to the jumptable in the first place, if the serial driver has a defined high-level interface, just expose that, once execution reaches that function it will just call its internal functions itself? You can even make the function in the jumptable go to a little wrapper yourself to be invariant against changes in the underlying serial driver, since you would have to maintain binary compatibility of the established functions in the jumptable.
The serial driver has a single well-defined interface: a C++ object. Its data contains a pointer to a vtable which in turn contains pointers to a bunch of functions with interestingly-mangled names. The compiler only calls those functions through the vtable if they’re virtual – otherwise it calls them directly.
I assume that you can now see how this could be a slight problem.