Changing upload binary file via pre-build script

Development Platforms
1

There’s got to be some super simple solution to this that I’m missing….

How do I use a Python pre-build script to change the default upload target path to something other than the default, e.g. from “firmware.elf” to “firmware-bootloader.elf”?

NOTE: This is not “changing the build output filename”…no, this is changing the uploaded filename after building with the default filename is complete. Completely useless unless as in my case, I’m doing an ELF merge via said build script…and want to upload the resulting file, without overwriting the original file.

Digging into the builder script, we can see where the OpenOCD command line is set:

This indicates that the path passed to OpenOCD is contained in an (environment? PIO context? SCONS context?) variable named “$SOURCE”.
There’s a hint in the PlatformIO documentation that this value is “replaced by a real firmware/binary”: (upload_command — PlatformIO latest documentation)

However, I have not been able to find where this “replacement” takes place. A PlatformIO Github search turns up no results for “$SOURCE” or even “SOURCE” (case sensitive: “/(?-i)SOURCE/”).

An environment dump in the script callback…

Import("env")

def before_upload(source, target, env):
	print(source, target, env.Dump())

env.AddPreAction("upload", before_upload)

…has the filename provided in the “target” argument (and writing to this makes no change)–but there’s no environment entry for “SOURCE”:

  'SMARTLINK': <function smart_link at 0x75235a069080>,
  'SPAWN': <function subprocess_spawn at 0x7523596623e0>,
  'STATIC_AND_SHARED_OBJECTS_ARE_THE_SAME': 0,
  'TARGET_ARCH': None,
  'TARGET_OS': None,

In the environment, the “firmware” is found in “PROGNAME”…but changing it with:

env.Replace(PROGNAME="${PIOENV}-bootloader")

(or as appropriate) does not make any difference to the path that “$SOURCE” is replaced with before invoking OpenOCD.

In this community post, someone asked a vaguely similar question:

but the solution ended up being different than here.

Where is this “$SOURCE” resolved, and how can I override it via a Python build script?

2

The $SOURCE for the upload action is defined in

Aka it’s target_firm, which in turn is defined as the either the .hex or .bin result of the firmware build.

So what you’re trying to do is similiar to Zephyr when the firmware includes MCUBoot.

https://github.com/platformio/platform-ststm32/blob/c6e392c616d141444421610a5d5ea65cf814e2d2/builder/main.py#L107-L127

However, I think the cheapest way to achieve this without trying to bend the target_firm, is to simply make the merge process so that it backups the original (pre-merge) firmware.elf as firware-nobootloader.elf, and merges that file with the bootloader so that it overwrites the original firmware.elf (now containing the bootloader). Then everything will work as normal.

Otherwise, you can also can set a custom upload command

upload_protocol = custom
upload_command = ${platformio.packages_dir}/tool-openocd/openocd -f <config files...> -c "program {.pio/build/myenvname/firmware-bootloader.elf} verify reset; shutdown;"
3

What I’m not understanding is where the “$SOURCE” text in a string or path is replaced with the predetermined path. I know enough about programming to know that nothing happens “magically”–code somewhere is replacing “$SOURCE” with a path (retrieved from somewhere) before UPLOADCMD is executed as a shell command. It would make sense if “$SOURCE” was a console context environment variable–in which case, it should be easily possible to change it…?

“$SOURCE” is also translated into a path in the console output for the upload hook:

e.g. “Uploading [file]” is printed to the console before OpenOCD is run.

Somewhat of a clue is that “$SOURCE” is a relative path to the project folder, not an absolute system path. Meanwhile, all of the “PROJPATH” and other PIO environment variables, are all absolute system paths.

The hack I’ve come up with thus far, is to iterate through and replace “$SOURCE” in “UPLOADFLAGS” with the path to the actual binary (and found that it’s important to leave the curly brackets around the path.) This of course does not display the correct file with “Uploading [path]” (nor with the firmware size check)…but at least OpenOCD is passed the correct path.

To make sure I’m understanding correctly, it is somehow not possible to override/replace/adjust the $SOURCE variable via pre- or post- Python script?

4

PlatformIO is built upon SCons (https://github.com/SCons/scons). SCons will substitute $SOURCE with the target of the Action, which again per above, is the .bin or .hex file. This does not happen inside PlatformIO, but the SCons build engine.

https://scons.org/doc/production/HTML/scons-user.html#cv-SOURCE

5

Thanks. Answers that part of the mystery, anyway.