PlatformIO Community

Running a post-build-script from library?

Hi!

Is it possible to run a post-build script from a library?
In the library.json I already added "build":{"extraScript:":"versioning.py"}.

The script runs before the build process and creates the files versioning.txt in the project folder and version.h in the src folder.

I would also like to add another script that runs after the build process and copies the resulting firmware.bin to a folder firmware and creates another file firmware.nfo.

Is this possible at all?

Note: I am not a python expert :innocent:

You can create a post action on the firmware.bin file as shown in the docs and execute arbitrary logic.

You might need to use the newly available projenv environment (and only this environment, not others) for this, as showcased in Force RTTI from library? - #23 by cziter15.

Hi @maxgerhardt

I have read the doc, but unfortunately I understand only half of it :frowning:

I have once created a simple version of the script:

Import("env", "projenv")

def copy_firmware(source, target, env):
    print("test copy_firmware() is called")

env.AddPostAction("buildprog", copy_firmware)
print("test: script gets called")

Import("env", "projenv") results in *** Import of non-existent variable ''projenv''

Removing “projenv” fixed the error, but the function copy_firmware() is never called.

This is the output from the build process:

> Executing task in folder Firmware_Builder: C:\Users\Boris\.platformio\penv\Scripts\platformio.exe run --environment ESP32 <

Processing ESP32 (platform: espressif32; board: esp32dev; framework: arduino)
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------Verbose mode can be enabled via `-v, --verbose` option
CONFIGURATION: https://docs.platformio.org/page/boards/espressif32/esp32dev.html
PLATFORM: Espressif 32 (4.4.0) > Espressif ESP32 Dev Module
HARDWARE: ESP32 240MHz, 320KB RAM, 4MB Flash
DEBUG: Current (cmsis-dap) External (cmsis-dap, esp-prog, iot-bus-jtag, jlink, minimodule, olimex-arm-usb-ocd, olimex-arm-usb-ocd-h, olimex-arm-usb-tiny-h, olimex-jtag-tiny, tumpa)
PACKAGES:
 - framework-arduinoespressif32 @ 3.20003.220619 (2.0.3)
 - tool-esptoolpy @ 1.30300.0 (3.3.0)
 - toolchain-xtensa-esp32 @ 8.4.0+2021r2-patch3
LDF: Library Dependency Finder -> https://bit.ly/configure-pio-ldf
LDF Modes: Finder ~ chain, Compatibility ~ soft
test: script gets called
Found 40 compatible libraries
Scanning dependencies...
Dependency Graph
|-- SimpleHTTPUpdater @ 0.1.0+sha.2c6fbc6
|   |-- HTTPClient @ 2.0.0
|   |   |-- WiFi @ 2.0.0
|   |   |-- WiFiClientSecure @ 2.0.0
|   |   |   |-- WiFi @ 2.0.0
|   |-- Update @ 2.0.0
|-- WiFi @ 2.0.0
Building in release mode
Compiling .pio\build\ESP32\src\main.cpp.o
Linking .pio\build\ESP32\firmware.elf
Retrieving maximum program size .pio\build\ESP32\firmware.elf
Checking size .pio\build\ESP32\firmware.elf
Advanced Memory Usage is available via "PlatformIO Home > Project Inspect"
RAM:   [=         ]  12.1% (used 39776 bytes from 327680 bytes)
Flash: [=======   ]  65.3% (used 855945 bytes from 1310720 bytes)
Building .pio\build\ESP32\firmware.bin
esptool.py v3.3
Creating esp32 image...
Merged 25 ELF sections
Successfully created esp32 image.

Addition:

Putting extra_scripts = versioning.py in platformio.ini works:

... shortened ...
Dependency Graph
|-- SimpleHTTPUpdater @ 0.1.0+sha.2c6fbc6
|   |-- HTTPClient @ 2.0.0
|   |   |-- WiFi @ 2.0.0
|   |   |-- WiFiClientSecure @ 2.0.0     
|   |   |   |-- WiFi @ 2.0.0
|   |-- Update @ 2.0.0
|-- WiFi @ 2.0.0
Building in release mode
test: script gets called
... shortened ...
Linking .pio\build\ESP32\firmware.elf
Retrieving maximum program size .pio\build\ESP32\firmware.elf
Checking size .pio\build\ESP32\firmware.elf
Advanced Memory Usage is available via "PlatformIO Home > Project Inspect"
RAM:   [=         ]  12.1% (used 39776 bytes from 327680 bytes)
Flash: [=======   ]  65.3% (used 855945 bytes from 1310720 bytes)
Building .pio\build\ESP32\firmware.bin
esptool.py v3.3
Creating esp32 image...
Merged 25 ELF sections
Successfully created esp32 image.
copy_firmware(["buildprog"], [".pio\build\ESP32\firmware.bin"])
test: copy_firmware() gets called

But my goal is to avoid manual changes to platformio.ini.

This feature is only available in the dev version for now, hasn’t yet been published into a new release. CLI + pio upgrade --dev to upgrade, then retry with projenv.

Since it’s a library, I would gladly do without the developer version. This could cause problems for the users of the library who do not have the developer version installed.

Is the execution of a post-build script dependent on the “projenv”?

Hm yes actually this doesn’t seem to be possible from a library’s extraScript at all, I tested this with

Import('env', 'projenv')

def copy_firmware(source, target, env):
    print("test copy_firmware() is called")

def post_bin_action(source, target, env):
    print("test post_bin_action() is called")

def post_progsize(source, target, env):
    print("test post_progsize() is called")

for e in (env, projenv, DefaultEnvironment()):
    e.AddPostAction("buildprog", copy_firmware)
    e.AddPostAction("checkprogsize", post_progsize)
    # for Uno projects: firmware.hex
    e.AddPostAction("$BUILD_DIR/${PROGNAME}.hex", post_bin_action)
    # for other projects
    e.AddPostAction("$BUILD_DIR/${PROGNAME}.bin", post_bin_action)
print("test: script gets called")

and non but the test: script gets called shows up – maybe these post actions are not available in library scripts? Could you confirm that @ivankravets? Test project is at https://github.com/maxgerhardt/pio-script-hooks-actions

(PS: Some library authors restor to just storing the python script in the library and then tell users to reference it in the platformio.ini, like in here).

1 Like

@maxgerhardt Thanks for testing and confirming.
I will wait for the feedback from @ivankravets

Hi there,

This is a historical issue. It is a bug from the PlatformIO’s perspective but from the SCons (which we use internally for the building) this is a feature. See

So, if the target is not ready yet, action will not be assigned.

Nevertheless, I agree that we should fix that. So, I played a little bit and wrapped SCons handlers to catch these cases when targets are not ready. See

Please re-test with PIO Core 6.1-dev. We increased PIO Core minor version. A lot of changes and new features. Does it work for you?

Thank you very much for the quick answer and implementation. Unfortunately, I am away for a few days. I will test it when I return and report back.

With 6.1.0b1 and the same test code, only in combination with environment = DefaultEnvironment() (not env or projenv), it triggers the e.AddPostAction("$BUILD_DIR/${PROGNAME}.hex", post_bin_action) action. Post actions for buildprog and checkprogsize are not triggered. Registering these callbacks only in env, only in projenv or in both makes no AddPostAction() callback trigger – strange.

1 Like

Thanks! Fixed! Please re-test with pio upgrade --dev. Should finally work.

After that, all actions trigger when using e = DefaultEnvironment()

avr-g++ -o .pio\build\uno\firmware.elf -mmcu=atmega328p -Os -Wl,--gc-sections -flto -fuse-linker-plugin .pio\build\uno\src\main.cpp.o -L.pio\build\uno -Wl,--start-group .pio\build\uno\libfc0\libTestLib.a .pio\build\uno\libFrameworkArduinoVariant.a .pio\build\uno\libFrameworkArduino.a -lm -Wl,--end-group
MethodWrapper(["checkprogsize"], [".pio\build\uno\firmware.elf"])
Advanced Memory Usage is available via "PlatformIO Home > Project Inspect"
RAM:   [          ]   0.4% (used 9 bytes from 2048 bytes)
Flash: [          ]   1.4% (used 444 bytes from 32256 bytes)
[...]
post_progsize(["checkprogsize"], [".pio\build\uno\firmware.elf"])
>>>>>>>>test post_progsize() is called<<<<<<<<<<<<
avr-objcopy -O ihex -R .eeprom .pio\build\uno\firmware.elf .pio\build\uno\firmware.hex
post_bin_action([".pio\build\uno\firmware.hex"], [".pio\build\uno\firmware.elf"])
>>>>>>>>test post_bin_action() is called<<<<<<<<<<<<
copy_firmware(["buildprog"], [".pio\build\uno\firmware.hex"])
>>>>>>>>test copy_firmware() is called<<<<<<<<<<<<

But still none of the callbacks fire when using env and projenv – maybe I’m misunderstanding these environments?

This is correct behavior. There are no targets in these envs on which you plan to assign actions. These targets are declared in the main/global env where “progenv” and “env” (lib env) are cloned/isolated environments. We have this info in the docs Construction Environments — PlatformIO latest documentation

1 Like

Sorry for the late feedback.
I have tested the latest dev version and 6.1.0.
In both versions it works with DefaultEnvironment() (as the note in the documentation describes).

1 Like