Help achieving a batch build with build variables

Hello everyone! First time here. I’ve been searching for a solution for my need and i think i saw some person with similar need but i could not understand how to do this cause i’m pretty much rookie in all this platformio.ini options and environments. I really need extra help. What i need might be easy but i don’t know where to start. Here it goes:

I need to make 200+ .Bin files for ESP8266 on Arduino C language, but each one of them has to have 2 string variables that change between .bin file. My idea was to have a “database” file where i would put a name, variable1 and variable2 200 times and then somehow platformio would help me iterate thru that database file and build independent .bin where name.bin would output, and the inner variable1 and variable2 would match.

Is this somehow possible? has anyone done similar?
Thank you so much for your attention.

Yes, you can Advanced scripting for that. You should first replace the two strings that are subject to change with a macro. You can then use scripting to set the value of that macro to anything sourced from anywhere.

E.g., take two values from the top line of a file, let it build, then, then remove the top line from the file).

# obtain values, e.g. from file..
val1 = ..
val2 = ..
# inject as macros into build system
env.Append(CPPDEFINES=[
   ("VALUE1_STR", "\"" + val1 + "\""),
   ("VALUE2_STR", "\"" + val2 + "\""),
])

You can also modify the firmware name to keep track of which .bin has what variable value in it (see custom firmware name.

env.Replace(PROGNAME="firmware_%s_%s" % (val1, val2))

Further, you can use a post-action on the firmware binary to copy it into some other folder (otherwise the binary will be overwritten in the next build run). See post actions.

env.AddPostAction("$BUILD_DIR/${PROGNAME}.bin", callback_that_copies_file_and_prepares_next_values)
1 Like

Wow this is amazing. Thanks for your time in advance, wasn’t expecting such an extensive reply!

Gotta check how to make a macro. I’ve been also reading about advanced scripting but i dont even recognise the syntax. This is all very new for me so i dont even know what to look for, exactly. I’ll try to start from your example file and see where i can get.

“Advanced scripting” is Python scripting with SCons extension (the build system), FYI. PlatformIO is written in Python.

I can deal with a bit of python, SCons is something i really have to deal with for the first time.

Just another question, my initial idea was to be able to iterate an object and you sugested something like striping the var values from the file. The build process would not work for some python like (pseudo code): for each value in values export variable, build, and then iterate the next value?

Sure, you can write a high-level script that calls into PlatformIO, too. There are multiple ways to do this. A Python (or even, a simple bash or .bat) script can modify the platformio.ini (or source code) with the new values, call pio run to build the firmware, copy the resulting firmware to somewhere and rename it, then build the next firmware.

PlatformIO is accessible from the CLI too.

The way I showed above was how to do it in PlatformIO as opposed to on top of it.

1 Like

I’ve been trying just to get started but i am not getting it.

I have the following:

extra_scripts = pre:extra_script.py on platformio.ini

this text on extra_script.py:

Import ("env")

val1 = "test"
val2 = "test2"
# inject as macros into build system
env.Append(CPPDEFINES=[
   ("VALUE1_STR", "\"" + val1 + "\""),
   ("VALUE2_STR", "\"" + val2 + "\""),
])

And

#define FDS VALUE1_STR
Serial.println(FDS);

On ESP code.

The build output fails for the following:

src/Main.cpp: In function ‘void setup()’:
:0:12: error: ‘test’ was not declared in this scope
src/Main.cpp:7:13: note: in expansion of macro ‘VALUE1_STR’
#define FDS VALUE1_STR
^
src/Main.cpp:1088:20: note: in expansion of macro ‘FDS’
Serial.println(FDS);

where am i messing up? I want to pass all values as STRING to the ESP code.
Thanks once again

Ah my bad, the string escaping is a bit weird, you need to actually also include a \ character at the front and back to have it properly passed to GCC, see here.

Import ("env")

val1 = "test"
val2 = "test2"
# inject as macros into build system
env.Append(CPPDEFINES=[
   ("VALUE1_STR", "\\\"" + val1 + "\\\""),
   ("VALUE2_STR", "\\\"" + val2 + "\\\""),
])

YES! That did the trick!!!
Now i’m almost getting there. Already have the custom names, bin generated with custom name, all fine.
My problem is now getting to the move part and to check if there is another firmware to be built and if so, run build again.

Post actions are not after the build is made as i thought initially. From other posts you helped i’ve seen that env.AddPostAction(“buildprog”, callback…) should help me running some action just after build is done. I’ve tested buildprog and build and nothing seems to happen.
env.AddPostAction(“checkprogsize”, after_build) also doesn’t cut it.

env.AddPostAction("$BUILD_DIR/${PROGNAME}.bin", after_build) like you stated above also not helping.

${PROGNAME}.bin changed before by the env.Replace, and is working correctly. So i’m guessing PROGNAME is the same for the addpostaction.

I am also trying shutil.move but as it runs before the .bin is generated, it returns an error.
shutil.move("$BUILD_DIR/${PROGNAME}.bin", “/${PROGNAME}.bin”)

Thanks again. I’m almost there! :stuck_out_tongue:

Anyways, am i supposed to re-run build as a CLI if i need to build the next firmware? like os.system(pio run)

Thank you so much!

I DID IT!!! :smiley: i’m very happy this is amazing! Eventually i read some other reply from you and i tested different things and managed to get to success!! this is huge for me! thank you so much for all!

Can you post the solution?
Thank you!!