Custom build step that mimics makefile rule?

I would like to have a python script that generates some source code from a data file in the project tree. Since the processing time can be considerable, it would be best to have it set up like a compilation step - to run only when needed, based on input and output files timestamps (like a regular makefile rule)

Is there a way to set up extra_script like that ?

you can use SCons as Make ( secondary build task )

I can’t find any documentation reference or examples of that - do you have any links ?

dependent-target is the closest I could find, but it has no provision for specifying the output file, so I don’t see how it can detect the need for rebuild.

what you need
secondary builder or “makefile” for main builder

this is imitation of “makefile” - add not compiled sourse files from framework

for secondary builder, from this file as example you can run SCons as independent Make

the SConscript() function might also work for you

So the first parameter would be “script file” in the SCons “language” ?

Hmm, where would be the right place to call this function ? Pre-build script ? Post-build script ?

(sorry, it takes some time to understand the relations between platformio and SCons system)

Edit: this is extremely confusing. In makefile, that’s two simple lines. I’ve browsed through thousands of lines of code and docs so far, and it looks I’m no closes to a solution.

You need to use Advanced Scripting and PRE-script that generates source files to src / includes folders. PlatformIO will do the rest for you using incremental build.

See very complex example just to get some hints on what you can do with advanced scripting.

Thanks !

Silly question regarding this example: it seems to spend a lot of effort on calculating and caching file hashes. Isn’t this functionality provided by SCons ?

explain in detail exactly what you want to do
make or makefile?

1 Like

The simplest example of what I look for would be:

  • the project folder contains “somedata.dat” file
  • there is a python function to read this file, and write “somedata.c” with something like "static uint8_t somedata[] = { … } ", based on the contents of this file.
  • I want this function to be called every time the somedata.dat file is changed, the same way compiler is executed and generates object file every time a source file is changed.

After just now reading SCons documentation on “deciders” i see that the criteria for it are somewhat different in makefiles and SCons, but that detail doesn’t really matter for my simple case.
I’m looking for a simplest solution to detect file change (hopefuly using built-in mechanism and not having to roll my own).

The equivalent makefile rule would have been:

somedata.c: somedata.dat
    python gen_script.py somedata.c somedata.dat

(appreciate your help guys)

preAction( generate_dat_c.py → somedata.c )
BuildSources(DST, somedata.c)

something like that

Where would I reference the “somedata.dat” source file in this approach ? It seems that PreAction is unconditional and is always executed.

I can’t find any documentation of BuildSources anywhere … where can I read what it does ?

From reading SCons documentation, it appears the correct mechanism for this task would be env.Command('somedata.c', 'somedata.dat', action=build_function)

However this seem to have no visible effect in platformio. The function is never called, and build tree displayed with SCONSFLAGS=--tree=all has no node for somedata.dat.
Tried executing from both pre and post scripts, and tried both env and projenv environments.

Any ideas ?

That was apparently because the task was not linked to the main build target. Adding Depends() call fixed it.

extra_script.py:

Import("env")
c = env.Command('msgs.c', 'msgs.txt', action="cp $SOURCE $TARGET")
env.Depends("$BUILD_DIR/$PROGNAME$PROGSUFFIX", c)