Call target C preprocessor from a script

Hello,

I have a pre-script that I use to pre-generate some files. I would like to use the C preprocessor to pre-process my input file (json) so that I can benefit from the #include and #ifdef directives before acutally processing the input json file.

I checked env, and I could not see anything usefull. I just need the cross compiler gcc path. env[“CC”] is equal to “cc” but I need the target gcc not the host one (to avoid having to install yet an other tool in my host).

Is it possible?

Thanks!

Correct. If you execute a command within the environment (PATH) that PlatformIO has created (env.Execute()) it should be fine.

Otherwise, duplicate of Change firmware name based on main.cpp - #2 by maxgerhardt.

thank you for the quick reply.
I checked again the link “duplicate” and I still fail to see how I can do it.

My point is that I need the xtensa or arm gcc, and env is returning me the host gcc.
I’ve searched all the links in the link you provided, and I see this:

Seems like there is no way to easily get the gcc of the toolchain without creating a search function myself?
If I can get ~/.platformio/packages/toolchain-xtensa-esp32/bin/xtensa-esp32-elf-gcc I will be happy! :sweat_smile:

o_O?

[env:esp32dev]
platform = espressif32
board = esp32dev
framework = arduino
extra_scripts = test.py
Import("env")

def search_compiler(env):
    from pathlib import Path, PurePath
    # Use any item in $PATH corresponding to a platformio toolchain bin folder
    path_separator = ':'
    gcc_exe = '*g++'
    if env['PLATFORM'] == 'win32':
        path_separator = ';'
        gcc_exe += ".exe"
    # Search for the compiler in PATH
    for ppath in map(Path, env['ENV']['PATH'].split(path_separator)):
        if ppath.match(env['PROJECT_PACKAGES_DIR'] + "/**/bin"):
            for gpath in ppath.glob(gcc_exe):
                gccpath = str(gpath.resolve())
                return gccpath
    gccpath = env.get('CXX')
    return gccpath

compiler = search_compiler(env)
print("Got compiler: " + str(compiler))
max@XXXX:~/temp/pio_find_compiler$ pio run -v
Processing esp32dev (platform: espressif32; board: esp32dev; framework: arduino; extra_scripts: test.py)
------------------------------------------------------------------------------------------------------------------------------
CONFIGURATION: https://docs.platformio.org/page/boards/espressif32/esp32dev.html
PLATFORM: Espressif 32 (6.0.0) > Espressif ESP32 Dev Module
HARDWARE: ESP32 240MHz, 320KB RAM, 4MB Flash
DEBUG: Current (cmsis-dap) External (cmsis-dap, esp-bridge, esp-prog, iot-bus-jtag, jlink, minimodule, olimex-arm-usb-ocd, olimex-arm-usb-ocd-h, olimex-arm-usb-tiny-h, olimex-jtag-tiny, tumpa)
PACKAGES: 
 - framework-arduinoespressif32 @ 3.20006.221224 (2.0.6) 
 - tool-esptoolpy @ 1.40400.0 (4.4.0) 
 - toolchain-xtensa-esp32 @ 8.4.0+2021r2-patch5
LDF: Library Dependency Finder -> https://bit.ly/configure-pio-ldf
LDF Modes: Finder ~ chain, Compatibility ~ soft
Found 33 compatible libraries
Scanning dependencies...
No dependencies
Building in release mode
Got compiler: /home/max/.platformio/packages/toolchain-xtensa-esp32@8.4.0+2021r2-patch5/bin/xtensa-esp32-elf-g++

And after that something like

cmd = [
  '"%s"' % compiler,
  "--version"
]
env.Execute(" ".join(cmd))

outputs

"/home/max/.platformio/packages/toolchain-xtensa-esp32@8.4.0+2021r2-patch5/bin/xtensa-esp32-elf-g++" --version
xtensa-esp32-elf-g++ (crosstool-NG esp-2021r2-patch5) 8.4.0
Copyright (C) 2018 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

And just to prove my point that indeed “CXX” or “CC” does 100% suffice:

Import("env")

cmd = [
  "$CXX",
  "--version"
]
env.Execute(" ".join(cmd))

outputs

xtensa-esp32-elf-g++ --version
xtensa-esp32-elf-g++ (crosstool-NG esp-2021r2-patch5) 8.4.0
Copyright (C) 2018 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

It really can’t get simpler and more elegant than that. SCons variable substitions in env.Execute() take care of it all.

Thanks!
I think I know what is the problem: in my case $CC is just cc, $CXX is g++ etc… and my script is a pre-script…
Seems like it is run before all the variables are set!

And I need to run before the compilation is started.
I’ve used in the past something like env.AddPreAction("buildprog", before_build) but it never worked (I suspect that the “buildprog” target does not exist anymore as “before_build” is never exectued)

If you reference the script like that and directly execute the preprocessing steps you need, not as a a pre or post action to something, it should work fine. Maybe check that the current target is “build” (and not Intellisense discovery) for safety as per Extra_scripts seems to be executed when I change GitHub branches in VSCode - #2 by maxgerhardt.

ok, got it. My issue was that the AddPreAction(“buildprog”… is only called if there is something to build! This is neat, but since I generate header files, I cannot rely on that mechanism.

So I’m facing a dilemma… If I use pre:extra_script.py, I got executed before the LDF and anything in platformio, but then the env is still not complete.
Or if I remove “pre:” and use a condition like in the link (is_pio_build()), the env is complete, but then I run after the LDF

This is maybe not a problem, since I explicitely set the deps here.

Anyway, thanks for your help, I have all the info I need to continue now! Thanks a lot!

Hello,

Just for information, it seems like I need to go back using a pre-extra script… When doing a clean build (ie, no generated source files yet), I can see the file being generated after the dependency graph being displayed and before the first compilation starts… however it does not pick up my generated C files for compilation (this is the case for ldf default value and ldf off)
If I run the compilation a second time, then it works (because the files are already generated)
Too bad, I didn’t expect this problem to be this difficult :slight_smile:

You can put them in a non-default build folder (like <root>/generated) and call env.BuildSources() on them, for example.