Issues using advanced scripting

Hi
I’m trying to start versioning my own project using platformio pre script

I’ve followed 2 methods, both ends with the same error. First one by adding !python:my_version_file.py into build_flags. (I’ll skip to the second method)

and the other method is by using extra_scripts.

I use git describe output to make version of my project.
I have 2 issues (3rd is less important now) :

When I hit pio run -v, It shows me less defines by 1 in verbose. For example it should define :

-DGIT_REVISION="%hash" 
-DGIT_REVISION_NO=0x%hash
-DMEEZANI_VERSION=0.0.1

what I really see is only the first 2.

The second (more important issue, I get this error if appended CPP Defines (by this method or the first one) :

xtensa-esp32-elf-g++: fatal error: no input files

a full log with verbose :

PS C:\Users\Hamza\Documents\PlatformIO\Projects\piotest> pio run -v
Processing esp32doit-devkit-v1 (framework: arduino; platform: espressif32; board: esp32doit-devkit-v1; monitor_speed: 115200; upload_speed: 921600; build_flags: -w; extra_scripts: pre:git_rev_macro.py; lib_deps: https://github.com/meezani/FastLED.git)
---------------------------------------------------------------------------------------------------------------------------------------
<SCons.Script.SConscript.SConsEnvironment object at 0x00000148D01FFA90>
CONFIGURATION: https://docs.platformio.org/page/boards/espressif32/esp32doit-devkit-v1.html
PLATFORM: Espressif 32 (2.0.0) > DOIT ESP32 DEVKIT V1
HARDWARE: ESP32 240MHz, 320KB RAM, 4MB Flash
DEBUG: Current (esp-prog) External (esp-prog, iot-bus-jtag, jlink, minimodule, olimex-arm-usb-ocd, olimex-arm-usb-ocd-h, olimex-arm-usb-tiny-h, olimex-jtag-tiny, tumpa)
PACKAGES:
 - framework-arduinoespressif32 3.10004.200129 (1.0.4)
 - tool-esptoolpy 1.20600.0 (2.6.0)
 - toolchain-xtensa32 2.50200.80 (5.2.0)
LDF: Library Dependency Finder -> http://bit.ly/configure-pio-ldf
LDF Modes: Finder ~ chain, Compatibility ~ soft
Found 27 compatible libraries
Scanning dependencies...
Dependency Graph
|-- <FastLED> 3.3.3+sha.1c3fbd0 [git+https://github.com/meezani/FastLED.git] (C:\Users\Hamza\Documents\PlatformIO\Projects\piotest\.pio\libdeps\esp32doit-devkit-v1\FastLED)
|   |-- <SPI> 1.0 (C:\Users\Hamza\.platformio\packages\framework-arduinoespressif32\libraries\SPI)
Building in release mode
xtensa-esp32-elf-g++ -o .pio\build\esp32doit-devkit-v1\lib237\FastLED\FastLED.cpp.o -c -fno-rtti -fno-exceptions -std=gnu++11 -w -Os -g3 -Wall -nostdlib -Wpointer-arith -Wno-error=unused-but-set-variable -Wno-error=unused-variable -mlongcalls -ffunction-sections -fdata-sections -fstrict-volatile-bitfields -Wno-error=deprecated-declarations -Wno-error=unused-function -Wno-unused-parameter -Wno-sign-compare -fstack-protector -fexceptions -Werror=reorder -DGIT_REVISION="2dd02317cfad6998e11fe6563beee92190454f0b" -DGIT_REVISION_NO=0x2dd02317cfad6998e11fe6563beee92190454f0b
xtensa-esp32-elf-g++ -o .pio\build\esp32doit-devkit-v1\lib237\FastLED\bitswap.cpp.o -c -fno-rtti -fno-exceptions -std=gnu++11 -w -Os -g3 -Wall -nostdlib -Wpointer-arith -Wno-error=unused-but-set-variable -Wno-error=unused-variable -mlongcalls -ffunction-sections -fdata-sections -fstrict-volatile-bitfields -Wno-error=deprecated-declarations -Wno-error=unused-function -Wno-unused-parameter -Wno-sign-compare -fstack-protector -fexceptions -Werror=reorder -DGIT_REVISION="2dd02317cfad6998e11fe6563beee92190454f0b" -DGIT_REVISION_NO=0x2dd02317cfad6998e11fe6563beee92190454f0b
xtensa-esp32-elf-g++: fatal error: no input files
compilation terminated.
xtensa-esp32-elf-g++ -o .pio\build\esp32doit-devkit-v1\lib237\FastLED\colorpalettes.cpp.o -c -fno-rtti -fno-exceptions -std=gnu++11 -w 
-Os -g3 -Wall -nostdlib -Wpointer-arith -Wno-error=unused-but-set-variable -Wno-error=unused-variable -mlongcalls -ffunction-sections -fdata-sections -fstrict-volatile-bitfields -Wno-error=deprecated-declarations -Wno-error=unused-function -Wno-unused-parameter -Wno-sign-compare -fstack-protector -fexceptions -Werror=reorder -DGIT_REVISION="2dd02317cfad6998e11fe6563beee92190454f0b" -DGIT_REVISION_NO=0x2dd02317cfad6998e11fe6563beee92190454f0b
xtensa-esp32-elf-g++: fatal error: no input files
compilation terminated.
xtensa-esp32-elf-g++: fatal error: no input files
compilation terminated.
xtensa-esp32-elf-g++ -o .pio\build\esp32doit-devkit-v1\lib237\FastLED\colorutils.cpp.o -c -fno-rtti -fno-exceptions -std=gnu++11 -w -Os -g3 -Wall -nostdlib -Wpointer-arith -Wno-error=unused-but-set-variable -Wno-error=unused-variable -mlongcalls -ffunction-sections -fdata-sections -fstrict-volatile-bitfields -Wno-error=deprecated-declarations -Wno-error=unused-function -Wno-unused-parameter -Wno-sign-compare -fstack-protector -fexceptions -Werror=reorder -DGIT_REVISION="2dd02317cfad6998e11fe6563beee92190454f0b" -DGIT_REVISION_NO=0x2dd02317cfad6998e11fe6563beee92190454f0b
*** [.pio\build\esp32doit-devkit-v1\lib237\FastLED\FastLED.cpp.o] Error 1
*** [.pio\build\esp32doit-devkit-v1\lib237\FastLED\bitswap.cpp.o] Error 1
*** [.pio\build\esp32doit-devkit-v1\lib237\FastLED\colorpalettes.cpp.o] Error 1
xtensa-esp32-elf-g++: fatal error: no input files
compilation terminated.
*** [.pio\build\esp32doit-devkit-v1\lib237\FastLED\colorutils.cpp.o] Error 1
===================================================== [FAILED] Took 4.37 seconds ===================================================== 

However I’ve created a sample project to reproduce the issue, It’s uploaded here.

For a quick eye :
git_rev_macro.py :

import subprocess

projectTag = (
    subprocess.run(['git', 'describe','--tags','--abbrev=0'], capture_output=True, text=True).stdout
)
revision = (
    subprocess.run(['git', 'rev-parse','HEAD','--short=10'], capture_output=True, text=True).stdout
)
gitDescribeOutput = (
    subprocess.run(['git', 'describe','--tags'], capture_output=True, text=True).stdout
)
info = gitDescribeOutput.split('-')


Import("env")

print(env)

env.Append(CPPDEFINES=[
  ("GIT_REV", "\"%s\""%revision.strip()),
  ("GIT_REV_NO", "0x%s"%revision),
  ("MEEZANI_VERSION", projectTag),
#   ("AHEAD_COMMITS", info[1]),
])

platformio.ini:

[platformio]
default_envs = esp32doit-devkit-v1

[common]
framework = arduino
monitor_speed = 115200
lib_deps = 
  https://github.com/meezani/FastLED.git

upload_speed = 921600

[env:esp32doit-devkit-v1]
framework = ${common.framework}
platform = espressif32
board = esp32doit-devkit-v1

monitor_speed = ${common.monitor_speed}
upload_speed = ${common.upload_speed}
build_flags = -w

extra_scripts = pre:git_rev_macro.py

lib_deps = ${common.lib_deps}

The project is empty. (except for #include <FastLED.h> after <Arduino.h>)

This has to be a string. It is an invalid double right now because of two .

I see,

For this exact issue, I’m wondering how it’s done without issues here in esp32 arduino core_version.h

their shell command : arduino-esp32/on-release.sh at be4d3b6cb87236a749a0abc8288b49cb6b79710b · espressif/arduino-esp32 · GitHub

EDIT :

I think it’s not used at all, And if someone wants to use it, He must encompass it with parenthesis like “ARDUINO_ESP32_GIT_DESC”

Try the method that I outlined in this answer:

From what you describe, its exactly what you want.

1 Like

Thanks for sharing, I’ve the following code, Any comment ?

"""Git revision"""

import subprocess


revision = subprocess.check_output(["git", "rev-parse", "HEAD"]).strip().decode("utf-8")
REVISION = revision[:10]
gitDescribeOutput = subprocess.check_output(["git", "describe", "--tags"]).strip().decode("utf-8")
info = gitDescribeOutput.split('-')
PROJECT_TAG = info[0]
AHEAD_COMMITS = 0
if len(info) == 3:
    AHEAD_COMMITS = info[1]
elif len(info) == 4:
    PROJECT_TAG = info[0]+"-"+info[1]
    AHEAD_COMMITS = info[2]

PROJECT_TAG_NO = PROJECT_TAG.replace('.','0')


print("-D GIT_REV='\"{}\"'".format(REVISION))
print("-D ITANK_VERSION='\"{}\"'".format(PROJECT_TAG))
print("-D ITANK_VERSION_NO='{}'".format(PROJECT_TAG_NO))
print("-D AHEAD_COMMITS='{}'".format(AHEAD_COMMITS))

1 Like

Looks great!
If it gets the values you want, that’s awesome. There may be easier ways to get the values you want, like hash and latest tag. I specifically wanted to use the git describe output itself, So that was easiest for me.

To inject those values into the PIO build, make sure to use:

Import("env")
env.Append(
    BUILD_FLAGS=[whatever_flag_string, another_flag_string]
)

Where the flag strings are the strings you printed out.

It is important that the string have the " escaped in the PIO env. So the string in python should be:
"-D FLAG_NAME=\\\"flag_value_string\\\"" or '-D FLAG_NAME=\\"flag_value_string\\"'
Which results in -D FLAG_NAME=\"flag_value\" in the PIO env

1 Like

Wow, great information!

I’m using a different strategy, Where the script is printing directly to extra_flags field in platformio.ini

for git describe, Why am I getting different version ?

PS F:\Projects\iTank\New Version\Software\Project\iTankGateway> git describe
0.7.0-4-g3bd261b
PS F:\Projects\iTank\New Version\Software\Project\iTankGateway> git describe --tags
1.0.0-rc-3-g3bd261b

@hamzahajeir, refer to the git documentation for the full explanation of the git describe command and its flags:

Git - git-describe Documentation

TLDR:
The reason you are getting a difference output is because git describe defaults to just using annotated tags. If you add the --tags flag, it uses any tag, regardless of whether it is annotated or not.

For your example, 0.7.0 is the most recent annotated tag, where 1.0.0-rc-3 is the most recent tag (of any type).

1 Like

Thank you @awbmilne for the clarification.