Extra_scripts and imprintig FW ver in code: rebuild all files always

1

I would like to see in my firmware its version and build time. So I found a python file that sets a defyne with this text and just output it to the UART:

Serial.print("Firmware Version: ");
Serial.println(AUTO_VERSION);

The file itself is connected like this:

extra_scripts = 
    pre:auto_firmware_version.py

But the problem is that now the project is rebuilt from scratch every time. If you fix BUILD_FLAGS to BUILD_FLAGS_ in the script, this behavior stops, so I think this is a consequence of the BUILD_FLAGS modification. But I couldn’t go any further in analyzing this problem.
The script itself looks like this:

 import subprocess
import datetime

Import("env")

def get_firmware_specifier_build_flag():
    ret = subprocess.run(["git", "describe", "--tags", "--long"], stdout=subprocess.PIPE, text=True) #Uses any tags
    build_time = datetime.datetime.now().strftime("%Y-%m-%d--%H:%M:%S")
    build_version = ret.stdout.strip()
    build_flag = "-D AUTO_VERSION=\\\"" + build_version+ "@" + build_time + "\\\""
    print ("Firmware Revision: " + build_version)
    return (build_flag)

env.Append(
    BUILD_FLAGS=[get_firmware_specifier_build_flag()]
)
2

Nice script!

It smells like it’s comming from the OpenDTU project?!

I think the reason for the complete rebuild is, that build_flags are modified, which applies to every source file (including the framework source files)

Modifying the build_src_flags instead should be the solution.

Unfortunately I’m not familiar with SCons and python and tried to set build_src_flags from within the script for 2 hours now without any success :cry:.

cc @ivankravets , @valeros Do you have a solution for this?

3

I honestly don’t remember where exactly I found it, and just edited it for git describe --tags --long and adding build times.

4

The project is rebuilt because the AUTO_VERSION macro changes every second.

5

Hi valeros! Thanks for stopping by!

Yes, that is correct.

But isn’t it possible to avoid recompiling the whole framework by using the build_src_flags instead of the build_flags?

Unfortunately I was not able to set the build_src_flags from within a script.

6

I read through the platformio documentation (it’s about time, yes), and discovered another way to do the same thing.

import subprocess
import datetime

Import("env")

def get_firmware_specifier_build_flag():
    #ret = subprocess.run(["git", "describe"], stdout=subprocess.PIPE, text=True) #Uses only annotated tags
    ret = subprocess.run(["git", "describe", "--tags", "--long"], stdout=subprocess.PIPE, text=True) #Uses any tags
    build_time = datetime.datetime.now().strftime("%Y-%m-%d--%H:%M:%S")
    build_version = ret.stdout.strip()
    build_flag = "\\\"" + build_version+ "@" + build_time + "\\\""
    print ("Firmware Revision: " + build_version)
    return (build_flag)

env.Append(CPPDEFINES=[ ("AUTO_VERSION", get_firmware_specifier_build_flag()) ])

It works exactly the same way, also forcing you to rebuild all the code. I realize that the macro changes every second, but it only affects one file, and I expected that only that file would be compiled every time I build the project.

What if I modify the script so that it records the time in an .h file, which I then add to my code via #include?

7

It should be possible to use SRC_BUILD_FLAGS instead of BUILD_FLAGS in an extra script. There is also an alternative way of adding build flags to a specific file via a middleware function, more info here.

1 Like
8

Yeah that’s the solution!!! Many thanks!

(btw: Where do I have to search for SRC_BUILD_FLAGS? There is nothing in the docs)

9

@vvzvlad final solution is to change

env.Append(
    BUILD_FLAGS=[get_firmware_specifier_build_flag()]
)

to

env.Append(
    SRC_BUILD_FLAGS=[get_firmware_specifier_build_flag()]
)

Works like a charm!

10

AFAIR there is no docs for BUILD_FLAGS or SRC_BUILD_FLAGS mainly because we don’t encourage developers to use environment variables used by PlatformIO build system internally, although it should be safe to do so. I’d rather recommend using a proper environment variable called PLATFORMIO_BUILD_SRC_FLAGS.

11

That was my 2nd approach.
Due to my lack of Python knowledge, I wasn’t able to do this either.

12

Chatgpt was kind enough to generate me a variant with the file addition and after a few minor edits it works fine.

import subprocess
import datetime
from os.path import join

Import("env")

def get_build_info():
    ret = subprocess.run(["git", "describe", "--tags", "--long"], stdout=subprocess.PIPE, text=True)
    build_version = ret.stdout.strip()
    build_time = datetime.datetime.now().strftime("%Y-%m-%d--%H:%M:%S")
    build_flag = f"\"{build_version}@{build_time}\""
    return build_flag

def generate_imprint_file(build_flag, project_dir):
    imprint_file_path = join(project_dir, 'include', 'imprint.h')
    with open(imprint_file_path, 'w') as f:
        f.write(f'#define AUTO_VERSION {build_flag}\n')

build_flag = get_build_info() # Get build information
project_dir = env.subst("$PROJECT_DIR") # Get project directory from environment
generate_imprint_file(build_flag, project_dir) # Generate imprint.h file with the build information
13

It’s interesting, but for some reason the SRC_BUILD_FLAGS option causes a few more files in the ui folder to be built each time. But of course this is not even a tenth of the project, but the behavior is strange.

Compiling .pio/build/esp-cyd/src/main.cpp.o
Compiling .pio/build/esp-cyd/src/ui/images.c.o
Compiling .pio/build/esp-cyd/src/ui/screens.c.o
Compiling .pio/build/esp-cyd/src/ui/styles.c.o
Compiling .pio/build/esp-cyd/src/ui/ui.c.o
Compiling .pio/build/esp-cyd/src/ui/ui_font_roboto14.c.o
Compiling .pio/build/esp-cyd/src/ui/ui_font_roboto20.c.o
Compiling .pio/build/esp-cyd/src/ui/ui_font_roboto25.c.o
Compiling .pio/build/esp-cyd/src/ui/ui_font_roboto32.c.o
14

That’s expected behavior because build_src_flags are applied all project source files.

Afaik the only way to avoid this is your second approach by creating a temporary header file which has the version string and include this to your source file.

15

Ah, so it will still rebuild all the project files inside src/, but it won’t rebuild the libraries?

16

Yes, that’s what I read from the description.
(Click on build_src_flags in my last post. It’s a link to the docs)

1 Like