Generated files using extraScript - generate only on-demand (if source changed)

PlatformIO Home
1

Hello!

I’m processing protobuf files with nanopb generator.
All works fine, expect that files are generated every time I build project. I would like to speedup build process and run generator only if source files were changed.

So, from *.proto it genarates 2 files *.pb.c and *.pb.h

I understand, that I need to add some dependency, to source .proto files but I don’t know how to do that with platformio env.
Seems that I need to add custom target instead of calling env.Execute(cmd) directly.
Can somebody help me example with custom target which has 1 dependecy and 2 output files?

library.json file:

"extraScript": "scripts/nanopb_generate.py"

nanopb_generate.py: (working example)

    import os
    import glob

    Import("env")


    mylib_root = os.path.join(os.getcwd(),'..')
    generated_src_dir = os.path.join(mylib_root, 'src', 'generated')
    protoc_generator = os.path.join(mylib_root, '3rd_party', 'nanopb', 'generator', 'protoc')

    mylib_proto_dirs = [
        os.path.join(mylib_root, 'mylib', 'proto'),
    ]

    nanopb_options = [
        f"--nanopb_out={generated_src_dir}",
    ]

    proto_files = []
    for proto_dir in mylib_proto_dirs:
        protos = glob.glob(os.path.join(proto_dir, '*.proto'))
        proto_files += protos
        nanopb_options.append("--proto_path=" + proto_dir)
        nanopb_options.append("--nanopb_opt=-I" + proto_dir)
        # print(proto_dir)

    for proto_file in proto_files:
        proto_file_basename = os.path.basename(proto_file)
        proto_file_without_ext = os.path.splitext(proto_file_basename)[0]
        generated_targets = [
            os.path.join(generated_src_dir, proto_file_without_ext + ".pb.c"),
            os.path.join(generated_src_dir, proto_file_without_ext + ".pb.h")
        ]
        # print(f"{proto_file_basename} -> {generated_targets}")
        cmd = protoc_generator + " " + " ".join(nanopb_options) + " " + proto_file_basename
        result = env.Execute(cmd)
        if (result != 0):
            print(f"Error({result}) processing cmd: '{cmd}'")
            exit(1)
2

Hm you can manually create a hash/checksum check where you save the last e.g. SHA1 hash of the .proto file and the output files in a file, so if the .proto file changes or the pb.c and pb.h files were manually changed, you can detect it and regenerate it.

Maybe there’s also a way to make it work automatically with the SCons build system. There is the env.Depends() function which allows you to declare dependencies between files, but then I’m not sure on how to give SCons the ‘build’ function protoc ... that it should execute once it detects a change in the dependencies.

Maybe @ivankravets has some thoughts about this.

1 Like
3

@maxgerhardt, thanks for the reply.
Of cause, I know that I can handle SHA1/MD5/… manually, but it not looks nice :frowning:
I’m looking for example how to do that with PIO+Scons :wink:

4

Have you checked our docs Redirecting... ?

5

Of cause I checked docs, but seems that there is no my case in it.

What is your suggestion? using env.AddCustomTarget() for each file?
How to depend target output from source file in this case?