Patching external library from GitHub with custom library.json

Libraries
1

Hej!

I would like to supply my own library.json for a specific library to be “independent” from the project sources and choose only the sources I need in my project.

Do to that in platformio.ini I added the library git URL and the extra script name which patches the library:

extra_scripts = pre:extra_script.py
lib_deps = https://github.com/KKoovalsky/JunglesDataStructs.git

I created a directory lib_overlay which contains my own library.json for that library with the relative path lib_overlay/JunglesDataStructs/library.json.

Then in my extra_script.py I perform the actual patching like that:

from shutil import copyfile
from os import path

Import("env")

ROOT_DIR = env['PROJECT_DIR']
PIO_PLATFORM = env['PIOPLATFORM']
LIBDEPS_DIR = env['PROJECT_LIBDEPS_DIR']
JDS_LIB_JSON_SRC_PATH = path.join(
    ROOT_DIR, 'lib_overlay/JunglesDataStructs/library.json')
JDS_LIB_JSON_DST_PATH = path.join(
    LIBDEPS_DIR, PIO_PLATFORM, 'JunglesDataStructs', 'library.json')

copyfile(JDS_LIB_JSON_SRC_PATH, JDS_LIB_JSON_DST_PATH)

  1. The problem with this approach is when the project is compiled for the first time the extra script is run BEFORE the library is downloaded, so the patching will not work. This approach works only if I comment out the copyfile... line, when running platformio run for the first time, and then when running for the second time I uncomment the mentioned line. So the question is: is there a clean way of patching the library AFTER it is downloaded, and BEFORE it is compiled?

  2. In the code of the extra_script.py there is a lot of “manual” path handling and so on. Is there a method like:

projenv.get_library_dir("JunglesDataStructs")

which will return the actual path to the library downloaded locally in the project directory? Basically it means to replace all the code which merges the JDS_LIB_JSON_DST_PATH to a single call.

  1. Is there some kind of documentation which documents how to use the python modules/methods inside the extra_scripts?
2

I refresh the topic as there is no answer yet.

3

Why you can’t fork this library, do modifications and use it directly in lib_deps?

4

I would rather not do it, because then I would need to maintain my own fork of the repository, to mirror it manually, etc.
What I only would like to do is to provide the library.json to choose only part of the source files. To choose only those compatible ones or compile just a part of it. This means that I don’t want to perform any changes on the repository.
What I do right now is that I use this script to copy my own library.json to the .pio/build/lib_deps/... directory, after the library is downloaded to the build directory.

1 Like
5

There are a bunch of triggers for when a extra_script can run… have you checked to see if any fire only at the start of the build, but after the library dependencies have been met?

https://docs.platformio.org/en/latest/projectconf/advanced_scripting.html#before-pre-and-after-post-actions

I’m sort of hoping a env.AddPreAction("buildprog", callback...) type callback would trigger at the right time… but I have this feeling it triggers before the library dependencies are processed. :-/

6

Yeah, firstly I checked whether I’m able to run a post action, after the dependencies are downloaded (or a pre-action before the actual build), but there seems no such trigger.

That would be really handy to have such a trigger.

1 Like