Is there a good way to "diff & patch" without overwriting the global core_dir version of a framework?

Hi all,

I’m working on a project that requires a small modification to the framework (in my case, framework-arduinoadafruitnrf52). I followed the instructions for overriding package files by using the provided python pre script to apply my patch, but I noticed that it applies the patch to the copy of the framework in core_dir (ie, ~/.platformio/packages/framework-arduinoadafruitnrf52), which means that the modification will be in effect for every pio project on my system that uses that framework, rather than just this one project.

I tried following the advice in other posts on the topic, but even when I specify the library in platform_packages, it seems to just use the version stored in core_dir rather than one local to the project directory. I know I could override the core_dir location for the whole project, and I will if that’s the only option, but

  1. That feels kind of like a “nuclear option”
  2. Ideally, I’d like the patched library to be part of my board definition, so I can reuse it in other projects, rather than something specific to this project.

Is there a way to either force platformio to use a project-local version of a single package (in this case a framework) OR to override/patch files from a package without modifying the core_dir version or maintaining a separate fork?

Thanks!

Then you’re not doing it correctly or your misinterpreting the results. Please share your platformio.ini and a pastebin.com of the project task → Advanced → Verbose Build.

platformio.ini:

[env:myboard]
platform = nordicnrf52
# adafruit feather just used as a base; all the relevant settings are overridden
# (and that's not relevant to the issue at hand here)
board = adafruit_feather_nrf52840
upload_speed=115200
board_build.mcu=nrf52833
framework=arduino
# I cleaned this up later when I refactored into a custom branch of the whole framework, but that doesn't
# matter to the issue here.
build_flags = -D EMBEDDED -std=c++11 -Wall -Iinclude/ -Iboards/nrf52833_files/include -Iboards/nrf52833_files/include/s140_nrf52_7.2.0_API/include/nrf52 -Iboards/nrf52833_files/include/s140_nrf52_7.2.0_API/include -Ilib_nrf/ -D INCLUDE_xTaskGetHandle=1 -g -D USE_TINYUSB  -D CFG_TUD_CDC -D NRF52833_XXAA
board_build.extra_flags=
board_build.arduino.ldscript=boards/nrf52833_files/nrf52833_s140_v7.ld
board_build.softdevice.sd_flags="-DS140"
board_build.softdevice.sd_name="s140"
board_build.softdevice.sd_version="7.2.0"
board_build.softdevice.sd_fwid="0x0100"
lib_ldf_mode=deep+

extra_scripts=pre:apply_patches.py
platform_packages=
  framework-arduinoadafruitnrf52 @ platformio/framework-arduinoadafruitnrf52

lib_deps=
  SPI

Logs (I’m using the command line, so this is the output of pio run -v): pastebin

Before I ran the pio command, I deleted ~/.platformio/packages/framework-arduinoadafruitnrf52, which forced it to reinstall. After that (as is obvious from the logs), the fresh copy in ~/.platformio/packages/ contains the result of the patching.

I’ve also tried lots of variations on the platform_packages lines (framework-arduinoadafruitnrf52=platformio/framework-arduinoadafruitnrf52, the same with @version or #branch on the end, etc). All of them either have the same result or throw errors. Have I just not tried the right incantation here?

Just so that I understand you right:

You have a base version of framework-arduinoadafruitnrf52. You want to apply some patches to it and make the project use that, without affecting any other project.

The way to do that is to first duplicate the unmodified version of that package (e.g., framework-arduinoadafruitnrf52-1.10600.0.tar.gz), apply the your patches, change the version in the package.json to something unique (e.g., 2.10600.0) then either reupload it somewhere on the internet (Github etc.) and reference the download link, or reference a local copy on disk, using plaform_packages.

platform_packages=
  framework-arduinoadafruitnrf52 @ symlink://C:/Users/user/patched_framework

or

platform_packages=
  framework-arduinoadafruitnrf52 @ https://github.com/someuser/somerepo/archive/refs/heads/main.zip

You do not need a apply_patches.py then anymore, since you are already sourcing the package from the fixed source.

The change of the version in the package.json because otherwise other projects might see your now custom installed version of the framework as also fitting their version requirements. For example,

will match any 1.10600.X version (semver), so if you dont match that format, PlatformIO should nicely use the patched framework version only for your project and not any other.

Of course the apply_patches.py can still work without disturbing other projects too: Considering only one project is compiled at a time (no parallelism), the script can undo the patches it has done to the framework at the end of each firmware build step, reverting it to its original state.

Yep, that’s right.

This is what I ended up doing, though more as a consequence of wanting to open a pull request with Adafruit than thinking it was the best way to accomplish the task I wanted for my project.

It would be great if someone could add this note to the Override package files documentation; when I read it, I assumed that the override only took place for the build that it was specified in and that platformio would take care of maintaining the integrity of the packages in its package store. Having an “unpatch” example post-build script, or at least a note that this affects the global version of the package, would help clarify that a lot, I think.

As an aside - where does the package.json come from in the framework-arduinoadafruitnrf52 package? Adafruit is unwilling to include one in their BSP, but I haven’t been able to find any source for the one that platformio adds. The framework that gets downloaded is identical to (an old version of) the Adafruit repository except for the addition of the package.json file, but I haven’t been able to find any forked repository or other code that adds it. It just magically seems to exist in the platformio registry’s version of the package and nowhere else.