I’ve managed to build the Arduino ESP32 framework libraries using the esp32-arduino-lib-builder repository and update some of the configuration options that I needed. I then added my updated library binaries to the existing framework-arduinoespressif32 package (in a new folder). I created a new repository on my Github account for this updated framework-arduinoespressif32 framework and updated my project platformio.ini file to refer to it. So far, so good, the project builds and works properly with the updated ESP32 Arduino framework. However, when I switch to another project that uses the ESP32 Arduino framework (but not my updated framework) it winds up renaming my framework package folder to something like framework-arduinoespressif32@src-7c60f8d39dccf4e1bff992b3033aecb9 with the long commit ID. Then when switching back to the original project that wants to use the modified framework the project will no longer build under Windows because the command line exceeds 32k due to the long folder name now used for the updated framework. I can see that there are other frameworks that seem to differ only by version number and they seem to get named framework-name@version. I haven’t been able to figure out how to get my custom framework to get named with just the version number (and not commit ID) so that the command line doesn’t exceed 32k in length when building. Anyone have suggestions?
Hello, Im trying to get a modified version of arduino libraries for esp32 with some changes in sdkconfig, I achieve to build with esp32-arduino-lib-builder and get something under /out folder, but I have trouble on how to update the platform io framework-arduinoespressif32 or include my new files in platformio.ini.
Can you please explain how to do it exactly, If I override all the folders I get some build errors when try to build my project
Hi Javier, I’ve managed to build a custom version of the Arduino ESP32 libraries with modified SDK config settings. I built the library under WSL2 on a Windows system but you should be able to do that on pretty much any Linux system I believe. I was then able to use these libraries in one of my PlatformIO projects and everything seemed to work. The only problem I’m having is that PlatformIO will “rename” my custom ESP32 library to a very long name (using a full commit ID from the Github account that I use to host the library) when switching back to a ESP32 project that doesn’t use this custom library. After that the project using the custom library fails to build because of a path length issue in one of the ESP32 tools. I haven’t been able to get past this issue but I have been able to continue making progress on the project using the custom library. Let me know if you are still interested in the steps I use to get where I am today.
Hi Spurlock, sorry for the delay. Yes I’m still interested in find a way to get an Arduino ESP32 libraries with modifications in SDK config. Can you tell me the steps from the compilation with esp32-arduino-lib-builder to extract the compiled files and include it in PlatformIO project? Is Github needed for that process? If would be nice to have a step by step procedure, im not so familiar with this. Thanks a lot.
If you use pioarduino/platform-espressif32 as a platform, you have an even easier option: “Hybrid-Compile”
This allows you to set SDK configurations directly in the platformio.ini. pioarduino takes care of the rest, you don’t need to do anything else.
Example:
custom_sdkconfig =
CONFIG_ESP_WIFI_GMAC_SUPPORT=y
CONFIG_ESP_WIFI_CSI_ENABLED=y
CONFIG_ESP_WIFI_ENABLE_WPA3_SAE=y
CONFIG_ESP_WIFI_ENABLE_SAE_PK=y
CONFIG_ESP_WIFI_ENTERPRISE_SUPPORT=y
CONFIG_ESP_WIFI_SOFTAP_SAE_SUPPORT=y
CONFIG_ESP_WIFI_ENABLE_WPA3_OWE_STA=y
I am interested in an answer to the original question:
I have built an Arduino framework using some custom settings. I would like to use that library in my platformIO (not pioarduino) project. So far I have made the changes to the configs on my linux vm. Built using esp32-arduino-lib-builder and have an “out” directory loaded with all sorts of interesting stuff. I have FTP’d the entire out directory to a “Framework” directory in my (Windows based) project. What do I have to do to my platformio.ini file to make the build use my new framework?
I have tried adding platform_packages = arduinoespressif32-libs @ file://./Framework/tools/esp32-arduino-libs to my platformio.ini file, but this results in my previously working build no longer building due to the inability to find header files (that it used to find without issue) that are part of my project. The header files it cannot find are part of my project (not part of the framework).
FYI: Adding that single line is the ONLY change made to the project. The project does not yet even use any of the new functionality in the framework.
So I deleted ./pio, rebuilt the arduino lib framework, copied the contents of ‘out’ to framework in my project, started the visual studio app and get the error below. To me, it looks like copy_tree is not working. Not sure how that happens since the directories were deleted prior to starting the app.
Tool Manager: Installing file://E:/users/mmueller/Documents/Arduino/ESPixelStick/Framework/tools/esp32-arduino-libs
Error: Traceback (most recent call last):
File "C:\Users\mmueller\.platformio\penv\Lib\site-packages\platformio\__main__.py", line 103, in main
cli() # pylint: disable=no-value-for-parameter
^^^^^
File "C:\Users\mmueller\.platformio\penv\Lib\site-packages\click\core.py", line 1157, in __call__
return self.main(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\mmueller\.platformio\penv\Lib\site-packages\click\core.py", line 1078, in main
rv = self.invoke(ctx)
^^^^^^^^^^^^^^^^
File "C:\Users\mmueller\.platformio\penv\Lib\site-packages\platformio\cli.py", line 85, in invoke
return super().invoke(ctx)
^^^^^^^^^^^^^^^^^^^
File "C:\Users\mmueller\.platformio\penv\Lib\site-packages\click\core.py", line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\mmueller\.platformio\penv\Lib\site-packages\click\core.py", line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\mmueller\.platformio\penv\Lib\site-packages\click\core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\mmueller\.platformio\penv\Lib\site-packages\click\core.py", line 783, in invoke
return __callback(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\mmueller\.platformio\penv\Lib\site-packages\platformio\project\commands\init.py", line 106, in project_init_cmd
install_project_dependencies(
File "C:\Users\mmueller\.platformio\penv\Lib\site-packages\platformio\package\commands\install.py", line 107, in install_project_dependencies
already_up_to_date = not install_project_env_dependencies(env, options)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\mmueller\.platformio\penv\Lib\site-packages\platformio\package\commands\install.py", line 132, in install_project_env_dependencies
_install_project_env_platform(project_env, options),
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\mmueller\.platformio\penv\Lib\site-packages\platformio\package\commands\install.py", line 149, in _install_project_env_platform
PlatformPackageManager().install(
File "C:\Users\mmueller\.platformio\penv\Lib\site-packages\platformio\package\manager\platform.py", line 62, in install
p.install_required_packages(force=force)
File "C:\Users\mmueller\.platformio\penv\Lib\site-packages\platformio\platform\_packages.py", line 76, in install_required_packages
self.install_package(name, force=force)
File "C:\Users\mmueller\.platformio\penv\Lib\site-packages\platformio\platform\_packages.py", line 70, in install_package
return self.pm.install(spec or self.get_package_spec(name), force=force)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\mmueller\.platformio\penv\Lib\site-packages\platformio\package\manager\_install.py", line 47, in install
pkg = self._install(spec, skip_dependencies=skip_dependencies, force=force)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\mmueller\.platformio\penv\Lib\site-packages\platformio\package\manager\_install.py", line 97, in _install
pkg = self.install_from_uri(spec.uri, spec)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\mmueller\.platformio\penv\Lib\site-packages\platformio\package\manager\_install.py", line 204, in install_from_uri
return self._install_tmp_pkg(pkg_item)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\mmueller\.platformio\penv\Lib\site-packages\platformio\package\manager\_install.py", line 296, in _install_tmp_pkg
shutil.copytree(tmp_pkg.path, dst_pkg.path, symlinks=True)
File "C:\Users\mmueller\.platformio\python3\Lib\shutil.py", line 561, in copytree
return _copytree(entries=entries, src=src, dst=dst, symlinks=symlinks,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\mmueller\.platformio\python3\Lib\shutil.py", line 515, in _copytree
raise Error(errors)
shutil.Error: [('e:\\users\\mmueller\\Documents\\Arduino\\ESPixelStick\\.pio\\cache\\tmp\\pkg-installing-pv0mc1gm\\esp32\\include\\espressif__esp_matter\\connectedhomeip\\connectedhomeip\\src\\app\\clusters\\boolean-state-configuration-server\\BooleanStateConfigurationTestEventTriggerHandler.h', 'e:\\users\\mmueller\\Documents\\Arduino\\ESPixelStick\\.pio\\packages\\framework-arduinoespressif32-libs\\esp32\\include\\espressif__esp_matter\\connectedhomeip\\connectedhomeip\\src\\app\\clusters\\boolean-state-configuration-server\\BooleanStateConfigurationTestEventTriggerHandler.h', "[Errno 2] No such file or directory: 'e:\\\\users\\\\mmueller\\\\Documents\\\\Arduino\\\\ESPixelStick\\\\.pio\\\\packages\\\\framework-arduinoespressif32-libs\\\\esp32\\\\include\\\\espressif__esp_matter\\\\connectedhomeip\\\\connectedhomeip\\\\src\\\\app\\\\clusters\\\\boolean-state-configuration-server\\\\BooleanStateConfigurationTestEventTriggerHandler.h'"), ('e:\\users\\mmueller\\Documents\\Arduino\\ESPixelStick\\.pio\\cache\\tmp\\pkg-installing-pv0mc1gm\\esp32\\include\\espressif__esp_matter\\connectedhomeip\\connectedhomeip\\src\\app\\clusters\\device-energy-management-server\\DeviceEnergyManagementTestEventTriggerHandler.h', 'e:\\users\\mmueller\\Documents\\Arduino\\ESPixelStick\\.pio\\packages\\framework-arduinoespressif32-libs\\esp32\\include\\espressif__esp_matter\\connectedhomeip\\connectedhomeip\\src\\app\\clusters\\device-energy-management-server\\DeviceEnergyManagementTestEventTriggerHandler.h', "[Errno 2] No such file or directory: 'e:\\\\users\\\\mmueller\\\\Documents\\\\Arduino\\\\ESPixelStick\\\\.pio\\\\packages\\\\framework-arduinoespressif32-libs\\\\esp32\\\\include\\\\espressif__esp_matter\\\\connectedhomeip\\\\connectedhomeip\\\\src\\\\app\\\\clusters\\\\device-energy-management-server\\\\DeviceEnergyManagementTestEventTriggerHandler.h'"), ('e:\\users\\mmueller\\Documents\\Arduino\\ESPixelStick\\.pio\\cache\\tmp\\pkg-installing-pv0mc1gm\\esp32\\include\\espressif__esp_matter\\connectedhomeip\\connectedhomeip\\src\\app\\clusters\\thread-border-router-management-server\\thread-border-router-management-server.h', 'e:\\users\\mmueller\\Documents\\Arduino\\ESPixelStick\\.pio\\packages\\framework-arduinoespressif32-libs\\esp32\\include\\espressif__esp_matter\\connectedhomeip\\connectedhomeip\\src\\app\\clusters\\thread-border-router-management-server\\thread-border-router-management-server.h', "[Errno 2] No such file or directory: 'e:\\\\users\\\\mmueller\\\\Documents\\\\Arduino\\\\ESPixelStick\\\\.pio\\\\packages\\\\framework-arduinoespressif32-libs\\\\esp32\\\\include\\\\espressif__esp_matter\\\\connectedhomeip\\\\connectedhomeip\\\\src\\\\app\\\\clusters\\\\thread-border-router-management-server\\\\thread-border-router-management-server.h'"), ('e:\\users\\mmueller\\Documents\\Arduino\\ESPixelStick\\.pio\\cache\\tmp\\pkg-installing-pv0mc1gm\\esp32\\include\\espressif__esp_matter\\connectedhomeip\\connectedhomeip\\src\\app\\clusters\\valve-configuration-and-control-server\\valve-configuration-and-control-delegate.h', 'e:\\users\\mmueller\\Documents\\Arduino\\ESPixelStick\\.pio\\packages\\framework-arduinoespressif32-libs\\esp32\\include\\espressif__esp_matter\\connectedhomeip\\connectedhomeip\\src\\app\\clusters\\valve-configuration-and-control-server\\valve-configuration-and-control-delegate.h', "[Errno 2] No such file or directory: 'e:\\\\users\\\\mmueller\\\\Documents\\\\Arduino\\\\ESPixelStick\\\\.pio\\\\packages\\\\framework-arduinoespressif32-libs\\\\esp32\\\\include\\\\espressif__esp_matter\\\\connectedhomeip\\\\connectedhomeip\\\\src\\\\app\\\\clusters\\\\valve-configuration-and-control-server\\\\valve-configuration-and-control-delegate.h'"), ('e:\\users\\mmueller\\Documents\\Arduino\\ESPixelStick\\.pio\\cache\\tmp\\pkg-installing-pv0mc1gm\\esp32\\include\\espressif__esp_matter\\connectedhomeip\\connectedhomeip\\src\\app\\clusters\\valve-configuration-and-control-server\\valve-configuration-and-control-server.h', 'e:\\users\\mmueller\\Documents\\Arduino\\ESPixelStick\\.pio\\packages\\framework-arduinoespressif32-libs\\esp32\\include\\espressif__esp_matter\\connectedhomeip\\connectedhomeip\\src\\app\\clusters\\valve-configuration-and-control-server\\valve-configuration-and-control-server.h', "[Errno 2] No such file or directory: 'e:\\\\users\\\\mmueller\\\\Documents\\\\Arduino\\\\ESPixelStick\\\\.pio\\\\packages\\\\framework-arduinoespressif32-libs\\\\esp32\\\\include\\\\espressif__esp_matter\\\\connectedhomeip\\\\connectedhomeip\\\\src\\\\app\\\\clusters\\\\valve-configuration-and-control-server\\\\valve-configuration-and-control-server.h'")]
============================================================
An unexpected error occurred. Further steps:
* Verify that you have the latest version of PlatformIO using
`python -m pip install -U platformio` command
* Try to find answer in FAQ Troubleshooting section
https://docs.platformio.org/page/faq/index.html
* Report this problem to the developers
https://github.com/platformio/platformio-core/issues
============================================================The file paths are getting so long that they likely break the Windows PATH_MAX restriction. Activate Win32 long paths.
Thank You for this tip. This resolved the copy issue. I no longer need to manually copy the files around. I did run into the same issue that I was working on after my manual copy: There are header files in the new platform build that are not present in the stock file set and I get compile errors when trying to access them.
Looking closer, I can see that the build is using headers from the “platform” definition when it should be using the file set found in the platform_packages definition, Example: The build is using:
.pio\packages\framework-arduinoespressif32\tools\sdk\esp32\include\lwip\lwip\src\include\lwip\prot\ethernet.h
when it should be using:
.pio\packages\framework-arduinoespressif32-libs\esp32\include\lwip\lwip\src\include\lwip\prot\ethernet.h