Libraries showing twice in .pio/libdeps

GeorgeFlorian · August 9, 2021, 12:01pm

Why are the libraries downloaded twice ?

what1

These are the libraries inside platformio.ini :

#libraries
lib_deps =  https://github.com/me-no-dev/ESPAsyncWebServer.git
            https://github.com/me-no-dev/AsyncTCP.git
            https://github.com/marcmerlin/Framebuffer_GFX.git
            https://github.com/marcmerlin/SmartMatrix_GFX.git
            https://github.com/adafruit/Adafruit_BusIO.git
            https://github.com/adafruit/Adafruit-GFX-Library.git
            https://github.com/FastLED/FastLED.git
            Wire

If I delete one of the same library from the /libdeps directory, Platformio will just download them again.

I don’t know how correct is any of this, because, for example, in this project both Adafruit GFX libraries have a Font directory inside, where I add fonts that I want to use. I am adding the font to both libraries, even if they are identical. Isn’t this making the build file larger ?

GeorgeFlorian · August 9, 2021, 2:09pm

Any ideas as to why I’m getting this strange behavior ?
I am getting conflicting code errors because it’s the same code twice.
I can’t modify a library because I don’t know which one will get used.

GeorgeFlorian · August 9, 2021, 2:27pm

On the same note, why do I have two lists of libraries both pointing to the same directory:

/home/georgeflorian/.platformio/packages/framework-arduinoespressif32/libraries

Why is everything doubled ?

GeorgeFlorian · August 9, 2021, 2:37pm

I can’t even build projects anymore:

Indexing .pio/build/esp32doit-devkit-v1/libFrameworkArduino.a
Linking .pio/build/esp32doit-devkit-v1/firmware.elf
.pio/build/esp32doit-devkit-v1/lib8d2/FastLED@src-840563c23c8dae79803694244a6818ef/FastLED.cpp.o: In function `CFastLED::CFastLED()':
/home/georgeflorian/Documents/PlatformIO/Projects/ESP_TimerOnInput-main/.pio/libdeps/esp32doit-devkit-v1/FastLED@src-840563c23c8dae79803694244a6818ef/src/FastLED.cpp:24: multiple definition of `CFastLED::CFastLED()'
.pio/build/esp32doit-devkit-v1/libb5a/FastLED/FastLED.cpp.o:/home/georgeflorian/Documents/PlatformIO/Projects/ESP_TimerOnInput-main/.pio/libdeps/esp32doit-devkit-v1/FastLED/src/FastLED.cpp:24: first defined here
.pio/build/esp32doit-devkit-v1/lib8d2/FastLED@src-840563c23c8dae79803694244a6818ef/FastLED.cpp.o: In function `CFastLED::CFastLED()':
FastLED.cpp:(.text._ZN8CFastLEDC2Ev+0x0): multiple definition of `CFastLED::CFastLED()'
.pio/build/esp32doit-devkit-v1/lib8d2/FastLED@src-840563c23c8dae79803694244a6818ef/power_mgt.cpp.o: In function `show_at_max_brightness_for_power()':
power_mgt.cpp:(.text._Z32show_at_max_brightness_for_powerv+0x0): multiple definition of `show_at_max_brightness_for_power()'
.pio/build/esp32doit-devkit-v1/libb5a/FastLED/power_mgt.cpp.o:power_mgt.cpp:(.text._Z32show_at_max_brightness_for_powerv+0x0): first defined here
.pio/build/esp32doit-devkit-v1/lib8d2/FastLED@src-840563c23c8dae79803694244a6818ef/power_mgt.cpp.o: In function `delay_at_max_brightness_for_power(unsigned short)':
power_mgt.cpp:(.text._Z33delay_at_max_brightness_for_powert+0x0): multiple definition of `delay_at_max_brightness_for_power(unsigned short)'
.pio/build/esp32doit-devkit-v1/libb5a/FastLED/power_mgt.cpp.o:power_mgt.cpp:(.text._Z33delay_at_max_brightness_for_powert+0x0): first defined here
collect2: error: ld returned 1 exit status
*** [.pio/build/esp32doit-devkit-v1/firmware.elf] Error 1

GeorgeFlorian · August 10, 2021, 10:05am

I can’t compile projects anymore.
Help, please ?

maxgerhardt · August 11, 2021, 7:51am

Does removing the .pio folder of the project and rebuild the project help?

Otherwise I’d suggest a clean PlatformIO reinstall (remove the VSCode extension, remove C:\Users\<user>\.platformio, reinstall extension).

GeorgeFlorian · August 11, 2021, 9:01am

I’ve done this and it’s the same exact behavior.

I’ve done this as well and it’s the same exact behavior.

Do you think that it has something to do with the way I get my libraries:

#libraries
lib_deps =  https://github.com/me-no-dev/ESPAsyncWebServer.git
            https://github.com/me-no-dev/AsyncTCP.git
            https://github.com/marcmerlin/Framebuffer_GFX.git
            https://github.com/marcmerlin/SmartMatrix_GFX.git
            https://github.com/adafruit/Adafruit_BusIO.git
            https://github.com/adafruit/Adafruit-GFX-Library.git
            https://github.com/FastLED/FastLED.git
            Wire

I am simply specifying the git page. Should I specify the version or ?
Can I improve the way I import libraries in my projects ?

maxgerhardt · August 11, 2021, 9:43am

That looks correct, but each library could have additional dependencies which are pulled via the regular PlatformIO registry and not via git, and that might result in that behavior. I’ll try it out.

GeorgeFlorian · August 11, 2021, 5:03pm

Any news on that ?

Also, do additional dependencies translate to duplicate libraries ?

maxgerhardt · August 11, 2021, 5:48pm

Hm yes it is as I thought. When you reference a library via git that has sub-dependencies, like e.g. ESPAsyncWebServer has on AsyncTCP, you will get the git version of ESPAsyncWebServer but then the regular version of the dependencies

github.com

me-no-dev/ESPAsyncWebServer/blob/1d46269cedf477661ca8a29518414f4b74e957d4/library.json#L19-L32


      
          "dependencies": [
            {
              "name": "ESPAsyncTCP",
              "platforms": "espressif8266"
            },
            {
              "name": "AsyncTCP",
              "platforms": "espressif32"
            },
            {
              "name": "Hash",
              "platforms": "espressif8266"
            }
          ]

If we take the simpler example

[env:esp32dev]
platform = espressif32
board = esp32dev
framework = arduino
lib_deps =  https://github.com/me-no-dev/ESPAsyncWebServer.git
            https://github.com/me-no-dev/AsyncTCP.git

with src\main.cpp as

#include <Arduino.h>
#include <ESPAsyncWebServer.h>
#include <AsyncTCP.h>

AsyncWebServer server(80);
AsyncClient asyncClient;

void setup() {}
void loop() {}

Then the verbose dependency graph is

Dependency Graph
|-- <ESP Async WebServer> 1.2.3+sha.1d46269 [git+https://github.com/me-no-dev/ESPAsyncWebServer.git] (C:\Users\Max\temp\double_deps\.pio\libdeps\esp32dev\ESP Async WebServer)
|   |-- <AsyncTCP> 1.1.1 (C:\Users\Max\temp\double_deps\.pio\libdeps\esp32dev\AsyncTCP)
|   |-- <FS> 1.0 (C:\Users\Max\.platformio\packages\framework-arduinoespressif32\libraries\FS)
|   |-- <WiFi> 1.0 (C:\Users\Max\.platformio\packages\framework-arduinoespressif32\libraries\WiFi)
|-- <AsyncTCP> 1.1.1+sha.ca8ac5f [git+https://github.com/me-no-dev/AsyncTCP.git] (C:\Users\Max\temp\double_deps\.pio\libdeps\esp32dev\AsyncTCP@src-7fb2940bccb78b8d2de6915ae328b7fc)
|-- <AsyncTCP> 1.1.1 (C:\Users\Max\temp\double_deps\.pio\libdeps\esp32dev\AsyncTCP)

So the git version of ESP Async Websever is itself of the git version, bubt the dependency is of the regular version. Then both AsyncTCP of the git and regular version are in the dependency graph, too.

Also, both versions of Async TCP are linked

xtensa-esp32-elf-g++ -o .pio\build\esp32dev\firmware.elf […] .pio\build\esp32dev\lib0e7\libAsyncTCP.a “.pio\build\esp32dev\lib5a7\libESP Async WebServer.a” .pio\build\esp32dev\lib042\libAsyncTCP@src-7fb2940bccb78b8d2de6915ae328b7fc.a […]

Per this the linker will then use the first-available library that satisfies the linking conditions… meaning that one of them will be ignored and it probably depends on the order of the library in the linker command.

That is a problem when the library code is different and you need the sub-dependency to be sourced from the later version as well…

Three things I can recommend:

only source the high-level library from git, in the example above ESP Async WebServer. It will internally then use the stable AsyncTCP version it was tested against.
if you need to a subdependency to also be sourced from git, you should fork the high-level library and change the library.json manifest to reference the git version as well, see dependencies in the library.json
open an issue in Issues · platformio/platformio-core · GitHub so that some resolution for the above mentioned situation can be found in easier managable terms without forking. E.g., if two libraries are present with the same name, and one from git and one regular, introduce an option to prefer the git version, etc.