PlatformIO Community

Pio compiles dependencies for all platforms when only a specific platform specified per dependency

Hello

I have developed this library: https://github.com/iotexproject/arduino-sdk

The library supports multiple platforms (esp8266, esp32, atmelsam). For atmelsam ONLY, the library needs the dependencies FlashStorage and WiFiNINA. These are specified in library.json as follows:

"dependencies":
    [
      {
        "name": "FlashStorage",
        "platforms": ["atmelsam"]
      },
      {
        "name": "WiFiNINA",
        "platforms": ["atmelsam"]
      }
    ]

However, when building any of the examples for esp8266 or esp32 platforms, platformio tries to build those dependencies too. This causes a build error because they are not compatible

In order to reproduce:

  1. Clone the repo
  2. Try to build any of the examples in the examples folder for esp8266 or esp32 environments

Note: as the library is not published yet to pio manager, you’ll need to add a lib_extra_dirs entry pointing to the directory where you have cloned the repository

Expected behavior: the FlashStorage and WiFiNINA libraries are not compiled and the example builds
Actual behavior: the FlashStorage and WiFiNINA libraries are compiled causing a build error

Am I missing something?

Thanks for the help

See Library options — PlatformIO latest documentation

By default, this value is set to lib_compat_mode = soft and means that LDF will check only for framework compatibility.

The library.json can also declare the strict mode instead of the user having to do it in the project’s platformio.ini, per documentation.

{
   "build": {
      "libCompatMode": "strict"
   }
}

Thank you for your quick replies
Unfortunately they don´t make any change

I tried adding lib_compat_mode = strict to platformio.ini and also "libCompatMode": "strict" as maxgerhardt suggested. The result is the same. Pio still tries to build the dependencies for esp32 and esp8266 even if the platform specified on them is atmelsam

I also tried changing the value of libLDFMode but nothing seems to make a change

Odd. Unfortunately, I can’t replicate your build:

CONFIGURATION: https://docs.platformio.org/page/boards/espressif8266/esp_wroom_02.html
PLATFORM: Espressif 8266 (3.2.0) > ESP-WROOM-02
HARDWARE: ESP8266 80MHz, 80KB RAM, 2MB Flash
PACKAGES:
 - framework-arduinoespressif8266 3.30002.0 (3.0.2)
 - tool-esptool 1.413.0 (4.13)
 - tool-esptoolpy 1.30000.201119 (3.0.0)
 - toolchain-xtensa 2.100300.210717 (10.3.0)
Converting SignMessage.ino
LDF: Library Dependency Finder -> https://bit.ly/configure-pio-ldf
LDF Modes: Finder ~ deep+, Compatibility ~ strict
Library Manager: Installing IoTeX-blockchain-client
Warning! Could not find the package with 'IoTeX-blockchain-client' requirements for your system 'darwin_arm64'
Found 35 compatible libraries
Scanning dependencies...
No dependencies
Building in release mode
Compiling .pio/build/esp8266/src/SignMessage.ino.cpp.o
Compiling .pio/build/esp8266/FrameworkArduino/Print.cpp.o
Compiling .pio/build/esp8266/FrameworkArduino/Schedule.cpp.o
Compiling .pio/build/esp8266/FrameworkArduino/StackThunk.cpp.o
Compiling .pio/build/esp8266/FrameworkArduino/Stream.cpp.o
Compiling .pio/build/esp8266/FrameworkArduino/StreamSend.cpp.o
Compiling .pio/build/esp8266/FrameworkArduino/Tone.cpp.o
Compiling .pio/build/esp8266/FrameworkArduino/TypeConversion.cpp.o
/Users/jcw/Desktop/arduino-sdk/examples/SignMessage/SignMessage.ino:3:10: fatal error: IoTeX-blockchain-client.h: No such file or directory

Note the Compatibility ~ strict which I added to SignMessage/platformio.ini.
It looks like the above issue is not related to this.

The library is not published to platformio library manager yet
In order for platformio to recognize the library you need to clone it manually from github and then add a lib_extra_dirs entry pointing to the directory where you have cloned the repo.

Alternatively you could set the PLATFORMIO_LIB_EXTRA_DIRS environment variable before calling platformio run

For example, if you are using bash:

git clone https://github.com/iotexproject/arduino-sdk.git
# Set the value of PLATFORMIO_LIB_EXTRA_DIRS to the current directory
export PLATFORMIO_LIB_EXTRA_DIRS=`pwd`
cd arduino-sdk/examples/GetAccountMetadata
platformio run --environment nano_33_iot
platformio run --environment esp32

nano_33_iot builds fine
esp32 build fails because it tries to build WifiNINA. See the output below (with strict mode):

export PLATFORMIO_LIB_EXTRA_DIRS="~/Documents/Arduino/libraries" && pio run --environment esp32
Processing esp32 (platform: espressif32; board: esp32dev; framework: arduino)
-------------------------------------------------------------------------------------------------------------------------------------------------
Verbose mode can be enabled via `-v, --verbose` option
CONFIGURATION: https://docs.platformio.org/page/boards/espressif32/esp32dev.html
PLATFORM: Espressif 32 (1.12.1) > Espressif ESP32 Dev Module
HARDWARE: ESP32 240MHz, 320KB RAM, 4MB Flash
DEBUG: Current (esp-prog) External (esp-prog, iot-bus-jtag, jlink, minimodule, olimex-arm-usb-ocd, olimex-arm-usb-ocd-h, olimex-arm-usb-tiny-h, olimex-jtag-tiny, tumpa)
PACKAGES: 
 - framework-arduinoespressif32 3.10004.200129 (1.0.4) 
 - tool-esptoolpy 1.20600.0 (2.6.0) 
 - toolchain-xtensa32 2.50200.80 (5.2.0)
Converting GetAccountMetadata.ino
LDF: Library Dependency Finder -> http://bit.ly/configure-pio-ldf
LDF Modes: Finder ~ deep+, Compatibility ~ strict
Found 28 compatible libraries
Scanning dependencies...
Dependency Graph
|-- <IoTeX-blockchain-client> 0.0.1
|   |-- <WiFiNINA> 1.8.13
|   |   |-- <SPI> 1.0
|   |-- <HTTPClient> 1.2
|   |   |-- <WiFiNINA> 1.8.13
|   |   |   |-- <SPI> 1.0
|   |   |-- <WiFiClientSecure> 1.0
|   |   |   |-- <WiFiNINA> 1.8.13
|   |   |   |   |-- <SPI> 1.0
|   |-- <WiFiClientSecure> 1.0
|   |   |-- <WiFiNINA> 1.8.13
|   |   |   |-- <SPI> 1.0
|   |-- <EEPROM> 1.0.3
|-- <WiFiNINA> 1.8.13
|   |-- <SPI> 1.0
Building in release mode
Compiling .pio/build/esp32/src/GetAccountMetadata.ino.cpp.o
Compiling .pio/build/esp32/lib9da/WiFiNINA/WiFi.cpp.o
Compiling .pio/build/esp32/lib9da/WiFiNINA/WiFiClient.cpp.o
Compiling .pio/build/esp32/lib9da/WiFiNINA/WiFiServer.cpp.o
Compiling .pio/build/esp32/lib9da/WiFiNINA/WiFiStorage.cpp.o
Compiling .pio/build/esp32/lib9da/WiFiNINA/WiFiUdp.cpp.o
Compiling .pio/build/esp32/lib9da/WiFiNINA/utility/spi_drv.cpp.o
Compiling .pio/build/esp32/lib9da/WiFiNINA/utility/wifi_drv.cpp.o
In file included from /Users/__/Documents/Arduino/libraries/WiFiNINA/src/WiFiStorage.h:23:0,
                 from /Users/__/Documents/Arduino/libraries/WiFiNINA/src/WiFi.h:38,
                 from /Users/__/Documents/Arduino/libraries/iotex-client/examples/GetAccountMetadata/GetAccountMetadata.ino:4:
/Users/__/Documents/Arduino/libraries/WiFiNINA/src/utility/wifi_drv.h:293:12: error: 'PinStatus' does not name a type
     static PinStatus digitalRead(uint8_t pin);
            ^
In file included from /Users/__/Documents/Arduino/libraries/WiFiNINA/src/WiFi.cpp:21:0:
/Users/__/Documents/Arduino/libraries/WiFiNINA/src/utility/wifi_drv.h:293:12: error: 'PinStatus' does not name a type
     static PinStatus digitalRead(uint8_t pin);
            ^
In file included from /Users/__/Documents/Arduino/libraries/WiFiNINA/src/WiFiClient.cpp:30:0:
/Users/__/Documents/Arduino/libraries/WiFiNINA/src/utility/wifi_drv.h:293:12: error: 'PinStatus' does not name a type
     static PinStatus digitalRead(uint8_t pin);
            ^
*** [.pio/build/esp32/lib9da/WiFiNINA/WiFi.cpp.o] Error 1
*** [.pio/build/esp32/lib9da/WiFiNINA/WiFiClient.cpp.o] Error 1
In file included from /Users/__/Documents/Arduino/libraries/iotex-client/src/protobuf/pb_api.h:15:0,
                 from /Users/__/Documents/Arduino/libraries/iotex-client/src/api/rpc.h:9,
                 from /Users/__/Documents/Arduino/libraries/iotex-client/src/api/wallet/wallets.h:5,
                 from /Users/__/Documents/Arduino/libraries/iotex-client/src/api/api.h:7,
                 from /Users/__/Documents/Arduino/libraries/iotex-client/src/connection/connection.h:5,
                 from /Users/__/Documents/Arduino/libraries/iotex-client/src/account/account.h:4,
                 from /Users/__/Documents/Arduino/libraries/iotex-client/src/IoTeX-blockchain-client.h:4,
                 from /Users/__/Documents/Arduino/libraries/iotex-client/examples/GetAccountMetadata/GetAccountMetadata.ino:17:
/Users/__/Documents/Arduino/libraries/iotex-client/src/helpers/json_helper.h: In member function 'virtual void iotex::json::Uint64JsonProp::appendValueToString(String&)':
/Users/__/Documents/Arduino/libraries/iotex-client/src/helpers/json_helper.h:228:36: warning: format '%lu' expects argument of type 'long unsigned int', but argument 3 has type 'uint64_t {aka long long unsigned int}' [-Wformat=]
   sprintf(buf, "%lu", _value.uint64);
                                    ^
/Users/__/Documents/Arduino/libraries/iotex-client/src/helpers/json_helper.h:228:36: warning: format '%lu' expects argument of type 'long unsigned int', but argument 3 has type 'uint64_t {aka long long unsigned int}' [-Wformat=]
In file included from /Users/__/Documents/Arduino/libraries/WiFiNINA/src/WiFiStorage.h:23:0,
                 from /Users/__/Documents/Arduino/libraries/WiFiNINA/src/WiFi.h:38,
                 from /Users/__/Documents/Arduino/libraries/WiFiNINA/src/WiFiServer.cpp:28:
/Users/__/Documents/Arduino/libraries/WiFiNINA/src/utility/wifi_drv.h:293:12: error: 'PinStatus' does not name a type
     static PinStatus digitalRead(uint8_t pin);
            ^
/Users/__/Documents/Arduino/libraries/iotex-client/examples/GetAccountMetadata/GetAccountMetadata.ino: In function 'void initWiFi()':
/Users/__/Documents/Arduino/libraries/iotex-client/examples/GetAccountMetadata/GetAccountMetadata.ino:31:10: error: 'class WiFiClass' has no member named 'mode'
     WiFi.mode(WIFI_STA);
          ^
/Users/__/Documents/Arduino/libraries/iotex-client/examples/GetAccountMetadata/GetAccountMetadata.ino:31:15: error: 'WIFI_STA' was not declared in this scope
     WiFi.mode(WIFI_STA);
               ^
*** [.pio/build/esp32/lib9da/WiFiNINA/WiFiServer.cpp.o] Error 1
In file included from /Users/__/Documents/Arduino/libraries/iotex-client/src/contract/contract.h:4:0,
                 from /Users/__/Documents/Arduino/libraries/iotex-client/src/IoTeX-blockchain-client.h:7,
                 from /Users/__/Documents/Arduino/libraries/iotex-client/examples/GetAccountMetadata/GetAccountMetadata.ino:17:
/Users/__/Documents/Arduino/libraries/iotex-client/src/abi/abi.h: At global scope:
/Users/__/Documents/Arduino/libraries/iotex-client/src/abi/abi.h:41:20: warning: 'iotex::abi::EthereumTypeNameToStringLT' defined but not used [-Wunused-variable]
 static const char* EthereumTypeNameToStringLT[] = {"uint",  "int",   "address", "bool",  "fixed",
                    ^
*** [.pio/build/esp32/src/GetAccountMetadata.ino.cpp.o] Error 1
In file included from /Users/__/Documents/Arduino/libraries/WiFiNINA/src/WiFiStorage.h:23:0,
                 from /Users/__/Documents/Arduino/libraries/WiFiNINA/src/WiFiStorage.cpp:20:
/Users/__/Documents/Arduino/libraries/WiFiNINA/src/utility/wifi_drv.h:293:12: error: 'PinStatus' does not name a type
     static PinStatus digitalRead(uint8_t pin);
            ^
In file included from /Users/__/Documents/Arduino/libraries/WiFiNINA/src/WiFiUdp.cpp:29:0:
/Users/__/Documents/Arduino/libraries/WiFiNINA/src/utility/wifi_drv.h:293:12: error: 'PinStatus' does not name a type
     static PinStatus digitalRead(uint8_t pin);
            ^
*** [.pio/build/esp32/lib9da/WiFiNINA/WiFiStorage.cpp.o] Error 1
*** [.pio/build/esp32/lib9da/WiFiNINA/WiFiUdp.cpp.o] Error 1
In file included from /Users/__/Documents/Arduino/libraries/WiFiNINA/src/WiFiStorage.h:23:0,
                 from /Users/__/Documents/Arduino/libraries/WiFiNINA/src/WiFi.h:38,
                 from /Users/__/Documents/Arduino/libraries/WiFiNINA/src/utility/spi_drv.cpp:25:
/Users/__/Documents/Arduino/libraries/WiFiNINA/src/utility/wifi_drv.h:293:12: error: 'PinStatus' does not name a type
     static PinStatus digitalRead(uint8_t pin);
            ^
/Users/__/Documents/Arduino/libraries/WiFiNINA/src/utility/spi_drv.cpp: In static member function 'static void SpiDrv::begin()':
/Users/__/Documents/Arduino/libraries/WiFiNINA/src/utility/spi_drv.cpp:94:24: error: 'PINS_COUNT' was not declared in this scope
       if (SLAVERESET > PINS_COUNT) {
                        ^
/Users/__/Documents/Arduino/libraries/WiFiNINA/src/utility/spi_drv.cpp:103:15: error: 'NINA_GPIO0' was not declared in this scope
       pinMode(NINA_GPIO0, OUTPUT);
               ^
/Users/__/Documents/Arduino/libraries/WiFiNINA/src/utility/spi_drv.cpp: In static member function 'static int SpiDrv::available()':
/Users/__/Documents/Arduino/libraries/WiFiNINA/src/utility/spi_drv.cpp:66:25: error: 'NINA_GPIO0' was not declared in this scope
 #define NINA_GPIOIRQ    NINA_GPIO0
                         ^
/Users/__/Documents/Arduino/libraries/WiFiNINA/src/utility/spi_drv.cpp:597:25: note: in expansion of macro 'NINA_GPIOIRQ'
     return (digitalRead(NINA_GPIOIRQ) != LOW);
                         ^
In file included from /Users/__/Documents/Arduino/libraries/WiFiNINA/src/utility/wifi_drv.cpp:27:0:
/Users/__/Documents/Arduino/libraries/WiFiNINA/src/utility/wifi_drv.h:293:12: error: 'PinStatus' does not name a type
     static PinStatus digitalRead(uint8_t pin);
            ^
*** [.pio/build/esp32/lib9da/WiFiNINA/utility/spi_drv.cpp.o] Error 1
/Users/__/Documents/Arduino/libraries/WiFiNINA/src/utility/wifi_drv.cpp:1063:1: error: 'PinStatus' does not name a type
 PinStatus WiFiDrv::digitalRead(uint8_t pin)
 ^
*** [.pio/build/esp32/lib9da/WiFiNINA/utility/wifi_drv.cpp.o] Error 1

Please me know if you are able to reproduce the error following my instructions

For pio run -e nano_33_iot, I get errors:

LDF Modes: Finder ~ deep+, Compatibility ~ strict
Found 12 compatible libraries
Scanning dependencies...
No dependencies
Building in release mode
Compiling .pio/build/nano_33_iot/src/GetAccountMetadata.ino.cpp.o
Compiling .pio/build/nano_33_iot/FrameworkArduinoVariant/variant.cpp.o
Compiling .pio/build/nano_33_iot/FrameworkArduino/Reset.cpp.o
Compiling .pio/build/nano_33_iot/FrameworkArduino/SERCOM.cpp.o
Compiling .pio/build/nano_33_iot/FrameworkArduino/Tone.cpp.o
Compiling .pio/build/nano_33_iot/FrameworkArduino/USB/CDC.cpp.o
Compiling .pio/build/nano_33_iot/FrameworkArduino/USB/USBCore.cpp.o
Compiling .pio/build/nano_33_iot/FrameworkArduino/USB/samd21_host.c.o
/Users/jcw/Desktop/arduino-sdk/examples/GetAccountMetadata/GetAccountMetadata.ino:12:14: fatal error: WiFiNINA.h: No such file or directory

Hrmph, I don’t even get to pio run -e esp32. Note the “No dependencies” message.
I’m sorry, there are too many moving parts for me to pursue this issue any further.
I suggest stripping your project way down to get to the core of the problem.

Thank you again

The behavior in your machine was completely different to mine. So I did more investigation. I found something very weird and confusing with PlatformIO, which is what is causing platformio to always build the FlashStorage and WiFiNINA libraries. I suspect this is a bug in platformio and shoulg go into a Github issue.

Here’s my setup:

In order to make development easier, I have the repo in my Arduino IDE libraries directory.
When I build with Arduino IDE, the WiFININA and FlashStorage libraries are downloaded in the same directory as my library. So, here’s the file structure:

/Users/__/Documents/Arduino/libraries/
├── FlashStorage
├── WiFiNINA
├── arduino-sdk
└── readme.txt

Now when I build any example from arduino-sdk/examples with platformio, it detects FlashStorage and WiFiNINA as dependencies. This happens even if I remove all the "dependencies" from library.json. Here’s the output:

|-- <IoTeX-blockchain-client> 0.0.1
|   |-- <HTTPClient> 1.2
|   |   |-- <WiFiNINA> 1.8.13
|   |   |   |-- <SPI> 1.0
|   |   |-- <WiFiClientSecure> 1.0
|   |   |   |-- <WiFiNINA> 1.8.13
|   |   |   |   |-- <SPI> 1.0
|   |-- <WiFiNINA> 1.8.13
|   |   |-- <SPI> 1.0
|   |-- <WiFiClientSecure> 1.0
|   |   |-- <WiFiNINA> 1.8.13
|   |   |   |-- <SPI> 1.0
|   |-- <EEPROM> 1.0.3
|-- <WiFiNINA> 1.8.13
|   |-- <SPI> 1.0

This explains why my build for nano_33_iot was working, while esp32 was failing. The FlashStorage and WiFiNINA libraries were always built, no matter what library.json said!

In order to replicate exactly your scenario. I cloned the repo to a new folder (outside of the arduino libraries directory) . After that, I run the same commands I posted above. I get exactly the same behavior you have seen.
Ie. the nano_33_iot build fails, while esp32 and esp8266 build successfully

The problem here though, is that platformio does not recognize WiFiNINA and FlashStorage as dependencies.