Unable to disable LDF for ESP32

Libraries
1

Hi,

First, thanks for this awesome project!

I’m developing a tool that builds custom ESP firmwares for users based on some configuration files, and internally it’s using platformio.

Some users have started complaining about compilation speed recently, so I wanted to remove as much stuff as possible for compiling. So ideally I would disable the library dependency finder with lib_ldf_mode = off.

However, that makes it impossible to import ESP32 WiFi.h AFAIK. Usually, the LDF would add the build-in WiFi library automagically. But as the LDF is disabled, I have to add it explicitly in lib_deps. But that leads platformio to import this Arduino library :confused:

I tried this all with a combination of lib_compat_mode = strict, lib_ignore = id=870 and so on but couldn’t come up with a solution.

Minmal platformio.ini:

[env:test]
platform = espressif32@1.6.0
board = nodemcu-32s
lib_ldf_mode = off
lib_deps =
    WiFi
; adding these doesn't help
lib_compat_mode = strict
lib_ignore = id=870

Is it somehow possible to mark WiFi as a built-in library, or make platformio not use the library with id 870 that clearly is incompatible with ESP32s?

Reference: this issue

2

Here’s what happens without adding lib_deps = WiFi

LDF MODES: FINDER(off) COMPATIBILITY(strict)
Collected 36 compatible libraries
Scanning dependencies...
Dependency Graph
|-- <AsyncTCP> 1.0.1
|-- <ESPmDNS> 1.0
|-- <Preferences> 1.0
|-- <Update> 1.0
|-- <esphomelib> 1.11.0-dev #2ff00db
|   |-- <AsyncMqttClient> 0.8.2
|   |   |-- <AsyncTCP> 1.0.1
|   |-- <ArduinoJson-esphomelib> 5.13.3
Compiling .pioenvs/test/src/main.cpp.o
Generating partitions .pioenvs/test/partitions.bin
Compiling .pioenvs/test/lib59e/AsyncTCP_ID1826/AsyncTCP.cpp.o
Compiling .pioenvs/test/lib617/ESPmDNS/ESPmDNS.cpp.o
In file included from .piolibdeps/esphomelib/src/esphomelib/application.h:22:0,
from src/main.cpp:3:
.piolibdeps/esphomelib/src/esphomelib/ota_component.h:10:24: fatal error: WiFiServer.h: No such file or directory

********************************************************************
* Looking for WiFiServer.h dependency? Check our library registry!
*
* CLI  > platformio lib search "header:WiFiServer.h"
* Web  > https://platformio.org/lib/search?query=header:WiFiServer.h
*
********************************************************************

compilation terminated.
Compiling .pioenvs/test/libad3/Preferences/Preferences.cpp.o
*** [.pioenvs/test/src/main.cpp.o] Error 1
/Users/ottowinter/.platformio/packages/framework-arduinoespressif32@2.10000.0/libraries/ESPmDNS/src/ESPmDNS.cpp:42:18: fatal error: WiFi.h: No such file or directory

**************************************************************
* Looking for WiFi.h dependency? Check our library registry!
*
* CLI  > platformio lib search "header:WiFi.h"
* Web  > https://platformio.org/lib/search?query=header:WiFi.h
*
**************************************************************

compilation terminated.
*** [.pioenvs/test/lib617/ESPmDNS/ESPmDNS.cpp.o] Error 1
========================== [ERROR] Took 3.37 seconds ==========================

and here with lib_deps = WiFi

LDF MODES: FINDER(off) COMPATIBILITY(strict)
Collected 36 compatible libraries
Scanning dependencies...
Dependency Graph
|-- <AsyncTCP> 1.0.1
|-- <ESPmDNS> 1.0
|-- <Preferences> 1.0
|-- <Update> 1.0
|-- <esphomelib> 1.11.0-dev #2ff00db
|   |-- <AsyncMqttClient> 0.8.2
|   |   |-- <AsyncTCP> 1.0.1
|   |-- <ArduinoJson-esphomelib> 5.13.3
|-- <WiFi> 1.2.7
Compiling .pioenvs/test/src/main.cpp.o
Generating partitions .pioenvs/test/partitions.bin
Compiling .pioenvs/test/lib59e/AsyncTCP_ID1826/AsyncTCP.cpp.o
Compiling .pioenvs/test/lib617/ESPmDNS/ESPmDNS.cpp.o
In file included from .piolibdeps/esphomelib/src/esphomelib/application.h:28:0,
from src/main.cpp:3:
.piolibdeps/esphomelib/src/esphomelib/wifi_component.h:9:24: fatal error: WiFiType.h: No such file or directory

******************************************************************
* Looking for WiFiType.h dependency? Check our library registry!
*
* CLI  > platformio lib search "header:WiFiType.h"
* Web  > https://platformio.org/lib/search?query=header:WiFiType.h
*
******************************************************************

compilation terminated.
Compiling .pioenvs/test/libad3/Preferences/Preferences.cpp.o*** [.pioenvs/test/src/main.cpp.o] Error 1

/Users/ottowinter/.platformio/packages/framework-arduinoespressif32@2.10000.0/libraries/ESPmDNS/src/ESPmDNS.cpp:42:18: fatal error: WiFi.h: No such file or directory

**************************************************************
* Looking for WiFi.h dependency? Check our library registry!
*
* CLI  > platformio lib search "header:WiFi.h"
* Web  > https://platformio.org/lib/search?query=header:WiFi.h
*
**************************************************************

compilation terminated.
*** [.pioenvs/test/lib617/ESPmDNS/ESPmDNS.cpp.o] Error 1
========================== [ERROR] Took 3.33 seconds ==========================
3

More background on why I want to disable LDF (because I think one suggested solution will be “use the LDF” :stuck_out_tongue: )

The LDF normal “chain” mode is already pretty quick. However, it includes all dependencies of my library automatically. For example each user of my project needs to download “FastLED”, “NeoPixelBus”, “ESP Async WebServer” etc each time they want to compile for a new node.

Ideally, I would like to remove the need to download (+ compile) all those libraries when the user doesn’t use them. And the tool that operates over platformio here (esphomeyaml) also knows exactly when the user needs a particular library.

I could of course also enable chain+ mode, which would make the LDF not include those libraries if they’re not used. However, in some quick benchmarking I found the chain+ mode to be pretty slow. Correct me if I’m wrong, but I think it makes platformio execute gcc for each file, no?

So the best solution for me would be to disable LDF, just … I can’t get it to work :stuck_out_tongue_winking_eye:

4

LDF looks on your #includes in the project. I recommend to do the next:

  1. Move non used includes under #ifdef USE_EXTRA_LIBS guard
  2. Use this guard via build_flags = -D USE_EXTRA_LIBS
  3. Set lib_ldf_mode = chain+.

Did it help?

5

Hi @ivankravets!

Yes, I actually tried that (see second post; a bit long, sorry). And I’m already doing all the feature guards you wrote in 1&2.

However, the problem is that the plus mode is quite slow. The thing I want to minimize is compile time, as that is one of the biggest problems of my application.

If I enable chain+ mode the whole LDF gets very slow (about 200 source files). For example for an ESP8266 project I get the following timings:

  • off: 1s
  • chain: 1.5s
  • chain+: 6s

And that is on a beefy laptop. Quite a few people run my software on RPi where it takes much longer. And waiting 6s for the LDF for each tiny change is a waste, plus I think it should be possible to disable LDF.

So I would like to disable the LDF (like it is possible for ESP8266). Just this one WiFi dependency is breaking everything. I see a few options:

  • Add a manual fix in espressif32 package that redirects WiFi dependency to internal one. (The WiFi library in libr registry doesn’t work anyway).
  • Make platformio prefer internal libraries over online ones. Don’t know how hard this would be with current platformio core design.
  • Make the WiFi library manifest not accept ESP32 platform (it’s not compatible). Problem is that in my experience the Arduino guys there are not very responsive to updates.