Problem with lib_deps in platformio.ini file while trying to compile multiple .cpp files

Hi. I am a newbie in platformio.

In my project I have a folder named “lab” in the main folder. This folder have some folders with test code that I want to run separately I wrote a python script so that I can change src dir based on some console inputs before compilation.

The problem is that some of these test codes compile succesfully and others don’t. I’m puzzled.

I tried a bunch of stuff. I read this post for example. It didn’t work.

I also moved my .cpp file to src folder alone. It did not work…

Then I found an ugly fix to my problem

In my platformio.ini file I have this

lib_deps =
  # RECOMMENDED
  # Accept new functionality in a backwards compatible manner and patches
  esphome/ESP32-audioI2S @ ^2.0.7
  # RECOMMENDED
  # Accept new functionality in a backwards compatible manner and patches
  bodmer/TFT_eSPI @ ^2.5.43
  # RECOMMENDED
  # Accept new functionality in a backwards compatible manner and patches
  bitbank2/AnimatedGIF @ ^2.1.0

In the .cpp files that don’t compile I don’t use some of these libraries. If I comment those libraries that I don’t use in a particular .cpp file, then it compiles. But I don’t understand what is the problem.

I get this log when trying to compile a troublesome .cpp file

CONFIGURATION: https://docs.platformio.org/page/boards/espressif32/esp32-s3-devkitc-1.html
PLATFORM: Espressif 32 (6.7.0) > Espressif ESP32-S3-DevKitC-1-N8 (8 MB QD, No PSRAM)
HARDWARE: ESP32S3 240MHz, 320KB RAM, 8MB Flash
DEBUG: Current (esp-builtin) On-board (esp-builtin) External (cmsis-dap, esp-bridge, esp-prog, iot-bus-jtag, jlink, minimodule, olimex-arm-usb-ocd, olimex-arm-usb-ocd-h, olimex-arm-usb-tiny-h, olimex-jtag-tiny, tumpa)
PACKAGES:
 - framework-arduinoespressif32 @ 3.20016.0 (2.0.16)
 - tool-esptoolpy @ 1.40501.0 (4.5.1) 
 - toolchain-riscv32-esp @ 8.4.0+2021r2-patch5
 - toolchain-xtensa-esp32s3 @ 8.4.0+2021r2-patch5
LDF: Library Dependency Finder -> https://bit.ly/configure-pio-ldf
LDF Modes: Finder ~ chain, Compatibility ~ soft
Found 41 compatible libraries
Scanning dependencies...
Dependency Graph
|-- ESP32-audioI2S @ 2.1.0
|-- TFT_eSPI @ 2.5.43
|-- AnimatedGIF @ 2.1.0
|-- FS @ 2.0.0
|-- SD_MMC @ 2.0.0
Building in release mode
Compiling .pio\build\esp32-s3-devkitc-1\src\SDMMC_Test_ESP32_S3.cpp.o
Compiling .pio\build\esp32-s3-devkitc-1\libcd2\ESP32-audioI2S\Audio.cpp.o
Compiling .pio\build\esp32-s3-devkitc-1\lib05b\TFT_eSPI\TFT_eSPI.cpp.o
In file included from .pio/libdeps/esp32-s3-devkitc-1/ESP32-audioI2S/src/Audio.cpp:9:
.pio/libdeps/esp32-s3-devkitc-1/ESP32-audioI2S/src/Audio.h:16:10: fatal error: WiFi.h: No such file or directory

**************************************************************
* Looking for WiFi.h dependency? Check our library registry!
*
* CLI  > platformio lib search "header:WiFi.h"
* Web  > https://registry.platformio.org/search?q=header:WiFi.h
*
**************************************************************

 #include <WiFi.h>
          ^~~~~~~~
compilation terminated.
In file included from .pio/libdeps/esp32-s3-devkitc-1/TFT_eSPI/TFT_eSPI.cpp:16:
.pio/libdeps/esp32-s3-devkitc-1/TFT_eSPI/TFT_eSPI.h:33:12: fatal error: SPI.h: No such file or directory

*************************************************************
* Looking for SPI.h dependency? Check our library registry!
*
* CLI  > platformio lib search "header:SPI.h"
* Web  > https://registry.platformio.org/search?q=header:SPI.h
*
*************************************************************

   #include <SPI.h>
            ^~~~~~~
compilation terminated.
*** [.pio\build\esp32-s3-devkitc-1\libcd2\ESP32-audioI2S\Audio.cpp.o] Error 1
*** [.pio\build\esp32-s3-devkitc-1\lib05b\TFT_eSPI\TFT_eSPI.cpp.o] Error 1

When I try to add these dependencies manually in lib_deps then it fails and says it cannot find more libraries, then I add them to lib_deps, and the same thing occurs on and on.

And I don’t want this kind of solution because it does not apply always.

I want to have the same platformio.ini file for all my test folders.

I find this very rare because those libraries that the compiler cannot find are official ESP32 libraries, so I shouldn’t need to include them manually.

I made a post days ago, someone suggested that I used multiple environments instead of my python script. I don’t know if this issue is related.

What is the reason for the “test-code”?
What do they do and how do they look like?

Your setup sounds a bit unusual and might be the root cause of your problems.

Can you share your project, so I can take a closer look?

1 Like

Hello sivar2311, thank you for your reply

The thing is that I’m not working on my main code per se. I’m writing the libraries that I’m going to need to write that main code easily. I put all those libraries in lib folder. These test codes are just some examples that help me test those libraries thoroughly.

I want to test them alone and together so that I have a bunch of files for test code of those libraries.

This is my project directory.

This is my platformio.ini file

; PlatformIO Project Configuration File
;
;   Build options: build flags, source filter
;   Upload options: custom upload port, speed and extra flags
;   Library options: dependencies, extra library storages
;   Advanced options: extra scripting
;
; Please visit documentation for the other options and examples
; https://docs.platformio.org/page/projectconf.html

[platformio]
default_envs = project

[env]
platform = espressif32
board = esp32-s3-devkitc-1
framework = arduino
monitor_speed = 115200

# extra_scripts = pre:choose_src_dir.py

lib_deps =
  # RECOMMENDED
  # Accept new functionality in a backwards compatible manner and patches
  esphome/ESP32-audioI2S @ ^2.0.7
  # RECOMMENDED
  # Accept new functionality in a backwards compatible manner and patches
  bodmer/TFT_eSPI @ ^2.5.43
  # RECOMMENDED
  # Accept new functionality in a backwards compatible manner and patches
  bitbank2/AnimatedGIF @ ^2.1.0
  
build_flags =
  -D USER_SETUP_LOADED=1

  -D ILI9341_DRIVER=1
  -D TFT_CS=38                       ; 10 or 34 (FSPI CS0
  -D TFT_MOSI=35                     ; 11 or 35 (FSPI D)
  -D TFT_SCLK=36                     ; 12 or 36 (FSPI CLK)
  -D TFT_MISO=37                     ; 13 or 37 (FSPI Q)
  -D TFT_DC=40                       ; 7
  -D TFT_RST=39                      ; 6
  -D LOAD_GLCD=1
  -D LOAD_FONT2=1
  -D LOAD_FONT4=1
  -D LOAD_FONT6=1
  -D LOAD_FONT7=1
  -D LOAD_FONT8=1
  -D LOAD_GFXFF=1
  -D SMOOTH_FONT=1
  -D USE_HSPI_PORT=1                 ; FSPI (or VSPI) port (SPI2) used unless following defined. HSPI port is (SPI3) on S3.
  -D SPI_FREQUENCY=40000000          ; Maximum for ILI9341
  -D SPI_READ_FREQUENCY=6000000      ; 6 MHz is the maximum SPI read speed for the ST7789V
  -D SPI_TOUCH_FREQUENCY=2500000

;;;     ESP32 CONFIGURATION     ;;;

;1 USD CDC On Boot
board_build.cdc_on_boot=0

;2 CPU Frequency
board_build.f_cpu=240000000L

;3 Core Debug Level
board_build.debug=5

;4 USD DFU On Boot
board_build.dfu_on_boot=0

;5 Erase All Flash Before Sketch Upload
board_upload.erase_cmd=-e ; Si

;6 Events Run On
board_build.event_core=1

;7 Flash Mode
board_build.f_flash=80000000L   ; 80MHz
board_build.flash_mode=qio

;8 Flash size
board_upload.flash_size=8MB
board_upload.maximum_size=8388608

;9 JTAG Adapter
board_build.openocdscript=esp32s3-builtin.cfg   ;Integrado
board_build.copy_jtag_files=1                   ;Integrado

;10 Events Run On
board_build.loop_core=1

;11 USD MSC On Boot
board_build.msc_on_boot=0

;12 Partition Scheme
board_build.partitions=default_8MB.csv

;13 PSRAM
board_build.defines=-DBOARD_HAS_PSRAM -mfix-esp32-psram-cache-issue
board_build.memory_type={build.boot}_qspi

;14 Upload Mode
board_upload.use_1200bps_touch=false      ;UART0/Hardware CDC
board_upload.wait_for_upload_port=false   ;UART0/Hardware CDC

;15 Upload speed
upload_speed=921600

;16 USB Mode
board_build.usb_mode=1      ;Hardware CDC and JTAG

[env:project]
# default src_filter

[env:audio-sd]
build_src_filter = +<../lab/audio-sd/*>

[env:gif-audio]
build_src_filter = +<../lab/gif-audio/*>

[env:gif-sd]
build_src_filter = +<../lab/gif-sd/*>

[env:gif-sd-lib]
build_src_filter = +<../lab/gif-sd-lib/*>

[env:sdmmc-test]
build_src_filter = +<../lab/sdmmc-test/*>

I tested using this multi-env scheme and this does not solve the problem, so I think this is not related to my python script.

One of the troublesome .cpp files is this

#include <Arduino.h>

#include "FS.h"
#include "SD_MMC.h"

#define ONE_BIT_MODE false

// Se configuro asi para que no interfiera con la PSRAM octal
// Pines que generan problemas: 37, 36, 35

#define PIN_D2 GPIO_NUM_4
#define PIN_D3 GPIO_NUM_5
#define PIN_CMD GPIO_NUM_6    // SD_MMC
#define PIN_CLK GPIO_NUM_7
#define PIN_D0 GPIO_NUM_15
#define PIN_D1 GPIO_NUM_16

int d2  = 4;
int d3  = 5; 
int cmd = 6;                  // SD_MMC
int clk = 7;
int d0  = 15;
int d1  = 16;

void listDir(fs::FS &fs, const char * dirname, uint8_t levels){
    Serial.printf("Listing directory: %s\n", dirname);

    File root = fs.open(dirname);
    if(!root){
        Serial.println("Failed to open directory");
        return;
    }
    if(!root.isDirectory()){
        Serial.println("Not a directory");
        return;
    }

    File file = root.openNextFile();
    while(file){
        if(file.isDirectory()){
            Serial.print("  DIR : ");
            Serial.println(file.name());
            if(levels){
                listDir(fs, file.path(), levels -1);
            }
        } else {
            Serial.print("  FILE: ");
            Serial.print(file.name());
            Serial.print("  SIZE: ");
            Serial.println(file.size());
        }
        file = root.openNextFile();
    }
}

void createDir(fs::FS &fs, const char * path){
    Serial.printf("Creating Dir: %s\n", path);
    if(fs.mkdir(path)){
        Serial.println("Dir created");
    } else {
        Serial.println("mkdir failed");
    }
}

void removeDir(fs::FS &fs, const char * path){
    Serial.printf("Removing Dir: %s\n", path);
    if(fs.rmdir(path)){
        Serial.println("Dir removed");
    } else {
        Serial.println("rmdir failed");
    }
}

void readFile(fs::FS &fs, const char * path){
    Serial.printf("Reading file: %s\n", path);

    File file = fs.open(path);
    if(!file){
        Serial.println("Failed to open file for reading");
        return;
    }

    Serial.print("Read from file: ");
    while(file.available()){
        Serial.write(file.read());
    }
}

void writeFile(fs::FS &fs, const char * path, const char * message){
    Serial.printf("Writing file: %s\n", path);

    File file = fs.open(path, FILE_WRITE);
    if(!file){
        Serial.println("Failed to open file for writing");
        return;
    }
    if(file.print(message)){
        Serial.println("File written");
    } else {
        Serial.println("Write failed");
    }
}

void appendFile(fs::FS &fs, const char * path, const char * message){
    Serial.printf("Appending to file: %s\n", path);

    File file = fs.open(path, FILE_APPEND);
    if(!file){
        Serial.println("Failed to open file for appending");
        return;
    }
    if(file.print(message)){
        Serial.println("Message appended");
    } else {
        Serial.println("Append failed");
    }
}

void renameFile(fs::FS &fs, const char * path1, const char * path2){
    Serial.printf("Renaming file %s to %s\n", path1, path2);
    if (fs.rename(path1, path2)) {
        Serial.println("File renamed");
    } else {
        Serial.println("Rename failed");
    }
}

void deleteFile(fs::FS &fs, const char * path){
    Serial.printf("Deleting file: %s\n", path);
    if(fs.remove(path)){
        Serial.println("File deleted");
    } else {
        Serial.println("Delete failed");
    }
}

void testFileIO(fs::FS &fs, const char * path){
    File file = fs.open(path);
    static uint8_t buf[512];
    size_t len = 0;
    uint32_t start = millis();
    uint32_t end = start;
    if(file){
        len = file.size();
        size_t flen = len;
        start = millis();
        while(len){
            size_t toRead = len;
            if(toRead > 512){
                toRead = 512;
            }
            file.read(buf, toRead);
            len -= toRead;
        }
        end = millis() - start;
        Serial.printf("%u bytes read for %u ms\n", flen, end);
        file.close();
    } else {
        Serial.println("Failed to open file for reading");
    }


    file = fs.open(path, FILE_WRITE);
    if(!file){
        Serial.println("Failed to open file for writing");
        return;
    }

    size_t i;
    start = millis();
    for(i=0; i<2048; i++){
        file.write(buf, 512);
    }
    end = millis() - start;
    Serial.printf("%u bytes written for %u ms\n", 2048 * 512, end);
    file.close();
}

void setup(){
    Serial.begin(115200);
    
    // If you want to change the pin assigment on ESP32-S3 uncomment this block and the appropriate
    // line depending if you want to use 1-bit or 4-bit line.
    // Please note that ESP32 does not allow pin change and will always fail.
    //if(! SD_MMC.setPins(clk, cmd, d0)){
    if(! SD_MMC.setPins(clk, cmd, d0, d1, d2, d3)){
        Serial.println("Pin change failed!");
        return;
    }

    //Personal. Se siguio la indicacion del README
    
    gpio_set_pull_mode(PIN_CLK, GPIO_PULLUP_ONLY);   // Lo agregue yo, siguiendo el README
    gpio_set_pull_mode(PIN_CMD, GPIO_PULLUP_ONLY);   // CMD, needed in 4- and 1- line modes
    gpio_set_pull_mode(PIN_D0, GPIO_PULLUP_ONLY);    // D0, needed in 4- and 1-line modes
    gpio_set_pull_mode(PIN_D1, GPIO_PULLUP_ONLY);    // D1, needed in 4- and 1-line modes
    gpio_set_pull_mode(PIN_D2, GPIO_PULLUP_ONLY);    // D1, needed in 4- and 1-line modes
    gpio_set_pull_mode(PIN_D3, GPIO_PULLUP_ONLY);    // D1, needed in 4- and 1-line modes
    
    if(!SD_MMC.begin("/sdcard", ONE_BIT_MODE)){                  
        Serial.println("Card Mount Failed");
        return;
    }
    uint8_t cardType = SD_MMC.cardType();

    if(cardType == CARD_NONE){
        Serial.println("No SD_MMC card attached");
        return;
    }

    Serial.print("SD_MMC Card Type: ");
    if(cardType == CARD_MMC){
        Serial.println("MMC");
    } else if(cardType == CARD_SD){
        Serial.println("SDSC");
    } else if(cardType == CARD_SDHC){
        Serial.println("SDHC");
    } else {
        Serial.println("UNKNOWN");
    }

    uint64_t cardSize = SD_MMC.cardSize() / (1024 * 1024);
    Serial.printf("SD_MMC Card Size: %lluMB\n", cardSize);

    listDir(SD_MMC, "/", 0);
    createDir(SD_MMC, "/mydir");
    listDir(SD_MMC, "/", 0);
    removeDir(SD_MMC, "/mydir");
    listDir(SD_MMC, "/", 2);
    writeFile(SD_MMC, "/hello.txt", "Hello ");
    appendFile(SD_MMC, "/hello.txt", "World!\n");
    readFile(SD_MMC, "/hello.txt");
    deleteFile(SD_MMC, "/foo.txt");
    renameFile(SD_MMC, "/hello.txt", "/foo.txt");
    readFile(SD_MMC, "/foo.txt");
    testFileIO(SD_MMC, "/test.txt");
    Serial.printf("Total space: %lluMB\n", SD_MMC.totalBytes() / (1024 * 1024));
    Serial.printf("Used space: %lluMB\n", SD_MMC.usedBytes() / (1024 * 1024));
}

void loop(){

}

This is just an adaptation of an official example of the SD_MMC library. This is the code that lead to the log I posted above, I just changed the file name, nothing else.

This code do not use none of my custom libraries, so I think this is better to isolate the problem and focusing on what is important.

As I said above, changing lib_dep as described above solved the problem, and this also builds in Arduino IDE.

I just want to use the same setup for everything, I don’t want to do this lib_dep fix I found for each of my test codes.

What can I do?

Tell me if you need more information about my project.

Thank you!

That is the cause of the problem.

Unfortunately, some libraries are designed in such a way that a project will not compile without the explicit inclusion (#include) of the library.

The (nevertheless very good) ESPAsyncWebServer library is a practical example of this.

Also the TFT_eSPI library causing this error. If it is present in the lib_deps your sketch must include at minimal SPI.h, FS.h and SPIFFS.h even they are not used by your sketch. Otherwise TFT_eSPI won’t compile. TFT_eSPI might not be used by your sketch, but the source code is present and will be compiled.

In this case, you have no choice but to remove the library from the lib_deps of the “global” environment and only include it in the environments where the library is actually used.

You must use lib_ignore in the relevant environments. Example

[platformio]
default_envs = project

[env]
platform = espressif32
board = esp32dev
framework = arduino
lib_deps = 
  bodmer/TFT_eSPI

[env:project]

[env:display-test]
build_src_filter = +<../lab/display-test/*>

[env:sdmmc-test]
build_src_filter = +<../lab/sdmmc-test/*>
lib_ignore = 
  TFT_eSPI
1 Like

Hello!

I tried your solution and it worked perfectly, everything compiles.

I didn’t want to manually change the global lib_dep when switching examples, but with lib_ignore this configuration can be done once, very nice.

Now I’m starting to see the benefits of multi-env schemes.

Thank you!

1 Like