Pio ci failes due to missing sdfat.h

kiliw · January 2, 2022, 7:51pm

Hey everyone,

I have a problem with pio ci command. If I build my project locally via the vs code platform io integration and upload it to the mega everything works fine. Unfortunately in CI not. Even if I run the following command locally it does not work

pio ci src/main.cpp --board=megaatmega2560

The problem is that SdFat.h cant be found… I tried putting the library int the lib folder. Tried different versions. As mentioned above, if I build the project locally everything works fine.

// github workflow
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - name: Cache pip
        uses: actions/cache@v2
        with:
          path: ~/.cache/pip
          key: ${{ runner.os }}-pip-${{ hashFiles('**/requirements.txt') }}
          restore-keys: |
            ${{ runner.os }}-pip-
      - name: Cache PlatformIO
        uses: actions/cache@v2
        with:
          path: ~/.platformio
          key: ${{ runner.os }}-${{ hashFiles('**/lockfiles') }}
      - name: Set up Python
        uses: actions/setup-python@v2
      - name: Install PlatformIO
        run: |
          python -m pip install --upgrade pip
          pip install --upgrade platformio
      - name: Install library dependencies
        run: cd packages/mc && pio lib -g install 1

      - name: Run PlatformIO
        run: cd packages/mc && pio ci src/main.cpp --board=megaatmega2560

Here is the error in the pipeline (which is the same as locally)

// Error
Run cd packages/game && pio ci src/main.cpp --board=megaatmega2560
  cd packages/game && pio ci src/main.cpp --board=megaatmega2560
  shell: /usr/bin/bash -e {0}
  env:
    pythonLocation: /opt/hostedtoolcache/Python/3.10.1/x64
    LD_LIBRARY_PATH: /opt/hostedtoolcache/Python/3.10.1/x64/lib
The next files/directories have been created in /tmp/tmpt4cll55k
include - Put project header files here
lib - Put here project specific (private) libraries
src - Put project source files here
platformio.ini - Project Configuration File

Project has been successfully initialized! Useful commands:
`pio run` - process/build project from the current directory
`pio run --target upload` or `pio run -t upload` - upload firmware to a target
`pio run --target clean` - clean project (remove compiled files)
`pio run --help` - additional information
Processing megaatmega2560 (platform: atmelavr; board: megaatmega2560; framework: arduino)
--------------------------------------------------------------------------------
Platform Manager: Installing atmelavr
Downloading...
Unpacking...
Platform Manager: atmelavr @ 3.4.0 has been installed!
The platform 'atmelavr' has been successfully installed!
The rest of the packages will be installed later depending on your build environment.
Tool Manager: Installing platformio/toolchain-atmelavr @ ~1.70300.0
Downloading...
Unpacking...
Tool Manager: toolchain-atmelavr @ 1.70300.191015 has been installed!
Tool Manager: Installing platformio/framework-arduino-avr @ ~5.1.0
Downloading...
Unpacking...
Tool Manager: framework-arduino-avr @ 5.1.0 has been installed!
Tool Manager: Installing platformio/tool-scons @ ~4.40300.0
Downloading...
Unpacking...
Tool Manager: tool-scons @ 4.40300.1 has been installed!
Verbose mode can be enabled via `-v, --verbose` option
CONFIGURATION: https://docs.platformio.org/page/boards/atmelavr/megaatmega2560.html
PLATFORM: Atmel AVR (3.4.0) > Arduino Mega or Mega 2560 ATmega2560 (Mega 2560)
HARDWARE: ATMEGA2560 16MHz, 8KB RAM, 248KB Flash
DEBUG: Current (avr-stub) On-board (avr-stub, simavr)
PACKAGES: 
 - framework-arduino-avr 5.1.0 
 - toolchain-atmelavr 1.70300.191015 (7.3.0)
LDF: Library Dependency Finder -> https://bit.ly/configure-pio-ldf
LDF Modes: Finder ~ chain, Compatibility ~ soft
Found 6 compatible libraries
Scanning dependencies...
Dependency Graph
|-- <SPI> 1.0
Building in release mode
Compiling .pio/build/megaatmega2560/src/main.cpp.o
Compiling .pio/build/megaatmega2560/lib716/SPI/SPI.cpp.o
src/main.cpp:14:10: fatal error: SdFat.h: No such file or directory

***************************************************************
* Looking for SdFat.h dependency? Check our library registry!
*
* CLI  > platformio lib search "header:SdFat.h"
* Web  > https://platformio.org/lib/search?query=header:SdFat.h
*
***************************************************************

 #include <SdFat.h> //Sd Card
          ^~~~~~~~~
compilation terminated.
*** [.pio/build/megaatmega2560/src/main.cpp.o] Error 1
========================= [FAILED] Took 12.73 seconds =========================
Error: Process completed with exit code 1.

Thanks in advance.

All the best and happy coding.
Kilian

maxgerhardt · January 2, 2022, 8:27pm

You may try to give it an extra option like

--project-option="lib_deps=arduino-libraries/SD@^1.2.4"

kiliw · January 3, 2022, 7:56am

@maxgerhardt thanks for the tip. Still a bit strange though. I could get past the sdfat error but then another accoured complaining about another lib. I tried adding it with
--project-option="lib_deps=miguelbalboa/MFRC522@^1.4.10" all together with the sdfat one but it did not work.
Finally I copopied again the the sdfat lib to the lib folder of the project and only included it in the build command. Still I dont understand why.

I come from a JS background and there you have your package.json in which all dependencies are defiened. I assumend that is similar to the platformio.ini. But why do I have to include some libraries in the ci command?

Thanks again for your solution.

maxgerhardt · January 3, 2022, 8:48am

You must accumulate all libraries into one lib_deps expression. See docs on lib_deps. So it could e.g. be

pio ci src/main.cpp --board=megaatmega2560 --project-option="lib_deps=arduino-libraries/SD@^1.2.4, miguelbalboa/MFRC522@^1.4.10"

As explained in the documentation, pio ci creates a new blank project on the fly with only the options specified in the command. PlatformIO doesn’t just go out in the world and try to download a library dependency that it encounters during compilation from the registry (an exception are framework built-in libraries, like SPI or Wire), they need to be either

globally installed libraries (pio lib -g install <name>)
in the lib/ folder of the project (does not apply to pio ci since it only takes the source files specified inthe command, ignores everything everything else by default, applies to pio run)
in the project platformio.ini’s lib_deps expression (which when doing pio ci is initially blank.)

If you already have the project set-up with a platformio.ini (which looks so if the path is src/main.cpp), you should just use pio run instead of pio ci to compile the project. pio run is a perfectly valid command even in CI situations and used by the offical repos, me and other people. pio ci (as used e.g. here) is still perfectly valid though if used correctly (i.e., mostly library preinstallation).

kiliw · January 3, 2022, 10:56am

Awesome! Thanks for your detailed explanation. Now everything works fine