V3 Build Error - Library Ordering?

A project that compiles and uploads fine with v2 is failing to compile with v3.

lib/Display/src/Display.cpp: In constructor 'Display::Display(int8_t, int8_t, int8_t, int8_t)':
lib/Display/src/Display.cpp:35:17: error: 'INITR_144GREENTAB' was not declared in this scope
_tft->initR(INITR_144GREENTAB);   // initialize a ST7735S chip, green tab
^

That symbol is declared in Adafruit_ST7735.h, that comes from a library in installed at the global package location:

$ grep 'INITR_144' ~/.platformio/lib/Adafruit_ST7735_ID12/Adafruit_ST7735.h 
#define INITR_144GREENTAB   0x1

The problem might be that v3 platformio is finding that header in a packages (platform?) location, which does not define this symbol…

~/.platformio/packages/framework-arduinoteensy/libraries/TFT/src/utility/Adafruit_ST7735.h

A gist of the full error output – gist:54d9567bf5894ad40c5232eac237a698 · GitHub

Switching back to v2, and running the ‘upload’ target, it compiles successfully – gist:927d3d90ace777343b74dea267607c30 · GitHub

The symbol is defined in .pioenvs/teensy31/Adafruit_ST7735_ID12/Adafruit_ST7735.h

It seems like the library ordering has changed? And in v3 my installed library isn’t taking precedent over the package/platform one?

$ grep '#ifndef' ~/.platformio/packages/framework-arduinoteensy/libraries/TFT/src/utility/Adafruit_ST7735.h
#ifndef _ADAFRUIT_ST7735H_
$ grep '#ifndef' ~/.platformio/lib/Adafruit_ST7735_ID12/Adafruit_ST7735.h 
#ifndef _ADAFRUIT_ST7735H_

I went looking at the documentation for anything that might influence the libraries and/or ordering used.

I added these two lines to platformio.ini and I’m now getting a clean compile:

lib_extra_dirs = /Users/home/.platformio/lib/Adafruit_ST7735_ID12
lib_deep_search = true

Could you create some test project to reproduce this issue? Or you can contact with via PM here. It is very important for me. Thanks.

This is incorrect using of lib_extra_dirs option. See warning block http://docs.platformio.org/en/feature-platformio-30/projectconf.html#lib-extra-dirs

I commented out the lib_extra_dirs, re-init and it still compiles correctly now.

Is there any reason the lib_deep_search is disabled by default? Or why the libs in the ~/.platformio/lib directory don’t take precedence over the ~/.platformio/package/ directory?

I don’t know what to do. I wrote this PlatformIO 3.0 Build System to be very smart. So… We have the problem with this “smart behaviour”. PlatformIO 2.0 has a big problem… It compiles each file and each library. It doesn’t understand about compatibility.

PlatformIO 3.0 has another behaviour. It has support for the different library manifests:

  • library.json - PlatformIO
  • library.properties - Arduino IDE
  • module.json ARM mbed

PlatformIO 3.0 Library Build System analyzes these files and manifests (if they present) and decides how to use it, to extend build environment with extra flags, includes etc. This is very cool! It works 2 times faster than PlatformIO 2.0 and allows to avoid a lot of problems and conflicts.

Nevertheless, this system could be broken when manifest contains incorrect data. This is your case. Adafruit GFX and ST7735 were compatible only with atmelavr development platform. So, the libraries were not used for compilation… ignored! I’ve just updated manifest for these libs:


The question: “What to do with other libs”? How to verify these manifests and update them? This feature is very cool and would good to keep it in PlatformIO 3.0. For example, you have 2 build environments for the project in platformio.ini. The environment is based on ESP8266 and another on Atmel AVR. However, some libs are compatible with ESP8266 and are not compatible with AVR. What is more, both these libs contain file some_header.h that you will include in your project. Of course, you want from PlatformIO for find automatically proper libs depending on the environment data. The current behaviour of PlatformIO 3.0 allows doing that. PlatformIO 2.0 - doesn’t allow.

Finally, PlatformIO 3.0 Build System understands #ifdef, macros and etc. It means, that if a library is under some macros, it will not be included to build process.


P.S: You can build your project now without problems. Just update libs mentioned above. Also, you need to add to dependency list this library

I’m having the same trouble now, with TinyGPS library.

[ 416 ] TinyGPS arduino, atmelavr "Mikal Hart": A compact NMEA (GPS) parsing library

And once installed, and working with PlatformIO v3, it can’t find the library. Works just fine under v2.

Thanks! I found a very cool solution. I’ve introduced today lib_compat_level option. It should resolve 99% further problems. Please upgrade to the new 3.0.0.dev10.

....
source pio3dev/bin/activate
pip install https://github.com/platformio/platformio/archive/feature/platformio-30.zip
platformio --version

See installation info for PlatformIO 3.0 PlatformIO 3.0 Roadmap · Issue #423 · platformio/platformio-core · GitHub

Also, I added to docs information about How does work Library Dependency Finder (LDF)?

Excellent. I did the upgrade (dev11) and re-initialised the project. Building/uploading is working fine.

Please upgraded to dev12. Sorry, I’ve refactored Library Dependency Finder again :frowning: Now, the lib_deep_search is turned ON by default. The new version has smart behaviour and understands nested dependencies and keeps order. Also, it prints Dependency Map.

If left “debug” code for testing. Could you provide me your value from console?

Looking for dependencies...
13 0.312541007996
Library Dependency Map

I need the value that follows after 13 .

  1. Remove .pioenvs directory.
  2. pio run
  3. Copy this value
  4. pio run
  5. Copy this value (it should be lower now)

You need to add a few new libs to your project in the repo:

lib_install = 12, 129, 291, 1106, 416, 355, 357

Thanks!

I’ve got different versions of those libs in my ~/.platformio/libs directory, and it seems to be picking them up correctly.

(pio3dev) $ platformio run --target build
[Wed Jul 27 18:46:36 2016] Processing teensy31 (platform: teensy, lib_install: 129, 291, 1106, 416, board: teensy31, framework: arduino, lib_extra_dirs: /Users/home/.platformio/lib)
-------------------------------------------------------------------------------------------------------------------------------------------------------------------
Collected 35 compatible libraries
Looking for dependencies...
13 0.0425350666046
Library Dependency Map
|-- <Bounce2> v2.1
|-- <TinyGPS>
|-- <Encoder>
|-- <Display>
|   |-- <TFT> v1.0
|   |   |-- <SPI> v1.0
|   |-- <SPI> v1.0
|   |-- <ADC>
|-- <MTS>
|-- <ADC>
|-- <RKW_Led> v0.1.0
Processing build
scons: *** Do not know how to make File target `build' (/Users/home/arduino/digihead/build).  Stop.
==================================================================== [ERROR] Took 0.41 seconds ====================================================================

Not sure why --target build isn’t working? When I open in CLion and tell it to build, it does.

There is no target “build” :slight_smile: it’s mapped to pio run In CLion

I have another library challenge for you Ivan :fearful: … I can’t get this one to work. It’s a struggle trying to trace out the various include stacks, but ultimately I can’t get it to compile even on a bare project.

GitHub - sumotoy/TFT_ST7735 at 1.0p1 (note the branch is 1.0p1)

Cloned into my arduino folder, and soft-linked over to ~/.platformio/lib/TFT_ST7735/

Performed the pio init --ide clion --board teensy31.

The only two include lines are:

#include <Arduino.h>
#include <TFT_ST7735.h>

And that will cause many build errors like:

/Users/home/.platformio/lib/TFT_ST7735/_fonts/defaultFont.c:39:1: error: unknown type name '_smCharType'
const _smCharType image_data_defaultFont_0x20[1]
^

However that _smCharType is defined for my teensy cpu type (__MK20DX256__) over in _includes/TFT_ST7735_cpuCommons.h:44

#define _smCharType unsigned char`

It’s an odd arrangement of includes, and might susceptible to ordering problems? Might be exploring some edge cases of the library dependency finder logic??

In a related note, I saw Teensy Paul mention that today’s new release of Arduino IDE is using gcc for finding includes… Arduino 1.6.10 | Teensy Forum

… using “gcc -E”, which is unfortunately slower, but it is absolutely guaranteed to give the same preprocessing as gcc run in normal compile mode. I shudder to think what a mess 1.6.5 could have been if they’d used two different preprocessors to discover required library headers

This isn’t a problem of the builder. We introduced support for 2 different build systems and their manifests:

  • library.properties (Arduino IDE specific structure)
  • module.json (ARM mbed, yotta build system)

Of course, the main and recommended manifest is library.json. When Library Dependency Finder collects compatible libraries from different source storages, it tries to initialize most compatible library sub-builder for the library. See source code:

  • class ArduinoLibBuilder(LibBuilderBase)
  • class MbedLibBuilder(LibBuilderBase)
  • class PlatformIOLibBuilder(LibBuilderBase)

If a library contains library.json, then the PlatformIOLibBuilder will be used. There is a big difference between these builders. For example (your case), this is Arduino IDE based library. ArduinoLibBuilder has specific behavior… If library doesn’t contain src folder in the root, then need to build only source files that are located in the root. All other folders should be ignored. If lib contains src, then all files should be ignored and only src folder should be built.

The other big story with ARM mbed’s yotta… I’ll not describe it here.


What do we have? This library contains library.json manifest file but doesn’t explain PlatformIO Build System how to build this library. In this case, PlatformIO Library Builder will build ALL files (the default behavior). In this case, need to explain in manifest which files should be built. See my PR:

https://github.com/sumotoy/TFT_ST7735/pull/7


I’ve just removed debug code and fixed a few issues. Please update to the latest commit. It should work very well now.

That is not a secret that our build system is super fast and doesn’t have competitors. We have a lot of tests within our community. As result, we decided to write universal build system that will support different manifests from the other build systems.

If you will see any problems with library, try to remove library.json. In this case, PlatformIO will try to find better sub-builder for the library automatically.

Nevertheless, the best way is to update library.json in the owner’s repo. It will allow to avoid problems for other developers. Here is a new docs for build field http://docs.platformio.org/en/feature-platformio-30/librarymanager/config.html#build

Thank you for such a detailed explanation.

I removed the library.json from my local checkout of that library, and my project built cleanly!

I’ll update to latest piodev3 and keep developing…