What's the procedure for developing nested dependant libraries using platformIO

Libraries
1

I maintain a few open-source libraries (hosted on pio), IoAbstraction, LiquidCrystalIO and tcMenu. My question revolves around how to use the source from a known directory instead of using a library dependency. I’ve tried this: [Nested Libraries] but it doesn’t work for me because it only includes the headers, so the code compiles but doesn’t link because the CPP source in the include_directories that I added was not compiled.

My environment is clion with the new platform io plugin. platformio and clion including plugin are up to date as of this morning. However, it fails on the command line so rules out clion there.

Is there a way to develop by including the source of a library into my current project instead of working with a dependency. I even tried using pio lib install file:pathtolib and that didn’t work either, I ended up with a cmake file full of references from the platformio hidden directory of the dependant library.

The main problem seems to be that when using library manager for this, it pulls in all the hidden directories, tests and everything else, and makes a local copy of it too. What most library developers need is to have a single copy of each library and reference them, as often there’s a need to change more than one library at once.

I have no issue having two library projects open as separate projects, I just want the nested library to directly reference its dependency. Similar to what happens with maven / nuget in Java / C#, or Visual Micro Arduino plugin for VS where the locally available source is taken over using a dependency.

Hopefully someone can help me with this, as I’m really liking the Clion / pio combination.

2

I kept digging through the options, and found that adding the following option to the environment in the leaf library works:

[env:someenv]
.. other stuff
lib_extra_dirs = path/to/root/of/lib

Compiles and uploads at least for AVR, will try mbed later and feed-back. In Clion, this also pulls all the source files into the project, (which I actually don’t mind). Could someone with more experience of pio confirm that this is the right thing to do as it certainly seems to work like that?

3

Although I managed to get things working for Arduino framework / AVR, I have not managed to get this to work for mbed. It is failing in a way that I cannot really understand, even after looking at the verbose output.

Basically, the library is an Arduino compatible directory structure available to view here along with platformio.ini:

The library it depends on is IoAbstraction, I’ve included that locally using the method above.

On Arduino: all working, LiquidCrystalIO.h is found in the src directory. All builds and uploads fine.

On mbed: not working, LiquidCrystalIO.h is not found and as far as I can see the src directory is not included in the build path. Is this expected on mbed, is there something I’m missing?

This is not consistent with IoAbstraction, where it pulled in the src directory during the build.

What i see looking at the output is that the libraries src directory is never pulled in (full output can be put somewhere if needed):

Processing STM439 (platform: ststm32; board: nucleo_f439zi; framework: mbed; build_flags: -D_NO_EEPROM_CLASS_=1, -DPIO_FRAMEWORK_MBED_RTOS_PRESENT, -DIO_LOGGING_DEBUG=1; lib_extra_dirs: /Users/dave/Documents/Arduino/libraries/IoAbstraction)
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
CONFIGURATION: https://docs.platformio.org/page/boards/ststm32/nucleo_f439zi.html
PLATFORM: ST STM32 6.1.0 > ST Nucleo F439ZI
HARDWARE: STM32F439ZIT6 180MHz, 256KB RAM, 2MB Flash
DEBUG: Current (stlink) On-board (stlink) External (blackmagic, jlink)
PACKAGES: 
 - framework-mbed 6.51401.200402 (5.14.1) 
 - tool-dfuutil 1.9.200310 
 - tool-openocd 2.1000.190707 (10.0) 
 - tool-stm32duino 1.0.1 
 - toolchain-gccarmnoneeabi 1.70201.0 (7.2.1)
Collecting mbed sources...
LDF: Library Dependency Finder -> http://bit.ly/configure-pio-ldf
LDF Modes: Finder ~ chain, Compatibility ~ soft
Framework incompatible library /Users/dave/Documents/Arduino/libraries/LiquidCrystalIO/src
More details about "Library Compatibility Mode": https://docs.platformio.org/page/librarymanager/ldf.html#ldf-compat-mode
Found 17 compatible libraries
Scanning dependencies...
Dependency Graph
|-- <src> (/Users/dave/Documents/Arduino/libraries/IoAbstraction/src)
Building in release mode

Results in:

*************************************************************************
* Looking for LiquidCrystalIO.h dependency? Check our library registry!
*
* CLI  > platformio lib search "header:LiquidCrystalIO.h"
* Web  > https://platformio.org/lib/search?query=header:LiquidCrystalIO.h
*
*************************************************************************

 #include "LiquidCrystalIO.h"
          ^~~~~~~~~~~~~~~~~~~
compilation terminated.
*** [.pio/build/STM439/src/helloMbed.o] Error 1

Directory structure:

% tree
.
├── README.md
├── examples
│   ├── Autoscroll
│   │   └── Autoscroll.ino
│   ├── Blink
│   │   └── Blink.ino
│   ├── Counter23017
│   │   └── Counter23017.ino
│   ├── Cursor
│   │   └── Cursor.ino
│   ├── CustomCharacter
│   │   └── CustomCharacter.ino
│   ├── Display
│   │   └── Display.ino
│   ├── HelloI2c
│   │   └── HelloI2c.ino
│   ├── HelloShiftReg
│   │   └── HelloShiftReg.ino
│   ├── HelloWorld
│   │   └── HelloWorld.ino
│   ├── Scroll
│   │   └── Scroll.ino
│   ├── SerialDisplay
│   │   └── SerialDisplay.ino
│   ├── TextDirection
│   │   └── TextDirection.ino
│   ├── helloMbed
│   │   └── helloMbed.cpp
│   └── setCursor
│       └── setCursor.ino
├── keywords.txt
├── library.json
├── library.properties
├── platformio.ini
└── src
    ├── LiquidCrystalIO.cpp
    └── LiquidCrystalIO.h

Ini file to save going to link to take a look:

; Please visit documentation for the other options and examples
; https://docs.platformio.org/page/projectconf.html

[platformio]
src_dir = examples/helloMbed
lib_dir = .

; Sourcecode version that requires IoAbstraction installed in parent directory
; to use external deps during development, remove the -I build flag and add IoAbstraction
[env:ATmega2560]
platform = atmelavr
board = megaatmega2560
framework = arduino
upload_port = /dev/cu.usbmodem14101
;upload_port = COM3
monitor_speed = 115200
lib_deps = Wire
build_flags = -DIO_LOGGING_DEBUG=1
              -DDEFAULT_TASK_SIZE=3
lib_extra_dirs = /Users/dave/Documents/Arduino/libraries/IoAbstraction

; source code version for mbed, that finds io abstraction in parent directory.
; to use external deps during development, remove the -I build flag and add IoAbstraction
[env:STM439]
platform = ststm32
board = nucleo_f439zi
framework = mbed
;upload_port = /dev/cu.usbmodem14103
;upload_port = COM8
;monitor_speed = 115200
build_flags = -D_NO_EEPROM_CLASS_=1
              -DPIO_FRAMEWORK_MBED_RTOS_PRESENT
              -DIO_LOGGING_DEBUG=1
lib_extra_dirs = /Users/dave/Documents/Arduino/libraries/IoAbstraction

The full output is too large to pasted into here, but if you wanted the full output I can attach. I don’t think I’ll be able to answer my own question this time :slight_smile:

4

Looks very unhealthy that src/ is the detected dependency.

Looking at the docs I think this should be set to the libraries/ folder which contains all libraries, not only this single library. What happens when you set

lib_extra_dirs = /Users/dave/Documents/Arduino/libraries/

… Arduino library folders in an mbed-os project ó_Ô? What’s the rationale here? These should all be non-compilable when they attempt to use Arduino library functions.

5

Many thanks, this has led me to think about this problem in a different way and got me up and running. Rather than try and turn all the libraries into projects with the build directories and build scripts, I just cloned all the libraries into the lib directory of a brand new project, and work with platformio as intended. Clion appears to pick this up properly as well.

It seems odd but the libraries are cross-compatible, they conditionally compile in the right code for each environment. It means that tcMenu can target both codebases with few changes.

IoAbstraction is quite good at providing abstractions around IO, and it turned out to be quite easy to provide an mbed port. I was initially dreading it, but it took a few days to get the basics working. in fact once IoAbstraction was ported, LiquidCrystalIO followed very quickly.