Library dependancy doesn't download

Hi everyone,

I work on a lib called Luos and this lib depend on another one, LuosHAL.

I released a new revision of LuosHAL (0.1.0) and I’m working on a new revision of Luos (1.0.0).

If I have Luos(1.0.0 unreleased yet) and LuosHAL(0.1.0 released) locally everything is compiling.

|-- <Luos> 1.0.0
|   |-- <LuosHAL> 0.1.0
|-- <Gate> 0.7.0
|   |-- <Luos> 1.0.0
|   |   |-- <LuosHAL> 0.1.0

If I have Luos but not LuosHAL locally, Platformio doesn’t try to find LuosHAL

|-- <Luos> 1.0.0
|-- <Gate> 0.7.0
|   |-- <Luos> 1.0.0

There is no download error or anything, except of course the compilation failure resulting of the missing lib : Examples update by nicolas-rabault · Pull Request #32 · Luos-io/Examples · GitHub

If I run :

pio lib install "LuosHAL"

LuosHAL is downloaded in 0.1.0 without trouble and after that the code is compiling.

Why Platformio doesn’t try to download LuosHAL ?
Did I miss something ?

Installing Luos via pio lib install Luos does install all its dependency libraries… but probably in the wrong version?

>pio lib install Luos
Library Storage: C:\Users\Max\Documents\PlatformIO\Projects\atmel_hello\.pio\libdeps\uno
Library Manager: Installing Luos
Library Manager: Warning! More than one package has been found by Luos requirements:
 - luos-io/Luos @ 0.7.0
 - luos/Luos @ 1.0.0
Library Manager: Please specify detailed REQUIREMENTS using package owner and version (showed above) to avoid name conflicts
Library Manager: Luos @ 0.7.0 has been installed!
Library Manager: Installing dependencies...
Library Manager: Installing LuosHAL
Library Manager: Warning! More than one package has been found by LuosHAL requirements:
 - nicolas_luos/LuosHAL @ 0.0.2
 - luos/LuosHAL @ 0.1.0
Library Manager: Please specify detailed REQUIREMENTS using package owner and version (showed above) to avoid name conflicts
Library Manager: LuosHAL @ 0.0.2 has been installed!

…with 4 warnings regarding which packet I want to choose since apparently luos-io, luos and nicolas_luos have registered libraries. Your dependencies declaration doesn’t use an owner name or a library version so PlatformIO is a little confused but still tries to do it.

https://github.com/Luos-io/Luos/blob/dc265d959ab2b8fefcbf1b72f84da0ce46aee274/library.json#L21-L24

See library.json docs.

When I clone your examples and run them, PlatformIO tells me that it has tried to install LuosHAL, but of version 0.0.2 – again, because you didn’t specify the neither owner nor dependency version in the library.json.

>platformio run -d Projects/Button
Processing l0 (platform: ststm32; board: l0; framework: stm32cube)
------------------------------------------------------------------------------------------------------------------------Tool Manager: Installing platformio/framework-stm32cube @ ~2.0.181130
Downloading  [####################################]  100%
Unpacking  [####################################]  100%
Tool Manager: framework-stm32cube @ 2.0.200813 has been installed!
Tool Manager: Installing platformio/tool-dfuutil @ ~1.9.190708
Tool Manager: tool-dfuutil @ 1.9.200310 has been installed!
Tool Manager: Installing platformio/tool-ldscripts-ststm32 @ ~0.1.0
Unpacking  [####################################]  100%
Tool Manager: tool-ldscripts-ststm32 @ 0.1.0 has been installed!
Verbose mode can be enabled via `-v, --verbose` option
CONFIGURATION: https://docs.platformio.org/page/boards/ststm32/l0.html
PLATFORM: ST STM32 (9.0.0) > L0
HARDWARE: STM32F072VBT6 48MHz, 16KB RAM, 128KB Flash
DEBUG: Current (stlink) External (stlink)
PACKAGES:
 - framework-stm32cube 2.0.200813
 - tool-dfuutil 1.9.200310
 - tool-ldscripts-ststm32 0.1.0
 - toolchain-gccarmnoneeabi 1.70201.0 (7.2.1)
LDF: Library Dependency Finder -> http://bit.ly/configure-pio-ldf
LDF Modes: Finder ~ chain, Compatibility ~ soft
Library Manager: Installing Luos @ ^1.0.0
Library Manager: Warning! More than one package has been found by Luos @ ^1.0.0 requirements:
 - luos-io/Luos @ 0.7.0
 - luos/Luos @ 1.0.0
Library Manager: Please specify detailed REQUIREMENTS using package owner and version (showed above) to avoid name conflicts
..
Downloading  [####################################]  100%
Library Manager: Luos @ 1.0.0 has been installed!
Library Manager: Installing dependencies...
Library Manager: Installing LuosHAL
Library Manager: Warning! More than one package has been found by LuosHAL requirements:
 - nicolas_luos/LuosHAL @ 0.0.2
 - luos/LuosHAL @ 0.1.0
Library Manager: Please specify detailed REQUIREMENTS using package owner and version (showed above) to avoid name conflicts
Library Manager: LuosHAL @ 0.0.2 has been installed!
Found 23 compatible libraries
Scanning dependencies...
Dependency Graph
|-- <Luos> 1.0.0
|   |-- <LuosHAL> 0.0.2
|-- <Button> 0.7.0
|   |-- <Luos> 1.0.0
|   |   |-- <LuosHAL> 0.0.2
Building in release mode
Compiling .pio\build\l0\src\gpio.o
Compiling .pio\build\l0\src\main.o
Compiling .pio\build\l0\src\stm32f0xx_hal_msp.o
Compiling .pio\build\l0\src\stm32f0xx_it.o
Compiling .pio\build\l0\src\system_stm32f0xx.o
Compiling .pio\build\l0\src\usart.o
Compiling .pio\build\l0\libc7b\LuosHAL\STM32F0\luosHAL.o
Compiling .pio\build\l0\lib7ca\Robus\src\msg_alloc.o
Compiling .pio\build\l0\lib7ca\Robus\src\port_manager.o
Compiling .pio\build\l0\lib7ca\Robus\src\reception.o
Compiling .pio\build\l0\lib7ca\Robus\src\robus.o
Compiling .pio\build\l0\lib7ca\Robus\src\target.o
Compiling .pio\build\l0\lib7ca\Robus\src\transmission.o
Compiling .pio\build\l0\lib7ca\Luos\luos.o
Compiling .pio\build\l0\lib7ca\Luos\routing_table.o
Compiling .pio\build\l0\lib7ca\Luos\streaming.o
.pio\libdeps\l0\Luos\Robus\src\port_manager.c:11:10: fatal error: luos_hal.h: No such file or directory

So I think your error should be gone if you declare in that luos/Luos@ 1.0.0 package that it needs the dependency luos/LuosHAL@~0.1.0 and update the project’s lib_deps to reflect the used owner and version, too.

2 Likes

Thank you @maxgerhardt,

Yes as i mentioned in this other post, I did a mess with my libs :( - #2 by maxgerhardt
Thank’s to you and @ivankravets everything is fine now.

2 Likes

This problem still happening even if I fix my lib mess.

You can find a compilation fail here : https://github.com/Luos-io/Examples/runs/1474032842?check_suite_focus=true

If i do the same thing on my computer it seems to work.

What could broke my compilation like that ?

After some more tests, I realize that some of my projects work some others don’t. For example the Gate project seems to work but Button doesn’t.

I also can reproduce it on my computer with libs on the PLATFORMIO_LIB_EXTRA_DIRS folder, so platformio doesn’t try to find it.

On the library.json file on the Luos 1.0.0 lib if I replace

    "dependencies": [
        {
            "name": "luos/LuosHAL",
            "version": "~0.1.0"
        }
    ],

by

    "dependencies": [
        {
            "name": "LuosHAL",
            "version": "~0.1.0"
        }
    ],

it’s working.

I updated my lib dependency (Luos 1.0.1) by using

    "dependencies": [
        {
            "name": "LuosHAL",
            "version": "~0.1.0"
        }
    ],

But it doesn’t seems to work all the time.
Here is a not working case : https://github.com/Luos-io/Luos/runs/1485635670
Here is the same one working fine : https://github.com/Luos-io/Examples/runs/1485604683?check_suite_focus=true

Did someone have any idea about why this thing sometime work and sometime not ?

I’m notifcing that the non-working thing has

PLATFORMIO_LIB_EXTRA_DIRS: ~/work/Luos

and the working thing has

PLATFORMIO_LIB_EXTRA_DIRS: ~/work/Examples

?

(which is weird because your Examples repo doesn’t contain any librariy, does it?)

And the first one is also only correct of ~/work/Luos is a folder in which further library folders are, e.g. ~/work/Luos/LuosHAL

But yes, I see that there’s a general fetching libraries problem. I’ll have a look at the library.json declarations again.

Yes it’s different because it’s a build test on different repo one is on Examples and the other on Luos but they do the same thing.
The only difference is that Luos use the Luos lib on PLATFORMIO_LIB_EXTRA_DIRS and example don’t.

I should remove PLATFORMIO_LIB_EXTRA_DIRS: ~/work/Examples by the way !
(done : https://github.com/Luos-io/Examples/runs/1485770127)

I revive this topic because I still have trouble with this dependancioes that doesn’t download : https://github.com/Luos-io/Luos/runs/1934270440

I can’t find anything wrong here but it’s like Platformio doesn’t care about dependancies of the Luos lib.

It sounds like the dependancies doesn’t download when the base lib is present localy.

Can you create a minimal reproducible example separate from GitHub Actions?

Hi @manuelbl,

Yes I think. You just have to compile this project : https://drive.google.com/file/d/1OAsmMal3r4I2wQOKvr9snaPVdkmJfTc-/view?usp=sharing

At the scanning dependancy phase you should see :

Scanning dependencies...
Dependency Graph
|-- <Luos> 1.1.1
|-- <Button> 0.7.0
|   |-- <Luos> 1.1.1

Instead of :

Scanning dependencies...
Dependency Graph
|-- <Luos> 1.1.1
|   |-- <LuosHAL> 0.2.0
|-- <Button> 0.7.0
|   |-- <Luos> 1.1.1
|   |   |-- <LuosHAL> 0.2.0

The main dependency in your project are:

ProjectLuosLuosHAL

The regular setup is:

  • platformio.ini declares Luos as a dependencies
  • lib folder is empty
  • LuosHAL is detected as an indirect dependency as it’s in declared in library.json of Luos

The regular setup works.

For your test, you use a special setup:

  • The lib folder contains a local copy of Luos
  • platformio.ini declares Luos as a dependencies but is ignored as the Luos copy in lib takes precedence
  • LuosHAL is supposed to be retrieved from the library repository

This setup does not work. The reason is that library.json files within the lib folder are not used and therefore the dependency from Luos to LuosHAL is not detected. As far as I know, this is by design.

The workaround is to add luos/LuosHAL as a dependency to platformio.ini:

lib_deps = 
    Button
    luos/LuosHAL

Ok I think I get it.
So to cover all the cases of my examples I have to declare the LuosHAL lib as a dependency for my examples projects :

Scanning dependencies...
Dependency Graph
|-- <Luos> 1.1.1
|   |-- <LuosHAL> 0.2.0
|-- <LuosHAL> 0.2.0
|-- <Button> 0.7.0
|   |-- <Luos> 1.1.1
|   |   |-- <LuosHAL> 0.2.0

In this case why dependancies on Button local lib seems to be detected if I have Luos and LuosHAL both localy or on the web and Luos as dependency into platformio.ini?
Is this dependency graph built differently in this case?

In my point of view this could be an issue for people (like me) that create libs on top of other libs. If the top lib is local (because we are working on), we have to list all the dependancies of all the libs on the project platformio.ini even if we don’t know their existence. I think it could be a problem to scale the library ecosystem.

I doesn’t try to install the Luos library to see what happen in this case. Is the dependancies installed with the top lib?

Anyway it’s sound like a reasonable patch for now.

Good question. I don’t understand either why the dependency isn’t initially detected but then listed anyway. People more familiar with how the library dependency finder works have to answer it.

Another approach for your case would be to declare the dependencies like so:

lib_deps =
    file://work/Luos

I haven’t tested it. The URL might be slightly different. This way, library.json should be evaluated. Might be worth a try.

The thing is I need to make examples working in any kind of situation (with some lib local or not, on different user computers, …) So the file thing don’t cover every situation.

Another interesting thing I discover!
If I make :

lib_deps = 
    Button
    luos/Luos
    luos/LuosHAL

Platformio always download the lib and don’t care about local ($PLATFORMIO_LIB_EXTRA_DIRS) libs.

To make it work in any situation I have to do :

lib_deps = 
    Button
    Luos
    LuosHAL

It could become a problem if another organization create a library using the same name…