Difficulties understanding the right way to include local libraries

1

I’ve been trying to figure out the best practice for sharing local libraries.

Example… I am writing a communications protocol for a group of embedded devices that will all interact with each other. Multiple projects, one communications library.

I’d like this library to be stored in it’s own location, as opposed to under any of the projects.

I originally tried to manage this as a submodule, which allowed me to put a copy in the “lib” folder of each project. But there are a lot of difficulties in managing this when you’re doing development of that library from multiple locations.

PlatformIO has so many options that for this old embedded programmer it’s a little confusing. In the “old days”, all I needed was an #include to the right header and a build flag to the right directory and I was off to the races.

I have not figured out how to manage this in VS Code and Platform IO.

If anyone had a recommendation, that would be really helpful. I don’t want to go back to submodules.

Here is an example of what I’m trying to do below.
Project 1 and Project 2 are independant from each other, but share the communication protocols.
I’d like to allow both projects to reference the comms_1 and comms_2.
Referencing the .h files is easy, but getting the .c files to compile is where I’m getting stuck.

|-- Project 1
|-- |-- src
|-- Project 2
|-- |-- src
|-- Communication
|—|-- Protocol 1
|-------|-- comms_1.c
|-------|-- comms_1.h
|—|-- Protocol 2
|-------|-- comms_2.c
|-------|-- comms_2.h

I appreciate the help, trying to stick to a best practice and I’m not sure I understand enough about the platformIOs different options.

2

Please see Lib_deps creating different versions of included file? - #4 by sivar2311

3

This continues to evade my understanding.
PlatformIO continues to tell me it cannot find a library.json file, even though I know it’s in the root location properly.

If the library.json were invalid, would I get a message?
The documentation says you can use the “pio pkg install” option to validate the library,json, but finding any example of exactly how to do this is difficult.

Also, I want these libraries to recompile every time they change. Will this approach still recompile each time?

4

In your file structure shown above, there is no library.json?!

Simply create a one per library. The minmal info is name and version.
Put it into the root of every library.
Then create a src directory and move the library’s source code in that directory.

└── LibA
    ├── src
    │   ├── LibA.cpp
    │   └── LibA.h
    └── library.json

library.json:

{
  "name" : "LibA",
  "version" : "1.0.0"
}
5

I feel like I’m missing something really obvious.

My file structure:

|-- MyProject
|—|-- platformio.ini
|—|—| src
|—|—|—| <files.cpp>
|—|—| lib
|—|—|—| Protocol1
|-- Comms
|—| library.json
|—| src
|—|—| code.h
|—|—| code.cpp

I started with the Comms folder in the “lib” folder under MyProject. Everything compiled just fine.

I then moved the Comms folder to it’s own location, added the library.json file, and update the reference int he MyProject code files to the location of the header file.

The lines in MyProject’s platformio.ini:

lib_deps =
symlink:/…/Temp_Comms

The library.json file:

{
“name” : “RemoteProtocol”,
“version” : “1.0.0”
}

Now when I compile I get:
Terminal

Removing unused dependencies…
MissingPackageManifestError: Could not find one of ‘library.json, library.properties, module.json’ manifest files in the package

Output

Resolving giga_r1_m7 dependencies…

Removing unused dependencies…

MissingPackageManifestError: Could not find one of ‘library.json, library.properties, module.json’ manifest files in the package

If I then comment out the lib_deps line in the platformio, copy the Temp_Comms folder to the lib folder, and update the reference to the .h file… it compiles fine.

So the issue is the library definition, I just can’t figure out what part.

6

The correct symlink for your file structure is

lib_deps = 
  symlink://../Comms

Don’t name the source files for the comms library “code.h” and “code.cpp”. This will lead to a naming conflict if you do the same for other libraries. Use more unique names like “comms.h” and “comms.cpp”.

Then include the library in your poroject with #include <comms.h>.

File structure:

.
├── Comms
│   ├── src
│   │   ├── comms.h
│   │   └── comms.cpp
│   └── library.json
└── MyProject
    ├── src
    │   └── main.cpp
    └── platformio.ini

Comms/library.json

{
    "name": "comms",
    "version": "1.0.0"
}

lib_deps inplatformio.ini

lib_deps =
  symlink://../Comms

This symlink is using a relative path - relative to the location of the platformio.ini and and consists of three parts

  1. symlink:// ← symlink directive
  2. .. ← parent directory (one level up)
  3. /Comms ← Comms directory
7

Sivar2311 - I really, really appreciate the help and support. Thank you.

This is not working. Either I’m missing a necessary PATH reference, or some other problem… but I get the same error message every single time.

I’ve updated my project using the exact filenames and references you recommended, and PlatformIO still tells me it cannot find the library.json file.

But it’s quite clearly in the root of the Comms directory.

image
image806×290 9.17 KB

Which is exactly what you recommended:

image
image484×220 9.31 KB

8

I’ve created a repository with all necessary files which should match your file and folder structure:

You may have to edit the platformio.ini to match your platform / board etc.

9

I downloaded the repository, and I believe I’ve got things setup the way you recommend.

I created a repository and copied the project and comms folder into it.
My project is called R3D2 (I was using much more generic names in my example).

JimmyTheBook/Project_LibraryReference: This repository is for troubleshooting library reference issues.

The project as uploaded gets the error message for lacking the library.json file.

If you copy the comms folder into the lib folder of R3D2, change the #incude reference in the “./include/includes_m7.h” file to the new location in the lib folder, and comment out the lib_deps line in platformio.ini… the project builds just fine.

10

After changing this line

to

#include <comms.h>

your project compiles without any issues:

11

We’re missing something else… because this doesn’t solve the problem for me.
I changed the line to use <comms.h>, and get the exact same error.

(sorry, I don’t know how to do a record of the window, but here is a screenshot of the VS Code window after the change, with the same compile error.)

Is there a PATH, or some other setting, that I’m missing?
I updated the repository as well, but the only change was that single line.

image
image2673×1310 197 KB

12

Do you have the same error on a brand new and clean project?

I’ve got the feeling that this error does not come form the library but from a broken platform / package!

13

Yes, you’re very quick!

I was just writing another response… I was traveling but just got home today. I pulled the repository onto my desktop, and it compiles just fine.

So first… thank you very much for all you help!

This afternoon I will start to troubleshoot why my laptop environment isn’t working properly.

1 Like
14
  • Close VS Code
  • Delete C:\Users\jimpo\.platformio\packages and C:\Users\jimpo\.platformio\platforms
  • Restart VS Code
  • Open your project and wait until PlatformIO reinstalled the necessary files (platform and packages)
15

I deleted the folders and restarted VS Code, which then attempted to reload the packages and platforms.

There was a “MissingPackageManifest” error at the end of the reload (see below).

And I’m still getting the same error message.

image
image2628×1304 481 KB

16

Repeat deleting the folders but also delete the .cache folder.

17

Still didn’t help. Got the same error message at the end of the reinstall process… (see below)

I leave again shortly, so it seems like my next move may need to be uninstall and reinstall VS Code. I need this to work from my laptop!

And again, thank you so much for the help!

image
image1579×1313 386 KB

18

I’ve resolved the issue! If you want the step by step details, feel free to read. Or jump to the bottom for the summary…

I started digging into the library issue from the PlatformIO pages. The call message shows up twice (once for the repository I sent to you, and once for the original project that wasn’t working). And no matter how long I wait for those spinning wheels to stop… they don’t stop.

image
image2097×860 102 KB

I deleted the platformio.ini files from both folders.
I then went into the “Project” interface and deleted the platformio-ini config files as they showed up in the PlatformIO screen.

Once I did those two things… the error messages stopped.

Next I commented out the reference to <comms.h> from the include file.
I deleted the three folders again (.cache, .platforms, .pakages).

I restarted VS Code, opened the project (which has no platformio.ini file) and accessed the library screen in the PlatformIO tool. I no longer recieved the error message.

Next, I rebuilt the platformio.ini file WITHOUT the lib_deps message. So there should have been nothing to attempt to load the comms library.

But I started to receive the error again.

Then I noticed a new folder in the .pio folder… .pio/libdeps/giga_r1_m7/ that had a file in it called Libraries.pio-link (this is in the repository that we are sharing).

When I open that file, there is only one line…

{“cwd”: “C:\Software_Projects\R3D2”, “spec”: {“owner”: null, “id”: null, “name”: “Libraries”, “requirements”: null, “uri”: “symlink://…/Libraries”}

The reference to “symlink://…/Libraries” was an old option I tried a while back when I was attempting to get all of this to work. I had added the symlink to the platformio.ini file when the Comms folder was in a subfolder called Libraries. This reference hasn’t been tried or used since before I posted this issue.

I deleted the entire libdeps directory, and restarted VS Code. And it worked!

Summary… there is a file created in /.pio/libdeps/ when you attempt to create a library link that was not deleted. Not sure what should trigger this deletion, but it wasn’t happening. Everytime I tried to load or build the project, it searched for this library that doesn’t exist. And because of this error, the library page in PlatformIO froze and wouldn’t show me the reference to the library to realize what it was trying to load.

1 Like
19

There seems there is a bug when it comes to clean up the .pio/libdeps folder.

If the lib_deps section is changed and no library is listed anymore, the remaining files/folders in the libdeps folder are not automatically deleted.

I could also observe that other dependencies were listed via symlink whose source had already been deleted. Unfortunately I was not able to reproduce this for the video.

I’ve created an issue for that: