PlatformIO Community

How to generate and use pre-compiled objects

I am currently in a project that will be developed in a distributed way in the future. For administration gitlab should be used, but not all source codes should be published in the parent project.
Now the idea is that only a part of the source codes will be published. But for this a part would have to be made available as precompiled objects, or so…

My question: Has anyone done something similar? Or is there a tutorial for this. (I already found something here and there, but it didn’t bring me to the goal yet).
Currently I am working on how to precompile source codes and how to reuse the objects.

I am looking forward to your answers!
Thanks a lot in advance!

Should be easily possible. I’ll write a quick tutorial here that should cover all cases. I’ll also assume 0 before-knowledge. And show how to make it maximally comfortable for PlatformIO users.

Creating and using precompiled libraries in PlatformIO projects

Creating the library project

I’ll be using a project for an Arduino Uno in the Atmel-AVR platform as example. This information applies in general to all other platforms too, like e.g. ARM based ones. Just the toolchain changes, from e.g. avr-gcc to arm-none-eabi-gcc and friends.

Let’s first inspect the PlatformIO build prcess, and what it generates and how we can use or build upon what is generated.

Project setup

We’ll write a short calculator application. The platformio.ini of the project is standard

[env:uno]
platform = atmelavr
board = uno
framework = arduino

Adding library code and firmware test code

We’ll write a calculator function library. I create a new a folder in lib/ and put the files Calculator.hpp and Calculator.cpp in it. (The process is identical if writing a C library instead of C++).

The contents are

Calculator.hpp

#include <stdint.h>

class Calculator {
public:
    /* a + b */
    static int Add(int a, int b);
    /* a * b */
    static int Mul(int a, int b);
    /* a * b + c */
    static int MultiplyAccumulate(int a, int b, int c);
};

and Calculator.cpp

#include "Calculator.hpp"

int Calculator::Add(int a, int b) {
    return a + b;
}
    
int Calculator::Mul(int a, int b) {
    return a * b;
}

int Calculator::MultiplyAccumulate(int a, int b, int c) {
    return (a * b) + c;
}

further a src\main.cpp is added which calls these functions.

#include <Arduino.h>
#include <Calculator.hpp>

void setup() {
  Serial.begin(9600);
  Serial.println("2 + 3 = " + String(Calculator::Add(2,3)));
  Serial.println("2 * 3 + 4 = " + String(Calculator::MultiplyAccumulate(2,3,4)));
}

void loop() {
}

As an additional, we also write a piece of code in the src/ folder which we would like to re-use as a library, this time as C code. The file src\LibraryInfo.c has content

const char* GetLibraryAuthor() {
    return "Max Gerhardt";
}

int GetLibraryVersion() {
    return 10;
}

with the header file src\LibraryInfo.h being

#ifndef LIBRARYINFO_H
#define LIBRARYINFO_H

#ifdef __cplusplus
extern "C" {
#endif

const char* GetLibraryAuthor();
int GetLibraryVersion();

#ifdef __cplusplus
}
#endif

#endif

The header file is standard also contains the right content to directly use it from C++ code – since the underlying library code is written in C, we have to declare all functions as extern "C" when importing it in C++ code, due to name mangling.

Building the library project

Let’s now build the project.

Processing uno (platform: atmelavr; board: uno; framework: arduino)
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------Verbose mode can be enabled via `-v, --verbose` option
CONFIGURATION: https://docs.platformio.org/page/boards/atmelavr/uno.html
PLATFORM: Atmel AVR (3.0.0) > Arduino Uno
HARDWARE: ATMEGA328P 16MHz, 2KB RAM, 31.50KB Flash
DEBUG: Current (avr-stub) On-board (avr-stub, simavr)
PACKAGES:
 - framework-arduino-avr 5.1.0
 - toolchain-atmelavr 1.50400.190710 (5.4.0)
LDF: Library Dependency Finder -> http://bit.ly/configure-pio-ldf
LDF Modes: Finder ~ chain, Compatibility ~ soft
Found 11 compatible libraries
Scanning dependencies...
Dependency Graph
|-- <Calculator>
Building in release mode
Compiling .pio\build\uno\src\LibraryInfo.c.o
Compiling .pio\build\uno\src\main.cpp.o
Compiling .pio\build\uno\lib952\Calculator\Calculator.cpp.o
Archiving .pio\build\uno\libFrameworkArduinoVariant.a
Compiling .pio\build\uno\FrameworkArduino\CDC.cpp.o
Compiling .pio\build\uno\FrameworkArduino\HardwareSerial.cpp.o
Compiling .pio\build\uno\FrameworkArduino\HardwareSerial0.cpp.o
Compiling .pio\build\uno\FrameworkArduino\HardwareSerial1.cpp.o
Compiling .pio\build\uno\FrameworkArduino\HardwareSerial2.cpp.o
Compiling .pio\build\uno\FrameworkArduino\HardwareSerial3.cpp.o
Compiling .pio\build\uno\FrameworkArduino\IPAddress.cpp.o
Compiling .pio\build\uno\FrameworkArduino\PluggableUSB.cpp.o
Compiling .pio\build\uno\FrameworkArduino\Print.cpp.o
Indexing .pio\build\uno\libFrameworkArduinoVariant.a
Compiling .pio\build\uno\FrameworkArduino\Stream.cpp.o
Compiling .pio\build\uno\FrameworkArduino\Tone.cpp.o
Compiling .pio\build\uno\FrameworkArduino\USBCore.cpp.o
Compiling .pio\build\uno\FrameworkArduino\WInterrupts.c.o
Archiving .pio\build\uno\lib952\libCalculator.a
Compiling .pio\build\uno\FrameworkArduino\WMath.cpp.o
Compiling .pio\build\uno\FrameworkArduino\WString.cpp.o
Compiling .pio\build\uno\FrameworkArduino\abi.cpp.o
Compiling .pio\build\uno\FrameworkArduino\hooks.c.o
Indexing .pio\build\uno\lib952\libCalculator.a
Compiling .pio\build\uno\FrameworkArduino\main.cpp.o
Compiling .pio\build\uno\FrameworkArduino\new.cpp.o
Compiling .pio\build\uno\FrameworkArduino\wiring.c.o
Compiling .pio\build\uno\FrameworkArduino\wiring_analog.c.o
Compiling .pio\build\uno\FrameworkArduino\wiring_digital.c.o
Compiling .pio\build\uno\FrameworkArduino\wiring_pulse.S.o
Compiling .pio\build\uno\FrameworkArduino\wiring_pulse.c.o
Compiling .pio\build\uno\FrameworkArduino\wiring_shift.c.o
Archiving .pio\build\uno\libFrameworkArduino.a
Indexing .pio\build\uno\libFrameworkArduino.a
Linking .pio\build\uno\firmware.elf
Checking size .pio\build\uno\firmware.elf
Building .pio\build\uno\firmware.hex
Advanced Memory Usage is available via "PlatformIO Home > Project Inspect"
RAM:   [=         ]  10.7% (used 220 bytes from 2048 bytes)
Flash: [=         ]  10.5% (used 3380 bytes from 32256 bytes)

Understanding the build process

An there’s already a few things that we can look at. First of all, we can see exactly how each of our c or cpp files (or the files of the Arduino framework) are being compiled into object files. This holds the compiled, binary version of the source code, specific for the configuration (platform: Atmel AVR, CPU: ATMega328P) we’re compiling for.

Compiling .pio\build\uno\src\LibraryInfo.c.o
Compiling .pio\build\uno\src\main.cpp.o
Compiling .pio\build\uno\lib952\Calculator\Calculator.cpp.o

Compiling is done, as one can see when using the “Advanced → Verbose Build” task, by running the compiler, avr-g++ or avr-gcc in this case, on the source code file with special flags (most importantly -c to generate object file without linking) to generate an output file (designated via the -o <output file> flag). Example:

avr-gcc -o .pio\build\uno\src\LibraryInfo.c.o -c -std=gnu11 -fno-fat-lto-objects -Os -Wall -ffunction-sections -fdata-sections -flto -mmcu=atmega328p -DPLATFORMIO=50101 -DARDUINO_AVR_UNO -DF_CPU=16000000L -DARDUINO_ARCH_AVR -DARDUINO=10808 -Iinclude -Isrc -Ilib\Calculator -IC:\Users\Max.platformio\packages\framework-arduino-avr\cores\arduino -IC:\Users\Max.platformio\packages\framework-arduino-avr\variants\standard src\LibraryInfo.c

Will generate the object file .pio\build\uno\src\LibraryInfo.c.o based on src\LibraryInfo.c with various options, defines (-D) and include paths (-I) given for compilation.

Also, some archives are created, in some cases. Archives are just collection of object files – think of them .zip files containing multiple .o files. For the Arduino framework, for the variant folder (that is seperate) and all libraries, archives are created. The source code in our main src/folder is however not converted to an archive. Example:

Archiving .pio\build\uno\lib952\libCalculator.a

Which is turn implemented by

avr-gcc-ar rc .pio\build\uno\lib952\libCalculator.a .pio\build\uno\lib952\Calculator\Calculator.cpp.o

Here the avr-gcc-ar tool is use to create the archive file libCalculator.a based on one or multiple object files, in this case only Calculator.cpp.o. See documentation for this tool.

Also, the step

Indexing .pio\build\uno\lib952\libCalculator.a

is done through

avr-gcc-ranlib .pio\build\uno\lib952\libCalculator.a

Which is the process of

[generating] an index to the contents of an archive and stores it in the archive.
An archive with such an index speeds up linking to the library and allows routines in the library to call each other without regard to their placement in the archive.

(These are technical details that are not super important, just as a side info that that is done).

As a last detail, we will look at the final build step: The linking. Linking links all previously compiled object files (or archive files) together to create the firmware image (an ELF file), from which then a pure binary image is produced that gets loaded into flash (the .bin file).

Linking is done in this case through

avr-g++ -o .pio\build\uno\firmware.elf -Os -mmcu=atmega328p -Wl,–gc-sections -flto -fuse-linker-plugin .pio\build\uno\src\LibraryInfo.c.o .pio\build\uno\src\main.cpp.o -L.pio\build\uno -Wl,–start-group .pio\build\uno\lib952\libCalculator.a .pio\build\uno\libFrameworkArduinoVariant.a .pio\build\uno\libFrameworkArduino.a -lm -Wl,–end-group

What we can see here is that firmware.elf will be generated, with some options regarding linker options or target MCU, but most importantly that is uses the files LibraryInfo.c.o, libCalculator.a, libFrameworkArduinoVariant.a and libFrameworkArduino.a as primary sources. There are also -l<name> flags in the command which also link against a library, in this case -lm for “lib math”, a compiler-builtin library.

For GCC, when linking, it needs to know which files you want to link and where they are. Most importantly per docs.

  • -l library or -llibrary will tell the linker that you want to link against a library called library. GCC will search for the file liblibrary.a (aka, lib prefix is implicit). So if I had downloaded a libSensor.a library from the internet and I’d like to use it, I’d use the flag -lSensor to refer to it.
  • -Ldir: Adds a foler to the library search path. In PlatformIO, paths are relative to the project folder but can also be given as absolute. If you’d e.g. have a library file downloaded at C:\MyLibrary\libSensor.a, you can enable the linker to find the file you mean (when you refer to it via -lSensor) by giving it the search path -L "C:\MyLibrary\". In the case above you can see, that .pio\build\uno is aded to the library search path.

Inspecting build artefacts

So, after compilation, all build artifacts of the project is saved in .pio\build\<environment name>. Let’s have a look.

grafik

The top leel folder contains the .elf file and the two archive files for the Arduino framework.

The lib952 folder is our calculator library.

grafik

The subfolder contains the pure object file.

grafik

The src/ folder from above contains the two object files for main.cpp an our other library code.

grafik

Testing the library code

Now when we upload the firmware and look at the output it ofc works.

— Miniterm on COM14 9600,8,N,1 —
— Quit: Ctrl+C | Menu: Ctrl+T | Help: Ctrl+T followed by Ctrl+H —
2 + 3 = 5
2 * 3 + 4 = 10

(See code above to verify).

That completes the build of the library, one in a lib/ folder an one bit of code in the src/ folder.

Creating a project using the library

Let’s now create a new project where we wish to use the 2 libraries we have created previously. A new project atmel_lib_user with initially the same platformio.ini as above is created.

Let’s first write a new firmware source code which uses both libraries. Let src\main.cpp contain.

#include <Arduino.h>
#include <Calculator.hpp>
#include <LibraryInfo.h>

void setup() {
  Serial.begin(9600);
  Serial.println("5 + 1 = " + String(Calculator::Add(5,1)));
  Serial.println("5 * 2 + 1 = " + String(Calculator::MultiplyAccumulate(5,2,1)));

  Serial.println("Library written by \"" + String(GetLibraryAuthor()) + "\"");
  Serial.println("Library version " + String(GetLibraryVersion()));
}

void loop() {
}

Adding library header files

Now of course with just that code, build will fail since the project doesn’t contain the header files yet. So, we add both of them into the include foler now. (Note: Adding to src/ is also possible. We will also take a look at creating a dedicated library folder later).

With the 2 header files copied, there’s still the errors

C:\Users\Max\AppData\Local\Temp\ccXbzREZ.ltrans0.ltrans.o: In function `main':
<artificial>:(.text.startup+0x10c): undefined reference to `Calculator::Add(int, int)'
<artificial>:(.text.startup+0x154): undefined reference to `Calculator::MultiplyAccumulate(int, int, int)'
<artificial>:(.text.startup+0x190): undefined reference to `GetLibraryAuthor'
<artificial>:(.text.startup+0x202): undefined reference to `GetLibraryVersion'
collect2.exe: error: ld returned 1 exit status

of course. We’re missing the actual implementation / object file code that implements these functions. So how do we do that?

Adding library object files

For this, we can use both archive files (.a) and object files (.o) from our previous library build. (Note: It is more common to only have one .a file for a precompiled library – we will take a look at creating our own .a file later).

So, we copy the libCalculator.a and LibraryInfo.c.o somwhere in our project directory. I choose to create a new folder called precompiled_lib where I put these two files in, as an example. The total folder structure is

grafik

Now, rebuilding the library will ofc result in the same error. We have to tell the linker to actually use these files. Therefore I use a build_flags directive in the platformio.ini with the appropiate linker flags, as discussed previously.

build_flags =
    -L precompiled_lib
    -lcalculator
    -Wl,precompiled_lib/LibraryInfo.c.o

The -Wl,precompiled_lib/LibraryInfo.c.o is a little quirk. We have to add the object file as a pure argument to the linker so that it understands to in that file. -Wl, is prefix to say that the next thing is linker option (rather than a build flag option that is appended during every object file compilation – which we don’t want, we only want this during the final link). The -L and -l options are standard an as previously discussed – they’re usually the norm when dealing with precompiled libraries.

When we now press “Build”,

Linking .pio\build\uno\firmware.elf
Building .pio\build\uno\firmware.hex
Checking size .pio\build\uno\firmware.elf
Advanced Memory Usage is available via "PlatformIO Home > Project Inspect"
RAM:   [=         ]  13.2% (used 270 bytes from 2048 bytes)
Flash: [=         ]  11.2% (used 3608 bytes from 32256 bytes)
=========================== [SUCCESS] Took 2.08 seconds ===========================

Our firmware links successfully now.

Testing the precompiled library project

Uploading and monitoring gives

— Miniterm on COM14 9600,8,N,1 —
— Quit: Ctrl+C | Menu: Ctrl+T | Help: Ctrl+T followed by Ctrl+H —
5 + 1 = 6
5 * 2 + 1 = 11
Library written by “Max Gerhardt”
Library version 10

So we have successfully created a project with some library code, compiled, and used the resulting build artefacts in a new project, which we only gave the header files with the function declarations and the object or archive files, not the original source code.

So, they way I would recommend doing this when writing a library that’s supposed to be distributed and included as a precompiled static library is to just put the library code in a new folder in lib/ in some project, then it’ll be automatically compiled to a .a archive file which can be use with only two build flag additions very easily.

However, I’ll take a short moment to talk about some extended stuff.

Creating own archive files

One can create .a files from arbitrary .o files at any point. The commands and documentation were already shown above for avr-gcc-ar and avr-gcc-ranlib.

For example, we can turn our LibraryInfo.c.o file into an archive file. For that, I’ll open a command prompt. I’ll use the same avr-gcc tools that PlatformIO uses, which are locate in C:\Users\<user>\.platformio\packages\toolchain-atmelavr\bin in this case for Windows and the Atmel AVR case.

To create a new archive with only the LibraryInfo.c.o in it, I run (relative to project directory in the library project)

C:\Users\Max\.platformio\packages\toolchain-atmelavr\bin\avr-gcc-ar rc libLibraryInfo.a .\.pio\build\uno\src\LibraryInfo.c.o

Whereas “rc” means "add member to archive while replacing if needed, create archive. (See previous doc link). Next I’ll index the archive.

C:\Users\Max\.platformio\packages\toolchain-atmelavr\bin\avr-gcc-ranlib libLibraryInfo.a

I’ll then go into the library user project and remove the precompiled_lib\LibraryInfo.c.o file but put the libLibraryInfo.a file in the folder instead.

I then change the build_flags to

build_flags =
    -L precompiled_lib
    -lcalculator
    -lLibraryInfo

And the library links

Linking .pio\build\uno\firmware.elf
Checking size .pio\build\uno\firmware.elf
Building .pio\build\uno\firmware.hex
Advanced Memory Usage is available via "PlatformIO Home > Project Inspect"
RAM:   [=         ]  13.2% (used 270 bytes from 2048 bytes)
Flash: [=         ]  11.2% (used 3608 bytes from 32256 bytes)
=========================== [SUCCESS] Took 2.16 seconds ===========================

and works the same.

— Miniterm on COM14 9600,8,N,1 —
— Quit: Ctrl+C | Menu: Ctrl+T | Help: Ctrl+T followed by Ctrl+H —
5 + 1 = 6
5 * 2 + 1 = 11
Library written by “Max Gerhardt”
Library version 10

Thus you can use that to group arbitrary object files together and use that.

Creating PlatformIO libraries using a precompiled library

The next comfort level step is creating a library which can be used directly in a PlatformIO project, without having to adjust any build flags whatsoever. For that, we can create a library folder with our code, header, and a library.json information file, which will take care of library-specific build instructions.

Let us first create on big unified .a library file, with one .h and the library.json, in a folder UltraLibrary.

As a header, we simply merge everything into UltraLibrary.h

#ifndef ULTRALIBRARY_H
#define ULTRALIBRARY_H

#include <stdint.h>

class Calculator {
public:
    /* a + b */
    static int Add(int a, int b);
    /* a * b */
    static int Mul(int a, int b);
    /* a * b + c */
    static int MultiplyAccumulate(int a, int b, int c);
};

#ifdef __cplusplus
extern "C" {
#endif

const char* GetLibraryAuthor();
int GetLibraryVersion();

#ifdef __cplusplus
}
#endif

#endif

We’ll create a unified archive file from both the calculator code and the LibraryInfo code. I’ll simply use the two object files directly

C:\Users\Max\.platformio\packages\toolchain-atmelavr\bin\avr-gcc-ar rc libUltraLibrary.a .\.pio\build\uno\src\LibraryInfo.c.o .\.pio\build\uno\lib952\Calculator\Calculator.cpp.o

C:\Users\Max\.platformio\packages\toolchain-atmelavr\bin\avr-gcc-ranlib .\libUltraLibrary.a

Now I have one header file and one .a file with all the functions.

(Yes, if I had just set up the library project such that my library code was all in the lib/ folder there, I would have automatically gotten the .a file which I want to use. I’m just trying to show a more complicated case for general usage here. It’s of course also okay to use multiple include files in the library, and not unify them. Again, just for demonstration purposes).

However, sticking bot the .h file and the .a file in the UltraLibrary folder will not make it work directly – we have to create the library.json metainformation file, too. Most importantly, the build flags and compatibility information for building. See linked docs for full information.

I decide on the file

{
    "name": "UltraLibrary",
    "authors": {
        "name": "Max Gerhardt"
    },
    "version": "1.0.0",
    "platforms": [
        "atmelavr"
    ],
    "frameworks": "*",
    "build": {
        "flags": [
            "-L.",
            "-lUltraLibrary"
        ]
    }
}

Notes:

  • platforms is set to atmelavr (see platform = .. in the original library project) because that is the microarchitecture / platform that the precompiled code was built for
  • frameworks is set to “all” (*) because the library does not use any Arduino specific code – theoritically it could be used anywhere. If I was using Arduino specific functions in there, I’d have to declared it as "arduino".
  • the build flags add the library folder itself to the library search path ("." refers the current directory, the library folder in this case) and the -l flag so that libUltraLibrary.a will be linked

Now the user project looks like

grafik

(The UltraLibrary folder can ofc be used anywhere else to and is not specific to the user project).

Now I can change the firmware test code to

#include <Arduino.h>
#include <UltraLibrary.h>

void setup() {
  Serial.begin(9600);
  Serial.println("5 + 1 = " + String(Calculator::Add(5,1)));
  Serial.println("5 * 2 + 1 = " + String(Calculator::MultiplyAccumulate(5,2,1)));

  Serial.println("Library written by \"" + String(GetLibraryAuthor()) + "\"");
  Serial.println("Library version " + String(GetLibraryVersion()));
}

void loop() {
}

And the, the platformio.ini can be absolute standard.

[env:uno]
platform = atmelavr
board = uno
framework = arduino

without build flags. I don’t even have to delcare lib_deps = UltraLibrary in this case because PlatformIO will pick up the usage of the folder in lib/ automatically. If the folder weren’t there but a user would like to use a published library, this is where the library declaration like lib_deps = someOwner/someLibrary@^1.0.0 would go.

The firmware links and uploads and executes as normal.

Processing uno (platform: atmelavr; board: uno; framework: arduino)
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------Verbose mode can be enabled via `-v, --verbose` option
CONFIGURATION: https://docs.platformio.org/page/boards/atmelavr/uno.html
PLATFORM: Atmel AVR (3.0.0) > Arduino Uno
HARDWARE: ATMEGA328P 16MHz, 2KB RAM, 31.50KB Flash   
DEBUG: Current (avr-stub) On-board (avr-stub, simavr)
PACKAGES:
 - framework-arduino-avr 5.1.0
 - tool-avrdude 1.60300.200527 (6.3.0)
 - toolchain-atmelavr 1.50400.190710 (5.4.0)
LDF: Library Dependency Finder -> http://bit.ly/configure-pio-ldf
LDF Modes: Finder ~ chain, Compatibility ~ soft
Found 11 compatible libraries
Scanning dependencies...
Dependency Graph        
|-- <UltraLibrary> 1.0.0
Building in release mode
Compiling .pio\build\uno\src\main.cpp.o
...
Linking .pio\build\uno\firmware.elf
Building .pio\build\uno\firmware.hex
Checking size .pio\build\uno\firmware.elf
Advanced Memory Usage is available via "PlatformIO Home > Project Inspect"
RAM:   [=         ]  13.2% (used 270 bytes from 2048 bytes)
Flash: [=         ]  11.2% (used 3608 bytes from 32256 bytes)
Configuring upload protocol...
..
--- Miniterm on COM14  9600,8,N,1 ---
--- Quit: Ctrl+C | Menu: Ctrl+T | Help: Ctrl+T followed by Ctrl+H ---
5 + 1 = 6
5 * 2 + 1 = 11
Library written by "Max Gerhardt"
Library version 10

And that’s now created a completely unified library folder with one big precompiled library code blob, one big header file which declares all the functions in there, an the library.json needed to automatically build and link this library correctly, without the user having to adjust the build_flags of the project himself for linkage.

The library folder would also be ready for distribution in the PlatformIO library registers with pio package publish <path to UltraLibrary folder>. (Docs).

Automated Build, CI, Testing (is not covered)

The final automation and comfortness level step would be ofc automated building in e.g. Github or Gitlab, where you commit or modify the library code, and out pops the .a file or better, the complete library folder (or zip archive thereof) containing the header file, library file and library.json file, ready to be used directly.

This is not covered here as I don’t have much experience with it, but PlatformIO has good docs for it. A lot in these systems is scriptable, with shell scripts or python scripts, so really anything can be done there to create such a library folder or file for an arbitrary library project structure.

As said, the easiest way is to let PlatformIO handle the most part – the path of

  1. putting the to-be-distributed code in a folder in lib/ (so that the .a file is generated automatically),
  2. starting a normal build (or CI build),
  3. then creating a new folder where all original header files, plus the generated .a files and a premade library.json is copied into

should be most comfortable.

After that, one can think about taking the created library folder (or zip file) and running it through further CI test projects that use this library. The sky is the limit here.

2 Likes

Thank you so much for the great step by step tutorial.
I tried a few things with it today and was already able to create my own precompiled library with corresponding .json description. The whole thing worked great after a few loops.
I think your idea for the PlatformIO release is great. It makes the whole workflow the easiest. Unfortunately I could not upload my library to PlatformIO yet. I think the problem sits as always in front of the computer :slight_smile: I haven’t quite understood the steps of what to do after zipping the folder with the .a, .h and .json files. I will deal with that tomorrow. Do you have another tip for me?
Thanks again!

Roughly, you need to open a CLI, then use pio account commands (like pio account login, pio account show) to login & verify. After that, pio package publish <path to library folder / .zip / .tar.gz> will upload your library and submit it to review.

>pio account login
Username or email: maximilian.gerhardt@myemailprovider.de
Password: <..>
Successfully logged in!
>pio package publish C:\Users\Max\Documents\PlatformIO\Projects\atmel_lib_user\lib\UltraLibrary
..

After that the PlatformIO team will review it and send email notifications. Then, the library should be globally visible in https://platformio.org/lib with the name you gave it in the JSON file. You can ofc also pio package unpublish a library if you don’t like it, or publish a new version using the same command.

Zipping is just a step for easier single-file distribution and not necessary if you just want to develop and publish a library.

The thought here is just that if you have a project where the library is created and out pops the library folder (with .h and .a and .json file), you can use locally it on another project by:

  • Copying it into the lib/ folder of the other project
  • Specifying the folder via lib_deps, e.g.
lib_deps = 
    file://C:\Users\user\Documents\MyBuiltLibrary

(also see these docs for allowed expressions in lib deps).

  • Specifying the release zip via lib_deps, as e.g.
lib_deps = 
   file://C:\Development\MyBuiltLibrary.zip

Once a finished library is released and in the PlatformIO library registry, users will just have to add

lib_deps = 
   owner/libraryname@^1.0.0

to the platformio.ini (or the GUI library manager) to use the library, in some abritrary version. (When a library is published, you will receive a link to where it was published, and it will automatically contain these inclusion instructions; example)

So a zipped folder or not doesn’t make a difference here. Of course, ZIP folders are the standard Arduino library distributation format. If you want to make your library also Arduino-IDE compatible, the library folder also needs to contain a library.properties file in the documented Arduino format. It does also support declaring the usage of a precompiled library – but if you don’t provide the library.properties file, the Arduino IDE won’t know how to build it correctly. The Bosch BSEC library has a good and simple example of this file. In PlatformIO, the library.json will always be used preferrably.

I greatly appreciate the amount of time and effort you have put into this tutorial.

Although I have more than ten years experience using the Arduino IDE and regularly use .a libraries, I am a complete Noob with VSCode/PlatformIO (VP). For the last couple of weeks, I have been happily compiling and running VP with great success. But working through your tutorial I am having troubles. I imagine either something has changed since you wrote this or I do not have my environment setup correctly. I recently finished another thread here and found one important take-away was that my Windows installation doesn’t always behave the same as a Linux install.

I first tried to do my own ported project using your guide and ran into troubles. So, I backed up and followed each of your steps using your exact code samples. I ran into the same troubles. I only get down to Building the library project. I’ve put copies of your files in the designated folders. The only difference I noted was the lib/ folder was already created during the project creation step. Here is my IDE at this step showing the layout and one of your files.
image

Doing a verbose compile, I get…

> Executing task: C:\Users\Dennis\.platformio\penv\Scripts\platformio.exe run --verbose --environment uno <

Processing uno (platform: atmelavr; board: uno; framework: arduino)
----------------------------------------------------------------------------------------
CONFIGURATION: https://docs.platformio.org/page/boards/atmelavr/uno.html
PLATFORM: Atmel AVR (3.4.0) > Arduino Uno
HARDWARE: ATMEGA328P 16MHz, 2KB RAM, 31.50KB Flash
DEBUG: Current (avr-stub) On-board (avr-stub, simavr)
PACKAGES:
 - framework-arduino-avr 5.1.0
 - toolchain-atmelavr 1.70300.191015 (7.3.0)
LDF: Library Dependency Finder -> https://bit.ly/configure-pio-ldf
LDF Modes: Finder ~ chain, Compatibility ~ soft
Found 5 compatible libraries
Scanning dependencies...
No dependencies
Building in release mode
avr-g++ -o .pio\build\uno\src\main.cpp.o -c -fno-exceptions -fno-threadsafe-statics -fpermissive -std=gnu++11 -Os -Wall -ffunction-sections -fdata-sections -flto -mmcu=atmega328p -DPLATFORMIO=50203 -DARDUINO_AVR_UNO -DF_CPU=16000000L -DARDUINO_ARCH_AVR -DARDUINO=10808 -Iinclude -Isrc -IC:\Users\Dennis\.platformio\packages\framework-arduino-avr\cores\arduino -IC:\Users\Dennis\.platformio\packages\framework-arduino-avr\variants\standard src\main.cpp
src\main.cpp:2:10: fatal error: Calculator.hpp: No such file or directory

**********************************************************************
* Looking for Calculator.hpp dependency? Check our library registry!
*
* CLI  > platformio lib search "header:Calculator.hpp"
* Web  > https://platformio.org/lib/search?query=header:Calculator.hpp
*
**********************************************************************

 #include <Calculator.hpp>
          ^~~~~~~~~~~~~~~~
compilation terminated.
avr-gcc -o .pio\build\uno\FrameworkArduino\wiring_analog.c.o -c -std=gnu11 -fno-fat-lto-objects -Os -Wall -ffunction-sections -fdata-sections -flto -mmcu=atmega328p -DPLATFORMIO=50203 -DARDUINO_AVR_UNO -DF_CPU=16000000L -DARDUINO_ARCH_AVR -DARDUINO=10808 -IC:\Users\Dennis\.platformio\packages\framework-arduino-avr\cores\arduino -IC:\Users\Dennis\.platformio\packages\framework-arduino-avr\variants\standard C:\Users\Dennis\.platformio\packages\framework-arduino-avr\cores\arduino\wiring_analog.c
avr-gcc -o .pio\build\uno\FrameworkArduino\wiring_digital.c.o -c -std=gnu11 -fno-fat-lto-objects -Os -Wall -ffunction-sections -fdata-sections -flto -mmcu=atmega328p -DPLATFORMIO=50203 -DARDUINO_AVR_UNO -DF_CPU=16000000L -DARDUINO_ARCH_AVR -DARDUINO=10808 -IC:\Users\Dennis\.platformio\packages\framework-arduino-avr\cores\arduino -IC:\Users\Dennis\.platformio\packages\framework-arduino-avr\variants\standard C:\Users\Dennis\.platformio\packages\framework-arduino-avr\cores\arduino\wiring_digital.c
avr-gcc -x assembler-with-cpp -Os -Wall -ffunction-sections -fdata-sections -flto -mmcu=atmega328p -DPLATFORMIO=50203 -DARDUINO_AVR_UNO -DF_CPU=16000000L -DARDUINO_ARCH_AVR -DARDUINO=10808 -IC:\Users\Dennis\.platformio\packages\framework-arduino-avr\cores\arduino
-IC:\Users\Dennis\.platformio\packages\framework-arduino-avr\variants\standard -c -o .pio\build\uno\FrameworkArduino\wiring_pulse.S.o C:\Users\Dennis\.platformio\packages\framework-arduino-avr\cores\arduino\wiring_pulse.S
avr-gcc -o .pio\build\uno\FrameworkArduino\wiring_pulse.c.o -c -std=gnu11 -fno-fat-lto-objects -Os -Wall -ffunction-sections -fdata-sections -flto -mmcu=atmega328p -DPLATFORMIO=50203 -DARDUINO_AVR_UNO -DF_CPU=16000000L -DARDUINO_ARCH_AVR -DARDUINO=10808 -IC:\U***
[.pio\build\uno\src\main.cpp.o] Error 1
sers\Dennis\.platformio\packages\framework-arduino-avr\cores\arduino -IC:\Users\Dennis\.platformio\packages\framework-arduino-avr\variants\standard C:\Users\Dennis\.platformio\packages\framework-arduino-avr\cores\arduino\wiring_pulse.c
avr-gcc -o .pio\build\uno\FrameworkArduino\wiring_shift.c.o -c -std=gnu11 -fno-fat-lto-objects -Os -Wall -ffunction-sections -fdata-sections -flto -mmcu=atmega328p -DPLATFORMIO=50203 -DARDUINO_AVR_UNO -DF_CPU=16000000L -DARDUINO_ARCH_AVR -DARDUINO=10808 -IC:\Users\Dennis\.platformio\packages\framework-arduino-avr\cores\arduino -IC:\Users\Dennis\.platformio\packages\framework-arduino-avr\variants\standard C:\Users\Dennis\.platformio\packages\framework-arduino-avr\cores\arduino\wiring_shift.c
============================== [FAILED] Took 1.00 seconds ==============================
The terminal process "C:\Users\Dennis\.platformio\penv\Scripts\platformio.exe 'run', '--verbose', '--environment', 'uno'" terminated with exit code: 1.

Terminal will be reused by tasks, press any key to close it.

It does not seem to recognize the lib folder even though VP created it. So, I tried moving the two lib files into the src/ folder and it successfully compiled.

This seems to defeat the logic that creates a *.a library as inspecting the output, I find the *.o files, but not an *.a file. However, I wasn’t totally clear if the build process was supposed to automatically create the *.a file or was I suppose to explicitly run the command:
avr-gcc-ar rc .pio\build\uno\lib952\libCalculator.a .pio\build\uno\lib952\Calculator\Calculator.cpp.o

That’s where I stopped in your tutorial. I hope it is simply a matter of some erroneous setting in my environment. Do you have any advice to get me over this stumble?

Thank you for your time.

Wrong filestructure. A library must be a folder inside the lib folder (refer README). Meaning you should do

|- lib
   |- Calculator
      |- Calculator.cpp
      |- Calculator.hpp

Restructure it like that and do a Ctrl+Shift+P → Rebuild IntelIiSense and it should all work.

Outstanding! That was it. This is very similar to ArduinoIDE’s layout and I should have tried it. I was being way too literal following your instructions.

I continue on with your tutorial…

That was the kicker I needed. I buzzed through with your sample, and applied it to my own library. And even though I know that libraries are created for the binary of the device, I expected troubles using it with the Arduino IDE. But I moved the *.a and header over to the Arduino folder structure, made the appropriate support files and sure enough the Arduino IDE was able to build a project using it.

Thank you for your great tutorial!