External Library and link issues (CLion / CMake ) - file not found

[mizraith]

I have been working on porting my CLion (2017 1.3)+ Arduino CMake project over to CLion + PlatformIO. I am completely stuck on the library imports.

Background

• Created project structure: platformio init --ide clion --board megaatmega2560 --board nanoatmega328
• Opened this structure in CLion
• Copied my main source PPC.ino over and put into /src
• Convert PPC.ino to PPC.cpp by adding forward declarations of functions.
• Copied all my additional local files over and put into /lib
• Merged my original CMakeLists.txt file with the new one that platformIO created.
• PLATFORMIO_REBUILD_PROJECT_INDEX
  • Issue: My CMakeLists.txt file gets completely rewritten by PlatformIO. Yikes.
  • I’ve learned to keep a backup of my CMakeLists.txt file

This is a big-ish project. I’m referencing an external (Adafruit_Neopixel) library that I keep in a folder far outside my build directory (so that other projects can use it).

Original Settings

CMakeLists.txt:

set(ARDUINO_SKETCH_FOLDER “/Users/me/Documents/ARDUINO_DEVELOPMENT”)
include_directories(“${ARDUINO_SKETCH_FOLDER}/libraries/Adafruit_Neopixel”)

PPC.ino (now PPC.cpp)

`#include “Adafruit_Neopixel.h”

This worked with ArduinoCMake (but I suspect they had some functions in there for making includes easier). However, once I was working with PlatformIO, I just kept on getting ‘Adafruit_Neopixel’ file not found.

Tried to resolve in CMake

I tried just about every CMake call I could try, including:

• include_directories()
• link_directories()
• set(SRC_LIST ${SRC_LIST} “path/to/Adafruit_Neopixel.cpp”

I verified all the INCLUDED_DIRECTORIES are correct in CMake…but it’s like PlatformIO is not even using that list to search for the “.h” files. Question: Does PlatformIO even look at the INCLUDED_DIRECTORIES from Cmake?

Hack Workaround in Source File

By putting the full path into the include, I was able to get it to compile. But why should I have to do this?

`#include “full/path/to/the/include/file/Adafruit_Neopixel.h”

Tryied to resolve in platformio.ini

In search of a better solution, within platformio.ini, I have tried:

• lib_deps_external = Adafruit_Neopixel
• lib_extra_dirs = “path/to/folder/Adafruit_Neopixel”
• lib_ldf_mode = deep+
  • (I have also done chain+). I’m using “+” because I have include guards to switch between Neopixels and WS2801’s.

Desperation Sets In

Finally, out of desperation, I copied the Adafruit_Neopixel library folder (as well as the other external libraries) into my /lib. After that I was able to get back to simple _`#include “Adafruit_Neopixel.h”, and now it can at least compile. However, there still seem to be issues with linking everything up. This also does not seem to be the correct solution.

Also an Include Chain Issue?

In my /lib file, I have a central include file my_PPC_main.h This file calls several other #includes, referencing all the other required headers within /lib. However, it does not appear that those files are being linked (do they need to be added to source)?

To sum up – I’m in desperate need of more documentation to support my port from CLion/CMake-Arduino to Clion/CMake/PlatformIO. Can anyone lend some guidance on how to resolve these linking issues?
[/quote]

• PLATFORMIO_BUILD
  • FAIL Can’t find external files.

This is a big-ish project. I’m referencing an external (Adafruit_Neopixel) library that I keep in a folder far outside my build directory (so that other projects can use it).

Original Settings

Within CMakeLists.txt:

set(ARDUINO_SKETCH_FOLDER “/Users/me/Documents/ARDUINO_DEVELOPMENT”)
include_directories(“${ARDUINO_SKETCH_FOLDER}/libraries/Adafruit_Neopixel”)

Within PPC.ino

`#include “Adafruit_Neopixel.h”

This worked with ArduinoCMake (but I suspect they had some functions in there for making includes easier). However, once I was working with PlatformIO, I just kept on getting ‘Adafruit_Neopixel’ file not found.

Trying to resolve in CMake

I tried just about every CMake call I could try, including:

• include_directories()
• link_directories()
• set(SRC_LIST ${SRC_LIST} “path/to/Adafruit_Neopixel.cpp”

I verified all the INCLUDED_DIRECTORIES are correct in CMake…but it’s like PlatformIO is not even using that list to search for the “.h” files. Question: Does PlatformIO even look at the INCLUDED_DIRECTORIES from Cmake?

Source File Workaround

By putting the full path into the include, I was able to get it to compile. But why should I have to do this?

`#include “full/path/to/the/include/file/Adafruit_Neopixel.h”

Trying to resolve in platformio.ini

In search of a better solution, within platformio.ini, I have tried:

• lib_deps_external = Adafruit_Neopixel
• lib_extra_dirs = “path/to/folder/Adafruit_Neopixel”
• lib_ldf_mode = deep+ (I have also done chain+). I’m using “+” because I have include guards to switch between Neopixels and WS2801’s.

Desperation Sets In

Finally, out of desperation, I copied the Adafruit_Neopixel library folder (as well as the other external libraries) into my /lib. After that I was able to get back to simple _`#include “Adafruit_Neopixel.h”, and now it can at least compile. However, there still seem to be issues with linking everything up. This does not seem to be the correct solution. Help?

Is There Also an Include Chain Issue?

In my /lib file, I have a central include file my_PPC_main.h This file calls several other #includes, referencing all the other required headers within /lib.

e.g. my_PPC_main.h

#ifndef _my_PPC_Main_H_
#define _my_PPC_Main_H_
 
 //-------PIN DEFINITIONS AND CONSTANTS------ must include first
#include "PPC_PinDefs_and_Constants.h"
//-----------------------------
//for saving structs out to EEPROM. 
#include "PPC_EEPROMAnything.h"
//save many of the common general strings into flash memory
#include "PPC_Strings.h"

However, it does not appear that these additional includes are being linked (do they need to be added to source)?

e.g. start of the compile output:

[Wed Jun 21 14:36:14 2017] Processing nanoatmega328 (framework: arduino; lib_ldf_mode: deep+; build_flags: -D USE_NEOPIXEL_LEDS -Wno-unknown-pragmas; platform: atmelavr; board: nanoatmega328; upload_port: /dev/cu.usbmodem1421)
--------------------------------------------------------------------------------
Collected 30 compatible libraries
Looking for dependencies...
Library Dependency Graph
|-- <U8glib_mizraith> (/Users/me/Documents/Computer_Science/ARDUINO_DEVELOPMENT/PLATFORMIO_Projects/CLI_Mega_Due/lib/U8glib_mizraith)
|-- <EEPROM> v2.0 (/Users/me/.platformio/packages/framework-arduinoavr/libraries/__cores__/arduino/EEPROM)
|-- <Adafruit_NeoPixel> (/Users/me/Documents/Computer_Science/ARDUINO_DEVELOPMENT/PLATFORMIO_Projects/CLI_Mega_Due/lib/Adafruit_NeoPixel)
|-- <SPI> v1.0 (/Users/me/.platformio/packages/framework-arduinoavr/libraries/__cores__/arduino/SPI)
avr-g++ -o .pioenvs/nanoatmega328/src/PixelPacerController.o -c -fno-exceptions -fno-threadsafe-statics -fpermissive -std=gnu++11 -g -Os -Wall -ffunction-sections 

To sum up – I’m in desperate need of more documentation to support my port from CLion/CMake-Arduino to Clion/CMake/PlatformIO. Can anyone lend some guidance on how to resolve these linking issues?

[ivankravets]

1. PlatformIO does not use Make or Cmake. It has own independent Build System. You should work with platformio.ini each time. See Redirecting...
2. lib_extra_dirs = "path/to/folder/Adafruit_Neopixel" , that is incorrect value. Please read documentation Redirecting... and WARNING block. The correct value is lib_extra_dirs = path/to/folder/where/libraries/are/located (not directly to a library).
3. Don’t forget to re-init project after all changes.

[mizraith]

Thanks for the fast response.

Modified platformio.ini:

[platformio]
lib_extra_dirs = “/Users/me/Documents/Computer_Science/ARDUINO_DEVELOPMENT/libraries”
[common]
lib_deps_external = Adafruit_NeoPixel, U8glib_mizraith
[env:nanoatmega328]
build_flags = -D DEBUG_MODE=1 -D USE_NEOPIXEL_LEDS -D USE_U8GLIB_GRAPHICS -Wno-unknown-pragmas
lib_ldf_mode = chain+
platform = atmelavr
board = nanoatmega328
framework = arduino

Ran PLATFORMIO_REBUILD_PROJECT_INDEX and re-ran but now it is having trouble with the include “U8glib.h” file.

The structure of the external library is:
/Users/me/Documents/Computer_Science/ARDUINO_DEVELOPMENT/libraries/U8glib_mizraith/U8glib.h

• Is there a better way to specify the include so that it looks under “U8glib_mizraith” for the U8glib.h file?

Then I ran the platformio init --ide clion --board due --board megaatmega2560 --board nanoatmega328 command and was pleasantly surprised that it did not clobber my existing platformio.ini file.

Still the same problem with U8glib.h as before. Not showing up in the dependency graph.
> Library Dependency Graph
> |-- EPROM> v2.0
> |-- SPI> v1.0

[ivankravets]

[platformio]
lib_extra_dirs = /Users/me/Documents/Computer_Science/ARDUINO_DEVELOPMENT/libraries

[env:nanoatmega328]
build_flags = -D DEBUG_MODE=1 -D USE_NEOPIXEL_LEDS -D USE_U8GLIB_GRAPHICS -Wno-unknown-pragmas
platform = atmelavr
board = nanoatmega328
framework = arduino
lib_deps = Adafruit_NeoPixel, U8glib_mizraith

[mizraith]

So the quotes around the lib_extra_dirs was evil? Removing that helped!
Now I get:

Collected 57 compatible libraries
Looking for dependencies...
Library Dependency Graph
|-- <EEPROM> v2.0
|-- <Adafruit_NeoPixel>
|-- <U8glib_mizraith>
|-- <Adafruit_GFX>
|-- <Wire> v1.0
|-- <Adafruit_SSD1306>
|   |-- <Adafruit_GFX>
|   |-- <SPI> v1.0
|   |-- <Wire> v1.0
|-- <SPI> v1.0
|-- <Adafruit WS2801 Library> v1.0.0
|   |-- <SPI> v1.0

Which isn’t quite right (I should see the SSD1306 with the include guards. So I reinstated

lib_ldf_mode = chain+

and with the preprocessor #ifdef’s, I now get the more correct (and less resource intensive):

Library Dependency Graph
|-- <EEPROM> v2.0
|-- <Adafruit_NeoPixel>
|-- <U8glib_mizraith>
|-- <SPI> v1.0

I’m still seeing the linking issue with the PPC_main.h file. This may be my problem (with my include structure) and I will look into that. It seem like only a few includes are missing, not all of them. , e.g. :

Linking .pioenvs/nanoatmega328/firmware.elf
/var/folders/_0/khxnwnmm8xj15y0059_xt4th0000gn/T//ccfK0GrP.ltrans0.ltrans.o: In function `setPotFlagIfChanged(unsigned char, unsigned char)':
ccfK0GrP.ltrans0.o:(.text+0x2fe): undefined reference to `getAbs(signed char)'
/var/folders/_0/khxnwnmm8xj15y0059_xt4th0000gn/T//ccfK0GrP.ltrans0.ltrans.o: In function `doProgrammingMode()':
ccfK0GrP.ltrans0.o:(.text+0x3f8): undefined reference to `displayInitialProgrammingModeScreen()'
ccfK0GrP.ltrans0.o:(.text+0x43c): undefined reference to `displayUpdateProgrammingMode()'

[ivankravets]

@mizraith

I’m still seeing the linking issue with the PPC_main.h file.

It means that some prototypes are not implemented. Where is located displayUpdateProgrammingMode()?

[mizraith]

Okay - i figured it out. Again, thanks for your awesome help.

Turns out I had one main file in /src: my_application.cpp

And I had put all my other files under /lib
my_main_include_file.h
my_other_headers.h
my_other_sources.cpp

The idea was that inside my_application.cpp there was a single include
#include "my_main_include_file.h"
Inside that include was a list of #include for all the other headers in /lib.

But platformIO couldn’t compile with the above include so I tried:
#include "../lib/my_main_include_file.h"

This worked, but didn’t link.
(1) Is there a way to get this to link up? Can I add /lib as a directory to scan for sources?

So, I copied all my source files from /lib into /src and removed the file path from my #include. And voila – EVERYTHING COMPILED!!!

[ivankravets]

Again, docs, docs, and docs.

See Library Dependency Finder (LDF) — PlatformIO latest documentation

Also, you have a README in lib folder where is detailed description how to use own libraries.

[mizraith]

Ivan,
Not sure I completely follow. I read the LDF docs closely, multiple times. The first issue (that paths must not have quotes) is never explicitly stated in the docs. What about paths with spaces? I went with the default (both Windows/Mac) and quoted my path.

The second issue, using /lib was surprising. From the docs, it appears LDF looks in /lib, and I had set the chain+ option. Why wouldn’t LDF have included the rest of the files linked by my .h?

I’m not knocking your support (which has been excellent), but having read the docs over multiple times, catching the little non-standard differences was difficult and time consuming. That said, not sure there would have been anyway to catch these issues (unless PIO had improved syntax checking for the platformio.ini file). And that’s why your support has been so critical to get me up and running.

Thanks!

[ivankravets]

@mizraith

What about paths with spaces?

PlatformIO handles paths with spaces automatically

@mizraith

Why wouldn’t LDF have included the rest of the files linked by my .h?

It looks for “a library” in lib folder, but not for a source code in a root of lib/*. So, lib folder is intended for libraries where each of them has an own folder.

@mizraith

(unless PIO had improved syntax checking for the platformio.ini file)

Would be thankful if you point us where we should improve documentation or other things. Please note, that our documentation is open source. You can help us with improving. See GitHub - platformio/platformio-docs: PlatformIO Documentation

[mizraith]

I think your support is helping fill in the gaps. As for docs – it looks like you already bolstered up the CLion docs a bit (Thanks!)

One suggestion would be a bit more on the Redirecting... page. This would be a good section to talk about paths (1) being non-quoted and (2) the [common] section (which currently is missing from that page)