CMakeLists.txt and CMakeListsPrivate.txt are not generated upon `pio project init --board zero --ide clion`

[divstar]

I updated to PlatformIO 6.1.10 and created a new project from CLI. Even this was not as straight forward as I’d like.

Problem

Apparently if I do: pio project init --board zero --project-dir "~/CLionProjects/MyNewProject" --ide clion, I get Error: Invalid value for '--project-dir' / '-d': Directory '~/CLionProjects/MyNewProject' does not exist..

If I create the folder and use an absolute path (it did not work with the home directory - even with double-quotes), it creates a folder with a platformio.ini and a couple of folders. I’d expect it to create CMakeLists.txt and CMakeListsPrivate.txt at this point.

However: even at a later point no such files are created. CLion does suggest creating it (and creates it if necessary) when opening any source file, but the file seems weird content-wise.

Since I have a CMakeLists.txt and a CMakeListsPrivate.txt flying around, it must have been generated at some point when I created a new project using CLion itself (along with the PlatformIO plugin). However: this does not work anymore (the only difference between then and now is the updated PlatformIO library).

So my questions are:

1. When are the 2 CMakeLists files generated?
2. When are they regenerated?
3. And - most importantly - what should I do since these files don’t seem to be generated on my machine.

Versions

I am using PlatformIO 6.1.10 (previously 6.1.9), CLion v2023.1.3 and PlatformIO Plugin for CLion v231.8109.141.
Also I am on a MacBook M1 Pro with MacOS Ventura (13.5).

If you need any additional information, please let me know. Any help is greatly appreciated.

Edit

I verified, that on PlatformIO version 6.1.9 using Tools → PlatformIO → Re-Init (or pio project init --board zeroUSB --ide clion via CLI) indeed does generate CMakeLists.txt and CMakeListsPrivate.txt on the same project. So I suspect there being an issue with version 6.1.10 (not sure about 6.1.9_1), which is why I filed a bug on platformio-core: CMakeLists.txt and CMakeListsPrivate.txt no longer regenerated upon Re-Init (neither CLI nor CLion) · Issue #4718 · platformio/platformio-core · GitHub.

[elanf]

Can confirm the exact same issue.

[divstar]

Thought I make sure, that this topic also contains the latest information.

In the bug ticket CMakeLists.txt and CMakeListsPrivate.txt no longer regenerated upon Re-Init (neither CLI nor CLion) · Issue #4718 · platformio/platformio-core · GitHub @ivankravets replied, that with CLion 2023.2 and the newest PlatformIO the CMakeLists.txt and CMakeListsPrivate.txt files are no longer needed to actually build the project. I am about to verify this and I wonder how syntax highlighting etc. works without a build target of sorts, but I’ll find out shortly.

That said: in CLion 2023.2 Released With Better PlatformIO Integration, Updated Debugger, and Brand-New AI Assistant | The CLion Blog it’s also statet, that PlatformIO integration has been improved. That said, if you want something, that works without having to update anything, you can stay with version 6.1.9 or 6.1.9_1, but you will probably end up having to update anyway, because I am sure there were also some useful bugfixes and enhancement there (or will be in newer versions anyway).

Edit

I have tried 6.1.10 with the newest CLion (2023.2) and the newest PlatformIO plugins and apparently the code completion and e.g. Clang-Tidy (for the code?) does not work (yet?). Furthermore I had troubles importing some libraries (e.g. #include <Wire.h> resulted in an error even though it should be part of the Arduino SAMD framework that I am using?).

Hence I decided to go back to version 6.1.9_1 for now. Especially the fact, that I do not get to see errors (and warnings) in the code and that code completition does not work, annoys me very much. I’ll keep an eye on the updates though.

[copperbot-d]

Hello @divstar,

I just try to import <Wire.h> and it works on my side.

Steps to reproduce:

• Create a blank folder, and open a CLI in it
• Type pio init --board zeroUSB --ide clion
• Type pio pkg update to be sure that samd framework is up to date
• Add Arduino Blinky example
• Open the folder with Clion
• Click on the PIO reload button, to be sure everything is generated correctly
• Add #include <Wire.h> in the main.cpp

… and code insight seems to work properly. (At least with Arduino framework)

Using the pio.ini you provide here, the main.cpp file is not recognized as a project file, so code insight won’t work properly.
I suspect this is an issue with the plugin and build_src_filter variable, if you comment it, Clion seems to recognize the file properly after a reload.

If you are moving all of you code in the src/ folder, and remove/comment build_src_filter in your platformio.ini, does code insight and code completion works again ?

[divstar]

I did roll back to version 6.1.9_1, but decided to give what you said a try. Therefore I upgraded to the newest CLion and PlatformIO (and PlatformIO Core on CLI) and proceeded to try a new project.

It does build the new project, but what I am missing, are suggestions for includable header files, function and other descriptions and suggestions, typechecks and code completion. I believe all that has been provided by Clang-Tidy in the past?

I also cannot jump to function declarations (within a header in a library) and I have no entries in “External Libraries” and it seems “Clang-Tidy” does not work as a .clang-format is no longer interpreted?

I also thought, that header files should be put under the include folder, but perhaps my understanding is wrong?

Also changing the file from Blink.cpp to main.cpp (and not even having a separate declaration header) does not change the code inspection and code completion capabilities.

Is there anything else I can try to make the libraries, especially the Arduino SAMD21 one, appear in “External Libraries”? Because in the case of other languages I am dealing with (Java with Maven and Gradle in particular, but I think it happened in other languages, too), library files (and their content) only became available to the workspace after the library properly appeared as an External Library.

Edit

Now that’s weird… I am not sure if that’s the thing I missed - and I did “Tools” → “PlatformIO” → “Reload PlatformIO Project” multiple times I am sure… but once I did that on the top most folder element in the “Project Files” view, all libraries appeared and the code completion and inspection and typechecking etc. started appearing when editing C++ files.

Edit2

Figured out, that it all started working once I added “System” as a Toolchain (with default values).

Note, that the libraries - albeit with a weird name - started appearing and code completion and inspection and so on, now apparently works. I am still not sure whether the formatting with Clang-Tidy will work, but the rest certainly seems to work now.
Also: it seems to not matter anymore whether I have any CMake profiles or not - it works regardless of them.

Edit3

I apologize for the edits. Perhaps one more issue: I also have a library.json, because my project is expected to be a library. However: as long as I am developing a library, I’d be interested in having code completition and inspection etc., but apparently in order to succed at building the firmware, I must provide a main file with void setup() and void loop(). What if it’s a library and I use neither of these functions? Am I supposed to sym-link the libraries into my actual project and develop them there? Or how would this work (or perhaps I should open another question for this?)?

TL;DR

If working on a firmware with PlatformIO 6.1.10+, CLion 2023.2+ and CLion PIO Plugin 232.8660.142+, you should:

1. Create a new project either via CLion or the CLI or open a project in CLion,
2. Ensure you have a Toolchain defined (look at the screenshots above to figure out what) - default values seem fine in my case,
3. Right-click on the top-most folder in the “Project” view, select “PlatformIO” and select “Reload PlatformIO Project”.
   
   

After that, the used external libraries should appear as seen in the screenshot and code inspection, completition and typechecks should be available.

[copperbot-d]

@divstar

I also thought, that header files should be put under the include folder, but perhaps my understanding is wrong?

It’s not an obligation, you can put all of your code in the src folder (including headers), it would still be compiled. But it’s a good practice to separate headers and sources when you begin to have a lot of project files.

@divstar

“Tools” → “PlatformIO” → “Reload PlatformIO Project” multiple times I am sure… but once I did that on the top most folder element in the “Project Files” view, all libraries appeared and the code completion and inspection and typechecking etc. started appearing when editing C++ files.

Yeah, same here, sometimes I have to force the reload again to be sure all of project files are detected.

@divstar

Figured out, that it all started working once I added “System” as a Toolchain (with default values).

If you had been on Windows, I would say the bundle MinGW is missing, it should come with Clion by default:

I also have another toolchain entry (System GNU arm), but it is a custom one, I don’t use it with PlatformIO.

But you’re on Mac, it seems to be a bit different, I suppose you already setup a custom toolchain, or installed Xcode command line developer tools.
In any case, you absolutely need a default toolchain (selected in Clion parameters) in order to get code insight / code completion / clang working.

@divstar

Also: it seems to not matter anymore whether I have any CMake profiles or not - it works regardless of them.

As said here, with the last version of the PIO plugin, Clion does not require any Cmake profiles / Cmake files anymore, it’s completely autonomous (PIO core is actually generating metadata of the project in the background, and the PIO plugin is translating it to Clion).

@divstar

However: as long as I am developing a library, I’d be interested in having code completition and inspection etc., but apparently in order to succed at building the firmware, I must provide a main file with void setup() and void loop().

I may be wrong here, but yes you need a file with the main() function defined (or in the case of Arduino, setup() and loop()), in order for PIO to generate proper metadata for the project (and thus for the library), and building the library.

In any case, with PIO or not, every time I need to develop a new library, I prefer creating a blank project with a blank void main(), but with the right framework, in order to get Clion code insight working properly when I am writing the library. (Could be a bad way to do it though, but I find it easier and more practical to update the library later.)

[divstar]

@copperbot-d
Thank you so much for your response! Especially since you described how you usually create a new library. I feel this is the way I might go (and simply exclude the main.* file(s) in the library.json file).

Yeah, I’ve got XCode installed and running (I believe I installed everything I could, since I am also about to create a userspace driver for the device and a configurator application, that should work on iPads, too, along with the driver).

Just a side note: as soon as one activates Clang-Tidy for C/C++ files in the settings in CLion, it starts working, too. For the .clang-format file to work, one has to enable it in the settings or somewhere in the status bar at the bottom.

[ivankravets]

See CLion 2023.2 Released With Better PlatformIO Integration, Updated Debugger, and Brand-New AI Assistant | The CLion Blog

You will need the latest CLion and updated PlatformIO plugin.

[elmot]

@divstar I am glad to see you are using our new plugin now, any kind of feedback is more than welcome.

As already said, the experimental approach with CMake is not used anymore for various reasons.

Library writing.
First of all there is a bug with libraries CPP-34184. I don’t know if it affects you or not. In any case, the fix is already on it’s way and will be published soon.

My vision, how libraries may be written using PlatformIO and CLion:

You start with a new empty testing project set to your primary target platform and framework. So you have either setup/loop pair or main in the project. The project may be published to some git place. If you plan to support several targets, make several envs, and you may have startup approaches, activating one them with source filters.
The library itself is put to lib/<lib_name> folder. The folder may be published as a separate git repo, see git submodule feature.
With this project structure you have full set of CLion feature and ability to run test code with your library.

[davetcc]

In fairness there are a few bugs at the moment in 2023.2, I have been testing 2023.2 since it was in preview. If you are using the debugger you may find more significant issues that are between easy and moderate to work around. Apologies I should have updated here earlier.

I have a Windows PC that I have dedicated to testing 2023.2 but I’m still mainly developing from my mac on 2023.1 with a slightly downgraded platformIO. If you search the CLion bug tracker for tag platformio you’ll pretty much see the list of current issues I’ve raised there.

In summary, the outstanding ones are:

1. Some library code not being included in indexing because the IDE is not indexing library subdirs.
2. Debug not recompiling before running, for boards with a heavy build such as mbed, this is a showstopper as you need to clean before each change/run cycle.
3. Even with upload I’ve occasionally had issues with changes not being picked up, but this is probably a different issue to (2).
4. Debug for heavy builds times out before the debug process can start, but it will still at least finish compiling in the background. You can then debug once the background process finishes. You have no indication of this as the IDE loses track thinking it has terminated.

BTW - I have a repo that I use as the starting point for library development, it is davetcc/tcLibraryDev. Up until the recent changes it had worked pretty well for me, but bear in mind it’s an unsupported starting point basically.

[divstar]

Thank you @davetcc for your suggestions. I’ll definitely checkout the repository.

As crazy as it sounds: I am about 5 months into my project (I am writing a new firmware-like application for an Arduino-based device) and I haven’t really had to debug the application a single time, though perhaps it would have helped solve a couple of issues if I did.

So since I am not debugging much, CLion 2023.2 and PIO 6.1.10 seems fine for me.

What I indeed saw, was, that if I do “Update platforms, toolchains, etc.”, it sometimes does not pick up changes from the source (another folder in my case). I already thought about sym-linking the folders, but it isn’t as handy as having it managed by e.g. PIO (I am usually a Java guy, who used and uses Ant, Ivy, Maven and Gradle, though I also like npm/yarm and docker - and PIO seems very similar to this).

As for the library, the state of affairs is a bit odd. I’d have loved it if PIO (CLI or CLion Plugin) provided a way to create a library and e.g. suggest if I should exclude platformio.ini etc. and how to build the library (is that even necessary?) standalone - without a main.(h|cpp).

So far I’m sticking with 2023.2 and PIO 6.1.10 and we’ll see how that goes.

Edit: what I also figured out, is, that transient dependencies seem to be ignored.

Example: I have library A and B, B depends on A. If I import B into my application, A isn’t automatically available (even though the library.json file of B would suggest so). Instead I have to import A in addition to B or the classes and files in B will not compile properly within my application.

[davetcc]

@divstar

So far I’m sticking with 2023.2 and PIO 6.1.10 and we’ll see how that goes.

That’s the best thing to do if you can raising any bugs found, even though I’m badly affected by a few of the issues, I keep a machine on 2023.2 as it will be a significant step forward once the bugs are ironed out.

Ref the transient dependencies, you should see them in the graph printed in the output window during build initialization.

[divstar]

In regards to the transient dependencies: I had defined library A as a dependency in library B, but only in the platformio.ini (lib_deps), which isn’t enough in this case, because it’s a library. After defining library A as a dependency in library B using library.json (dependencies attribute, explained here: dependencies — PlatformIO latest documentation), library A was picked up next to library B in my firmware application.

@davetcc : I have looked into GitHub - davetcc/tcLibraryDev, but I think it’s missing a library.json. It’s not a problem in my case, but it might irritate others, who could read this topic at some point. My library repository is probably not perfect (it’s the “library A” in my example), but it seems to do its job.

[davetcc]

@divstar

I have looked into GitHub - davetcc/tcLibraryDev, but I think it’s missing a library.json. It’s not a problem in my case, but it might irritate others, who could read this topic at some point. My library repository is probably not perfect (it’s the “library A” in my example), but it seems to do its job.

Ah no, it is not a library itself, just instructions how to set up a lib directory and install libraries into it including a few setups that help with testing across a very wide range of boards, I clone that repo, then in the lib directory clone all the libraries I need, mainly tcMenu, IoAbstraction, TaskManagerIO and a few others for me.