Anybody able to successfully compile Azure SDK ESP32-dev with Arduino in PlatformIO 5

greg.svitak · October 24, 2020, 2:56pm

Hello,

I am curious if anybody has been able to run a recent version of Azure SDK samples with Platformio. I have been able to compile and run using Arduino IDE but not Platformio.

Details:
I am trying to compile the latest Azure SDK sample(s) with Arduino using CLion and Platformio. My board is the ESP32-WRover so I believe I cannot use the libraries that are available on the online Platformio libraries repo based on the documentation on Microsoft’s website.

It appears that I have a dependency management issue. Every time, I try to compile I have issues with the WiFiClientSecure class… specifically with AzureIoTUtility. It appears that AzureIoTUtility it expecting a specific version of Wifi.h and WiFiClientSecure.h.

Scanning dependencies...
Dependency Graph
|-- <AzureIoTProtocol_HTTP> 1.3.9
|   |-- <AzureIoTUtility> 1.3.9
|   |-- <AzureIoTHub> 1.3.9
|   |   |-- <AzureIoTUtility> 1.3.9
|-- <AzureIoTProtocol_MQTT> 1.3.9
|   |-- <AzureIoTHub> 1.3.9
|   |   |-- <AzureIoTUtility> 1.3.9
|   |-- <AzureIoTUtility> 1.3.9
|-- <AzureIoTHub> 1.3.9
|   |-- <AzureIoTUtility> 1.3.9
Building in debug mode
Compiling .pio/build/esp32-dev/lib6f2/AzureIoTUtility/adapters/sslClient_arduino.cpp.o
lib/AzureIoTUtility/src/adapters/sslClient_arduino.cpp:15:18: fatal error: WiFi.h: No such file or directory

I am able to build using the Arduino IDE. However I had to follow the following.

Specifically I copied the resulting files from make_sdk.py and added them to the libraries directory in my Arduino path.
Copied the platform.txt to correct spot.
https://github.com/Azure/azure-iot-pal-arduino/pull/17

Based on what I was reading with other posts, I think I have to do something similar for the Platformio setup.

I copied the directories from the Azure SDK make_sdk.py to my lib directory in my Platformio project.
I copied the platform.text to /Users/user_dir/.platformio/packages/framework-arduinoespressif32/platform.txt

Here is my platformio.ini

[env:gatekit_esp_dev]
platform = espressif32
board = esp-wrover-kit
framework = arduino
debug_tool = esp-prog
build_flags = -DCORE_DEBUG_LEVEL=5 -DBA_BOARD=0 -DDONT_USE_UPLOADTOBLOB
board_build.partitions = min_spiffs.csv
monitor_speed = 115200

Any help would be appreciated.
Thanks
Greg

maxgerhardt · October 24, 2020, 3:16pm

I don’t see WiFi in your library dependencies. Have you done a #include <WiFi.h> in your main CPP file?

maxgerhardt · October 24, 2020, 3:17pm

Also, PlatformIO doesn’t process platform.txt (or boards.txt for that matter) in any form. This is all done in Python code (build scripts) and JSON (board definitions).

greg.svitak · October 24, 2020, 10:14pm

Hi @maxgerhardt,

Thanks for your suggestion.

yes I have tried to include in main.cpp. Here is what happens:

LDF Modes: Finder ~ chain, Compatibility ~ soft
Found 31 compatible libraries
Scanning dependencies...
Dependency Graph
|-- <AzureIoTProtocol_HTTP> 1.3.9
|   |-- <AzureIoTUtility> 1.3.9
|   |   |-- <WiFi> 1.2.7
|   |   |   |-- <SPI> 1.0
|   |   |-- <SPI> 1.0
|   |-- <AzureIoTHub> 1.3.9
|   |   |-- <AzureIoTUtility> 1.3.9
|   |   |   |-- <WiFi> 1.2.7
|   |   |   |   |-- <SPI> 1.0
|   |   |   |-- <SPI> 1.0
|-- <AzureIoTProtocol_MQTT> 1.3.9
|   |-- <AzureIoTHub> 1.3.9
|   |   |-- <AzureIoTUtility> 1.3.9
|   |   |   |-- <WiFi> 1.2.7
|   |   |   |   |-- <SPI> 1.0
|   |   |   |-- <SPI> 1.0
|   |-- <AzureIoTUtility> 1.3.9
|   |   |-- <WiFi> 1.2.7
|   |   |   |-- <SPI> 1.0
|   |   |-- <SPI> 1.0
|-- <AzureIoTHub> 1.3.9
|   |-- <AzureIoTUtility> 1.3.9
|   |   |-- <WiFi> 1.2.7
|   |   |   |-- <SPI> 1.0
|   |   |-- <SPI> 1.0
|-- <WiFi> 1.2.7
|   |-- <SPI> 1.0
Building in debug mode
Compiling .pio/build/gatekit_moduino/src/main.cpp.o
Compiling .pio/build/gatekit_moduino/lib47d/SPI/SPI.cpp.o
Compiling .pio/build/gatekit_moduino/lib7b6/WiFi/WiFi.cpp.o
Compiling .pio/build/gatekit_moduino/lib7b6/WiFi/WiFiClient.cpp.o
Compiling .pio/build/gatekit_moduino/lib7b6/WiFi/WiFiServer.cpp.o
Compiling .pio/build/gatekit_moduino/lib7b6/WiFi/WiFiUdp.cpp.o
Compiling .pio/build/gatekit_moduino/lib7b6/WiFi/utility/server_drv.cpp.o
Compiling .pio/build/gatekit_moduino/lib7b6/WiFi/utility/spi_drv.cpp.o
Compiling .pio/build/gatekit_moduino/lib7b6/WiFi/utility/wifi_drv.cpp.o
Compiling .pio/build/gatekit_moduino/lib6f2/AzureIoTUtility/adapters/sslClient_arduino.cpp.o
lib/AzureIoTUtility/src/adapters/sslClient_arduino.cpp:16:30: fatal error: WiFiClientSecure.h: No such file or directory

and if I add both

#include <Arduino.h>
#include <WiFi.h>
#include "WiFiClientSecure.h"
#include <AzureIoTHub.h>
#include <stdio.h>
#include <stdlib.h>

Found 31 compatible libraries
Scanning dependencies...
Dependency Graph
|-- <AzureIoTProtocol_HTTP> 1.3.9
|   |-- <AzureIoTUtility> 1.3.9
|   |   |-- <WiFi> 1.2.7
|   |   |   |-- <SPI> 1.0
|   |   |-- <WiFiClientSecure> 1.0
|   |   |   |-- <WiFi> 1.2.7
|   |   |   |   |-- <SPI> 1.0
|   |   |-- <SPI> 1.0
|   |-- <AzureIoTHub> 1.3.9
|   |   |-- <AzureIoTUtility> 1.3.9
|   |   |   |-- <WiFi> 1.2.7
|   |   |   |   |-- <SPI> 1.0
|   |   |   |-- <WiFiClientSecure> 1.0
|   |   |   |   |-- <WiFi> 1.2.7
|   |   |   |   |   |-- <SPI> 1.0
|   |   |   |-- <SPI> 1.0
|-- <AzureIoTProtocol_MQTT> 1.3.9
|   |-- <AzureIoTHub> 1.3.9
|   |   |-- <AzureIoTUtility> 1.3.9
|   |   |   |-- <WiFi> 1.2.7
|   |   |   |   |-- <SPI> 1.0
|   |   |   |-- <WiFiClientSecure> 1.0
|   |   |   |   |-- <WiFi> 1.2.7
|   |   |   |   |   |-- <SPI> 1.0
|   |   |   |-- <SPI> 1.0
|   |-- <AzureIoTUtility> 1.3.9
|   |   |-- <WiFi> 1.2.7
|   |   |   |-- <SPI> 1.0
|   |   |-- <WiFiClientSecure> 1.0
|   |   |   |-- <WiFi> 1.2.7
|   |   |   |   |-- <SPI> 1.0
|   |   |-- <SPI> 1.0
|-- <WiFiClientSecure> 1.0
|   |-- <WiFi> 1.2.7
|   |   |-- <SPI> 1.0
|-- <AzureIoTHub> 1.3.9
|   |-- <AzureIoTUtility> 1.3.9
|   |   |-- <WiFi> 1.2.7
|   |   |   |-- <SPI> 1.0
|   |   |-- <WiFiClientSecure> 1.0
|   |   |   |-- <WiFi> 1.2.7
|   |   |   |   |-- <SPI> 1.0
|   |   |-- <SPI> 1.0
|-- <WiFi> 1.2.7
|   |-- <SPI> 1.0
Building in debug mode
Compiling .pio/build/gatekit_moduino/src/main.cpp.o
Archiving .pio/build/gatekit_moduino/lib47d/libSPI.a
Indexing .pio/build/gatekit_moduino/lib47d/libSPI.a
Archiving .pio/build/gatekit_moduino/lib7b6/libWiFi.a
Indexing .pio/build/gatekit_moduino/lib7b6/libWiFi.a
Compiling .pio/build/gatekit_moduino/lib42e/WiFiClientSecure/WiFiClientSecure.cpp.o
Compiling .pio/build/gatekit_moduino/lib42e/WiFiClientSecure/ssl_client.cpp.o
Compiling .pio/build/gatekit_moduino/lib6f2/AzureIoTUtility/adapters/sslClient_arduino.cpp.o
Compiling .pio/build/gatekit_moduino/lib6f2/AzureIoTUtility/azure_c_shared_utility/sha1.c.o
/Users/greg.svitak/.platformio/packages/framework-arduinoespressif32/libraries/WiFiClientSecure/src/WiFiClientSecure.cpp: In constructor 'WiFiClientSecure::WiFiClientSecure()':
/Users/greg.svitak/.platformio/packages/framework-arduinoespressif32/libraries/WiFiClientSecure/src/WiFiClientSecure.cpp:33:5: error: '_connected' was not declared in this scope
     _connected = false;
     ^
/Users/greg.svitak/.platformio/packages/framework-arduinoespressif32/libraries/WiFiClientSecure/src/WiFiClientSecure.cpp: In constructor 'WiFiClientSecure::WiFiClientSecure(int)':
/Users/greg.svitak/.platformio/packages/framework-arduinoespressif32/libraries/WiFiClientSecure/src/WiFiClientSecure.cpp:50:5: error: '_connected' was not declared in this scope
     _connected = false;

Any suggestions?

Thanks,
Greg

greg.svitak · October 24, 2020, 10:17pm

Hi @maxgerhardt

Ohh thanks I did not know that piece of information.

Can you please provide a link to information/dcos on how I can modify the similar setup as platform.txt?

Based on my experience of setting up the Arduino IDE, I have a hunch this might be the issue.

Any nudge in the right direction would be really appreciated.

Thanks,
Greg

maxgerhardt · October 24, 2020, 10:19pm

Whatever WiFi library it’s compiling there it’s not the one from the Espressif32 core.

Remove any global WiFi library that you may have installed in <user home directory>\.platformio\lib. And remove the .pio\libdeps folder of the project. Then clean the project and build again.

If compilation is still not going through, do a “Advanced -> Verbose Build” and post only the library dependency graph. Should show where this wrong WiFi library is hiding.

maxgerhardt · October 24, 2020, 10:41pm

When I compare their platforms.local.txt against Arduino-ESP32’s, the main addition just seems to be

The addition of 3 defines in the build process. As shown in the docs, you can get the exact same behavior when you write

build_flags = -DDONT_USE_UPLOADTOBLOB -DUSE_BALTIMORE_CERT -DUSE_MBEDTLS

in the platformio.ini.

Notice that per their documentation, you might want / need different certs

github.com

Azure/azure-iot-sdk-c/blob/main/certs/certs.c#L12-L17


      
          */
          #if !defined(USE_AZURE_CLOUD_RSA_CERT) && !defined(USE_AZURE_CLOUD_ECC_CERT) && \
              !defined(USE_MICROSOFTAZURE_DE_CERT) && !defined(USE_PORTAL_AZURE_CN_CERT)
          // For legacy, if no certificates were explicitly selected then include all of them
          #define USE_AZURE_CLOUD_RSA_CERT
          #define USE_AZURE_CLOUD_ECC_CERT

greg.svitak · October 25, 2020, 3:26pm

Hi @maxgerhardt

Thanks for pointing me in a direction. Based on your help, I am hopeful I am close to resolving.

I am trying to find the advanced menu to find the verbose build option. Can you please point me in the direction you are thinking? Are you thinking about the following

pio run --verbose

maxgerhardt · October 25, 2020, 3:28pm

That is the equivalent command line command. I was assuming you’re using VSCode + PIO extension, in which “Verbose Build” is found in the project task pane as documented. pio run -v output (the dependency graph) suffices too, of course.

maxgerhardt · October 25, 2020, 3:59pm

Based on your response time you still seem to be having trouble.

I’ve created project GitHub - maxgerhardt/pio-azure-iot-arduino-demo: PlatformIO compilable version of the azure-iot-arduino repository for ESP32. as a compilable PlatformIO project example. Have a look if something is not working.

greg.svitak · October 25, 2020, 5:30pm

Hi @maxgerhardt

Thanks for taking the time to help. I was using CLion and had to setup VSCode. This step was very helpful.

I think I figured it out as the example compiles!! Thanks for the help!!

Before responding I was trying a few things. I noticed that AzureIoTSocket_WiFi.h seemed to be the issue. So I tried to create a new project with just library to see if it would compile on its own. This lead me to add the “include AzureIoTSocket_WiFi.h”

I had to add the following to get the sample to compile to main.cpp

#include <Arduino.h>
#include <WiFi.h>
#include <WiFiClientSecure.h>
#include “AzureIoTSocket_WiFi.h”
#include “AzureIoTHub.h”

#include “iot_configs.h” // You must set your wifi SSID, wifi PWD, and your IoTHub Device Connection String in iot_configs.h
#include “sample_init.h”

However I still have a circular dependency issue.

Found 30 compatible libraries
Scanning dependencies...
Warning! Circular dependencies detected between `/Users/user_dir/git/firmware/lib/AzureIoTUtility` and `/Users/user_dir/git/firmware/lib/AzureIoTHub`
Warning! Circular dependencies detected between `/Users/user_dir/git/firmware/lib/AzureIoTUtility` and `/Users/user_dir/git/firmware/lib/AzureIoTHub`
Warning! Circular dependencies detected between `/Users/user_dir/git/firmware/lib/AzureIoTUtility` and `/Users/user_dir/git/firmware/lib/AzureIoTHub`
Warning! Circular dependencies detected between `/Users/user_dir/git/firmware/lib/AzureIoTUtility` and `/Users/user_dir/git/firmware/lib/AzureIoTSocket_WiFi`
Dependency Graph
|-- <AzureIoTHub> 1.3.9 (/Users/user_dir/git/firmware/lib/AzureIoTHub)
|   |-- <AzureIoTUtility> 1.3.9 (/Users/user_dir/git/firmware/lib/AzureIoTUtility)
|   |   |-- <WiFi> 1.0 (/Users/user_dir/.platformio/packages/framework-arduinoespressif32/libraries/WiFi)
|   |   |-- <WiFiClientSecure> 1.0 (/Users/user_dir/.platformio/packages/framework-arduinoespressif32/libraries/WiFiClientSecure)
|   |   |   |-- <WiFi> 1.0 (/Users/user_dir/.platformio/packages/framework-arduinoespressif32/libraries/WiFi)
|-- <AzureIoTProtocol_HTTP> 1.3.9 (/Users/user_dir/git/firmware/lib/AzureIoTProtocol_HTTP)
|   |-- <AzureIoTUtility> 1.3.9 (/Users/user_dir/git/firmware/lib/AzureIoTUtility)
|   |   |-- <WiFi> 1.0 (/Users/user_dir/.platformio/packages/framework-arduinoespressif32/libraries/WiFi)
|   |   |-- <WiFiClientSecure> 1.0 (/Users/user_dir/.platformio/packages/framework-arduinoespressif32/libraries/WiFiClientSecure)
|   |   |   |-- <WiFi> 1.0 (/Users/user_dir/.platformio/packages/framework-arduinoespressif32/libraries/WiFi)
|   |-- <AzureIoTHub> 1.3.9 (/Users/user_dir/git/firmware/lib/AzureIoTHub)
|   |   |-- <AzureIoTUtility> 1.3.9 (/Users/user_dir/git/firmware/lib/AzureIoTUtility)
|   |   |   |-- <WiFi> 1.0 (/Users/user_dir/.platformio/packages/framework-arduinoespressif32/libraries/WiFi)
|   |   |   |-- <WiFiClientSecure> 1.0 (/Users/user_dir/.platformio/packages/framework-arduinoespressif32/libraries/WiFiClientSecure)
|   |   |   |   |-- <WiFi> 1.0 (/Users/user_dir/.platformio/packages/framework-arduinoespressif32/libraries/WiFi)
|-- <AzureIoTProtocol_MQTT> 1.3.9 (/Users/user_dir/git/firmware/lib/AzureIoTProtocol_MQTT)
|   |-- <AzureIoTHub> 1.3.9 (/Users/user_dir/git/firmware/lib/AzureIoTHub)
|   |   |-- <AzureIoTUtility> 1.3.9 (/Users/user_dir/git/firmware/lib/AzureIoTUtility)
|   |   |   |-- <WiFi> 1.0 (/Users/user_dir/.platformio/packages/framework-arduinoespressif32/libraries/WiFi)
|   |   |   |-- <WiFiClientSecure> 1.0 (/Users/user_dir/.platformio/packages/framework-arduinoespressif32/libraries/WiFiClientSecure)
|   |   |   |   |-- <WiFi> 1.0 (/Users/user_dir/.platformio/packages/framework-arduinoespressif32/libraries/WiFi)
|   |-- <AzureIoTUtility> 1.3.9 (/Users/user_dir/git/firmware/lib/AzureIoTUtility)
|   |   |-- <WiFi> 1.0 (/Users/user_dir/.platformio/packages/framework-arduinoespressif32/libraries/WiFi)
|   |   |-- <WiFiClientSecure> 1.0 (/Users/user_dir/.platformio/packages/framework-arduinoespressif32/libraries/WiFiClientSecure)
|   |   |   |-- <WiFi> 1.0 (/Users/user_dir/.platformio/packages/framework-arduinoespressif32/libraries/WiFi)
|-- <AzureIoTSocket_WiFi> 1.0.0 (/Users/user_dir/git/firmware/lib/AzureIoTSocket_WiFi)
|   |-- <WiFi> 1.0 (/Users/user_dir/.platformio/packages/framework-arduinoespressif32/libraries/WiFi)
|   |-- <AzureIoTUtility> 1.3.9 (/Users/user_dir/git/firmware/lib/AzureIoTUtility)
|   |   |-- <WiFi> 1.0 (/Users/user_dir/.platformio/packages/framework-arduinoespressif32/libraries/WiFi)
|   |   |-- <WiFiClientSecure> 1.0 (/Users/user_dir/.platformio/packages/framework-arduinoespressif32/libraries/WiFiClientSecure)
|   |   |   |-- <WiFi> 1.0 (/Users/user_dir/.platformio/packages/framework-arduinoespressif32/libraries/WiFi)
|-- <WiFi> 1.0 (/Users/user_dir/.platformio/packages/framework-arduinoespressif32/libraries/WiFi)
|-- <WiFiClientSecure> 1.0 (/Users/user_dir/.platformio/packages/framework-arduinoespressif32/libraries/WiFiClientSecure)
|   |-- <WiFi> 1.0 (/Users/user_dir/.platformio/packages/framework-arduinoespressif32/libraries/WiFi)
Building in release mode

Any suggestions?

Thanks again,
Greg

PS. As I was writing this, I see your replied. I will check your sample link.

maxgerhardt · October 25, 2020, 11:06pm

greg.svitak:

Scanning dependencies...
Warning! Circular dependencies detected between `/Users/user_dir/git/firmware/lib/AzureIoTUtility` and `/Users/user_dir/git/firmware/lib/AzureIoTHub`
Warning! Circular dependencies detected between `/Users/user_dir/git/firmware/lib/AzureIoTUtility` and `/Users/user_dir/git/firmware/lib/AzureIoTHub`
Warning! Circular dependencies detected between `/Users/user_dir/git/firmware/lib/AzureIoTUtility` and `/Users/user_dir/git/firmware/lib/AzureIoTHub`
Warning! Circular dependencies detected between `/Users/user_dir/git/firmware/lib/AzureIoTUtility` and `/Users/user_dir/git/firmware/lib/AzureIoTSocket_WiFi`

Interesting, doesn’t occur for me.

Found 35 compatible libraries
Scanning dependencies...
Dependency Graph
|-- <AzureIoTHub> 1.3.9
|   |-- <AzureIoTUtility> 1.3.9
|   |   |-- <WiFi> 1.0
|   |   |-- <WiFiClientSecure> 1.0
|   |   |   |-- <WiFi> 1.0
|-- <AzureIoTUtility> 1.3.9
|   |-- <WiFi> 1.0
|   |-- <WiFiClientSecure> 1.0
|   |   |-- <WiFi> 1.0
|-- <AzureIoTProtocol_MQTT> 1.3.9
|   |-- <AzureIoTHub> 1.3.9
|   |   |-- <AzureIoTUtility> 1.3.9
|   |   |   |-- <WiFi> 1.0
|   |   |   |-- <WiFiClientSecure> 1.0
|   |   |   |   |-- <WiFi> 1.0
|   |-- <AzureIoTUtility> 1.3.9
|   |   |-- <WiFi> 1.0
|   |   |-- <WiFiClientSecure> 1.0
|   |   |   |-- <WiFi> 1.0
|-- <AzureIoTProtocol_HTTP> 1.3.9
|   |-- <AzureIoTUtility> 1.3.9
|   |   |-- <WiFi> 1.0
|   |   |-- <WiFiClientSecure> 1.0
|   |   |   |-- <WiFi> 1.0
|   |-- <AzureIoTHub> 1.3.9
|   |   |-- <AzureIoTUtility> 1.3.9
|   |   |   |-- <WiFi> 1.0
|   |   |   |-- <WiFiClientSecure> 1.0
|   |   |   |   |-- <WiFi> 1.0
|-- <AzureIoTSocket_WiFi> 1.0.0
|   |-- <WiFi> 1.0
|   |-- <AzureIoTUtility> 1.3.9
|   |   |-- <WiFi> 1.0
|   |   |-- <WiFiClientSecure> 1.0
|   |   |   |-- <WiFi> 1.0
|-- <WiFi> 1.0
|-- <WiFiClientSecure> 1.0
|   |-- <WiFi> 1.0
Building in release mode

Probably because you’ve put the AzureIoTUtility in the lib/ folder so it’s being analyzed in some way…

I just declare them via lib_deps, so the libraries are auto-downloaded into .pio\libdeps. The only folder in lib/ I have is the AzureIoTSocket_WiFi library that I had to self-generate from their SDK…

Also if it compiles the warning from the library dependency finder can be ignored I think… If in doubt you can try my project and/or use the latest PlatformIO core which might behave differently (upgrade via pio upgrade --dev)