How do I configure mbedtls within PlatformIO

Development Platforms
1

I am new to PlafrormIO and have just successfully imported my ESP32 project from the Arduino IDE to PlatformIO to work on. I would like to configure a couple of mbedtls parameters which I was given to understand is possible in PlatformIO?

According to this information, I need a file called mbed_app.json in my project directory:
https://docs.platformio.org/en/stable/frameworks/mbed.html

However, where do I find this file? I have searched but cannot find even a template for it anywhere. At this stage I know nothing about mbedtls. I am only slightly aware of json and am new to PlatformIO. If I have to construct one, what is its format and where do I start?

Esspresif has an mbedtls configuration section in its menuconfig with lots of settings that can be tweaked, but in order to port the project it would have to be completely re-written. PlatformIO allows for an easy import, but where do I find the equivalent of the mbedtls configuration in PlatformIO?

2

Hm, you seem to be a bit on the wrong path here. If you’re using an ESP32 with the Arduino framework, then you’re not using the mbed(-os) framework. Mbed-Os doesn’t even support the ESP32. see mbed-os/targets at master · ARMmbed/mbed-os · GitHub. If you open the platformio.ini of your project you’ll very likely find framework = arduino, not framework = mbed. So the linked documentation page has nothing to do with what you need.

When you use Arduino-ESP32, mbedtls is already preconfigured by the ESP-IDF base that Arduino-ESP32 is built upon and inserted into the core using a static library. This is evident by the fact that https://github.com/espressif/arduino-esp32/tree/master/tools/sdk/esp32/lib has the libmedtls.a and libmbedcrypto.a static library files and the config header files in https://github.com/espressif/arduino-esp32/tree/master/tools/sdk/esp32/include/mbedtls.

If you want to change mbedtls settings used in Arduino-ESP32 (which I would actually doubt in the first place because Espressif has already done that in the component and optimized it for the ESP32), then you would need to use the “Arduino-ESP32 as an ESP-iDF component” way. This recompiles the ESP-IDF base under the settings you want and doesn’t use the precompiled static libraries with their default settings. An official example of that is provided in platform-espressif32/examples/espidf-arduino-blink at develop · platformio/platform-espressif32 · GitHub.

By invoking the menuconfig in that project then, you can change the options exposed in the KConfig of their mbedtls comonent, which are basically all mebdtls options.

3

Maximillian, thank you for your very helpful reply.

The only reason I am looking to configure mbedtls is that I am getting this error in my project:

[189311][E][ssl_client.cpp:37] _handle_error(): [send_ssl_data():382]: (-27136) SSL - A buffer is too small to receive or write a message

It seems that the buffer can be configured via mbedtls. The flag to vary the buffer size according to the request is apparently turned off by default.

I actually tried going down the ESP-IDF with Arduino component route before I looked at PlatformIO. Unfortunately, I discovered that Arduino component was not compatible with the current version 5.x.x of the IDF. Version 5.x.x is what I got by default when trying to follow the installation steps on the Esspressif website and I can’t see any download option for version 4.x.x. I must admit, I did wonder whether the Arduino component was being deprecated? (That was suggested by someone on IRC). One drawback with the IDF is that the project will have to be re-programmed from scratch. The IDF is also seems to be extremely disk space hungry.

It may be that the above error is down to something else, but I have run out of options and ideas to test. The problem does seem to occur in ssl_client.cpp which is part of the ESP32 SDK and which appears to have dependencies in mbedtls.

Looking at my platformio.ini, it does indeed contain framework=arduino. I had understood that it was not possible to tweak mbedtls in the Arduino IDE, but I was given to understand that it is possible in PlatformIO. Your comments seem to suggest othewise, which is interesting. It is entirely possible that what I was told is incorrect, or I may have mis-understood or mis-interpreted. On the other hand, if what you say is true (an I have no reason to doubt it) then it seems that I have gained nothing, in terms of solving the SSL buffer too small problem, by importing my project from Arduino IDE into PlatformIO. I do find PlatformIO a pleasant environment to use though.

Can I ask, does the IDF use the mbed framework? If I created a project using the mbed instead of Arduino framework, would I then be working with the same code structures as within the IDF?

I should mention that we are talking about tweaking one setting here for test purposes. I didn’t expect that to be so problematic. This week has been quite a learning curve!

4

No, PlatformIO lets you tweak the mbedtls settings because you can build an Arduino project with the “Arduino-ESP32 as ESP-IDF component” easily.

Again, mbed framework (= GitHub - ARMmbed/mbed-os: Arm Mbed OS is a platform operating system designed for the internet of things) does not support ESP32. ESP-IDF hence can’t use it. ESP-IDF is the basis of all other ESP32 frameworks (like Arduino-ESP32).

ESP-IDF does have a copy of mbedTLS with their specific additions / config. Not mbed.

5

Thanks again. Obviously I am not making the distinction between mbed framework and one part of it which is mbedtls. So Espressif have included mbedtls within the ESP32 SDK even though the ESP32 is not supported by mbed.

If I want to pusue the ESP-IFD route then I need to figure out where to get ESP-IDF version 4.x.x.

In the meantime, I am looking at testing another approach to the problem.

6

I don’t quite understand? You can just download the linked example and compile it, with the latest espressif32 platform version that gives you Arduino-ESP32 2.0.3 with ESP-IDF 4.4.1. It’s all handled within PlatformIO, it downloads the frameworks for you…

7

grafik
grafik1320Ă—803 61 KB

grafik
grafik1320Ă—803 68.6 KB

Works nicley in PlatformIO just as I’ve described.

8

Maximillian, thank you again.

The confusion on my part is probably down to the fact that I had downloaded and experimented with the Espressif IDF tools before trying the vscode+PlatformIO combination. I was using the IDF as a standalone command line tool as per their instructions:

https://docs.espressif.com/projects/esp-idf/en/latest/esp32/get-started/linux-macos-setup.html
https://docs.espressif.com/projects/arduino-esp32/en/latest/esp-idf_component.html

Unfortunately the install procedure automatically downloads IDF 5.x.x with which the Arduino component is not compatible. I was therefore seeking a way to download 4.x.x for which there seems to be no link on the Espressif site…

Although I have used the Arduino IDE for quite some time, I am still very new to vscode+PlatformIO and still very much learning. In fact, I have only had it installed on my PC for a couple of days! It has some neat features but I still find the interface rather confusing. For example, yesterday I added ESP8266 to my test project and now the IDE compiles for both platforms at the same time and also tries to upload the code for both platforms to my ESP32! I couldn’t find an option to prevent it from doing so. I am sure I will get the hang of it in the end.

Going back to the Espressif IDF, your screenshots clearly show that it can be integrated with and function inside vscode+PlaformIO. I will therefore download and have a look at the example project to see how this works. I expect that I will probably be able to delete the previously downloaded standalone version entirely and just use the IDE. I already had a look at the Github page and tried those useful looking pio commands. These seem to run only within the vscode terminal. My standard OS terminal just returns:

Command 'pio' not found, but there are 16 similar ones.

It probably needs a path setting somewhere? In any case, I will work through the example.

I also noted in your previous comment the point that tweaking the mbedtls config is unlikely to be needed and this led me to consider a different approach. I had been using a library called httpsRedirect in my project. This is essentially a wrapper for WiFiClientSecure that includes a method called GET. As its name suggests, this performs a GET request, but under the hood it calls the WiFiClientSecure print() method. It also then parses the returned header, checks whether it is a redirection and if so, performs another request to the redirected URL and then finally returns the result. I decided to do an experiment by starting a new test project without the library and using only direct calls to WiFiClientSecure. The first GET request succeeded and returned a header containing the re-direction URL. I still have some way to go in completing the investigation, but I am now wondering whether the problem might be being caused by the library rather than mbedtls.

I appreciate your insight and please excuse my confusion/information overload!

1 Like
9

Hello, this may be a little offtopic. I would be interested to learn how you have managed to include HTTPSRedirect in one of your Platformio projects. I am struggling and would be happy to learn more from you if you find the time. Thank you!

10

I originally added it to my Arduino IDE ESP8266 project. For test purposes the two library files httpsRedirect.h and httpsRedirect.cpp were copied to the project directory and included the .h file in my project with:

#include "HTTPSRedirect.h"

I realised that I will need more pins for my project so I then set about porting it to the ESP32. Porting the code was no problem, but I did problems running it on the ESP32. The initial suspect was mbedtls and a possible tweak appeared to be required. This was not possible in the Arduino IDE, so a couple of days ago the project was imported into vscode+PlatformIO.

This thread came about as a result of that step. However, last night I decided to re-write the code without using the httpsRedirect library, using direct calls to WiFiClientSecure instead. I have spent quite a bit more time completing this today and it seems to have been worth it as it made possible to see what was happening “behind the scenes” in the httpsRedirect library and has led me to what I think is the underlying problem.

I don’t know whether mbedtls is involved or not, but the ESP32 seems to be very unreliable when making WiFi connections to the Internet. Usually the first connection fails, but the 2nd or 3rd succeeds. The problem is that although the project code performed up to 3 re-tries for the initial connection, the library does not attempt to re-try the subsequent re-directed connection so it just fails. There is no problem with running it on the ESP8266, but on the ESP32, WiFiClientSecure seems to fail almost every time on the 1st attempt.

The underlying problem seems to be the unreliability of the ESP32. My current code re-tries both connections up to three times and eventually succeeds, although I am not particularly happy with such a state of things.

Since I had already logged and issue with the author of the httpsRedirect library regarding the problem, I have updated the logged issue with my findings.

Incidentally, I had already noticed that the ESP32 is very unreliable when doing NTP requests. A generous amount of re-tries and timeout intervals seems to enable one to work around the problem, but again its not entirely satisfactory. I expect that the NTP client uses the same underlying WiFiClient class to do its work but I hadn’t really made the connection until just now.

The question is why WiFiSecureClient/WiFiClient on the ESP32 so unreliable? Is it a problem with this particular ESP devkit board, the ESP12E firmware code, or ESP32s in general? Could it be the fact that I am using client.setInsecure() ? I don’t know. My experiment has shown me that I don’t need to use the httpsRedirect library and I can work around the problem suffering some delays, but as for fixing the underlying problem, I have no idea at present. I did see a video from Ralph S Bacon where he had problems with the ESP32 dropping connections and his research led him to identify a problem with early ESP firmware, but this board I am currently using is a recent purchase. I have an older one from around 3 years ago which I also tried with the same result.

11

I am glad you are making progress. Also, thank you for the long answer. In the meantime, I managed to somehow set up HTTPSRedirect within PlatformIO. This is great. I am using it to write directly to a Google Spreadsheet from a Wemos D1 mini. Good luck with your project!