Cannot compile project based on espidf framework

tom.holzwurm · July 1, 2020, 8:59am

Dear colleagues,

after working a while with the ESP32 arduino framework I like to switch for some application to the espidf framework.

But the first attempt created a lot of compile errors caused by problems within the framework.

Especially I was puzzled by looking into “transport_ws.c”: I had assumed, that this file is copied from the library structure into the build directory of the project.

But the file “transport_ws.c” in the “.platformio” structure is completely different with the version in the project build dir, which is also wrong and causes errors.

BTW: the projects goal is simply to compile the example included in the MQTT library from Andrey Mitrokhin.

I hope somebody has a clue whats going wrong here, as this should be a very primitive setup.

Thanks in advance,

Regards Thomas

PS: I’ve tested on Windows 10 and Debian 10 Linux

the platformio.ini file:


[env:nodemcu-32s]

platform = espressif32

#platform = https://github.com/platformio/platform-espressif32.git

#platform_packages =  framework-espidf @ https://github.com/espressif/esp-idf.git

board = nodemcu-32s

framework = espidf

board_build.mcu = esp32

board_build.f_cpu = 240000000L

monitor_speed = 115200

monitor_port = COM7

upload_port = COM7

#build_flags = -w

lib_deps =

  # Using a library name

  MQTT library for ESP32 (ESP-IDF)

    

  # ... or using library Id

  5670

      

  # ... or depend on a specific version

  MQTT library for ESP32 (ESP-IDF)@1.0.0

    

  # Semantic Versioning Rules

  # http://docs.platformio.org/page/userguide/lib/cmd_install.html#description

  MQTT library for ESP32 (ESP-IDF)@^1.0.0

  MQTT library for ESP32 (ESP-IDF)@~1.0.0

  MQTT library for ESP32 (ESP-IDF)@>=1.0.0

the espidf version:


espressif32                     @ 1.12.4 

Updating toolchain-xtensa32     @ 2.50200.80

the error messages:


Compiling .pio/build/nodemcu-32s/lib339/MQTT library for ESP32 _ESP-IDF__ID5670/http_parser.o

.pio/libdeps/nodemcu-32s/MQTT library for ESP32 _ESP-IDF__ID5670/src/http_parser.c: In function 'http_parser_execute':

.pio/libdeps/nodemcu-32s/MQTT library for ESP32 _ESP-IDF__ID5670/src/http_parser.c:1816:31: warning: this statement may fall through [-Wimplicit-fallthrough=]

               parser->upgrade = 1;

               ~~~~~~~~~~~~~~~~^~~

.pio/libdeps/nodemcu-32s/MQTT library for ESP32 _ESP-IDF__ID5670/src/http_parser.c:1818:13: note: here

             case 1:

             ^~~~

.pio/libdeps/nodemcu-32s/MQTT library for ESP32 _ESP-IDF__ID5670/src/http_parser.c: In function 'http_parser_parse_url':

.pio/libdeps/nodemcu-32s/MQTT library for ESP32 _ESP-IDF__ID5670/src/http_parser.c:2375:18: warning: this statement may fall through [-Wimplicit-fallthrough=]

         found_at = 1;

         ~~~~~~~~~^~~

.pio/libdeps/nodemcu-32s/MQTT library for ESP32 _ESP-IDF__ID5670/src/http_parser.c:2378:7: note: here

       case s_req_server:

       ^~~~

Compiling .pio/build/nodemcu-32s/lib339/MQTT library for ESP32 _ESP-IDF__ID5670/mqtt_client.o

Compiling .pio/build/nodemcu-32s/lib339/MQTT library for ESP32 _ESP-IDF__ID5670/mqtt_msg.o

Archiving .pio/build/nodemcu-32s/bootloader/esp-idf/xtensa/libxtensa.a

Linking .pio/build/nodemcu-32s/bootloader.elf

In file included from .pio/libdeps/nodemcu-32s/MQTT library for ESP32 _ESP-IDF__ID5670/include/platform_esp32_idf.h:21,

                 from .pio/libdeps/nodemcu-32s/MQTT library for ESP32 _ESP-IDF__ID5670/include/platform.h:11,

                 from .pio/libdeps/nodemcu-32s/MQTT library for ESP32 _ESP-IDF__ID5670/src/mqtt_client.c:2:

/home/thomas/.platformio/packages/framework-espidf/components/esp32/include/rom/queue.h:1:2: warning: #warning rom/queue.h is deprecated, please use sys/queue.h instead [-Wcpp]

 #warning rom/queue.h is deprecated, please use sys/queue.h instead

  ^~~~~~~

In file included from .pio/libdeps/nodemcu-32s/MQTT library for ESP32 _ESP-IDF__ID5670/include/platform_esp32_idf.h:21,

                 from .pio/libdeps/nodemcu-32s/MQTT library for ESP32 _ESP-IDF__ID5670/include/platform.h:11,

                 from .pio/libdeps/nodemcu-32s/MQTT library for ESP32 _ESP-IDF__ID5670/src/mqtt_msg.c:35:

/home/thomas/.platformio/packages/framework-espidf/components/esp32/include/rom/queue.h:1:2: warning: #warning rom/queue.h is deprecated, please use sys/queue.h instead [-Wcpp]

 #warning rom/queue.h is deprecated, please use sys/queue.h instead

  ^~~~~~~

Building .pio/build/nodemcu-32s/bootloader.bin

Compiling .pio/build/nodemcu-32s/lib339/MQTT library for ESP32 _ESP-IDF__ID5670/mqtt_outbox.o

esptool.py v2.6

Compiling .pio/build/nodemcu-32s/lib339/MQTT library for ESP32 _ESP-IDF__ID5670/platform_esp32_idf.o

In file included from .pio/libdeps/nodemcu-32s/MQTT library for ESP32 _ESP-IDF__ID5670/include/platform_esp32_idf.h:21,

                 from .pio/libdeps/nodemcu-32s/MQTT library for ESP32 _ESP-IDF__ID5670/include/platform.h:11,

                 from .pio/libdeps/nodemcu-32s/MQTT library for ESP32 _ESP-IDF__ID5670/include/mqtt_outbox.h:8,

                 from .pio/libdeps/nodemcu-32s/MQTT library for ESP32 _ESP-IDF__ID5670/src/mqtt_outbox.c:1:

/home/thomas/.platformio/packages/framework-espidf/components/esp32/include/rom/queue.h:1:2: warning: #warning rom/queue.h is deprecated, please use sys/queue.h instead [-Wcpp]

 #warning rom/queue.h is deprecated, please use sys/queue.h instead

  ^~~~~~~

In file included from .pio/libdeps/nodemcu-32s/MQTT library for ESP32 _ESP-IDF__ID5670/src/mqtt_outbox.c:4:

/home/thomas/.platformio/packages/framework-espidf/components/esp32/include/rom/queue.h:1:2: warning: #warning rom/queue.h is deprecated, please use sys/queue.h instead [-Wcpp]

 #warning rom/queue.h is deprecated, please use sys/queue.h instead

  ^~~~~~~

In file included from .pio/libdeps/nodemcu-32s/MQTT library for ESP32 _ESP-IDF__ID5670/include/platform_esp32_idf.h:21,

                 from .pio/libdeps/nodemcu-32s/MQTT library for ESP32 _ESP-IDF__ID5670/include/platform.h:11,

                 from .pio/libdeps/nodemcu-32s/MQTT library for ESP32 _ESP-IDF__ID5670/src/platform_esp32_idf.c:1:

/home/thomas/.platformio/packages/framework-espidf/components/esp32/include/rom/queue.h:1:2: warning: #warning rom/queue.h is deprecated, please use sys/queue.h instead [-Wcpp]

 #warning rom/queue.h is deprecated, please use sys/queue.h instead

  ^~~~~~~

Compiling .pio/build/nodemcu-32s/lib339/MQTT library for ESP32 _ESP-IDF__ID5670/transport.o

In file included from .pio/libdeps/nodemcu-32s/MQTT library for ESP32 _ESP-IDF__ID5670/src/transport.c:4:

/home/thomas/.platformio/packages/framework-espidf/components/esp32/include/rom/queue.h:1:2: warning: #warning rom/queue.h is deprecated, please use sys/queue.h instead [-Wcpp]

 #warning rom/queue.h is deprecated, please use sys/queue.h instead

  ^~~~~~~

Compiling .pio/build/nodemcu-32s/lib339/MQTT library for ESP32 _ESP-IDF__ID5670/transport_ssl.o

Compiling .pio/build/nodemcu-32s/lib339/MQTT library for ESP32 _ESP-IDF__ID5670/transport_tcp.o

In file included from .pio/libdeps/nodemcu-32s/MQTT library for ESP32 _ESP-IDF__ID5670/include/platform_esp32_idf.h:21,

                 from .pio/libdeps/nodemcu-32s/MQTT library for ESP32 _ESP-IDF__ID5670/include/platform.h:11,

                 from .pio/libdeps/nodemcu-32s/MQTT library for ESP32 _ESP-IDF__ID5670/src/transport.c:8:

/home/thomas/.platformio/packages/framework-espidf/components/esp32/include/rom/queue.h:1:2: warning: #warning rom/queue.h is deprecated, please use sys/queue.h instead [-Wcpp]

 #warning rom/queue.h is deprecated, please use sys/queue.h instead

  ^~~~~~~

Compiling .pio/build/nodemcu-32s/lib339/MQTT library for ESP32 _ESP-IDF__ID5670/transport_ws.o

In file included from .pio/libdeps/nodemcu-32s/MQTT library for ESP32 _ESP-IDF__ID5670/include/platform_esp32_idf.h:21,

                 from .pio/libdeps/nodemcu-32s/MQTT library for ESP32 _ESP-IDF__ID5670/include/platform.h:11,

                 from .pio/libdeps/nodemcu-32s/MQTT library for ESP32 _ESP-IDF__ID5670/src/transport_tcp.c:12:

/home/thomas/.platformio/packages/framework-espidf/components/esp32/include/rom/queue.h:1:2: warning: #warning rom/queue.h is deprecated, please use sys/queue.h instead [-Wcpp]

 #warning rom/queue.h is deprecated, please use sys/queue.h instead

  ^~~~~~~

In file included from .pio/libdeps/nodemcu-32s/MQTT library for ESP32 _ESP-IDF__ID5670/include/platform_esp32_idf.h:21,

                 from .pio/libdeps/nodemcu-32s/MQTT library for ESP32 _ESP-IDF__ID5670/include/platform.h:11,

                 from .pio/libdeps/nodemcu-32s/MQTT library for ESP32 _ESP-IDF__ID5670/src/transport_ssl.c:24:

/home/thomas/.platformio/packages/framework-espidf/components/esp32/include/rom/queue.h:1:2: warning: #warning rom/queue.h is deprecated, please use sys/queue.h instead [-Wcpp]

 #warning rom/queue.h is deprecated, please use sys/queue.h instead

  ^~~~~~~

In file included from .pio/libdeps/nodemcu-32s/MQTT library for ESP32 _ESP-IDF__ID5670/include/platform_esp32_idf.h:21,

                 from .pio/libdeps/nodemcu-32s/MQTT library for ESP32 _ESP-IDF__ID5670/include/platform.h:11,

                 from .pio/libdeps/nodemcu-32s/MQTT library for ESP32 _ESP-IDF__ID5670/src/transport_ws.c:5:

/home/thomas/.platformio/packages/framework-espidf/components/esp32/include/rom/queue.h:1:2: warning: #warning rom/queue.h is deprecated, please use sys/queue.h instead [-Wcpp]

 #warning rom/queue.h is deprecated, please use sys/queue.h instead

  ^~~~~~~

.pio/libdeps/nodemcu-32s/MQTT library for ESP32 _ESP-IDF__ID5670/src/transport_ws.c: In function 'ws_connect':

.pio/libdeps/nodemcu-32s/MQTT library for ESP32 _ESP-IDF__ID5670/src/transport_ws.c:100:53: error: '258EAFA5-E914-47DA-95CA-C5AB...' directive writing 36 bytes into a region of size between 33 and 64 [-Werror=format-overflow=]

     int key_len = sprintf((char*)client_key_b64, "%s258EAFA5-E914-47DA-95CA-C5AB0DC85B11", (char*)client_key);

                                                     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.pio/libdeps/nodemcu-32s/MQTT library for ESP32 _ESP-IDF__ID5670/src/transport_ws.c:100:19: note: 'sprintf' output between 37 and 68 bytes into a destination of size 64

     int key_len = sprintf((char*)client_key_b64, "%s258EAFA5-E914-47DA-95CA-C5AB0DC85B11", (char*)client_key);

                   ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

cc1: some warnings being treated as errors

*** [.pio/build/nodemcu-32s/lib339/MQTT library for ESP32 _ESP-IDF__ID5670/transport_ws.o] Error 1``

maxgerhardt · July 1, 2020, 11:32am

tom.holzwurm:

  # Using a library name
  MQTT library for ESP32 (ESP-IDF)
  # ... or using library Id
  5670
  # ... or depend on a specific version
  MQTT library for ESP32 (ESP-IDF)@1.0.0
  # Semantic Versioning Rules
  # http://docs.platformio.org/page/userguide/lib/cmd_install.html#description
  MQTT library for ESP32 (ESP-IDF)@^1.0.0
  MQTT library for ESP32 (ESP-IDF)@~1.0.0
  MQTT library for ESP32 (ESP-IDF)@>=1.0.0

Why are you declaring this library 6 times? Pick one declaration style

Also this lib is 2 years old and no longer maintained.

Well the compiler is right here, buffer overflow. That’s a critical error elligable for compilation abortion. The safe snprintf() should be used here with a more appropriate buffer size.

This line also matches 1:1 the file content in the repository

github.com

Andrey-m/MQTT-lib-for-esp-idf/blob/master/src/transport_ws.c#L100


      
                  ESP_LOGE(TAG, "Error read response for Upgrade header %s", ws->buffer);
                  return -1;
              }
              char *server_key = get_http_header(ws->buffer, "Sec-WebSocket-Accept:");
              if (server_key == NULL) {
                  ESP_LOGE(TAG, "Sec-WebSocket-Accept not found");
                  return -1;
              }
          
              unsigned char client_key_b64[64], valid_client_key[20], accept_key[32] = {0};
              int key_len = sprintf((char*)client_key_b64, "%s258EAFA5-E914-47DA-95CA-C5AB0DC85B11", (char*)client_key);
              mbedtls_sha1(client_key_b64, (size_t)key_len, valid_client_key);
              mbedtls_base64_encode(accept_key, 32,  &outlen, valid_client_key, 20);
              accept_key[outlen] = 0;
              ESP_LOGD(TAG, "server key=%s, send_key=%s, accept_key=%s", (char *)server_key, (char*)client_key, accept_key);
              if (strcmp((char*)accept_key, (char*)server_key) != 0) {
                  ESP_LOGE(TAG, "Invalid websocket key");
                  return -1;
              }
              return 0;
          }

So I don’t see

Where that comes from. What differences do you see in the files?

Anyways, what I would highly recommend is to drop this 2-year old library which is forked from the official GitHub - espressif/esp-mqtt: ESP32 mqtt component but use the official library instead. This is already integrated into ESP-IDF, no need to declare any additional libraries.

Just #include <mqtt_client.h> and follow an example and refer to the docs.

tom.holzwurm · July 1, 2020, 1:12pm

Thanks a lot for your reply and your effort:
It’s always puzzling for me which examples and which modules have to be chosen.
I wasn’t aware that MQTT is included in the framework directly which is of course much better and also explains the discrepancy of the files and the compilers behaviour.
My platformio.ini file came directly from the libraries example code, because I was looking for MQTT libraries related to espidf.
So if I get you right, you’re saying: stick with the examples mentioned in your reply, which should be sufficient, except there is no module available for e.g. special hardware ?
I’d appreciate a short reply and thanks again,
Regards Thomas