I made some local modifications to the arduino-esp32 core I am depending from. With the recent platform update I decided to throw away my local copy and installed the update. I did the modifications again, but that seems to have no impact, I am getting the standard core instead.
So I am wondering if the platform code is being cached somewhere by PlatformIO and if so, how to have my modifications applied to it?
Which file did you modify exactly? When using the project tasks “Clean” and “Advanced → Verbose Build”, which files are compiled?
It is esp32-hal-uart.c. I did a “Clean” - but I do not know where to find “Advanced”? I can confirm that the file was compiled, I can see the freshly created esp32-hal-uart.c.o file in the file system.
Per link above.
Oops… The “Quick Access” window was maximized, so I missed the “Project Tasks” above it. Sorry. The verbose build has run now.
The output busted the terminal’s scroll capacity, though. Can I redirect the output into a file or such?
Wait a second: it was in the saved scroll lines in fact:
xtensa-esp32-elf-gcc -o .pio\build\RTU06example\FrameworkArduino\esp32-hal-uart.c.o -c -std=gnu99 -Wno-old-style-declaration -Os -g3 -Wall -nostdlib -Wpointer-arith -Wno-error=unused-but-set-variable -Wno-error=unused-variable -mlongcalls -ffunction-sections -fdata-sections -fstrict-volatile-bitfields -Wno-error=deprecated-declarations -Wno-error=unused-function -Wno-unused-parameter -Wno-sign-compare -fstack-protector -fexceptions -Werror=reorder -DPLATFORMIO=50101 -DARDUINO_ESP32_DEV -DLOG_LEVEL=LOG_LEVEL_ERROR -DESP32 -DESP_PLATFORM -DF_CPU=240000000L -DHAVE_CONFIG_H -DMBEDTLS_CONFIG_FILE=\"mbedtls/esp_config.h\" -DARDUINO=10805 -DARDUINO_ARCH_ESP32 -DARDUINO_VARIANT=\"esp32\" "-DARDUINO_BOARD=\"AZ-Delivery ESP-32 Dev Kit C V4\"" -IC:\Users\Micha\.platformio\packages\framework-arduinoespressif32\tools\sdk\include\config -IC:\Users\Micha\.platformio\packages\framework-arduinoespressif32\tools\sdk\include\app_trace -IC:\Users\Micha\.platformio\packages\framework-arduinoespressif32\tools\sdk\include\app_update -IC:\Users\Micha\.platformio\packages\framework-arduinoespressif32\tools\sdk\include\asio -IC:\Users\Micha\.platformio\packages\framework-arduinoespressif32\tools\sdk\include\bootloader_support -IC:\Users\Micha\.platformio\packages\framework-arduinoespressif32\tools\sdk\include\bt -IC:\Users\Micha\.platformio\packages\framework-arduinoespressif32\tools\sdk\include\coap -IC:\Users\Micha\.platformio\packages\framework-arduinoespressif32\tools\sdk\include\console -IC:\Users\Micha\.platformio\packages\framework-arduinoespressif32\tools\sdk\include\driver -IC:\Users\Micha\.platformio\packages\framework-arduinoespressif32\tools\sdk\include\efuse -IC:\Users\Micha\.platformio\packages\framework-arduinoespressif32\tools\sdk\include\esp-tls -IC:\Users\Micha\.platformio\packages\framework-arduinoespressif32\tools\sdk\include\esp32 -IC:\Users\Micha\.platformio\packages\framework-arduinoespressif32\tools\sdk\include\esp_adc_cal -IC:\Users\Micha\.platformio\packages\framework-arduinoespressif32\tools\sdk\include\esp_event -IC:\Users\Micha\.platformio\packages\framework-arduinoespressif32\tools\sdk\include\esp_http_client -IC:\Users\Micha\.platformio\packages\framework-arduinoespressif32\tools\sdk\include\esp_http_server -IC:\Users\Micha\.platformio\packages\framework-arduinoespressif32\tools\sdk\include\esp_https_ota -IC:\Users\Micha\.platformio\packages\framework-arduinoespressif32\tools\sdk\include\esp_https_server -IC:\Users\Micha\.platformio\packages\framework-arduinoespressif32\tools\sdk\include\esp_ringbuf
-IC:\Users\Micha\.platformio\packages\framework-arduinoespressif32\tools\sdk\include\esp_websocket_client -IC:\Users\Micha\.platformio\packages\framework-arduinoespressif32\tools\sdk\include\espcoredump -IC:\Users\Micha\.platformio\packages\framework-arduinoespressif32\tools\sdk\include\ethernet -IC:\Users\Micha\.platformio\packages\framework-arduinoespressif32\tools\sdk\include\expat -IC:\Users\Micha\.platformio\packages\framework-arduinoespressif32\tools\sdk\include\fatfs -IC:\Users\Micha\.platformio\packages\framework-arduinoespressif32\tools\sdk\include\freemodbus -IC:\Users\Micha\.platformio\packages\framework-arduinoespressif32\tools\sdk\include\freertos -IC:\Users\Micha\.platformio\packages\framework-arduinoespressif32\tools\sdk\include\heap -IC:\Users\Micha\.platformio\packages\framework-arduinoespressif32\tools\sdk\include\idf_test -IC:\Users\Micha\.platformio\packages\framework-arduinoespressif32\tools\sdk\include\jsmn -IC:\Users\Micha\.platformio\packages\framework-arduinoespressif32\tools\sdk\include\json -IC:\Users\Micha\.platformio\packages\framework-arduinoespressif32\tools\sdk\include\libsodium -IC:\Users\Micha\.platformio\packages\framework-arduinoespressif32\tools\sdk\include\log -IC:\Users\Micha\.platformio\packages\framework-arduinoespressif32\tools\sdk\include\lwip -IC:\Users\Micha\.platformio\packages\framework-arduinoespressif32\tools\sdk\include\mbedtls -IC:\Users\Micha\.platformio\packages\framework-arduinoespressif32\tools\sdk\include\mdns -IC:\Users\Micha\.platformio\packages\framework-arduinoespressif32\tools\sdk\include\micro-ecc -IC:\Users\Micha\.platformio\packages\framework-arduinoespressif32\tools\sdk\include\mqtt -IC:\Users\Micha\.platformio\packages\framework-arduinoespressif32\tools\sdk\include\newlib -IC:\Users\Micha\.platformio\packages\framework-arduinoespressif32\tools\sdk\include\nghttp -IC:\Users\Micha\.platformio\packages\framework-arduinoespressif32\tools\sdk\include\nvs_flash -IC:\Users\Micha\.platformio\packages\framework-arduinoespressif32\tools\sdk\include\openssl -IC:\Users\Micha\.platformio\packages\framework-arduinoespressif32\tools\sdk\include\protobuf-c -IC:\Users\Micha\.platformio\packages\framework-arduinoespressif32\tools\sdk\include\protocomm -IC:\Users\Micha\.platformio\packages\framework-arduinoespressif32\tools\sdk\include\pthread -IC:\Users\Micha\.platformio\packages\framework-arduinoespressif32\tools\sdk\include\sdmmc -IC:\Users\Micha\.platformio\packages\framework-arduinoespressif32\tools\sdk\include\smartconfig_ack -IC:\Users\Micha\.platformio\packages\framework-arduinoespressif32\tools\sdk\include\soc -IC:\Users\Micha\.platformio\packages\framework-arduinoespressif32\tools\sdk\include\spi_flash -IC:\Users\Micha\.platformio\packages\framework-arduinoespressif32\tools\sdk\include\spiffs -IC:\Users\Micha\.platformio\packages\framework-arduinoespressif32\tools\sdk\include\tcp_transport -IC:\Users\Micha\.platformio\packages\framework-arduinoespressif32\tools\sdk\include\tcpip_adapter -IC:\Users\Micha\.platformio\packages\framework-arduinoespressif32\tools\sdk\include\ulp -IC:\Users\Micha\.platformio\packages\framework-arduinoespressif32\tools\sdk\include\unity -IC:\Users\Micha\.platformio\packages\framework-arduinoespressif32\tools\sdk\include\vfs -IC:\Users\Micha\.platformio\packages\framework-arduinoespressif32\tools\sdk\include\wear_levelling -IC:\Users\Micha\.platformio\packages\framework-arduinoespressif32\tools\sdk\include\wifi_provisioning -IC:\Users\Micha\.platformio\packages\framework-arduinoespressif32\tools\sdk\include\wpa_supplicant -IC:\Users\Micha\.platformio\packages\framework-arduinoespressif32\tools\sdk\include\xtensa-debug-module -IC:\Users\Micha\.platformio\packages\framework-arduinoespressif32\tools\sdk\include\esp-face -IC:\Users\Micha\.platformio\packages\framework-arduinoespressif32\tools\sdk\include\esp32-camera -IC:\Users\Micha\.platformio\packages\framework-arduinoespressif32\tools\sdk\include\esp-face -IC:\Users\Micha\.platformio\packages\framework-arduinoespressif32\tools\sdk\include\fb_gfx -IC:\Users\Micha\.platformio\packages\framework-arduinoespressif32\cores\esp32 -IC:\Users\Micha\.platformio\packages\framework-arduinoespressif32\variants\esp32 C:\Users\Micha\.platformio\packages\framework-arduinoespressif32\cores\esp32\esp32-hal-uart.c
It seems to have been built, so I probably will have to look for other reasons.
Indeed it is that file that is built.
And that is also the file that you modified?
You can add
#error "Yep mods were built"
at the top of that c file and press build again to make absolutely sure that your modified file is built. The build must then fail with the custom error message above.
Yes, that is the only copy I have left on my system and it is the one I modified. Strange.
There is one thing left: the change is effective only if a certain preprocessor define is existing. I am currently on the hunt to find its definition.
Well, I have to come back to the original suspicion. I manually removed the blocking preprocessor define to no avail.
The change is minor, but has quite an impact. The original implementation of the UART handling sets a threshold of 112 bytes in buffer before a program is notified by interrupt that UART data is available. My application typically has messages much shorter and a critical timing (MODBUS), so I had the interrupt firing at every single character:
void uartEnableInterrupt(uart_t* uart)
{
UART_MUTEX_LOCK();
uart->dev->conf1.rxfifo_full_thrhd = 1; // was 112!
uart->dev->conf1.rx_tout_thrhd = 2;
uart->dev->conf1.rx_tout_en = 1;
uart->dev->int_ena.rxfifo_full = 1;
uart->dev->int_ena.frm_err = 1;
uart->dev->int_ena.rxfifo_tout = 1;
uart->dev->int_clr.val = 0xffffffff;
esp_intr_alloc(UART_INTR_SOURCE(uart->num), (int)ESP_INTR_FLAG_IRAM, _uart_isr, NULL, &uart->intr_handle);
UART_MUTEX_UNLOCK();
}
It has been working since, also with other users having applied the same patch. 
Can you do a
#include <soc/uart_struct.h>
in the code and then print out the value of
UART0.conf1.rxfifo_full_thrhd
(or the UART peripheral you’re working with, 0 to 2) to verify that this value is now indeed 1?
You may also be able to modify it directly with UART0.conf1.rxfifo_full_thrhd = 1;
If the value is indeed 1 but the behavior doesn’t change, then maybe the new Arduino core version (recently 1.0.5 was released) has changed something else. In that case one can go back with platform_packages. Also note that one can use Advanced scripting to implement an auto-patch feature.
Thanks! Doing as you advised resulted in a FIFO threshold reported as 1, as was supposed to.
So this trace seems to have been misleading. Hell knows where to look else… 
Looks like the recent arduino-esp32 core has introduced something weird here. I found spurious 0xFF bytes being sprinkled into the received UART data, between the bytes of the message proper. Not exactly between every byte, but depending on the timing sometimes more, sometimes less.
Well, puzzling time again.
Available Arduino framework versions for fallback are listed at https://api.registry.platformio.org/v3/packages/platformio/tool/framework-arduinoespressif32, e.g.
platform_packages =
framework-arduinoespressif32@~3.10004.0
takes you back to the previous 1.0.4 version. You’ll have to reapply the patch in the framework file though (or modify the right UART’s register after the initilization has happened, as shown per code above).
Thanks, will try that tomorrow - if the issue is gone, I will know it really is caused by the new core.
I found the reason being another flaw in esp32-hal-uart.c, where available() can return a 1 even if there is no byte available for read() yet. This has been mentioned (and even a fix was proposed) by another user in this arduino-esp32 issue report, but was nonchalantly ignored by the maintainers of the core - as I have seen several times in the past 
.
To not have to patch the core again and again, I now am doing like
if (serial.available()) {
int c = serial.read();
if (c >= 0) {
... (use c) ...
}
}
A bit clumsy, but working at least.