ESP32 runs TWO copies of the firmware SIMULTANEOUSLY. How so?

kotyara12 · October 12, 2021, 7:53am

Good day!
I apologize in advance for the English, I translate it through googol.

Wrote firmware for ESP32 on ESP-IDF. Since Saturday I have come across an unpleasant problem - judging by the notifications in the telegrams, TWO copies of the program are working on the controller at the same time. What happened and why - I can’t understand …
I suspect that the OTA update was not completed correctly, and now both images, which are written to FLASH, are launched at once. Can this be ???
Another sign that two copies are running is that gpio_install_isr_service (0) returns an “already installed” error. The sensors connected to the controller also work and do not work at the same time.
But if you connect a laptop to the device and look at the logs, then you can see the logs from only one copy. Can not understand…
Where to look?

maxgerhardt · October 12, 2021, 8:02am

I’ve never seen this happen. The bootloader decides at boot-time which is the active app partition and boots into that. Not both at the same time.

Is it possible that you provide a minimal reproducable example of this behavior? Hard to judge what’s going on otherwise.

kotyara12 · October 12, 2021, 8:19am

Main firmware file:

/ *
   Bathroom ventilation control + control of freezing pipes under the floor
   -------------------------------------------------- -----------------------------------------------
   (с) 2021 Razzhivin Alexander | Razzhivin Alexander
   kotyara12@yandex.ru | https://kotyara12.ru | tg: @ kotyara1971
* /

#include "esp_log.h"
#include "project_config.h"
#include "rLog.h"
#include "rStrings.h"
#include "reLed.h"
#include "reNvs.h"
#include "reStates.h"
#include "reParams.h"
#include "reEvents.h"
#include "reEsp32.h"
#include "reWiFi.h"
#include "reSNTP.h"
#include "reMQTT.h"
#include "reSysInfo.h"
#include "reScheduler.h"
#include "reI2C.h"
#if CONFIG_PINGER_ENABLE
#include "rePinger.h"
#endif // CONFIG_PINGER_ENABLE
#if CONFIG_TELEGRAM_ENABLE
#include "reTgSend.h"
#endif // CONFIG_TELEGRAM_ENABLE
#if CONFIG_OPENMON_ENABLE
#include "reOpenMon.h"
#endif // CONFIG_OPENMON_ENABLE
#include "sensors.h"

extern "C" {

// Main function
void app_main (void)
{
  // Debug messages about memory allocation errors
  heap_caps_register_failed_alloc_callback (heap_caps_alloc_failed_hook);
  
  // Suppress the output of unnecessary system messages (wifi, tcpip, ...)
  esp_log_level_set ("wifi", ESP_LOG_NONE);
  esp_log_level_set ("event", ESP_LOG_ERROR);
  esp_log_level_set ("tcpip", ESP_LOG_ERROR);
  esp_log_level_set ("tcpip_adapter", ESP_LOG_ERROR);
  esp_log_level_set ("phy", ESP_LOG_ERROR);
  esp_log_level_set ("phy_version", ESP_LOG_ERROR);
  esp_log_level_set ("phy: phy_version", ESP_LOG_ERROR);
  esp_log_level_set ("i2c", ESP_LOG_ERROR);
  esp_log_level_set ("HTTP_CLIENT", ESP_LOG_NONE);
  esp_log_level_set ("MQTT_CLIENT", ESP_LOG_NONE);
  esp_log_level_set ("TRANS_TCP", ESP_LOG_NONE);

  // Initialize "our" log and display the firmware version
  rlog_empty ();
  rloga_i ("Firmware initialization, version% s", APP_VERSION);

  // Initialize LEDs
  ledSysInit (CONFIG_GPIO_SYSTEM_LED, true, nullptr);
  ledSysOn (false);

  // Start the main event loop
  eventLoopCreate ();

  // Initialize the device tracking system
  statesInit (true);

  // Initialize the parameter storage system
  paramsInit ();

  // Initialize the I2C bus
  initI2C (0, CONFIG_I2C_PORT0_SDA, CONFIG_I2C_PORT0_SCL, CONFIG_I2C_PORT0_PULLUP, CONFIG_I2C_PORT0_FREQ_HZ);
  scanI2C (0);
  initI2C (1, CONFIG_I2C_PORT1_SDA, CONFIG_I2C_PORT1_SCL, CONFIG_I2C_PORT1_PULLUP, CONFIG_I2C_PORT1_FREQ_HZ);
  scanI2C (1);

  // Register the "minute timer" and scheduling service, but don't start
  schedulerEventHandlerRegister ();

  #if CONFIG_PINGER_ENABLE
  // Register a service for periodic checking of external servers and access to the Internet
  pingerEventHandlerRegister ();
  #endif // CONFIG_PINGER_ENABLE

  // Start and register the time synchronization service
  sntpTaskCreate (true);

  // Start and register the MQTT client
  mqttTaskStart (true);

  // Register event handlers for parameters
  paramsEventHandlerRegister ();

  // Start and register the notification service in Telegram
  #if CONFIG_TELEGRAM_ENABLE
  tgTaskCreate (true);
  #endif // CONFIG_TELEGRAM_ENABLE

  // Start the service for sending data to open-monitoring.online
  #if CONFIG_OPENMON_ENABLE
  omTaskCreate (false);
  #endif // CONFIG_OPENMON_ENABLE

  // Start the service for reading data from sensors and control of ventilation and heating
  sensorsTaskStart ();

  // Connect to WiFi
  if (wifiStartSTA ()) {
    // Telegram notification about successful launch
    #if CONFIG_TELEGRAM_ENABLE
      // We are waiting for the WiFi connection and time synchronization
      if (statesInetWait (portMAX_DELAY) && statesTimeWait (portMAX_DELAY)) {
        tgSend (false, CONFIG_TELEGRAM_DEVICE, CONFIG_MESSAGE_TG_VERSION,
          APP_VERSION, getResetReason (), getResetReasonRtc (0), getResetReasonRtc (1));
      };
    #endif // CONFIG_TELEGRAM_ENABLE
  } else {
    ledSysBlinkOn (1, 100, 250);
  };
}

}

kotyara12 · October 12, 2021, 8:33am

The most interesting thing … everything worked perfectly fine until Saturday. Of course, I will try to reflash to another crystal in the evening, but I want to understand the reason

kotyara12 · October 12, 2021, 10:15am

Still, the problem was in the OTA sections. After flashing to another controller of the same type, the problem went away by itself. If you change the partition table by shifting the OTA addresses, then the problem is also solved. This means that in an exceptional case, firmwares from different partitions can still be launched simultaneously.