There was no change involving ArduinoOTA at all applied to the sketch code or build environment (Espressif8266 core 2.6.3, as was installed by PlatformIO). The ArduinoOTA onError() callback reports an OTA_BEGIN_ERROR (1) on the ESP8285.
In the rare cases of OTA working still, there is no indication of any problem.
Several other TCP services are running okay as before, like Telnet, Fauxmo (Philips Hue emulation) and a Modbus server. The same sketch is doing OTA nicely on another’s device in his network.
I uninstalled and installed again the ESP8266 framework to no avail - I doubted it would change a thing, but did not want to let anything untested.
The only change I am aware of is that my router (an AVM Fritz.box 6591) got an OS update from 7.13 to 7.21 recently.
Concluding, indications are it may be a network issue, but I have no idea left as of where to look.
If I omit the OTA password, the upload is successful every time.
I did some more investigation and it looks like the MD5 hash espota.py is using does not match that the ArduinoOTA from the ESP8266 core is generating. This has been working previously, so either one must have been changed recently.
Is there a way to see when the last updates of the respective files were? The OS time stamps are 2020-31-07 for both, but these files must have been working until about 3-4 weeks ago.
I moved the code to a NodeMCU unit to have Serial output. I snapped the UDP packets going back and forth, but if the second hadn’t AUTH on it, I could not make heads or tails from it.
Now only the error message follows. The NodeMCU gives me “End failed” as compared to the ESP8285’s “Begin failed”.
Various online MD5 calculators are giving me ad347226dcc1d205f58e693925c28783 for the given password XXXX, but I do not see that in the packet sent as response to the AUTH packet.
The protocol is a bit more complicate here, see code
The first packet is parsed with the command number, which must be one of
followed by the OTA port number, the length of the firwmare and an MD5 hash. So that’s what’s in
0 → U_FLASH
34143 → OTA port
323152 → Firmware update length
4a4f4f69612fc82d8b9a92062e67d3ea → MD5 Hash of firmware update data (not authentication).
The hash is then just internally saved as _md5. Since the _password was previously set by calling ArduinoOTAClass::setPassword or setPasswordHash(), it triggers this code
Which uses the micros() value as a “random value”, MD5’s the String represenetation of that number, and sends it back with the “AUTH” prefix. So that’s the packet
And b02aa39c36370598d1f151010cf81f14 is the hash of micros() at that point there, and it saves the hash as _nonce.
Then it waits for another packet and does
So here it reads two strings, the first space seperated and then the rest until the newline.
For the packet
It reads out the values
86dbad6d84f4615583318014257ab9d3 → cnonce
2c48186f2051f60f2c101ec4788ab8d1 → response
where _password is already the MD5 of the original hash and _nonce was the nonce generated in the first answer, so in this case b02aa39c36370598d1f151010cf81f14 . So the final challenge is
The ESP8266 OTA logic, if it internally has the MD5 hash of the password, generates a random number “nonce” and sends that, then asks esptool.py to
This enables the EPS8266 to also calculate the same string (since it has its original password and receives cnonce from the client again) and check whether they get the same result. If they do not get the same result, then esptool.py must have used a different value for original_password and thus the hashes don’t match. If esptool.py does all the calculations correctly, the hashes must turn out correct.
So for your example, if the password is ASCII XXXX and the ESP8266 sends back b02aa39c36370598d1f151010cf81f14 as the _nonce, and esptool.py has chosen 86dbad6d84f4615583318014257ab9d3 as its cnonce, then it should compute
and the hash of that that again is 2c48186f2051f60f2c101ec4788ab8d1 per https://md5generator.de/.
So esptool.py does everything correctly here because we can also see it that that’s what it sends as the response hash
Since the ESP8266 is still rejecting it, it either generally calculates something wrong in this protocol (which is highly unlikely and also there haven’t been changes to the ArduinoOTA library since 8 months, History for libraries/ArduinoOTA - esp8266/Arduino · GitHub) or it internally has a different saved password(hash).
Are you able to reproduce this in a minimal example that any OTA uploads that are password protected fail? With what exact platformio.ini and code is that?
Hi Max, thank you for being the one helping me again!
The sketch is quite small already, as I am using it to nail down exactly this issue. It will print out the current time every 10s only - you might have to fill in another NTP server in the fist #define:
// Copyright 2021 blah
#include <Arduino.h>
#include <ArduinoOTA.h>
#include <time.h>
#include <ESP8266WiFi.h>
// NTP definitions
#define MY_NTP_SERVER "fritz.box"
#define MY_TZ "CET-1CEST-2,M3.5.0/2:00,M10.5.0/3:00"
#define C_SSID "xxxxx"
#define C_PWD "xxxxx"
#define WHOAMI "OTAtest"
// -----------------------------------------------------------------------------
// Setup WiFi
// -----------------------------------------------------------------------------
void wifiSetup(const char *hostname) {
// Set WIFI module to STA mode
WiFi.mode(WIFI_STA);
WiFi.hostname(hostname);
// Connect
WiFi.begin(C_SSID, C_PWD);
// Wait for connection. ==> We will hang here in RUN mode forever without a WiFi!
while (WiFi.status() != WL_CONNECTED) {
delay(50);
}
}
// -----------------------------------------------------------------------------
void setup() {
// STart WiFI
wifiSetup(WHOAMI);
// ArduinoOTA setup
ArduinoOTA.setHostname(WHOAMI); // Set OTA host name
ArduinoOTA.setPassword("XXXX"); // Set OTA password
ArduinoOTA.begin(); // start OTA scan
ArduinoOTA.onError([](ota_error_t error) {
Serial.printf("Error[%u]: ", error);
if (error == OTA_AUTH_ERROR) Serial.println("Auth Failed");
else if (error == OTA_BEGIN_ERROR) Serial.println("Begin Failed");
else if (error == OTA_CONNECT_ERROR) Serial.println("Connect Failed");
else if (error == OTA_RECEIVE_ERROR) Serial.println("Receive Failed");
else if (error == OTA_END_ERROR) Serial.println("End Failed");
});
// Start NTP
configTime(MY_TZ, MY_NTP_SERVER);
// Init Serial
Serial.begin(115200);
Serial.println(".");
Serial.println("__OK__");
}
void loop() {
static uint32_t tick = millis();
// Check for OTA update requests
ArduinoOTA.handle();
if (millis() - tick > 10000) {
time_t now = time(NULL);
tm tm;
localtime_r(&now, &tm);
Serial.printf("%02d:%02d:%02d\n", tm.tm_hour, tm.tm_min, tm.tm_sec);
tick = millis();
}
}
py>esptool.py flash_id
esptool.py v3.0
Found 2 serial ports
Serial port COM9
Connecting....
Detecting chip type... ESP8266
Chip is ESP8266EX
Features: WiFi
Crystal is 26MHz
MAC: 60:01:94:0d:XX:XX
Uploading stub...
Running stub...
Stub running...
Manufacturer: ef
Device: 4016
Detected flash size: 4MB
Hard resetting via RTS pin...
)
Sounds more like a flash configuration issue to me?
Whith what exact platformio.ini and chip do you expect the OTA to work but it doesn’t?
The devices are Chinese smart plugs with (“Gosund SP-1” and “Maxcio DE-W004” brands) equipped with an ESP8285. These devices do have 1MB flash, and my application is using a LittleFS file system - therefore the 64kB.
These used to work flawlessly with the above platformio.ini; I only picked a NodeMCU for testing to have Serial output, as the smart plug devices are using all their GPIOs for other purposes.
You are right in that this device has a 4MB flash as well:
esptool.py v2.6
Found 1 serial ports
Serial port COM3
Connecting....
Detecting chip type... ESP8266
Chip is ESP8266EX
Features: WiFi
MAC: 48:3f:da:7e:48:34
Uploading stub...
Running stub...
Stub running...
Manufacturer: 20
Device: 4016
Detected flash size: 4MB
Hard resetting via RTS pin...
Unfortunately I cannot use esptool on the smart plug devices to verify and have to rely on findings on the Internet. So far everything was fine…
Yes, I do. One of each brand, to be precise. The other 10+ are on site and running. This exactly is the reason I need OTA in the first row.
Sure, but each device has a different name. The “NTPtest” f.i. is another than the “OTAtest” I am currently using. WiFi host name and ArduinoOTA host name are identical, if that matters.
By the way: commenting out the two lines like you did does not change a bit, unfortunately.
So there’s a network problem from the ESP8285 side to the PC? Because in start update it should at least trigger the UDP packet
to be sent from the ESP8285 to the PC. Can you put a print statement there to verify that the ESP8285 tries to send this packet, but confirm via wireshark that no such packet is seen?
