I did not try that, I may have an outdated Arduino IDE installed somewhere but I never used it for the ESP32. The sketch I am working on has some 3000+ lines of code now, so I rather would write a test sketch instead. I honestly will try that as a last resort only, given the efforts.
Yes, it is working fine. I am doing NTP requests regularly and the ESP is properly reacting on TCP requests on port 502, so the module seems to work and the network settings are okay.
Excerpt from my code:
...
// SPI and Ethernet libs for W5500 Ethernet module
#include <SPI.h>
#include <IPAddress.h>
#include <Ethernet.h>
// W5500 reset pin
#define RESET_P GPIO_NUM_26
...
// OTA update interface
#include <ArduinoOTA.h>
...
// Server on port 502 (MODBUS)
EthernetServer server(IPport);
...
// A UDP instance to let us send and receive packets over UDP
EthernetUDP Udp;
...
void setup()
{
...
// SPI CS line is GPIO_NUM_5
Ethernet.init(GPIO_NUM_5);
// Hard boot W5500 module
WizReset();
...
// start the Ethernet connection:
bool connected = false;
if(IPDHCP) // DHCP required?
{
Trace(0, "Trying to get an IP address using DHCP\n");
if (Ethernet.begin(mac) == 0)
{
Trace(0, "Failed to configure Ethernet using DHCP\n");
}
else connected = true;
}
// Connection established?
if(!connected)
{
// No. Either DHCP was not required or did not work.
// initialize the Ethernet device not using DHCP:
Ethernet.begin(mac, Reverse(IPaddress), Reverse(IPDNS), Reverse(IPgateway), Reverse(IPmask));
// Check for Ethernet hardware present
if (Ethernet.hardwareStatus() == EthernetNoHardware)
{
Trace(0, "Ethernet shield was not found. Sorry, can't run without hardware. :(\n");
}
else
{
if (Ethernet.linkStatus() == LinkOFF)
{
Trace(0, "Ethernet cable is not connected.\n");
}
else connected = true;
}
}
// print local IP address:
lIP = Ethernet.localIP();
Trace(TRACE_BASIC, "My IP address: %u.%u.%u.%u\n", lIP[0], lIP[1], lIP[2], lIP[3]);
// Update server to configured port
if(IPport) server = EthernetServer(IPport);
// Now start it
server.begin();
...
// Set up UDP for NTP requests
setenv("TZ", NTP_TZ, 1);
tzset();
Udp.begin(localPort);
...
// Start ArduinoOTA service
ArduinoOTA.begin(lIP, "myBridge", "xxxx", InternalStorage);
}
...
void loop()
{
...
// Check for OTA updates
ArduinoOTA.poll();
...
}
Not at all, as I wrote above. The flag does not have a single effect, that is why I asked where to look for the additional output.
For the time being I give up now. I could not find a reason why the OTA invitation was not answered. The packets looked good and were sent to the W5500 module, but inside the ESP32 nothing was happening.
So you did try the minimal sketch with just enough added on to initialize the W5500 interface? I think the issue here is that when the OTA library listens on port 3232, it does so on the general WiFi (or hardware ethernet interface), but it doesn’t know anything about the W5500 chip attached via SPI? There may be some extensions that must be done to the network stack there. I’m sure people at Issues · espressif/arduino-esp32 · GitHub would know more.
The ArduinoOTA library is claiming to work with Ethernet.h, but you may be right and that is only meant for the “real Arduino” flavor. I will give your suggestion another try - it would be wonderful to have a working OTA…
Eh revisiting the code a bit, isn’t it a bit suspicious that the ArduinoOTA class has a member
WiFiUDP? …Maybe a hack with #include <Ethernet.h> and replacing WiFiUDP the EthernetUDP class (ref) works?
You should be able to do that as temporary hack in C:\Users\<user>\.platformio\packages\framework-arduinoespressif32\libraries\ArduinoOTA\src\ArduinoOTA.h, given that all the interface functions are the same the code should still work
@maxgerhardt: I changed the ArduinoOTA.h file as you proposed - compile went without issues, but no change, unfortunately.
I am somewhat irritated by the location of the Arduino.h file you hinted me to - there is another in C:\Users\<user>\.platformio\lib\ArduinoOTA_ID6178\src. Most probably this is the one installed with the PlatformIO Library Manager. I experimentally de-installed it, but then got a couple of errors when compiling:
src\main.cpp: In function 'void setup()':
src\main.cpp:1412:50: error: 'InternalStorage' was not declared in this scope
ArduinoOTA.begin(lIP, "myBridge", "xxxx", InternalStorage);
^
src\main.cpp: In function 'void loop()':
src\main.cpp:1747:14: error: 'class ArduinoOTAClass' has no member named 'poll'
ArduinoOTA.poll();
^
Compiling .pio\build\az-delivery-devkit-v4\lib547\ArduinoOTA\ArduinoOTA.cpp.o
*** [.pio\build\az-delivery-devkit-v4\src\main.cpp.o] Error 1
C:\Users\Micha\.platformio\packages\framework-arduinoespressif32\libraries\ArduinoOTA\src\ArduinoOTA.cpp: In member function 'void ArduinoOTAClass::begin()':
C:\Users\Micha\.platformio\packages\framework-arduinoespressif32\libraries\ArduinoOTA\src\ArduinoOTA.cpp:120:9: error: 'WiFi' was not declared in this scope
WiFi.macAddress(mac);
^
C:\Users\Micha\.platformio\packages\framework-arduinoespressif32\libraries\ArduinoOTA\src\ArduinoOTA.cpp: In member function 'void ArduinoOTAClass::_runUpdate()':
C:\Users\Micha\.platformio\packages\framework-arduinoespressif32\libraries\ArduinoOTA\src\ArduinoOTA.cpp:257:5: error: 'WiFiClient' was not declared in this scope
WiFiClient client;
^
C:\Users\Micha\.platformio\packages\framework-arduinoespressif32\libraries\ArduinoOTA\src\ArduinoOTA.cpp:258:10: error: 'client' was not declared in this scope
if (!client.connect(_ota_ip, _ota_port)) {
^
C:\Users\Micha\.platformio\packages\framework-arduinoespressif32\libraries\ArduinoOTA\src\ArduinoOTA.cpp:267:36: error: 'client' was not declared in this scope
while (!Update.isFinished() && client.connected()) {
^
C:\Users\Micha\.platformio\packages\framework-arduinoespressif32\libraries\ArduinoOTA\src\ArduinoOTA.cpp:326:9: error: 'client' was not declared in this scope
client.print("OK");
^
C:\Users\Micha\.platformio\packages\framework-arduinoespressif32\libraries\ArduinoOTA\src\ArduinoOTA.cpp:341:27: error: 'client' was not declared in this scope
Update.printError(client);
^
*** [.pio\build\az-delivery-devkit-v4\lib547\ArduinoOTA\ArduinoOTA.cpp.o] Error 1
I am suspecting that I unintentionally may have mixed up the ArduinoOTA from the core lib with that that I installed with the Library Manager.
By the way: a platformio.ini line as -upload_flags= -p 65280 seems not to be honored, as the debug out put still has the 3232 port: I tried the 65280 port as well, as it is in the code of the downloaded ArduinoOTA library. Not that it had any effect at all…
Seems like this is the library to go. If the library in the framework is troublesome (it unfortunately has the same name…), try doing a
; use bleeding edge version
lib_deps =
ArduinoOTA=https://github.com/jandrassy/ArduinoOTA.git
or temporarily remove it from the framework-arduinoespressif32\libraries folder. It can be brought back any time by simply deleting framework-arduinoespressif32 and recompiling the project (auto redownload).
Thanks again! I missed the bit with “one option per line” for -upload_flags - at least that is sorted out now.
But regarding the basic issue I am back at square one. The library (defined in lib_deps now) was downloaded and installed okay, the sketch compiles without errors or warnings, but still I am getting the notorious
13:03:06 [DEBUG]: Options: {'timeout': 10, 'esp_ip': '192.168.199.40', 'host_port': 49341, 'image': '.pio\\build\\az-delivery-devkit-v4\\firmware.bin', 'host_ip': '0.0.0.0', 'auth': 'xxxx', 'esp_port': 3232, 'spiffs': False, 'debug': True, 'progress': True}
13:03:06 [INFO]: Starting on 0.0.0.0:49341
13:03:06 [INFO]: Upload size: 403200
Sending invitation to 192.168.199.40 ..........
13:04:46 [ERROR]: No response from the ESP
*** [upload] Error 1
Basically the ArduinoOTA library you’ve linked is entirely incompatible with the ArduinoOTA that is provided by default in the Arduino ESP32 package. It uses an entirely different upload mechanism too: The external library opens a HTTP Web Server on a TCP port and does HTTP Basic authentication. It doesn’t work with the espota tool. I use the curl tool (curl for Windows) to push the sketch. espota uses a UDP advertisment custom TCP stream to do the update.
Note that that might not be the best thing to use this custom ArduinoOTA which differs so much from these tools. A modification or expansion request to the Arduino-ESP32 provided ArduinoOTA might still make sense.
@maxgerhardt: thank you very much for your time and effort to help me out!
I tend to refrain from adapting your proprietary solution and rather go to attempt getting the core ArduinoOTA modified. I already opened a bug report yesterday ( https://github.com/espressif/arduino-esp32/issues/4122), but that attracted no reaction yet.
For now I will deactivate the OTA code in my sketch and am hoping a modification of the core will happen in time before my rollout…
YAY! I cannot believe it was that easy in the end! Your information and tips were the clue - I brute-forcely changed all occurrences of WiFi objects in ArduinoOTA.h and ArduinoOTA.cpp with their Ethernet.h complements (leaving out WiFi.mac(), as Ethernet.h seems not to have that) and:
The code change to the core ArduinoOTA will be a few lines only, but I am afraid it will need a preprocessor #define before #including <ArduinoOTA.h> to switch between WiFi and Ethernet.
Great to hear! I think for now you can even make it cleaner by just copying it into a new library, e.g. called ArduinoEthernetOTA or something, with all instances of the old class name ArduinoOTA being changed to new ones. Also adapt the library.properties accordingly. Then put the lib into the lib/ folder. Then you don’t have to modify framework files and have a portable solution.
A really great thing would of course be if you could tell the original ArduinoOTA library which objects to use for the socket, server and mDNS service, just like the old lib does with its templated clases. Templates really make sense here.
There is Ethernet::MACAddress which you can use to save the MAC address into a given 6-byte buffer of yours.
Works like a charm in the local net; my firewall is rejecting the TCP part after the initial UDP communication, though. Another topic anyway, so thank you so much again for your help!
While the original problem has been solved I’d like to add a new idea for a slightly different solution.
Opening up ports and forwarding them is always kind of messy. If your device would initiate an outgoing connection, that would be the way to go. But how? Simply: not with ArduinoOTA but there are other ways.
I always liked the idea of MQTT and yes, while MQTT adds another infrastructure component, this can add way more manageability to your devices.
Using premade open solutions like Homie you get a nice standardized and well documented (https://homieiot.github.io/) way of communication to and from your device and it adds another nice feature: OTA over MQTT. So you don’t need access to each device but simply to an MQTT server they all use in common. That one does not even need to be within the remote site but it could.
Homie adds autmatically sending telemetry data of the device and its connection, you may add additional data if you want to. Also there are libs to log stack traces on exceptions and reboot reasons. So if your device is remote, you can gather a lot of information remotely and can even send OTA Updates. This works with any size of installation as it relys only on MQTT which was built for large scale infrastructures - but also works like a charm in small environments.
Just my 2c
Here are those two libs (both available on github):
homie-esp8266 (old name - it supports ESP8266 and ESP32!)
EspSaveCrash
If you want to use Homie add these lines to platformio.ini lib_deps:
ArduinoJson @ 6.16.1
Homie
(There seems to be a small incompatibility with newest Homie lib and the newest ArduinoJson lib!)
Hi @miq19 … do you think your solution could work also for a different ethernet shield ENC28J60 …
I have already a modified version of the lib EthernetENC which works…
What do you thinK?
