Serial monitor prints regular letters, internally code sees �?H!�?

1

Hi,

I’m having a problem that is rather difficult to explain, let alone illustrate using code. What I mean by that is that when I try to boil down the code to just the items that have the problem, it doesn’t actually show me the problem. If there’s anyone who’s interested in accessing my github page to see of they can find the problem, be sure to let me know.

Hardware: Adafruit ESP32 and DOIT DevKit 1

Here’s what I’m building:

I have an application that reads wifi settings from a json (ArduinoJson 6) file to set up a webserver (ASyncWebServer).

Original json file:

{
    "wifi_ssid": "shufobox",
    "wifi_password": "",
    "shutter_duration": 20,
    "focus_mode": "SIMULTANEOUS"
}

On there I have an html page that gives the user the option to change the ssid and password, among other things. When I submit and save these settings back into a json file, the file is perfectly normal and all the settings are exactly what I expected they would be when printing to the browser or Serial prompt.

Here’s what the saved json file looks like:

{
  "wifi_ssid": "shufoboxxes",
  "wifi_password": "shootit",
  "shutter_duration": 25,
  "focus_mode": "SIMULTANEOUS"
}

However, after restarting the webserver, the SSID (even though the Serial prompt shows me the correct one) is a nonsense word with characters I don’t even recognize. If I load the html page again (assuming I can connect), there seems to be random text where the SSID and password fields should be:

<input type="text" name="wifi_ssid" placeholder="AP SSID" required value="�?H!�?" />
<input type="text" name="wifi_password" placeholder="AP password" value="" />

I’m not sure where to start looking for the error. I thought it was perhaps that I’m restarting the board before it’s finished writing, but I do have a clean and finished JSON file that I can read on the next boot. Could it be that variables are lingering in memory and therefor are scrambling up my settings on future boots? I reset the board and even unplugged it but that doesn’t seem to change it.

Any help would be greatly appreciated, and again I’m happy to give access to the github repo.
Thanks,
Jan

2

So you

  • load SPIFFS with initial JSON file
  • read it, modify it, save it back
  • first readback yields same content
  • after reboot and readback the contents are scrambled

right? Without having the code it’s hard to guess the bug. Especially when you say that a minimal firmware does not show the problem. Can you show the full code?

3

It has been brought to my attention that it could possibly be a character encoding problem.

The original settings.json was written in PlatformIO and apparently it’s in UTF-8 (according to notebook++). It doesn’t even need to be from a json, it can just be from variables that I hard-code.

However, when I allow a user to tyoe a new SSID and password through a http form and the software uses it to set up a new softAP() the program interprets it as �?H!�?, or similar style characters. That means the software can’t determine that it’s 1-byte, 2-byte or 4-byte characters (I think) and is thus seeing this nonsense. The reason I couldn’t find the problem initially is because the Serial monitor prints this variable out correctly, meaning it interprets it differently from the softAP() method.

I do have the following tag in my HTML document:
<meta charset="utf-8" />

Is there a way to safeguard against this? Can characters be converted in a way that avoids this problem?