Hi Paul,
It’s no problem, I like a challenge. I’m between contracts at the moment, so when Mrs D allows, I have time to “play”. 
I ordered 5! One costs £5.79 but 5 costs £3.00 extra. AZ Delivery are a good company to deal with, I’ve bought from them before.
I’m puzzled that notepad sees ascii values though. However, I’ll get to the bottom of this problem!
Cheers,
Norm.
Hi Paul,
It came to me in the night! The print() and println() functions convert their parameters to text! For the various readXxx() functions, we need to read binary from the file.
Try this… change these two function calls:
// ND - Write a byte, and an iint to the SD card. I deleted the original text written.
myFile.print(0xA5);
myFile.print ((int)12345);
// ND Ends - End of this bit. More later!
To this:
// ND - Write a byte, and an iint to the SD card. I deleted the original text written.
myFile.write(0xA5);
myFile.write((int)12345);
// ND Ends - End of this bit. More later!
The read back should work.
Cheers,
Norm.
Well Norman I’m pleased to report that Fred is reporting the correct value 165 but unfortunately Wilma has the following error
src\main.cpp: In function ‘void setup()’:
src\main.cpp:99:28: warning: large integer implicitly truncated to unsigned type [-Woverflow]
myFile.write ((int)12345);
If you shorten the int to 123 it compiles okay and gives the correct value.
If you come up with any more blinding flashes of inspiration please let me know
Thanks
Paul
Hmmm. Given that it was an integer when written, is defined as an integer variable to be read into, I’m a little puzzled (again) as to why it thinks it will be truncated.
Looks like I mighgt need the new hardware after all!
I’ll get back to you ASAP.
Cheers,
Norm.
Hi Norman
well I have been playing today as I’ve got the day off, and I have hit upon a way of doing it I think?
I will do some more testing and see if it will do the job, I’m sure there are a million ways better to do the this, but this is about maximum for my learning level at the moment.
Thank you again for all your help and encouragement and going the extra mile, I hope you will make use of the SD cards when you get them, may I encourage you to get a ESP32 or 8266, they are both super fun to play with and to create many super projects around the house.
The code is very simple
#include <M5Stack.h>
#include <TaskScheduler.h>
#include <SPI.h>
#include <SD.h>
void SDW();
void SDR();
typedef struct Board1 {
byte sensornumber; // Sensor number provided by e.g. Sensor=3
String Time; //Time
float Temperature; // DHT Temperature
float Humidity; // DHT Humidity
float BMP_Temperature; // BMP Temperature
float BMP_Pressure; // Bmp Pressure
int Light; // Light
} Board1;
struct Board1 Board_1;
File myFile;
void setup() {
Serial.begin(115200);
M5.begin();
M5.Lcd.fillScreen(DARKGREEN);
M5.Lcd.setCursor(10, 10);
M5.Lcd.setTextColor(WHITE);
M5.Lcd.setTextSize(3);
M5.Lcd.println("Starting!");
M5.update();
Board_1.BMP_Pressure=1010.2;
Board_1.BMP_Temperature=26.6;
Board_1.Humidity=77.77;
Board_1.Light=555;
Board_1.sensornumber=44;
Board_1.Temperature=25.5;
Board_1.Time="10:20:30";
SDW();
SDR();
}
void loop() {
// put your main code here, to run repeatedly:
}
//####################################################################################
void SDW(){
auto filename="/Data3.csv";
auto myFile = SD.open(filename, FILE_WRITE );
if(myFile){
myFile.println(Board_1.sensornumber);
//myFile.print(",");
myFile.println(Board_1.Time);
//myFile.print(",");
myFile.println(Board_1.Temperature);
//myFile.print(",");
myFile.println(Board_1.Humidity);
//myFile.print(",");
myFile.println(Board_1.BMP_Temperature);
//myFile.print(",");
myFile.println(Board_1.BMP_Pressure);
//myFile.print(",");
myFile.println(Board_1.Light);
//myFile.close();
Serial.println("Done");
}
else{
Serial.println("Error opening Data1.csv");
}
}
//#########################################################################
void SDR(){
float a1,a2,a3,a4;
int sen,lig;
String s,tim;
myFile = SD.open("/Data3.csv");
if(!myFile){
Serial.println("Failed to open file for reading");
return;
}
else
{
Serial.println(myFile.size());
Serial.println("File is open file for reading");
s=myFile.readStringUntil('\n');
sen= s.toInt();
s=myFile.readStringUntil('\n');
tim = s;
s=myFile.readStringUntil('\n');
a1=s.toFloat();
s=myFile.readStringUntil('\n');
a2=s.toFloat();
s=myFile.readStringUntil('\n');
a3=s.toFloat();
s=myFile.readStringUntil('\n');
a4=s.toFloat();
s=myFile.readStringUntil('\n');
lig=s.toInt();
myFile.close();
Serial.println(sen);
Serial.println(tim);
Serial.println(a1);
Serial.println(a2);
Serial.println(a3);
Serial.println(a4);
Serial.println(lig);
}
}That code will work. Yes. However, your original query was about reading data back into a structure. 
Strings are something to be careful about using in a microcontroller. When you “do things with them” they internally allocate a new chunk of Static RAM, which is very scarce – only 2KB, copy themselves over then get rid of the old version.
This could cause data in RAM to overwrite other variables and corrupt your program. Your code will probably suffer from that at some point when you have more going on with Strings. Just so you are aware.
There is advice (somewhere) from someone involved with Arduino Libraries, who advises they are “best avoided”. I can post the URL when I’ve looked it up.
When my bits arrive, I’ll have another look at this problem. I’ll be in touch!
Cheers,
Norm.
PS. Found it. The String class uses a lot of dynamic memory allocation. This means that there could be runtime errors when used on AVR microcontrollers with minimal available RAM. The author of the Arduino JSON library, Benoit Blanchon, has a few warnings and pointers as to why you should never use the String class. You can read the article, or at least the “Disclaimer,” at https://blog. benoitblanchon.fr/arduino-json-v5-0/#disclaimer and then, if you so wish, avoid the use of the String class in your Arduino code.
Afternoon All, Hi @meathome2017.
I did say that I liked a challenge, however I never thought that the Arduino SD card system, at least for an Uno board, was so fsck'd! (Unix joke!)
Here’s what I found:
FILE_WRITE will open it at the end, it is effectively FILE_APPEND (which doesn’t exist in my SD library.)write() to write a byte, as in fred, works fine and my readByte() can read it back happily.write() function’s docs on the Arduino Reference site, the number of bytes written is always 1 no matter what type of variable data you write. So int wilma = 123245; myFile.write(wilma); only writes one byte.int you have to myFile.write((uint8_t *)&wilma, sizeof(int)); which will write an int, and – even better – that int value can be read back happily!I am not happy with the code in the ReadWrite example sketch, so I rewrote it, with lots of commentary as it was going along, the working version is as follows:
//----------------------------------------------------------
// SD Card "exerciser". Based on the ReadWrite sketch under
// File->Examples->SD->ReadWrite in the Arduino IDE. There
// are problems with that sketch!
//
// Pinout:
//
// D13 = CLK
// D12 = MISO
// D11 = MOSI
// D4 = Card Select pin.
//
// Opening with FILE_WRITE appends to the file. (Fixed below)
//
// The write() function only ever write one byte unless you
// give it an array and a size. Write(myInt) will only do a
// single byte. The docs say otherwise!
//
//
// Norman Dunbar
// 29th August 2020.
//----------------------------------------------------------
#include <SPI.h>
#include <SD.h>
Nothing has changed in the above from the original, except the comments! However, I’ve had to add the folloiwng “bug” fix (feature fix?) as opening a file in FILE_WRITE mode always opened at the end.
//----------------------------------------------------------
// How to fix the Arduino SD library, it's borked!
//
// FILE_READ is ok.
// FILE_WRITE is actually an APPEND. I've renamed that below
// to FILE_APPEND.
// The new FILE_WRITE will open at the start of the file.
//----------------------------------------------------------
#ifdef ARDUINO
#ifdef FILE_WRITE
#undef FILE_WRITE
#define FILE_WRITE (O_READ | O_WRITE | O_CREAT)
#define FILE_APPEND ((FILE_WRITE) | O_APPEND)
#endif // FILE_WRITE
#endif // ARDUINO
The above has to go after the above includes. I removed the O_APPEND from FILE_WRITE and added in a new FILE_APPEND macro to do what the current FILE_WRITE does. Opening a file with FILE_WRITE mode will correctly start writing from the beginning of the file.
Moving on. In the next chunk of code, I’ve pulled out all the useful functions from the original setup() code. I prefer it that way. These functions are very wordy in that they tell you what’s happening as we go along. It was this that helped me track down the write() problem. I’ve parameterised the fileOpen() and cardInit() functions too.
//----------------------------------------------------------
// File name for testing purposes.
//----------------------------------------------------------
#define FILE_NAME "test.txt"
//----------------------------------------------------------
// Card select pin.
//----------------------------------------------------------
#define CARD_SELECT_PIN 4
//----------------------------------------------------------
// Initialise the SD Card system, returns true on success.
// It's a very wordy function this one!
//----------------------------------------------------------
bool cardInit(byte cardSelectPin) {
Serial.print("Initializing SD card...");
if (!SD.begin(cardSelectPin)) {
Serial.println("initialization failed!");
return false;
}
Serial.println("initialization done.");
return true;
}
//----------------------------------------------------------
// Open the test file on the card, returns handle on success
// or NULL on failure.
//
// WARNING: Calling with mode = FILE_WRITE will actually
// open at the end. For this reason, I'm setting
// the position to the start after opening. Reading
// the file opens at the start anyway.
//
// NOTE: Only one file can be open at any time.
//----------------------------------------------------------
File openFile(const char *fileName, uint8_t mode) {
File theFile;
Serial.print("\nOpening file: ");
Serial.print(fileName);
Serial.print(" ...");
theFile = SD.open(fileName, mode);
if (theFile) {
Serial.println("opened successfully.");
} else {
Serial.println("failed to open.");
return theFile;
}
// File is open, where are we?
Serial.print("File position is currently: ");
Serial.println(theFile.position());
// If the mode is FILE_WRITE, reset to the start.
if (mode == FILE_WRITE) {
theFile.seek(0);
Serial.print("File position is now: ");
Serial.println(theFile.position());
}
return theFile;
}
The WARNING in the comments above fileOpen() apply to the original FILE_WRITE in the Arduino library. It no longer applies if you are using my version as per the correction back at the start of the code. The code also prints out to Serial the file position after the open and after resetting to the start in FILE_WRITE mode. With the bug fix above, that can all be removed.
//----------------------------------------------------------
// Close the test file on the card.
//----------------------------------------------------------
void closeFile(const File fileHandle) {
Serial.print("Closing file...");
fileHandle.close();
Serial.println("closed");
}
//----------------------------------------------------------
// This will read a single byte from the SD card file.
//----------------------------------------------------------
void readByte(File dataFile, byte *data) {
dataFile.read((uint8_t *)data, sizeof(byte));
}
//----------------------------------------------------------
// This will read a single int from the SD card file.
//----------------------------------------------------------
void readInt(File dataFile, int *data) {
dataFile.read((uint8_t *)data, sizeof(int));
}
Now we get to setup() where it all happens. In the event of the SD Card system no initialising, or not being able to open the file, the code just enters a busy loop and stops further execution.
I’ve had to change my original myFile.write((int)12345); line to be myFile.write((uint8_t *)&Wilma, sizeof(int)); to get write() to actually write the two bytes that make up an int. This is contrary to the docs.
//----------------------------------------------------------
// Setup. Does everything here.
//----------------------------------------------------------
void setup() {
Serial.begin(9600);
// Initialise the SD card system.
if (!cardInit(CARD_SELECT_PIN)) {
while (1) ;
}
// Open our test file.
File myFile = openFile(FILE_NAME, FILE_WRITE);
if (!myFile) {
while (1) ;
}
// Test data.
byte Fred = 0xA5;
int Wilma = 12345;
uint32_t sizeWritten = 0;
// Write a byte and an int.
sizeWritten = myFile.write(Fred);
Serial.print("Writing byte: Size written = ");
Serial.println(sizeWritten);
sizeWritten = myFile.write((uint8_t *)&Wilma, sizeof(int));
Serial.print("Writing int: Size written = ");
Serial.println(sizeWritten);
// Close, re-open for reading.
closeFile(myFile);
myFile = openFile(FILE_NAME, FILE_READ);
if (!myFile) {
while (1) ;
}
// Read back one byte and one int.
Fred = Wilma = 0;
readByte(myFile, &Fred);
Serial.print("Fred = 0x");
Serial.println(Fred, HEX);
readInt(myFile, &Wilma);
Serial.print("Wilma (signed int) = ");
Serial.println(Wilma, DEC);
Serial.print("Wilma (unsigned int) = ");
Serial.println((unsigned)Wilma, DEC);
// All done, shutdown.
closeFile(myFile);
}
void loop() {
// There's nothing to see here! Move along!
}
So, there you have it. An SD Card example that works. Now, as for Paul’s original query about writing and reading from a struct…
//----------------------------------------------------------
// SD Card exampel of writing a struct to, and reading it
// from an SD Card file.
//
// Pinout:
//
// D13 = CLK
// D12 = MISO
// D11 = MOSI
// D4 = Card Select pin.
//
// This code uses my fix for the FILE_WRITE problem. Also,
// The write() function only ever writes one byte unless you
// give it an array and a size.
//
//
// Norman Dunbar
// 29th August 2020.
//----------------------------------------------------------
#include <SPI.h>
#include <SD.h>
//----------------------------------------------------------
// How to fix the Arduino SD library, it's borked!
//
// FILE_READ is ok.
// FILE_WRITE is actually an APPEND. I've renamed that below
// to FILE_APPEND.
// The new FILE_WRITE will open at the start of the file.
//----------------------------------------------------------
#ifdef ARDUINO
#ifdef FILE_WRITE
#undef FILE_WRITE
#define FILE_WRITE (O_READ | O_WRITE | O_CREAT)
#define FILE_APPEND ((FILE_WRITE) | O_APPEND)
#endif // FILE_WRITE
#endif // ARDUINO
//----------------------------------------------------------
// File name for testing purposes.
//----------------------------------------------------------
#define FILE_NAME "struct.txt"
//----------------------------------------------------------
// Card select pin.
//----------------------------------------------------------
#define CARD_SELECT_PIN 4
//----------------------------------------------------------
// Initialise the SD Card system, returns true on success.
// It's a very wordy function this one!
//----------------------------------------------------------
bool cardInit(byte cardSelectPin) {
Serial.print("Initializing SD card...");
if (!SD.begin(cardSelectPin)) {
Serial.println("initialization failed!");
return false;
}
Serial.println("initialization done.");
return true;
}
//----------------------------------------------------------
// Open the test file on the card, returns handle on success
// or NULL on failure.
//
// WARNING: Calling with mode = FILE_WRITE will actually
// open at the end. For this reason, I'm setting
// the position to the start after opening. Reading
// the file opens at the start anyway.
//
// NOTE: Only one file can be open at any time.
//----------------------------------------------------------
File openFile(const char *fileName, uint8_t mode) {
File theFile;
Serial.print("\nOpening file: ");
Serial.print(fileName);
Serial.print(" ...");
theFile = SD.open(fileName, mode);
if (theFile) {
Serial.println("opened successfully.");
} else {
Serial.println("failed to open.");
return theFile;
}
// File is open, where are we?
Serial.print("File position is currently: ");
Serial.println(theFile.position());
// If the mode is FILE_WRITE, reset to the start.
// No longer required with the above fix. Was appending.
if (mode == FILE_WRITE) {
theFile.seek(0);
Serial.print("File position is now: ");
Serial.println(theFile.position());
}
return theFile;
}
//----------------------------------------------------------
// Close the test file on the card.
//----------------------------------------------------------
void closeFile(const File fileHandle) {
Serial.print("Closing file...");
fileHandle.close();
Serial.println("closed");
}
//----------------------------------------------------------
// Setup. Does everything here.
//----------------------------------------------------------
void setup() {
Serial.begin(9600);
// Initialise the SD card system.
if (!cardInit(CARD_SELECT_PIN)) {
while (1) ;
}
// Open our test file.
File myFile = openFile(FILE_NAME, FILE_WRITE);
if (!myFile) {
while (1) ;
}
// Test data.
typedef struct testData {
byte aByte;
short aShort;
int anInt;
long aLong;
float aFloat;
char charArray[15];
String aString;
} testData_t;
testData_t myData;
myData.aByte = 1;
myData.aShort = 2;
myData.anInt = 3;
myData.aLong = 4;
myData.aFloat = 3.14;
strcpy(myData.charArray, "Norman Dunbar");
myData.aString = "Hello World, again!";
uint32_t sizeWritten = myFile.write((uint8_t *)&myData, sizeof(myData));
Serial.print("Writing myData struct: Size written = ");
Serial.println(sizeWritten);
// Close, re-open for reading.
closeFile(myFile);
myFile = openFile(FILE_NAME, FILE_READ);
if (!myFile) {
while (1) ;
}
// Read back testdata. Wipe it first though.
memset(&myData, 0, sizeof(testData_t));
uint32_t sizeRead = myFile.read((uint8_t *)&myData, sizeof(myData));
Serial.print("Reading myData struct: Size read = ");
Serial.println(sizeRead);
// All done, shutdown.
closeFile(myFile);
// Print the test data.
Serial.println("\nYour test data looks like this:");
Serial.print("myData.aByte = ");
Serial.println(myData.aByte);
Serial.print("myData.aShort = ");
Serial.println(myData.aShort);
Serial.print("myData.anInt = ");
Serial.println(myData.anInt);
Serial.print("myData.aLong = ");
Serial.println(myData.aLong);
Serial.print("myData.aFloat = ");
Serial.println(myData.aFloat);
Serial.print("myData.charArray = ");
Serial.println(myData.charArray);
Serial.print("myData.aString = ");
Serial.println(myData.aString);
}
void loop() {
// There's nothing to see here! Move along!
}
The code outputs the folloiwng:
Initializing SD card...initialization done.
Opening file: struct.txt ...opened successfully.
File position is currently: 0
File position is now: 0
Writing myData struct: Size written = 34
Closing file...closed
Opening file: struct.txt ...opened successfully.
File position is currently: 0
Reading myData struct: Size read = 34
Closing file...closed
Your test data looks like this:
myData.aByte = 1
myData.aShort = 2
myData.anInt = 3
myData.aLong = 4
myData.aFloat = 3.14
myData.charArray = Norman Dunbar
myData.aString = Hello World, again!
Dumping out the contents of the file, I see the following:
00000000 01 02 00 03 00 04 00 00 00 c3 f5 48 40 4e 6f 72 |...........H@Nor|
00000010 6d 61 6e 20 44 75 6e 62 61 72 00 a3 44 06 13 00 |man Dunbar..D...|
00000020 13 00 |..|
Byte 0 is the struct member aByte.
Bytes 1 and 2 are the struct member aShort in little endian format.
Bytes 3 and 4 are the struct member anInt in little endian format.
Bytes 5 to 8 are aLong in little endian format.
Bytes 9 to 12 are aFloat in who cares what format! Floats are 32 bits or 4 bytes.
Bytes 13 to 25 are the contents of charArray.
Byte 26 is the zero byte terminator of the array.
Bytes 27 to 33 are aString. However, do you see “Hello World, again!”? No neither do I. I suspect that given there are only 7 bytes we are looking at the internal representation of the String class, and not the actual data it (should) contain.
So, how come it worked on the print back? I would suspect that somewhere in that 7 bytes is a pointer, or the address, of wherever the String's data was created in Static RAM. As the code didn’t affect it in any way, it was still present at the address when we read the String back from file.
That’s what I suspect, don’t quote me on that though!
Hopefully, this helps? And don;t forget, stay away from Strings!
Cheers,
Norm.
Good evening Norman
Somewhere in the correspondence that we have been exchanging you said “I like a challenge” well you certainly had one, with what appeared to be a nice simple coding job and it’s turned into a bit of a nightmare. It’s no surprise that mere mortals like me struggle at times.
All I can say again thank you so very much for your efforts and I think you deserve and other glass or two of wine. Enjoy your bank holiday and hopefully it will remain sunny.
I have copied this section into my file and I will keep it for future reference.
Thank you again you certainly have gone the extra mile.
Paul
And a good Evening to you too Paul.
Yes indeed, it was a challenge - but I did get some new toys to play with. That’s always a bonus.
My problem was to do with the Arduino docs about the SD write() function. It does not work as described. It only writes a single byte regardless of the data type passed. It’s fine when writing an array of bytes, that’s ok.
Given the “traumas” encountered, I’m mulling over the idea of a new C++ class, an SDFileHandler, to do all the file work for SD files, and allowing a `write()" function to actually do what it says on the tin! (Regardless of data type!).
Take care.
Cheers,
Norm.
Is it really ‘not as described’ or ‘not described well’ though? 
As the documentation does say for the file.write(data) implementation that data: the byte, char, or string (char *) to write - so isn’t that a single byte in all cases, meaning you need to use the file.write(buf, len) form for more than one byte?
Not that I’m disagreeing with it being traumatic, and a real oversight, since it means a user needs some pretty specific C++ knowledge to do some relatively simple stuff with SD cards… if you just want to append a string (or multi-byte data type) to a SD card, I don’t think you can just do myFile.write(myString);, can you?
I don’t mind people disagreeing with me. Everyine is entitled to an opinon – as long as it’s mine! 
It does say that it returns the number of bytes written. Whinch makes me think that writing a char * type C string should write it all and return the length.
Likewise, writing an int should write both bytes and return 2. Otherwise, the docs should be explicit and say returns 1 – the number of bytes written.
With the file open with FILE_WRITE, it will append (one byte!), if myString is an Arduino String data type, then which byte it writes, I don’t know. I think String inherits from Printable so it’s probably the first byte of the contents.
In my struct example, with a String field, it wrote 7 bytes for the String, when writing the struct as a byte array with a sizeof length supplied. The actual contents of the field were not written. This leads me to assume that a String holds a pointer to the contents and reading it back just set the pointer to where it was pointing before.
It would be interesting to write another sketch to read in the data … waits for wife to go out shopping…
Cheers,
Norm.
You rule breaker… that’s not how the quote goes!
Ah, but the number of bytes in the case of myFile.write(myByte); is 1, whereas myFile.write(myInt,sizeof(myInt)); is 2… so it’s the ‘overloaded’ version of .write() that is needed for multi-byte data.
Hence myFile.write(myString); also misbehaving! Those silly Arduinoites… should have added extra overloads for int, float, String, etc… 
Can’t help with the String behaviour… but is quite possible - perhaps .c_str() is needed there?
Apologies if Granny already knows how to suck eggs!"
I consulted “the Book” and the source code. I was incorrect somewhere above when I mentioned that I thought that String inherited from Printable, it does not, so it’s not streamable. However, the Print class knows how to print and println a String:
size_t Print::print(const String &s)
{
return write(s.c_str(), s.length());
}
...
size_t Print::println(const String &s)
{
size_t n = print(s);
n += println();
return n;
}
The write() functions are virtual in the Print class and must be defined in any descendant classes.
So, what about the File class?
class File : public Stream {
...
virtual size_t write(uint8_t);
virtual size_t write(const uint8_t *buf, size_t size);
...
using Print::write;
};
So, it inherits from Stream which is just a base class to allow things like Serial.read() etc. All the reading functions are in this class. Stream inherits from Print – so we have reading and writing abilities – and we have those two write() functions to define. So far so good.
As File is using Print::write; then it should be able to call upon the various write() functions in that class. Checking the source, I see this:
size_t write(const char *str) {
if (str == NULL) return 0;
return write((const uint8_t *)str, strlen(str));
}
size_t write(const char *buffer, size_t size) {
return write((const uint8_t *)buffer, size);
}
So, the Print class knows how to write a C string (char *) correctly – which implies that the File class should as well? And yet, when I call a File.write() with a C string, it prints off the first byte only.
That’s what a print() with a String does, it just dumps out the c_str() part and length. That’s built in to the Print class. There’s no corresponding write() function though. Bummer!
Anyway, the summary of all this is simple, the File class is borked and incomplete. The example(s) given show you how to write a char * string but nothing else, you have to write your own write functions to get any ssort of meaningful data written in binary to the file - it seems to assume you will only ever want to print() data as ASCII text.
And I was correct about what a struct with a String field writes to file when the etire struct is written as a byte array, it writes a 4 byte pointer to the data, a two byte int capacity and a 2 byte int length.
char *buffer; // the actual char array
unsigned int capacity; // the array length minus one (for the '\0')
unsigned int len; // the String length (not counting the '\0')
Cheers,
Norm.
This should help anyone wanting to write data, as opposed to ASCII, to an SD card file.
Cheers,
Norm.
I have now uploaded some documentation for the above SDFileHandler class.
Cheers,
Norm.