AWS S3 HTTPS error SignatureDoesNotMatch

Hi, I am trying to connect to S3 with my ESP32 and I am using the arduino SDK. I followed these two AWS documentations on how to properly make a HTTP Get request: Create a signed AWS API request and Signature Calculations for the Authorization Header.

I verified my SHA256 caluclations with this online tool and my HMAC-SHA256 calcs with this tool

This is the error message I get (I removed the StringToSignBytes and CanonicalRequestBytes):

<Error><Code>SignatureDoesNotMatch</Code><Message>The request signature we calculated does not match the signature you provided. Check your key and signing method.</Message><AWSAccessKeyId>AKIAZOKIXEXAMPLE</AWSAccessKeyId>
<StringToSign>
AWS4-HMAC-SHA256
20230310T134304Z
20230310/eu-central-1/s3/aws4_request
5263692f39c1379ed930dfd5c7ed75b40ba0619c7dc316bb7630152162a5d5ef
</StringToSign><SignatureProvided>
2b974103174d38e1946911813cd76aff7271f4ace1b75947b09c67487d2c39da
</SignatureProvided><StringToSignBytes></StringToSignBytes>
<CanonicalRequest>
GET
/firmware.bin

content-type:application/octet-stream
host:esp-data-exchange.s3.amazonaws.com
x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
x-amz-date:20230310T134304Z

content-type;host;x-amz-content-sha256;x-amz-date
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855

According to this AWS site I have a

Canonicalization errors

If you incorrectly calculated the canonical request or the string to sign, the signature verification step performed by the service fails with the following error message:

The request signature we calculated does not match the signature you provided

The error response includes the canonical request and the string to sign that the service calculated. You can compare these strings with the strings that you calculated.

You can also verify that you didn’t send the request through a proxy that modifies the headers or the request.

Checking the content of the error message, especially the Canonical request and String to Sign of my request are identical (or did I miss something?), see them below.
Canonical request:

GET
/firmware.bin

content-type:application/octet-stream
host:esp-data-exchange.s3.amazonaws.com
x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
x-amz-date:20230310T134304Z

content-type;host;x-amz-content-sha256;x-amz-date
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855

My string to sign:

AWS4-HMAC-SHA256
20230310T134304Z
20230310/eu-central-1/s3/aws4_request
5263692f39c1379ed930dfd5c7ed75b40ba0619c7dc316bb7630152162a5d5ef

Besides, the final signature, also present in the response, is also identical with my hash caluclations, both are: 2b974103174d38e1946911813cd76aff7271f4ace1b75947b09c67487d2c39da

Has anyone an idea what could be wrong? Do I misinterpret the error code from AWS and it could be something else? Any help is appreciated.

Since I wanted to make my code public anyway, this is a “minimal” example of my HTTPS S3 communication attempts. It has been tested in VScode, platformIO project (copy all content into main.cpp). It works if you add your wifi credentials. However, the request is outdated and my keys are not included - but for support it might help to understand what I am doing and help find the error.
PlatformIO.ini:

[env:firebeetle32]
platform = espressif32
board = firebeetle32
framework = arduino

EDIT: Here a “minimal” example.

  • Parameters like date and time are hardcoded but are the same like above.
  • Obviously, I cannot provide my keys. For the sig_calc I can provide the resulting hash from the first calculation where the AWS_SECRET_KEY is needed: 43d0ce257da1febfc3e750bb7c6eeab529041a7b69cc420a97039ce7e4e97a87
  • You need to be connected to WIFI - add your SSID and PASSWORD
  • The error you get is now, that the request is outdated otherwise the calculation results are like exaplained above.
    #include <Arduino.h>
    #include "mbedtls/md.h"
    #include "WiFiClientSecure.h"
    #include "WiFi.h"
    
    
    String generate_hmac_SHA256_hash(const char *hmac_key, const char *payload)
    {
        /* Instantiating hash context structure */
        mbedtls_md_context_t ctx;
        /* Defining hash type */
        mbedtls_md_type_t md_type = MBEDTLS_MD_SHA256;
        /* Byte buffer array for the hash result */
        byte hash_result[32];
        /* Initialize final hash string to be returned */
        String hash_str = "";
    
        /* Initialize the hash calculator structur */
        mbedtls_md_init(&ctx);
        /* Setup the hash type */
        mbedtls_md_setup(&ctx, mbedtls_md_info_from_type(md_type), 1);
        /* Proivde the key used for hashing */
        mbedtls_md_hmac_starts(&ctx, (const unsigned char *)hmac_key, strlen(hmac_key));
        /* Provide the payload to be hashed */
        mbedtls_md_hmac_update(&ctx, (const unsigned char *)payload, strlen(payload));
        /* Perform the hash process */
        mbedtls_md_hmac_finish(&ctx, hash_result);
        /* Free memory allocated for has process */
        mbedtls_md_free(&ctx);
        /* Convert the bytes into a hexadecimal string */
        for (int i = 0; i < sizeof(hash_result); i++)
        {
            /* Buffer for storing hex values */
            char hex_buffer[3];
            /* Converting the bytes and storing them in hex buffer */
            sprintf(hex_buffer, "%02x", (int)hash_result[i]);
            /* Concatenating the hex bytes to a string */
            hash_str += String(hex_buffer);
        }
    
        return hash_str;
    }
    
    String generate_SHA256_hash(const char *payload)
    {
        /* Instantiating hash context structure */
        mbedtls_md_context_t ctx;
        /* Defining hash type */
        mbedtls_md_type_t md_type = MBEDTLS_MD_SHA256;
        /* Byte buffer array for the hash result */
        byte hash_result[32];
        /* Initialize final hash string to be returned */
        String hash_str = "";
    
        /* Initialize the hash calculator structur */
        mbedtls_md_init(&ctx);
        /* Setup the hash type */
        mbedtls_md_setup(&ctx, mbedtls_md_info_from_type(md_type), 0);
        /* Start the message-digest computation */
        mbedtls_md_starts(&ctx);
        /* Provide the payload to be hashed */
        mbedtls_md_update(&ctx, (const unsigned char *)payload, strlen(payload));
        /* Perform the hash process */
        mbedtls_md_finish(&ctx, hash_result);
        /* Free memory allocated for has process */
        mbedtls_md_free(&ctx);
        /* Convert the bytes into a hexadecimal string */
        for (int i = 0; i < sizeof(hash_result); i++)
        {
            /* Buffer for storing hex values */
            char hex_buffer[3];
            /* Converting the bytes and storing them in hex buffer */
            sprintf(hex_buffer, "%02x", (int)hash_result[i]);
            /* Concatenating the hex bytes to a string */
            hash_str += String(hex_buffer);
        }
    
        return hash_str;
    }
    
    /* Step 1: Create a canonical request */
    String canonical_request(String payload, String method, String object, String query, String content_type, String bucket_name, String service, String host, String date, String time)
    {
      String payload_hash = "";
      if (payload == "")
      {
        payload_hash = "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855";
      }
      else
      {
        payload_hash = generate_SHA256_hash(payload.c_str());
      }
      return method + "\n" +
              "/" + object + "\n" +
              query + "\n" +
              "content-type:" + content_type + "\n" +
              "host:" + bucket_name + "." + service + "." + host + "\n" +
              "x-amz-content-sha256:" + payload_hash + "\n" +
              "x-amz-date:" + date + "T" + time + "Z\n" +
              "\n" +
              "content-type;host;x-amz-content-sha256;x-amz-date\n" +
              payload_hash;
    }
    
    /* Step 2: Create a canonical request hash */
    String canoncial_request_hash(String canoncial_request)
    {
      return generate_SHA256_hash(canoncial_request.c_str());
    }
    
    /* Step 3: Create a string to sign */
    String string_2_sign(String algorithm, String date, String time, String region, String service, String hashed_canonical_request)
    {
      return algorithm + "\n" +
              date + "T" + time + "Z\n" +
              date + "/" + region + "/" + service + "/aws4_request\n" +
              hashed_canonical_request;
    }
    
    /* Step 4: Calculate the signature */
    String signature_calculation(int length, String array[])
    {
      String signing_key = array[0];
    
      for (int i = 1; i < length; i++)
      {
        Serial.println(signing_key.c_str());
        Serial.println(array[i].c_str());
        signing_key = generate_hmac_SHA256_hash(signing_key.c_str(), array[i].c_str());
      }
      return signing_key;
    }
    
    /* Step 5: Add the signature to the request */
    String authorization_header(String algorithm, String aws_access_key, String date, String region, String service, String signature)
    {
      return algorithm + " Credential=" + aws_access_key + "/" + date + "/" + region + "/" + service + "/aws4_request," +
              "SignedHeaders=content-type;host;x-amz-content-sha256;x-amz-date," + "Signature=" + signature;
    }
    
    // https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetObject.html
    /* Create a sample request header */
    String GET_header(String object, String content_type, String bucket, String service, String date, String time, String authorization_header)
    {
      return "GET /" + object + " HTTP/1.1\r\n" +
              "Content-Type: " + content_type + "\r\n" +
              "Host: " + bucket + "." + service + "." + "amazonaws.com\r\n" +
              "Authorization: " + authorization_header + "\r\n" +
              "x-amz-content-sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855\r\n" +
              "x-amz-date: " + date + "T" + time + "Z" + "\r\n\r\n";
    }
    
    
    void setup()
    {
      WiFiClientSecure ota1_client;
    
      String algorithm = "AWS4-HMAC-SHA256";
      String region = "eu-central-1";
      String method = "GET";
      String service = "s3";
      String bucket_name = "esp-data-exchange";
      String aws_secret_key = "AWS_SECRET_KEY";
      String aws_access_key = "AWS_ACCESS_KEY";
    
      String object = "firmware.bin";
      String content_type = "application/octet-stream";
      String host = "amazonaws.com";
      String payload = "";
      String query = "";
      String date = "20230310";
      String time = "134304";
      String date_time = "20230310T134304Z";
      int length = 6;
    
      Serial.begin(9600);
      delay(5000);
    
      WiFi.begin("SSID", "PASSWORD");
      
      String can_req = canonical_request(payload, method, object, query, content_type, bucket_name, service, host, date, time);
      String can_req_hash = canoncial_request_hash(can_req);
      String str_2_sign = string_2_sign(algorithm, date, time, region, service, can_req_hash);
      String hash_array[length] = {"AWS4" + aws_secret_key, date, region, service, "aws4_request", str_2_sign};
      String sig_calc = signature_calculation(length, hash_array);
      String auth_header = authorization_header(algorithm, aws_access_key, date, region, service, sig_calc);
      String http_header = GET_header(object, content_type, bucket_name, service, date, time, auth_header);
      String bucket_url = bucket_name + "." + service + "." + region + "." + host;
      
      //ota1_client.setCACert(AWS_CERT_CA);
      ota1_client.setInsecure();
      ota1_client.connect(bucket_url.c_str(), 443);
      Serial.println("Test");
      ota1_client.print(http_header);
    
      // Wait for the response
      while (ota1_client.connected() && !ota1_client.available())
        ;
    
      // Read and print the response
      while (ota1_client.available())
      {
        String line = ota1_client.readStringUntil('\r');
        Serial.print(line);
      }
      ota1_client.stop();
    }
    
    void loop()
    {
      // put your main code here, to run repeatedly:
    }

So I was finally able to get a valid signature. I know that my error was using the function generate_hmac_SHA256_hash they way I used it inside signature_calculation() but to be honest, until the end I did not find the explanation why only the first iteration was correct and afterwards using the hash result of the first calculation as the hash key for the second calculation (and so on…) did not work.
Anyway, I used a javascript to check if my hmac-sha256 calculation were correct. This is the verification code:

const crypto = require("crypto");

function HMAC(key, text) {
    return crypto.createHmac("sha256", key).update(text).digest();
}


const kSecret = "hCVSg1n1+mZgRwjktOv52NqA70jeayS/Z4yqJcD4";
const kDate = HMAC("AWS4" + kSecret,"20230314");
console.log("kDate hash is    :", kDate.toString('hex'));
const kRegion = HMAC(kDate,"eu-central-1");
console.log("kRegion hash is    :", kRegion.toString('hex'));
const kService = HMAC(kRegion,"s3");
console.log("kService hash is    :", kService.toString('hex'));
const kSigning = HMAC(kService,"aws4_request");
console.log("kSigning hash is    :", kSigning.toString('hex'));

I compiled it using this online compiler.
To ensure my credentials were working - I used postman as an online tool. It helped me to understand HTTP header design and response interpretation.

My posted approach from above did not work because I returned the hex-encoded version of my hash and used this hex-encoded string for the next hash, but as indicated in the documentation from AWS you have to provide the unsigned char buffer directly from the previous hash calculation. You are not allowed to convert this unsigned char into a hex-String or const char hex.

So far, so good but I was simply not able to calculate the correct hash result using the following approach. As you can see, I added a buffer as input argument for the generate_hmac_SHA256_hash() function to store the result in a buffer making it possible to directly use the previously calculated hash as key for the next hash calculation.

	String kDate_str, kRegion_str, kService_str, kSign_str, signature = "";
	byte hash_buffer[32];

    kDate_str = generate_hmac_SHA256_hash(AWS4_key.c_str(), "20230313", hash_buffer);
	kRegion_str = generate_hmac_SHA256_hash((const char*) hash_buffer, "eu-central-1", hash_buffer);
	kService_str = generate_hmac_SHA256_hash((const char*) hash_buffer, "s3", hash_buffer);
	kSign_str = generate_hmac_SHA256_hash((const char*) hash_buffer, "aws4_request", hash_buffer);
	signature = generate_hmac_SHA256_hash((const char*) hash_buffer, string_2_sign, hash_buffer);

Only the first hash was correct. Can someone explain to me why this approach is not working? Looks like I am overseeing something very simple I guess…

Finally, I ended up with this function, calculating the correct signature:

String signature_calculation(String* payload, size_t payload_size)
{
    /* Instantiating hash context structure */
    mbedtls_md_context_t ctx;
    /* Defining hash type */
    mbedtls_md_type_t md_type = MBEDTLS_MD_SHA256;
    /* Byte buffer array for the hash result */
	size_t buffer_length = 32;
    unsigned char hash_buffer[buffer_length]; 
    /* Initialize final hash string to be returned */
    String hash_str = "";
    
    /* Initialize the hash calculator structur */
    mbedtls_md_init(&ctx);
    /* Setup the hash type */
    mbedtls_md_setup(&ctx, mbedtls_md_info_from_type(md_type), 1);
	/* First iteration for signature has different key length */
	/* Start the message-digest computation */
	mbedtls_md_hmac_starts(&ctx, (const unsigned char *)payload[0].c_str(), strlen(payload[0].c_str()));
	/* Provide the payload to be hashed */
	mbedtls_md_hmac_update(&ctx, (const unsigned char *)payload[1].c_str(), strlen(payload[1].c_str()));
	/* Store result in hash_buffer */
	mbedtls_md_hmac_finish(&ctx, hash_buffer);
	/* Convert hash result into hex string an print it */
	hash_byte_to_hex(buffer_length, hash_buffer);

	/* Calculate a cascade of hashs using previous hash results as keys */
	for(int i = 2; i < payload_size; i++)
	{
		mbedtls_md_hmac_starts(&ctx, hash_buffer, buffer_length);
		mbedtls_md_hmac_update(&ctx, (const unsigned char *)payload[i].c_str(), strlen(payload[i].c_str()));
		mbedtls_md_hmac_finish(&ctx, hash_buffer);
		hash_str = hash_byte_to_hex(buffer_length, hash_buffer);
	}
	/* Free memory allocated for has process */
  	mbedtls_md_free(&ctx);
	/* Return final hash result as hex string*/
  	return hash_str;
}

Note that the hash_byte_to_hex() function simply converts the hash result into a hex string and prints it.

Hi, sorry if this a bit off-topic, but you can run the javascript on your desktop, too: You can just install nodejs and just test your code interactively, or if you’re using CLion or Visual Studio code those can also run javascript (I believe visual studio code requires an additional plugin, though)

Good point, I think my background as embedded developer, where you cannot simply perform quick code snippet tests in the same IDE (e.g. Code Composer Studio from TI) caused me to automatically go to “online compiler” :slight_smile:

@lkyslevin Would you mind to share the code reference. I am also working on esp32 and getting Error - “The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method”.

Not sure what you mean by code reference. I think I have posted an example and also described all invoked functions related to it.

Thanks @lkyslevin for replying back.
I tried it with esp32 and getting The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method.

I don’t find this function in the reference ( hash_byte_to_hex ). It will be great help if you can share aws signature code example. I am searching on google from couple of days and no luck.

Sorry for the late reply, I have attached my .cpp file with several utility functions.

/******************************************************************************
 Includes
 *****************************************************************************/
#include "aws_http.h"

/******************************************************************************
 Function prototypes
 *****************************************************************************/
void hashByteToHex(unsigned char *hashBuffer, size_t bufferLength, char *result);

/******************************************************************************
 Function definitions
 *****************************************************************************/
/**
 * @brief Converts an unsigned 32-bit integer to a hexadecimal string.
 *
 * @param integer The integer to be converted.
 * @param result A pointer to the buffer that will contain the hexadecimal string.
 * @return None
 */
void uintToHex(uint32_t integer, char *result)
{
    char hexStr[MAX_UINT32_TO_HEX_LENGTH] = {0};
    sprintf(hexStr, "%04X", integer);
    /* Remove leading zeros from hex string */
    int i;
    for (i = 0; i < 3; i++)
    {
        if (hexStr[i] != '0')
        {
            break;
        }
    }
    strcpy(result, &hexStr[i]);
}

/**
 * @brief Converts a byte buffer to a hexadecimal string representation.
 *
 * @param hashBuffer A pointer to the buffer containing the bytes to be converted.
 * @param bufferLength The length of the byte buffer.
 * @param result A pointer to the buffer that will contain the hexadecimal string.
 * @return None
 */
void hashByteToHex(unsigned char *hashBuffer, size_t bufferLength, char *result)
{
    for (int i = 0; i < bufferLength; i++)
    {
        /* Converting the bytes and storing them in hex buffer */
        sprintf(result + (i * 2), "%02x", (int)hashBuffer[i]);
    }
    // Serial.printf("Hex hash: %s\r\n", result);
}

// https://techtutorialsx.com/2018/01/25/esp32-arduino-applying-the-hmac-sha-256-mechanism/
/**
 * @brief Generates a SHA256 hash of a given payload.
 *
 * @param payload A pointer to the payload to be hashed.
 * @param payloadLength The length of the payload.
 * @param result A pointer to the buffer that will contain the generated hash.
 * @return None
 */
void generateSHA256Hash(const char *payload, size_t payloadLength, char *result)
{
    /* Instantiating hash context structure */
    mbedtls_md_context_t ctx;
    /* Defining hash type */
    mbedtls_md_type_t mdType = MBEDTLS_MD_SHA256;

    /* Initialize the hash calculator structur */
    mbedtls_md_init(&ctx);
    /* Setup the hash type */
    mbedtls_md_setup(&ctx, mbedtls_md_info_from_type(mdType), 0);
    /* Start the message-digest computation */
    mbedtls_md_starts(&ctx);
    /* Provide the payload to be hashed */
    mbedtls_md_update(&ctx, (const unsigned char *)payload, payloadLength);
    /* Perform the hash process */
    mbedtls_md_finish(&ctx, (unsigned char *)result);
    /* Free memory allocated for has process */
    mbedtls_md_free(&ctx);
}

/**
 * @brief Generates an HMAC-SHA256 hash of the provided payload using the provided key.
 *
 * @param hmacKey A pointer to the key used for hashing.
 * @param keyLength The length of the key used for hashing.
 * @param payload A pointer to the payload to be hashed.
 * @param payloadLength The length of the payload to be hashed.
 * @param result A pointer to the buffer that will contain the generated hash.
 * @return None
 */
void generateHmacSHA256Hash(const char *hmacKey, size_t keyLength, const char *payload, size_t payloadLength, char *result)
{
    /* Instantiating hash context structure */
    mbedtls_md_context_t ctx;
    /* Defining hash type */
    mbedtls_md_type_t mdType = MBEDTLS_MD_SHA256;

    /* Initialize the hash calculator structur */
    mbedtls_md_init(&ctx);
    /* Setup the hash type */
    mbedtls_md_setup(&ctx, mbedtls_md_info_from_type(mdType), 1);
    /* Proivde the key used for hashing */
    mbedtls_md_hmac_starts(&ctx, (const unsigned char *)hmacKey, keyLength);
    /* Provide the payload to be hashed */
    mbedtls_md_hmac_update(&ctx, (const unsigned char *)payload, payloadLength);
    /* Perform the hash process */
    mbedtls_md_hmac_finish(&ctx, (unsigned char *)result);
    /* Free memory allocated for has process */
    mbedtls_md_free(&ctx);
}

/**
 * @brief Extracts the hexadecimal payload hash from a canonical request.
 *
 * @param canonicalRequest A string containing the canonical request.
 * @param result A pointer to the buffer that will contain the extracted hash.
 * @return None
 */
void extractPayloadHexHash(const char *canonicalRequest, char *result)
{
    /* Find the start of the payload hash */
    const char *payloadHashStart = strstr(canonicalRequest, "x-amz-content-sha256:");
    const char *payloadHashEnd;
    size_t payloadHashLength = 0;
    if (!payloadHashStart)
    {
        /* Payload hash header not found */
        return;
    }
    /* Move pointer to start of the hash */
    payloadHashStart += strlen("x-amz-content-sha256:");
    payloadHashEnd = payloadHashStart;
    /* Move pointer to the end of the line */
    while (*payloadHashEnd != '\n')
    {
        payloadHashEnd++;
        payloadHashLength++;
    }
    /* Extract the payload hash into a new C string and add 0 pointer */
    memcpy(result, payloadHashStart, payloadHashLength);
    result[payloadHashLength] = '\0';
}

/**
 * @brief Extracts the signed headers list from the canonical request.
 *
 * @param canonicalRequest The canonical request to extract the signed headers from
 * @param canonicalRequestLength The length of the canonical request
 * @param endLine The position of the last newline in the canonical request header section
 * @param result Pointer to a buffer to store the extracted signed headers list
 */
void extractSignedHeaderList(char *canonicalRequest, size_t canonicalRequestLength, size_t endLine, char *result)
{
    char *signedHeaderListPos = canonicalRequest + canonicalRequestLength - endLine - 2;
    int signedHeaderLength = 0;
    /* Moving backwards through the signed header list counting the chars until the previous \n is found */
    while (*signedHeaderListPos != '\n')
    {
        signedHeaderListPos--;
        signedHeaderLength++;
    }
    /* The while loop iterates one iteration too often */
    signedHeaderListPos++;
    memcpy(result, signedHeaderListPos, signedHeaderLength);
    result[signedHeaderLength] = '\0';
}

// https://docs.aws.amazon.com/general/latest/gr/create-signed-request.html
// https://docs.aws.amazon.com/AmazonS3/latest/API/sig-v4-header-based-auth.html
/* Step 1: Create a canonical request */
/**
 * @brief Creates the canonical GET request
 * 
 * @param payload The request payload
 * @param object The S3 object name
 * @param query The S3 query string
 * @param contentType The content type of the payload
 * @param host The S3 bucket endpoint
 * @param date The request date
 * @param time The request time
 * @param result The output string containing the canonical request
 * @param sizeResult The size of the output string
 * @return true if the canonical request was created successfully, false otherwise
 */
bool canonicalGetRequest(const char *payload, const char *object, const char *query, const char *contentType,
                         const char *host, const char *date, const char *time, char *result, size_t sizeResult)
{
    bool ret = false;
    char payloadHash[MAX_BYTE_SHA256_HASH_LENGTH] = {0};
    char payloadHexHash[MAX_HEX_HASH_LENGTH] = {0};
    int snprintf_ret = 0;

    /* Stores an SHA256 hash */
    generateSHA256Hash(payload, strlen(payload), payloadHash);
    /* Convert the bytes into a hexadecimal string */
    hashByteToHex((unsigned char *)payloadHash, sizeof(payloadHash), payloadHexHash);

    snprintf_ret = snprintf(result,
                            sizeResult,
                            "GET\n"
                            "/%s\n"
                            "%s\n"
                            "content-type:%s\n"
                            "host:%s\n"
                            "x-amz-content-sha256:%s\n"
                            "x-amz-date:%sT%sZ\n"
                            "\n"
                            "content-type;host;x-amz-content-sha256;x-amz-date\n"
                            "%s",
                            object, query, contentType, host, payloadHexHash, date, time, payloadHexHash);
    if (snprintf_ret < 0)
    {
        return ret;
    }
    else if (snprintf_ret >= sizeResult)
    {
        Serial.printf("Canonical request failed: %d\r\n", snprintf_ret);
        return ret;
    }
    else
    {
        ret = true;
        return ret;
    }
}

/* Step 1: Create a canonical request */
/**
 * @brief Generates a canonical request for a PUT operation and stores it in the result buffer.
 *
 * @param payload        the payload of the PUT request.
 * @param object         the S3 object being PUT.
 * @param query          the query string (without the '?') to add to the object name.
 * @param host           the S3 endpoint being accessed.
 * @param date           the date in ISO 8601 format (YYYYMMDD).
 * @param time           the time in ISO 8601 format (HHMMSSZ).
 * @param storageClass   the S3 storage class to use for the object.
 * @param result         the buffer to store the canonical request in.
 * @param sizeResult     the size of the result buffer.
 *
 * @return True if the canonical request was generated successfully, false otherwise.
 */
bool canonicalPutRequest(const char *payload, const char *object, const char *query, const char *host,
                         const char *date, const char *time, const char *storageClass, char *result,
                         size_t sizeResult)
{
    bool ret = false;
    char payloadHash[32] = {0};
    char payloadHexHash[65] = {0};
    int snprintf_ret = 0;
    /* Stores an SHA256 hash */
    generateSHA256Hash(payload, strlen(payload), payloadHash);
    /* Convert the bytes into a hexadecimal string */
    hashByteToHex((unsigned char *)payloadHash, sizeof(payloadHash), payloadHexHash);
    snprintf_ret = snprintf(result,
                            sizeResult,
                            "PUT\n"
                            "/%s\n"
                            "%s\n"
                            "host:%s\n"
                            "x-amz-content-sha256:%s\n"
                            "x-amz-date:%sT%sZ\n"
                            "x-amz-storage-class:%s\n"
                            "\n"
                            "host;x-amz-content-sha256;x-amz-date;x-amz-storage-class\n"
                            "%s",
                            object, query, host, payloadHexHash, date, time, storageClass, payloadHexHash);
    if (snprintf_ret < 0)
    {
        return ret;
    }
    else if (snprintf_ret >= sizeResult)
    {
        Serial.printf("Canonical request failed: %d\r\n", snprintf_ret);
        return ret;
    }
    else
    {
        ret = true;
        return ret;
    }
}

/* Step 1: Create a canonical request */
/**
 * @brief Constructs the canonical request for an HTTP Put request (e.g. to AWS S3).
 *
 * @param object The S3 object key
 * @param query Query string to include in the request
 * @param contentLength Total content length of the PUT request
 * @param host The S3 bucket host address
 * @param date The date of the request in ISO8601 format
 * @param time The time of the request in ISO8601 format
 * @param payloadLength The length of the actual data to be stored in the destination file
 * @param storageClass The AWS storage class
 * @param result The buffer to store the canonical request
 * @param sizeResult The size of the buffer for the canonical request
 *
 * @return True if successful, false otherwise
 */
bool seedCanonicalPutRequest(const char *object, const char *query, uint32_t contentLength, const char *host,
                             const char *date, const char *time, uint32_t payloadLength, const char *storageClass,
                             char *result, size_t sizeResult)
{
    bool ret = false;
    int snprintf_ret = 0;
    /* Payload lenght relates to the the actual data to be stored in the destination file
    Content length is payload length + meta data length! */
    snprintf_ret = snprintf(result,
                            sizeResult,
                            "PUT\n"
                            "/%s\n"
                            "%s\n"
                            "content-encoding:aws-chunked\n"
                            "content-length:%lu\n"
                            "host:%s\n"
                            "x-amz-content-sha256:STREAMING-AWS4-HMAC-SHA256-PAYLOAD\n"
                            "x-amz-date:%sT%sZ\n"
                            "x-amz-decoded-content-length:%lu\n"
                            "x-amz-storage-class:%s\n"
                            "\n"
                            "content-encoding;content-length;host;x-amz-content-sha256;x-amz-date;x-amz-decoded-content-length;x-amz-storage-class\n"
                            "STREAMING-AWS4-HMAC-SHA256-PAYLOAD",
                            object, query, contentLength, host, date, time, payloadLength, storageClass);
    if (snprintf_ret < 0)
    {
        return ret;
    }
    else if (snprintf_ret >= sizeResult)
    {
        Serial.printf("Canonical request failed: %d\r\n", snprintf_ret);
        return ret;
    }
    else
    {
        ret = true;
        return ret;
    }
}

/* Step 2: Create a canonical request hash */
/**
 * @brief Generates an SHA256 hash for the canonical request string and
 *        stores the resulting hash as a hexadecimal string.
 * 
 * @param canoncialRequest The canonical request string
 * @param result The buffer to store the resulting hash as a hexadecimal string
 */
void canoncialRequestHash(const char *canoncialRequest, char *result)
{
    char payloadHash[MAX_BYTE_SHA256_HASH_LENGTH] = {0};
    /* Stores an SHA256 hash in hex format */
    generateSHA256Hash(canoncialRequest, strlen(canoncialRequest), payloadHash);
    /* Convert the bytes into a hexadecimal string */
    hashByteToHex((unsigned char *)payloadHash, sizeof(payloadHash), result);
}

/**
 * @brief Calculates the string to sign for AWS S3 REST requests with streaming body.
 *        The string to sign is required for the signature calculation of the REST request.
 * 
 * @param payload pointer to the buffer containing the payload to be sent in the request.
 * @param payloadLength length of the payload in bytes.
 * @param prevSignature the signature calculated for previous chunks, or for the full payload.
 * @param date pointer to the date string in ISO8601 format, as required by AWS.
 * @param time pointer to the time string in ISO8601 format, as required by AWS.
 * @param region pointer to the region string, as required by AWS.
 * @param service pointer to the service string, as required by AWS.
 * @param result pointer to the buffer where the result string will be stored.
 * 
 * @note The result buffer must have at least MAX_STRING2SIGN_STREAM_LENGTH bytes of space.
 */
void stringToSignStream(const char *payload, size_t payloadLength, const char *prevSignature, const char *date,
                        const char *time, const char *region, const char *service, char *result)
{
    char payloadHash[MAX_BYTE_SHA256_HASH_LENGTH] = {0};
    char payloadHexHash[MAX_HEX_HASH_LENGTH] = {0};

    /* Stores an SHA256 hash */
    generateSHA256Hash(payload, payloadLength, payloadHash);
    /* Convert the bytes into a hexadecimal string */
    hashByteToHex((unsigned char *)payloadHash, sizeof(payloadHash), payloadHexHash);

    snprintf(result, MAX_STRING2SIGN_STREAM_LENGTH,
             "AWS4-HMAC-SHA256-PAYLOAD\n"
             "%sT%sZ\n"
             "%s/%s/%s/aws4_request\n"
             "%s\n"
             "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855\n"
             "%s",
             date, time, date, region, service, prevSignature, payloadHexHash);
}

/* Step 3: Create a string to sign */
/**
 * @brief Generates the string to sign for AWS request signing.
 * 
 * @param algorithm The signing algorithm.
 * @param date The date of the request.
 * @param time The time of the request.
 * @param region The AWS region of the request.
 * @param service The AWS service of the request.
 * @param hashedCanonicalRequest The hashed canonical request of the request.
 * @param result The resulting string to sign.
 * @param sizeResult The size of the result buffer.
 */
void stringToSign(const char *algorithm, const char *date, const char *time, const char *region, const char *service,
                  const char *hashedCanonicalRequest, char *result, size_t sizeResult)
{
    snprintf(result,
             sizeResult,
             "%s\n%sT%sZ\n%s/%s/%s/aws4_request\n%s",
             algorithm, date, time, date, region, service, hashedCanonicalRequest);
}

/* Step 4: Calculate the signature */
/**
 * @brief Calculates the signature for an AWS4 request
 *
 * This function takes in the AWS4 secret key, date, region, service, and stringToSign and uses them
 * to calculate the signature for an AWS4 request. The signature is stored in the result argument.
 *
 * @param AWS4secretKey The AWS4 secret key to be used for signature calculation
 * @param date The date to be used for signature calculation
 * @param region The region to be used for signature calculation
 * @param service The service to be used for signature calculation
 * @param stringToSign The string to be used for signature calculation
 * @param result The result string to store the signature
 */
void signatureCalculation(const char *AWS4secretKey, const char *date, const char *region, const char *service,
                          const char *stringToSign, char *result)
{
    char kDate[MAX_BYTE_SHA256_HASH_LENGTH] = {0};
    char kRegion[MAX_BYTE_SHA256_HASH_LENGTH] = {0};
    char kService[MAX_BYTE_SHA256_HASH_LENGTH] = {0};
    char kSigning[MAX_BYTE_SHA256_HASH_LENGTH] = {0};
    char kSignature[MAX_BYTE_SHA256_HASH_LENGTH] = {0};
    /* Calculate date key */
    generateHmacSHA256Hash(AWS4secretKey, strlen(AWS4secretKey), date, strlen(date), kDate);
    /* Convert hash result into hex string and print it */
    hashByteToHex((unsigned char *)kDate, MAX_BYTE_SHA256_HASH_LENGTH, result);

    /* Calculate region key */
    generateHmacSHA256Hash(kDate, MAX_BYTE_SHA256_HASH_LENGTH, region, strlen(region), kRegion);
    /* Convert hash result into hex string and print it */
    hashByteToHex((unsigned char *)kRegion, MAX_BYTE_SHA256_HASH_LENGTH, result);

    /* Calculate service key */
    generateHmacSHA256Hash(kRegion, MAX_BYTE_SHA256_HASH_LENGTH, service, strlen(service), kService);
    /* Convert hash result into hex string and print it */
    hashByteToHex((unsigned char *)kService, MAX_BYTE_SHA256_HASH_LENGTH, result);

    /* Calculate signing key */
    generateHmacSHA256Hash(kService, MAX_BYTE_SHA256_HASH_LENGTH, "aws4_request", strlen("aws4_request"), kSigning);
    /* Convert hash result into hex string and print it */
    hashByteToHex((unsigned char *)kSigning, MAX_BYTE_SHA256_HASH_LENGTH, result);

    /* Calculate signature */
    generateHmacSHA256Hash(kSigning, MAX_BYTE_SHA256_HASH_LENGTH, stringToSign, strlen(stringToSign), kSignature);
    /* Convert hash result into hex string and print it */
    hashByteToHex((unsigned char *)kSignature, MAX_BYTE_SHA256_HASH_LENGTH, result);
}

/**
 * @brief Creates the chunk body definition string for a given chunk size and signature.
 * 
 * @param chunkSize The size of the chunk in bytes.
 * @param signature The signature of the chunk.
 * @param payload The payload to which the chunk body will be added.
 * 
 * @note The chunk body will be added at the beginning of the payload and the payload will be shifted
 * to make space for it. The resulting payload will end with CRLF and a 0 terminator.
 */
void getChunkPayload(uint32_t chunkSize, const char *signature, char *payload)
{
    char chunkHexString[UNSIGNED_INT_HEX_STR_LENGTH] = {0};
    char chunkBody[MAX_CHUNK_BODY_LENGTH] = {0};
    /* Convert the decoded payload length from int into hex */
    uintToHex(chunkSize, chunkHexString);
    /* Create the chunk body string */
    snprintf(chunkBody, MAX_CHUNK_BODY_LENGTH, "%s;chunk-signature=%s\r\n", chunkHexString, signature);
    size_t chunkBodyLen = strlen(chunkBody);
    /* Shift the payload to make space for the chunk body
    since the destination/source are overlapping memmove() is used */
    memmove(payload + strlen(chunkBody), payload, chunkSize);
    /* Copy the chunk body at the beginning of the payload */
    memcpy(payload, chunkBody, strlen(chunkBody));
    /* Add CRLF and 0 termiantor at the end of the payload */
    payload[chunkSize + strlen(chunkBody)] = '\r';
    payload[chunkSize + strlen(chunkBody) + 1] = '\n';
    payload[chunkSize + strlen(chunkBody) + 2] = '\0';
}

/* Step 5: Add the signature to the request */
/**
 * @brief Generates an AWS authorization header string for a given set of credentials and request parameters
 *
 * This function generates an authorization header string as required by AWS for authenticated requests. The generated
 * authorization header is added to the HTTP request headers to authenticate the request. The AWS signature version 4
 * signing process is used to generate the signature part of the authorization header. This function is used to sign
 * requests to AWS S3 and AWS API Gateway endpoints.
 *
 * @param algorithm       The algorithm used for generating the signature. This should be 'AWS4-HMAC-SHA256'.
 * @param awsAccessKey    The AWS access key to be used for authentication.
 * @param date            The date and time of the request in the format yyyyMMdd'T'HHmmss'Z'.
 * @param region          The AWS region of the service being requested.
 * @param service         The AWS service to be requested.
 * @param signedHeaders   A string with all header names and values that should be included in the signature
 *                        separated by semicolons. This should include all headers that are being sent with the
 *                        request except for 'Authorization'.
 * @param signature       The signature generated using the signing key and string to sign.
 * @param result          A pointer to a buffer to store the generated authorization header.
 * @param sizeResult      The size of the buffer to store the generated authorization header.
 *
 * @note The size of the buffer pointed to by 'result' should be at least 600 bytes to accommodate the longest possible
 * authorization header string.
 *
 * @return None.
 */
void authorizationHeader(const char *algorithm, const char *awsAccessKey, const char *date, const char *region,
                         const char *service, const char *signedHeaders, const char *signature,
                         char *result, size_t sizeResult)
{
    snprintf(result,
             sizeResult,
             "%s Credential=%s/%s/%s/%s/aws4_request, "
             "SignedHeaders=%s, "
             "Signature=%s",
             algorithm, awsAccessKey, date, region, service, signedHeaders, signature);
}

// https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetObject.html
/* Step 6: Create an http request */
/**
 * @brief Creates an HTTP GET request header string for AWS S3.
 *
 * @param object The name of the object to get.
 * @param contentType The MIME type of the content to get.
 * @param host The S3 bucket host URL.
 * @param payloadHexHash The hexadecimal hash of the payload.
 * @param date The current date in ISO8601 format.
 * @param time The current time in ISO8601 format.
 * @param authorizationHeader The AWS signature authorization header.
 * @param result The output string for the HTTP GET request header.
 * @param sizeResult The size of the output string.
 *
 * @return true if the function executed successfully, false otherwise.
 *
 * @note The size of the result string must be at least 256 bytes.
 */
bool httpGetReq(const char *object, const char *contentType, const char *host, const char *payloadHexHash,
                const char *date, const char *time, const char *authorizationHeader, char *result,
                size_t sizeResult)
{
    bool ret = false;
    int snprintf_ret = snprintf(result,
                                sizeResult,
                                "GET /%s HTTP/1.1\r\n"
                                "Content-Type: %s\r\n"
                                "Host: %s\r\n"
                                "x-amz-content-sha256: %s\r\n"
                                "x-amz-date: %sT%sZ\r\n"
                                "Authorization: %s\r\n"
                                "User-Agent: ESP-32\r\n"
                                "Cache-Control: no-cache\r\n"
                                "Accept: */*\r\n"
                                "Connection: close\r\n"
                                "\r\n",
                                object, contentType, host, payloadHexHash, date, time, authorizationHeader);
    if (snprintf_ret < 0)
    {
        return ret;
    }
    else if (snprintf_ret >= sizeResult)
    {
        Serial.printf("httpGetReq failed: %d\r\n", snprintf_ret);
        return ret;
    }
    else
    {
        ret = true;
        return ret;
    }
}

// https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutObject.html
/* Step 6: Create an http request */
/**
 * @brief Creates an HTTP PUT request header for single chunk data transmission.
 *
 * @param object The name of the object to store in the destination file
 * @param contentType The type of the content being uploaded
 * @param contentLength The length of the content being uploaded (including metadata)
 * @param host The host address to connect to
 * @param payloadHexHash The C-string holding the 32 byte SHA256 hash.
 * @param date The current date in ISO8601 format.
 * @param time The current time in ISO8601 format.
 * @param storageClass The class of storage to be used
 * @param authorizationHeader The authorization header for the request
 * @param result The buffer to write the HTTP request header into
 * @param sizeResult The size of the result buffer
 *
 * @return Returns true if the function is successful, false otherwise
 *
 * @note This function assumes that the result buffer is large enough to hold the entire header
 */
bool httpPutReq(const char *object, const char *contentType, uint32_t contentLength, const char *host,
                const char *payloadHexHash, const char *date, const char *time, const char *storageClass,
                const char *authorizationHeader, char *result, size_t sizeResult)
{
    bool ret = false;
    int snprintf_ret = snprintf(result,
                                sizeResult,
                                "PUT /%s HTTP/1.1\r\n"
                                "Content-Length: %lu\r\n"
                                "Content-Type: %s\r\n"
                                "Host: %s\r\n"
                                "X-Amz-Content-Sha256: %s\r\n"
                                "X-Amz-Date: %sT%sZ\r\n"
                                "X-Amz-Storage-Class: %s\r\n"
                                "Authorization: %s\r\n"
                                "User-Agent: ESP32\r\n"
                                "Accept: */*\r\n"
                                "Cache-Control: no-cache\r\n"
                                "Expect: 100-continue\r\n"
                                "Connection: keep-alive\r\n"
                                "\r\n",
                                object, contentLength, contentType, host, payloadHexHash,
                                date, time, storageClass, authorizationHeader);
    Serial.printf("HTTP header length: %d\r\n", snprintf_ret);
    if (snprintf_ret < 0)
    {
        return ret;
    }
    else if (snprintf_ret >= sizeResult)
    {
        Serial.printf("httpPutReq failed: %d\r\n", snprintf_ret);
        return ret;
    }
    else
    {
        ret = true;
        return ret;
    }
}

// https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutObject.html
/* Step 6: Create an http request */
/**
 * @brief Creates an HTTP PUT request header with necessary metadata for chunked data transmission.
 *
 * This function creates an HTTP PUT request header with the necessary metadata including content length,
 * content type, host, date, time, and authorization headers. The header is written into the result buffer.
 * The function returns true if successful, false otherwise.
 *
 * @param object The name of the object to store in the destination file
 * @param contentType The type of the content being uploaded
 * @param contentLength The length of the content being uploaded (including metadata)
 * @param host The host address to connect to
 * @param date The date in YYYYMMDD format
 * @param time The time in HHMMSSZ format
 * @param payloadLength The length of the actual data to be stored in the destination file
 * @param storageClass The class of storage to be used
 * @param authorizationHeader The authorization header for the request
 * @param result The buffer to write the HTTP request header into
 * @param sizeResult The size of the result buffer
 *
 * @return Returns true if the function is successful, false otherwise
 *
 * @note This function assumes that the result buffer is large enough to hold the entire header
 */
bool httpPutReqStream(const char *object, const char *contentType, uint32_t contentLength, const char *host,
                      const char *date, const char *time, uint32_t payloadLength, const char *storageClass,
                      const char *authorizationHeader, char *result, size_t sizeResult)
{
    /* Payload lenght relates to the the actual data to be stored in the destination file
    Content length is payload length + meta data length! */
    bool ret = false;
    int snprintf_ret = snprintf(result,
                                sizeResult,
                                "PUT /%s HTTP/1.1\r\n"
                                "Content-Encoding: aws-chunked\r\n"
                                "Content-Length: %lu\r\n"
                                "Content-Type: %s\r\n"
                                "Host: %s\r\n"
                                "X-Amz-Content-Sha256: STREAMING-AWS4-HMAC-SHA256-PAYLOAD\r\n"
                                "X-Amz-Date: %sT%sZ\r\n"
                                "X-Amz-Decoded-Content-Length: %lu\r\n"
                                "X-Amz-Storage-Class: %s\r\n"
                                "Authorization: %s\r\n"
                                "User-Agent: ESP32\r\n"
                                "Accept: */*\r\n"
                                "Cache-Control: no-cache\r\n"
                                "Expect: 100-continue\r\n"
                                "Connection: keep-alive\r\n"
                                "\r\n",
                                object, contentLength, contentType, host, date, time,
                                payloadLength, storageClass, authorizationHeader);
    Serial.printf("HTTP header length: %d\r\n", snprintf_ret);
    if (snprintf_ret < 0)
    {
        return ret;
    }
    else if (snprintf_ret >= sizeResult)
    {
        Serial.printf("httpPutReq failed: %d\r\n", snprintf_ret);
        return ret;
    }
    else
    {
        ret = true;
        return ret;
    }
}