Removing Secrets from SRC

1

Hey Guys,
i want to publish my project to github, and i try to find a way to remove my Credentials out of my repository.

I found out that it is possible to define secrets via buildflag, and if i define them in platform.ini it works.
I don’t want to simply exlude the files, because i also manage dependencys there.
If i put the same string, that worked in the platformio.ini into the corresponding enviroment variable (which is quite easy, because i use the github ci), it doesn’t work.

So i have two Questions:
What is the best practice to hide these secrets?
What format is required inside the enviroment variable?

The build happens on a linux server.

Greetings

2

What exaxct code / commands have you tried there?

Which ones, on the basis of what documentation?

When I see sketches / firmwares using e.g. WiFi access credetnials, they offload it into another file, e.g. wifi_secrets.h. This file is not published in the repo and everyone building the firmware is expected to create this file themselves in the format specified in the firmware.

Another possibility be to expect a certain set of environment variables on the building system, which can be translated into additional build_flags by a python script. The basis of this is shown in the Advanced Scripting doc. You can e.g. add the line

extra_scripts = pre:add_secrets.py

to the platformio.ini and then have add_secrets.py

Import("env", "projenv")
import os

# check if environment variable exists (try/except), then read out.. 
secret = os.environ['MAGIC_ENVIRON_VAR_SECRET']

# add macro definition for code to use in projenv (visible to src/ files) 
projenv.Append(CPPDEFINES=[
  ("SECRET_VALUE_MACRO", secret)
])
1 Like
3

By the way, if you already published your creds once: You have to do some extra work to get them removed from the history, another commit removing the file is not enough. There’s a nice story about it.

1 Like
4

I tried it with the github ci pipeline (actions).
This requires a yaml file like this:

 env: 
        PLATFORMIO_BUILD_FLAGS: ${{ secrets.pio_build_flag }}

I verifyed that the variable where available and the “right” ones, with a followed up little bash script.

PLATFORMIO_BUILD_FLAGS

5

Thank you for your suggestion. Adding just a .h file seemd very blunt to me, but if it is usual that users have to do this, then well…

I did not publish anything yet, i just need to rewrite the git history, but i will do it with this tool:

6

Well this looks good an like just what you want… If that’s not working it’s either the wrong option, the format of the content of this variable is wrong or the code doesn’t handle it. Maybe @ivankravets knows more?

Doesn’t work meaning there’s a compiler error? When you use it as environment variable, do you apply some string escaping to it as shown in the docs?

7

It depends, but generally yes. Most times it didn’t got recognized, so the compiler throws an error becouse the token is not defined.

The string escaping in the docs didn’t work, but yes, i tried it with and without string escaping.

But i got an working solution now, which don’t even looks so ugly.
I define the build flags with dummy values in the platformio.ini, and replace in the cd pipeline the dummys with the real values. sed:

sed -i ‘s/verySecretPassword/${{ secrets.WIFI_PASS }}/g’ platformio.ini

My Platformio.ini has now this line in it:

build_flags = ‘-DWIFI_PASS=“verySecretPassword”’, ‘-DWIFI_SSID=“niceWIFIName”’, ‘-DThingSpeakChannelNumber=0000’, ‘-DThingSpeakWriteAPIKey=“ABCDEF”’

Sorry for the hassle.

1 Like
8

What is a content of this template variable? have you tried to print env in shell to check environment variable has valid value?

9

something along this:

‘-DWIFI_PASS=“PASSWORD”’ ‘-DWIFI_SSID=“SSID”’

Yes, i echod them and piped them into a file. They were correctly placed.

I also tried some escaping:
'-DWIFI_PASS="PASSWORD"' '-DWIFI_SSID="SSID"'
‘-DWIFI_PASS="PASSWORD"’ ‘-DWIFI_SSID="SSID"’

10

i just notice, that my password has an _ in it. I did not escape this, maybe this was the reason.

11

Meaning this was the reason, it’s solved now? It works with simple strings which have like no spaces and no special characters in it?

12

No need to put ', just use -DWIFI_PASS=mypass…

13

This does not work.

i tried:
build_flags = -DWIFI_PASS=“verySecretPassword”, -DWIFI_SSID=“niceWIFIName”, 'DThingSpeakChannelNumber=0000, -DThingSpeakWriteAPIKey=“ABCDEF”

build_flags = ‘-DWIFI_PASS=“verySecretPassword”’, ‘-DWIFI_SSID=“niceWIFIName”’, ‘-DThingSpeakChannelNumber=0000’, ‘-DThingSpeakWriteAPIKey=“ABCDEF”’

The second one works, the first one does not not work

14

I don’t know yet, but i can test it tomorrow.

15

No need to use comma.

You can also pass each new macro in a new line.

build_flags = 
  -D1
  -D2
16

I tried this, and it works.
I still need to enclose them in ’ ’ though, becouse the strings don’t get recognized as strings, if i leave them out.
But it already looks better like this.

build_flags = 
  '-DWIFI_PASS="verySecretPassword"'
  '-DWIFI_SSID="niceWIFIName"'
  '-DThingSpeakChannelNumber=0000'
  '-DThingSpeakWriteAPIKey="ABCDEF"'
1 Like