PlatformIO Community

Help sought to integrate Github repo as project

Hi guys,

I am looking for a step-by step guide or similar to set up a Github repository (namely, this one) as a project in PlatformIO. I read several advice I must confess did not completely understand about Github actions, YAML etc. I imagine there must be a standard way to do it, as it should be a common task - or am I wrong?

The general documentation is at https://docs.platformio.org/en/latest/integration/ci/github-actions.html.

The exact configuration depends on what you want to do (compile/build check, automatically deploying to some board, automatically unit testing on some board, …)

Well, I read the doc, but found it being a bit too general. What I am looking for is a cooking recipe like

  • git clone your repository
  • Go into the local repo folder and create a platformio.ini file with the following content: …
  • open VSCode/PlatformIO, go to PlatformIO Home and…

Is that too simple-minded? I reckon these first steps will be quite similar for anyone starting.

The only reason I am diving into this is to make some local changes I applied to the library available as pull request for the original author. I am not into Github normally, just for using some libraries for my own projects.

How come Github is able to open the VSCode GUI and do stuff? o_o
These are Github server based actions, not on your local computer. It still can fully operate PlatformIO from the shell and build the project or the library as a CI project etc.

The stuff behind “and…” is criticial. What’s Github supposed to do with the PlatformIO project?

I think there is some confusion about what exactly you want to do. I take it you want to know how to use that library in a new project?

Something along the lines of:

  • Open VSCode
  • Goto PlatformIO Home
  • Click the ‘New Project’ button and create a new Arduino project choosing your particular ESP32 board
  • Click Finish
  • Once the new project has been created, open the platformio.ini and add lib_deps = Miq1/esp32ModbusRTU to the bottom of the file.

Sorry if I wasn’t clear with that. I already am doing what you are suggesting, so my project is in fact utilizing the modified library.
But I would like to be able to (automagically) modify my local copy of the library files and have these changes available for commits to the Github repository.
As I understood it so far, the lib_deps= parameter lets me use the library as it is on Github and will take care to keep it up-to-date in PlatformIO, but not vice versa.
What I am looking for is the opposite way, integrated into PlatformIO.
I did not try it yet, but will it at least work to use git to accept the lib_deps/esp32ModbusRTU directory as local clone directory? Then I could do changes from within VSCode/PlatformIO in the project tree and do my commits with git from the command line without having to maintain another copy of the library for Github.

1 Like

Right. So, to do that - edit the library and manage git commits and pushes to GitHub via VSCode - since the repo already exists on GitHub, I think your best option is to (if you haven’t already) clone the repo somewhere, and then simply open that folder in VSCode (File -> Open folder, or if you want to have a project open as well as the library File -> Add Folder to Workspace). VSCode won’t know anything about PlatformIO support for the library, but you will be able to edit the source code, and use VSCode’s built in git/source control support to make commits and push them to github.

The VSCode source control panel:
image

Options to pull, push, etc.
image

Maybe have a look at the VSCode documentation on this feature also, to get acquainted with it.

Thanks a lot! I will it give a try today :grin:

1 Like

Don’t hesitate to ask questions if you get stuck… it’s not ‘simple’, but it’s worth the learning curve (in my opinion) :wink: Means you can made changes to code, commit, make changes, commit, then push it when you’re ready, etc.

@pfeerick: my first pull request done the way you suggested is on the way. Regarding the learning curve I see the first upward slope :wink:
I got a number of error lines as

> git rev-parse --show-toplevel
fatal: not a git repository (or any of the parent directories): .git

Is that something I should be worrying about? The functions I used (checkout, stage, confirm, push) worked nevertheless.

Is that from the Git logs? i.e.

image

If so, not sure why you’re getting that. But if it is working, probably safe to ignore it. Maybe scroll further up the log, and see what repository it was trying to open - might give a clue as to what is going on.

Yes, that is in the git log, multiple times. There are streaks of 10 and more identical 2-line messages, with the “real” messages spread between those.

Is the “.git” mentioned in the message somewhere to have a look at?

I think it’s likely the that VSCode is simply seeing a folder/project in the workspace that isn’t tracked by git, and isn’t smart enough to only run the git commands in non-git-tracked folders/projects.

For every directory that has had a git repo initialised, or has been cloned, etc, there should be a .git folder (often hidden by your OS), so once you work out which .git folder it is talking about, you may be able to delete it, and try again. I.e. on my output, if I scrolled back , found a ‘opening repo’ line, and then started seeing the not a git repository after that, I’d be suspecting it as the culprit.

Unfortunately nothing is to be seen before the messages are starting. This is the log right away after starting VSCode:

Looking for git in: C:\Program Files\Git\cmd\git.exe
Using git 2.27.0.windows.1 from C:\Program Files\Git\cmd\git.exe
> git rev-parse --show-toplevel
fatal: not a git repository (or any of the parent directories): .git
> git rev-parse --show-toplevel
fatal: not a git repository (or any of the parent directories): .git
> git rev-parse --show-toplevel
fatal: not a git repository (or any of the parent directories): .git
> git rev-parse --show-toplevel
fatal: not a git repository (or any of the parent directories): .git
> git rev-parse --show-toplevel
fatal: not a git repository (or any of the parent directories): .git
> git rev-parse --show-toplevel
fatal: not a git repository (or any of the parent directories): .git
> git rev-parse --show-toplevel
> git rev-parse --git-dir
Open repository: c:\Users\Micha\esp32ModbusRTU
> git fetch
> git status -z -u
> git symbolic-ref --short HEAD
> git rev-parse master
> git rev-parse --symbolic-full-name master@{u}
> git rev-list --left-right master...refs/remotes/origin/master
> git check-ignore -v -z --stdin
> git for-each-ref --sort -committerdate --format %(refname) %(objectname)
> git remote --verbose
Warning: Failed to watch ref 'c:\Users\Micha\esp32ModbusRTU\.git\refs\remotes\origin\master', is most likely packed.
> git config --get commit.template
> git rev-parse --show-toplevel
> git rev-parse --show-toplevel
> git rev-parse --show-toplevel
> git rev-parse --show-toplevel
fatal: not a git repository (or any of the parent directories): .git
> git rev-parse --show-toplevel
> git rev-parse --show-toplevel
fatal: not a git repository (or any of the parent directories): .git
> git rev-parse --show-toplevel
> git status -z -u
fatal: not a git repository (or any of the parent directories): .git
> git rev-parse --show-toplevel
> git rev-parse --show-toplevel
> git rev-parse --show-toplevel
> git symbolic-ref --short HEAD
fatal: not a git repository (or any of the parent directories): .git
> git rev-parse --show-toplevel
> git rev-parse master
fatal: not a git repository (or any of the parent directories): .git
> git rev-parse --show-toplevel
> git rev-parse --show-toplevel
> git rev-parse --symbolic-full-name master@{u}
fatal: not a git repository (or any of the parent directories): .git
> git rev-parse --show-toplevel
> git rev-list --left-right master...refs/remotes/origin/master
fatal: not a git repository (or any of the parent directories): .git
> git rev-parse --show-toplevel
> git rev-parse --show-toplevel
> git for-each-ref --sort -committerdate --format %(refname) %(objectname)
> git remote --verbose
fatal: not a git repository (or any of the parent directories): .git
> git rev-parse --show-toplevel
> git rev-parse --show-toplevel
Warning: Failed to watch ref 'c:\Users\Micha\esp32ModbusRTU\.git\refs\remotes\origin\master', is most likely packed.
> git config --get commit.template
fatal: not a git repository (or any of the parent directories): .git
> git rev-parse --show-toplevel
fatal: not a git repository (or any of the parent directories): .git
> git rev-parse --show-toplevel
fatal: not a git repository (or any of the parent directories): .git
> git rev-parse --show-toplevel
> git rev-parse --show-toplevel
fatal: not a git repository (or any of the parent directories): .git
> git rev-parse --show-toplevel
fatal: not a git repository (or any of the parent directories): .git
> git rev-parse --show-toplevel
> git rev-parse --show-toplevel
fatal: not a git repository (or any of the parent directories): .git
> git rev-parse --show-toplevel
> git rev-parse --show-toplevel
fatal: not a git repository (or any of the parent directories): .git
> git rev-parse --show-toplevel
fatal: not a git repository (or any of the parent directories): .git
> git rev-parse --show-toplevel
fatal: not a git repository (or any of the parent directories): .git
> git rev-parse --show-toplevel
> git rev-parse --show-toplevel
fatal: not a git repository (or any of the parent directories): .git
> git rev-parse --show-toplevel
fatal: not a git repository (or any of the parent directories): .git
> git rev-parse --show-toplevel
> git rev-parse --show-toplevel
fatal: not a git repository (or any of the parent directories): .git
> git rev-parse --show-toplevel
> git rev-parse --show-toplevel
fatal: not a git repository (or any of the parent directories): .git
> git rev-parse --show-toplevel
fatal: not a git repository (or any of the parent directories): .git
> git rev-parse --show-toplevel
fatal: not a git repository (or any of the parent directories): .git
> git rev-parse --show-toplevel
> git rev-parse --show-toplevel
fatal: not a git repository (or any of the parent directories): .git
> git rev-parse --show-toplevel
fatal: not a git repository (or any of the parent directories): .git
> git rev-parse --show-toplevel
> git rev-parse --show-toplevel
fatal: not a git repository (or any of the parent directories): .git
> git rev-parse --show-toplevel
> git rev-parse --show-toplevel
fatal: not a git repository (or any of the parent directories): .git
> git rev-parse --show-toplevel
fatal: not a git repository (or any of the parent directories): .git
> git rev-parse --show-toplevel
fatal: not a git repository (or any of the parent directories): .git
> git rev-parse --show-toplevel
> git rev-parse --show-toplevel
fatal: not a git repository (or any of the parent directories): .git
> git rev-parse --show-toplevel
fatal: not a git repository (or any of the parent directories): .git
> git rev-parse --show-toplevel
> git rev-parse --show-toplevel
> git rev-parse --show-toplevel
> git rev-parse --show-toplevel
fatal: not a git repository (or any of the parent directories): .git
> git rev-parse --show-toplevel
> git rev-parse --show-toplevel
fatal: not a git repository (or any of the parent directories): .git
> git rev-parse --show-toplevel
fatal: not a git repository (or any of the parent directories): .git
> git rev-parse --show-toplevel
> git rev-parse --show-toplevel
> git rev-parse --show-toplevel
fatal: not a git repository (or any of the parent directories): .git
> git rev-parse --show-toplevel
fatal: not a git repository (or any of the parent directories): .git
> git rev-parse --show-toplevel
> git rev-parse --show-toplevel
fatal: not a git repository (or any of the parent directories): .git
> git rev-parse --show-toplevel
fatal: not a git repository (or any of the parent directories): .git
> git show --textconv :src/esp32ModbusRTU.cpp
> git ls-files --stage -- C:\Users\Micha\esp32ModbusRTU\src\esp32ModbusRTU.cpp
> git cat-file -s 2bad4392c08657f5bf0d7c9aaa789e28d9dfd143
> git show --textconv :src/esp32ModbusRTU.cpp
> git ls-files --stage -- C:\Users\Micha\esp32ModbusRTU\src\esp32ModbusRTU.cpp
> git cat-file -s 2bad4392c08657f5bf0d7c9aaa789e28d9dfd143
> git rev-parse --show-toplevel
fatal: not a git repository (or any of the parent directories): .git
> git rev-parse --show-toplevel
fatal: not a git repository (or any of the parent directories): .git
> git rev-parse --show-toplevel
fatal: not a git repository (or any of the parent directories): .git
> git rev-parse --show-toplevel
fatal: not a git repository (or any of the parent directories): .git
> git rev-parse --show-toplevel
fatal: not a git repository (or any of the parent directories): .git
> git rev-parse --show-toplevel
fatal: not a git repository (or any of the parent directories): .git

The repository is opened without problems; it looks like every single git command is triggering these messages somehow, while working correctly. The only indication that something could be wrong ist this line:

Warning: Failed to watch ref 'c:\Users\Micha\esp32ModbusRTU\.git\refs\remotes\origin\master', is most likely packed.

In the directory C:\Users\Micha\esp32ModbusRTU there is in fact no .git subdirectory. Should it?

Addendum: if I do git rev-parse --show-toplevel by hand in VSCode’s terminal, all seems to be fine:

PS C:\Users\Micha\esp32ModbusRTU> git rev-parse --show-toplevel
C:/Users/Micha/esp32ModbusRTU
PS C:\Users\Micha\esp32ModbusRTU> 

I googled around a bit and found numerous mentionings of both the effects. They seem to be somewhat intrinsic to VSCode and so far no solution seems to exist. As these things seem to do no harm to my git operations, so I am going to ignore them.

I am confused, though, about the best way to maintain my changes to the lib. The setup is as follows:

  • original repo is https://github.com/bertmelis/esp32ModbusRTU
  • my fork is https://github.com/Miq1/esp32ModbusRTU
  • I created several branches, each one containing the changes for one topic, and committed these to my fork
  • I created pull requests from the commits
  • the original author accepted some of the pull requests and merged those into his repo
  • I did a fetch and merge of the original repo to integrate these changes into my master branch
  • I did a manual git branch -d for the branches that were fetched/merged

I got some questions regarding this multi-step workflow

  1. Is there a better way to achieve the same result?
  2. why are pull requests based on commits and not branches? Or did I mistake that?
  3. why are the changes fetched from the original repo not identified to be identical to my pull request branches?

I am finding that a tad complicated; I imagined the changes I made in a branch would be recognizable throughout the whole process of pull request/merge original/fetch/merge master, but it looks to me like the connections have to be made by me manually. :thinking:

They’re not - they are based on branches - hence a message when you do a PR that indicates you can keep pushing commits to that branch to update the PR. Or as the documentation puts it:

you can push commits from your topic branch to add them to your existing pull request.

As far as ‘is there a better way to achieve the same result’ - Probably not. That would be my workflow also if making non-related suggestions/corrections, etc.

Thank you for the clarification. Well, so be it.

I got the impression of pull requests basing on commits because I saw all commits regardless of the branch they had been created from in a single list on the Github page, the header being the name of just one branch. I will have a closer look to find out why.

1 Like

Possibly once they were merged into the original project which you forked. Then they effectively lose the connection to your branch, which was really only a way to group a bunch of commits anyway.

That would in fact explain it. Thanks again!

1 Like