How to deploy a release with travis & co?

squix78 · May 26, 2016, 8:01am

I would like to use platformio ci on Travis to create a firmware of a tagged release of an example file and use Travis’ deploy command to write it back to github. Using the OTA feature connected clients could then download that released firmware automatically. The problem seems to be that platformio ci seems to remove the created binaries right after the build. I experimented with the “keep-build-dir” flag and the build dir attribute but either platformio complains that the directory does not exists or it just is not visible after the build.

platformio ci --build-dir="/home/travis/build" --keep-build-dir --lib="." --board=nodemcuv2

Any idea?

ivankravets · May 26, 2016, 2:18pm

You need to override PLATFORMIO_ENVS_DIR environemnt variable. For example,

export PLATFORMIO_ENVS_DIR=~/.pioenvs

squix78 · May 31, 2016, 7:17am

Hi Ivan

Thanks for your answer. This configuration solved the problem for me:

script:

mkdir /tmp/build

platformio ci --build-dir=“/tmp/build” --keep-build-dir --board=nodemcuv2

deploy:
on:
tags: true
all_branches: true
skip_cleanup: true
provider: releases
overwrite: true
api_key:
secure:
file: “/tmp/build/.pioenvs/nodemcuv2/firmware.bin”
on:
repo: squix78/esp8266-ci-ota

snoerenberg · July 29, 2018, 6:07pm

@ivankravets
Hi, I still cannot get it to work, that the gitlab runner can find the artifacts.

I’ve tested it with

    export PLATFORMIO_ENVS_DIR=~/.pioenvs

and with the --keep-build-dir option as well no luck

gitlab runner:

    buildarduino:
      image: python:2.7
      before_script:
       - "pip install -U platformio"
       - "mkdir /tmp/build"
      stage: build
      script: "platformio ci --build-dir=\"/tmp/build\" --keep-build-dir --project-conf platformio.ini --board=megaatmega2560"
      variables: {PLATFORMIO_CI_SRC: "src", PLATFORMIO_ENVS_DIR: "/tmp/build/.pioenvs"}
      artifacts:
        paths:
        - /tmp/build/.pioenvs/megaatmega2560/firmware.elf
        - /tmp/build/.pioenvs/megaatmega2560/firmware.hex

In the log is the following:

The next files/directories have been created in /tmp/build
platformio.ini - Project Configuration File
src - Put your source files here
lib - Put here project specific (private) libraries
...
Cloning into '/tmp/build/.piolibdeps/_tmp_installing-tj7dXm-package'...

    Archiving .pioenvs/megaatmega2560/libFrameworkArduino.a
    Indexing .pioenvs/megaatmega2560/libFrameworkArduino.a
    Linking .pioenvs/megaatmega2560/firmware.elf
    Building .pioenvs/megaatmega2560/firmware.hex
    Calculating size .pioenvs/megaatmega2560/firmware.elf
    AVR Memory Usage
    ----------------
    Device: atmega2560

    Program:   36626 bytes (14.0% Full)
    (.text + .data + .bootloader)

    Data:       3490 bytes (42.6% Full)
    (.data + .bss + .noinit)


    ========================= [SUCCESS] Took 62.76 seconds =========================
    Uploading artifacts...
    WARNING: /tmp/build/.pioenvs/megaatmega2560/firmware.elf: no matching files 
    WARNING: /tmp/build/.pioenvs/megaatmega2560/firmware.hex: no matching files 
    Uploading artifacts to coordinator... ok            id=85176969 responseStatus=201 Created token=svcybWWW
    Job succeeded

Any idea? Thank you
Stephan

ivankravets · July 29, 2018, 6:11pm

Please remove this line.

snoerenberg · July 29, 2018, 6:12pm

Hi, @ivankravets

I already removed it. I just tried it. I now updated my post with correct code blocks showing the actual version.

BTW: I love PlatformIO …

Thanks

ivankravets · July 29, 2018, 6:19pm

Are you sure that remove PLATFORMIO_ENVS_DIR? I’ve just tried locally and can’t reproduce this issue.

snoerenberg · July 29, 2018, 7:06pm

Hi @ivankravets

same behavior - I tested this already but this is the actual config and log

config:

stages:
 - build
 - documentation

buildarduino:
  image: python:2.7
  before_script:
   - "pip install -U platformio"
   - "mkdir /tmp/build"
  stage: build
  script: "platformio ci --build-dir=\"/tmp/build\" --keep-build-dir --project-conf platformio.ini --board=megaatmega2560"
  variables: {PLATFORMIO_CI_SRC: "src"}
  artifacts:
    paths:
    - /tmp/build/.pioenvs/megaatmega2560/firmware.elf
    - /tmp/build/.pioenvs/megaatmega2560/firmware.hex

log:

Running with gitlab-runner 11.1.0 (081978aa)
  on docker-auto-scale 0277ea0f
Using Docker executor with image python:2.7 ...
Pulling docker image python:2.7 ...
Using docker image sha256:17c0fe4e76a553da47dcbd8ea15173482d4733b0237e08ef670a813ab17ac02f for python:2.7 ...
Running on runner-0277ea0f-project-7502892-concurrent-0 via runner-0277ea0f-srm-1532890894-297024c8...
Cloning repository...
Cloning into '/builds/hugo123/planting'...
Checking out 64bc06f6 as master...
Skipping Git submodules setup
$ pip install -U platformio
Collecting platformio
  Downloading https://files.pythonhosted.org/packages/8c/f9/1da2df1b4b66e5ec777df79613ea3e8db3228f6121c7ae15803d79bc5157/platformio-3.5.4-py27-none-any.whl (138kB)
Collecting lockfile<0.13,>=0.9.1 (from platformio)
  Downloading https://files.pythonhosted.org/packages/c8/22/9460e311f340cb62d26a38c419b1381b8593b0bb6b5d1f056938b086d362/lockfile-0.12.2-py2.py3-none-any.whl
Collecting click<6,>=5 (from platformio)
  Downloading https://files.pythonhosted.org/packages/8f/98/14966b6d772fd5fba1eb3bb34a62a7f736d609572493397cdc5715c14514/click-5.1-py2.py3-none-any.whl (65kB)
Collecting semantic-version<3,>=2.5.0 (from platformio)
  Downloading https://files.pythonhosted.org/packages/72/83/f76958017f3094b072d8e3a72d25c3ed65f754cc607fdb6a7b33d84ab1d5/semantic_version-2.6.0.tar.gz
Collecting colorama (from platformio)
  Downloading https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting bottle<0.13 (from platformio)
  Downloading https://files.pythonhosted.org/packages/bd/99/04dc59ced52a8261ee0f965a8968717a255ea84a36013e527944dbf3468c/bottle-0.12.13.tar.gz (70kB)
Collecting requests<3,>=2.4.0 (from platformio)
  Downloading https://files.pythonhosted.org/packages/65/47/7e02164a2a3db50ed6d8a6ab1d6d60b69c4c3fdf57a284257925dfc12bda/requests-2.19.1-py2.py3-none-any.whl (91kB)
Collecting pyserial!=3.3,<4,>=3 (from platformio)
  Downloading https://files.pythonhosted.org/packages/0d/e4/2a744dd9e3be04a0c0907414e2a01a7c88bb3915cbe3c8cc06e209f59c30/pyserial-3.4-py2.py3-none-any.whl (193kB)
Collecting chardet<3.1.0,>=3.0.2 (from requests<3,>=2.4.0->platformio)
  Downloading https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl (133kB)
Collecting urllib3<1.24,>=1.21.1 (from requests<3,>=2.4.0->platformio)
  Downloading https://files.pythonhosted.org/packages/bd/c9/6fdd990019071a4a32a5e7cb78a1d92c53851ef4f56f62a3486e6a7d8ffb/urllib3-1.23-py2.py3-none-any.whl (133kB)
Collecting certifi>=2017.4.17 (from requests<3,>=2.4.0->platformio)
  Downloading https://files.pythonhosted.org/packages/7c/e6/92ad559b7192d846975fc916b65f667c7b8c3a32bea7372340bfe9a15fa5/certifi-2018.4.16-py2.py3-none-any.whl (150kB)
Collecting idna<2.8,>=2.5 (from requests<3,>=2.4.0->platformio)
  Downloading https://files.pythonhosted.org/packages/4b/2a/0276479a4b3caeb8a8c1af2f8e4355746a97fab05a372e4a2c6a6b876165/idna-2.7-py2.py3-none-any.whl (58kB)
Building wheels for collected packages: semantic-version, bottle
  Running setup.py bdist_wheel for semantic-version: started
  Running setup.py bdist_wheel for semantic-version: finished with status 'done'
  Stored in directory: /root/.cache/pip/wheels/60/bb/50/215d669d31f992767f5dd8d3c974e79261707ee7f898f0dc10
  Running setup.py bdist_wheel for bottle: started
  Running setup.py bdist_wheel for bottle: finished with status 'done'
  Stored in directory: /root/.cache/pip/wheels/76/a0/b4/2a3ee1a32d0506931e558530258de1cc04b628eff1b2f008e0
Successfully built semantic-version bottle
Installing collected packages: lockfile, click, semantic-version, colorama, bottle, chardet, urllib3, certifi, idna, requests, pyserial, platformio
Successfully installed bottle-0.12.13 certifi-2018.4.16 chardet-3.0.4 click-5.1 colorama-0.3.9 idna-2.7 lockfile-0.12.2 platformio-3.5.4 pyserial-3.4 requests-2.19.1 semantic-version-2.6.0 urllib3-1.23
$ mkdir /tmp/build
$ platformio ci --build-dir="/tmp/build" --keep-build-dir --project-conf platformio.ini --board=megaatmega2560
********************************************************************************
If you like PlatformIO, please:
- follow us on Twitter to stay up-to-date on the latest project news > https://twitter.com/PlatformIO_Org
- star it on GitHub > https://github.com/platformio/platformio
- try PlatformIO IDE for IoT development > https://platformio.org/platformio-ide
********************************************************************************

The next files/directories have been created in /tmp/build
platformio.ini - Project Configuration File
src - Put your source files here
lib - Put here project specific (private) libraries

Project has been successfully initialized!
Useful commands:
`platformio run` - process/build project from the current directory
`platformio run --target upload` or `platformio run -t upload` - upload firmware to embedded board
`platformio run --target clean` - clean project (remove compiled files)
`platformio run --help` - additional information
Processing megaatmega2560 (platform: atmelavr; board: megaatmega2560; framework: arduino)
--------------------------------------------------------------------------------
LibraryManager: Installing id=136
Downloading...
Unpacking...
LibraryManager: Installing id=1923
Downloading...
Unpacking...
LibraryManager: Installing id=274
Downloading...
Unpacking...
LibraryManager: Installing id=129
Downloading...
Unpacking...
LibraryManager: Installing id=721
Downloading...
Unpacking...
LibraryManager: Installing id=1
Downloading...
Unpacking...
LibraryManager: Installing id=54
Downloading...
Unpacking...
LibraryManager: Installing id=1256
Downloading...
Unpacking...
LibraryManager: Installing DHT_nonblocking
git version 2.11.0
Cloning into '/tmp/build/.piolibdeps/_tmp_installing-dw9Hgl-package'...
PlatformManager: Installing atmelavr
Downloading...
Unpacking...
atmelavr @ 1.9.0 has been successfully installed!
The platform 'atmelavr' has been successfully installed!
The rest of packages will be installed automatically depending on your build environment.
PackageManager: Installing toolchain-atmelavr @ ~1.40902.0
Downloading...
Unpacking...
PackageManager: Installing framework-arduinoavr @ ~1.10621.1
Downloading...
Unpacking...
CorePackageManager: Installing tool-scons @ ~2.20501.4
Downloading...
Unpacking...
tool-scons @ 2.20501.4 has been successfully installed!
Verbose mode can be enabled via `-v, --verbose` option
PLATFORM: Atmel AVR > Arduino Mega or Mega 2560 ATmega2560 (Mega 2560)
SYSTEM: ATMEGA2560 16MHz 8KB RAM (248KB Flash)
Library Dependency Finder -> http://bit.ly/configure-pio-ldf
LDF MODES: FINDER(chain) COMPATIBILITY(soft)
Collected 33 compatible libraries
Scanning dependencies...
Dependency Graph
|-- <LiquidCrystal> 1.3.4
|   |-- <Wire> 1.0
|-- <LCDMenuLib2> 1.2.7
|-- <RTC> 2.1.1
|-- <Encoder>
|-- <TaskScheduler> 2.6.1
|-- <OneWire> 2.3.4
|-- <DallasTemperature> 3.8.0
|   |-- <OneWire> 2.3.4
|-- <FanController> 1.0.2
|-- <DHT_nonblocking> 1.0 #7f4da74
|-- <Wire> 1.0
|-- <EEPROM> 2.0
Compiling .pioenvs/megaatmega2560/src/main.cpp.o
Compiling .pioenvs/megaatmega2560/lib8a1/Wire/Wire.cpp.o
Compiling .pioenvs/megaatmega2560/lib8a1/Wire/utility/twi.c.o
Archiving .pioenvs/megaatmega2560/lib8a1/libWire.a
Indexing .pioenvs/megaatmega2560/lib8a1/libWire.a
Compiling .pioenvs/megaatmega2560/libb64/LiquidCrystal_ID136/FastIO.cpp.o
Compiling .pioenvs/megaatmega2560/libb64/LiquidCrystal_ID136/I2CIO.cpp.o
Compiling .pioenvs/megaatmega2560/libb64/LiquidCrystal_ID136/LCD.cpp.o
Compiling .pioenvs/megaatmega2560/libb64/LiquidCrystal_ID136/LiquidCrystal.cpp.o
Compiling .pioenvs/megaatmega2560/libb64/LiquidCrystal_ID136/LiquidCrystal_I2C.cpp.o
Compiling .pioenvs/megaatmega2560/libb64/LiquidCrystal_ID136/LiquidCrystal_I2C_ByVac.cpp.o
Compiling .pioenvs/megaatmega2560/libb64/LiquidCrystal_ID136/LiquidCrystal_SI2C.cpp.o
Compiling .pioenvs/megaatmega2560/libb64/LiquidCrystal_ID136/LiquidCrystal_SR.cpp.o
Compiling .pioenvs/megaatmega2560/libb64/LiquidCrystal_ID136/LiquidCrystal_SR1W.cpp.o
In file included from .piolibdeps/LiquidCrystal_ID136/LiquidCrystal_SR1W.cpp:36:0:
.piolibdeps/LiquidCrystal_ID136/LiquidCrystal_SR1W.h:157:1: warning: multi-line comment [-Wcomment]
//             |    |      0.1uF            |     \
^
Compiling .pioenvs/megaatmega2560/libb64/LiquidCrystal_ID136/LiquidCrystal_SR2W.cpp.o
Compiling .pioenvs/megaatmega2560/libb64/LiquidCrystal_ID136/LiquidCrystal_SR3W.cpp.o
Compiling .pioenvs/megaatmega2560/libb64/LiquidCrystal_ID136/SI2CIO.cpp.o
Archiving .pioenvs/megaatmega2560/libb64/libLiquidCrystal_ID136.a
Indexing .pioenvs/megaatmega2560/libb64/libLiquidCrystal_ID136.a
Compiling .pioenvs/megaatmega2560/lib717/LCDMenuLib2_ID1923/LCDMenuLib2.cpp.o
Compiling .pioenvs/megaatmega2560/lib717/LCDMenuLib2_ID1923/LCDMenuLib2_menu.cpp.o
Archiving .pioenvs/megaatmega2560/lib717/libLCDMenuLib2_ID1923.a
Indexing .pioenvs/megaatmega2560/lib717/libLCDMenuLib2_ID1923.a
Compiling .pioenvs/megaatmega2560/lib8a8/RTC_ID274/RtcDateTime.cpp.o
Compiling .pioenvs/megaatmega2560/lib8a8/RTC_ID274/RtcUtility.cpp.o
Archiving .pioenvs/megaatmega2560/lib8a8/libRTC_ID274.a
Indexing .pioenvs/megaatmega2560/lib8a8/libRTC_ID274.a
Compiling .pioenvs/megaatmega2560/lib645/Encoder_ID129/Encoder.cpp.o
Archiving .pioenvs/megaatmega2560/lib645/libEncoder_ID129.a
Indexing .pioenvs/megaatmega2560/lib645/libEncoder_ID129.a
Archiving .pioenvs/megaatmega2560/lib3b7/libTaskScheduler_ID721.a
Indexing .pioenvs/megaatmega2560/lib3b7/libTaskScheduler_ID721.a
Compiling .pioenvs/megaatmega2560/lib379/OneWire_ID1/OneWire.cpp.o
Archiving .pioenvs/megaatmega2560/lib379/libOneWire_ID1.a
Indexing .pioenvs/megaatmega2560/lib379/libOneWire_ID1.a
Compiling .pioenvs/megaatmega2560/lib7a0/DallasTemperature_ID54/DallasTemperature.cpp.o
Archiving .pioenvs/megaatmega2560/lib7a0/libDallasTemperature_ID54.a
Indexing .pioenvs/megaatmega2560/lib7a0/libDallasTemperature_ID54.a
Compiling .pioenvs/megaatmega2560/lib24a/FanController_ID1256/FanController.cpp.o
.piolibdeps/FanController_ID1256/FanController.cpp: In member function 'unsigned int FanController::getSpeed()':
.piolibdeps/FanController_ID1256/FanController.cpp:37:1: warning: control reaches end of non-void function [-Wreturn-type]
}
^
Archiving .pioenvs/megaatmega2560/lib24a/libFanController_ID1256.a
Indexing .pioenvs/megaatmega2560/lib24a/libFanController_ID1256.a
Compiling .pioenvs/megaatmega2560/libf3a/DHT_nonblocking/dht_nonblocking.cpp.o
Archiving .pioenvs/megaatmega2560/libf3a/libDHT_nonblocking.a
Indexing .pioenvs/megaatmega2560/libf3a/libDHT_nonblocking.a
Archiving .pioenvs/megaatmega2560/lib26e/libEEPROM.a
Indexing .pioenvs/megaatmega2560/lib26e/libEEPROM.a
Archiving .pioenvs/megaatmega2560/libFrameworkArduinoVariant.a
Indexing .pioenvs/megaatmega2560/libFrameworkArduinoVariant.a
Compiling .pioenvs/megaatmega2560/FrameworkArduino/CDC.cpp.o
Compiling .pioenvs/megaatmega2560/FrameworkArduino/HardwareSerial.cpp.o
Compiling .pioenvs/megaatmega2560/FrameworkArduino/HardwareSerial0.cpp.o
Compiling .pioenvs/megaatmega2560/FrameworkArduino/HardwareSerial1.cpp.o
Compiling .pioenvs/megaatmega2560/FrameworkArduino/HardwareSerial2.cpp.o
Compiling .pioenvs/megaatmega2560/FrameworkArduino/HardwareSerial3.cpp.o
Compiling .pioenvs/megaatmega2560/FrameworkArduino/IPAddress.cpp.o
Compiling .pioenvs/megaatmega2560/FrameworkArduino/PluggableUSB.cpp.o
Compiling .pioenvs/megaatmega2560/FrameworkArduino/Print.cpp.o
Compiling .pioenvs/megaatmega2560/FrameworkArduino/Stream.cpp.o
Compiling .pioenvs/megaatmega2560/FrameworkArduino/Tone.cpp.o
Compiling .pioenvs/megaatmega2560/FrameworkArduino/USBCore.cpp.o
Compiling .pioenvs/megaatmega2560/FrameworkArduino/WInterrupts.c.o
Compiling .pioenvs/megaatmega2560/FrameworkArduino/WMath.cpp.o
Compiling .pioenvs/megaatmega2560/FrameworkArduino/WString.cpp.o
Compiling .pioenvs/megaatmega2560/FrameworkArduino/abi.cpp.o
Compiling .pioenvs/megaatmega2560/FrameworkArduino/hooks.c.o
Compiling .pioenvs/megaatmega2560/FrameworkArduino/main.cpp.o
Compiling .pioenvs/megaatmega2560/FrameworkArduino/new.cpp.o
Compiling .pioenvs/megaatmega2560/FrameworkArduino/wiring.c.o
Compiling .pioenvs/megaatmega2560/FrameworkArduino/wiring_analog.c.o
Compiling .pioenvs/megaatmega2560/FrameworkArduino/wiring_digital.c.o
Compiling .pioenvs/megaatmega2560/FrameworkArduino/wiring_pulse.S.o
Compiling .pioenvs/megaatmega2560/FrameworkArduino/wiring_pulse.c.o
Compiling .pioenvs/megaatmega2560/FrameworkArduino/wiring_shift.c.o
Archiving .pioenvs/megaatmega2560/libFrameworkArduino.a
Indexing .pioenvs/megaatmega2560/libFrameworkArduino.a
Linking .pioenvs/megaatmega2560/firmware.elf
Building .pioenvs/megaatmega2560/firmware.hex
Calculating size .pioenvs/megaatmega2560/firmware.elf
AVR Memory Usage
----------------
Device: atmega2560

Program:   36626 bytes (14.0% Full)
(.text + .data + .bootloader)

Data:       3490 bytes (42.6% Full)
(.data + .bss + .noinit)


========================= [SUCCESS] Took 52.69 seconds =========================
Uploading artifacts...
WARNING: /tmp/build/.pioenvs/megaatmega2560/firmware.elf: no matching files 
WARNING: /tmp/build/.pioenvs/megaatmega2560/firmware.hex: no matching files 
Uploading artifacts to coordinator... ok            id=85181012 responseStatus=201 Created token=qeSHX2DW
Job succeeded

snoerenberg · July 30, 2018, 5:42am

@ivankravets
I got it to work with the following config:
The artifacts must be within the $CI_PROJECT_DIR when using Gitlab Runner with artifacts.

buildarduino:
  image: python:2.7
  before_script:
   - "pip install -U platformio"
   - "mkdir -p $CI_PROJECT_DIR/tmp/build"
  stage: build
  script: "platformio ci --build-dir=\"$CI_PROJECT_DIR/tmp/build\" --keep-build-dir --project-conf platformio.ini --board=megaatmega2560"
  variables: {PLATFORMIO_CI_SRC: "src"}
  artifacts:
    paths:
    - tmp/build/.pioenvs/megaatmega2560/firmware.elf
    - tmp/build/.pioenvs/megaatmega2560/firmware.hex

snoerenberg · July 30, 2018, 6:35am

Where is the “.pioenvs” directory normally created? Maybe you could change the behavior that it is using $CI_PROJECT_DIR automagically?

It seems not to be created in the working directory in the case of the gitlab running?

Thanks
Stephan