Add a git repository as a lib to travis build

Trying to get travis to build a library example using platformio.

It has library dependencies that are not platformio libs. that I would like to clone into travis for use a lib.

Do you have an example of how to do this?

many thanks

ok scratch that. found the docs. silly me

OK so something else. Can’t get travis to work. can’t see why. Using it to build a library.

here is the travis output

Using worker: worker-linux-docker-26e14ea0.prod.travis-ci.org:travis-linux-13
system_info
Build system information
Build language: python
Build group: stable
Build dist: precise
Build image provisioning date and time
Thu Feb  5 15:09:33 UTC 2015
Operating System Details
Distributor ID:	Ubuntu
Description:	Ubuntu 12.04.5 LTS
Release:	12.04
Codename:	precise
Linux Version
3.13.0-29-generic
Cookbooks Version
a68419e https://github.com/travis-ci/travis-cookbooks/tree/a68419e
GCC version
gcc (Ubuntu/Linaro 4.6.3-1ubuntu5) 4.6.3
Copyright (C) 2011 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
LLVM version
clang version 3.4 (tags/RELEASE_34/final)
Target: x86_64-unknown-linux-gnu
Thread model: posix
Pre-installed Ruby versions
ruby-1.9.3-p551
Pre-installed Node.js versions
v0.10.36
Pre-installed Go versions
1.4.1
Redis version
redis-server 2.8.19
riak version
2.0.2
MongoDB version
MongoDB 2.4.12
CouchDB version
couchdb 1.6.1
Neo4j version
1.9.4
RabbitMQ Version
3.4.3
ElasticSearch version
1.4.0
Installed Sphinx versions
2.0.10
2.1.9
2.2.6
Default Sphinx version
2.2.6
Installed Firefox version
firefox 31.0esr
PhantomJS version
1.9.8
ant -version
Apache Ant(TM) version 1.8.2 compiled on December 3 2011
mvn -version
Apache Maven 3.2.5 (12a6b3acb947671f09b81f49094c53f426d8cea1; 2014-12-14T17:29:23+00:00)
Maven home: /usr/local/maven
Java version: 1.7.0_76, vendor: Oracle Corporation
Java home: /usr/lib/jvm/java-7-oracle/jre
Default locale: en_US, platform encoding: ANSI_X3.4-1968
OS name: "linux", version: "3.13.0-29-generic", arch: "amd64", family: "unix"
fix.CVE-2015-7547
$ export DEBIAN_FRONTEND=noninteractive
W: Size of file /var/lib/apt/lists/us.archive.ubuntu.com_ubuntu_dists_precise-backports_multiverse_source_Sources.gz is not what the server reported 5886 5888
W: Size of file /var/lib/apt/lists/ppa.launchpad.net_ubuntugis_ppa_ubuntu_dists_precise_main_binary-amd64_Packages.gz is not what the server reported 36669 36677
W: Size of file /var/lib/apt/lists/ppa.launchpad.net_ubuntugis_ppa_ubuntu_dists_precise_main_binary-i386_Packages.gz is not what the server reported 36729 36733
Reading package lists...
Building dependency tree...
Reading state information...
The following extra packages will be installed:
  libc-bin libc-dev-bin libc6-dev
Suggested packages:
  glibc-doc
The following packages will be upgraded:
  libc-bin libc-dev-bin libc6 libc6-dev
4 upgraded, 0 newly installed, 0 to remove and 234 not upgraded.
Need to get 8,840 kB of archives.
After this operation, 14.3 kB disk space will be freed.
Get:1 http://us.archive.ubuntu.com/ubuntu/ precise-updates/main libc6-dev amd64 2.15-0ubuntu10.15 [2,943 kB]
Get:2 http://us.archive.ubuntu.com/ubuntu/ precise-updates/main libc-dev-bin amd64 2.15-0ubuntu10.15 [84.7 kB]
Get:3 http://us.archive.ubuntu.com/ubuntu/ precise-updates/main libc-bin amd64 2.15-0ubuntu10.15 [1,177 kB]
Get:4 http://us.archive.ubuntu.com/ubuntu/ precise-updates/main libc6 amd64 2.15-0ubuntu10.15 [4,636 kB]
Fetched 8,840 kB in 0s (36.7 MB/s)
Preconfiguring packages ...
(Reading database ... 72431 files and directories currently installed.)
Preparing to replace libc6-dev 2.15-0ubuntu10.10 (using .../libc6-dev_2.15-0ubuntu10.15_amd64.deb) ...
Unpacking replacement libc6-dev ...
Preparing to replace libc-dev-bin 2.15-0ubuntu10.10 (using .../libc-dev-bin_2.15-0ubuntu10.15_amd64.deb) ...
Unpacking replacement libc-dev-bin ...
Preparing to replace libc-bin 2.15-0ubuntu10.10 (using .../libc-bin_2.15-0ubuntu10.15_amd64.deb) ...
Unpacking replacement libc-bin ...
Processing triggers for man-db ...
Setting up libc-bin (2.15-0ubuntu10.15) ...
(Reading database ... 72430 files and directories currently installed.)
Preparing to replace libc6 2.15-0ubuntu10.10 (using .../libc6_2.15-0ubuntu10.15_amd64.deb) ...
Unpacking replacement libc6 ...
Setting up libc6 (2.15-0ubuntu10.15) ...
Setting up libc-dev-bin (2.15-0ubuntu10.15) ...
Setting up libc6-dev (2.15-0ubuntu10.15) ...
Processing triggers for libc-bin ...
ldconfig deferred processing now taking place
git.checkout
1.92s$ git clone --depth=50 --branch=ASYNC https://github.com/sticilface/ESPmanager.git sticilface/ESPmanager
Cloning into 'sticilface/ESPmanager'...
remote: Counting objects: 235, done.
remote: Compressing objects: 100% (147/147), done.
remote: Total 235 (delta 115), reused 191 (delta 74), pack-reused 0
Receiving objects: 100% (235/235), 657.78 KiB | 0 bytes/s, done.
Resolving deltas: 100% (115/115), done.
Checking connectivity... done.
$ cd sticilface/ESPmanager
$ git checkout -qf fcde2134db34828318617ad06fa482fe7e7f40c6
This job is running on container-based infrastructure, which does not allow use of 'sudo', setuid and setguid executables.
If you require sudo, add 'sudo: required' to your .travis.yml
See https://docs.travis-ci.com/user/workers/container-based-infrastructure/ for details.
Setting environment variables from .travis.yml
$ export PLATFORMIO_CI_SRC=examples/ESPmanager-example/ESPmanager-example.ino
0.00s$ source ~/virtualenv/python2.7/bin/activate
cache.1
Setting up build cache
$ export CASHER_DIR=$HOME/.casher
0.05s$ Installing caching utilities
0.00s
1.16sattempting to download cache archive
fetching ASYNC/cache-linux-precise-0219438f21a1a5708a860383863ceabc701efe9f68761c08be8d841bc6647f17--python-2.7.tgz
found cache
0.00s
1.80sadding /home/travis/.platformio to cache
$ python --version
Python 2.7.9
$ pip --version
pip 6.0.7 from /home/travis/virtualenv/python2.7.9/lib/python2.7/site-packages (python 2.7)
install.1
2.31s$ pip install -U platformio
Collecting platformio
  Downloading platformio-2.9.4-py27-none-any.whl (181kB)
    100% |################################| 184kB 1.7MB/s 
Collecting click<6,>=3.2 (from platformio)
  Downloading click-5.1-py2.py3-none-any.whl (65kB)
    100% |################################| 65kB 3.9MB/s 
Collecting requests<3,>=2.4.0 (from platformio)
  Downloading requests-2.10.0-py2.py3-none-any.whl (506kB)
    100% |################################| 507kB 955kB/s 
Collecting pyserial<4 (from platformio)
  Downloading pyserial-3.1-py2.py3-none-any.whl (93kB)
    100% |################################| 94kB 5.0MB/s 
Collecting colorama (from platformio)
  Downloading colorama-0.3.7-py2.py3-none-any.whl
Collecting bottle<0.13 (from platformio)
  Downloading bottle-0.12.9.tar.gz (69kB)
    100% |################################| 69kB 2.5MB/s 
Collecting lockfile<0.13,>=0.9.1 (from platformio)
  Downloading lockfile-0.12.2-py2.py3-none-any.whl
Installing collected packages: lockfile, bottle, colorama, pyserial, requests, click, platformio
  Running setup.py install for bottle
    changing mode of build/scripts-2.7/bottle.py from 664 to 775
    changing mode of /home/travis/virtualenv/python2.7.9/bin/bottle.py to 775
  Compiling /tmp/pip-build-ZrFK4r/pyserial/serial/aio.py
  
Successfully installed bottle-0.12.9 click-5.1 colorama-0.3.7 lockfile-0.12.2 platformio-2.9.4 pyserial-3.1 requests-2.10.0
install.2
0.36s$ platformio lib install 293
Installing library [ 293 ]:
Already installed
install.3
0.42s$ git clone https://github.com/sticilface/ESPAsyncWebServer /tmp/ESPAsyncWebServer
Cloning into '/tmp/ESPAsyncWebServer'...
remote: Counting objects: 523, done.
remote: Compressing objects: 100% (14/14), done.
remote: Total 523 (delta 6), reused 0 (delta 0), pack-reused 509
Receiving objects: 100% (523/523), 157.16 KiB | 0 bytes/s, done.
Resolving deltas: 100% (330/330), done.
Checking connectivity... done.
install.4
0.00s$ cd /tmp/ESPAsyncWebServer
install.5
0.01s$ git checkout FileFallbackHandler
Branch FileFallbackHandler set up to track remote branch FileFallbackHandler from origin.
Switched to a new branch 'FileFallbackHandler'
install.6
0.00s$ mkdir /tmp/build
0.45s$ platformio ci --build-dir="/tmp/build" --keep-build-dir --lib="." --lib="/tmp/ESPAsyncWebServer" --board=nodemcu
The next files/directories will be created in /tmp/build
platformio.ini - Project Configuration File. |-> PLEASE EDIT ME <-|
src - Put your source files here
lib - Put here project specific (private) libraries
Project has been successfully initialized!
Useful commands:
`platformio run` - process/build project from the current directory
`platformio run --target upload` or `platformio run -t upload` - upload firmware to embedded board
`platformio run --target clean` - clean project (remove compiled files)
`platformio run --help` - additional information
[Thu Jun  9 20:15:56 2016] Processing nodemcu (platform: espressif, board: nodemcu, framework: arduino)
--------------------------------------------------------------------------------
Error: Nothing to build. Please put your source code files to '/tmp/build/src' folder
========================= [ ERROR ] Took 0.26 seconds =========================
The command "platformio ci --build-dir="/tmp/build" --keep-build-dir --lib="." --lib="/tmp/ESPAsyncWebServer" --board=nodemcu" exited with 1.
cache.2
store build cache
0.00s
0.88snothing changed, not updating cache
Done. Your build exited with 1.

here is the travis.ml

language: python
python:
    - "2.7"

# Cache PlatformIO packages using Travis CI container-based infrastructure
sudo: false
cache:
    directories:
        - "~/.platformio"

env:

    - PLATFORMIO_CI_SRC=examples/ESPmanager-example/ESPmanager-example.ino

install:
    - pip install -U platformio
    - platformio lib install 293  
    - git clone https://github.com/sticilface/ESPAsyncWebServer /tmp/ESPAsyncWebServer
    - cd /tmp/ESPAsyncWebServer
    - git checkout FileFallbackHandler
    - mkdir /tmp/build

script:
    
    - platformio ci --build-dir="/tmp/build" --keep-build-dir --lib="." --lib="/tmp/ESPAsyncWebServer" --board=nodemcu

The problem here is that you changed project directory to cd /tmp/ESPAsyncWebServer

Try this

install:
    - pip install -U platformio
    - platformio lib install 293  
    - git clone --recursive -b FileFallbackHandler https://github.com/sticilface/ESPAsyncWebServer /tmp/ESPAsyncWebServe
    - mkdir /tmp/build

that fixed it! thanks as always for the super fast support.

I’ll probably be chasing you again. I’m trying to have complete .bin uploaded back to my repository, with files from the /data folder, and generation of a json manifest with appropriate md5s… so a lot to try and get done!

Take a look at the @squix78 article ESP8266: Continuous Delivery Pipeline – Push To Production

Happy Coding with PlatformIO! :blush:

1 Like

Thats what gave me the idea! few differences though as i need to generate md5, and copy the data folder to githubio as well! not just the bin. bit to do…

any hints…

so including this in my header for the lib

// Stringifying the BUILD_TAG parameter
#define TEXTIFY(A) #A
#define ESCAPEQUOTE(A) TEXTIFY(A)

String buildTag = ESCAPEQUOTE(BUILD_TAG);
.pioenvs/nodemcu/libESPmanager.a -Wl,--end-group
.pioenvs/nodemcu/libESPmanager.a(ESPmanager.o):(.bss.buildTag+0x0): multiple definition of `buildTag'
.pioenvs/nodemcu/src/tmp_ino_to.o:(.bss.buildTag+0x0): first defined here
collect2: error: ld returned 1 exit status
scons: *** [.pioenvs/nodemcu/firmware.elf] Error 1

How about to use BUILD_TAG macros? See my comment ESP8266: Continuous Delivery Pipeline – Push To Production – Squix – TechBlog

OK, so lots of progress.

I’ve got travis to build, and collect all the files for SPIFFS, generate a json manifest file with correct relative links and md5 checksum, and upload it all to my own home server via SSH… so nearly there…

but… there is a bit that I don’t quite understand… and need some help with…

this line PLATFORMIO_BUILD_FLAGS=!echo '-DBUILD_TAG='${TRAVIS_TAG:-"0.0.0"}

takes TRAVIS_TAG and outputs it as a build flag…

now i need this to happen for a few things… TRAVIS_COMMIT and TRAVIS_BRANCH as well as some custom enviromentals…

could u kindly explain the syntax here as i do not follow the !echo… and such…

with the !echo in, the tags get passed as global environmental variables and not passed as flags to the builder.

 - PLATFORMIO_CI_SRC=examples/ESPmanager-example PLATFORMIO_BUILD_FLAGS= !echo "-DBUILD_TIMESTAMP=$UNIX_TIME -DBUILD_TAG='$TRAVIS_TAG' -DCOMMIT_TAG='$TRAVIS_COMMIT'"
$ export PLATFORMIO_CI_SRC=examples/ESPmanager-example
$ export DBUILD_TIMESTAMP=$UNIX_TIME
$ export DBUILD_TAG='$TRAVIS_TAG'
$ export DCOMMIT_TAG='$TRAVIS_COMMIT'

if i take out the echo then it works

$ export PLATFORMIO_BUILD_FLAGS="-DBUILD_TIMESTAMP=$UNIX_TIME -DBUILD_TAG='$TRAVIS_TAG' -DCOMMIT_TAG='$TRAVIS_COMMIT'"

but i’ve also found that in the example Redirecting...

you can specify per build and globals… but it is not obvious that globals will overwrite individual ones…

!echo ... will be replaced with the result of output. It means, that

  • when TRAVIS_TAG does not exist, then PLATFORMIO_BUILD_FLAGS=-DBUILD_TAG=0.0.0
  • otherwise, PLATFORMIO_BUILD_FLAGS=-DBUILD_TAG=x.x.x

How about to use platformio run instead platformio ci? Again, see my comment ESP8266: Continuous Delivery Pipeline – Push To Production – Squix – TechBlog