BIOS introduction

BIOS is short for “Basic Input/Output System”. It is a simple and fast way to automate feedback on GitHub pull requests.

With BIOS, the input is a bios.sh script you add to a repo that will run on every push to GitHub.

The primary output is a commit status – a success or failure – based on the exit status of the script. Another output is a commit comment with the script logs.

This makes certain types of automation – like running linters, static analysis, compilers and simple test suites – easy to add to a GitHub repo.

Take a look at a pull request on the nzoschke/run repo to see it in action. Note how within seconds BIOS points out linting and building failures.

A simple bios.sh script…

#!/bin/bash
set -ex

export GIT_DIR=src/$PKG/.git
run -s "Cloning"  git clone $URL --branch $REF --single-branch src/$PKG
git reset --hard $SHA

PKGS=$(go list $PKG/...)
run -s "Linting"  golint -set_exit_status $PKGS
run -s "Vetting"  go vet -x $PKGS
run -s "Building" go build -v $PKGS
run -s "Testing"  go test -v $PKGS

…gives quality feedback in seconds.

Commit Status Commit Comment

Installation

BIOS is a GitHub App that you authorize to get webhooks on pull requests, read your repo contents, and write commit statuses and comments. The app can not write to your repo.

  1. Browse to the BIOS GitHub app listing and click “Install”.
  2. On the “Where do you want to install BIOS?” screen, select a GitHub user or organization.
  3. On the “Install BIOS” screen, select what repositories you’d like to grant permission to read.
  4. On the “Install BIOS” screen, review the permissions.
    • BIOS requests read access to your code and comments for cloning the repo and “listening” for control comments
    • BIOS requests write access to commit statuses and pull requests
  5. On the “Install BIOS” screen click “Install”.

BIOS is now installed!

At any time you can browse to app listing or the Installed GitHub Apps to update the app configuration or to uninstall it.

bios.sh

BIOS looks for a bios.sh script in your repo. If no script is present no commit status is set.

Exit code

The exit code of bios.sh controls the GitHub commit status. If the script exits 0 it will set a succcess status, and if it exits non-zero it will set a failure status.

stdout / stderr

The stdout and stderr output of bios.sh is captured and posted as a GitHub commit comment.

As a rule of thumb, turn on command traces with set -x to print every command that runs onto stderr. This helps your team understand script failures and software to prettify the commit comment.

STAT and FAIL lines

Output lines that start with STAT: or FAIL: will set a new pending status. This helps you see progress as the script runs.

echo "STAT: Cloning"
git clone $URL || { echo "FAIL: Cloning"; exit 1; }

Note: FAIL: lines are highligted in the commit comment but do not cause a failed commit status. That is controlled by the exit code.

run utility

The execution environment includes a run command. Prefix every important or long-running command with run -s <step> to automatically add STAT: and FAIL: lines along with other human and machine friendly decoration.

$ run -s Cloning git clone $URL

STAT: Cloning
EXEC: "git clone https://github.com/nzoschke/bios.git"
    Cloning into 'bios'...
EXIT: 0
TIME: 0.6s

$ run -s "Cloning again" git clone $URL

STAT: Cloning again
EXEC: "git clone https://github.com/nzoschke/bios.git"
    fatal: destination path 'bios' already exists and is not an empty directory.
EXIT: 128
TIME: 0.1s
FAIL: Cloning again

Check out the run project on GitHub for more info.

bios development tool

More complex automations may take some hacking on a bios.sh script to get right. To quickly iterate, there is a bios tool that will run your bios.sh script locally in a Docker environment.

## Install the bios CLI
$ go get -u github.com/nzoschke/bios

## Check out a repo with `bios.sh`
$ git clone https://github.com/nzoschke/bios.git && cd bios

## Run the local `bios.sh` will run in the BIOS execution environment
$ bios

DIR:  /tmp/bios
USER: <disabled>
PASS: <disabled>
BREF: master
BSHA: c6719771a11546d45869ef3d008d33492870a986
REF:  master
SHA:  c6719771a11546d45869ef3d008d33492870a986
URL:  https://github.com/nzoschke/bios.git

000:0 $ run -s Cloning git clone file:///tmp/repo/.git ...
...

## Results

Succeeded in 1.4 seconds. 🆗

## Statuses

+ Cloning
+ Resetting
+ Fetching
+ Whitespacing
+ Linting
+ Vetting
+ Building
+ Testing

Check out the bios project on GitHub for more info.

Environment

Information from the GitHub pull request event is passed to bios.sh as environment variables. This set of variables make common operations like checking out the repo to the latest push easy.

USER / PASS

USER is a basic auth username, e.g. x-access-token.

PASS is a basic auth password, e.g. v1.1f699f1069f60xxx.

These are tokens that are scoped to BIOS app permissions and expire after one hour.

These values are also written to a .netrc file, which enables a git clone command without additional authentication effort.

Security tip: Use $PASS and env | grep -v PASS to avoid exposing credentials in logs.

OWNER / REPO

OWNER is a repository owner, e.g. nzoschke.

REPO is a repository name, e.g. run.

PKG

PKG is a golang package URL, e.g. github.com/nzoschke/run.

This enables a go get $PKG command without additional URL parsing.

REF / SHA

REF is a pull request branch, e.g. my-feature.

SHA is a SHA of the latest push to the pull request, e.g. 1d8dfce4137d0a8e5a0b6260c2ac860d89a8c524.

This enables setting up a code directory efficiently with git clone $URL --branch $REF --single-branch && git reset --hard $SHA.

BREF / BSHA

BREF is a pull request base branch, e.g. master.

BSHA is a SHA of the base of pull request, e.g. cfce341d2beb70c41e326cf51faee09e3ca393b4.

This enables looking at a diff with git clone diff $BSHA $SHA.

URL

URL is the repo clone URL, e.g. https://github.com/nzoschke/run.git.

GOPATH / HOME / PATH / TMP

BIOS executes in a Lambda function. System environment variables are automatically set to reflect the Lambda sandbox and temporary work directory.

Custom environment

You can set custom environment variables through the BIOS dashboard. This allows you to define additional environment variables when bios.sh runs. For example, you can add a personal GitHub API key with additional permissions, say to automate creating a GitHub release.

Authorization

The BIOS dashboard is a web app that you authorize for single sign on with your GitHub account, and to perform actions within the scope of the BIOS GitHub App. The dashboard can not do anything the app can not do.

  1. Browse to the BIOS Dashboard and click “Sign In”.
  2. On the “BIOS would like to verify your identity on GitHub” screen, review the permissions.
    • BIOS verifies your account for single sign on
    • BIOS requests access to list repositories it is installed on
  3. Click “Authorize”.

You are now authorized to log into the dashboard!

At any time you can browse to Authorized GitHub Apps to revoke the authorization.

Repo configuration

Select one of the repos in the list and hit “Config”. In the config section, set your custom variables as JSON in key/value format:

{
  "FOO": "bar",
  "GITHUB_API_KEY": "..."
}

And click “Update”.

On the next run of bios.sh you can access $FOO and $GITHUB_API_KEY in your script.

Control Comments

bios initialize

On a pull request, add a comment that says bios initialize and BIOS will re-run the script.

Examples

Mix and match the following snippets to write a bios.sh script:

Git clone

run -s "Cloning" git clone $URL --branch $REF --single-branch
git reset --hard $SHA

Git whitespace

run -s "Whitespacing" git diff-tree --check $BSHA $SHA

Build a Go package

run -s "Getting" go get $PKG
cd $GOPATH/src/$PKG
git reset --hard $SHA
run -s "Building" go build .

Block merges on Saturday

#!/bin/bash
set -x
[ $(date +%A) == "Saturday" ] && { echo "FAIL: Working too hard" ; exit 1 }

Try to push on the weekend and we expect a failure status.

FAQ

Pre-installed dependencies

BIOS uses the AWS Lambda Python 2.7 execution environment.

It vendors git, a go toolchain, golint and zip into /var/task/usr.

It would be reasonable to add additional Lambda runtimes (i.e. Python 3, Javascript), and more utilities (jq, etc.) over time.

Custom dependencies

At present you must check custom tools into your GitHub repo or download them manually with curl.

The LambCI Builder Images are a good resource for building binaries for AWS Lambda.