Python Release

This action is responsible for making releases of the Python code, both the beta builds based on the develop branch or “real” releases that are based on the master branch.

Triggers

If we push something to the master branch that warrants a new release then this action is triggered

push:
  branches:
  - master
  paths:
  - 'arlunio/**'
  - 'docs/users/**'
  - 'setup.py'
  - 'MANIFEST.in'

We also have a cron trigger that runs every day at 02:00 on the develop branch

schedule:
  - cron: '0 2 * * *'

Jobs

Since code has to be contributed via a PR against the relevant branch, testing across all the supported Python versions and platforms will already have been handled by the Python PR Builds action. Also as this project consists of pure Python code we can produce a single wheel and have it work across all platforms, so it is enough to have our release process run on a single platform, python version combination.

Release:
  runs-on: ubuntu-latest

Steps

Should Release?

  - uses: actions/checkout@v1

  - name: 'Should Release?'
    id: dorel
    run: |
        if [[ "$REF" = 'refs/heads/develop' ]]; then
          ./scripts/should-release.sh
        else
          echo "::set-output name=should_release::true"
        fi
    env:
        REF: ${{github.ref}}

The first thing we do is check whether we should be doing a release in the first place. Here we make use of the set-output workflow command to set the value of a boolean output should_release. The rest of the steps in this workflow check it to see if they should be running, effectively cancelling the build while still having it show as a success on Github.

Question

Is there a better way to cleanly exit a build early?

In the case of a push to master we of course want to trigger a release so this is hardwired to set should_release to true. Otherwise we run a bash script that checks to see if any files of interest have changed since the last release.

#!/bin/bash
# Script to check if we should trigger a beta release or not.

# Find the tag name of the latest release.
tag=$(curl -s "https://api.github.com/repos/swyddfa/arlunio/releases" | jq -r '.[0].tag_name')
echo "Latest Release: ${tag}"

# Determine which files have changed since the last release.
files=$(git diff --name-only ${tag}..HEAD)
echo -e "Files Changed:\n\n$files"

# Do any of them warrant a new release?
changes=$(echo $files | grep -E '^arlunio|setup\.py|docs/users')
echo

if [ -z "$changes" ]; then
    echo "There is nothing to do."
else
    echo "Changes detected, cutting release!"
    echo "::set-output name=should_release::true"
fi

Setup

We then proceed as normal, setting up Python and the build environment.

- name: Setup Python 3.7
  uses: actions/setup-python@v1
  with:
    python-version: 3.7
  if: steps.dorel.outputs.should_release

- name: Setup Environment
  run: |
    python --version
    python -m pip install --upgrade pip
    python -m pip install --upgrade tox
  if: steps.dorel.outputs.should_release

Beta Version Number

So that there is the option of testing/playing with the upcoming release of arlunio as it is being developed, we publish a beta release that includes a beta signifier in the version number. So that we get an unique version number for each build we make use of the einaregilsson/build-number action to generate that for us.

We can then modifiy arlunio’s version number in arlunio/_version.py to include the beta tag.

- name: Get Version Number
  uses: einaregilsson/build-number@v1
  with:
    token: ${{secrets.github_token}}
  if: github.ref == 'refs/heads/develop' && steps.dorel.outputs.should_release

- name: Set Version Number
  shell: bash
  run : |
    sed -i 's/"\(.*\)"/"\1b'"${BUILD_NUMBER}"'"/' arlunio/_version.py
    cat arlunio/_version.py
  if: github.ref == 'refs/heads/develop' && steps.dorel.outputs.should_release

Export Release Info

One of the tasks performed by this workflow is to create a GitHub release so we have a step that exposes the version number and release date to the rest of the workflow. This makes use of the set-env and set-output commands available to an action.

- name: Export release info
  id: info
  run: |
     version=$(sed 's/.*"\(.*\)".*/\1/' arlunio/_version.py)
     release_date=$(date +%Y-%m-%d)

     echo "::set-env name=VERSION::$version"
     echo "::set-output name=VERSION::$version"

     echo "::set-env name=RELEASE_DATE::$release_date"
     echo "::set-output name=RELEASE_DATE::$release_date"
  if: steps.dorel.outputs.should_release

Exposing the results as an environment variable means that the values are available to subsequent script blocks while a step’s output is available to be used as an argument to some YAML field.

Notice how we’re giving this step an explicit id, we’ll use this later when referencing the exposed values.

Build Wheel Package

Time to build the wheel package that we upload to PyPi, the details of which are handled by the pkg tox environment.

- name: Build Package
  run: |
    tox -e pkg
  if: steps.dorel.outputs.should_release

Export Release Assets

In order to make the whl and sdist packages available on the releases page they have to be uploaded by the actions/upload-release-asset action which in turn requires us to know the filepath(s) that we’re going to publish.

- name: Export release assets
  id: pkg
  run: |
    whl=$(find dist/ -name '*.whl' -exec basename {} \;)
    echo "::set-output name=WHL::$whl"

    src=$(find dist/ -name '*.tar.gz' -exec basename {} \;)
    echo "::set-output name=SRC::$src"
  if: steps.dorel.outputs.should_release

Notice how we’re giving this step an explicit id, we’ll use this later when referencing the exposed values

Tag Release

Time to start preparing the release object in GitHub itself by creating a tag for the new version number which we will reference in the next step. At the time of writing the easiest way to do this appears to be just call the GitHub API directly.

- name: Tag Release
  run: |
    commit=$(git rev-parse HEAD)

    # POST a new ref to repo via Github API
    curl -s -X POST https://api.github.com/repos/${{ github.repository }}/git/refs \
    -H "Authorization: token $GITHUB_TOKEN" \
    -d @- << EOF
    {
      "ref": "refs/tags/V$VERSION",
      "sha": "$commit"
    }
    EOF
  env:
    GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
  if: steps.dorel.outputs.should_release

To ensure we get the repository name right we can reference the github context

Create Release

Now that we have something to reference we can go ahead and create a formal release in GitHub. Depending on if the build is taking place on develop or master the release will be tagged as a pre-release by checking the github context

- name: Create Release
  id: release
  uses: actions/create-release@v1.0.0
  env:
    GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
  with:
    tag_name: V${{ steps.info.outputs.VERSION }}
    release_name: V${{ steps.info.outputs.VERSION}} - ${{ steps.info.outputs.RELEASE_DATE }}
    draft: false
    prerelease: ${{ github.ref == 'refs/heads/develop' }}
  if: steps.dorel.outputs.should_release

We also extract the version number and release date from our earlier info step via the steps context. Also notice how we’re giving this step an explicit id, we’ll use this later when we upload the release assets

Upload Release Assets

With the release created we can now upload all the assets we want to publish as part of release. Currently this is just the sdist and whl distributions

- name: Upload Release Asset
  uses: actions/upload-release-asset@v1.0.1
  env:
    GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
  with:
    upload_url: ${{ steps.release.outputs.upload_url }}
    asset_path: dist/${{ steps.pkg.outputs.WHL }}
    asset_name: ${{ steps.pkg.outputs.WHL }}
    asset_content_type: application/octet-stream
  if: steps.dorel.outputs.should_release

- name: Upload Release Asset
  uses: actions/upload-release-asset@v1.0.1
  env:
    GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
  with:
    upload_url: ${{ steps.release.outputs.upload_url }}
    asset_path: dist/${{ steps.pkg.outputs.SRC }}
    asset_name: ${{ steps.pkg.outputs.SRC }}
    asset_content_type: application/octet-stream
  if: steps.dorel.outputs.should_release

Publish Package to PyPi

Finally time to make arlunio pip installable by uploading it to PyPi the details of which are handled by the twine project. The only thing we have to do is provide the twine command with the required credentials stored in GitHub secrets.

- name: Publish to PyPi
  run: |
    python -m pip install twine
    twine upload dist/* -u alcarney -p ${{ secrets.PYPI_PASS }}
  if: steps.dorel.outputs.should_release