The most important tool to have as a developer (Unity edition)

alt text

The CI/CD pipeline is one of the best practices any developer team could implement, for delivering code changes more frequently and reliably.

Before we start diving into the various details, let's get familiar with the services we'll be using.



Gitlab

Every developer having worked in a team should be familiar with source control like Github or SVN. Source control offers powerful collaboration tools like issue tracking, branching, history and even the ability to run scripts at the various stages of the merging process. And it was just that, the ability to run scripts that made us switch from Github to Gitlab. In addition to offer the same features as Github, Gitlab offers a rich management toolkit, taking you through the steps from creating an issue, to code ready for production. The most important capability is running a pipeline, which is where we'll do all of our automation. Pipelines work by stepping through a yaml file which defines various stages of scripts to run inside a docker container. Gitlab even features docker registries, which are quite handy.


Itch.io

Itch.io is the platform we'll be publishing our game to.


Docker

Docker is a powerful tool for running applications inside an isolated linux container. Unity works with linux now, so that shouldn't be problem, unless windows only packages are used. The first thing we need to do is to create a repository for our dockerfile(s).


Dockerfile for Unity

This is the Dockerfile we use, it starts by getting a ubuntu image, then proceeds to install the needed dependencies for Unity and finally downloads and install Unity itself.

FROM ubuntu:16.04

RUN apt-get update -qq; \
  apt-get install -qq -y \
  gconf-service \
  lib32gcc1 \
  lib32stdc++6 \
  libasound2 \
  libc6 \
  libc6-i386 \
  libcairo2 \
  libcap2 \
  libcups2 \
  libdbus-1-3 \
  libexpat1 \
  libfontconfig1 \
  libfreetype6 \
  libgcc1 \
  libgconf-2-4 \
  libgdk-pixbuf2.0-0 \
  libgl1-mesa-glx \
  libglib2.0-0 \
  libglu1-mesa \
  libgtk2.0-0 \
  libnspr4 \
  libnss3 \
  libpango1.0-0 \
  libstdc++6 \
  libx11-6 \
  libxcomposite1 \
  libxcursor1 \
  libxdamage1 \
  libxext6 \
  libxfixes3 \
  libxi6 \
  libxrandr2 \
  libxrender1 \
  libxtst6 \
  libsoup2.4-1 \
  libarchive13 \
  zlib1g \
  debconf \
  npm \
  xdg-utils \
  lsb-release \
  libpq5 \
  xvfb \
  wget \
  && apt-get clean \
  && rm -rf /var/lib/apt/lists/*

RUN wget -nv https://beta.unity3d.com/download/77a142ac9989/UnitySetup-2018.1.0b13; \
    chmod +x UnitySetup-2018.1.0b13; \
    echo y | ./UnitySetup-2018.1.0b13 --unattended --install-location=/opt/Unity --components=Unity,Windows-Mono

WORKDIR /root

This Dockerfile can either be built manually, or ran using gitlab's CI tool:

variables:
  DOCKER_DRIVER: overlay2
  VERSION: 2018.1.0b13
  CONTAINER_IMAGE: ${CI_PROJECT_PATH}:${VERSION}

stages:
  - release

docker:
  image: docker:latest
  services:
    - docker:dind
  stage: release
  script:
    - docker login -u ${CI_REGISTRY_USER} -p ${CI_BUILD_TOKEN} ${CI_REGISTRY}
    - docker build --network host -t ${CONTAINER_IMAGE} ./${VERSION}
    - docker tag ${CONTAINER_IMAGE} ${CI_REGISTRY}/${CONTAINER_IMAGE}
    - docker push ${CI_REGISTRY}/${CONTAINER_IMAGE}
  only:
    - master

This .gitlab-ci.yml contains a single stage, which runs docker-in-docker to build and push the docker image to the gitlab registry. The final path should look something like this registry.gitlab.com/company/repository:2018.1.0b13.


Activation

The most annoying problem with this setup is that Unity needs to be activated. The only way to activate Unity from cli is passing a pro license key, but it's possible to get around that limitation with some tinkering. The first thing to do is create an activation stage in the .gitlab-ci.yml file. When running this stage, you will get a licensing request in xml printed in the output. Save that as "license.alf" and go through the manual activation on http://license.unity3d.com/manual. The license you get needs to be passed as an environment variable in the pipeline.

This is how our activation stage looks

variables:
  UNITY_USERNAME: person@company.com
  UNITY_PASSWORD: XXXXXXXXXXX

stages:
  - license

activation:
  image: registry.gitlab.com/company/repository:2018.1.0b13
  stage: license
  script:
    - >-
      xvfb-run --server-args="-screen 0 1024x768x24"
      /opt/Unity/Editor/Unity
      -nographics
      -batchmode
      -logFile
      -quit
      -username ${UNITY_USERNAME}
      -password ${UNITY_PASSWORD}
  when: manual


Building

Now, the most important stage; building our game. This stage might take some time depending on the size of your game. After the stage is complete, it will create an artifact containing the finished build. Artifacts will be persisted between stages, which is important, as we'll be using it in the release stage.

This is how we build for linux:

build linux:
  image: registry.gitlab.com/company/repository:2018.1.0b13
  stage: build
  before_script:
    - mkdir -p /root/.local/share/unity3d/Unity/
    - echo "${UNITY_LICENSE}" > /root/.local/share/unity3d/Unity/Unity_lic.ulf
  script:
    - >-
      /opt/Unity/Editor/Unity
      -buildLinux64Player ./build/linux64/Pareidolia
      -buildTarget Linux64
      -projectPath "$(pwd)"
      -nographics
      -batchmode
      -logFile
      -quit
  artifacts:
    paths:
      - build/
    expire_in: 1 day

This is how we build for windows:

build win:
  image: registry.gitlab.com/company/repository:2018.1.0b13
  stage: build
  before_script:
    - mkdir -p /root/.local/share/unity3d/Unity/
    - echo "${UNITY_LICENSE}" > /root/.local/share/unity3d/Unity/Unity_lic.ulf
  script:
    - >-
      /opt/Unity/Editor/Unity
      -buildWindows64Player ./build/win64/Pareidolia.exe
      -buildTarget Win64
      -projectPath "$(pwd)"
      -nographics
      -batchmode
      -logFile
      -quit
  artifacts:
    paths:
      - build/
    expire_in: 1 day
Releasing

To keep versioning consistent we added a stage to update version number after every build stage. This allows us to keep track of which build everyone is running.

version:
  image: alpine:latest
  stage: version
  before_script:
    - apk --no-cache add curl git openssh-client
    - eval $(ssh-agent -s)
    - echo "${GITLAB_PRIVATE_KEY}" | tr -d '\r' | ssh-add - > /dev/null
    - mkdir -p ~/.ssh
    - chmod 700 ~/.ssh
    - ssh-keyscan gitlab.com >> ~/.ssh/known_hosts
    - chmod 644 ~/.ssh/known_hosts
    - git config --global user.email "runner@gitlab.com"
    - git config --global user.name "Gitlab Runner"
  script:
    - git clone git@gitlab.com:company/game.git
    - cd pareidolia
    - VERSION=$(cat "version")
    - VERSION=$((VERSION+1))
    - echo $VERSION > version
    - git add version
    - git commit -m "Bumped version number [skip ci]"
    - git push origin master
  only:
    - master

For the final stage we'll be using butler to push our game to itch.io. Remember to provide the itch.io API key in the BUTLER_API_KEY environment variable.

release stable win:
  image: registry.gitlab.com/lucidum/butler:alpine
  stage: release
  before_script:
    - VERSION=$(cat "version")
    - VERSION=$((VERSION+1))
    - echo $VERSION > version
  script:
    - butler push ./build/win64 lucidumstudio/pareidolia:windows --userversion-file version
  only:
    - master
  when: manual
Result

There's alot of setup, but after all of the configuration we're able to rapidly deliver bugfixes and features to the players.