Drone Drone Drone

A brief introduction to Drone CI usage at Monsanto.

Posted by Justin Schlechte on June 20, 2017

CI at Monsanto

Continuous integration is a given in any modern software shop. While there are a plethora of options today, in a traditional Java shop, CI seems to mean Jenkins. Over the past few years, however, there has been a big shift toward the cloud and team autonomy. As part of that shift, teams began choosing different tech stacks and different deployment pipelines to best support their users. When our team made the hop to the cloud, we opted to use Drone CI as our CI system.

What is Drone?

Drone CI is a CI system built on Docker. Each build step is run in a temporary container. Build configurations specify which image to use for each step. And build configurations are stored in the source code. When it all comes together, you have a flexible build system based on lightweight containers that can spin up additional support containers during the build.

Why Choose Drone?

Late 2015, my team began development on its first cloud based application. At the time I had recently heard about Drone CI from a friend at another company and convinced my team to give it a shot. We were hoping that Drone would alleviate some of the issues that we had experienced with Jenkins.

For one, as our tech stack changed, Jenkins agents had to have ALL of the tools that any branch of any repository required. I know technically you can prescribe builds to run on specific agents, but then you end up with A LOT of agents just waiting for orders. Drone, however, runs each build step in a seperate container, so there wouldn’t be just One Agent Image To Rule Them All.

Also, as with any healthy software team, pipelines evolved. Jenkins made it difficult to manage different pipelines for different branches of each of the many repositories we maintained. Drone keeps its build configuration in the repository making branch specific build processes easier to manage.

Finally, we develop a lot of APIs that depend on databases or other resources. In the past, we would run integration tests against the existing dev environment. This was especially troublesome as we had data model transitions. Drone would hopefully allow us to spin up ephemeral resources for testing purposes.

Analysis

Configuration

Configuring Drone was extremely simple. After the host and agents are running, iteraction with Drone itself tends to be fairly small. We see the build status on our PRs in GitHub and build-breakers are notified in Slack. Pipelines are managed in the .drone.yml files. Drone is unopinionated about your pipeline, so we were able to develop a process that fits our environment and our application. We have had hiccups along the way, but we’ve ultimately found ourselves managing our .drone.yml files very little after our pipeline was stabilized. We found the ability to spin up service containers to be extremely valuable since we have integration tests that depend on ephemeral instances of Postgres, Kafka, MongoDB, ElasticSearch, and more.

Example Host Configuration

DRONE_OPEN: true
DRONE_ORGS: MyGitHubOrg
DRONE_GITHUB: true
DRONE_ADMIN: my_github_user,your_github_user
DRONE_GITHUB_URL: https://github.my-company.com
DRONE_GITHUB_CLIENT: ****
DRONE_GITHUB_SECRET: ****
DRONE_GITHUB_PRIVATE_MODE: true
DRONE_SECRET: a_secret_password_for_the_agent_server_communications

Example Agent Configuration

DRONE_SERVER: wss://drone.my-company.com/ws/broker
DRONE_SECRET: a_secret_password_for_the_agent_server_communications

Example .drone.yml

This is a pretty near approximation to how we’ve been structuring our pipeline for node apps.

pipeline:
  check_signature:
    image: alpine
    commands:
      - '[ $DRONE_YAML_VERIFIED = true ] && [ $DRONE_YAML_SIGNED = true ]'

  patch_version:
    image: node
    commands:
      - npm version patch -m '[npm-release][skip ci] %s'
    when:
      events: push
      branch: release

  prerelease_version:
    image: node
    commands:
      - npm version prerelease -m '[npm-release][skip ci] %s'
    when:
      events: push
      branch: develop

  test:
    image: node
    commands:
      - npm install
      - npm test
    when:
      events: [push, pull_request]

  push_tags:
    image: plugins/git-push
    remote_name: origin
    local_branch: refs/tags/*
    branch: refs/tags/*
    when:
      events: push
      branch: release

  push_to_release:
    image: plugins/git-push
    remote_name: origin
    branch: release
    when:
      events: push
      branch: release

  push_to_develop:
    image: plugins/git-push
    remote_name: origin
    branch: develop
    when:
      events: push
      branch:
        - develop
        - release

  # redacted deployment, artifact archival, notifications

Plugins (aka Scripts in Docker Containers)

Creating a new Drone plugin is conceptually straightforward as well. Write a script and get parameters from the environment. Put that script in a docker image and post to your docker registry.

Secrets

We’ve only used versions 0.4 and 0.5 so far. Secret management in version 0.5 is far better that version 0.4, but they revamped the secret system again in version 0.6. In version 0.5, secrets are stored on the Drone host and are exposed only to certain images. As an additional layer of security, your build configuration file has to be signed for the secrets to be injected.

Should You Use Drone?

Yes, unless…

  • You don’t know what docker is. Drone is most potent when placed in the hands of those who know and understand docker.
  • You are unwilling to ask for help. The Drone developers are active on Gitter and have been very helpful.
  • You refuse to use beta software. At this time, the developers are quickly moving toward a 1.0 release, but the software is still in beta.
  • You are already super happy with Jenkins.

Conclusion

Our team has benefitted greatly from Drone, and I encourage you to take a look. Reminder though, Drone is still beta software and things have been changing fairly quickly in the last few months as the development team approaches a version 1.0. As beta software, we sometimes ended up source diving to better understand how Drone would behave in specific situations.

posted on June 20, 2017 by
Justin Schlechte