CI at Monsanto
Continuous integration is a given in any modern software shop. While there are a plethora of options today, in a traditional Java shop, CI seems to mean Jenkins. Over the past few years, however, there has been a big shift toward the cloud and team autonomy. As part of that shift, teams began choosing different tech stacks and different deployment pipelines to best support their users. When our team made the hop to the cloud, we opted to use Drone CI as our CI system.
What is Drone?
Drone CI is a CI system built on Docker. Each build step is run in a temporary container. Build configurations specify which image to use for each step. And build configurations are stored in the source code. When it all comes together, you have a flexible build system based on lightweight containers that can spin up additional support containers during the build.
Why Choose Drone?
Late 2015, my team began development on its first cloud based application. At the time I had recently heard about Drone CI from a friend at another company and convinced my team to give it a shot. We were hoping that Drone would alleviate some of the issues that we had experienced with Jenkins.
For one, as our tech stack changed, Jenkins agents had to have ALL of the tools that any branch of any repository required. I know technically you can prescribe builds to run on specific agents, but then you end up with A LOT of agents just waiting for orders. Drone, however, runs each build step in a seperate container, so there wouldn’t be just One Agent Image To Rule Them All.
Also, as with any healthy software team, pipelines evolved. Jenkins made it difficult to manage different pipelines for different branches of each of the many repositories we maintained. Drone keeps its build configuration in the repository making branch specific build processes easier to manage.
Finally, we develop a lot of APIs that depend on databases or other resources. In the past, we would run integration tests against the existing dev environment. This was especially troublesome as we had data model transitions. Drone would hopefully allow us to spin up ephemeral resources for testing purposes.
Configuring Drone was extremely simple.
After the host and agents are running, iteraction with Drone itself tends to be fairly small.
We see the build status on our PRs in GitHub and build-breakers are notified in Slack.
Pipelines are managed in the
Drone is unopinionated about your pipeline, so we were able to develop a process that fits our environment and our application.
We have had hiccups along the way, but we’ve ultimately found ourselves managing our
.drone.yml files very little after our pipeline was stabilized.
We found the ability to spin up service containers to be extremely valuable since we have integration tests that depend on ephemeral instances of Postgres, Kafka, MongoDB, ElasticSearch, and more.
Example Host Configuration
DRONE_OPEN: true DRONE_ORGS: MyGitHubOrg DRONE_GITHUB: true DRONE_ADMIN: my_github_user,your_github_user DRONE_GITHUB_URL: https://github.my-company.com DRONE_GITHUB_CLIENT: **** DRONE_GITHUB_SECRET: **** DRONE_GITHUB_PRIVATE_MODE: true DRONE_SECRET: a_secret_password_for_the_agent_server_communications
Example Agent Configuration
DRONE_SERVER: wss://drone.my-company.com/ws/broker DRONE_SECRET: a_secret_password_for_the_agent_server_communications
This is a pretty near approximation to how we’ve been structuring our pipeline for node apps.
pipeline: check_signature: image: alpine commands: - '[ $DRONE_YAML_VERIFIED = true ] && [ $DRONE_YAML_SIGNED = true ]' patch_version: image: node commands: - npm version patch -m '[npm-release][skip ci] %s' when: events: push branch: release prerelease_version: image: node commands: - npm version prerelease -m '[npm-release][skip ci] %s' when: events: push branch: develop test: image: node commands: - npm install - npm test when: events: [push, pull_request] push_tags: image: plugins/git-push remote_name: origin local_branch: refs/tags/* branch: refs/tags/* when: events: push branch: release push_to_release: image: plugins/git-push remote_name: origin branch: release when: events: push branch: release push_to_develop: image: plugins/git-push remote_name: origin branch: develop when: events: push branch: - develop - release # redacted deployment, artifact archival, notifications
Plugins (aka Scripts in Docker Containers)
Creating a new Drone plugin is conceptually straightforward as well. Write a script and get parameters from the environment. Put that script in a docker image and post to your docker registry.
We’ve only used versions 0.4 and 0.5 so far. Secret management in version 0.5 is far better that version 0.4, but they revamped the secret system again in version 0.6. In version 0.5, secrets are stored on the Drone host and are exposed only to certain images. As an additional layer of security, your build configuration file has to be signed for the secrets to be injected.
Should You Use Drone?
- You don’t know what docker is. Drone is most potent when placed in the hands of those who know and understand docker.
- You are unwilling to ask for help. The Drone developers are active on Gitter and have been very helpful.
- You refuse to use beta software. At this time, the developers are quickly moving toward a 1.0 release, but the software is still in beta.
- You are already super happy with Jenkins.
Our team has benefitted greatly from Drone, and I encourage you to take a look. Reminder though, Drone is still beta software and things have been changing fairly quickly in the last few months as the development team approaches a version 1.0. As beta software, we sometimes ended up source diving to better understand how Drone would behave in specific situations.