Exploring GitHub Enterprise

Using the API to make exploration and discovery possible in GitHub Enterprise

Posted by Patrick Byrne on January 13, 2017

Don’t care about the backstory? Skip ahead

A story of two GitHubs

Earlier this year Monsanto began the process of migrating from github.com to a private GitHub Enterprise instance. Frankly, we love almost everything about it.

GitHub.com

Let’s start out with the things that were amazing about github.com when we first started using it.

Being able to access our source code without a VPN tunnel was a huge boon for all of our developers that don’t happen to work at our main campus. The global nature of github.com is what allowed us to start our migration to other people’s computers (i.e. the cloud). On top of that, github.com has an amazing search feature. It’s fast, it’s accurate, and it just works. The ability to make some of our code open source, something that our dev teams had been trying to get approved for years, finally became a possibility. Many of our teams were happy to abandon whatever they were using for issue tracking in favor of per project issue tracking.

There were some issues though.

In github.com we had to do crazy things like make all our repos private, add all org members to a special team with read only access to things, and then remember to add that team to every repo we created. Lots of time was wasted trying to figure out why someone couldn’t clone a repo, but someone else could. In addition, we started to quickly creep up on our private repo limit. GitHub pages also were usually a no go since, even if the repo was private, the published pages were still open to the world. There was the possibility that something critical could be set to public by accident. When people left the company, they were never actually removed from our org! Most of this was eventually solved with some clever automation, but some of these issues left a bad taste behind.

GitHub Enterprise

Some of us were tasked with investigating some alternatives to github.com. The evaluation process could probably be its own post, but eventually we settled on GitHub Enterprise (GHE).

Instantly some of our bigger issues were resolved.

GHE uses SAML to connect directly to our federated identity source. No more needing to worry about adding new users. We had as many repos as we had hard drive space for. Pages became an awesome way to add documentation to our repos that are both interesting and interactive. Repos could be public by default so that all our devs could browse and search code, and nobody had to worry about accidentally exposing sensitive info. We could group our code into orgs that actually represented a product group, or a team.

But that’s where the next set of problems started. Exploration and discovery in GHE was pretty painful. If you had a good idea what you were looking for, the search would usually get you there, but if you needed to do a bit of exploration, you were probably going to hit a wall.

Exploration and discovery of organizations is painful in GHE.

There is no built in UI to simply list the orgs that are in a GHE instance. By default, org membership is private and has to be changed by each individual user for each org they are part of. Even if you found the org you wanted, you had no way of knowing who to ask about it. Our final hope rested with the GitHub API.

Enter the GitHub Org Explorer

Org Explorer is a React + material-ui based tool with a couple of built in services, all served from a minimal Express instance. The service layer simply queries your GHE API so there isn’t any persistent data store.

Exploring the organizations in your GHE instance

Login Screen

Dummy data provided by the npm-expansion and lorem-ipsum modules

The main screen of the app provides some simple client side sorting and filtering. The org’s avatar and description are displayed, and the org’s login (the part that shows in the URL) is humanized and titlized. Unfortunately, the GitHub API doesn’t provide the org’s display name on the /v3/orgs endpoint. Each org has a link to itself in GHE as well as a link to the Org Explorer’s details page for that org.

Getting down to details

Login Screen

Personal info has been blurred to protect the innocent.

The details page displays some of the more hidden information that can be pulled from the API. To be able to pull back most of this info, you’ll need to be running the app with a Personal Access Token for someone that’s a member of every org. An easy way to do this is the GHE CLI command ghe-org-admin-promote which will add site admins to all orgs as org admins. Honestly this is our biggest complaint with GHE. We would love to see some GHE specific APIs to help out site admins. Even being an org admin, the API still doesn’t let us find who org owners are, for example. Because of this work around, GHE site admins are pulled into their own member’s sections. Hopefully this will eventually be something that can be removed if an improved API is ever provided.

Running it yourself

Specifics are provided in the project’s README, but the short and sweet of it is run npm install, set some environment variables, and run npm start. By default the app will start up on localhost:3099 and you’ll be ready to go. To get the avatar icons to display you’ll need to be logged into your GHE UI. There’s no built in security so you’ll want to either run this inside your firewall, or behind a reverse proxy like nginx.

posted on January 13, 2017 by
Patrick Byrne