FRANS CAISAR RAMADHAN

My personal blog

How to publish Homebrew formula in private Git repository

2020.09.26

This article is still related with How to publish your Go binary as Homebrew Formula with GoReleaser Full source code of this example can be seen in https://github.com/franzramadhan/homebrew-go-example

Introduction

This article will cover how to publish Homebrew formula to Private Github repository easily using GoReleaser.

Background

Sometimes, we need to publish our artifact or Homebrew formula privately. So it is only accessible within our organization or our teammates. Therefore, we need to create one logic to Homebrew so it is able to handle this scenario.

Solution

To do that we need to create a script as a download strategy, then tells the Homebrew formula to use the strategy instead of using normal one.

Luckily, GoReleaser support this adjustment. We only need to set it in the .goreleaser.yml.

Workflow

First we need to implement the download strategy. For this example, we will put it inside lib/ directory with private.rb name.

require "download_strategy"

# GitHubPrivateRepositoryDownloadStrategy downloads contents from GitHub
# Private Repository. To use it, add
# `:using => :github_private_repo` to the URL section of
# your formula. This download strategy uses GitHub access tokens (in the
# environment variables `HOMEBREW_GITHUB_API_TOKEN`) to sign the request.  This
# strategy is suitable for corporate use just like S3DownloadStrategy, because
# it lets you use a private GitHub repository for internal distribution.  It
# works with public one, but in that case simply use CurlDownloadStrategy.
class GitHubPrivateRepositoryDownloadStrategy < CurlDownloadStrategy
  require "utils/formatter"
  require "utils/github"

  def initialize(url, name, version, **meta)
    super
    parse_url_pattern
    set_github_token
  end

  def parse_url_pattern
    unless match = url.match(%r{https://github.com/([^/]+)/([^/]+)/(\S+)})
      raise CurlDownloadStrategyError, "Invalid url pattern for GitHub Repository."
    end

    _, @owner, @repo, @filepath = *match
  end

  def download_url
    "https://#{@github_token}@github.com/#{@owner}/#{@repo}/#{@filepath}"
  end

  private

  def _fetch(url:, resolved_url:)
    curl_download download_url, to: temporary_path
  end

  def set_github_token
    @github_token = ENV["HOMEBREW_GITHUB_API_TOKEN"]
    unless @github_token
      raise CurlDownloadStrategyError, "Environment variable HOMEBREW_GITHUB_API_TOKEN is required."
    end

    validate_github_repository_access!
  end

  def validate_github_repository_access!
    # Test access to the repository
    GitHub.repository(@owner, @repo)
  rescue GitHub::HTTPNotFoundError
    # We only handle HTTPNotFoundError here,
    # becase AuthenticationFailedError is handled within util/github.
    message = <<~EOS
      HOMEBREW_GITHUB_API_TOKEN can not access the repository: #{@owner}/#{@repo}
      This token may not have permission to access the repository or the url of formula may be incorrect.
    EOS
    raise CurlDownloadStrategyError, message
  end
end

# GitHubPrivateRepositoryReleaseDownloadStrategy downloads tarballs from GitHub
# Release assets. To use it, add `:using => :github_private_release` to the URL section
# of your formula. This download strategy uses GitHub access tokens (in the
# environment variables HOMEBREW_GITHUB_API_TOKEN) to sign the request.
class GitHubPrivateRepositoryReleaseDownloadStrategy < GitHubPrivateRepositoryDownloadStrategy
  def initialize(url, name, version, **meta)
    super
  end

  def parse_url_pattern
    url_pattern = %r{https://github.com/([^/]+)/([^/]+)/releases/download/([^/]+)/(\S+)}
    unless @url =~ url_pattern
      raise CurlDownloadStrategyError, "Invalid url pattern for GitHub Release."
    end

    _, @owner, @repo, @tag, @filename = *@url.match(url_pattern)
  end

  def download_url
    "https://#{@github_token}@api.github.com/repos/#{@owner}/#{@repo}/releases/assets/#{asset_id}"
  end

  private

  def _fetch(url:, resolved_url:)
    # HTTP request header `Accept: application/octet-stream` is required.
    # Without this, the GitHub API will respond with metadata, not binary.
    curl_download download_url, "--header", "Accept: application/octet-stream", to: temporary_path
  end

  def asset_id
    @asset_id ||= resolve_asset_id
  end

  def resolve_asset_id
    release_metadata = fetch_release_metadata
    assets = release_metadata["assets"].select { |a| a["name"] == @filename }
    raise CurlDownloadStrategyError, "Asset file not found." if assets.empty?

    assets.first["id"]
  end

  def fetch_release_metadata
    release_url = "https://api.github.com/repos/#{@owner}/#{@repo}/releases/tags/#{@tag}"
    GitHub.open_api(release_url)
  end
end

class DownloadStrategyDetector
  class << self
    module Compat

      def detect_from_symbol(symbol)
        case symbol
        when :github_private_repo
          GitHubPrivateRepositoryDownloadStrategy
        when :github_private_release
          GitHubPrivateRepositoryReleaseDownloadStrategy
        else
          super(symbol)
        end
      end
    end

    prepend Compat
  end
end

Then we need to also add additional configuration in .goreleaser.yml so it will update the previous formula to use the download strategy we created previously. We need to add following lines in brew section

    download_strategy: GitHubPrivateRepositoryReleaseDownloadStrategy
    custom_require: "lib/private"

Complete section should look like this:

# Check https://goreleaser.com/customization/homebrew/
brews:
  - homepage: 'https://github.com/franzramadhan/homebrew-go-example'
    description: 'Example binary distribution using homebrew.'
    folder: Formula
    download_strategy: GitHubPrivateRepositoryReleaseDownloadStrategy
    custom_require: "lib/private"
    commit_author:
      name: franzramadhan
      email: franzramadhan@gmail.com
    tap:
      owner: franzramadhan
      name: homebrew-go-example

Once the new release is triggered. The homebrew formula should be updated to use the download strategy. See homebrew-go-example.rb:

# This file was generated by GoReleaser. DO NOT EDIT.
require_relative "lib/private"
class HomebrewGoExample < Formula
  desc "Example binary distribution using homebrew."
  homepage "https://github.com/franzramadhan/homebrew-go-example"
  version "0.0.3"
  bottle :unneeded

  if OS.mac?
    url "https://github.com/franzramadhan/homebrew-go-example/releases/download/v0.0.3/homebrew-go-example_0.0.3_darwin_x86_64.tar.gz", :using => GitHubPrivateRepositoryReleaseDownloadStrategy
    sha256 "dbf982698730b6f8f2e34a4b2e6e9fe4033bb7d9b388a15a6d2d94f6261261e3"
  elsif OS.linux?
    if Hardware::CPU.intel?
      url "https://github.com/franzramadhan/homebrew-go-example/releases/download/v0.0.3/homebrew-go-example_0.0.3_linux_x86_64.tar.gz", :using => GitHubPrivateRepositoryReleaseDownloadStrategy
      sha256 "3f4b0784acfb83217f0335135f91975210c092b055108f9b2a08d1104932c070"
    end
  end

  def install
    bin.install "homebrew-go-example"
  end
end

See require_relative "lib/private" and GitHubPrivateRepositoryReleaseDownloadStrategy line. It tells the Homebrew client to use the download strategy method defined in lib/private.rb

Testing

Try to install the formula

brew install homebrew-go-example

Updating Homebrew...
==> Installing homebrew-go-example from franzramadhan/go-example
Error: Failed to download resource "homebrew-go-example"
Download failed: Environment variable HOMEBREW_GITHUB_API_TOKEN is required.

Since the repository is private, we need to set the Github authentication. To authenticate, we need to generate Github Personal Access Token with repo scope and export it as HOMEBREW_GITHUB_API_TOKEN environment variable. e.g

export HOMEBREW_GITHUB_API_TOKEN=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

Try install again

brew install homebrew-go-example

Updating Homebrew...
==> Installing homebrew-go-example from franzramadhan/go-example
==> Downloading https://github.com/franzramadhan/homebrew-go-example/releases/download/v0.0.3/homebrew-go-example_0.0.3_darwin_x86_64.tar.gz
Already downloaded: /Users/franscaisarramadhan/Library/Caches/Homebrew/downloads/9189775534d2c9a1a759ec2fc48ea9bf349c2e00efa501ba8cd7c464b5d48882--homebrew-go-example_0.0.3_darwin_x86_64.tar.gz
🍺  /usr/local/Cellar/homebrew-go-example/0.0.3: 5 files, 2MB, built in 3 seconds

Test the installed binary

homebrew-go-example
2020/09/27 01:03:26 [DEBUG] GET http://numbersapi.com/9/27/date?json
{
    "found": true,
    "number": 271,
    "text": "September 27th is the day in 1331 that the Battle of Płowce between the Kingdom of Poland and the Teutonic Order is fought.",
    "type": "date",
    "year": 1331
}

Conclusion

Homebrew formula is easily extensible to accomodate generic use case
GoReleaser is powerful tool that can also help internal usage of Homebrew formula.