Over the past two months or so I have become a contributor to the Debian Project. This is something that I’ve wanted to do for a while. Firstly, just because I’ve got so much out of Debian over the last five or six years—both as a day-to-day operating system and a place to learn about computing—and I wanted to contribute something back. And secondly, in following the work of Joey Hess for the past three or four years I’ve come to share various technical and social values with Debian. Of course, I’ve long valued the project of making it possible for people to run their computers entirely on Free Software, but more recently I’ve come to appreciate how Debian’s mature technical and social infrastructure makes it possible for a large number of people to work together to produce and maintain high quality packages. The end result is that the work of making a powerful software package work well with other packages on a Debian system is carried out by one person or a small team, and then as many users who want to make use of that software need only apt-get it. It’s hard to get the systems and processes to make this possible right, especially without a team being paid full-time to set it all up. Debian has managed it on the backs of volunteers. That’s something I want to be a part of.

So far, most of my efforts have been confined to packaging addons for the Emacs text editor and the Firefox web browser. Debian has common frameworks for packaging these and lots of scripts that make it pretty easy to produce new packages (I did one yesterday in about 30 minutes). It’s valuable to package these addons because there are a great many advantages for a user in obtaining them from their local Debian mirror rather than downloading them from the de facto Emacs addons repository or the Mozilla addons site. Users know that trusted Debian project volunteers have reviewed the software—I cannot yet upload my packages to the Debian archive by myself—and the whole Debian infrastructure for reporting and classifying bugs can be brought to bear. The quality assurance standards built into these processes are higher than your average addon author’s, not that I mean to suggest anything about authors of the particular addons I’ve packaged so far. And automating the installation of such addons is easy as there are all sorts of tools to automate installations of Debian systems and package sets.

I hope that I can expand my work beyond packaging Emacs and Firefox addons in the future. It’s been great, though, to build my general knowledge of the Debian toolchain and the project’s social organisation while working on something that is both relatively simple and valuable to package. Now I said at the beginning of this post that it was following the work of Joey Hess that brought me to Debian development. One thing that worries me about becoming involved in more contentious parts of the Debian project is the dysfunction that he saw in the Debian decision-making process, dysfunction which eventually led to his resignation from the project in 2014. I hope that I can avoid getting quagmired and demotivated.

Posted Thu 28 Jan 2016 19:00:28 UTC

As I understand it, having a GitHub profile as a portfolio has become an essential element in applying for entry-level computer programming jobs—insightfully, a friend of mine draws a comparison with the rise of unpaid internships in other fields. Something about GitHub that gets in the way of maintaining a presentable portfolio is that forks of other people’s repositories made just to submit a pull request can crowd out repositories showcasing one’s work. Sometimes pull requests can take months to be responded to by upstream maintainers, leaving unimpressive repositories sitting around on one’s profile for all that time.

The following Perl script, git-gh-fork, forks a repository and then sets various attributes of it to make it as obvious as GitHub allows that it’s just a temporary fork made in order to submit a pull request. Invoke it like this:

$ cd repo
$ git gh-fork

You will need the following perl libraries: Net::GitHub, Git::Wrapper, Config::GitLike, Term::UI and File::XDG. On a Debian-based system, most of these can be installed with apt-get install libgit-wrapper-perl libconfig-gitlike-perl libterm-ui-perl libnet-github-perl. You’ll need to obtain File::XDG from CPAN.

#!/usr/bin/perl

# git-gh-fork --- Create tidy GitHub forks for pull requests
#
# Copyright (C) 2017  Sean Whitton
#
# git-gh-fork is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# git-gh-fork is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with git-gh-fork.  If not, see <http://www.gnu.org/licenses/>.

use strict;
use warnings;
no warnings "experimental::smartmatch";

use Data::Dumper;

use Net::GitHub;
use Git::Wrapper;
use Config::GitLike;
use Term::UI;
use Term::ReadLine;
use File::XDG;
use File::Temp qw/tempdir/;
use File::chdir;
use Path::Class;
use Sys::Hostname;
use autodie; # die if problem reading or writing a file

my $xdg = File::XDG->new(name => 'net-github');
my $term = Term::ReadLine->new('brand');

my $cache_dir = $xdg->cache_home();
my $oauth_token_file = $cache_dir->file("oauth_token");
my $oauth_token;
if (-f "$oauth_token_file") {
    $oauth_token = $oauth_token_file->slurp();
    chomp $oauth_token;
} else {
    $oauth_token = get_new_oauth_token();
}

my $github = Net::GitHub->new(access_token => $oauth_token);
my $repos = $github->repos;
my $github_user = $github->user;
my $user = $github_user->show()->{'login'};
my $git = Git::Wrapper->new(".");
my $config = Config::GitLike->new( confname => 'config' );
$config->load_file('.git/config');

# TODO check all remotes, not just origin
# TODO resolve the URI, taking account of insteadOf and pushInsteadOf
# in ~/.gitconfig, and then then match against the URIs github accepts
my $origin_url = $config->get(key => "remote.origin.url");
$origin_url =~ m|([a-zA-Z0-9]+)/(.*)(?:\.git)?/?$|;
my $org = $1;
my $repo = $2;
my $prompt = "Do you want to submit a PR from $user against repo $repo belonging to $org?";
my $confirm = $term->ask_yn(prompt => $prompt, default => 'y',);
die "looks like I need a better regexp" unless $confirm;

unless (fork_exists()) {
    $repos->create_fork($org, $repo);
    until (fork_exists()) {
        print "Waiting for fork to be created ...\n";
        sleep 5;
    }
}
$repos->set_default_user_repo($user, $repo);
my $fork = $repos->get();

my @branches = $repos->branches;
unless (grep { $_->{name} eq "github" } @branches) {
    my $worktree = dir(tempdir());
    my $readme = $worktree->file("README.md");
    system "git worktree add --detach $worktree";
    {
        local $CWD = $worktree;
        system "git checkout --orphan github";
        system "git rm -rf .";
        my $fh = $readme->openw();
        $fh->print("This repository is just a fork made in order to submit a pull request");
        close $fh;
        system "git add README.md";
        system "git commit -m 'fork for a pull request'";
    }
    $worktree->rmtree();
    system "git worktree prune";

    system "git remote add -f fork $fork->{html_url}";
    system "git push fork +github";
    system "git branch -D github";
    $repos->update({
                    name => "$repo",
                    has_wiki => 0,
                    homepage => "",
                    description => "Temporary fork for a pull request",
                    has_issues => 0,
                    has_downloads => 0,
                    default_branch => "github",
                   });

    my $branches = "";
    for my $branch (@branches) {
        unless ($branch->{name} eq "github") {
            $branches .= " :$branch->{name}";
        }
    }
    system "git push fork $branches";
}

sub get_new_oauth_token {
    $cache_dir->mkpath();
    my $user = $term->get_reply(prompt => 'GitHub username');
    my $pass = $term->get_reply(prompt => 'GitHub password');
    my $github = Net::GitHub::V3->new( login => "$user", pass => "$pass" );
    my $oauth = $github->oauth;
    # TODO this will fail if a token has already been created on this
    # host -- we should be able to re-use it
    my $o = $oauth->create_authorization({scopes => ['repo'], note => 'git gh-fork@' . hostname});
    my $oauth_token_file_handle = $oauth_token_file->openw();
    $oauth_token_file_handle->print($o->{token} . "\n");
    return $o->{token}
}

sub fork_exists {
    my @user_repos = $repos->list_user($user);
    return grep { $_->{name} eq $repo } @user_repos;
}

If you have any suggestions for git gh-fork, please send me a patch or a pull request against the version in my dotfiles repository.

Update 2017/ii/14: Applied patches from Tom Hoover. Thanks.

Update 2017/ii/18: Rewritten in Perl, with various improvements. Python version still available.

apt-get install python-github

#!/usr/bin/env python

# clean-github-pr --- Create tidy repositories for pull requests
#
# Copyright (C) 2016  Sean Whitton
#
# clean-github-pr is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# clean-github-pr is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with clean-github-pr.  If not, see <http://www.gnu.org/licenses/>.

import github

import sys
import time
import tempfile
import shutil
import subprocess
import os

CREDS_FILE = os.getenv("HOME") + "/.cache/clean-github-pr-creds"

def main():
    # check arguments
    if len(sys.argv) != 2:
        print sys.argv[0] + ": usage: " + sys.argv[0] + " USER/REPO"
        sys.exit(1)

    # check creds file
    try:
        f = open(CREDS_FILE, 'r')
    except IOError:
        print sys.argv[0] + ": please put your github username and password, separated by a colon, in the file ~/.cache/clean-github-pr-creds"
        sys.exit(1)

    # just to be sure
    os.chmod(CREDS_FILE, 0600)

    # make the fork
    creds = f.readline()
    username = creds.split(":")[0]
    pword = creds.split(":")[1].strip()
    token = f.readline().strip()

    if len(token) != 0:
        g = github.Github(token)
    else:
        g = github.Github(username, pword)

    u = g.get_user()

    source = sys.argv[1]
    if '/' in source:
        fork = sys.argv[1].split("/")[1]
        print "forking repo " + source
        u.create_fork(g.get_repo(source))
    else:
        fork = sys.argv[1]

    while True:
        try:
            r = u.get_repo(fork)
        except github.UnknownObjectException:
            print "still waiting"
            time.sleep(5)
        else:
            break

    # set up & push github branch
    user_work_dir = os.getcwd()
    work_area = tempfile.mkdtemp()
    os.chdir(work_area)
    subprocess.call(["git", "clone", "https://github.com/" + username + "/" + fork])
    os.chdir(work_area + "/" + fork)
    subprocess.call(["git", "checkout", "--orphan", "github"])
    subprocess.call(["git", "rm", "-rf", "."])
    with open("README.md", 'w') as f:
        f.write("This repository is just a fork made in order to submit a pull request; please ignore.")
    subprocess.call(["git", "add", "README.md"])
    subprocess.call(["git", "commit", "-m", "fork for a pull request; please ignore"])
    subprocess.call(["git", "push", "origin", "+github"])
    os.chdir(user_work_dir)
    shutil.rmtree(work_area)

    # make sure the branch has been pushed
    time.sleep(5)

    # set clean repository settings
    r.edit(fork,
           has_wiki=False,
           description="Fork for a pull request; please ignore",
           homepage="",
           has_issues=False,
           has_downloads=False,
           default_branch="github")

if __name__ == "__main__":
    main()
Posted Sun 31 Jan 2016 23:15:18 UTC Tags: