Git Reflector Script

2021-09-29

Here’s a small script to reflect a git repo from some source like GitHub to some destination like an internal gitea mirror. That’s my usecase. You could also use this to go the other direction, or whatever you like.

It’s designed to be run periodically with something like cron. If anything goes wrong during pull/push (for example, upstream rewrites history), that repo sync will fail but others will continue. Modify this to watch for errors if you like or run it manually on occasion to check for errors yourself.

Intermediary copies of repos are stored locally in $PWD/repos/. The repos/ folder is created automatically. Run the script from whatever working directory you want to be the parent of repos/, or change the script to put them somewhere else.

Repo list is in format:

local/repo/name    source_url    dest_url

See script repos= line for examples. Change this to set up your own repo list. You must use spaces as the delimiter, or change the script to use a different delimiter by changing IFS=. For example, to use tabs, set IFS=$'\t'.

Here’s the script:

#!/usr/bin/env sh

# FORMAT:
# local/path   source_url   dest_url
#
# repos are cloned from source_url into folder <local/path>
#   local/path must not contain spaces because i dont want to deal with that.
#   If the folder doesn't exist, its a clone
#   If the folder exists, its an incremental pull
#
# after that, push the repo to dest_url.
#   If there's an error (like source overwrote history with --force) then this
#   doesn't happen. Resolve errors manually.

repos='linux         https://github.com/torvalds/linux.git       git@192.168.69.69:mirrors/linux.git
faithanalog/rtmouse  https://github.com/faithanalog/rtmouse.git  git@192.168.69.69:mirrors/faithanalog-rtmouse.git
xf86-video-amdgpu    https://gitlab.freedesktop.org/xorg/driver/xf86-video-amdgpu.git  git@192.168.69.69:mirrors/xf86-video-amdgpu.git
'

printf '%s' "$repos" | while IFS=' ' read -r path source_url dest_url; do
    # work in repos subdir so we can easily gitignore it
    path="./repos/$path"

    # make parent dir of repo
    mkdir -p "$(dirname "$path")"

    # clone if it doesnt exist already
    if ! [ -d "$path" ]; then
        echo "cloning $source_url into $path"
        git clone --mirror "$source_url" "$path"
    fi

    (
        # enter repo
        cd "$path"
        
        # update all branches from source
        # docs for `git fetch -t` say it fetches tags in addition to a normal
        # fetch. in our testing this was not fully the case, and `git fetch -t`
        # did not fetch new branches.
        echo "pulling updates from $source_url"
        git fetch "$source_url"
        git fetch -t "$source_url"

        # push branches to dest
        echo "pushing updates to $dest_url"
        git push --all "$dest_url"
        git push --tags "$dest_url"
    )
done