Let's migrate to Omnibus Gitlab
2016-01-25 von Artikel von HannesHannes

  •  Howto
  •  Tech

Over the last year Gitlab has become a central component in our development process. We use it since version 6.x.x and the project has constantly evolved ever since. If you haven't heard about Gitlab think of it as your own Github. The feature set is very close to superpowers Github provides and the community version is free and painlessly self hosted. It is a perfect match for us.

From version 7 upwards the preferred installation method is the omnibus package. Omnibus is a Ruby gem by the guys behind Opscode Chef to create full-stack installers. The installer includes a chef client that performs all installation tasks. This simplifies the Gitlab upgrade and configuration process a lot because the configuration files are separated from the real application. As I said before we started when this option didn't exist and upgrading the source installation always felt harder than it should be.

Now is the time to fix this!

In this article I will walk you through the process of how to upgrade your legacy Gitlab source installation to a shiny Omnibus one.

0. Before you start make sure that...

  • you are currently running Gitlab installed from source.
  • you have a fresh server for the new Omnibus Gitlab.
  • you are currently using MySQL as your Gitlab database OR you use Postgres and skip step 4!
  • you are ready to get your hands dirty.

1. Get yourself a new Server

Since VMs are cheep I decided to start with a fresh Debian 8 box.

NOTE: I thought about dockerizing the new installation, but since we are only a small company Gitlab runs very well on a single box and in my opinion the overhead of dockerizing the individual Gitlab services hasn't much benefit for us right now. Nevertheless there is a great project for that. Perhaps I will change my mind in the future.

2. Find out your current Gitlab version

Before we start we have to check our current Gitlab version. Just run cat /home/git/gitlab/VERSION.

In our case it is version 8.0.2.

3. Create a backup of your non-Omnibus installation

Before we start to make our hands dirty we should create a backup of our current installation.

cd /home/git/gitlab
sudo -u git -H bundle exec rake gitlab:backup:create RAILS_ENV=production

After that you should find a new backup file with the current timestamp in /home/git/gitlab/tmp/backups.

4. Convert your backup from MySQL to Postgres

As I mentioned before we started using Gitlab some time ago. Back then we decided to use a MySQL Database. Omnibus Gitlabs database of choice is Postgres and the backup we just created is not compatible to postgres. Luckyly Gitlab has our back. The conversion process is described here.

NOTE: In addition I had to install apt-get install -y ed before.

Basically we create a postgres compatible database dump using a database converter provided by the Gitlab team. The MySQL database dump in our backup is then replaced with postgres dump. That will allow us to restore the backup to an Omnibus Gitlab installation later on.

5. Install Gitlab from Omnibus package

Next we install Gitlab from the Omnibus package. The general instructions on how to install the package can be found here.

IMPORTANT: Make sure you install the same version you had before! (8.0.2 in our case):

# list the available versions
apt-cache madison gitlab-ce

# Install the same version (in my case 8.0.2-ce.1)
sudo apt-get install gitlab-ce=8.0.2-ce.1
sudo gitlab-ctl reconfigure

After that we should have a clean Omnibus Gitlab running on Port 80.

DISCLAIMER: I told you before to install Omnibus Gitlab to a new server. In fact I installed it on the same server our old Gitlab was running on when I tried it for the first time. This might not be the best idea, but I had no problems with 2 simultaneous installations. Nevertheless I would recommend against it and if you decide to try it anyway at least take a snapshot of the system first!

6. Restore the backup to your Omnibus installation

Now we are on the finishing line. Let's restore the backup we created before to get our data and configuration back in place. To do that we have to copy the backup to the backup folder of the Omnibus installation.

sudo mkdir /var/opt/gitlab/backups/
sudo chown git:git /var/opt/gitlab/backups
sudo cp -p xxxxxx_gitlab_backup.tar /var/opt/gitlab/backups/

To acually restore the backup follow the instructions provided here.

fingers crossed ... BÄHHM!

7. Update Omnibus Gitlab to the latest version

We did all of this to take the pain out of the upgrade process right?! Now let's harvest the fruits of our labor and simply run:

sudo gitlab-rake gitlab:backup:create
sudo apt-get update && sudo apt-get install gitlab-ce

Latest version RUNNIG! At least it should be :) You can find more detailed information about the update process here.

8. Configuration

One last thing! Remember I told you that the nice thing about the Omnibus installation is that the configuration is separate from the application logic. Well, we actually have to create the configuration file /etc/gitlab/gitlab.rbfor that. This is the minimal set of configuration we use.

external_url 'https://gitlab.yourdomain.local'

# FuF configuration #

gitlab_rails['gitlab_email_from'] = 'gitlab@froehlichundfrei.de'
gitlab_rails['gitlab_email_reply_to'] = 'noreply@froehlichundfrei.de'
gitlab_rails['gitlab_email_display_name'] = 'FUF GitLab'
gitlab_rails['smtp_enable'] = true
gitlab_rails['smtp_address'] = "smtp.server.de"
gitlab_rails['smtp_port'] = 587
gitlab_rails['smtp_user_name'] = "username"
gitlab_rails['smtp_password'] = "password"
gitlab_rails['smtp_domain'] = "froehlichundfrei.de"
gitlab_rails['smtp_authentication'] = "plain"
gitlab_rails['smtp_enable_starttls_auto'] = true
gitlab_rails['smtp_openssl_verify_mode'] = 'none'
gitlab_rails['backup_keep_time'] = 1209600
nginx['redirect_http_to_https'] = true

You can find a list of all available configuration options in the omnibus-gitlab repo.

Some other tasks you might consider:

Enable HTTPS
Configure cron to make daily backups

Now we have to rerun sudo gitlab-ctl reconfigure and all configuration files will get updated.

Paths and Commands Overview

This section is more or less a reminder for myself, but perhaps it will save you some time too.

sudo gitlab-ctl help

# runs all configuration tasks
sudo gitlab-ctl reconfigure

# checks gitlab status
sudo gitlab-rake gitlab:check

# create and restore backups
sudo gitlab-rake gitlab:backup:create
sudo gitlab-rake gitlab:backup:restore BACKUP=XXXXXX

Some paths to remember:

# main configuration file

# ssl cert folder

# backup folder

Wrap up

I hope this article will make life easier for one or two people on the Gitlab upgrade trail. Feel free to leave feedback or tell me about nasty typos. I will then open an issue in our totally up-to-date self hosted issue tracker ;D

Happy Coding!