---
title: Problems and solutions with Gitea 1.15
tags: [tutorial, gitea, mariadb, postgresql]
updated: 2021-09-13 14:48:00
description: This blog is now self-hosted
---

My Gitea instance at [software.franco.net.eu.org](https://software.franco.net.eu.org)
has been on SQLite since its inception in October 2018, up until some weeks ago.
I never had any problems concerning database updates. Then I decided to migrate
to something more scalable: [MariaDB](https://mariadb.org/).

<!--TOC-->

- [Migration of Gitea < 1.15.x from SQLite to MariaDB](#migration-of-gitea--115x-from-sqlite-to-mariadb)
- [Update to 1.15.2](#update-to-1152)
  - [Revert](#revert)
  - [PostgreSQL](#postgresql)
    - [Steps](#steps)
      - [1. setup the database](#1-setup-the-database)
      - [2. recover the repositories](#2-recover-the-repositories)
      - [3. Adminer](#3-adminer)
        - [Apache configuration](#apache-configuration)
      - [4. set the packages on hold](#4-set-the-packages-on-hold)
      - [5. restore issues and comments](#5-restore-issues-and-comments)
      - [6. restore mirrors from bare repositories](#6-restore-mirrors-from-bare-repositories)
      - [7. restore users](#7-restore-users)
      - [8. Database backups](#8-database-backups)
- [Extras](#extras)
  - [Torification](#torification)
  - [HTTP2](#http2)
  - [Caching](#caching)
    - [Solution](#solution)
      - [Redis](#redis)
      - [Gitea configuration](#gitea-configuration)
  - [Updating gitea next time](#updating-gitea-next-time)
  - [Apache2 reverse proxy for Gitea](#apache2-reverse-proxy-for-gitea)
  - [GitHub issue](#github-issue)

<!--TOC-->

## Migration of Gitea < 1.15.x from SQLite to MariaDB

Migration was made possible using the provided dump tool.
Here is what I did (these instructions have been written
after some trials and errors):

1. create a MySQL user

   ```shell
   mysql -u root
   ```

   ```sql
   CREATE USER 'gitea' IDENTIFIED BY '${DB_PASSWORD}';
   ```

2. create a MySQL database

   ```sql
   CREATE DATABASE giteadb CHARACTER SET 'utf8mb4' COLLATE 'utf8mb4_unicode_ci';
   GRANT ALL PRIVILEGES ON giteadb.* TO 'gitea';
   FLUSH PRIVILEGES;
   exit
   ```

3. Go into the site administration and delete all authentication sources.
   [This issue](https://github.com/go-gitea/gitea/issues/16831)
   also happened to me.
4. Stop Gitea

   ```shell
   systemctl stop gitea
   ```

5. disable all cron jobs at gitea boot. Gitea didn't start while cron jobs
   were enabled at boot. Edit `/etc/gitea/app.ini`:

   ```ini
   [cron]
   ; Enable running cron tasks periodically.
   ENABLED      = true
   ; Run cron tasks when Gitea starts.
   RUN_AT_START = false
   ```

6. start and stop Gitea to have a clean restart without cron jobs running:

   ```shell
   systemctl start gitea
   sleep 60
   systemctl stop gitea
   ```

7. create a database dump translated from sqlite to mysql. This command creates
   a zip file called `gitea-dump-${id}.zip`:

   ```shell
   sudo -i -u gitea
   HOME=/var/lib/gitea GITEA_WORK_DIR=/var/lib/gitea /usr/bin/gitea dump --skip-repository --skip-log --skip-custom-dir --skip-lfs-data --skip-attachment-data --database mysql -c /etc/gitea/app.ini
   ```

8. change the database type in `/etc/gitea/app.ini`. Comment out the old section and
   add a new mysql section like this:

   ```ini
   [database]
   ; Either "mysql", "postgres", "mssql" or "sqlite3", it's your choice
   DB_TYPE             = mysql
   HOST                = 127.0.0.1:3306
   NAME                = giteadb
   USER                = gitea
   ; Use PASSWD = `your password` for quoting if you use special characters in the password.
   PASSWD              = `${DB_PASSWORD}`
   ```

9. restore the database:

   ```shell
   unzip gitea-dump-${id}.zip
   mysql --default-character-set=utf8mb4 -ugitea -p${DB_PASSWORD} giteadb <gitea-db.sql
   ```

10. `systemctl start gitea`
11. re-add the authentication sources

This worked although some warnings remained when I lauched Gitea's doctor command.

## Update to 1.15.2

After updating the packages, Gitea did not start.
There were a bunch of errors (for example
[this](https://github.com/go-gitea/gitea/issues/16961))! I was able to recover the database
by hand by having a look at the errors from the log.

After some time, however, I noticed that the mirror feature did not work
and the repositories on the web UI did not update after pushing from git.

[Jump to the solution](#solution)

### Revert

I tried reverting back to a previous database version but once you migrate
you cannot go back easily.

### PostgreSQL

I tried with a fresh PostgreSQL 13 database and Gitea 1.15.2.
The repostitory and mirror problem was still there so at least I knew it
wasn't a problem with the database.

What I did in the end was to re-install Gitea 1.14.7 and to switch to
another new PostgreSQL database instead. The problem now was that
I needed to recover most "issues", mirrors, users, etc... from the original
MariaDB database.
Once that was done after hard work I made a backup and proceeded
to the migration of this new database to Gitea 1.15.0.

#### Steps

These steps work on Debian GNU/Linux 10 which is now *oldstable*.

##### 1. setup the database

1. install PostgreSQL 13 from the [official website](https://www.postgresql.org/download/linux/debian/)
2. Follow the [database preparations instructions](https://docs.gitea.io/en-us/database-prep/#postgresql)
3. use the UNIX socket instead of TCP. This configuration might improve performance a little bit. Set this in `/etc/gitea/app.ini`

   ```ini
   [database]
   DB_TYPE             = postgres
   HOST                = /var/run/postgresql
   LOG_SQL             = false
   ```

##### 2. recover the repositories

Create the users and push or "adopt" the repositories through the `admin/repos` URL. You will see
the `Unadopted Repository` button there.

##### 3. Adminer

[Adminer](https://www.adminer.org/) is similar to phpMyAdmin. You can install it with:

```shell
apt-get install adminer/buster-backports
```

The backports version seem to be fully compatible with PostgreSQL 13.

###### Apache configuration

1. if you use [PHP-FPM](https://www.php.net/manual/en/install.fpm.php)
   you can use this snippet in a `VirtualHost` directive:

   ```apache
   Alias /dbadmin /usr/share/adminer/adminer/
   <Directory "/usr/share/adminer/adminer/">
       Require ip 127.0.0.1
       Require ip 192.168.0.
       Options FollowSymlinks
   </Directory>
   Include conf-enabled/php7.3-fpm.conf
   ```

2. add this to the php.ini file of Apache, in `/etc/php/7.3/apache2/php.ini`:

   ```ini
   [HOST=my.host]
   open_basedir = /tmp/:/usr/share/adminer:/usr/share/php:/var/log/adminer
   ```

3. finally:

   ```shell
   systemctl restart apache2
   ```

4. connect to `http://my.host/dbadmin`

##### 4. set the packages on hold

Set these packages on hold so when you update the system they don't get
updated by accident.

```shell
apt-mark hold gitea linux-image-amd64 postgresql-13 postgresql-client-13 postgresql-client-common postgresql-common
```

This is what you should get when you run `apt-mark showhold`:

```shell
gitea
linux-image-amd64
postgresql-13
postgresql-client-13
postgresql-client-common
postgresql-common
```

##### 5. restore issues and comments

1. open the original database with adminer
1. get the repository id. Use the `repository` table to get it. Let's say it's `38`
2. go to the `issue` table
3. click on the search filter on the top. Set `repo_id` `=` `38`. Then click `Select`
4. on the web-ui with the new database create new issues within the repository. In our example we have 9
   issues so we need to create 9 dummy issues. Comment content and the user creating them is irrelevant
5. now take note of the issues ids. In our case they are [`1`-`9`]
6. open the comment table and do a new query, filtering by issue id. Let's start with `1`
   In this case we only have 1 comment so we must create a dummy comment in the issue on web UI
7. for issue `2` we have 2 comments so we need to create 2 dummy comments
8. do this for issues [`1`-`9`]
9. once we have all the dummy data we can open the new database on adminer.
   What we have to do now is just to manually copy-paste all the data from
   one database to the other. Check the UI once in a while if everything is working

{% include image.html file="adminer_gitea_issue_table.png" alt="Gitea issue table on Adminer" caption="Gitea issue table on Adminer" %}

{% include image.html file="adminer_gitea_comment_table.png" alt="Gitea comment table on Adminer" caption="Gitea comment table on Adminer" %}

{% include image.html file="adminer_gitea_comment_table_2.png" alt="Gitea comment table for issue 2 on Adminer" caption="Gitea comment table for issue 2 on Adminer" %}

##### 6. restore mirrors from bare repositories

1. once you have the repositories go into the `repositories` table
2. select the to-be mirrors repositories and set the `is_mirror` variable to `true`
2. go into the `mirror` table and create a new element. Use the correct `repo_id`.

{% include image.html file="adminer_gitea_repository_table_mirror.png" alt="Mirror variable in a repository" caption="Mirror variable in a repository" %}

{% include image.html file="adminer_gitea_mirror_table.png" alt="An example row of the mirror table" caption="An example row of the mirror table" %}

##### 7. restore users

To restore users simply create new users using Gitea's admin interface and then copy-paste the data
in the rows.

##### 8. Database backups

I now use 15 minute separated backups using borgmatic. See also
[https://docs.franco.net.eu.org/automated-tasks/scripts.html#borgmatic-hooks-py](https://docs.franco.net.eu.org/automated-tasks/scripts.html#borgmatic-hooks-py)

This is just an example for reference:

```yaml
#
# borgmatic.iron_postgresql_giteadb.yaml
#
# Copyright (C) 2014-2020 Dan Helfman <https://torsion.org/borgmatic/docs/reference/config.yaml>
#               2021 Franco Masotti
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program.  If not, see <https://www.gnu.org/licenses/>.

location:
    source_directories: []
    repositories:
        - user@remotepc:backups/postgresql_giteadb.borg
storage:
    checkpoint_interval: 900
    lock_wait: 120
retention:
    keep_within: 1w
    keep_monthly: 1
consistency:
    checks:
        - archives
output:
    color: false
hooks:
    after_everything:
        - /home/jobs/scripts/by-user/root/borgmatic_hooks.py /home/jobs/scripts/by-user/root/borgmatic_hooks.iron_postgresql_giteadb.yaml 'finish' "{configuration_filename}" "{repository}" "{output}" "{error}"
    on_error:
        - /home/jobs/scripts/by-user/root/borgmatic_hooks.py /home/jobs/scripts/by-user/root/borgmatic_hooks.iron_postgresql_giteadb.yaml 'error' "{configuration_filename}" "{repository}" "{output}" "{error}"

    postgresql_databases:
        - name: giteadb
          # Use unix sockets instead of TCP.
          # See
          # https://torsion.org/borgmatic/docs/reference/configuration/
          #
          # hostname: 127.0.0.1
          # port: 5432
          username: gitea
          password: ${DB_PASS}
          format: tar
          options: "--verbose"
```

## Extras

### Torification

This is useful if you need to clone repositories, or do mirroring via TOR.
I did this before switching to PostgreSQL:

1. Install TOR and configure it

   ```shell
   apt-get install tor
   ```

2. run `systemctl edit gitea.service` and add this content:

   ```systemd
   [Unit]
   Description=Gitea (Git with a cup of tea)
   After=syslog.target
   After=network.target
   After=mysqld.service
   After=postgresql.service
   After=memcached.service
   After=redis.service

   # Comment or change these.
   Requires=network.target
   Requires=postgresql.service
   Requires=redis.service

   # Comment this if you don't need it.
   # See
   # https://docs.franco.net.eu.org/automated-tasks/scripts.html#notify-unit-status-py
   OnFailure=notify-unit-status@%n.service

   [Service]
   ExecStart=
   ExecStart=/usr/bin/torsocks --isolate /usr/bin/gitea web -c /etc/gitea/app.ini

   User=gitea
   Group=gitea
   Type=simple
   WorkingDirectory=~
   RuntimeDirectory=gitea
   LogsDirectory=gitea
   StateDirectory=gitea
   Environment=USER=gitea HOME=/var/lib/gitea GITEA_WORK_DIR=/var/lib/gitea
   Restart=always
   RestartSec=2s
   CapabilityBoundingSet=
   NoNewPrivileges=false
   #SecureBits=noroot-locked
   ProtectSystem=strict
   ProtectHome=true
   ReadWritePaths=/etc/gitea/app.ini
   PrivateTmp=true
   PrivateDevices=false
   PrivateUsers=false
   ProtectHostname=false
   ProtectClock=false
   ProtectKernelTunables=false
   ProtectKernelModules=false
   ProtectKernelLogs=false
   ProtectControlGroups=true
   LockPersonality=false
   MemoryDenyWriteExecute=false
   RestrictRealtime=false
   RestrictSUIDSGID=false
   SystemCallArchitectures=
   SystemCallFilter=
   SystemCallErrorNumber=EPERM
   ReadWriteDirectories=/var/spool/postfix/maildrop
   ```

3. restart gitea

   ```shell
   systemctl restart gitea.servvice
   ```

4. Test by cloning a repository from *The Tor Project*. Mirror this
   repository for example:

   [http://eweiibe6tdjsdprb4px6rqrzzcsi22m4koia44kc5pcjr7nec2rlxyad.onion/tpo/anti-censorship/bridgedb.git](http://eweiibe6tdjsdprb4px6rqrzzcsi22m4koia44kc5pcjr7nec2rlxyad.onion/tpo/anti-censorship/bridgedb.git)

### HTTP2

1. enable HTTP2 on Apache

   ```shell
   a2enmod http2
   ```

2. add this to your Apache configuration file. This setting can be global or
   per-virtualhost:

   ```apache
   Protocols h2 h2c http/1.1
   ```

3. if you have problems with HTTP2 add this to `/etc/gitea/app.ini`:

   ```ini
   [ui.notification]
   EVENT_SOURCE_UPDATE_TIME=-1
   ```

   See also [https://github.com/go-gitea/gitea/issues/11978](https://github.com/go-gitea/gitea/issues/11978)

### Caching

#### Solution

The repository/mirror problem was still there. After lots of trials and errors I found
a solution.

##### Redis

1. install Redis

   ```shell
   apt-get install redis-server
   ```

2. setup Redis to listen on a UNIX socket exclusively. Edit `/etc/redis/redis.conf`:

   ```conf
   unixsocket /var/run/redis/redis-server.sock
   unixsocketperm 770
   bind 127.0.0.1 ::1

   # This disables listening on TCP.
   port 0

   # Avoid saving on disk
   save ""

   # Remove authentication.
   requirepass ""
   ```

3. add the `gitea` user to the `redis` group so Gitea can have
   access to the socket.

   ```
   usermod -aG redis gitea
   ```

4. restart Redis and Gitea:

   ```shell
   systemctl restart redis-server.service gitea.service
   ```

5. check the logs at `/var/log/redis/redis-server.log`. If you have a warning
   about transparent hugepages. Add this service to Systemd.

   ```systemd
   # See
   # https://unix.stackexchange.com/a/363887
   # https://stackoverflow.com/a/64945381
   # CC BY-SA 3.0
   # (c) 2017 nelaaro

   [Unit]
   Description=madvise Transparent Huge Pages
   Before=redis-server.service
   Before=apache2.service
   Before=gitea.service
   Before=postgresql.service

   [Service]
   Type=oneshot
   ExecStart=/bin/sh -c "/usr/bin/echo "madvise" | tee /sys/kernel/mm/transparent_hugepage/enabled"
   ExecStart=/bin/sh -c "/usr/bin/echo "madvise" | tee /sys/kernel/mm/transparent_hugepage/defrag"

   [Install]
   WantedBy=multi-user.target
   ```

6. have a look at the [Redis ArchWiki page](https://wiki.archlinux.org/title/Redis).

##### Gitea configuration

To be able to use Redis in Gitea edit these sections in `/etc/gitea/app.ini`:

```ini
[cache]
ADAPTER  = redis
HOST     =  network=unix,addr=/var/run/redis/redis-server.sock,db=0,pool_size=100,idle_timeout=180s
ITEM_TTL = 24h

[session]
PROVIDER          = redis
PROVIDER_CONFIG  =  network=unix,addr=/var/run/redis/redis-server.sock,db=1,pool_size=100,idle_timeout=180s

[queue]
TYPE = redis
CONN_STR = network=unix,addr=/var/run/redis/redis-server.sock,db=2,pool_size=100,idle_timeout=180s

[queue.task]
QUEUE_TYPE = redis
QUEUE_CONN_STR = network=unix,addr=/var/run/redis/redis-server.sock,db=2,pool_size=100,idle_timeout=180s

[task]
QUEUE_TYPE = redis
QUEUE_CONN_STR = network=unix,addr=/var/run/redis/redis-server.sock,db=2,pool_size=100,idle_timeout=180s
```

**Note: using Redis for queues and tasks seems to have solved the original problem.**

### Updating gitea next time

1. temporarly disable the service

   ```
   systemctl stop gitea.service
   systemctl mask gitea.service
   ```

2. copy the original database into a new one

   ```sql
   CREATE DATABASE backup_giteadb_1.15.2 WITH TEMPLATE giteadb;
   ```

3. run the doctor on the original database

   ```shell
   HOME=/var/lib/gitea GITEA_WORK_DIR=/var/lib/gitea /usr/bin/gitea -c /etc/gitea/app.ini doctor --all
   HOME=/var/lib/gitea GITEA_WORK_DIR=/var/lib/gitea /usr/bin/gitea -c /etc/gitea/app.ini doctor --all --fix
   ```

4. update Gitea

    ```shell
    apt-mark unhold gitea
    apt-get update
    apt-get dist-upgrade
    ```

5. migrate the database

   ```bash
   HOME=/var/lib/gitea GITEA_WORK_DIR=/var/lib/gitea /usr/bin/gitea -c /etc/gitea/app.ini migrate
   ```

3. run the doctor again

   ```shell
   HOME=/var/lib/gitea GITEA_WORK_DIR=/var/lib/gitea /usr/bin/gitea -c /etc/gitea/app.ini doctor --all
   HOME=/var/lib/gitea GITEA_WORK_DIR=/var/lib/gitea /usr/bin/gitea -c /etc/gitea/app.ini doctor --all --fix
   ```

2. re-enable the service

   ```shell
   systemctl unmask gitea.service
   systemctl start gitea.service
   ```

### Apache2 reverse proxy for Gitea

1. modify Gitea configuration (`/etc/gitea/app.ini`):

   ```ini
   [server]
   PROTOCOL                   = unix
   DOMAIN                     = localhost
   HTTP_ADDR                   = /var/run/gitea/gitea.sock
   UNIX_SOCKET_PERMISSION      = 770

   ; Do not set this variable if PROTOCOL is set to 'unix'.
   # LOCAL_ROOT_URL             = %(PROTOCOL)s://%(HTTP_ADDR)s:%(HTTP_PORT)s/
   ```

2. Add the www-data user to gitea's group so it can access the socket.

   ```shell
   usermod -aG gitea www-data
   ```
3. Restart gitea and apache

   ```shell
   systemctl restart gitea.service apache2.service
   ```

4. clearnet configuration in `/etc/apache2/apache2.conf` or a virtual host file:

   variables (shell style):

   - `ONION_ADDRESS`: the TOR address where Gitea is exposed
   - `TCP_PORT`: the TCP/IP port where gitea should be listenting.
     Even if we are using a UNIX socket this is applicable. You can use Gitea's default port.
   - `SERVER_NAME`: the FQDN

   ```apache
   #########
   # Gitea #
   #########
   <IfModule mod_ssl.c>
   <VirtualHost *:443>
       UseCanonicalName on

       ProxyPreserveHost On

       Keepalive On
       RewriteEngine on
       AllowEncodedSlashes NoDecode

       ServerName ${SERVER_NAME}

       ProxyBadHeader Ignore

       # TOR.
       # Remove this if you don't want Gitea being avaliable on TOR.
       ServerAlias ${ONION_ADDRESS}
       Header set Onion-Location "http://${ONION_ADDRESS}%{REQUEST_URI}s"

       SSLCompression      off

       ProxyPass  / unix:/var/run/gitea/gitea.sock|http://127.0.0.1:${TCP_PORT}/ nocanon
       ProxyPassReverse  / unix:/var/run/gitea/gitea.sock|http://127.0.0.1:${TCP_PORT}/

       Include /etc/letsencrypt/options-ssl-apache.conf
       SSLCertificateFile /etc/letsencrypt/live/${SERVER_NAME}/fullchain.pem
       SSLCertificateKeyFile /etc/letsencrypt/live/${SERVER_NAME}/privkey.pem
   </VirtualHost>
   ```

5. if you use TOR add this configuration in `/etc/apache2/apache2.conf` or a virtual host file:

   variables (shell style):

   - `TCP_PORT`: the TCP/IP port where Gitea should be listenting. Even if we are using a UNIX socket this is applicable. You can use Gitea's default port
   - `APACHE_TCP_PORT`:  the TCP/IP port where Apache is listening
   - `SERVER_NAME`: TOR's FQDN

   ```apache
   #########
   # Gitea #
   #########
   <IfModule mod_ssl.c>
   <VirtualHost *:${APACHE_TCP_PORT}>
       UseCanonicalName on

       ProxyPreserveHost On
       ProxyRequests off
       AllowEncodedSlashes NoDecode

       Keepalive On
       RewriteEngine on

       ServerName ${SERVER_NAME}

       SSLCompression      off

       # Disable HTTP push and cloning for TOR.
       # TODO: enable only cloning but not push.
       RewriteRule  ^(.*)/info/refs /errors
       <Location "/errors">
           Deny from all
       </Location>

        # Redirect api and login to black holes.
        # Specific rules first, generic rules last.
        ProxyPass  /user/login unix:/var/run/gitea/gitea.sock|http://127.0.0.1:${TCP_PORT}/404 nocanon
        ProxyPassReverse  /user/login/ unix:/var/run/gitea/gitea.sock|http://127.0.0.1:${TCP_PORT}/404
        ProxyPass  /api unix:/var/run/gitea/gitea.sock|http://127.0.0.1:${TCP_PORT}/404 nocanon
        ProxyPassReverse  /api unix:/var/run/gitea/gitea.sock|http://127.0.0.1:${TCP_PORT}/404
        ProxyPass  / unix:/var/run/gitea/gitea.sock|http://127.0.0.1:${TCP_PORT}/ nocanon
        ProxyPassReverse  / unix:/var/run/gitea/gitea.sock|http://127.0.0.1:${TCP_PORT}/

       RequestHeader set X-Forwarded-Proto "http"
       RequestHeader set X-Forwarded-Port "${APACHE_TCP_PORT}"
       RequestHeader set X-Forwarded-Host "${SERVER_NAME}"

       # Distinguish normal traffic from tor's.
       RequestHeader set X-Forwarded-For "tor"
   </VirtualHost>
   </IfModule>
   ```

6. if you use TOR, add this to `/etc/tor/torrc`

   variables (shell style):

   - `APACHE_TCP_PORT`:  the TCP/IP port where Apache is listening

   ```conf
   HiddenServiceDir /var/lib/tor/gitea/
   HiddenServicePort 80 127.0.0.1:${APACHE_TCP_PORT}
   ```

   Follow [these](https://2019.www.torproject.org/docs/tor-onion-service) instructions.

See these links for the UNIX socket explanation:

- [https://httpd.apache.org/docs/trunk/mod/mod_proxy.html#proxypass](https://httpd.apache.org/docs/trunk/mod/mod_proxy.html#proxypass)
- [https://wiki.archlinux.org/title/Talk:Gitea#Apache_Reverse_Proxy_Over_Unix_Socket](https://wiki.archlinux.org/title/Talk:Gitea#Apache_Reverse_Proxy_Over_Unix_Socket)

### GitHub issue

Follow it [here](https://github.com/go-gitea/gitea/issues/16992)