blog/_pages/software_instructions.md

---
layout: page
title: Software instructions
permalink: /software/instructions/
---

## Table of contents

<!--TOC-->

- [Table of contents](#table-of-contents)
- [Introduction](#introduction)
- [Extract](#extract)
- [Signing keys](#signing-keys)
- [Terminology](#terminology)
- [Methods](#methods)
  - [Upload (what I have to do)](#upload-what-i-have-to-do)
    - [Get the variables](#get-the-variables)
    - [Create an archive](#create-an-archive)
    - [Signing](#signing)
    - [Checksums](#checksums)
    - [Python project on PyPI](#python-project-on-pypi)
    - [Update the entry](#update-the-entry)
    - [Create a new release file](#create-a-new-release-file)
    - [Update the changelog](#update-the-changelog)
    - [Update the table of contents](#update-the-table-of-contents)
  - [Download (what you have to do)](#download-what-you-have-to-do)
    - [Get the public key](#get-the-public-key)
    - [Download the repository](#download-the-repository)
    - [Check the signature](#check-the-signature)
    - [Run the checksums](#run-the-checksums)
    - [Extract](#extract-1)
    - [Python project on PyPI](#python-project-on-pypi-1)

<!--TOC-->

## Introduction

Here you will find methods to assert the authenticity of the presented software packages.

## Extract

The following extract is from a [post by Mike Gerwitz](https://mikegerwitz.com/2012/05/a-git-horror-story-repository-integrity-with-signed-commits#trust):

> Git Host
>>
>>    Git hosting providers are probably the most easily overlooked trustees—providers like Gitorious, GitHub, Bitbucket, SourceForge, Google Code, etc. Each provides hosting for your repository and “secures” it by allowing only you, or other authorized users, to push to it, often with the use of SSH keys tied to an account. By using a host as the primary holder of your repository—the repository from which most clone and push to—you are entrusting them with the entirety of your project; you are stating, “Yes, I trust that my source code is safe with you and will not be tampered with”. This is a dangerous assumption. Do you trust that your host properly secures your account information? Furthermore, bugs exist in all but the most trivial pieces of software, so what is to say that there is not a vulnerability just waiting to be exploited in your host’s system, completely compromising your repository?
>>
>>    It was not too long ago (March 4th, 2012) that a public key security vulnerability at GitHub was exploited by a Russian man named Egor Homakov, allowing him to successfully commit to the master branch of the Ruby on Rails framework repository hosted on GitHub. Oops.

    Copyright © 2019 Mike Gerwitz. Licensed under the Creative Commons Attribution-ShareAlike 4.0 International License.

## Signing keys

You may contact me directly to obtain the public key fingerprint in a different way.

| Key | Fingerprint |
|-----|-------------|
| [pgp_pubkey_since_2019.txt]({{ site.baseurl }}/pubkeys/pgp_pubkey_since_2019.txt) | [pgp_pubkey_fingerprint_since_2019.txt]({{ site.baseurl }}/pubkeys/pgp_pubkey_fingerprint_since_2019.txt) |

## Terminology

- `project_dir`: the full path directory of the project
- `project`: the project name
- `project_python_module`: the python module name of the project.
                           For example: `md-toc` is `md_toc`
- `tag`: the git tag name which is usually [semver](https://semver.org/)ed.
         Get this variable with:

   ```shell
   git describe --tags $(git rev-list --tags --max-count=1)
   ```

- `tag_raw`: same as `tag` but pad each component of the tag with 6 zeros.
             For example: `12.121.5` becomes `000012.000121.000005`
- `signing_key`: the public key file used to sign the archive file
- `has_changelog`: the project has a changelog entry for a specific release. Value must be either `true` or `false`
- `is_on_pypi`: the project is on PyPI. Value must be either `true` or `false`
- `project_version_release_date`: the release date of a software version (tag) in UTC format.
                                  This is the one liner command to get the date:

  ```shell
  python3 -c 'from dateutil import parser as dateutil_parser; from dateutil.tz import UTC; import sys; p = dateutil_parser.parse(sys.argv[1]); u=p.astimezone(UTC); print(u.strftime("%Y-%m-%d"))' "$(git tag -l --format='%(taggerdate)' $(git describe --tags $(git rev-list --tags --max-count=1)))"
  ```

- `project_version_release_timestamp`: the timestamp of a software version (tag) in UTC format.
                                       This is the one liner command to get the timestamp:

  ```shell
  python3 -c 'from dateutil import parser as dateutil_parser; from dateutil.tz import UTC; import sys; p = dateutil_parser.parse(sys.argv[1]); u=p.astimezone(UTC); print(u.strftime("%Y-%m-%d %T"))' "$(git tag -l --format='%(taggerdate)' $(git describe --tags $(git rev-list --tags --max-count=1)))"
  ```

- `changelog_slugified_header`: the slugified header corresponding to a tag in a changelog file
- `url`: a generic url
- `PyPI_download_page`: the URL of the download page of the package on PyPI

## Methods

### Upload (what I have to do)

What follows are the steps I use to upload the software on this page.

#### Get the variables

Go into the project directory and then:

    export project_dir="$(pwd)"
    export project="$(basename "$(pwd)")"
    export tag="$(git tag | sort --human-numeric-sort --ignore-leading-blanks | tail --lines=1)"

#### Create an archive

    cd /tmp
    git -C ${project_dir} archive --format=tar.gz --output=/tmp/${project}-${tag}.tar.gz --prefix=${project}-${tag}/ ${tag}

#### Signing

    gpg --armor --output ${project}-${tag}.tar.gz.sig --detach-sig ${project}-${tag}.tar.gz

#### Checksums

    sha512sum ${project}-${tag}.tar.gz > ${project}-${tag}.tar.gz.SHA512SUM.txt
    sha256sum ${project}-${tag}.tar.gz > ${project}-${tag}.tar.gz.SHA256SUM.txt

#### Python project on PyPI

    make dist
    cd dist
    sha256sum ${project_python_module}-${tag}-py3-none-any.whl > ${project_python_module}-${tag}-py3-none-any.whl.SHA256SUM.txt
    md5sum ${project_python_module}-${tag}-py3-none-any.whl > ${project_python_module}-${tag}-py3-none-any.whl.MD5SUM.txt

#### Update the entry

Create a new entry in the `_pages/software.md` file

```
- [`${tag}`]({{ site.baseurl }}/software/${project}-${tag}/release.html)
```

#### Create a new release file

Create a new file called `_software/${project}-${tag}/release.md` and add the following.
If it's not a Python project you must omit `software_name_python_module`:

```yaml
---
layout: software_release
enable_markdown: true
title: release
excerpt: none
software_name: ${project}
software_name_python_module: ${project_python_module}
software_version: ${tag}
software_version_raw: ${tag_raw}
release_date: ${project_version_release_date}
is_on_pypi: ${is_on_pypi}
has_changelog: ${has_changelog}
signing_public_key: ${signing_key}
---
```

#### Update the changelog

Add a changelog file in `_software/${project}-${tag}/changelog.md`.
Add the `### Added`, `### Removed`, etc... contents if applicable.

In `./_software/CHANGELOG-${project}.md`:

- update the front matter with the appropriate data:
  - `updated` and `date` must correspond to `${project_version_release_timestamp}`
  - `last_version` must correspond to `${tag}`

#### Update the table of contents

Run this command manually or use pre-commit:

    md_toc -p github -l 6 software.md

### Download (what you have to do)

Run the following to download and verify the software.

#### Get the public key

If the public key is unknown you must import it from a trusted source:

    cd /tmp
    wget "${public_key_url}"
    gpg --import "${public_key_file}"

#### Download the repository

    cd /tmp
    wget ${url}/${project}-${tag}.tar.gz.sig

#### Check the signature

    wget ${url}/${project}-${tag}.tar.gz
    gpg --verify ${project}-${tag}.tar.gz.sig

#### Run the checksums

    sha512sum --check ${project}-${tag}.tar.gz.SHA512SUM.txt
    sha256sum --check ${project}-${tag}.tar.gz.SHA256SUM.txt

#### Extract

    tar -xvzf ${project}-${tag}.tar.gz

#### Python project on PyPI

    wget ${PyPI_download_page}/${project_python_module}-${tag}-py3-none-any.whl
    sha256sum --check ${project_python_module}-${tag}-py3-none-any.whl.SHA256SUM.txt
    md5sum --check ${project_python_module}-${tag}-py3-none-any.whl.MD5SUM.txt