Apache Unomi Release Guide

The Apache Unomi project periodically declares and publishes releases. A release is one or more packages of the project artifact(s) that are approved for general public distribution and use. They may come with various degrees of caveat regarding their perceived quality and potential for change, such as "alpha", "beta", "stable", etc.

The Unomi community treats releases with great importance. They are a public face of the project and most users interact with the project only through the releases. Releases are signed off by the entire Unomi community in a public vote.

Each release is executed by a Release Manager, who is selected among the Unomi committers. This document describes the process that the Release Manager follows to perform a release. Any changes to this process should be discussed and adopted on the dev@ mailing list.

Please remember that publishing software has legal consequences. This guide complements the foundation-wide Product Release Policy and Release Distribution Policy.

Overview

Apache Unomi release guide

The release process consists of several steps:

  1. Decide to release
  2. Prepare for the release
  3. Build a release candidate
  4. Vote on the release candidate
  5. During vote process, run validation tests
  6. If necessary, fix any issues and go back to step 3.
  7. Finalize the release
  8. Promote the release

Decide to release

Deciding to release and selecting a Release Manager is the first step of the release process. This is a consensus-based decision of the entire community.

Anybody can propose a release on the dev@ mailing list, giving a solid argument and nominating a committer as the Release Manager (including themselves). There's no formal process, no vote requirements, and no timing requirements. Any objections should be resolved by consensus before starting the release.

In general, the community prefers to have a rotating set of 3-5 Release Managers. Keeping a small core set of managers allows enough people to build expertise in this area and improve processes over time, without Release Managers needing to re-learn the processes for each release. That said, if you are a committer interested in serving the community in this way, please reach out to the community on the dev@ mailing list.

Basically we will be following the procedure described here with a few more steps and details before and after.

Checklist to proceed to the next step

  1. Community agrees to release
  2. Community selects a Release Manager

Prepare for the release

Pre-flight checks

  1. Make sure all copyright years have been changed in NOTICE and any other file
  2. Check that all SNAPSHOT versions are correct by searching for SNAPSHOT in all XML files

GPG Key configuration

  1. Make sure to have a PGP key for your @apache.org email. if not you can create a new one locally:
    gpg --gen-key
  2. Make sure that your default GPG key is configured to use an signature with an @apache.org email. You can do this by adding the following to your ~/.gnupg/gpg.conf :
    default-key <key-uid>
    replacing <key-uid> with the key uid you want to use by default. You can get the <key-uid> using:
    gpg --list-secret-keys
    You should then check by signing a file and verifying its signature that you have the right default key:
    gpg -ab test.txt
    gpg --verify test.txt.asc test.txt
                          
    This should tell you which key was used and display the email address.
  3. Make sure your public KEY is available in this files (if not add your key to the files and commit/push):
    1. GIT https://github.com/apache/unomi/blob/master/KEYS (cherry-pick to the branches if necessary)
    2. SVN https://dist.apache.org/repos/dist/dev/unomi/KEYS
    3. SVN https://dist.apache.org/repos/dist/release/unomi/KEYS
  4. Make sure your public KEY is also available in public pgp server like https://keys.openpgp.org/. You can export your key to file with:
    gpg --export USERNAME@apache.org > USERNAME@apache.pub
    Then upload it on https://keys.openpgp.org/upload

Jira

  1. Setup new version in JIRA project
  2. Update all JIRAs, close the ones that are still not closed, review tickets that can be closed, postpone to another release tickets that are not managed now. You can use this JIRA page: Unomi JIRA releases. Then review the list of tickets that need to be updated, also additionally you can check the list of commits since previous release and insure they are closed using fix version: 2.0.0. Ideally all tickets listed for version: 2.0.0 should be closed or moved to the next version in case they are not finished/started.
  3. Then you can also generate a release notes automatically using: JIRA release note, By choosing the version: 2.0.0. It will be useful for linking the release notes in the voting mail.

Git

  1. Clone and checkout the project using:
    git clone https://gitbox.apache.org/repos/asf/unomi.git
    in a separate directory, to make sure you don???t have any additional files in the build (such as allCountries.zip or other binary files that should not be there)
  2. Simply checkout an existing branch for release
    cd unomi
    git checkout unomi-1.6.x
                        
  3. OR Create a new branch if needed for the release using:
    cd unomi
    git checkout -b unomi-1.6.x
                        

Build the project

  1. Make sure you have a Docker installation properly configured on the machine building the project, as it is required to build everything.
  2. Tests source, javadocs, docker and signature generation using:
    mvn clean install -P apache-release,integration-tests,docker
  3. Make sure we generate all sub-projects using:
    mvn clean install -DskipITs=true -DskipTests=true -P integration-tests,rat,apache-release,docker,\!run-tests
  4. Check that there are no JARs or ZIP files in the source code, you can do this by looking at the generated RAT report here :
    less target/unomi-root-2.0.0-SNAPSHOT.rat
  5. Check that the KEYS file only contains signatures with @apache.org addresses (if there are non @apache.org don???t remove them because they have been used to sign older releases)
  6. Check the Copyright notice years in the NOTICE file and anywhere else where it might be (by using search and replace)
  7. If you get JavaDoc errors or other errors you must correct all of them before proceeding further
  8. If you get a error while signing despite having the proper keys install that looks like this:
    [INFO] --- maven-gpg-plugin:1.5:sign (default) @ unomi-root ---
    gpg: 'echec de la signature : Inappropriate ioctl for device
    gpg: signing failed: Inappropriate ioctl for device
    you can either try to sign a file like this:
    gpg -ab test.txt
    this will ask for your password and remember it in the agent for a while, or if you want a more permanent solution: https://dan.cx/2016/11/gpg-inappropriate-ioctl-for-device-errors

Publish the release

Repository

  1. Setup server passwords for publishing as document here: https://www.apache.org/dev/publishing-maven-artifacts.html and https://maven.apache.org/guides/mini/guide-encryption.html#How_to_encrypt_server_passwords
  2. Check into the target directory and unzip the source release and compile it using:
    pushd target
    gpg --verify unomi-root-2.0.0-SNAPSHOT-source-release.zip.asc unomi-root-2.0.0-SNAPSHOT-source-release.zip
    shasum -a 512 unomi-root-2.0.0-SNAPSHOT-source-release.zip
    cat unomi-root-2.0.0-SNAPSHOT-source-release.zip.sha512
    unzip unomi-root-2.0.0-SNAPSHOT-source-release.zip
    cd unomi-root-2.0.0-SNAPSHOT
    mvn clean install
    popd
    to check that the packaged source build properly
  3. Go back to the root project directory (pushd/popd did that for you normally) and run:
    mvn release:prepare -DskipITs=true -DskipTests=true -Drelease.arguments="-Papache-release,integration-tests -DskipITs=true -DskipTests=true"  -DdryRun=true -P apache-release,integration-tests,docker,\!run-tests
  4. Publish a snapshot to test the deployment passwords:
    mvn deploy -DskipITs=true -DskipTests=true -P \!run-tests
  5. Delete the release.properties file
    rm release.properties
  6. mvn -P apache-release,integration-tests,docker,\!run-tests release:prepare
  7. mvn -DskipITs=true -DskipTests=true -Drelease.arguments="-Papache-release,integration-tests -DskipITs=true -DskipTests=true" -P integration-tests,docker,\!run-tests release:perform
  8. If something fails, make sure you first drop the staging repository created here: https://repository.apache.org/#stagingRepositories.
    If you need to relaunch the release:perform and don???t have a release.properties, create a release.properties file with the following contents:
    scm.url=scm:git:https://gitbox.apache.org/repos/asf?p=unomi.git
    scm.tag=unomi-root-2.0.0
    and run mvn release:perform
  9. Connect to https://repository.apache .org/#stagingRepositories and look for the open staging repositories, you should have two (one for everything and the other for KAR/features) for releases <= 1.2 and just one for versions >= 1.3.0 (because of changes in the Karaf plugins). Close the repositories and given a meaningful comment when closing such as:
    Apache Unomi 2.0.0 Release Candidate 1

Vote

  1. Upload the source and binary distributions to the Apache Dist repository before the PMC vote:
    svn checkout https://dist.apache.org/repos/dist/dev/unomi unomi-dev
  2. cd unomi-dev
    mkdir 2.0.0
  3. Copy all the Zip and Tarbars including ASC (but do not copy the SHA1 or MD5 sum) files from: https://repository.apache.org/content/repositories/orgapacheunomi-1014/org/apache/unomi/unomi/2.0.0/ and https://repository.apache.org/content/repositories/orgapacheunomi-1014/org/apache/unomi/unomi-root/2.0.0/ using commands such as :
    export REPO_ID=orgapacheunomi-1033
    wget https://repository.apache.org/content/repositories/$REPO_ID/org/apache/unomi/unomi-root/2.0.0/unomi-root-2.0.0-source-release.zip
    wget https://repository.apache.org/content/repositories/$REPO_ID/org/apache/unomi/unomi-root/2.0.0/unomi-root-2.0.0-source-release.zip.asc
    wget https://repository.apache.org/content/repositories/$REPO_ID/org/apache/unomi/unomi/2.0.0/unomi-2.0.0.tar.gz
    wget https://repository.apache.org/content/repositories/$REPO_ID/org/apache/unomi/unomi/2.0.0/unomi-2.0.0.tar.gz.asc
    wget https://repository.apache.org/content/repositories/$REPO_ID/org/apache/unomi/unomi/2.0.0/unomi-2.0.0.zip
    wget https://repository.apache.org/content/repositories/$REPO_ID/org/apache/unomi/unomi/2.0.0/unomi-2.0.0.zip.asc
  4. Rename the source and binary files to something shorter and consistent with previous releases and generate the SHA 512 checksum manually:
    
    mv unomi-root-2.0.0-source-release.zip unomi-2.0.0-src.zip
    mv unomi-root-2.0.0-source-release.zip.asc unomi-2.0.0-src.zip.asc
    shasum -a 512 unomi-2.0.0-src.zip > unomi-2.0.0-src.zip.sha512
    
    mv unomi-2.0.0.zip unomi-2.0.0-bin.zip
    mv unomi-2.0.0.zip.asc unomi-2.0.0-bin.zip.asc
    shasum -a 512 unomi-2.0.0-bin.zip > unomi-2.0.0-bin.zip.sha512
    
    mv unomi-2.0.0.tar.gz unomi-2.0.0-bin.tar.gz
    mv unomi-2.0.0.tar.gz.asc unomi-2.0.0-bin.tar.gz.asc
    shasum -a 512 unomi-2.0.0-bin.tar.gz > unomi-2.0.0-bin.tar.gz.sha512
                    
  5. cd ..
    svn add 2.0.0
  6. svn commit -m "Apache 2.0.0 Release (for PMC voting)"
  7. Send out to the Unomi mailing list a mail to start the voting process, see [1]
  8. If the vote is refused or cancelled, peform the following steps to restart the release process:
    1. Drop the release in Nexus
    2. Remove the tag in Git:
      git push --delete origin unomi-root-2.0.0
      git tag -d unomi-root-2.0.0
    3. Correct any problems in the source, make sure to do them in master and cherry-pick them to the relevant branches
    4. Reset all versions with the following command:
      mvn versions:set -DskipITs=true -DskipTests=true -P integration-tests,rat,apache-release,docker,\!run-tests
    5. Restart at the release:prepare step
  9. If the vote is refused, you will need to restart the release process. Make sure you remove the staging repositories as well as the release tag. You will also need to reset the version in the project using:
    mvn versions:set
  10. Send out a vote summary in the same thread to the Unomi mailing list once the voting period has expired, see [2]
  11. Promote the staging repositories now that the release is approved

Finalize the release

  1. Move the files uploaded to the unomi-dev repository to the unomi-release repository by doing the following:
    svn mv https://dist.apache.org/repos/dist/dev/unomi/2.0.0 https://dist.apache.org/repos/dist/release/unomi/2.0.0 -m "Apache Unomi 2.0.0 Release"
  2. In JIRA mark the version as released and add a release date
  3. Connect to https://repository.apache.org/#stagingRepositories and look for the open staging repositories, you should have one. Select the staging repositoriy and click on release.
  4. Update the website (in project `unomi-site`) to point to the new release on the download page by first copying the current release to the archive page. Make sure you change the links to use the archive.apache.org instead of the closer.lua script (see other archived versions for example). Then you can update the download page with the proper links to the new release (no direct links to www.apache.org/dist, make sure you use the closer.lua script). Be careful with links that contain & (ampersand) characters, they must be replaced with & Finally on the home page update the news section to add the release.
    git clone https://gitbox.apache.org/repos/asf/unomi-site.git
    Perform any modifications, notably change the versions in the pom.xml file:
        <properties>
            <latest.stable.version>1.5.4</latest.stable.version>
            <next.development.version>1.5.5-SNAPSHOT</next.development.version>
            <next.stable.version>1.5.5</next.stable.version>
            <latest.development.version>2.0.0-SNAPSHOT</latest.development.version>
        </properties>
    
    Then modify the index.html, download.html to add the new files. For the documentation see the next step.
  5. Test the site modifications using the generated site HTML using Jekyll serve
    ./mvnw clean install gem:exec@jekyll-serve-filtered
    Connect to http://127.0.0.1:4000 to check the modifications. If you need to make more modifications, simply relaunch the above line after modifying the HTML files.
  6. Deploy the web site changes using:
    ./mvnw clean install
    ./mvnw install scm-publish:publish-scm -Dusername=YOUR_APACHE_USERNAME -Dpassword=YOUR_APACHE_PASSWORD
  7. Then make sure to commit all the changes to the web site's git project:
    git commit -m "Apache Unomi 2.0.0 Release website update"
    git push

Documentation

  1. In the `unomi` project, on the master, branch, make sure that you have a archive for the current branch in the `manual/src/archives` directory. If not, create one with the current main version and copy the contents of `manual/src/main` into the newly created directory. Make sure you add all the files to the git repository.
  2. Modify the `generate-site.sh` and `generate-site-and-upload.sh` if needed to add the new version you added in the previous step.
  3. Launch
    ./generate-site.sh
    from the project's root directory.
  4. Check the generated site by opening target/staging/manual/index.html and check your changes and verify that there are no broken links.
  5. Upload the site modification by using:
    ./generate-site-and-upload.sh APACHE_LDAP_USERNAME APACHE_LDAP_PASSWORD
  6. You will now need to publish the new version of the manual as downloadable resources
    cd manual
    cd target
    svn co https://dist.apache.org/repos/dist/release/unomi/2.0.0
    mv unomi-manual-1_6_x.pdf 2.0.0
    mv unomi-manual-1_6_x.pdf.asc 2.0.0
    mv unomi-manual-1_6_x.zip 2.0.0
    mv unomi-manual-1_6_x.pdf.sha512 2.0.0
    mv unomi-manual-1_6_x.zip.asc 2.0.0
    mv unomi-manual-1_6_x.zip.sha512 2.0.0
    cd 2.0.0
    svn add unomi-manual*
    svn commit -m "Update Unomi manual packages"
  7. Go back to the site project and make sure you update the documentation page to link to the proper versions of the documents.

Docker image

  1. Checkout the tagged version:
    git checkout tags/unomi-root-2.0.0
  2. Change to the Docker directory. Make sure you have docker running locally (start Docker Desktop for example):
    cd docker
  3. Launch the docker build and push
    mvn clean install
    mvn docker:push

Rollback

  1. Delete the tag:
    git push --delete origin unomi-root-2.0.0
    git tag --delete unomi-root-2.0.0
  2. Reset to the previous commit before the release preparation:
    git reset --hard c65f9897ec5f31d9d22ad639738c7db9d109aa77
    git push origin -f

Announce

  1. Send an announce to the project mailing list and the incubator mailing list to announce the new release using mail template [3]
  2. If you were releasing from a branch, make sure you merge all the branch changes for the release back to the master.

Social media

Tweet, post on Facebook, LinkedIn, and other platforms. Ask other contributors to do the same.

Remove old releases

In order to lighten the load on mirrors, remove old releases from the dist server once the new release has properly been deployed to all the mirrors. Old releases are automatically archived so they are not needed on the mirrors anymore. Just make sure that all the links in the download page do point to the archive server.

Checklist to declare the process completed

  1. Release announced on the user@ mailing list.
  2. Release recorded in reporter.apache.org.
  3. Release announced on social media.
  4. Completion declared on the dev@ mailing list.
  5. Check that old release where removed from the dist server

Improve the process

It is important that we improve the release processes over time. Once you???ve finished the release, please take a step back and look what areas of this process and be improved. Perhaps some part of the process can be simplified. Perhaps parts of this guide can be clarified.

If we have specific ideas, please start a discussion on the dev@ mailing list and/or propose a pull request to update this guide. Thanks!


Mail templates

[1] Mail template for the Unomi PMC vote:

Subject: [VOTE] Apache Unomi 2.0.0 release [TAKE2]
Body:
Hi all,

I submit Apache Unomi 2.0.0 release [TAKE2] to your vote.

The following corrections were done since TAKE 1:
NOTICE year (2016) has been updated to 2018
unexpected binaries inside zip file [1] : zip file has been completely removed
All artefacts are now signed in an apache email address

Staging Repository:
*https://repository.apache.org/content/repositories/orgapacheunomi-1021/
<https://repository.apache.org/content/repositories/orgapacheunomi-1021/>*

You can find the sources here :

*https://repository.apache.org/content/repositories/orgapacheunomi-1021/org/apache/
unomi/unomi-root/2.0.0/unomi-root-2.0.0-source-release.zip
<https://repository.apache.org/content/repositories/orgapacheunomi-1021/org/apache/
unomi/unomi-root/2.0.0/unomi-root-2.0.0-source-release.zip>*

Convenience binaries are also available here:
*https://repository.apache.org/content/repositories/orgapacheunomi-1021/org/apache/
unomi/unomi/2.0.0/
<https://repository.apache.org/content/repositories/orgapacheunomi-1021/org/apache/
unomi/unomi/2.0.0/>*

Git tag:
unomi-root-2.0.0

Release Notes:
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319220&
version=12341610

Please vote to approve this release:

[ ] +1 Approve the release
[ ] -1 Don't approve the release (please provide specific comments)

This vote will be open for at least 72 hours, or until the necessary number
of binding votes (3 +1) is reached.

Please let me know if you have any questions.

Thanks,
Regards
            
[2] Mail template for the results of the Unomi PMC vote:

Subject: [RESULT][VOTE] Apache Unomi 2.0.0 release [TAKE2]
Body:
Hi,

Thank you for your votes.

Here's the summary :

3 binding votes +1:

John Doe 1
John Doe 2
John Doe 3

2 non-binding vote +1:
John Doe 4
John Doe 5

No 0 or -1.

The proposal to release Unomi 2.0.0 is approved by the team.

Thanks,
John Doe 1
            
[3] Announce mailing list template:

Subject : [ANNOUNCE] Apache Unomi 2.0.0 Release
Body:
The Apache Unomi team would like to announce the release of Apache
Unomi 2.0.0.

Release notes are here:
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319220&
version=12338361

Apache Unomi is a Java Open Source customer data platform, a Java server
designed to manage customers, leads and visitors??? data and help personalize customers???
experiences.

More details regarding Apache Unomi can be found here:
http://unomi.apache.org/

The release artifacts can be downloaded here:
https://dist.apache.org/repos/dist/release/incubator/unomi/2.0.0/

All JIRAs completed for this release are tagged with 'FixVersion =
2.0.0'; the JIRA release notes can be found here:
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319220&
version=12338361

Thanks!
The Apache Unomi Team