RenJuan/TASKPM
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Merge pull request #1997 from kwoot/Adding-postgresql-db-migration-script-workflow

Browse source 
Adding postgresql db migration script workflow
This commit is contained in:
Jeroen Baten 2025-11-18 15:42:22 +01:00 • committed by GitHub
commit f574bbcc7f
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
4 changed files with 292 additions and 39 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

186
.forgejo/workflows/generate-db-upgrade-script-psql.yml Normal file
View file

@ -0,0 +1,186 @@
# This workflow will generate the database upgrade script for PostgreSQL
name: Generate PostgreSQL database upgrade script.
on:
workflow_dispatch:
jobs:
build:
runs-on: docker
container:
image: ubuntu:24.04
services:
postgres:
image: postgres:16.0
env:
POSTGRES_USER: libreplan
POSTGRES_PASSWORD: libreplan
POSTGRES_DB: libreplandev
ports:
- 5432:5432
options: >-
--health-cmd pg_isready
--health-interval 10s
--health-timeout 5s
--health-retries 5
steps:
# Define job-id
- name: Debug info
run: |
cat <<'EOF'
${{ toJSON(forgejo) }}
EOF
- name: My current run-id
run: echo "${{ forgejo.run_id }}"
# Define job-id
- name: install needed software
run: |
apt-get update
apt-get install -y postgresql-client nodejs git maven libpostgresql-jdbc-java
# Define job-id
- name: Checkout code
uses: actions/checkout@v3
with:
fetch-depth: 0
fetch-tags: true
show-progress: true
# This is only possible AFTER the checkout step.
- name: actions prep
id: artifact-upload-prep
run: echo "artifact-id=$(git rev-parse --short=10 HEAD)" >> "$GITHUB_OUTPUT"
# Let's try caching maven stuff.
- name: Cache Maven repository
uses: actions/cache@v4
with:
path: ~/.m2/repository
key: maven-${{ hashFiles('**/pom.xml') }}
restore-keys: maven-
# Example 1: create extra database
# - name: Create libreplandevtest database
# env:
# PGPASSWORD: ${{ secrets.POSTGRES_PASSWORD }}
# run: |
# PGPASSWORD='libreplan' psql -h postgres -U libreplan -d postgres -v ON_ERROR_STOP=1 -c "CREATE DATABASE libreplandevtest;"
- name: Set up Java 8
uses: actions/setup-java@v4
with:
distribution: temurin # of adopt, zulu, corretto, liberica …
java-version: 8
- name: Verify Java version
run: java -version
# Determine maven version in container
- name: Show maven version number
run: mvn -v
# Wait for PostgreSQL to be ready
- name: Wait for PostgreSQL to be ready
shell: bash
env:
PGPASSWORD: libreplan
run: |
for i in {1..30}; do
psql -h postgres -U libreplan -d libreplandev -c 'select 1' && break
sleep 2
done
# - name: "Checkout LibrePlan ${{ vars.full_last_release}} first."
# shell: bash
# run: |
# echo "First checkout older LibrePlan ${{ vars.full_last_release}} release"
# git checkout ${{ vars.full_last_release }}
# echo "Current branch: $(git rev-parse --abbrev-ref HEAD)"
# - name: Intermediare patch step. Fix old HTTP repo URLs in POMs
# shell: bash
# run: |
# find . -name "pom.xml" -print0 | xargs -0 sed -i \
# -e 's#http://gettext-commons.googlecode.com/svn/maven-repository#https://gettext-commons.googlecode.com/svn/maven-repository#g' \
# -e 's#http://nexus.***.org/content/repositories/thirdparty#https://nexus.***.org/content/repositories/thirdparty#g'
# # Setup clean empty database
# - name: Build clean ${{ vars.short_last_release }} db with Maven
# shell: bash
# run: |
# echo "Running maven to setup a ${{ vars.short_last_release }} database for a clean install"
# mvn clean install --no-transfer-progress -DskipTests -P-userguide,-reports,-i18n
# Instead of trying to do a build of the old version, just load the old database from the repo files.
- name: Populate database with database layout of last release.
shell: bash
run: |
echo "Populate database with database layout of ${{ vars.full_last_release }}"
cat scripts/database/install.sql | PGPASSWORD='libreplan' psql -h postgres -U libreplan -d libreplandev
- name: Switch to the new release.
shell: bash
run: |
echo "Switch to ${{ vars.short_new_release }}."
echo "We do not need to switch as we did not swotch before :-)"
# git checkout "${{ vars.full_new_release }}"
echo "Current branch: $(git rev-parse --abbrev-ref HEAD)"
# Yes, this looks ugly. But when cleanup over several lines mvn
# starts complaining: The POM for -DdataSource.url=jdbc:postgresql:jar://postgres is missing
# Starts thinking the datasource url is a pom it should download. Weird.
- name: "Generate database changes since version ${{ vars.short_last_release }}"
shell: bash
run: |
mvn clean install -DskipTests -P-userguide,-reports,-i18n,-liquibase-update,liquibase-updatesql -DdataSource.url=jdbc:postgresql://postgres:5432/libreplandev -DdataSource.user=libreplan -DdataSource.password=libreplan -DjdbcDriver.className=org.postgresql.Driver
# This will/should generate a file libreplan-business/target/liquibase/migrate.sql
# with the SQL script to migrate from previous version to the new one.
ls -l libreplan-business/target/liquibase/migrate.sql
- name: "The details of the generated migration script are:"
shell: bash
run: |
ls -l libreplan-business/target/liquibase/migrate.sql
lines=$( cat libreplan-business/target/liquibase/migrate.sql | grep -v '^--' | wc -l )
echo "The resulting migration script contained $lines lines. Processing migration script."
- name: "Copy the sql-file to database scripts folder"
shell: bash
run: |
cp -v libreplan-business/target/liquibase/migrate.sql scripts/database/upgrade_${{ vars.short_new_release }}.sql
- name: "Add information about version in the script"
shell: bash
run: |
sed -i "s/-- Update Database Script/-- Update Database Script - LibrePlan ${{ vars.short_new_release }}/" scripts/database/upgrade_${{ vars.short_new_release }}.sql
- name: "Append script at the end of ``install.sql`` file (with a separation of 2 new lines)"
shell: bash
run: |
echo -ne "\n\n" >> scripts/database/install.sql
cat scripts/database/upgrade_${{ vars.short_new_release }}.sql >> scripts/database/install.sql
# Upload the result
- name: Upload upgrade_${{ vars.short_new_release }}.sql
uses: actions/upload-artifact@v3
with:
name: upgrade_${{ vars.short_new_release }}.sql
path: scripts/database/upgrade_${{ vars.short_new_release }}.sql
retention-days: 3
- name: Upload install.sql
uses: actions/upload-artifact@v3
with:
name: install.sql
path: scripts/database/install.sql
retention-days: 3

40
.forgejo/workflows/generate-news-file.yml
View file

@ -10,27 +10,6 @@ name: Generate NEWS file.
on:
workflow_dispatch:
inputs:
full-last-release:
description: 'What was the full name of the last release'
default: "libreplan-"
required: true
type: string
short-last-release:
description: 'What was the short name of the last release'
default: "1.4."
required: true
type: string
full-new-release:
description: 'What is the full name of the new release'
default: "libreplan-"
required: true
type: string
short-new-release:
description: 'What is the short name of the new release'
default: "1.4."
required: true
type: string
jobs:
build:
@ -73,7 +52,7 @@ jobs:
echo "Writing to file: $FILE"
#Version 1.4.0 (29 Apr 2013)
#---------------------------
headstr="Version ${{ inputs.short-new-release }} ($(LANG=en;date '+%d %b %Y'))"
headstr="Version ${{ vars.short_new_release }} ($(LANG=en;date '+%d %b %Y'))"
headstrlen=${#headstr}
# echo "headstrlen: $headstrlen"
headline="$(printf -- '-%.0s' $(seq 1 "$headstrlen"))"
@ -86,7 +65,7 @@ jobs:
run: |
filepart=summary
FILE="NEWS.rst.summary"
#verstr="Version ${{ inputs.short-new-release }} ($(LANG=en;date +'%d %b %Y'))"
#verstr="Version ${{ vars.short_new_release }} ($(LANG=en;date +'%d %b %Y'))"
#echo "$verstr"
#verstrlen=${#verstr} | tee -a "${FILE}"
#echo "$verstrlen"
@ -94,7 +73,7 @@ jobs:
#echo "$linestr" | tee -a "${FILE}"
echo "Summary" | tee -a "${FILE}"
echo -e "~~~~~~~\n" | tee -a "${FILE}"
echo "Description of the new version, explaining the main features and bugs " | tee -a "${FILE}"
echo "Description of the new version ${{ vars.short_new_release }}, explaining the main features and bugs " | tee -a "${FILE}"
echo "included. It's usually similar to the piece of news that will be published " | tee -a "${FILE}"
echo "on the website. " | tee -a "${FILE}"
echo "" | tee -a "${FILE}"
@ -109,9 +88,9 @@ jobs:
# yes, I know about HERE files. I just don't like them!
echo "Notes" | tee -a "${FILE}"
echo -e "~~~~~\n\n" | tee -a "${FILE}"
echo "If you are upgrading from ${{ inputs.short-last-release }} without using a " | tee -a "${FILE}"
echo "If you are upgrading from ${{ vars.short_last_release }} without using a " | tee -a "${FILE}"
echo "package, you will need to manually execute on your database the SQL commands" | tee -a "${FILE}"
echo "from file: 'scripts/database/upgrade_${{ inputs.short-new-release }}.sql'." | tee -a "${FILE}"
echo "from file: 'scripts/database/upgrade_${{ vars.short_new_release }}.sql'." | tee -a "${FILE}"
echo "" | tee -a "${FILE}"
echo "If you are upgrading from earlier version without using the Debian package," | tee -a "${FILE}"
echo "you will need to manually execute on your database the SQL commands from all files." | tee -a "${FILE}"
@ -125,8 +104,8 @@ jobs:
echo "Contributors" | tee -a "${FILE}"
echo -e "~~~~~~~~~~~~\n" | tee -a "${FILE}"
echo -e "A big thanks to all the contributors to version ${{ inputs.short-new-release }} :\n" | tee -a "${FILE}"
#git shortlog -ns "${{ steps.set-release-tag.outputs.last-release-tag }}.." | cut -f2- | sed -e 's/^/* /' | sort -u | tee -a "${FILE}"
git shortlog -ns "${{ inputs.full-last-release }}.." | cut -f2- | sed -e 's/^/* /' | sort -u | tee -a "${FILE}"
#git shortlog -ns "${{ vars.full_last_release }}.." | cut -f2- | sed -e 's/^/* /' | sort -u | tee -a "${FILE}"
git shortlog -ns "${{ vars.full_last_release }}.." | cut -f2- | sed -e 's/^/* /' | sort -u | tee -a "${FILE}"
echo "" | tee -a "${FILE}"
- name: Start writing NEWS.rst.translators for NEWS.
@ -152,8 +131,7 @@ jobs:
echo "Writing to file: $FILE"
echo "Changes" | tee -a "${FILE}"
echo -e "~~~~~~~\n" | tee -a "${FILE}"
#git log --pretty="* %s" "${{ steps.set-release-tag.outputs.last-release-tag }}.." | tee -a "${FILE}"
git log --pretty="* %s" "${{ inputs.full-last-release }}.." | tee -a "${FILE}"
git log --pretty="* %s" "${{ vars.full_last_release }}.." | tee -a "${FILE}"
# Concatenate the files
- name: Generate resulting NEWS.rst.part
@ -185,7 +163,7 @@ jobs:
- name: Upload LATEST-NEWS.rst
uses: actions/upload-artifact@v3
with:
name: LATEST-NEWS.rst
name: LATEST-NEWS-for-${{ vars.short_new_release }}.rst
path: LATEST-NEWS.rst
compression-level: 0 # no compression
retention-days: 3

55
.forgejo/workflows/ubuntu_24.04.yml
View file

@ -4,10 +4,19 @@
name: Ubuntu 24.04 (Noble Numbat)
on:
workflow_dispatch:
inputs:
create-release:
description: 'Create a release from this build?'
default: false
type: boolean
push:
branches: [ "main" ]
pull_request:
branches: [ "main" ]
# branches: [ "main" ]
tags:
- "v*"
# pull_request:
# branches: [ "main" ]
# Prevent cancellation of already running actions when pushing concurrently. Test
concurrency:
@ -39,9 +48,14 @@ jobs:
steps:
# Define job-id
- name: actions prep
id: artifact-upload-prep
run: echo ::set-output name=artifact-id::$(git rev-parse --short=10 HEAD)
- name: Debug info
run: |
cat <<'EOF'
${{ toJSON(forgejo) }}
EOF
- name: My current run-id
run: echo "${{ forgejo.run_id }}"
- name: install needed software
run: |
@ -51,6 +65,12 @@ jobs:
- name: Checkout code
uses: actions/checkout@v3
# This is only possible AFTER the checkout step.
- name: actions prep
id: artifact-upload-prep
run: echo "artifact-id=$(git rev-parse --short=10 HEAD)" >> "$GITHUB_OUTPUT"
# Let's test caching maven stuff.
- name: Cache Maven repository
uses: actions/cache@v4
@ -96,18 +116,37 @@ jobs:
shell: bash
run: mvn clean install --no-transfer-progress -Ddefault.passwordsControl=false -Ddefault.exampleUsersDisabled=false -DdataSource.url=jdbc:postgresql://postgres:5432/libreplandev -DdataSource.user=libreplan -DdataSource.password=libreplan -DjdbcDriver.className=org.postgresql.Driver
- name: Copy libreplan-webapp.war to libreplan.war
run: cp libreplan-webapp/target/libreplan-webapp.war libreplan-webapp/target/libreplan.war
# Upload the result
- name: Upload libreplan.war attempt 1
uses: actions/upload-artifact@v3
with:
name: libreplan.war
path: libreplan-webapp/target/libreplan-webapp.war
path: libreplan-webapp/target/libreplan.war
retention-days: 3
- name: Upload libreplan.war with git hash
uses: actions/upload-artifact@v3
with:
name: libreplan-${{ steps.artifact-upload-prep.outputs.artifact-id }}.war
path: libreplan-webapp/target/libreplan-webapp.war
path: libreplan-webapp/target/libreplan.war
retention-days: 3
- name: Prep release dir for upload
run: |
mkdir -p dist/release
cp libreplan-webapp/target/libreplan.war dist/release/libreplan.war
# Also store as a release.
- uses: actions/forgejo-release@v2.7.3
with:
direction: upload
url: http://10.1.1.158:3000
repo: jeroen/libreplan
token: ${{ secrets.WRITE_TOKEN_TO_MYREPO }}
tag: v2025
release-dir: dist/release
release-notes: "No RELEASE NOTES"
verbose: true

50
doc/src/technical/howto-setup-forgejo.txt Normal file
View file

@ -0,0 +1,50 @@
How to setup Forgejo
Forgo is a git repository tool that is completely open source and
contains powerful CI/CD capabilities in the form of 'actions' defined in
'workflow' files.
Those are located in the .forgejo/workflows directory.
Why Forgejo? Well, because it is completely open source and currently has
the best chance that time invested is not wasted because features become
closed or behind a paywall.
Currently a lot of work is being done to ease the compilation and packaging
of LibrePlan.
First result was a simple workflow to build the current LibrePlan repo.
Next step was creating a workflow that would generate the NEWS.rst file
that is needed during packaging. At first that needed user input in the
form of strings describing version numbering (old and new), but I realised
it would be much easier if I store those in repository environment
variables because they do not change very often.
So, the repository needs the follwing environment variables:
full_last_release:
description: 'What was the full name of the last release'
default: "libreplan-"
required: true
type: string
short_last_release:
description: 'What was the short name of the last release'
default: "1.4.1"
required: true
type: string
full_new_release:
description: 'What is the full name of the new release'
default: "libreplan-1.5.0"
required: true
type: string
short_new_release:
description: 'What is the short name of the new release'
default: "1.5.0"
required: true
type: string
You can define those if you go to your repository settings page,
and under 'actions' create them in the 'variables' section.
One might think that it would be possible to generate the short name from the long name,
but it is very likely the release name is going to change.