Skip to content

Comments

cronjobs: expand script to run stand-alone#1215

Merged
neteler merged 16 commits intoOSGeo:grass8from
neteler:cron_standalone_scripts
Nov 12, 2024
Merged

cronjobs: expand script to run stand-alone#1215
neteler merged 16 commits intoOSGeo:grass8from
neteler:cron_standalone_scripts

Conversation

@neteler
Copy link
Member

@neteler neteler commented Sep 29, 2024

With this PR instructions to set up the source code repo have been turned into executable script chunks. Hence, if the scripts are executed on a new server, the GitHub repo cloning is done as well as needed directories are created.

Furthermore, some minor script tuning.

With this PR instructions to set up the source code repo have been turned into executable script chunks.
Hence, if the scripts are executed on a new server, the GitHub repo cloning is done as well as needed directories are created.

Furthermore, some minor script tuning.
@neteler neteler added the enhancement New feature or request label Sep 29, 2024
@neteler neteler requested a review from wenzeslaus September 29, 2024 21:01
@neteler
Copy link
Member Author

neteler commented Sep 29, 2024

FYI: Installation of dependencies on grass.osgeo.org:

ssh neteler@grasslxd:~$
cd $HOME/src/main/
git pull
sudo apt install $(cat .github/workflows/apt.txt)

Due to sudo this cannot be automated, though.

Copy link
Member

@wenzeslaus wenzeslaus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for looking into this. I didn't get far in a Docker container (just guessing what the setup needs to be):

docker run -it ubuntu bash

apt update
apt install git
git clone https://github.com/neteler/grass-addons.git
cd grass-addons/
git switch cron_standalone_scripts
cd ..
git clone https://github.com/OSGeo/grass
apt install $(cat grass/.github/workflows/apt.txt)
grass-addons/utils/cronjobs_osgeo_lxd/cron_grass_current_stable_build_binaries.sh

grass-addons/utils/cronjobs_osgeo_lxd/cron_grass_current_stable_build_binaries.sh: 141: cd: can't cd to /home//src//releasebranch_8_4/

Perhaps, you can try to get just one of these run in a Docker container and, if that still runs on the lxd server, you can then sync the other ones too.

@neteler
Copy link
Member Author

neteler commented Sep 30, 2024

... cd: can't cd to /home//src//releasebranch_8_4/

Should be addressed in 250c4a8

I got this running:

# adapt as needed, path to cronjob scripts in addons repo
cd ~/software/grass-addons/utils/cronjobs_osgeo_lxd/

# using ubuntu:22.04 as ubuntu is still missing PDAL
docker run -it --volume="$(pwd)/:/data" ubuntu:22.04 bash
apt update -y && apt install git gettext python3-sphinx doxygen graphviz zip -y

USER=`id -u -n`
MAINDIR=/home/$USER
SOURCE=$MAINDIR/src/
mkdir -p $SOURCE $MAINDIR/cronjobs/
cd $SOURCE

# GRASS GIS addons: get a shallow clone into docker container
cd $SOURCE
git clone --depth=1 https://github.com/OSGeo/grass.git grass8-addons
# add links to cronjob scripts 
(cd $MAINDIR/cronjobs/ && ln -s $SOURCE/grass8-addons/utils/cronjobs_osgeo_lxd/*.sh .)
(cd $MAINDIR/cronjobs/ && ln -s $SOURCE/grass8-addons/utils/cronjobs_osgeo_lxd/*.py .)

# needed for script runs in docker, to avoid major path complexity in scripts
ln -s /home/root/src /root/src

# GRASS GIS release branches: get a shallow clone into docker container
for REPO in releasebranch_7_8 releasebranch_8_3 releasebranch_8_4 ; do
 git clone -b $REPO --single-branch --depth=1 https://github.com/OSGeo/grass.git $REPO
done

# GRASS GIS main: get a shallow clone into docker container
BRANCH=main
git clone --single-branch --depth=1 https://github.com/OSGeo/grass.git $BRANCH

# install dependencies
apt install $(cat $SOURCE/$BRANCH/.github/workflows/apt.txt) -y

# define python3 = python
update-alternatives --install /usr/bin/python python /usr/bin/python3 1

cd /data/

# run all cronjob scripts
bash cron_grass_current_stable_build_binaries.sh && bash cron_grass_current_stable_src_snapshot.sh &&  bash cron_grass_legacy_build_binaries.sh &&  bash cron_grass_legacy_src_snapshot.sh && bash cron_grass_old_build_binaries.sh &&  bash cron_grass_old_src_snapshot.sh &&  bash cron_grass_preview_build_binaries.sh &&  bash cron_grass_preview_src_snapshot.sh

@neteler neteler requested a review from wenzeslaus October 7, 2024 13:19
@neteler
Copy link
Member Author

neteler commented Oct 7, 2024

Instructions added to README.md in 0daf05f

@neteler neteler self-assigned this Oct 23, 2024
@neteler
Copy link
Member Author

neteler commented Oct 23, 2024

@wenzeslaus any objections to merge this PR?
I have a new one on top which better reflects the Python strategy of publishing multiple manual pages which I can only submit once this PR is merged.

@neteler neteler requested a review from wenzeslaus November 1, 2024 21:56
Copy link
Member

@echoix echoix left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At one point we need to iterate and improve each time.

Copy link
Member

@wenzeslaus wenzeslaus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The changes look okay. I was not able to run it at this moment due to connection issues with the Ubuntu package archives.

Co-authored-by: Vaclav Petras <[email protected]>
@neteler neteler merged commit fe58963 into OSGeo:grass8 Nov 12, 2024
@neteler neteler deleted the cron_standalone_scripts branch November 12, 2024 22:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants