Showing posts with label server management. Show all posts
Showing posts with label server management. Show all posts

Saturday, January 31, 2026

Hetzner VPS initial configuration notes

Having just configured the Oracle cloud server, and having used DigitalOcean for years, Hetzner's interface for initial configuration is closer to DigitalOcean than Oracle cloud. And the price of $3.50/month for 2 CPUs and 4GB of RAM with 40GB of storage is mindblowing compared to DigitalOcean's 2 CPUs with 1 GB of RAM and 25GB of storage for $6/month.

Configuration and management interface is at https://console.hetzner.com/

Create user

I logged into the server using

  $ ssh -i ~/.ssh/KEYNAMEHERE root@SERVERIPHERE
  
and then create a new user and group
# sudo adduser pdg
# sudo adduser pdg sudo
# sudo addgroup pdg_grp
# sudo usermod -aG pdg_grp pdg

The -a (append) and -G (groups) options ensure the user is added to the new group without being removed from their other groups.

Validating,

# groups pdg
pdg : pdg sudo users pdg_grp
# id pdg
uid=1000(pdg) gid=1000(pdg) groups=1000(pdg),27(sudo),100(users),1001(pdg_grp)

SSH

Next, disable root from logging in via SSH. First, copy my public cert from my laptop to the server
$ ssh-copy-id pdg@SERVERIPHERE
On the server I exited the root SSH session and logged back in as pdg:
$ ssh -i ~/.ssh/KEYNAMEHERE pdg@SERVERIPHERE
$ sudo vi /etc/ssh/sshd_config
     PermitRootLogin no
     PubkeyAuthentication yes
     PasswordAuthentication no
     PermitEmptyPasswords no
     AuthorizedKeysFile      .ssh/authorized_keys .ssh/authorized_keys2

$ mkdir ~/.ssh
$ chmod 700 ~/.ssh
$ cat ~/.ssh/authorized_keys
$ sudo systemctl daemon-reload
$ sudo systemctl restart ssh.socket

Updating bash configuration

Added to ~/.bash_aliases
lias vi='vim'

alias s='git status'
alias p='git push'

# do not overwrite existing files
set -o noclobber

alias nothing='docker ps; git pull; git push; git status'
alias ll="ls -hal"
alias ..="cd .."
alias grin="grep -R -i -n --color"
and to ~/.bashrc
# HISTSIZE determines the number of commands remembered in memory during the current session.
# HISTFILESIZE determines the maximum number of lines allowed in the history file
export HISTSIZE=100000
export HISTFILESIZE=200000

export HISTTIMEFORMAT="%h %d %H:%M:%S "

shopt -s histappend
shopt -s cmdhist

UFW = Uncomplicated Firewall

  • 104.210.140.141 = OpenAI, observed 2025-12-04
  • 114.119.147.137 = PetalBot (for Huawei), observed 2025-12-04
  • 156.59.198.136 = bytedance, observed 2025-12-04
Best practice for order of firewall rules is:
  1. Specific DENY rules (blocking known bad actors).
  2. Specific ALLOW rules (allowing trusted hosts/networks).
  3. General ALLOW rules (allowing public services).
  4. General DENY rules (the default policy, often implied).
$ sudo ufw status verbose
Status: inactive
$ sudo nft list ruleset
add rules
$ sudo ufw deny from 156.59.198.136/24
$ sudo ufw deny from 114.119.147.0/24
$ sudo ufw deny from 104.210.140.0/24
$ sudo ufw allow ssh
$ sudo ufw allow 443
$ sudo ufw allow 80
Now I have
$ sudo ufw status numbered
Status: active

     To                         Action      From
     --                         ------      ----
[ 1] Anywhere                   DENY IN     156.59.198.0/24           
[ 2] Anywhere                   DENY IN     114.119.147.0/24          
[ 3] Anywhere                   DENY IN     104.210.140.0/24          
[ 4] 22/tcp                     ALLOW IN    Anywhere                  
[ 5] 443                        ALLOW IN    Anywhere                  
[ 6] 80                         ALLOW IN    Anywhere                  
[ 7] 22/tcp (v6)                ALLOW IN    Anywhere (v6)             
[ 8] 443 (v6)                   ALLOW IN    Anywhere (v6)             
[ 9] 80 (v6)                    ALLOW IN    Anywhere (v6)     

Update OS

Running Ubuntu 24.04.3 LTS, default load is
 System information as of Sat Jan 31 02:24:09 AM UTC 2026
  System load:  0.04              Processes:             123
  Usage of /:   3.0% of 37.23GB   Users logged in:       1
  Memory usage: 5%                IPv4 address for eth0: 65.21.252.29
  Swap usage:   0%                IPv6 address for eth0: 2a01:4f9:c013:5897::1
  
$ sudo apt update
$ sudo apt upgrade

Install Docker

Docker
$ sudo apt update
$ sudo apt install apt-transport-https curl
$ curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
$ echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
$ sudo apt update
$ sudo apt install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
$ sudo systemctl is-active docker
active
as per https://linuxiac.com/how-to-install-docker-on-ubuntu-24-04-lts/ Then add pdg to docker group
$ sudo usermod -a -G docker pdg
as per https://stackoverflow.com/questions/47854463/docker-got-permission-denied-while-trying-to-connect-to-the-docker-daemon-socke/48450294#48450294
$ sudo apt -y install make

Enable Upload to GitHub

$ ssh-keygen
$ cat id_ed25519.pub
Upload public key to my profile, https://github.com/settings/keys

Clone git repo

On the server
$ git clone [email protected]:allofphysicsgraph/ui_v8_website_flask_neo4j.git
$ cd /home/pdg/ui_v8_website_flask_neo4j
On my laptop
$ scp -i ~/.ssh/KEYNAMEHERE neo4j_pdg/plugins/apoc.jar pdg@SERVERIPHERE:/home/pdg
On the server
cd /home/pdg/ui_v8_website_flask_neo4j
echo "UID=$(id -u)" > .env
echo "GID=$(id -g)" >> .env

Build containers

$ cd /home/pdg/ui_v8_website_flask_neo4j
$ make container_build
Which results in
$ docker images
IMAGE                                ID             DISK USAGE   CONTENT SIZE   EXTRA
ui_v8_flask_webserver:latest-amd64   68046aa182c6       3.61GB          872MB       
From my laptop I copy the secrets file
$ scp -i ~/.ssh/KEYNAMEHERE .env.google pdg@SERVERIPHERE:/home/pdg/ui_v8_website_flask_neo4j
On the server I have a few changes to enact,
cd /home/pdg/ui_v8_website_flask_neo4j
mkdir logs
cd certs
openssl genrsa > privkey.pem
openssl req -new -x509 -key privkey.pem > fullchain.pem
openssl dhparam -out dhparam.pem 2048
I then tried running the server using
cd /home/pdg/ui_v8_website_flask_neo4j
make launch_webserver
but encountered some permissions issues that were fixed using
cd /home/pdg/ui_v8_website_flask_neo4j
sudo chown -R pdg:pdg neo4j_pdg/
sudo chown -R pdg:pdg dumping_grounds/
I was then able to successfully run
$ make launch_webserver COMPOSE_FLAGS=--detach
The webserver is then running at SERVERIPHERE on the internet!

DNS

Log into https://account.squarespace.com/domains 
On the page https://account.squarespace.com/domains/managed/allofphysics.com/dns/domain-nameservers change the Name Servers to Hetzner's:
  • helium.ns.hetzner.com
  • hydrogen.ns.hetzner.com
  • oxygen.ns.hetzner.com

On the Hetzner console webpage, select "DNS" and then "Add DNS zone"

Set the domain (allofphysics.com) and "Create an Empty zone." (The other options are "Import Zone file" and "Secondary".)

On DigitalOcean's DNS console, change TTL to 300 for each of the domains.

Since the nameservers are ns1.digitalocean.com, then I will need to change the records in the DigitalOcean Control Panel.

In DigitalOcean's Control Panel, set the "A" records to point to Hetzner's SERVERIPHERE and "NS" to point to Hetzner's

In Hetzner's panel, set "@" and "www" and "*" to Hetzner's SERVERIPHERE.

Certbot

In Hetzner's DNS panel, added TXT record "_acme-challenge" with TTL 300. The "value" will later be provided by certbot

Followed instructions from https://certbot.eff.org/instructions?ws=nginx&os=pip

cd /home/pdg/
sudo apt update
sudo apt upgrade
sudo apt install python3 python3-dev python3-venv libaugeas-dev gcc
sudo python3 -m venv /opt/certbot/
sudo /opt/certbot/bin/pip install --upgrade pip
sudo /opt/certbot/bin/pip install certbot certbot-nginx
sudo ln -s /opt/certbot/bin/certbot /usr/local/bin/certbot
Since my nginx isn't running baremetal, I can't use the recommended sudo certbot certonly --nginx
Instead, I used
sudo certbot certonly --manual --preferred-challenges dns \
--server https://acme-v02.api.letsencrypt.org/directory \
-d derivationmap.net -d www.derivationmap.net \
-d allofphysics.com -d www.allofphysics.com

Saving debug log to /var/log/letsencrypt/letsencrypt.log
Requesting a certificate for derivationmap.net and 3 more
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Please deploy a DNS TXT record under the name:

_acme-challenge.allofphysics.com.

with the following value:

ARANDOMLOOKINGSTRINGHERE

Before continuing, verify the TXT record has been deployed. Depending on the DNS
provider, this may take some time, from a few seconds to multiple minutes. You can
check if it has finished deploying with aid of online tools, such as the Google
Admin Toolbox: https://toolbox.googleapps.com/apps/dig/#TXT/_acme-challenge.allofphysics.com.
Look for one or more bolded line(s) below the line ';ANSWER'. It should show the
value(s) you've just added.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Press Enter to Continue

Successfully received certificate.
Certificate is saved at: /etc/letsencrypt/live/derivationmap.net/fullchain.pem
Key is saved at:         /etc/letsencrypt/live/derivationmap.net/privkey.pem
This certificate expires on 2026-05-01.
These files will be updated when the certificate renews.

NEXT STEPS:
- This certificate will not be renewed automatically. 
Autorenewal of --manual certificates requires the use of 
an authentication hook script (--manual-auth-hook) but one was not provided. 
To renew this certificate, repeat this same certbot command 
before the certificate's expiry date.

The Google Admin Toolbox page (https://toolbox.googleapps.com/apps/dig/#TXT/_acme-challenge.allofphysics.com) didn't pick up the TXT record, but nslookup did:

$ nslookup -q=TXT _acme-challenge.allofphysics.com hydrogen.ns.hetzner.com
Server:		hydrogen.ns.hetzner.com
Address:	213.133.100.98#53

_acme-challenge.allofphysics.com	text = "ARANDOMLOOKINGSTRINGHERE"

Now https://allofphysics.com/ looks correct and has no errors! (Same for derivationmap.net)

The renewal instructions on https://physicsderivationgraph.blogspot.com/2021/10/periodic-renewal-of-https-letsencrypt.html should be valid when these certs expire?

sudo certbot certonly --webroot \
-w /home/pdg/ui_v8_website_flask_neo4j/certs \
--server https://acme-v02.api.letsencrypt.org/directory \
-d derivationmap.net -d www.derivationmap.net \
-d allofphysics.com -d www.allofphysics.com

Saturday, January 24, 2026

VPS price comparison September 2024

Other posts in this series:

Netcup

https://www.netcup.com/en

German company. Has US hosting capability.

as of 2024-09-02, lowest cost relevant configuration: "VPS 250 G11s"

3.35 eur is currently $3.71 usd
2 vCore (x86)
2 GB ECC RAM
64 GB SSD
Traffic included
  

Unfortunately, the US-based servers are only available for beefy configurations. The "VPS 250 G11s" is hosted in Germany; see https://www.netcup.com/en/server/vps

Namecheap

https://www.namecheap.com/hosting/shared/

Is not a VPS. Provides disk space and pre-configured web services. Not suitable for running Docker.

hetzner

https://www.hetzner.com/cloud/

Servers hosted in Germany or Finland

CX22 is

2 vCPU
4GB RAM
40 GB SSD

for 4.51 eur ($5 USD) per month

AWS S3

AWS S3 is suitable for static sites only. Dynamic content would need to live in lambdas for on-demand execution.

As per https://www.reddit.com/r/aws/comments/mfbgot/is_aws_a_good_alternative_to_a_20mo_vps_for/

AWS LightSail

AWS VPS is "LightSail"

https://calculator.aws/#/createCalculator/Lightsail

The configuration "Bundle:2GB" features

Storage: 60GB
vCPU: 2
Memory: 2GB
Data Transfer Quota: 3TB
and costs $11.77/month

Google Cloud (GCP)

https://cloud.google.com/compute/vm-instance-pricing#general-purpose_machine_type_family

"e2-standard-2"

2 vCPUs
8GB RAM
for $55.08726/month in Virgina, or $48/month in Iowa. Cheaper if you sign up for 1 or 3 years.

linode

https://www.linode.com/pricing/ as of 2024-11-29 offers

1GB RAM, 25GB storage for $5/month
2GB RAM, 50GB storage for $12/month

ioflood

https://ioflood.com/bare-metal-cloud-server.php

Dual Intel Xeon E5-2695v4 CPUs
36 CPU cores
64GB RAM
960GB NVMe SSD
100TB @ 10Gbps (Inbound unmetered)
for $99/mo monthly

https://manage.ioflood.com/orderform/index.php?form=Max-95v4-September-2024

VPS price comparison January 2026

Other posts in this series:

See hhttps://github.com/allofphysicsgraph/task-tracker/issues/56

DigitalOcean

Currently (as of 2026-01-24) paying $6/month for 2 CPUs, 1GB of RAM, 

Was paying $12/month for 2 CPUs, 2GB of RAM (to support Neo4j)

Hetzner


"CPX11" in US host is 2 vCPUs, 2GB of RAM, 40GB SSD for 5 euros. Doesn't include VAT

Oracle

https://www.oracle.com/cloud/compute/arm/pricing/

https://www.oracle.com/cloud/free/

"AMD Compute Instance" has 1/8 OCPU and 1 GB memory

"Arm Compute Instance" has 4 Arm-based Ampere A1 cores and 24 GB of memory usable as 1 VM or up to 4 VMs Always Free 3,000 OCPU hours and 18,000 GB hours per month

I attempted to use Oracle (see https://physicsderivationgraph.blogspot.com/2026/01/oracle-cloud-configuration-for-free-tier.html) but their free tier didn't have any available instances.

Google Compute Engine


"E2" with 2vCPUs and 2GB of RAM is $37.11/month

https://docs.cloud.google.com/free/docs/free-cloud-features says "1 non-preemptible e2-micro VM instance per month in one of the following US regions: Oregon: us-west1. Iowa: us-central1. South Carolina: us-east1. 30 GB-months standard persistent disk. 1 GB of outbound data transfer from North America to all region destinations (excluding China and Australia) per month."

The e2-micro is 0.25 CPUs and 1GB of RAM.

Microsoft Azure

B2pts v2 = 1GB of RAM, 2 vCPUs for $6.13/month
next step up is 
B2pls v2 = 4GB of RAM, 2 vCPUs for $24.5280/month

$3.8909 per GB/month

AWS Lightsail

"Bundles resources like memory, vCPU, and solid-state drive (SSD) storage into one plan, so budgeting is easy." 
aka AWS simplified using pre-set configurations.

$12/month for 2 GB Memory 2 vCPUs 60 GB SSD Disk 3 TB Transfer

AWS

https://aws.amazon.com/free/
https://aws.amazon.com/free/free-tier-faqs/

2026-01-28: "Free plan eligible instances include: T3.micro, T3.small, T4g.micro, T4g.small, C7i-flex.large, M7i-flex.large" (sourcehttps://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-free-tier-usage.html)

"The free plan allows you to experiment with AWS services and build proof-of-concepts at no cost for up to 6 months until you upgrade to a paid plan."

Saturday, April 22, 2023

intermittent Internal Server Error on webpage - activity log of diagnosis

My login to the server provides an overview,

Welcome to Ubuntu 20.04.2 LTS (GNU/Linux 5.4.0-45-generic x86_64)

  System information as of Sat Apr 22 23:49:00 UTC 2023

  System load:                      0.09
  Usage of /:                       70.8% of 24.06GB
  Memory usage:                     72%
  Swap usage:                       0%
  Processes:                        131
  Users logged in:                  1

I first confirmed disk utilization.

$ df -h
Filesystem      Size  Used Avail Use% Mounted on
udev            474M     0  474M   0% /dev
tmpfs            99M  1.2M   97M   2% /run
/dev/vda1        25G   18G  7.1G  71% /

I next looked at the error logs. When the error occurred, the fault trace ended with

  File "/home/appuser/app/controller.py", line 3424, in review_derivation
    dat = clib.read_db(path_to_db)
  File "/home/appuser/app/common_lib.py", line 142, in read_db
    for row in cur.execute("SELECT * FROM data"):
sqlite3.OperationalError: no such table: data
  
Probably a race condition in the Python? If so, this problem is due to my hack of writing the database to SQL. My motivation to resolve this issue is low because I had intended to switch to Neo4j. I haven't complete the migration from JSON/SQL to Neo4j yet.

I used top and saw 2 and 3 instances of gunicorn

I checked for redundant Docker containers running but saw only what's expected:

$ docker ps
CONTAINER ID   IMAGE                           COMMAND                  CREATED       STATUS       PORTS                                                                      NAMES
632ee101f714   v7_pickle_web_interface_nginx   "nginx -g 'daemon of…"   4 weeks ago   Up 4 weeks   0.0.0.0:80->80/tcp, :::80->80/tcp, 0.0.0.0:443->443/tcp, :::443->443/tcp   v7_pickle_web_interface_nginx_1
80bb13ec66ca   v7_pickle_web_interface_flask   "gunicorn --reload -…"   4 weeks ago   Up 4 weeks   5000/tcp                                                                   v7_pickle_web_interface_flask_1

Friday, September 4, 2020

ufw allow status numbered delete -- managing the firewall

$ sudo ufw status verbose
Status: active
Logging: on (low)
Default: deny (incoming), allow (outgoing), deny (routed)
New profiles: skip

To                         Action      From
--                         ------      ----
22/tcp                     LIMIT IN    Anywhere                  
443/tcp                    ALLOW IN    Anywhere                  
80/tcp                     ALLOW IN    Anywhere                  
25                         ALLOW IN    Anywhere                  
22/tcp (v6)                LIMIT IN    Anywhere (v6)             
443/tcp (v6)               ALLOW IN    Anywhere (v6)             
80/tcp (v6)                ALLOW IN    Anywhere (v6)             
25 (v6)                    ALLOW IN    Anywhere (v6)             


$ sudo ufw allow 8000/tcp
Rule added
Rule added (v6)


$ sudo  ufw status numbered
Status: active

     To                         Action      From
     --                         ------      ----
[ 1] 22/tcp                     LIMIT IN    Anywhere     
[ 2] 443/tcp                    ALLOW IN    Anywhere  
[ 3] 80/tcp                     ALLOW IN    Anywhere  
[ 4] 22/tcp (v6)                LIMIT IN    Anywhere (v6) 
[ 5] 443/tcp (v6)               ALLOW IN    Anywhere (v6) 
[ 6] 80/tcp (v6)                ALLOW IN    Anywhere (v6) 
[ 7] 25 (v6)                    ALLOW IN    Anywhere (v6) 

$ sudo ufw delete 7
Deleting:
 allow 25
Proceed with operation (y|n)? y

Rule deleted (v6)