README.md
1
2
3
# KornShell 93u+m
5
Welcome to the repository where the KornShell is under active development.
6
This is where we develop bugfixes and new features for the shell, and where
7
users can download the latest releases or the current development version in
8
source code form.
9
The project started off from last stable release (93u+ 2012-08-01) of
10
[ksh93](http://www.kornshell.com/),
11
formerly developed by AT&T Software Technology (AST).
12
The sources in this repository were forked from the
13
GitHub [AST repository](https://github.com/att/ast)
14
which is no longer under active development.
16
For user-visible fixes, see [NEWS](https://github.com/ksh93/ksh/blame/dev/NEWS)
17
and click on commit messages for full details.
18
For all fixes, see [the commit log](https://github.com/ksh93/ksh/commits/).
19
To see what's left to fix, see [the issue tracker](https://github.com/ksh93/ksh/issues).
21
## Table of contents ##
22
23
* [Policy](#user-content-policy)
24
* [Why?](#user-content-why)
25
* [Installing from source](#user-content-installing-from-source)
26
* [Supported systems](#user-content-supported-systems)
27
* [Prepare](#user-content-prepare)
28
* [Build](#user-content-build)
29
* [Test](#user-content-test)
30
* [Install](#user-content-install)
31
* [What is ksh93?](#user-content-what-is-ksh93)
32
33
## Policy
35
1. Feature development for future releases happens on the dev branch.
36
The numbered release branch(es) are feature-frozen and get bugfixes
37
and maintenance only, usually cherry-picked from the dev branch.
38
2. No major rewrites. No refactoring code that is not fully understood.
39
Even gradual and careful development may culminate in profound changes.
40
Bit rot is prevented by cleaning up unused and obsolete code.
41
3. Maintain documented behaviour. Changes required for compliance with the
42
[POSIX shell language standard](https://pubs.opengroup.org/onlinepubs/9699919799/utilities/contents.html)
43
are implemented for the `posix` mode only to avoid breaking legacy scripts.
44
4. No 100% bug compatibility. Broken and undocumented behaviour gets fixed.
45
5. No bureaucracy, no formalities. Just fix it, or report it: create issues,
46
send pull requests. Every interested party is invited to contribute.
47
6. To help increase everyone's understanding of this code base, fixes and
48
significant changes should be fully documented in commit messages.
49
Each commit should be a complete, self-contained and self-documenting
50
change, including updates to documentation and regression tests where
51
applicable. Pull requests are therefore squashed into a single commit.
52
7. Code style varies somewhat in this historic code base.
53
Your changes should match the style of the code surrounding them.
54
Indent with tabs, assuming an 8-space tab width.
55
Opening braces are on a line of their own, at the same indentation level
56
as their corresponding closing brace.
57
Comments always use `/*`...`*/`.
58
8. Good judgment may override this policy.
60
## Why?
61
62
Between 2017 and 2020 there was an ultimately unsuccessful
63
[attempt](https://github.com/att/ast/tree/2020.0.1)
64
to breathe new life into the KornShell by extensively refactoring the last
65
unstable AST beta version (93v-).
66
While that ksh2020 effort is now abandoned and still has many critical bugs,
67
it also had a lot of bugs fixed. More importantly, the AST issue tracker
68
now contains a lot of documentation on how to fix those bugs, which made
69
it possible to backport many of them to the last stable release instead.
70
This ksh 93u+m reboot now incorporates many of these bugfixes,
71
plus patches from
72
[OpenSUSE](https://github.com/ksh93/ksh/wiki/Patch-Upstream-Report:-OpenSUSE),
73
[Red Hat](https://github.com/ksh93/ksh/wiki/Patch-Upstream-Report:-Red-Hat),
74
and
75
[Solaris](https://github.com/ksh93/ksh/wiki/Patch-Upstream-Report:-Solaris),
76
as well as many new fixes from the community
77
([1](https://github.com/ksh93/ksh/pulls?q=is%3Apr+is%3Amerged),
78
[2](https://github.com/ksh93/ksh/issues?q=is%3Aissue+is%3Aclosed+label%3Abug)).
79
Though there are many
80
[bugs left to fix](https://github.com/ksh93/ksh/issues),
81
we are confident at this point that 93u+m is already the least buggy version
82
of ksh93 ever released.
83
As of late 2021, distributions such as Debian and Slackware have begun
84
to package it as their default version of ksh93.
86
## Installing from source
87
88
You can download a [release](releases) tarball,
89
or clone the current code from your preferred branch.
90
New features for the future release series are developed on the `dev` branch.
91
Stable releases are currently based on the `1.0` branch.
92
93
### Supported systems
94
95
KornShell 93u+m is currently known to build and run on:
96
* Android/Termux
97
* Cygwin
98
* DragonFly BSD
99
* FreeBSD
100
* Haiku
101
* illumos distributions (e.g., OmniOS)
102
* Linux: all distributions with glibc or musl libc
103
* macOS
104
* NetBSD
105
* OpenBSD
106
* QNX Neutrino (6.5.0)
107
* Solaris
108
109
Systems that may work, but that we have not been able to test lately, include:
110
* AIX
111
* HP-UX
112
* UnixWare
113
114
KornShell 93u+m supports systems that use the ASCII character set as the
115
lowest common denominator. This includes Linux on IBM zSeries, but not z/OS.
116
Support for the EBCDIC character set has been removed, as we do not have
117
access to a mainframe with z/OS to test and maintain it.
118
119
### Prepare
120
121
The build system requires only a basic POSIX-compatible shell, utilities and
122
compiler environment. The `cc`, `ar` and `getconf` commands are needed at
123
build time. The `tput` and `getconf` commands are used at runtime if
124
available (for multiline editing and to complete the `getconf` built-in,
125
respectively). Not all systems come with all of these preinstalled. Here are
126
system-specific instructions for making them available:
127
128
* **Android/[Termux](https://termux.dev/):**
129
install dependencies using `pkg install`.
130
* Build dependencies: `clang`, `binutils`, `getconf`
131
* Runtime dependencies (optional): `ncurses-utils`, `getconf`
132
* **macOS:**
133
install the Xcode Command Line Tools:
134
`xcode-select --install`
135
* (to be completed)
136
137
### Build
139
To build ksh with a custom configuration of features, edit
140
[`src/cmd/ksh93/SHOPT.sh`](https://github.com/ksh93/ksh/blob/dev/src/cmd/ksh93/SHOPT.sh).
142
On systems such as NetBSD and OpenBSD, where `/bin/ksh` is not ksh93 and the
143
preinstalled `/etc/ksh.kshrc` profile script is incompatible with ksh93, you'll
144
want to disable `SHOPT_SYSRC` to avoid loading it on startup -- unless you can
145
edit it to make it compatible with ksh93. This generally involves differences
146
in the declaration and usage of local variables in functions.
147
148
Then `cd` to the top directory and run:
151
bin/package make
152
```
154
To suppress compiler output, use `quiet make` instead of `make`.
156
In some non-POSIX shells you might need to prepend `sh` to all calls to `bin/package`.
158
Parallel building is supported by appending `-j` followed by the
159
desired maximum number of concurrent jobs, e.g., `bin/package make -j4`.
160
This speeds up building on systems with more than one CPU core.
161
(Type `bin/package host cpu` to find out how many CPU cores your system has.)
162
163
The compiled binaries are stored in the `arch` directory, in a subdirectory
164
that corresponds to your architecture. The command `bin/package host type`
165
outputs the name of this subdirectory.
166
167
Dynamically linked binaries, if supported for your system, are stored in
168
`dyn/bin` and `dyn/lib` subdirectories of your architecture directory.
169
If built, they are built in addition to the statically linked versions.
170
Export `AST_NO_DYLIB` to deactivate building dynamically linked versions.
172
If you have trouble or want to tune the binaries, you may pass additional
173
compiler and linker flags. It is usually best to export these as environment
174
variables *before* running `bin/package` as they could change the name of
175
the build subdirectory of the `arch` directory, so exporting them is a
176
convenient way to keep them consistent between build and test commands.
177
**Note that this system uses `CCFLAGS` instead of the usual `CFLAGS`.**
178
An example that makes Solaris Studio cc produce a 64-bit binary:
181
export CCFLAGS="-m64 -O" LDFLAGS="-m64"
182
bin/package make
183
```
185
Alternatively you can append these to the command, and they will only be
186
used for that command. You can also specify an alternative shell in which
187
to run the build scripts this way. For example:
190
bin/package make SHELL=/bin/bash CCFLAGS="-O2 -I/opt/local/include" LDFLAGS="-L/opt/local/lib"
191
```
193
**Note:** Do not add compiler flags that cause the compiler to emit terminal
194
escape codes, such as `-fdiagnostics-color=always`; this will cause the
195
build to fail as the probing code greps compiler diagnostics. Additionally,
196
do not add the `-ffast-math` compiler flag; arithmetic bugs will occur when
197
using that flag.
199
For more information run
202
bin/package help
203
```
205
Many other commands in this repo self-document via the `--help`, `--man` and
206
`--html` options; those that do have no separate manual page.
210
After compiling, you can run the regression tests.
211
To run the default test sets for ksh and the build system, use:
214
bin/package test
215
```
217
For ksh, use the `shtests` command directly to control the regression test runs.
218
Start by reading the information printed by:
221
bin/shtests --man
222
```
224
To hand-test ksh (as well as the utilities and the autoloadable functions
225
that come with it) without installing, run:
228
bin/package use
229
```
231
### Install
232
233
Usage: `bin/package install` *install_root_directory* [ *command* ... ]
234
235
Any command from the `arch` directory can be installed. If no *command* is
236
specified, `ksh` and `shcomp` are assumed.
237
238
The *install_root_directory* is the directory from which the command(s) will
239
actually be run. It will be created if it does not exist. Commands are
240
installed into its `bin` subdirectory, any shared libraries into `lib`, C
241
development header files into `include/ast`, and each command's manual page,
242
if available, is installed into `share/man`.
244
If a dynamically linked version of ksh and associated commands has been
245
built, then the `install` subcommand will prefer that: commands, dynamic
246
libraries and associated header files will be installed then. To install the
247
statically linked version instead (and skip the header files), either delete
248
the `dyn` subdirectory, or export `AST_NO_DYLIB=y` before building to prevent
249
it from being created in the first place.
250
251
An additional install prefix directory path can be passed in `DESTDIR`, which
252
can be either passed as an environment variable or specified on the comannd
253
line as an extra assignment-like argument. The value of `DESTDIR` will be
254
prefixed to the path of every destination file when installing it, but not
255
when configuring the install root directory in the installed files (as may be
256
required by individual systems, e.g., to find dynamic libraries). This feature
257
is designed for packagers who need to install ksh into a directory other than
258
the one from which it will be run in order to package it.
259
260
## What is ksh93?
261
262
The following is the official AT&T description from 1993 that came with the
263
ast-open distribution. The text is original, but hyperlinks were added here.
264
265
----
266
267
KSH-93 is the most recent version of the KornShell Language described in
268
"The KornShell Command and Programming Language," by Morris Bolsky and David
269
Korn of AT&T Bell Laboratories, ISBN 0-13-182700-6. The KornShell is a shell
270
programming language, which is upward compatible with "sh" (the Bourne
271
Shell), and is intended to conform to the IEEE P1003.2/ISO 9945.2
272
[Shell and Utilities standard](https://pubs.opengroup.org/onlinepubs/9699919799/utilities/contents.html).
273
KSH-93 provides an enhanced programming environment in addition to the major
274
command-entry features of the BSD shell "csh". With KSH-93, medium-sized
275
programming tasks can be performed at shell-level without a significant loss
276
in performance. In addition, "sh" scripts can be run on KSH-93 without
277
modification.
278
279
The code should conform to the
280
[IEEE POSIX 1003.1 standard](https://www.opengroup.org/austin/papers/posix_faq.html)
281
and to the proposed ANSI C standard so that it should be portable to all
282
such systems. Like the previous version, KSH-88, it is designed to accept
283
eight bit character sets transparently, thereby making it internationally
284
compatible. It can support multi-byte characters sets with some
285
characteristics of the character set given at run time.
286
287
KSH-93 provides the following features, many of which were also inherent in
288
KSH-88:
289
290
* Enhanced Command Re-entry Capability: The KSH-93 history function records
291
commands entered at any shell level and stores them, up to a
292
user-specified limit, even after you log off. This allows you to re-enter
293
long commands with a few keystrokes - even those commands you entered
294
yesterday. The history file allows for eight bit characters in commands
295
and supports essentially unlimited size histories.
296
* In-line Editing: In "sh", the only way to fix mistyped commands is to
297
backspace or retype the line. KSH-93 allows you to edit a command line
298
using a choice of EMACS-TC or "vi" functions. You can use the in-line
299
editors to complete filenames as you type them. You may also use this
300
editing feature when entering command lines from your history file. A user
301
can capture keystrokes and rebind keys to customize the editing interface.
302
* Extended I/O Capabilities: KSH-93 provides several I/O capabilities not
303
available in "sh", including the ability to:
304
* specify a file descriptor for input and output
305
* start up and run co-processes
306
* produce a prompt at the terminal before a read
307
* easily format and interpret responses to a menu
308
* echo lines exactly as output without escape processing
309
* format output using printf formats.
310
* read and echo lines ending in "\\".
311
* Improved performance: KSH-93 executes many scripts faster than the System
312
V Bourne shell. A major reason for this is that many of the standard
313
utilities are built-in. To reduce the time to initiate a command, KSH-93
314
allows commands to be added as built-ins at run time on systems that
315
support dynamic loading such as System V Release 4.
316
* Arithmetic: KSH-93 allows you to do integer arithmetic in any base from
317
two to sixty-four. You can also do double precision floating point
318
arithmetic. Almost the complete set of C language operators are available
319
with the same syntax and precedence. Arithmetic expressions can be used to
320
as an argument expansion or as a separate command. In addition, there is an
321
arithmetic for command that works like the for statement in C.
322
* Arrays: KSH-93 supports both indexed and associative arrays. The subscript
323
for an indexed array is an arithmetic expression, whereas, the subscript
324
for an associative array is a string.
325
* Shell Functions and Aliases: Two mechanisms - functions and aliases - can
326
be used to assign a user-selected identifier to an existing command or
327
shell script. Functions allow local variables and provide scoping for
328
exception handling. Functions can be searched for and loaded on first
329
reference the way scripts are.
330
* Substring Capabilities: KSH-93 allows you to create a substring of any
331
given string either by specifying the starting offset and length, or by
332
stripping off leading or trailing substrings during parameter
333
substitution. You can also specify attributes, such as upper and lower
334
case, field width, and justification to shell variables.
335
* More pattern matching capabilities: KSH-93 allows you to specify extended
336
regular expressions for file and string matches.
337
* KSH-93 uses a hierarchical name space for variables. Compound variables can
338
be defined and variables can be passed by reference. In addition, each
339
variable can have one or more disciplines associated with it to intercept
340
assignments and references.
341
* Improved debugging: KSH-93 can generate line numbers on execution traces.
342
Also, I/O redirections are now traced. There is a DEBUG trap that gets
343
evaluated before each command so that errors can be localized.
344
* Job Control: On systems that support job control, including System V
345
Release 4, KSH-93 provides a job-control mechanism almost identical to
346
that of the BSD "csh", version 4.1. This feature allows you to stop and
347
restart programs, and to move programs between the foreground and the
348
background.
349
* Added security: KSH-93 can execute scripts which do not have read
350
permission and scripts which have the setuid and/or setgid set when
351
invoked by name, rather than as an argument to the shell. It is possible
352
to log or control the execution of setuid and/or setgid scripts. The
353
noclobber option prevents you from accidentally erasing a file by
354
redirecting to an existing file.
355
* KSH-93 can be extended by adding built-in commands at run time. In
356
addition, KSH-93 can be used as a library that can be embedded into an
357
application to allow scripting.
358
359
Documentation for KSH-93 consists of an "Introduction to KSH-93",
360
"Compatibility with the Bourne Shell" and a manual page and a README file.
361
In addition, the "New KornShell Command and Programming Language" book is
362
available from Prentice Hall.