Announcing: KornShell 93u+m/1.0.9
https://github.com/ksh93/kshHere is the ninth ksh 93u+m/1.0 bugfix release. This release contains many
fixes of old and not-so-old bugs (including a couple of regressions in the
printf built-in introduced in 93u+m/1.0.5), adds Android/Termux as a
supported platform, and reintroduces the ability to build a dynamically
linked ksh (with libast, libdll, libcmd, and libshell available for other
applications to use) on most supported platforms.
Further below is an overview of the main changes. For greater detail, see
the NEWS file in the distribution. For complete detail, see the git(1)
commit log, which has full documentation of every significant change.
### HOW TO GET IT ###
Please download the source code tarball from our GitHub releases page:
https://github.com/ksh93/ksh/releasesTo build, follow the instructions in README.md or src/cmd/ksh93/README.
Or ask your distribution package maintainer to upgrade ksh to this version.
### ABOUT KSH ###
KornShell (ksh) is a full-featured and very fast shell script interpreter
and interactive command shell with a distinguished lineage: it is a direct
descendant of the Bourne shell and, like its ancestor, was developed at
AT&T, the birthplace of UNIX. ksh has been open source since 2000.
But when AT&T terminated development in 2020, ksh was left buggy and
unreliable. ksh 93u+m aims to fix this situation whilst maintaining and
growing the tradition. For now, we are focusing mostly on fixing bugs and
egregious flaws but we also prioritise backward compatibility, performance,
portability, and occasionally adding a feature. Work on ksh 93u+m started in
May 2020, based on the last AT&T stable release, ksh 93u+.
Unique ksh features include discipline functions (every variable expansion
or assignment can trigger a shell function call determining its value),
static scoping of local variables in functions, the ability to define your
own data types, customisable tilde expansion (new in 93u+m), a shell option
for file system case (in)sensitivity detection for pathname expansion and
file name completion (new in 93u+m), and much more.
### CONTRIBUTORS ###
Main ksh 93u+m developers: Martijn Dekker, Johnothan King, hyenias
Direct contributors: Andy Fiddaman, Anuradha Weeraman, atheik, Chase,
Cy Schubert, Govind Kamat, Harald van Dijk, hyousatsu, K. Eugene Carlson,
Lev Kujawski, Marc Wilson, Phi, Ryan Schmidt, rymrg, Sterling Jensen,
Trey Valenta, Vincent Mihalkovic
Also includes backported contributions by: David Korn, Glenn Fowler,
Lefteris Koutsofios, Siteshwar Vashisht, Kurtis Rader, Roland Mainz,
Finnbarr P. Murphy, Lijo George, OpenSUSE ksh 93u+ patch authors, Red Hat
ksh 93u+ path authors, Solaris ksh 93u+ patch authors, Debian ksh 93u+
patch authors, Apple ksh 93u+ patch authors, Graphviz maintainers
Many fixes have also been backported from the AT&T 93v- beta as well as the
former AT&T ksh2020 project lead by Kurtis Rader and Siteshwar Vashisht; we
appreciate and benefit from their work. Many thanks also to Siteshwar for
graciously donating his 'ksh93' GitHub organisation account!
### HOW TO GET INVOLVED ###
To report a bug, please open an issue at our GitHub page (see above).
Alternatively, email me at
martijn@inlv.org with your report.
To get involved in development, read the brief policy information in
README.md and then jump right in with a pull request or email a patch.
Feel free to use Discussions to introduce yourself to the community.
You can also join the mailing list/Google group at:
https://groups.google.com/g/korn-shell### MAIN CHANGES between ksh 93u+m/1.0.8 and 93u+m/1.0.9 ###
- Android/Termux is now a supported platform. Build dependencies: binutils,
clang, getconf. Runtime dependencies (optional): ncurses-utils, getconf.
- Reintroduced support for building a dynamically linked ksh(1)/shcomp(1),
with libast, libdll, libcmd, and libshell available to other programs as
dynamic libraries. 'bin/package install /your/basepath' will install
development headers. The dynamically linked version is built in a 'dyn'
subdirectory; there are no changes to the statically linked version.
Dynamic linking is currently tested and supported on Linux, Android,
macOS, all the BSDs, illumos, Solaris, QNX, and Haiku.
- On systems where the external printf(1) utility supports deprecated
pre-POSIX syntax for formatters starting with '-', ksh now adapts its
built-in printf to match, for compatibility with system scripts. However,
ksh's built-in printf options such as -v or --man are not affected.
- Fixed a regression in the 'printf' built-in, introduced in 93u+m/1.0.5,
where each instance of '\0' or '%Z' in the format operand caused a string
argument to be incorrectly skipped.
- Fixed a regression, introduced in 93u+m/1.0.5, in ordinal specifiers in
'printf %T' date specifications. For example,
printf '%(%F)T\n' '4th tuesday in march 2016'
wrongly printed '2016-04-09' and now again correctly prints '2016-03-22'.
- Fixed a regression of 'return' within traps, reintroduced in 93u+m/1.0.8
after being fixed in 93u+m/1.0.0. The regression caused a 'return' or
'exit' with no arguments to assume the before-trap exit status instead of
that of the last-run command. This broke the shipped 'autocd' function.
- Fixed a longstanding bug in shell arithmetic: the representation of
negative integers with a base other than 10 was incorrectly treated as
unsigned long. For example,
typeset -i16 n=-12; echo $n
now correctly outputs '-16#c' and no longer ouputs '16#fffffffffffffff4'.
- Fixed a bug, introduced in ksh93q+ 2005-05-22, that stopped an append
assignment from working together with a declaration command. For example,
'typeset var+=value' or 'export var+=value' now again work as expected.
- Fixed a longstanding bug where the default terminal width for typeset -L, -R,
or -Z, if not given, was miscalculated for multibyte or control characters.
- Fixed: expansions of name references in loops were incorrectly treated as
invariant so they yielded the wrong values.
- If a .get or .getn discipline function is set for a variable, it is no
longer incorrectly triggered when performing an arithmetic assignment on
that variable; only the .set discipline is now triggered (as documented).
- Many other bug fixes (see the NEWS file).
### MAIN CHANGES between ksh 93u+m/1.0.7 and 93u+m/1.0.8 ###
- Fixed a regression in the behavior of 'exit' in a trap action. The exit
status used when no argument is given to 'exit' is now once again the exit
status of the last command executed *before* the trap action.
- Fixed a race condition, introduced in 1.0.7, that occurred on some systems
when running an external command with a standard output redirection from a
command substitution.
- Fixed an init-time crash on failure to trim the shell command history file
due to a non-writable parent directory; ksh now prints a warning instead.
- The 'kill' built-in command now correctly refuses to issue SIGSTOP to the
shell's own process if the shell is a login shell.
### MAIN CHANGES between ksh 93u+m/1.0.6 and 93u+m/1.0.7 ###
- Fixed a hang in command substitutions (introduced in 93u+m/1.0.0) that was
triggered when redirecting standard output within a command substitution,
in combination with other factors. E.g., the following no longer hangs:
{ v=$(redirect 2>&1 1>&9); } 9>&1
- Fixed a crash on trying to append an indexed array value to an unset name
reference, e.g.: nameref unsetref; unsetref+=(foo bar). This now produces
a "removing nameref attribute" warning before performing the assignment.
- Fixed: assignments like name=(...) to arrays did not preserve the array
and variable types; similarly, assigning an empty set () to a compound
indexed array caused the -C attribute to be lost.
- Fixed incorrect rejection of the tab key while reading input using the
'read' built-in command.
- Fixed a bug in printf %T: when using dates and times in the past, time
zones for the present were incorrectly used, ignoring historical changes.
### MAIN CHANGES between ksh 93u+m/1.0.5 and 93u+m/1.0.6 ###
- Fixed a serious regression in pathname expansion where quoted wildcard
characters were incorrectly expanded if a pattern contains both a brace
expansion and a variable expansion.
- Fixed a bug where the command to launch a full-screen editor (^X^E in
emacs and 'v' in vi) could cause the wrong command line to be edited
if two shell sessions share a .sh_history file.
### MAIN CHANGES between ksh 93u+m/1.0.4 and 93u+m/1.0.5 ###
- Fixed various bugs causing crashes.
- Fixed many bugs in the emacs and vi line editors, in command completion,
and in file name completion.
- Fixed various bugs in the handling of quotes, backslash escapes and braces
when processing shell glob patterns (e.g. in pathname expansion and 'case').
- ksh now throws a panic and exits if a read error (such as an I/O error)
occurs while trying to read the next command(s) from a running script.
- Fixed many bugs in 'printf' and 'print -f' built-in commands, including:
. Multiple bugs causing incorrect output for relative date specifications,
e.g., printf %T\n 'exactly 20 months ago' now outputs a correct result.
. More printf bugs with mix and match of % and %x$.
. A data corruption bug when using %B with 'printf -v varname'.
. A bug causing double evaluation of arithmetic expressions.
- Fixed a bug where 'unset -f commandname', executed in a subshell, hides
any built-in command by the same name for the duration of that subshell.
- Fixed ${var/#/string} and ${var/%/string} (with anchored empty pattern)
to work as on mksh, bash and zsh; these are no longer ineffective.
- Fixed incorrect result of array slicing ${array[@]:offset:length} where
'length' is a nested expansion involving an array.
- Command names can now end in ':' as they can on other shells.
- Fixed a spurious syntax error in compound assignments upon encountering a
pair of repeated opening parentheses '(('.
- Fixed spurious syntax error in ${parameter:offset:length}: the arithmetic
expressions 'offset' and 'length' may now contain the operators ( ) & |.
- Fixed a parsing bug in the declaration of .sh.math.* arithmetic functions.
- Fixed nameref self-reference loop detection for more than two namerefs.
- Several improvements to the POSIX compatibility mode.
- Many more minor and/or esoteric bugfixes.
### MAIN CHANGES between ksh 93u+m/1.0.3 and 93u+m/1.0.4 ###
- Fixed multiple scoping-related bugs in the += additive assignment operator.
- A number of crashing bugs have been fixed.
- Various fixes for the Haiku operating system, notably 'ulimit -a' now works.
- Fixed the expansion of out-of-range \n back references in the string part of
${parameter//pattern/string}. For example: v=AB; echo "${v/@(A)B/\0:\1:\2}"
now yields 'AB:A:' instead of 'AB:A:\2'.
- Fixed quoted '!', '^' and '-' within [bracket] expressions in glob patterns;
single or double quotes failed to disable their operator behaviour.
- Fixed a bug introduced on 2021-04-04 that incorrectly allowed 'typeset' to
turn off the readonly and export attributes on a readonly variable.
- In the emacs line editor, the Ctrl+R reverse-search prompt is now visually
distinct from a literal control character ("^R: " instead of "^R").
- In the vi line editor, fixed the behaviour of 'C', 'c$' and 'I' to be
consistent with standard vi(1) and with Bolsky & Korn (1995, p. 121).
- Aliases for many GNU long options have been added to the /opt/ast/bin
built-in commands. Additionally, 'kill -s' now has a --signal long option
alias compatible with the util-linux option.
- Backported support for 'print -u p' from ksh 93v- for compatibility with
scripts written for 93v-/ksh2020 (this is equivalent to 'print -p').
### MAIN CHANGES between ksh 93u+m/1.0.2 and 93u+m/1.0.3 ###
This point release fixes the following:
- An old bug in history expansion (set -H) where any use of the history
comment character caused processing to be aborted as if it were an invalid
history expansion.
- A bug in command line options processing that caused short-form
option equivalents on some built-in commands to be ignored after one use,
e.g., the new read -a equivalent of read -A.
- Ksh freezing or using excessive memory if HISTSIZE is assigned a
pathologically large value.
- A bug that caused ksh in the vi editor mode to crash or produce invalid
completions if ESC = was used at the beginning of a line.
### MAIN CHANGES between ksh 93u+m/1.0.1 and 93u+m/1.0.2 ###
This bugfix release fixes the interactive shell crashing when one of the
predefined aliases (currently 'history' and 'r') is redefined, whether from
a profile/kshrc script or manually. This crash occurred in two scenarios:
1. when redefining and then unsetting a predefined alias;
2. when redefining a predefined alias and then executing a shell script that
does not begin with a #! path.
### MAIN CHANGES between ksh 93u+m/1.0.0 and 93u+m/1.0.1 ###
This is an urgent bugfix release that removes an incorrect exec
optimization that was capable of terminating the execution of scripts
prematurely in certain corner cases. It is known to make the build scripts
of GNU binutils produce corrupted results if ksh is used as /bin/sh.
See
https://github.com/ksh93/ksh/issues/507 for more information.
### MAIN CHANGES between ksh 93u+ and 93u+m/1.0.0 ###
Roughly a thousand bugs have been fixed, including many serious/critical
bugs. See the NEWS file for more information, and the git commit log for
complete documentation of every fix. Incompatible changes have been
minimised, but not at the expense of fixing bugs. For a list of
potentially incompatible changes, see src/cmd/ksh93/COMPATIBILITY.
Though there was a "no new features, bugfixes only" policy, some new
features were found necessary, either to fix serious design flaws or to
complete functionality that was evidently intended, but not finished.
Below is a summary of these new features.
New command line editor features:
- The forward-delete and End keys are now handled as expected in the
emacs and vi built-in line editors.
- In the vi and emacs line editors, repeat counts can now also be used for
arrow keys and the forward-delete key, e.g., <ESC> 7 <left-arrow> works.
- Various keys on extended PC keyboards are now handled as expected in the
emacs and vi built-in line editors.
New shell language features:
- Pathname expansion (a.k.a. globbing) now never matches the special names
'.' (current directory) and '..' (parent directory). This change makes a
pattern like .* useful; it now matches all hidden files (dotfiles) in the
current directory, without the harmful inclusion of '.' and '..'.
- Tilde expansion can now be extended or modified by defining a .sh.tilde.get
or .sh.tilde.set discipline function. See the manual for details.
- The &>file redirection shorthand (for >file 2>&1) is now available for all
scripts and interactive sessions and not only for profile/login scripts.
- Arithmetic expressions in native ksh mode no longer interpret a number
with a leading zero as octal in any context. Use 8#octalnumber instead
(e.g. 8#400 == 256). Arithmetic expressions now also behave identically
within and outside ((...)) and $((...)). If the POSIX mode is turned on,
a leading zero now denotes an octal number in all arithmetic contexts.
New features in built-in commands:
- Usage error messages now show the --help/--man self-documentation options.
- Path-bound built-ins (such as /opt/ast/bin/cat) can now be executed by
invoking the canonical path, so the following will now work as expected:
$ /opt/ast/bin/cat --version
version cat (AT&T Research) 2012-05-31
- 'cd' now supports an -e option that, when combined with -P, verifies
that $PWD is correct after changing directories; this helps detect
access permission problems. See:
https://www.austingroupbugs.net/view.php?id=253- 'command -x' now looks for external commands only, skipping built-ins.
In addition, its xargs-like functionality no longer freezes the shell on
Linux and macOS, making it effectively a new feature on these systems.
- 'printf' now supports a -v option as in bash. This assigns formatted
output directly to variables, which is very fast and will not strip
final newline (\n) characters.
- 'redirect' now checks if all arguments are valid redirections before
performing them. If an error occurs, it issues an error message instead
of terminating the shell.
- 'return', when used to return from a function, can now return any
status value in the 32-bit signed integer range, like on zsh. However,
due to a traditional Unix kernel limitation, $? is still trimmed to its
least significant 8 bits whenever a shell or subshell exits.
- 'suspend' now refuses to suspend a login shell, as there is probably no
parent shell to return to and the login session would freeze.
- 'test'/'[' now supports all the same operators as [[ (including =~,
\<, \>) except for the different 'and'/'or' operators. Note that
'test'/'[' remains deprecated due to its unfixable pitfalls;
[[ ... ]] is recommended instead.
- 'times' now gives high precision output in a POSIX compliant format.
- 'type'/'whence': Two bash-like flags were backported from ksh 93v-:
- 'whence -P/type -P' is an alias to the existing -p flag.
- 'whence -t/type -t' will print only the type of a command in a
simple format that is designed to be easy to use for scripts.
- 'typeset' has a new '-g' flag that forces variables to be created or
modified at the global scope regardless of context, as on bash 4.2+.
- 'typeset' now gives an informative error message if an incompatible
combination of options is given.
- 'ulimit': Added three options inspired by bash:
- 'ulimit -k' sets the maximum number of kqueues.
- 'ulimit -P' sets the maximum number of pseudo-terminals.
- 'ulimit -R' sets the maximum time in microseconds a real-time process
can run before blocking.
Note that not all operating systems support the limits set by these options.
- 'whence -v/-a' now reports the location of autoloadable functions.
New features in shell options:
- When the -b/--notify shell option is on and the vi or emacs/gmacs shell
line editor is in use, 'Done' and similar notifications from completed
background jobs are now inserted directly above the line you're typing,
without affecting your command line display.
- A new --functrace long-form shell option causes the -x/--xtrace option's
state and the DEBUG trap action to be inherited by function scopes instead
of being reset to default. Changes made to them within a function scope
still do not propagate back to the parent scope. Similarly, this option
also causes the DEBUG trap action to be inherited by subshells.
- A new --globcasedetect shell option is added on operating systems where
we can check for a case-insensitive file system (currently Linux, macOS,
QNX 7.0+, and Windows/Cygwin). When this option is turned on, pathname
expansion (globbing), as well as tab completion on interactive shells,
automatically become case-insensitive depending on the file system.
This is separately determined for each pathname component.
- Enhancement to -G/--globstar: symbolic links to directories are now
followed if they match a normal (non-**) glob pattern. For example, if
'/lnk' is a symlink to a directory, '/lnk/**' and '/l?k/**' now work as
you would expect.
- The new --histreedit and --histverify options modify history expansion
(--histexpand). If --histreedit is on and a history expansion fails, the
command line is reloaded into the next prompt's edit buffer, allowing
corrections. If --histverify is on, the results of a history expansion are
not immediately executed but instead loaded into the next prompt's edit
buffer, allowing further changes.
- A new --nobackslashctrl shell option disables the special escaping
behaviour of the backslash character in the emacs and vi built-in editors.
Particularly in the emacs editor, this makes it much easier to go back,
insert a forgotten backslash into a command, and then continue editing
without having your next arrow key replace your backslash with garbage.
- A new --posix shell option has been added to ksh 93u+m that makes the
ksh language more compatible with other shells by following the POSIX
standard more closely. See the manual page for details. It is enabled by
default if ksh is invoked as sh, otherwise it is disabled by default.
[ANN] ksh 93u+m/1.0.9