sbcl ¶

1.3.1 How to Report Bugs Effectively ¶

Please include enough information in a bug report that someone reading it can reproduce the problem, i.e. don’t write

Subject: apparent bug in PRINT-OBJECT (or *PRINT-LENGTH*?)
PRINT-OBJECT doesn't seem to work with *PRINT-LENGTH*. Is this a bug?

but instead

Subject: apparent bug in PRINT-OBJECT (or *PRINT-LENGTH*?)
In sbcl-1.2.3 running under OpenBSD 4.5 on my Alpha box, when
I compile and load the file
   (DEFSTRUCT (FOO (:PRINT-OBJECT (LAMBDA (X Y)
                                    (LET ((*PRINT-LENGTH* 4))
                                      (PRINT X Y)))))
     X Y)
then at the command line type
   (MAKE-FOO)
the program loops endlessly instead of printing the object.

A more in-depth discussion on reporting bugs effectively can be found at

http://www.chiark.greenend.org.uk/~sgtatham/bugs.html.

1.3.2 How to Report Signal-related Bugs ¶

If you run into a signal related bug, you are getting fatal errors such as signal N is [un]blocked or just hangs, and you want to send a useful bug report then:

• Compile SBCL with ldb enabled (feature :sb-ldb, see base-target-features.lisp-expr).
• Isolate a smallish test case, run it.
• If it just hangs kill it with sigabrt: kill -ABRT <pidof sbcl>.
• Print the backtrace from ldb by typing ba.
• Attach gdb: gdb -p <pidof sbcl> and get backtraces for all threads: thread apply all ba.
• If multiple threads are in play then still in gdb, try to get Lisp backtrace for all threads: thread apply all call backtrace_from_fp($ebp, 100, 0). Substitute $ebp with $rbp on x86-64. The backtraces will appear in the stdout of the SBCL process.
• Send a report with the backtraces and the output (both stdout and stderr) produced by SBCL.
• Don’t forget to include OS and SBCL version.
• If available, include information on outcome of the same test with other versions of SBCL, OS, ...

2.3.1 Declarations ¶

Declarations are generally treated as assertions. This general principle, and its implications, and the bugs which still keep the compiler from quite satisfying this principle, are discussed in Declarations as Assertions.

2.3.2 FASL format ¶

SBCL fasl-format is binary compatible only with the exact SBCL version it was generated with. While this is obviously suboptimal, it has proven more robust than trying to maintain fasl compatibility across versions: accidentally breaking things is far too easy, and can lead to hard to diagnose bugs.

The following snippet handles fasl recompilation automatically for ASDF-based systems, and makes a good candidate for inclusion in the user or system initialization file (see Initialization Files).

(require :asdf)

;;; If a fasl was stale, try to recompile and load (once).
(defmethod asdf:perform :around ((o asdf:load-op)
                                 (c asdf:cl-source-file))
   (handler-case (call-next-method o c)
      ;; If a fasl was stale, try to recompile and load (once).
      (sb-ext:invalid-fasl ()
         (asdf:perform (make-instance 'asdf:compile-op) c)
         (call-next-method))))

2.3.3 Compiler-only Implementation ¶

SBCL is essentially a compiler-only implementation of Common Lisp. That is, for all but a few special cases, eval creates a lambda expression, calls compile on the lambda expression to create a compiled function, and then calls funcall on the resulting function object. A more traditional interpreter is also available on default builds; it is usually only called internally. This is explicitly allowed by the ANSI standard but leads to some oddities; e.g. at default settings, functionp and compiled-function-p are equivalent, and they collapse into the same function when SBCL is built without the interpreter.

2.3.4 Defining Constants ¶

SBCL is quite strict about ANSI’s definition of defconstant. ANSI says that doing defconstant of the same symbol more than once is undefined unless the new value is eql to the old value. Conforming to this specification is a nuisance when the "constant" value is only constant under some weaker test like string= or equal.

It’s especially annoying because, in SBCL, defconstant takes effect not only at load time but also at compile time, so that just compiling and loading reasonable code like

(defconstant +foobyte+ '(1 4))

runs into this undefined behavior. Many implementations of Common Lisp try to help the programmer around this annoyance by silently accepting the undefined code and trying to do what the programmer probably meant.

SBCL instead treats the undefined behavior as an error. Often such code can be rewritten in portable ANSI Common Lisp which has the desired behavior. E.g., the code above can be given an exactly defined meaning by replacing defconstant either with defparameter or with a customized macro which does the right thing, e.g.

(defmacro define-constant (name value &optional doc)
  `(defconstant ,name (if (boundp ',name) (symbol-value ',name) ,value)
                      ,@(when doc (list doc))))

or possibly along the lines of the sb-int:defconstant-eqx macro used internally in the implementation of SBCL itself. In circumstances where this is not appropriate, the programmer can handle the condition type sb-ext:defconstant-uneql and choose either the continue restart or abort restart as appropriate.

2.3.5 Style Warnings ¶

SBCL gives style warnings about various kinds of perfectly legal code, e.g.

• multiple defuns of the same symbol in different units;
• special variables not named in the conventional *foo* style, and lexical variables unconventionally named in the *foo* style.

This causes friction with people who point out that other ways of organizing code (especially avoiding the use of defgeneric) are just as aesthetically stylish. However, these warnings should be read not as warning, bad aesthetics detected, you have no style but as warning, this style keeps the compiler from understanding the code as well as you might like. That is, unless the compiler warns about such conditions, there’s no way for the compiler to warn about some programming errors which would otherwise be easy to overlook. (Related bug: The warning about multiple defuns is pointlessly annoying when you compile and then load a function containing defun wrapped in eval-when, and ideally should be suppressed in that case, but still isn’t as of SBCL 0.7.6.)

2.4.1 Editor Integration ¶

Though SBCL can be used running "bare", the recommended mode of development is with an editor connected to SBCL, supporting not only basic lisp editing (paren-matching, etc), but providing among other features an integrated debugger, interactive compilation, and automated documentation lookup.

Currently SLIME (Superior Lisp Interaction Mode for Emacs) together with Emacs is recommended for use with SBCL, though other options exist as well. Historically, the ILISP package at http://ilisp.cons.org/ provided similar functionality, but it does not support modern SBCL versions.

SLIME can be downloaded from https://slime.common-lisp.dev/.

2.4.2 Language Reference ¶

CLHS (Common Lisp Hyperspec) is a hypertext version of the ANSI standard, made freely available by LispWorks – an invaluable reference.

See https://www.lispworks.com/documentation/HyperSpec/Front/index.htm.

2.4.3 Generating Executables ¶

SBCL can generate stand-alone executables. The generated executables include the SBCL runtime itself, so no restrictions are placed on program functionality. For example, a deployed program can call compile and load, which requires the compiler to be present in the executable. For further information, sb-ext:save-lisp-and-die.

2.5.1 SBCL Homepage ¶

The SBCL website at http://www.sbcl.org/ has some general information, plus links to mailing lists devoted to SBCL, and to archives of these mailing lists. Subscribing to the mailing lists sbcl-help and sbcl-announce is recommended: both are fairly low-volume, and help you keep abreast with SBCL development.

2.5.2 Online Documentation ¶

Documentation for non-ANSI extensions for various commands is available online from the SBCL executable itself. The extensions for functions which have their own command prompts (e.g. the debugger, and inspect) are documented in text available by typing help at their command prompts. The extensions for functions which don’t have their own command prompt (such as trace) are described in their documentation strings, unless your SBCL was compiled with an option not to include documentation strings, in which case the documentation strings are only readable in the source code.

2.5.3 Additional Documentation Files ¶

Besides this user manual both SBCL source and binary distributions include some other SBCL-specific documentation files, which should be installed along with this manual on your system, e.g. in /usr/local/share/doc/sbcl/.

• copying: Licence and copyright summary.
• credits: Authorship information on various parts of SBCL.
• install: Covers installing SBCL from both source and binary distributions on your system, and also has some installation related troubleshooting information.
• news: Summarizes changes between various SBCL versions.

2.5.4 Internals Documentation ¶

If you’re interested in the development of the SBCL system itself, then subscribing to sbcl-devel is a good idea.

SBCL internals documentation – besides comments in the source – is available in the Web Archive:

https://web.archive.org/web/20120814000933/http://sbcl-internals.cliki.net/index.

Some low-level information describing the programming details of the conversion from CMUCL to SBCL is available in the doc/FOR-CMUCL-DEVELOPERS file.

2.6.1 Internet Community ¶

IRC channels on https://libera.chat/:

• #common-lisp: "Common Lisp, the #1=(programmable . #1#) programming language"
• #lispcafe: "The Lisp Café; sit down, have a drink, chat about anything, and enjoy your stay. | https://www.cliki.net/lispcafe | Be insuperable to each other".
• #sbcl: "Steel Bank Common Lisp Dev Hangout"

You can use https://web.libera.chat or a normal IRC client.

Also, see https://www.reddit.com/r/Common_Lisp/, as well as https://www.lisp.org and https://cliki.net, which contain numerous pointers places in the net where lispers talks shop.

2.6.2 Third-party Libraries ¶

For a wealth of information about free Common Lisp libraries and tools we recommend checking out CLiki: https://cliki.net/.

The most popular library manager is Quicklisp: https://www.quicklisp.org/beta/.

2.6.3 Common Lisp Books ¶

If you’re not a programmer and you’re trying to learn, many introductory Lisp books are available. However, we don’t have any standout favorites.

If you are an experienced programmer in other languages but need to learn about Common Lisp, some books stand out:

• Practical Common Lisp, by Peter Seibel

  An excellent introduction to the language, covering both the basics and "advanced topics" like macros, CLOS, and packages. Available both in print format and on the web: https://gigamonkeys.com/book/.

• Paradigms Of Artificial Intelligence Programming, by Peter Norvig

  Good information on general Common Lisp programming, and many nontrivial examples. Whether or not your work is AI, it’s a very good book to look at.

• On Lisp, by Paul Graham

  An in-depth treatment of macros, but not recommended as a first Common Lisp book, since it is slightly pre-ANSI so you need to be on your guard against non-standard usages, and since it doesn’t really even try to cover the language as a whole, focusing solely on macros. Downloadable from https://www.paulgraham.com/onlisp.html.

• Object-Oriented Programming In Common Lisp, by Sonya Keene

  With the exception of Practical Common Lisp, most introductory books don’t emphasize CLOS. This one does. Even if you’re very knowledgeable about object oriented programming in the abstract, it’s worth looking at this book if you want to do any OO in Common Lisp. Some abstractions in CLOS (especially multiple dispatch) go beyond anything you’ll see in most OO systems, and there are a number of lesser differences as well. This book tends to help with the culture shock.

• Art Of Metaobject Programming, by Gregor Kiczales et al.

  Currently the prime source of information on the Common Lisp Metaobject Protocol, which is supported by SBCL. Section 2 (Chapters 5 and 6) are freely available at http://mop.lisp.se/www.alu.org/mop/.

3.1.1 Running from Shell ¶

To run SBCL, type sbcl at the command line.

You should end up in the toplevel REPL (read-eval-print loop), where you can interact with SBCL by typing expressions.

$ sbcl
This is SBCL 0.8.13.60, an implementation of ANSI Common Lisp.
More information about SBCL is available at <http://www.sbcl.org/>.

SBCL is free software, provided as is, with absolutely no warranty.
It is mostly in the public domain; some portions are provided under
BSD-style licenses.  See the CREDITS and COPYING files in the
distribution for more information.
* (+ 2 2)
4
* (exit)
$

Also see Command Line Options and Stopping SBCL.

3.1.2 Running from Emacs ¶

To run SBCL as an inferior-lisp from Emacs, in your .emacs do something like:

;;; The SBCL binary and command-line arguments
(setq inferior-lisp-program "/usr/local/bin/sbcl --noinform")

For more information on using SBCL with Emacs, see Editor Integration.

3.1.3 Shebang Scripts ¶

Standard Unix tools that are interpreters follow a common command line protocol that is necessary to work with "shebang scripts". SBCL supports this via the --script command line option (see Command Line Options).

Example file (hello.lisp):

#!/usr/local/bin/sbcl --script
(write-line "Hello, World!")

Usage from the command line:

$ ./hello.lisp
Hello, World!

Note that SBCL skips the shebang line when it reads the file:

$ sbcl --script hello.lisp
Hello, World!

3.2.1 Exit ¶

SBCL can be stopped at any time by calling sb-ext:exit, optionally returning a specified numeric value to the calling process. See Threading for information about terminating individual threads.

Function: sb-ext:exit &key code abort timeout ¶

Terminates the process, causing SBCL to exit with code. code defaults to 0 when abort is false, and 1 when it is true.

When abort is false (the default), current thread is first unwound, *exit-hooks* are run, other threads are terminated, and standard output streams are flushed before SBCL calls exit(3) – at which point atexit(3) functions will run. If multiple threads call exit with abort being false, the first one to call it will complete the protocol.

When abort is true, SBCL exits immediately by calling _exit(2) without unwinding stack, or calling exit hooks. Note that _exit(2) does not call atexit(3) functions unlike exit(3).

Recursive calls to exit cause exit to behave as if abort was true.

timeout controls waiting for other threads to terminate when abort is nil. Once current thread has been unwound and *exit-hooks* have been run, spawning new threads is prevented and all other threads are terminated by calling sb-thread:terminate-thread on them. The system then waits for them to finish using sb-thread:join-thread, waiting at most a total timeout seconds for all threads to join. Those threads that do not finish in time are simply ignored while the exit protocol continues. timeout defaults to *exit-timeout*, which in turn defaults to 60. timeout nil means to wait indefinitely.

Note that timeout applies only to sb-thread:join-thread, not *exit-hooks*. Since sb-thread:terminate-thread is asynchronous, getting multithreaded application termination with complex cleanups right using it can be tricky. To perform an orderly synchronous shutdown use an exit hook instead of relying on implicit thread termination.

Consequences are unspecified if serious conditions occur during exit excepting errors from *exit-hooks*, which cause warnings and stop execution of the hook that signaled, but otherwise allow the exit process to continue normally.

3.2.2 End of File ¶

By default SBCL also exits on end of input, caused either by user pressing Control-D on an attached terminal, or end of input when using SBCL as part of a shell pipeline.

3.2.3 Saving a Core Image ¶

SBCL has the ability to save its state as a file for later execution. This functionality is important for its bootstrapping process, and is also provided as an extension to the user.

Function: sb-ext:save-lisp-and-die core-file-name &key toplevel executable save-runtime-options callable-exports purify root-structures environment-name compression ¶

Save a "core image", i.e. enough information to restart a Lisp process later in the same state, in the file of the specified name. Only global state is preserved: the stack is unwound in the process.

The following &key arguments are defined:

• :toplevel

  The function to run when the created core file is resumed. The default function handles command line toplevel option processing (see Toplevel Options) and runs the top level read-eval-print loop. This function returning is equivalent to (sb-ext:exit :code 0) being called.

  toplevel functions should always provide an abort restart: otherwise code they call will run without one.

• :executable

  If true, arrange to combine the SBCL runtime and the core image to create a standalone executable. If false (the default), the core image will not be executable on its own. Executable images always behave as if they were passed the --noinform runtime option. If :executable is :elf-object, then the resulting core will be wrapped in a .o which requires further linking. (EXPERIMENTAL)

• :save-runtime-options

  If true, values of runtime options --dynamic-space-size and --control-stack-size that were used to start SBCL are stored in the standalone executable, and restored when the executable is run. This also inhibits normal runtime option processing, causing all command line arguments to be passed to the toplevel. If :accept-runtime-options then --dynamic-space-size and --control-stack-size are still processed by the runtime. Meaningless if :executable is nil.

• :callable-exports

  This should be a list of symbols to be initialized to the appropriate alien callables on startup. All exported symbols should be present as global symbols in the symbol table of the runtime before the saved core is loaded. When this list is non-empty, the :toplevel argument cannot be supplied.

• :purify

  If true (the default), then some objects in the restarted core will be memory-mapped as read-only. Among those objects are numeric vectors that were determined to be compile-time constants, and any immutable values according to the language specification such as symbol names.

• :root-structures

  This should be a list of the main entry points in any newly loaded systems. This need not be supplied, but locality and/or gc performance may be better if they are. This has two different but related meanings: If :purify is true - and only for cheneygc - the root structures are those which anchor the set of objects moved into static space. On gencgc - and only on platforms supporting immobile code - these are the functions and/or function-names which commence a depth-first scan of code when reordering based on the statically observable call chain. The complete set of reachable objects is not affected per se. This argument is meaningless if neither enabling precondition holds.

• :environment-name

  This has no purpose; it is accepted only for legacy compatibility.

• :compression

  This is only meaningful if the runtime was built with the :sb-core-compression feature enabled. If nil (the default), saves to uncompressed core files. If :sb-core-compression was enabled at build-time, the argument may also be an integer from -7 to 22, corresponding to zstd compression levels, or t (which is equivalent to the default compression level, 9).

• :application-type

  Present only on Windows and is meaningful only with :executable t. Specifies the subsystem of the executable, :console or :gui. The notable difference is that :gui doesn’t automatically create a console window. The default is :console.

The save/load process changes the values of some global variables:

• *standard-output*, *debug-io*, etc

  Everything related to open streams is necessarily changed, since the OS won’t let us preserve a stream across save and load.

• *default-pathname-defaults*

  This is reinitialized to reflect the working directory where the saved core is loaded.

save-lisp-and-die interacts with sb-alien:load-shared-object: see its documentation for details.

On threaded platforms only a single thread may remain running after sb-ext:*save-hooks* have run. Applications using multiple threads can be save-lisp-and-die friendly by registering a save-hook that quits any additional threads, and an init-hook that restarts them.

This implementation is not as polished and painless as you might like:

• It corrupts the current Lisp image enough that the current process needs to be killed afterwards. This can be worked around by forking another process that saves the core.
• There is absolutely no binary compatibility of core images between different runtime support programs. Even runtimes built from the same sources at different times are treated as incompatible for this purpose.

This isn’t because we like it this way, but just because there don’t seem to be good quick fixes for either limitation and no one has been sufficiently motivated to do lengthy fixes.

Variable: sb-ext:*save-hooks* ¶

A list of function designators which are called in an unspecified order before creating a saved core image.

Unused by SBCL itself: reserved for user and applications.

In cases where the standard initialization files have already been loaded into the saved core, and alternative ones should be used (or none at all), SBCL allows customizing the initfile pathname computation.

Variable: sb-ext:*sysinit-pathname-function* ¶

Designator for a function of zero arguments called to obtain a pathname designator for the default sysinit file, or nil. If the function returns nil, no sysinit file is used unless one has been specified on the command-line.

Variable: sb-ext:*userinit-pathname-function* ¶

Designator for a function of zero arguments called to obtain a pathname designator or a stream for the default userinit file, or nil. If the function returns nil, no userinit file is used unless one has been specified on the command-line.

To facilitate distribution of SBCL applications using external resources, the filesystem location of the SBCL core file being used is available from Lisp.

Variable: sb-ext:*core-pathname* ¶

The absolute pathname of the running SBCL core.

3.2.4 Exit on Errors ¶

SBCL can also be configured to exit if an unhandled error occurs, which is mainly useful for acting as part of a shell pipeline; doing so under most other circumstances would mean giving up large parts of the flexibility and robustness of Common Lisp. See Debugger Entry and the command line option --disable-debugger in Runtime Options.

3.3.1 Runtime Options ¶

• --core <corefilename>

  Run the specified Lisp core file instead of the default. Note that if the Lisp core file is a user-created core file, it may run a nonstandard toplevel which does not recognize the standard toplevel options.

• --dynamic-space-size <megabytes>

  Size of the dynamic space reserved on startup in megabytes. Default value is platform dependent.

• --control-stack-size <megabytes>

  Size of control stack reserved for each thread in megabytes. Default value is 2.

• --tls-limit <positive integer>

  Maximum number of thread-local symbols in threaded builds. Default value is 4096.

• --noinform

  Suppress the printing of any banner or other informational message at startup. This makes it easier to write Lisp programs which work cleanly in Unix pipelines. See also the --noprint and --disable-debugger options.

• --disable-ldb

  Disable the low-level debugger. Only effective if SBCL is compiled with ldb.

• --lose-on-corruption

  There are some dangerous low-level errors (for instance, control stack exhausted, memory fault) that (or whose handlers) can corrupt the image. By default, SBCL prints a warning, then tries to continue and handle the error in Lisp, but this will not always work, and SBCL may malfunction or even hang. With this option, upon encountering such an error, SBCL will exit instead of invoking ldb (if present and enabled).

• --script <filename>

  As a runtime option, this is equivalent to --noinform --disable-ldb --lose-on-corruption --end-runtime-options --script <filename>. See the description of --script as a toplevel option below. If there are no other command line arguments following --script, the filename argument can be omitted.

• --merge-core-pages

  When platform support is present, provide hints to the operating system that identical pages may be shared between processes until they are written to. This can be useful to reduce the memory usage on systems with multiple SBCL processes started from similar but differently-named core files, or from compressed cores. Without platform support, do nothing. By default only compressed cores trigger hinting.

• --no-merge-core-pages

  Ensures that no sharing hint is provided to the operating system.

• --help

  Print some basic information about SBCL, then exit.

• --version

  Print SBCL’s version information, then exit.

In the future, runtime options may be added to control behaviour such as lazy allocation of memory.

Runtime options, including any --end-runtime-options option, are stripped out of the command line before the Lisp toplevel logic gets a chance to see it.

3.3.2 Toplevel Options ¶

The following options are processed and removed by the default toplevel (see sb-ext:save-lisp-and-die).

• --sysinit <filename>

  Load filename instead of the default system initialization file (see Initialization Files).

• --no-sysinit

  Don’t load a system-wide initialization file. If this option is given, the --sysinit option is ignored.

• --userinit <filename>

  Load filename instead of the default user initialization file (see Initialization Files.)

• --no-userinit

  Don’t load a user initialization file. If this option is given, the --userinit option is ignored.

• --eval <command>

  After executing any initialization file, but before starting the read-eval-print loop on standard input, read and evaluate command. More than one --eval option can be used, and all will be read and executed, in the order they appear on the command line.

• --load <filename>

  This is equivalent to --eval '(load "<filename>")'. The special syntax is intended to reduce quoting headaches when invoking SBCL from shell scripts.

• --noprint

  When ordinarily the toplevel "read-eval-print loop" would be executed, execute a "read-eval loop" instead, i.e. don’t print a prompt and don’t echo results. Combined with the --noinform runtime option, this makes it easier to write Lisp "scripts" which work cleanly in Unix pipelines.

• --disable-debugger

  By default when SBCL encounters an error, it enters the builtin debugger, allowing interactive diagnosis and possible intercession. This option disables the debugger, causing errors to print a backtrace and exit with status 1 instead. When given, this option takes effect before loading of initialization files or processing --eval and --load options. See sb-ext:disable-debugger and Debugger Entry.

• --script <filename>

  Implies --no-userinit --no-sysinit --disable-debugger --end-toplevel-options.

  Causes the system to load the specified file instead of entering the read-eval-print-loop, and exit afterwards. If the file begins with a shebang line, it is ignored.

  If there are no other command line arguments following, the filename can be omitted: this causes the script to be loaded from standard input instead. Shebang lines in standard input script are currently not ignored.

  In either case, if there is an unhandled error (e.g. end of file, or a broken pipe) on either standard input, standard output, or standard error, the script silently exits with code 0. This allows e.g. safely piping output from SBCL to head -n1 or similar.

  Additionally, the option sets *compile-verbose* and *load-verbose* to nil while loading the file to avoid potentially verbose diagnostic messages printed on the standard output.

4.1.1 Controlling Verbosity ¶

The compiler can be quite verbose in its diagnostic reporting, rather more then some users would prefer – the amount of noise emitted can be controlled, however.

To control emission of compiler diagnostics (of any severity other than error: Diagnostic Severity) use the sb-ext:muffle-conditions and sb-ext:unmuffle-conditions declarations, specifying the type of condition that is to be muffled (the muffling is done using an associated muffle-warning restart).

Global control:

;;; Muffle compiler-notes globally
(declaim (sb-ext:muffle-conditions sb-ext:compiler-note))

Local control:

;;; Muffle compiler-notes based on lexical scope
(defun foo (x)
  (declare (optimize speed) (fixnum x)
           (sb-ext:muffle-conditions sb-ext:compiler-note))
  (values (* x 5) ; no compiler note from this
    (locally
      (declare (sb-ext:unmuffle-conditions sb-ext:compiler-note))
      ;; this one gives a compiler note
      (* x -5))))

Declaration: sb-ext:muffle-conditions ¶

Syntax: (sb-ext:muffle-conditions &rest types).

Muffle the diagnostic messages that would be caused by compile-time signals of types.

Declaration: sb-ext:unmuffle-conditions ¶

Syntax: (sb-ext:muffle-conditions &rest types).

Cancel the effect of a previous sb-ext:muffle-conditions declaration.

Various details of how the compiler messages are printed can be controlled via the alist sb-ext:*compiler-print-variable-alist*.

Variable: sb-ext:*compiler-print-variable-alist* ¶

An association list describing new bindings for special variables to be used by the compiler for error-reporting, etc. E.g. ((*print-length* . 10) (*print-level* . 6) (*print-pretty* . nil)).

The variables in the car positions are bound to the values in the cdr during the execution of some debug commands. When evaluating arbitrary expressions in the debugger, the normal values of the printer control variables are in effect.

Initially empty, *compiler-print-variable-alist* is typically used to specify bindings for printer control variables.

For information about muffling warnings signaled outside of the compiler, see Customization Hooks for Users.

4.1.2 Diagnostic Severity ¶

There are four levels of compiler diagnostic severity:

• error
• warning
• style warning
• note

The first three levels correspond to condition classes which are defined in the ANSI standard for Common Lisp and which have special significance to the compile and compile-file functions. These levels of compiler error severity occur when the compiler handles conditions of these classes.

The fourth level of compiler error severity, note, corresponds to the sb-ext:compiler-note, and is used for problems which are too mild for the standard condition classes, typically hints about how efficiency might be improved. The sb-ext:code-deletion-note, a subtype of sb-ext:compiler-note, is signalled when the compiler deletes user-supplied code after proving that the code in question is unreachable.

Future work for SBCL includes expanding this hierarchy of types to allow more fine-grained control over emission of diagnostic messages.

Condition: sb-ext:compiler-note ¶

Root of the hierarchy of conditions representing information discovered by the compiler that the user might wish to know, but which does not merit a style-warning (or any more serious condition).

Condition: sb-ext:code-deletion-note ¶

A condition type signalled when the compiler deletes code that the user has written, having proved that it is unreachable.

4.1.3 Understanding Compiler Diagnostics ¶

The messages emitted by the compiler contain a lot of detail in a terse format, so they may be confusing at first. The messages will be illustrated using this example program:

(defmacro zoq (x)
  `(roq (ploq (+ ,x 3))))

(defun foo (y)
  (declare (symbol y))
  (zoq y))

The main problem with this program is that it is trying to add 3 to a symbol. Note also that the functions roq and ploq aren’t defined anywhere.

• Parts of a Compiler Diagnostic
• Original and Actual Source
• Processing Path

; file: /tmp/foo.lisp
; in: DEFUN FOO
;     (ZOQ Y)
; --> ROQ PLOQ
; ==>
;   (+ Y 3)
;
; caught WARNING:
;   Asserted type NUMBER conflicts with derived type (VALUES SYMBOL &OPTIONAL).

; file: /tmp/foo.lisp
; in: DEFUN FOO
;     (ZOQ Y)
; --> ROQ
; ==>
;   (PLOQ (+ Y 3))
;
; caught STYLE-WARNING:
;   undefined function: PLOQ

; ==>
;   (ROQ (PLOQ (+ Y 3)))
;
; caught STYLE-WARNING:
;   undefined function: ROQ

; file: /tmp/foo.lisp
; in: DEFUN BAR
;     (LET (A)
;     (DECLARE (FIXNUM A))
;     (SETQ A (FOO X))
;     A)
;
; caught WARNING:
;   Asserted type FIXNUM conflicts with derived type (VALUES NULL &OPTIONAL).

(defun foo (n)
  (dotimes (i n *undefined*)))

; in: DEFUN FOO
;     (DOTIMES (I N *UNDEFINED*))
; --> DO BLOCK LET TAGBODY RETURN-FROM
; ==>
;   (PROGN *UNDEFINED*)
;
; caught WARNING:
;   undefined variable: *UNDEFINED*

(do ((i 0 (1+ i)) (#:g1 n))
    ((>= i #:g1) *undefined*)
  (declare (type unsigned-byte i)))

(block nil
  (let ((i 0) (#:g1 n))
    (declare (type unsigned-byte i))
    (tagbody (go #:g3)
      #:g2    (psetq i (1+ i))
      #:g3    (unless (>= i #:g1) (go #:g2))
      (return-from nil (progn *undefined*)))))

4.2.1 Declarations as Assertions ¶

The SBCL compiler treats type declarations differently from most other Lisp compilers. Under default compilation policy the compiler doesn’t blindly believe type declarations, but considers them assertions about the program that should be checked: all type declarations that have not been proven to always hold are asserted at runtime.

Remaining bugs in the compiler’s handling of types unfortunately provide some exceptions to this rule, see Implementation Limitations.

CLOS slot types form a notable exception. Types declared using the :type slot option in defclass are asserted if and only if the class was defined in safe code and the slot access location is in safe code as well. This laxness does not pose any internal consistency issues, as the CLOS slot types are not available for the type inferencer, nor do CLOS slot types provide any efficiency benefits.

There are three type checking policies available in SBCL, selectable via optimize declarations.

• Full Type Checks

  All declarations are considered assertions to be checked at runtime, and all type checks are precise. The default compilation policy provides full type checks.

  Used when (or (>= safety 2) (>= safety speed 1)).

• Weak Type Checks

  Declared types may be simplified into faster to check supertypes: for example, (or (integer -17 -7) (integer 7 17)) is simplified into (integer -17 17).

  Warning: It is relatively easy to corrupt the heap when weak type checks are used if the program contains type-errors.

  Used when (and (< safety 2) (< safety speed)).

• No Type Checks

  All declarations are believed without assertions. Also disables argument count and array bounds checking.

  Warning: Any type errors in code where type checks are not performed are liable to corrupt the heap.

  Used when (= safety 0).

4.2.2 Precise Type Checking ¶

Precise checking means that the check is done as though typep had been called with the exact type specifier that appeared in the declaration.

If a variable is declared to be (integer 3 17), then its value must always be an integer between 3 and 17. If multiple type declarations apply to a single variable, then all the declarations must be correct; it is as though all the types were intersected producing a single and type specifier.

To gain maximum benefit from the compiler’s type checking, you should always declare the types of function arguments and structure slots as precisely as possible. This often involves the use of or, member, and other list-style type specifiers.

4.2.3 Getting Existing Programs to Run ¶

Since SBCL’s compiler does much more comprehensive type checking than most Lisp compilers, SBCL may detect type errors in programs that have been debugged using other compilers. These errors are mostly incorrect declarations, although compile-time type errors can find actual bugs if parts of the program have never been tested.

Some incorrect declarations can only be detected by run-time type checking. It is very important to initially compile a program with full type checks (high safety optimization) and then test this safe version. After the checking version has been tested, then you can consider weakening or eliminating type checks. This applies even to previously debugged programs because the SBCL compiler does much more type inference than other Common Lisp compilers, so an incorrect declaration can do more damage.

The most common problem is with variables whose constant initial value doesn’t match the type declaration. Incorrect constant initial values will always be flagged by a compile-time type error, and they are simple to fix once located. Consider this code fragment:

(prog (foo)
  (declare (fixnum foo))
  (setq foo ...)
  ...)

Here foo is given an initial value of nil but is declared to be a fixnum. Even if it is never read, the initial value of a variable must match the declared type. There are two ways to fix this problem. Change the declaration

(prog (foo)
  (declare (type (or fixnum null) foo))
  (setq foo ...)
  ...)

or change the initial value

(prog ((foo 0))
  (declare (fixnum foo))
  (setq foo ...)
  ...)

It is generally preferable to change to a legal initial value rather than to weaken the declaration, but sometimes it is simpler to weaken the declaration than to try to make an initial value of the appropriate type.

Another declaration problem occasionally encountered is incorrect declarations on defmacro arguments. This can happen when a function is converted into a macro. Consider this macro:

(defmacro my-1+ (x)
  (declare (fixnum x))
  `(the fixnum (1+ ,x)))

Although legal and well-defined Common Lisp code, this meaning of this definition is almost certainly not what the writer intended. For example, this call is illegal:

(my-1+ (+ 4 5))

This call is illegal because the argument to the macro is (+ 4 5), which is a list, not a fixnum. Because of macro semantics, it is hardly ever useful to declare the types of macro arguments. If you really want to assert something about the type of the result of evaluating a macro argument, then put a the in the expansion:

(defmacro my-1+ (x)
  `(the fixnum (1+ (the fixnum ,x))))

In this case, it would be stylistically preferable to change this macro back to a function and declare it inline.

Some more subtle problems are caused by incorrect declarations that can’t be detected at compile time. Consider this code:

(do ((pos 0 (position #a string :start (1+ pos))))
  ((null pos))
  (declare (fixnum pos))
  ...)

Although pos is almost always a fixnum, it is nil at the end of the loop. If this example is compiled with full type checks (the default), then running it will signal a type error at the end of the loop. If compiled without type checks, the program will go into an infinite loop (or perhaps position will complain because (1+ nil) isn’t a sensible start.) Why? Because if you compile without type checks, the compiler just quietly believes the type declaration. Since the compiler believes that pos is always a fixnum, it believes that pos is never nil, so (null pos) is never true, and the loop exit test is optimized away. Such errors are sometimes flagged by unreachable code notes, but it is still important to initially compile and test any system with full type checks, even if the system works fine when compiled using other compilers.

In this case, the fix is to weaken the type declaration to (or fixnum null). (Actually, this declaration is unnecessary in SBCL, since it already knows that position returns a non-negative fixnum or nil.)

Note that there is usually little performance penalty for weakening a declaration in this way. Any numeric operations in the body can still assume that the variable is a fixnum, since nil is not a legal numeric argument. Another possible fix would be to say:

(do ((pos 0 (position #a string :start (1+ pos))))
    ((null pos))
  (let ((pos pos))
    (declare (fixnum pos))
    ...))

This would be preferable in some circumstances, since it would allow a non-standard representation to be used for the local pos variable in the loop body.

4.2.4 Implementation Limitations ¶

If an ftype is placed after the function definition the function won’t perform any type checks, and the calls to the function will blindly trust the declared types. (optimize (debug 3)) will not trust any ftype declarations.

4.4.1 Type Errors at Compile Time ¶

If the compiler can prove at compile time that some portion of the program cannot be executed without a type error, then it will give a warning at compile time.

It is possible that the offending code would never actually be executed at run-time due to some higher level consistency constraint unknown to the compiler, so a type warning doesn’t always indicate an incorrect program.

For example, consider this code fragment:

(defun raz (foo)
  (let ((x (case foo
              (:this 13)
              (:that 9)
              (:the-other 42))))
    (declare (fixnum x))
    (foo x)))

Compilation produces this warning:

; in: DEFUN RAZ
;     (CASE FOO (:THIS 13) (:THAT 9) (:THE-OTHER 42))
; --> LET COND IF COND IF COND IF
; ==>
;   (COND)
;
; caught WARNING:
;   This is not a FIXNUM:
;   NIL

In this case, the warning means that if foo isn’t any of :this, :that or :the-other, then x will be initialized to nil, which the fixnum declaration makes illegal. The warning will go away if ecase is used instead of case, or if :the-other is changed to t.

This sort of spurious type warning happens moderately often in the expansion of complex macros and in inline functions. In such cases, there may be dead code that is impossible to correctly execute. The compiler can’t always prove this code is dead (could never be executed), so it compiles the erroneous code (which will always signal an error if it is executed) and gives a warning.

4.4.2 Errors During Macroexpansion ¶

The compiler handles errors that happen during macroexpansion, turning them into compiler errors. If you want to debug the error (to debug a macro), you can set *break-on-signals* to error. For example, this definition:

(defun foo (e l)
  (do ((current l (cdr current))
       ((atom current) nil))
      (when (eq (car current) e) (return current))))

gives this error:

; in: DEFUN FOO
;     (DO ((CURRENT L (CDR CURRENT))
;        ((ATOM CURRENT) NIL))
;       (WHEN (EQ (CAR CURRENT) E) (RETURN CURRENT)))
;
; caught ERROR:
;   (in macroexpansion of (DO # #))
;   (hint: For more precise location, try *BREAK-ON-SIGNALS*.)
;   DO step variable is not a symbol: (ATOM CURRENT)

4.4.3 Read Errors ¶

SBCL’s compiler does not attempt to recover from read errors when reading a source file, but instead just reports the offending character position and gives up on the entire source file.

5.1.1 Debugger Banner ¶

When you enter the debugger, it looks something like this:

debugger invoked on a TYPE-ERROR in thread 11184:
  The value 3 is not of type LIST.

You can type HELP for debugger help, or (SB-EXT:QUIT) to exit from SBCL.

restarts (invokable by number or by possibly-abbreviated name):
  0: [ABORT   ] Reduce debugger level (leaving debugger, returning to toplevel).
  1: [TOPLEVEL] Restart at toplevel READ/EVAL/PRINT loop.
(CAR 1 3)
0]

The first group of lines describe what the error was that put us in the debugger. In this case car was called on 3, causing a type-error.

This is followed by the "beginner help line", which appears only if sb-debug:*debug-beginner-help-p* is true (default).

Next comes a listing of the active restart names, along with their descriptions – the ways we can restart execution after this error. In this case, both options return to top-level. Restarts can be selected by entering the corresponding number or name.

The current frame appears right underneath the restarts, immediately followed by the debugger prompt.

5.1.2 Debugger Invocation ¶

The debugger is invoked when:

• error is called, and the condition it signals is not handled.
• break is called, or signal is called with a condition that matches the current *break-on-signals*.
• The debugger is explicitly entered with the invoke-debugger function.

When the debugger is invoked by a condition, ANSI mandates that the value of *debugger-hook*, if any, be called with two arguments: the condition that caused the debugger to be invoked and the previous value of *debugger-hook*. When this happens, *debugger-hook* is bound to nil to prevent recursive errors. However, ANSI also mandates that *debugger-hook* not be invoked when the debugger is to be entered by the break function. For users who wish to provide an alternate debugger interface (and thus catch break entries into the debugger), SBCL provides sb-ext:*invoke-debugger-hook*, which is invoked during any entry into the debugger.

Variable: sb-ext:*invoke-debugger-hook* ¶

This is either nil or a designator for a function of two arguments, to be run when the debugger is about to be entered. The function is run with *invoke-debugger-hook* bound to nil to minimize recursive errors, and receives as arguments the condition that triggered debugger entry and the previous value of *invoke-debugger-hook*.

This mechanism is an SBCL extension similar to the standard *debugger-hook*. In contrast to *debugger-hook*, it is observed by invoke-debugger even when called by break.

5.3.1 Stack Motion ¶

These commands move to a new stack frame and print the name of the function and the values of its arguments in the style of a Lisp function call:

• up: Move up to the next higher frame. More recent function calls are considered to be higher on the stack.
• down: Move down to the next lower frame.
• top: Move to the highest frame, that is, the frame where the debugger was entered.
• bottom: Move to the lowest frame.
• frame [<n>]: Move to the frame with the specified number. Prompts for the number if not supplied. The frame with number 0 is the frame where the debugger was entered.

5.3.2 How Arguments are Printed ¶

A frame is printed to look like a function call, but with the actual argument values in the argument positions. So the frame for this call in the source:

(myfun (+ 3 4) 'a)

would look like this:

(MYFUN 7 A)

All keyword and optional arguments are displayed with their actual values; if the corresponding argument was not supplied, the value will be the default. So this call:

(subseq "foo" 1)

would look like this:

(SUBSEQ "foo" 1 3)

And this call:

(string-upcase "test case")

would look like this:

(STRING-UPCASE "test case" :START 0 :END NIL)

The arguments to a function call are displayed by accessing the argument variables. Although those variables are initialized to the actual argument values, they can be set inside the function; in this case the new value will be displayed.

&rest arguments are handled somewhat differently. The value of the rest argument variable is displayed as the spread-out arguments to the call, so:

(format t "~A is a ~A." "This" 'test)

would look like this:

(FORMAT T "~A is a ~A." "This" 'TEST)

Rest arguments cause an exception to the normal display of keyword arguments in functions that have both &rest and &key arguments. In this case, the keyword argument variables are not displayed at all; the rest arg is displayed instead. So for these functions, only the keywords actually supplied will be shown, and the values displayed will be the argument values, not values of the (possibly modified) variables.

If the variable for an argument is never referenced by the function, it will be deleted. The variable value is then unavailable, so the debugger prints #<unused-arg> instead of the value. Similarly, if for any of a number of reasons the value of the variable is unavailable or not known to be available (Variable Access), then #<unavailable-arg> will be printed instead of the argument value.

Note that inline expansion and open-coding affect what frames are present in the debugger, see Debugger Policy Control.

5.3.3 Function Names ¶

If a function is defined by defun it will appear in backtrace by that name. Functions defined by labels and flet will appear as (FLET <name>) and (LABELS <name>) respectively. Anonymous lambdas will appear as (LAMBDA <lambda-list>).

• Entry Point Details

5.3.4 Debug Tail Recursion ¶

The compiler is properly tail recursive. If a function call is in a tail-recursive position, the stack frame will be deallocated at the time of the call, rather than after the call returns. Consider this backtrace:

(BAR ...)
(FOO ...)

Because of tail recursion, it is not necessarily the case that foo directly called bar. It may be that foo called some other function foo2, which then called bar tail-recursively, as in this example:

(defun foo ()
  ...
  (foo2 ...)
  ...)

(defun foo2 (...)
  ...
  (bar ...))

(defun bar (...)
  ...)

Usually the elimination of tail-recursive frames makes debugging more pleasant, since these frames are mostly uninformative. If there is any doubt about how one function called another, it can usually be eliminated by finding the source location in the calling frame. See Source Location Printing.

The elimination of tail-recursive frames can be prevented by disabling tail-recursion optimization, which happens when the debug optimization quality is greater than 2. See Debugger Policy Control.

5.3.5 Unknown Locations and Interrupts ¶

The debugger operates using special debugging information attached to the compiled code. This debug information tells the debugger what it needs to know about the locations in the code where the debugger can be invoked. If the debugger somehow encounters a location not described in the debug information, then it is said to be unknown. If the code location for a frame is unknown, then some variables may be inaccessible, and the source location cannot be precisely displayed.

There are three reasons why a code location could be unknown:

• There is inadequate debug information due to the value of the debug optimization quality. See Debugger Policy Control.
• The debugger was entered because of an interrupt such as C-c.
• A hardware error such as a bus error occurred in code that was compiled unsafely due to the value of the safety optimization quality.

In the last two cases, the values of argument variables are accessible, but may be incorrect. For more details on when variable values are accessible, see Variable Value Availability.

It is possible for an interrupt to happen when a function call or return is in progress. The debugger may then flame out with some obscure error or insist that the bottom of the stack has been reached, when the real problem is that the current stack frame can’t be located. If this happens, return from the interrupt and try again.

5.4.1 Variable Value Availability ¶

The value of a variable may be unavailable to the debugger in portions of the program where Lisp says that the variable is defined. If a variable value is not available, the debugger will not let you read or write that variable. With one exception, the debugger will never display an incorrect value for a variable. Rather than displaying incorrect values, the debugger tells you the value is unavailable.

The one exception is this: if you interrupt (e.g. with C-c) or if there is an unexpected hardware error such as a bus error (which should only happen in unsafe code), then the values displayed for arguments to the interrupted frame might be incorrect. This exception applies only to the interrupted frame: any frame farther down the stack will be fine.

Note: Since the location of an interrupt or hardware error will always be an unknown location, non-argument variable values will never be available in the interrupted frame. See Unknown Locations and Interrupts.)

The value of a variable may be unavailable for these reasons:

• The value of the debug optimization quality may have omitted debug information needed to determine whether the variable is available. Unless a variable is an argument, its value will only be available when debug is at least 2.
• The compiler did lifetime analysis and determined that the value was no longer needed, even though its scope had not been exited. Lifetime analysis is inhibited when the debug optimization quality is 3.
• The variable’s name is an uninterned symbol (gensym). To save space, the compiler only dumps debug information about uninterned variables when the debug optimization quality is 3.
• The frame’s location is unknown (see Unknown Locations and Interrupts) because the debugger was entered due to an interrupt or unexpected hardware error. Under these conditions the values of arguments will be available, but might be incorrect. This is the exception mentioned above.
• The variable (or the code referencing it) was optimized out of existence. Variables with no reads are always optimized away. The degree to which the compiler deletes variables will depend on the value of the compilation-speed optimization quality, but most source-level optimizations are done under all compilation policies.
• The variable is never set and its definition looks like

  (LET ((var1 var2))
     ...)
  

  In this case, var1 is substituted with var2.

• The variable is never set and is referenced exactly once. In this case, the reference is substituted with the variable initial value.

Since it is especially useful to be able to get the arguments to a function, argument variables are treated specially when the speed optimization quality is less than 3 and the debug quality is at least 1. With this compilation policy, the values of argument variables are almost always available everywhere in the function, even at unknown locations. For non-argument variables, debug must be at least 2 for values to be available, and even then, values are only available at known locations.

5.4.2 Note On Lexical Variable Access ¶

When the debugger command loop establishes variable bindings for available variables, these variable bindings have lexical scope and dynamic extent. You can close over them, but such closures can’t be used as upward function arguments.

Note: The variable bindings are actually created using the Lisp symbol-macrolet special form.

You can also set local variables using setq, but if the variable was closed over in the original source and never set, then setting the variable in the debugger may not change the value in all the functions the variable is defined in. Another risk of setting variables is that you may assign a value of a type that the compiler proved the variable could never take on. This may result in bad things happening.

5.5.1 How the Source is Found ¶

If the code was defined from Lisp by compile or eval, then the source can always be reliably located. If the code was defined from a FASL file created by compile-file, then the debugger gets the source forms it prints by reading them from the original source file. This is a potential problem, since the source file might have moved or changed since the time it was compiled.

The source file is opened using the truename of the source file pathname originally given to the compiler. This is an absolute pathname with all logical names and symbolic links expanded. If the file can’t be located using this name, then the debugger gives up and signals an error.

If the source file can be found, but has been modified since the time it was compiled, the debugger prints this warning:

; File has been modified since compilation:
;   <filename>
; Using form offset instead of character position.

where <filename> is the name of the source file. It then proceeds using a robust but not foolproof heuristic for locating the source. This heuristic works if:

• No top-level forms before the top-level form containing the source have been added or deleted, and
• the top-level form containing the source has not been modified much. (More precisely, none of the list forms beginning before the source form have been added or deleted.)

If the heuristic doesn’t work, the displayed source will be wrong, but will probably be near the actual source. If the "shape" of the top-level form in the source file is too different from the original form, then an error will be signaled. When the heuristic is used, the source location commands are noticeably slowed.

Source location printing can also be confused if (after the source was compiled) a read-macro you used in the code was redefined to expand into something different, or if a read-macro ever returns the same eq list twice. If you don’t define read macros and don’t use ## in perverted ways, you don’t need to worry about this.

5.5.2 Source Location Availability ¶

Source location information is only available when the debug optimization quality is at least 2. If source location information is unavailable, the source commands will give an error message.

If source location information is available, but the source location is unknown because of an interrupt or unexpected hardware error (see Unknown Locations and Interrupts), then the command will print

Unknown location: using block start.

and then proceed to print the source location for the start of the basic block enclosing the code location. It’s a bit complicated to explain exactly what a basic block is, but here are some properties of the block start location:

• The block start location may be the same as the true location.
• The block start location will never be later in the program’s flow of control than the true location.
• No conditional control structures (such as if, cond, or) will intervene between the block start and the true location (but note that some conditionals present in the original source could be optimized away.) Function calls do not end basic blocks.
• The head of a loop will be the start of a block.
• The programming language concept of block structure and the Lisp block special form are totally unrelated to the compiler’s basic block.

In other words, the true location lies between the printed location and the next conditional (but watch out because the compiler may have changed the program on you.)

5.9.1 Breakpoint Example ¶

Consider this definition of the factorial function:

(defun ! (n)
  (if (zerop n)
      1
      (* n (! (1- n)))))

This debugger session demonstrates the use of breakpoints:

• (break) ; invoke debugger

debugger invoked on a SIMPLE-CONDITION in thread 11184: break

restarts (invokable by number or by possibly-abbreviated name):
  0: [CONTINUE] Return from BREAK.
  1: [ABORT   ] Reduce debugger level (leaving debugger, returning to toplevel).
  2: [TOPLEVEL] Restart at toplevel READ/EVAL/PRINT loop.
("varargs entry for top level local call BREAK" "break")
0] ll #'!

0-1: (SB-INT:NAMED-LAMBDA ! (N) (BLOCK ! (IF (ZEROP N) 1 (* N (! #)))))
2: (BLOCK ! (IF (ZEROP N) 1 (* N (! (1- N)))))
3: (ZEROP N)
4: (* N (! (1- N)))
5: (1- N)
6: (! (1- N))
7-8: (* N (! (1- N)))
9-10: (IF (ZEROP N) 1 (* N (! (1- N))))
0] br 4

(* N (! (1- N)))
1: 4 in !
added
0] toplevel

> (! 10) ; Call the function

*Breakpoint hit*

Restarts:
  0: [CONTINUE] Return from BREAK.
  1: [ABORT   ] Return to Top-Level.

Debug  (type H for help)

(! 10) ; We are now in first call (arg 10) before the multiply
Source: (* N (! (1- N)))
3] step*

*Step*

(! 10) ; We have finished evaluation of (1- n)
Source: (1- N)
3] step*

*Breakpoint hit*

Restarts:
  0: [CONTINUE] Return from BREAK.
  1: [ABORT   ] Return to Top-Level.

Debug  (type H for help)

(! 9) ; We hit the breakpoint in the recursive call
Source: (* N (! (1- N)))
3]

Note: The step* command differs from the single stepping commands in that it also functions in compiled code which has not been compiled with stepping instrumentation. It simply steps to the next compiled code location. In the future, this form of stepping may be improved enough to subsume the instrumentation based stepping commands, which have much higher overhead.

6.1.1 Structure Object Slot Access ¶

Structure slot accessors are efficient only if the compiler is able to open code them: compiling a call to a structure slot accessor before the structure is defined, declaring one notinline, or passing it as a functional argument to another function causes severe performance degradation.

6.1.2 Standard Object Slot Access ¶

The most efficient way to access a slot of a standard-object is by using slot-value with a constant slot name argument inside a defmethod body, where the variable holding the instance is a specializer parameter of the method and is never assigned to. The cost is roughly 1.6 times that of an open coded structure slot accessor.

Second most efficient way is to use a CLOS slot accessor, or slot-value with a constant slot name argument, but in circumstances other than specified above. This may be up to 3 times as slow as the method described above.

Example:

(defclass foo () ((bar)))

;; Fast: specializer and never assigned to
(defmethod quux ((foo foo) new)
  (let ((old (slot-value foo 'bar)))
    (setf (slot-value foo 'bar) new)
    old))

;; Slow: not a specializer
(defmethod quux ((foo foo) new)
  (let* ((temp foo)
         (old (slot-value temp 'bar)))
    (setf (slot-value temp 'bar) new)
    old))

;; Slow: assignment to FOO
(defmethod quux ((foo foo) new)
  (let ((old (slot-value foo 'bar)))
    (setf (slot-value foo 'bar) new)
    (setf foo new)
    old))

Note that when profiling code such as this, the first few calls to the generic function are not representative, as the dispatch mechanism is lazily set up during those calls.

6.3.1 Signed Modular Arithmetic ¶

Sign-extending the result in the following way will be translated into signed modular arithmetic:

(defun add (a b)
  (declare (type (signed-byte 64) a b))
  (let ((u (ldb (byte 64 0) (+ a b))))
    (logior u (- (mask-field (byte 1 63) u)))))

6.4.1 Count Trailing Zeros ¶

(defun ctz (n)
  (declare (type (unsigned-byte 64) n))
  (integer-length (ldb (byte 64 0) (lognor n (- n)))))

is turned into hardware instructions on arm64 and x86-64. It returns 64 when n is 0. n can also be (signed-byte 64) or fixnum.

7.1.1 Extended Package Prefix Syntax ¶

SBCL supports extended package prefix syntax, which allows specifying an alternate package instead of *package* for the reader to use as the default package for interning symbols:

<package-name>::<form-with-interning-into-package>

Example:

'foo::(bar quux zot) == '(foo::bar foo::quux foo::zot)

*package* is not rebound during the course of reading a form with extended package prefix syntax; if foo::bar would cause a read-time package lock violation, so does foo::(bar).

7.1.2 Symbol Name Normalization ¶

SBCL also extends the reader to normalize all symbols to Normalization Form KC in builds with Unicode enabled. Whether symbols are normalized is controlled by

Function: sb-ext:readtable-normalization readtable ¶

Returns t if readtable normalizes symbols to NFKC, and nil otherwise. The readtable-normalization of the standard readtable is t.

Symbols created by intern and similar functions are not affected by this setting. If sb-ext:readtable-normalization is t, symbols that are not normalized are escaped during printing.

7.1.3 Decimal Syntax for Rationals ¶

SBCL supports a decimal syntax for rationals, modelled after the standard syntax for floating-point numbers. If a number with floating-point syntax has an exponent marker of r or r (rather than one of the standard exponent markers), it is read as the rational with the exact value of the decimal number expressed as a float.

In addition, setting or binding the value of *read-default-float-format* to rational around a call to read or read-from-string has the effect that floating-point numbers without exponent markers are read as rational numbers, as if there had been an explicit r or r marker.

Floating point numbers of all types are printed with an exponent marker while the value of *read-default-float-format* is rational; however, rational numbers are printed in their standard syntax, irrespective of the value of *read-default-float-format*.

7.4.1 Finalization ¶

Finalization allows code to be executed after an object has been garbage collected. This is useful for example for releasing foreign memory associated with a Lisp object.

Function: sb-ext:finalize object function &key dont-save ¶

Arrange for the designated function to be called when there are no more references to object, including references in function itself.

If dont-save is true, the finalizer will be cancelled when save-lisp-and-die is called: this is useful for finalizers deallocating system memory, which might otherwise be called with addresses from the old image.

In a multithreaded environment function may be called in any thread. In both single and multithreaded environments function may be called in any dynamic scope: consequences are unspecified if function is not fully re-entrant.

Errors from function are handled and cause a warning to be signalled in whichever thread the function was called in.

Examples:

;;; GOOD, assuming RELEASE-HANDLE is re-entrant.
(let* ((handle (get-handle))
       (object (make-object handle)))
 (finalize object (lambda () (release-handle handle)))
 object)

;;; BAD, finalizer refers to object being finalized, causing
;;; it to be retained indefinitely!
(let* ((handle (get-handle))
       (object (make-object handle)))
  (finalize object
            (lambda ()
              (release-handle (object-handle object)))))

;;; BAD, not re-entrant!
(defvar *rec* nil)

(defun oops ()
 (when *rec*
   (error "recursive OOPS"))
 (let ((*rec* t))
   (gc))) ; or just cons enough to cause one

(progn
  (finalize "oops" #'oops)
  (oops)) ; GC causes re-entry to #'oops due to the finalizer
          ; -> ERROR, caught, WARNING signalled

Function: sb-ext:cancel-finalization object ¶

Cancel all finalizations for object, returning t if it had a finalizer.

7.4.2 Weak Pointers ¶

Weak pointers allow references to objects to be maintained without keeping them from being garbage collected: useful for building caches among other things.

Hash tables can also have weak keys and values. See Hash Table Extensions.

Function: sb-ext:make-weak-pointer object ¶

Allocate and return a weak pointer which points to object.

Function: sb-ext:weak-pointer-value weak-pointer ¶

If weak-pointer is valid, return the value of weak-pointer and t. If the referent of weak-pointer has been garbage collected, returns the values nil and nil.

7.4.3 Introspection and Tuning ¶

Variable: sb-ext:*gc-run-time* ¶

Total CPU time spent doing garbage collection (as reported by get-internal-run-time.) Initialized to zero on startup. It is safe to bind this to zero in order to measure gc time inside a certain section of code, but doing so may interfere with results reported by eg. time.

Variable: sb-ext:*gc-real-time* ¶

Total real time spent doing garbage collection (as reported by get-internal-real-time.) Initialized to zero on startup.

Function: sb-ext:bytes-consed-between-gcs ¶

The amount of memory that will be allocated before the next garbage collection is initiated. This can be set with setf.

On GENCGC platforms this is the nursery size, and defaults to 5% of dynamic space size.

Note that currently, changes to this value are lost when saving core.

Function: sb-ext:dynamic-space-size ¶

Size of the dynamic space in bytes.

Function: sb-ext:get-bytes-consed ¶

Return the number of bytes consed since the program began. Typically this result will be a consed bignum, so if you have an application (e.g. profiling) which can’t tolerate the overhead of consing bignums, you’ll probably want either to hack in at a lower level (as the code in the sb-profile package does), or to design a more microefficient interface and submit it as a patch.

Function: sb-ext:gc-logfile ¶

Return the pathname used to log garbage collections. Can be setf. Default is nil, meaning collections are not logged. If non-null, the designated file is opened before and after each collection, and generation statistics are appended to it.

Function: sb-ext:generation-average-age generation ¶

Average age of memory allocated to GENERATION: average number of times objects allocated to the generation have seen younger objects promoted to it. Available on GENCGC platforms only.

Experimental: interface subject to change.

Function: sb-ext:generation-bytes-allocated generation ¶

Number of bytes allocated to GENERATION currently. Available on GENCGC platforms only.

Experimental: interface subject to change.

Function: sb-ext:generation-bytes-consed-between-gcs generation ¶

Number of bytes that can be allocated to GENERATION before that generation is considered for garbage collection. This value is meaningless for generation 0 (the nursery): see bytes-consed-between-gcs instead. Default is 5% of the dynamic space size divided by the number of non-nursery generations. Can be assigned to using setf. Available on GENCGC platforms only.

Experimental: interface subject to change.

Function: sb-ext:generation-minimum-age-before-gc generation ¶

Minimum average age of objects allocated to GENERATION before that generation is may be garbage collected. Default is 0.75. See also generation-average-age. Can be assigned to using setf. Available on GENCGC platforms only.

Experimental: interface subject to change.

Function: sb-ext:generation-number-of-gcs-before-promotion generation ¶

Number of times garbage collection is done on GENERATION before automatic promotion to the next generation is triggered. Default is 1. Can be assigned to using setf. Available on GENCGC platforms only.

Experimental: interface subject to change.

Function: sb-ext:generation-number-of-gcs generation ¶

Number of times garbage collection has been done on GENERATION without promotion. Available on GENCGC platforms only.

Experimental: interface subject to change.

7.4.4 Tracing Live Objects Back to Roots ¶

This feature is intended to help expert users diagnose rare low-level issues and should not be needed during normal usage. On top of that, the interface and implementation are experimental and may change at any time without further notice.

It is sometimes important to understand why a given object is retained in the Lisp image instead of being garbage collected. To help with this problem, SBCL provides a mechanism that searches through the different memory spaces, builds a path of references from a root to the object in question and finally reports this paths:

Function: sb-ext:search-roots weak-pointers &key criterion ignore print ¶

Find roots keeping the targets of weak-pointers alive.

weak-pointers must be a single sb-ext:weak-pointer or a list of those, pointing to objects for which roots should be searched.

criterion determines just how rooty (how deep) a root must be in order to be considered. Possible values are:

• :oldest

  This says we can stop upon seeing an object in the oldest gen to gc, or older. This is the easiest test to satisfy.

• :pseudo-static

  This is usually the same as :oldest, unless the oldest gen to gc has been decreased.

• :static

  To find a root of an image-backed object, you want to stop only at a truly :static object.

ignore is a list of objects to treat as if nonexistent in the heap. It can often be useful for finding a path to an interned symbol other than through its package by specifying the package as an ignored object.

print controls whether discovered paths should be returned or printed. Possible values are

• :verbose

  Return no values. Print discovered paths using a verbose format with each node of each path on a separate line.

• true (other than :verbose)

  Return no values. Print discovered paths using a compact format with all nodes of each path on a single line.

• nil

  Do not print any output. Instead return the discovered paths as a list of lists. Each list has the form

  (TARGET . (ROOT NODE*))
  

  where target is one of the target of one of the weak-pointers.

  root is a description of the root at which the path starts and has one of the following forms:

  • :static

    If the root of the path is a non-collectible heap object.

  • :pinned

    If an unknown thread stack pins the root of the path.

  • ((thread-name | thread-object) symbol currentp)

    If the path begins at a special binding of symbol in a thread. currentp is a boolean indicating whether the value is current or shadowed by another binding.

  • ((thread-name | thread-object) guessed-pc)

    If the path begins at a lexical variable in the function whose code contains guessed-pc.

  Each node in the remainder of the path is a cons (object . slot) indicating that the slot at index slot in object references the next path node.

Experimental: subject to change without prior notice.

An example of using this could look like this:

• (defvar *my-string* (list 1 2 "my string"))

*MY-STRING*

• (sb-ext:search-roots (sb-ext:make-weak-pointer (third *my-string*)))

-> ((simple-vector 3)) #x10004E9EAF[2] -> (symbol) #x5044100F[1] -> (cons) #x100181FAE7[1] -> (cons) #x100181FAF7[1] -> (cons) #x100181FB07[0] -> #x100181F9AF

The single line of output on *standard-output* shows the path from a root to "my string": the path starts with SBCL’s internal package system data structures followed by the symbol (cl-user:*my-string*) followed the three cons cells of the list.

The :print :verbose argument produces similar behavior but describes the path elements in more detail:

• (sb-ext:search-roots (sb-ext:make-weak-pointer (third *my-string*)) :print :verbose)

Path to "my string": 6 10004E9EAF [ 2] a (simple-vector 3) 0 5044100F [ 1] COMMON-LISP-USER::*MY-STRING* 0 100181FAE7 [ 1] a cons 0 100181FAF7 [ 1] a cons 0 100181FB07 [ 0] a cons

The :print nil argument is a bit different:

• (sb-ext:search-roots (sb-ext:make-weak-pointer (third *my-string*)) :print nil)

(("my string" :static (#(*MY-STRING* 0 0) . 2) (*MY-STRING* . 1) ((1 2 "my string") . 1) ((2 "my string") . 1) (("my string") . 0)))

There is no output on *standard-output*, and the return value is a single path for the target object "my string". As before, the path shows the symbol and the three cons cells.

7.7.1 AMOP Compatibility of Metaobject Protocol ¶

SBCL supports a metaobject protocol which is intended to be compatible with AMOP; present exceptions to this (as distinct from current bugs) are:

• sb-mop:compute-effective-method only returns one value, not two. There is no record of what the second return value was meant to indicate, and apparently no clients for it.
• The direct superclasses of sb-mop:funcallable-standard-object are (function standard-object) instead of the correct (standard-object function).

  This is to ensure that the standard-object class is the last of the standardized classes before class t appearing in the precedence list of generic-function and standard-generic-function, as required by clhs 1.4.4.5.

• The arguments :declare and :declarations are both accepted by ensure-generic-function, with the leftmost argument defining the declarations to be stored and returned by sb-mop:generic-function-declarations.

  Where AMOP specifies :declarations as the keyword argument to ensure-generic-function, the Common Lisp standard specifies :declare. Portable code should use :declare.

• Although SBCL obeys the requirement in AMOP that sb-mop:validate-superclass should treat standard-class and sb-mop:funcallable-standard-class as compatible metaclasses, we impose an additional requirement at class finalization time: a class of metaclass sb-mop:funcallable-standard-class must have function in its superclasses, and a class of metaclass standard-class must not.

  After a class has been finalized, it is associated with a class prototype which is accessible by a standard MOP function sb-mop:class-prototype. The user can then ask whether this object is a function or not in several different ways: whether it is a function according to typep; whether its class-of is subtypep function, or whether function appears in the superclasses of the class. The additional consistency requirement comes from the desire to make all of these answers the same.

  The following class definitions are bad, and will lead to errors either immediately or if an instance is created:

  (defclass bad-object (funcallable-standard-object)
    ()
    (:metaclass standard-class))
  (defclass bad-funcallable-object (standard-object)
    ()
    (:metaclass funcallable-standard-class))
  

  The following definition is acceptable:

  (defclass mixin ()
    ((slot :initarg slot)))
  (defclass funcallable-object (funcallable-standard-object mixin)
    ()
    (:metaclass funcallable-standard-class))
  

  and leads to a class whose instances are funcallable and have one slot.

  Note that this requirement also applies to the class sb-mop:funcallable-standard-object, which has metaclass sb-mop:funcallable-standard-class rather than standard-class as AMOP specifies.

• The requirement that no portable class may inherit, by virtue of being a direct or indirect subclass of a specified class, any slot for which the name is a symbol accessible in the common-lisp-user package or exported by any package defined in the ANSI Common Lisp standard. is interpreted to mean that the standardized classes themselves should not have slots named by external symbols of public packages.

  The rationale behind the restriction is likely to be similar to the ANSI Common Lisp restriction on defining functions, variables and types named by symbols in the Common Lisp package: preventing two independent pieces of software from colliding with each other.

• Specializations of the new-value argument to (setf sb-mop:slot-value-using-class) are not allowed: all user-defined methods must have a specializer of the class t.

  This prohibition is motivated by a separation of layers: the sb-mop:slot-value-using-class family of functions is intended for use in implementing different and new slot allocation strategies, rather than in performing application-level dispatching. Additionally, with this requirement, there is a one-to-one mapping between metaclass, class and slot-definition-class tuples and effective methods of (setf sb-mop:slot-value-using-class), which permits optimization of (setf sb-mop:slot-value-using-class)’s discriminating function in the same manner as for sb-mop:slot-value-using-class and sb-mop:slot-boundp-using-class.

  Note that application code may specialize on the new-value argument of slot accessors.

• The class named by the name argument to sb-mop:ensure-class, if any, is only redefined if it is the proper name of that class; otherwise, a new class is created.

  This is consistent with the description sb-mop:ensure-class in AMOP as the functional version of defclass, which has this behaviour; however, it is not consistent with the weaker requirement in AMOP, which states that any class found by find-class, no matter what its class-name, is redefined.

• An error is not signaled in the case of the :name initialization argument for sb-mop:slot-definition objects being a constant, when the slot definition is of type sb-pcl::structure-slot-definition (i.e. it is associated with a class of type structure-class).

  This allows code which uses constant names for structure slots to continue working as specified in ANSI, while enforcing the constraint for all other types of slot.

• The class t is not an instance of the built-in-class metaclass.

  AMOP specifies, in the _Inheritance Structure of Metaobject Classes_ section, that the class t should be an instance of built-in-class. However, it also specifies that sb-mop:validate-superclass should return true (indicating that a direct superclass relationship is permissible) if the second argument is the class t. Also, ANSI specifies that classes with metaclass built-in-class may not be subclassed using defclass, and also that the class t is the universal superclass, inconsistent with it being a built-in-class.

• Uses of change-class and redefinitions of classes with defclass (or the functional interfaces sb-mop:ensure-class or sb-mop:ensure-class-using-class) must ensure that for each slot with allocation :instance or :class, the set of applicable methods on the sb-mop:slot-value-using-class family of generic functions is the same before and after the change.

  This is required for correct operation of the protocol to update instances for the new or redefined class, and can be seen as part of the contract of the :instance or :class allocations.

• Metaobject protocol users may wish to override sb-mop:compute-discriminating-function for their own generic function classes. Overriding implementations of sb-mop:compute-discriminating-function must, in order to participate in the no-applicable-method and sb-pcl:no-primary-method protocols, perform appropriate checks on the return value of compute-applicable-methods before processing the effective method; the standard effective method contains error-invoking forms, but those forms have no access to the generic function invocation’s arguments.

7.7.2 Metaobject Protocol Extensions ¶

In addition, SBCL supports extensions to the Metaobject protocol from AMOP; at present, they are:

• Compile-time support for generating specializer metaobjects from specializer names in defmethod forms is provided by the sb-pcl:make-method-specializers-form function, which returns a form which, when evaluated in the lexical environment of the defmethod, returns a list of specializer metaobjects. This operator suffers from similar restrictions to those affecting sb-mop:make-method-lambda, namely that the generic function must be defined when the defmethod form is expanded, so that the correct method of sb-pcl:make-method-specializers-form is invoked. The system-provided method on sb-pcl:make-method-specializers-form generates a call to find-class for each symbol specializer name, and a call to sb-mop:intern-eql-specializer for each (EQL <x>) specializer name.
• Run-time support for converting between specializer names and specializer metaobjects, mostly for the purposes of find-method, is provided by sb-pcl:parse-specializer-using-class and sb-pcl:unparse-specializer-using-class, which dispatch on their first argument, the generic function associated with a method with the given specializer. The system-provided methods on those methods convert between classes and proper names and between lists of the form (EQL <x>) and interned eql specializer objects.
• Distinguishing unbound instance allocated slots from bound ones when using sb-mop:standard-instance-access and sb-mop:funcallable-standard-instance-access is possible by comparison to the symbol-macro sb-pcl:+slot-unbound+.

7.8.1 Iterator Protocol ¶

The general iterator protocol allows subsequently accessing some or all elements of a sequence in forward or reverse direction. Users first call sb-sequence:make-sequence-iterator to create an iteration state and receive functions to query and mutate it. These functions allow, among other things, moving to, retrieving or modifying elements of the sequence. The iteration state consists of a state object, a limit object, a from-end indicator and six functions to query or mutate this state.

An iterator is created by calling:

Function: sb-sequence:make-sequence-iterator sequence &key from-end start end ¶

Returns a sequence iterator for sequence or, if start and/or end are supplied, the subsequence bounded by start and end as nine values:

1. iterator state 2. limit 3. from-end 4. step function 5. endp function 6. element function 7. setf element function 8. index function 9. copy state function

If from-end is nil, the constructed iterator visits the specified elements in the order in which they appear in sequence. Otherwise, the elements are visited in the opposite order.

The six functions (items 4-9 in the list) have the same contract as the generic functions described in Simple Iterator Protocol. In fact, when there is no specialized method for a particular sequence subclass, sb-sequence:make-sequence-iterator calls sb-sequence:make-simple-sequence-iterator and returns those six generic functions.

The following convenience macros simplify traversing sequences using iterators:

Macro: sb-sequence:with-sequence-iterator (&optional iterator limit from-end-p step endp element set-element index copy) (sequence &key from-end start end) &body body ¶

Executes body with the elements of vars bound to the iteration state returned by sequence:make-sequence-iterator for sequence and args. Elements of vars may be nil in which case the corresponding value returned by sequence:make-sequence-iterator is ignored.

Macro: sb-sequence:with-sequence-iterator-functions (&optional step endp elt setf index copy) (sequence &rest args &key from-end start end) &body body ¶

Executes body with the names step, endp, elt, setf, index and copy bound to local functions which execute the iteration state query and mutation functions returned by sequence:make-sequence-iterator for sequence and args. When some names are not supplied or nil is supplied for a given name, no local functions are established for those names. The functions established for step, endp, elt, setf, index and copy have dynamic extent.

7.8.2 Simple Iterator Protocol ¶

For cases in which the full flexibility and performance of the general sequence iterator protocol is not required, there is a simplified sequence iterator protocol consisting of a few generic functions which can be specialized for iterator classes:

Generic function: sb-sequence:iterator-step sequence iterator from-end ¶

Moves iterator one position forward or backward in sequence depending on the iteration direction encoded in from-end.

Generic function: sb-sequence:iterator-endp sequence iterator limit from-end ¶

Returns non-nil when iterator has reached limit (which may correspond to the end of sequence) with respect to the iteration direction encoded in from-end.

Generic function: sb-sequence:iterator-element sequence iterator ¶

Returns the element of sequence associated to the position of iterator.

Setf generic function: sb-sequence:iterator-element sequence iterator ¶

Returns the element of sequence associated to the position of iterator.

Generic function: sb-sequence:iterator-index sequence iterator ¶

Returns the position of iterator in sequence.

Generic function: sb-sequence:iterator-copy sequence iterator ¶

Returns a copy of iterator which also traverses sequence but can be mutated independently of iterator.

Iterator objects implementing the above simple iteration protocol are created by calling the following generic function:

Generic function: sb-sequence:make-simple-sequence-iterator sequence &key from-end start end ¶

Returns a sequence iterator for sequence, start, end and from-end as three values:

1. iterator state 2. limit 3. from-end

The returned iterator can be used with the generic iterator functions described in Simple Iterator Protocol.

7.9.1 Running external programs ¶

External programs can be run with sb-ext:run-program.

Note: In SBCL versions prior to 1.0.13, sb-ext:run-program searched for executables in a manner somewhat incompatible with other languages. As of this version, SBCL uses the system library routine execvp(3), and no longer contains the function find-executable-in-search-path, which implemented the old search. Users who need this function may find it in run-program.lisp versions 1.67 and earlier in SBCL’s CVS repository here http://sbcl.cvs.sourceforge.net/sbcl/sbcl/src/code/run-program.lisp?view=log. However, we caution such users that this search routine finds executables that system library routines do not.

Function: sb-ext:run-program program args &key env environment wait search pty input if-input-does-not-exist output if-output-exists error if-error-exists status-hook external-format directory preserve-fds use-posix-spawn ¶

run-program creates a new process specified by program. args is a list of strings to be passed literally to the new program. In POSIX environments, this list becomes the array supplied as the second parameter to the execv() or execvp() system call, each list element becoming one array element. The strings should not contain shell escaping, as there is no shell involvement. Further note that while conventionally the process receives its own pathname in argv[0], that is automatic, and the 0th string should not be present in args.

The program arguments and the environment are encoded using the default external format for streams.

run-program will return a process structure. See the CMU Common Lisp Users Manual for details about the process structure.

Notes about Unix environments (as in the :environment and :env args):

• The SBCL implementation of run-program, like Perl and many other programs, but unlike the original CMU CL implementation, copies the Unix environment by default.
• Running Unix programs from a setuid process, or in any other situation where the Unix environment is under the control of someone else, is a mother lode of security problems. If you are contemplating doing this, read about it first. (The Perl community has a lot of good documentation about this and other security issues in script-like programs.)

The &key arguments have the following meanings:

• :environment

  A list of strings describing the new Unix environment (as in "man environ"). The default is to copy the environment of the current process.

• :env An alternative lossy representation of the new Unix environment, for compatibility with CMU CL.
• :search

  Look for program in each of the directories in the child’s $PATH environment variable. Otherwise an absolute pathname is required.

• :wait

  If non-nil (default), wait until the created process finishes. If nil, continue running Lisp until the program finishes.

• :pty (not supported on win32)

  Either t, nil, or a stream. Unless nil, the subprocess is established under a pty. If :pty is a stream, all output to this pty is sent to this stream, otherwise the process-pty slot is filled in with a stream connected to pty that can read output and write input.

• :input

  Either t, nil (the default), a pathname, a stream, or :stream.

  • t: the standard input for the current process is inherited.
  • nil: /dev/null (nul on win32) is used.
  • Pathname: the specified file is used.
  • Stream: all the input is read from that stream and sent to the subprocess.
  • :stream: the process-input slot is filled in with a stream that sends its output to the process.
• :if-input-does-not-exist (when :input is the name of a file)

  It is one of:

  • :error to generate an error
  • :create to create an empty file
  • nil (the default) to return nil from run-program
• :output

  Either t, nil (the default), a pathname, a stream, or :stream.

  • t: the standard output for the current process is inherited.
  • nil: /dev/null (nul on win32) is used.
  • Pathname: the specified file is used.
  • Stream: all the output from the process is written to this stream.
  • :stream: the process-output slot is filled in with a stream that can be read to get the output.
• :error

  Same as :output, additionally accepts :output, making all error output routed to the same place as normal output. Defaults to :output.

• :if-output-exists (when :output is the name of a file)

  It is one of:

  • :error (the default) to generate an error
  • :supersede to supersede the file with output from the program
  • :append to append output from the program to the file
  • nil to return nil from run-program, without doing anything
• :if-error-exists

  Same as :if-output-exists, controlling :error output to files. Ignored when :error :output. Defaults to :error.

• :status-hook

  This is a function the system calls whenever the status of the process changes. The function takes the process as an argument.

• :external-format

  The external-format to use for :input, :output, and :error :streams.

• :directory

  Specifies the directory in which the program should be run. nil (the default) means the directory is unchanged.

• :preserve-fds

  A sequence of file descriptors which should remain open in the child process.

Windows specific options:

• :escape-arguments (default t)

  Controls escaping of the arguments passed to CreateProcess.

• :window (default nil)

  When nil, the subprocess decides how it will display its window. The following options control how the subprocess window should be displayed: :hide, :show-normal, :show-maximized, :show-minimized, :show-no-activate, :show-min-no-active, :show-na.

  Note: console application subprocesses may or may not display a console window depending on whether the SBCL runtime is itself a console or GUI application. Invoke cmd /c start to consistently display a console window or use the :window :hide option to consistently hide the console window.

When sb-ext:run-program is called with :wait nil, an process object is returned. The following functions are available for use with processes:

Function: sb-ext:process-p object ¶

t if object is a process, nil otherwise.

Function: sb-ext:process-input instance ¶

The input stream of the process or nil.

Function: sb-ext:process-output instance ¶

The output stream of the process or nil.

Function: sb-ext:process-error instance ¶

The error stream of the process or nil.

Function: sb-ext:process-alive-p process ¶

Return t if process is still alive, nil otherwise. Can return a false positive on a closed process.

Function: sb-ext:process-status process ¶

Return the current status of process. The result is one of :running, :stopped, :exited, :signaled.

Function: sb-ext:process-wait process &optional check-for-stopped ¶

Wait for process to quit running for some reason. When check-for-stopped is t, also returns when process is stopped. Returns process.

Function: sb-ext:process-exit-code process ¶

The exit code or the signal of a stopped process.

Function: sb-ext:process-core-dumped instance ¶

t if a core image was dumped by the process.

Function: sb-ext:process-close process ¶

Close all streams connected to process, stop maintaining the status slot. After process-close, process-alive-p and process-exit-code can return stale information about a process, so should not be used.

Function: sb-ext:process-kill process signal &optional whom ¶

Hand signal to process. If whom is :pid, use the kill Unix system call. If whom is :process-group, use the killpg(1) Unix system call. Returns t if successful, otherwise returns nil and error number (two values).

7.10.1 Unicode property access ¶

The following functions can be used to find information about a Unicode codepoint.

Function: sb-unicode:general-category character ¶

Returns the general category of character as it appears in UnicodeData.txt

Function: sb-unicode:bidi-class character ¶

Returns the bidirectional class of character

Function: sb-unicode:combining-class character ¶

Returns the canonical combining class (CCC) of character

Function: sb-unicode:decimal-value character ¶

Returns the decimal digit value associated with character or nil if there is no such value.

The only characters in Unicode with a decimal digit value are those that are part of a range of characters that encode the digits 0-9. Because of this, (decimal-digit c) <=> (digit-char-p c 10) in #+sb-unicode builds

Function: sb-unicode:digit-value character ¶

Returns the Unicode digit value of character or nil if it doesn’t exist.

Digit values are guaranteed to be integers between 0 and 9 inclusive. All characters with decimal digit values have the same digit value, but there are characters (such as digits of number systems without a 0 value) that have a digit value but no decimal digit value

Function: sb-unicode:numeric-value character ¶

Returns the numeric value of character or nil if there is no such value. Numeric value is the most general of the Unicode numeric properties. The only constraint on the numeric value is that it be a rational number.

Function: sb-unicode:mirrored-p character ¶

Returns t if character needs to be mirrored in bidirectional text. Otherwise, returns nil.

Function: sb-unicode:bidi-mirroring-glyph character ¶

Returns the mirror image of character if it exists. Otherwise, returns nil.

Function: sb-unicode:age character ¶

Returns the version of Unicode in which character was assigned as a pair of values, both integers, representing the major and minor version respectively. If character is not assigned in Unicode, returns nil for both values.

Function: sb-unicode:hangul-syllable-type character ¶

Returns the Hangul syllable type of character. The syllable type can be one of :l, :v, :t, :lv, or :lvt. If the character is not a Hangul syllable or Jamo, returns nil

Function: sb-unicode:east-asian-width character ¶

Returns the East Asian Width property of character as one of the keywords :n (Narrow), :a (Ambiguous), :h (Halfwidth), :w (Wide), :f (Fullwidth), or :na (Not applicable)

Function: sb-unicode:script character ¶

Returns the Script property of character as a keyword. If character does not have a known script, returns :unknown

Function: sb-unicode:char-block character ¶

Returns the Unicode block in which character resides as a keyword. If character does not have a known block, returns :no-block

Function: sb-unicode:unicode-1-name character ¶

Returns the name assigned to character in Unicode 1.0 if it is distinct from the name currently assigned to character. Otherwise, returns nil. This property has been officially obsoleted by the Unicode standard, and is only included for backwards compatibility.

Function: sb-unicode:proplist-p character property ¶

Returns t if character has the specified property. property is a keyword representing one of the properties from PropList.txt, with underscores replaced by dashes.

Function: sb-unicode:uppercase-p character ¶

Returns t if character has the Unicode property Uppercase and nil otherwise

Function: sb-unicode:lowercase-p character ¶

Returns t if character has the Unicode property Lowercase and nil otherwise

Function: sb-unicode:cased-p character ¶

Returns t if character has a (Unicode) case, and nil otherwise

Function: sb-unicode:case-ignorable-p character ¶

Returns t if character is Case Ignorable as defined in Unicode 6.3, Chapter 3

Function: sb-unicode:alphabetic-p character ¶

Returns t if character is Alphabetic according to the Unicode standard and nil otherwise

Function: sb-unicode:ideographic-p character ¶

Returns t if character has the Unicode property Ideographic, which loosely corresponds to the set of "Chinese characters"

Function: sb-unicode:math-p character ¶

Returns t if character is a mathematical symbol according to Unicode and nil otherwise

Function: sb-unicode:whitespace-p character ¶

Returns t if character is whitespace according to Unicode and nil otherwise

Function: sb-unicode:soft-dotted-p character ¶

Returns t if character has a soft dot (such as the dots on i and j) which disappears when accents are placed on top of it. and nil otherwise

Function: sb-unicode:hex-digit-p character &key ascii ¶

Returns t if character is a hexadecimal digit and nil otherwise. If :ascii is non-nil, fullwidth equivalents of the Latin letters A through F are excluded.

Function: sb-unicode:default-ignorable-p character ¶

Returns t if character is a DefaultIgnorableCode_Point

Function: sb-unicode:grapheme-break-class char ¶

Returns the grapheme breaking class of character, as specified in UAX #29.

Function: sb-unicode:word-break-class char ¶

Returns the word breaking class of character, as specified in UAX #29.

Function: sb-unicode:sentence-break-class char ¶

Returns the sentence breaking class of character, as specified in UAX #29.

Function: sb-unicode:line-break-class character &key resolve ¶

Returns the line breaking class of character, as specified in UAX #14. If :resolve is nil, returns the character class found in the property file. If :resolve is non-nil, certain line-breaking classes will be mapped to other classes as specified in the applicable standards. Additionally, if :resolve is :east-asian, Ambigious (class :ai) characters will be mapped to the Ideographic (:id) class instead of Alphabetic (:al).

7.10.2 String operations ¶

SBCL can normalize strings using:

Function: sb-unicode:normalize-string string &optional form filter ¶

Normalize string to the Unicode normalization form form. Acceptable values for form are :nfd, :nfc, :nfkd, and :nfkc. If filter is a function it is called on each decomposed character and only characters for which it returns t are collected.

Function: sb-unicode:normalized-p string &optional form ¶

Tests if string is normalized to form

SBCL implements the full range of Unicode case operations with the functions

Function: sb-unicode:uppercase string &key locale ¶

Returns the full uppercase of string according to the Unicode standard. The result is not guaranteed to have the same length as the input. If :locale is nil, no language-specific case transformations are applied. If :locale is a keyword representing a two-letter ISO country code, the case transforms of that locale are used. If :locale is t, the user’s current locale is used (Unix and Win32 only).

Function: sb-unicode:lowercase string &key locale ¶

Returns the full lowercase of string according to the Unicode standard. The result is not guaranteed to have the same length as the input. :locale has the same semantics as the :locale argument to uppercase.

Function: sb-unicode:titlecase string &key locale ¶

Returns the titlecase of string. The resulting string can be longer than the input. :locale has the same semantics as the :locale argument to uppercase.

Function: sb-unicode:casefold string ¶

Returns the full casefolding of string according to the Unicode standard. Casefolding removes case information in a way that allows the results to be used for case-insensitive comparisons. The result is not guaranteed to have the same length as the input.

It also extends standard Common Lisp case functions such as string-upcase and string-downcase to support a subset of Unicode’s casing behavior. Specifically, a character is both-case-p if its case mapping in Unicode is one-to-one and invertable.

The sb-unicode package also provides functions for collating/sorting strings according to the Unicode Collation Algorithm.

Function: sb-unicode:unicode< string1 string2 &key start1 end1 start2 end2 ¶

Determines whether string1 sorts before string2 using the Unicode Collation Algorithm. The function uses an untailored Default Unicode Collation Element Table to produce the sort keys. The function uses the Shifted method for dealing with variable-weight characters, as described in UTS #10

Function: sb-unicode:unicode= string1 string2 &key start1 end1 start2 end2 strict ¶

Determines whether string1 and string2 are canonically equivalent according to Unicode. The start and end arguments behave like the arguments to string=. If :strict is nil, unicode= tests compatibility equavalence instead.

Function: sb-unicode:unicode-equal string1 string2 &key start1 end1 start2 end2 strict ¶

Determines whether string1 and string2 are canonically equivalent after casefolding (that is, ignoring case differences) according to Unicode. The start and end arguments behave like the arguments to string=. If :strict is nil, unicode= tests compatibility equavalence instead.

Function: sb-unicode:unicode<= string1 string2 &key start1 end1 start2 end2 ¶

Tests if string1 and string2 are either unicode< or unicode=

Function: sb-unicode:unicode> string1 string2 &key start1 end1 start2 end2 ¶

Tests if string2 is unicode< string1.

Function: sb-unicode:unicode>= string1 string2 &key start1 end1 start2 end2 ¶

Tests if string1 and string2 are either unicode= or unicode>

The following functions are provided for detecting visually confusable strings:

Function: sb-unicode:confusable-p string1 string2 &key start1 end1 start2 end2 ¶

Determines whether string1 and string2 could be visually confusable according to the IDNA confusableSummary.txt table

7.10.3 Breaking strings ¶

The sb-unicode package includes several functions for breaking a Unicode string into useful parts.

Function: sb-unicode:graphemes string ¶

Breaks string into graphemes according to the default grapheme breaking rules specified in UAX #29, returning a list of strings.

Function: sb-unicode:words string ¶

Breaks string into words according to the default word breaking rules specified in UAX #29. Returns a list of strings

Function: sb-unicode:sentences string ¶

Breaks string into sentences according to the default sentence breaking rules specified in UAX #29

Function: sb-unicode:lines string &key margin ¶

Breaks string into lines that are no wider than :margin according to the line breaking rules outlined in UAX #14. Combining marks will always be kept together with their base characters, and spaces (but not other types of whitespace) will be removed from the end of lines. If :margin is unspecified, it defaults to 80 characters

7.16.1 Timeout Parameters ¶

Certain operations accept :timeout keyword arguments. These only affect the specific operation and must be specified at each call site by passing a :timeout keyword argument and a corresponding timeout value to the respective operation. Expiration of the timeout before the operation completes results in either a normal return with a return value indicating the timeout or in the signaling of a specialized condition such as sb-thread:join-thread-error.

Example:

(defun join-thread-within-5-seconds (thread)
  (multiple-value-bind (value result)
      (sb-thread:join-thread thread :default nil :timeout 5)
    (when (eq result :timeout)
      (error "Could not join ~A within 5 seconds" thread))
    value))

The above code attempts to join the specified thread for up to five seconds, returning its value in case of success. If the thread is still running after the five seconds have elapsed, sb-thread:join-thread indicates the timeout in its second return value. If a :default value was not provided, sb-thread:join-thread would signal a sb-thread:join-thread-error instead.

To wait for an arbitrary condition, optionally with a timeout, the sb-ext:wait-for macro can be used:

Macro: sb-ext:wait-for test-form &key timeout ¶

Wait until test-form evaluates to true, then return its primary value. If timeout is provided, waits at most approximately timeout seconds before returning nil.

If with-deadline has been used to provide a global deadline, signals a deadline-timeout if test-form doesn’t evaluate to true before the deadline.

Experimental: subject to change without prior notice.

7.16.2 Synchronous Timeouts ¶

Deadlines, in contrast to timeout parameters, are established for a dynamic scope using the sb-sys:with-deadline macro and indirectly affect operations within that scope. In case of nested uses, the effective deadline is the one that expires first unless an inner use explicitly overrides outer deadlines.

Macro: sb-sys:with-deadline (&key seconds override) &body body ¶

Arranges for a timeout condition to be signalled if an operation respecting deadlines occurs either after the deadline has passed, or would take longer than the time left to complete.

Currently only sleep, blocking IO operations, sb-thread:get-mutex, and sb-thread:condition-wait respect deadlines, but this includes their implicit uses inside SBCL itself.

Unless override is true, existing deadlines can only be restricted, not extended. Deadlines are per thread: children are unaffected by their parent’s deadlines.

Experimental.

Expiration of deadlines set up this way only has an effect when it happens before or during the execution of a deadline-aware operation (Operations Supporting Timeouts and Deadlines). In this case, a sb-sys:deadline-timeout is signaled. A handler for this condition type may use the sb-sys:defer-deadline or sb-sys:cancel-deadline restarts to defer or cancel the deadline respectively and resume execution of the interrupted operation.

Condition: sb-sys:deadline-timeout ¶

Signaled when an operation in the context of a deadline takes longer than permitted by the deadline.

Function: sb-sys:defer-deadline seconds &optional condition ¶

Find the defer-deadline restart associated with condition, and invoke it with seconds as argument (deferring the deadline by that many seconds.) Otherwise return nil if the restart is not found.

Function: sb-sys:cancel-deadline &optional condition ¶

Find and invoke the cancel-deadline restart associated with condition, or return nil if the restart is not found.

When a thread is executing the debugger, signaling of sb-sys:deadline-timeout conditions for that thread is deferred until it exits the debugger.

Example:

(defun read-input ()
  (list (read-line) (read-line)))

(defun do-it ()
  (sb-sys:with-deadline (:seconds 5))
    (read-input)
    (sleep 2)
    (sb-ext:run-program "my-program"))

The above code establishes a deadline of five seconds within which the body of the do-it function should execute. All calls of deadline-aware functions in the dynamic scope, in this case two read-line calls, a sleep call and a sb-ext:run-program call, are affected by the deadline. If, for example, the first read-line call completes in one second and the second read-line call completes in three seconds, a sb-sys:deadline-timeout condition will be signaled after the sleep call has been executing for one second.

7.16.3 Asynchronous Timeouts ¶

Asynchronous timeouts are established for a dynamic scope using the sb-ext:with-timeout macro:

Macro: sb-ext:with-timeout expires &body body ¶

Execute the body, asynchronously interrupting it and signalling a timeout condition after at least expires seconds have passed.

Note that it is never safe to unwind from an asynchronous condition. Consider:

(defun call-with-foo (function)
  (let (foo)
    (unwind-protect
       (progn
         (setf foo (get-foo))
         (funcall function foo))
     (when foo
       (release-foo foo)))))

If timeout occurs after get-foo has executed, but before the assignment, then release-foo will be missed. While individual sites like this can be made proof against asynchronous unwinds, this doesn’t solve the fundamental issue, as all the frames potentially unwound through need to be proofed, which includes both system and application code – and in essence proofing everything will make the system uninterruptible.

Expiration of the timeout will cause the operation being executed at that moment to be interrupted by an asynchronously signaled sb-ext:timeout condition, (almost) irregardless of the operation and its context.

Condition: sb-ext:timeout ¶

Signaled when an operation does not complete within an allotted time budget.

7.16.4 Operations Supporting Timeouts and Deadlines ¶

| Operation              | Timeout parameter | Affected by deadlines |
|------------------------+-------------------+-----------------------|
| cl:sleep               | -                 | since SBCL 1.4.3      |
| cl:read-line, etc.     | no                | yes                   |
| wait-for               | yes               | yes                   |
| process-wait           | no                | yes                   |
| grab-mutex             | yes               | yes                   |
| condition-wait         | yes               | yes                   |
| wait-on-semaphore      | yes               | yes                   |
| join-thread            | yes               | yes                   |
| receive-message        | yes               | yes?                  |
| wait-on-gate           | yes               | yes?                  |
| frlock-write           | yes               | yes?                  |
| grab-frlock-write-lock | yes               | yes?                  |