→ ↺ λ

SBCL Manual

Table of Contents

[in package SB-MANUAL]

← ↑ → ↺ λ

1 Getting Support and Reporting Bugs

← ↑ → ↺ λ

1.1 Volunteer Support

Your primary source of SBCL support should probably be the mailing list sbcl-help: in addition to other users SBCL developers monitor this list and are available for advice. As an anti-spam measure subscription is required for posting:

https://lists.sourceforge.net/lists/listinfo/sbcl-help

Remember that the people answering your question are volunteers, so you stand a much better chance of getting a good answer if you ask a good question.

Before sending mail, check the list archives at either

http://sourceforge.net/mailarchive/forum.php?forum_name=sbcl-help

or

http://news.gmane.org/gmane.lisp.steel-bank.general

to see if your question has been answered already. Checking the bug database is also worth it (see Reporting Bugs), to see if the issue is already known.

For general advice on asking good questions, see

http://www.catb.org/~esr/faqs/smart-questions.html.

← ↑ → ↺ λ

1.2 Commercial Support

There is no formal organization developing SBCL, but if you need a paid support arrangement or custom SBCL development, we maintain the list of companies and consultants below. Use it to identify service providers with appropriate skills and interests, and contact them directly.

The SBCL project cannot verify the accuracy of the information or the competence of the people listed, and they have provided their own blurbs below: you must make your own judgement of suitability from the available information - refer to the links they provide, the CREDITS file, mailing list archives, CVS commit messages, and so on. Please feel free to ask for advice on the sbcl-help list.

(At present, no companies or consultants wish to advertise paid support or custom SBCL development in this manual).

← ↑ → ↺ λ

1.3 Reporting Bugs

SBCL uses Launchpad to track bugs. The bug database is available at

https://bugs.launchpad.net/sbcl

Reporting bugs there requires registering at Launchpad. However, bugs can also be reported on the mailing list sbcl-bugs, which is moderated but does not require subscribing.

Simply send email to sbcl-bugs@lists.sourceforge.net and the bug will be checked and added to Launchpad by SBCL maintainers.

← ↑ → ↺ λ

1.3.1 How to Report Bugs Effectively

Please include enough information in a bug report that someone reading it can reproduce the problem, i.e. don't write

Subject: apparent bug in PRINT-OBJECT (or *PRINT-LENGTH*?)
PRINT-OBJECT doesn't seem to work with *PRINT-LENGTH*. Is this a bug?

but instead

Subject: apparent bug in PRINT-OBJECT (or *PRINT-LENGTH*?)
In sbcl-1.2.3 running under OpenBSD 4.5 on my Alpha box, when
I compile and load the file
   (DEFSTRUCT (FOO (:PRINT-OBJECT (LAMBDA (X Y)
                                    (LET ((*PRINT-LENGTH* 4))
                                      (PRINT X Y)))))
     X Y)
then at the command line type
   (MAKE-FOO)
the program loops endlessly instead of printing the object.

A more in-depth discussion on reporting bugs effectively can be found at

http://www.chiark.greenend.org.uk/~sgtatham/bugs.html.

← ↑ → ↺ λ

1.3.2 How to Report Signal-related Bugs

If you run into a signal related bug, you are getting fatal errors such as signal N is [un]blocked or just hangs, and you want to send a useful bug report then:

← ↑ → ↺ λ

2 Introduction

SBCL is a mostly-conforming implementation of the ANSI Common Lisp standard. This manual focuses on behavior which is specific to SBCL, not on behavior which is common to all implementations of ANSI Common Lisp.

← ↑ → ↺ λ

2.1 ANSI Conformance

Essentially every type of non-conformance is considered a bug. (The exceptions involve internal inconsistencies in the standard.) See Reporting Bugs.

← ↑ → ↺ λ

2.2 Extensions

SBCL comes with numerous extensions, some in core and some in modules loadable with require(0 1). Unfortunately, not all of these extensions have proper documentation yet.

← ↑ → ↺ λ

2.3 Idiosyncrasies

The information in this section describes some of the ways that SBCL deals with choices that the ANSI standard leaves to the implementation.

← ↑ → ↺ λ

2.3.1 Declarations

Declarations are generally treated as assertions. This general principle, and its implications, and the bugs which still keep the compiler from quite satisfying this principle, are discussed in Declarations as Assertions.

← ↑ → ↺ λ

2.3.2 FASL format

SBCL fasl-format is binary compatible only with the exact SBCL version it was generated with. While this is obviously suboptimal, it has proven more robust than trying to maintain fasl compatibility across versions: accidentally breaking things is far too easy, and can lead to hard to diagnose bugs.

The following snippet handles fasl recompilation automatically for ASDF-based systems, and makes a good candidate for inclusion in the user or system initialization file (see Initialization Files).

(require :asdf)

;;; If a fasl was stale, try to recompile and load (once).
(defmethod asdf:perform :around ((o asdf:load-op)
                                 (c asdf:cl-source-file))
   (handler-case (call-next-method o c)
      ;; If a fasl was stale, try to recompile and load (once).
      (sb-ext:invalid-fasl ()
         (asdf:perform (make-instance 'asdf:compile-op) c)
         (call-next-method))))

← ↑ → ↺ λ

2.3.3 Compiler-only Implementation

SBCL is essentially a compiler-only implementation of Common Lisp. That is, for all but a few special cases, eval creates a lambda expression, calls compile on the lambda expression to create a compiled function, and then calls funcall on the resulting function object. A more traditional interpreter is also available on default builds; it is usually only called internally. This is explicitly allowed by the ANSI standard but leads to some oddities; e.g. at default settings, functionp and compiled-function-p are equivalent, and they collapse into the same function when SBCL is built without the interpreter.

← ↑ → ↺ λ

2.3.4 Defining Constants

SBCL is quite strict about ANSI's definition of defconstant. ANSI says that doing defconstant of the same symbol more than once is undefined unless the new value is eql(0 1) to the old value. Conforming to this specification is a nuisance when the "constant" value is only constant under some weaker test like string= or equal.

It's especially annoying because, in SBCL, defconstant takes effect not only at load time but also at compile time, so that just compiling and loading reasonable code like

(defconstant +foobyte+ '(1 4))

runs into this undefined behavior. Many implementations of Common Lisp try to help the programmer around this annoyance by silently accepting the undefined code and trying to do what the programmer probably meant.

SBCL instead treats the undefined behavior as an error. Often such code can be rewritten in portable ANSI Common Lisp which has the desired behavior. E.g., the code above can be given an exactly defined meaning by replacing defconstant either with defparameter or with a customized macro which does the right thing, e.g.

(defmacro define-constant (name value &optional doc)
  `(defconstant ,name (if (boundp ',name) (symbol-value ',name) ,value)
                      ,@(when doc (list doc))))

or possibly along the lines of the sb-int:defconstant-eqx macro used internally in the implementation of SBCL itself. In circumstances where this is not appropriate, the programmer can handle the condition type sb-ext:defconstant-uneql and choose either the continue restart or abort restart as appropriate.

← ↑ → ↺ λ

2.3.5 Style Warnings

SBCL gives style warnings about various kinds of perfectly legal code, e.g.

This causes friction with people who point out that other ways of organizing code (especially avoiding the use of defgeneric) are just as aesthetically stylish. However, these warnings should be read not as warning, bad aesthetics detected, you have no style but as warning, this style keeps the compiler from understanding the code as well as you might like. That is, unless the compiler warns about such conditions, there's no way for the compiler to warn about some programming errors which would otherwise be easy to overlook. (Related bug: The warning about multiple defuns is pointlessly annoying when you compile and then load a function containing defun wrapped in eval-when, and ideally should be suppressed in that case, but still isn't as of SBCL 0.7.6.)

← ↑ → ↺ λ

2.4 Development Tools

← ↑ → ↺ λ

2.4.1 Editor Integration

Though SBCL can be used running "bare", the recommended mode of development is with an editor connected to SBCL, supporting not only basic lisp editing (paren-matching, etc), but providing among other features an integrated debugger, interactive compilation, and automated documentation lookup.

Currently SLIME (Superior Lisp Interaction Mode for Emacs) together with Emacs is recommended for use with SBCL, though other options exist as well. Historically, the ILISP package at http://ilisp.cons.org/ provided similar functionality, but it does not support modern SBCL versions.

SLIME can be downloaded from https://slime.common-lisp.dev/.

← ↑ → ↺ λ

2.4.2 Language Reference

CLHS (Common Lisp Hyperspec) is a hypertext version of the ANSI standard, made freely available by LispWorks -- an invaluable reference.

See https://www.lispworks.com/documentation/HyperSpec/Front/index.htm.

← ↑ → ↺ λ

2.4.3 Generating Executables

SBCL can generate stand-alone executables. The generated executables include the SBCL runtime itself, so no restrictions are placed on program functionality. For example, a deployed program can call compile and load, which requires the compiler to be present in the executable. For further information, sb-ext:save-lisp-and-die.

← ↑ → ↺ λ

2.5 More SBCL Information

← ↑ → ↺ λ

2.5.1 SBCL Homepage

The SBCL website at http://www.sbcl.org/ has some general information, plus links to mailing lists devoted to SBCL, and to archives of these mailing lists. Subscribing to the mailing lists sbcl-help and sbcl-announce is recommended: both are fairly low-volume, and help you keep abreast with SBCL development.

← ↑ → ↺ λ

2.5.2 Online Documentation

Documentation for non-ANSI extensions for various commands is available online from the SBCL executable itself. The extensions for functions which have their own command prompts (e.g. the debugger, and inspect) are documented in text available by typing help at their command prompts. The extensions for functions which don't have their own command prompt (such as trace(0 1)) are described in their documentation strings, unless your SBCL was compiled with an option not to include documentation strings, in which case the documentation strings are only readable in the source code.

← ↑ → ↺ λ

2.5.3 Additional Documentation Files

Besides this user manual both SBCL source and binary distributions include some other SBCL-specific documentation files, which should be installed along with this manual on your system, e.g. in /usr/local/share/doc/sbcl/.

← ↑ → ↺ λ

2.5.4 Internals Documentation

If you're interested in the development of the SBCL system itself, then subscribing to sbcl-devel is a good idea.

SBCL internals documentation -- besides comments in the source -- is available in the Web Archive:

https://web.archive.org/web/20120814000933/http://sbcl-internals.cliki.net/index.

Some low-level information describing the programming details of the conversion from CMUCL to SBCL is available in the doc/FOR-CMUCL-DEVELOPERS file.

← ↑ → ↺ λ

2.6 More Common Lisp Information

← ↑ → ↺ λ

2.6.1 Internet Community

IRC channels on https://libera.chat/:

You can use https://web.libera.chat or a normal IRC client.

Also, see https://www.reddit.com/r/Common_Lisp/, as well as https://www.lisp.org and https://cliki.net, which contain numerous pointers places in the net where lispers talks shop.

← ↑ → ↺ λ

2.6.2 Third-party Libraries

For a wealth of information about free Common Lisp libraries and tools we recommend checking out CLiki: https://cliki.net/.

The most popular library manager is Quicklisp: https://www.quicklisp.org/beta/.

← ↑ → ↺ λ

2.6.3 Common Lisp Books

If you're not a programmer and you're trying to learn, many introductory Lisp books are available. However, we don't have any standout favorites.

If you are an experienced programmer in other languages but need to learn about Common Lisp, some books stand out:

← ↑ → ↺ λ

2.7 History and Implementation of SBCL

You can work productively with SBCL without knowing or understanding anything about where it came from, how it is implemented, or how it extends the ANSI Common Lisp standard. However, a little knowledge can be helpful in order to understand error messages, to troubleshoot problems, to understand why some parts of the system are better debugged than others, and to anticipate which known bugs, known performance problems, and missing extensions are likely to be fixed, tuned, or added.

SBCL is descended from CMUCL, which is itself descended from Spice Lisp, including early implementations for the Mach operating system on the IBM RT, back in the 1980s. Some design decisions from that time are still reflected in the current implementation:

SBCL also inherited some newer architectural features from CMUCL. The most important is that on some architectures it has a generational garbage collector (GC), which has various implications (mostly good) for performance. These are discussed in another chapter, Efficiency.

SBCL has diverged from CMUCL in that SBCL is now essentially a compiler-only implementation of Common Lisp. This is a change in implementation strategy, taking advantage of the freedom "any of these facilities might share the same execution strategy" guaranteed in clhs 3.1 (Evaluation). It does not mean SBCL can't be used interactively, and in fact the change is largely invisible to the casual user, since SBCL still can and does execute code interactively by compiling it on the fly. (It is visible if you know how to look, like using compiled-function-p; and it is visible in the way that SBCL doesn't have many bugs which behave differently in interpreted code than in compiled code.) What it means is that in SBCL, the eval function only truly "interprets" a few easy kinds of forms, such as symbols which are boundp. More complicated forms are evaluated by calling compile and then calling funcall on the returned result.

The direct ancestor of SBCL is the x86 port of CMUCL. This port was in some ways the most cobbled-together of all the CMUCL ports, since a number of strange changes had to be made to support the register-poor x86 architecture. Some things (like tracing and debugging) do not work particularly well there. SBCL should be able to improve in these areas (and has already improved in some other areas), but it takes a while.

On the x86 SBCL -- like the x86 port of CMUCL -- uses a conservative GC. This means that it doesn't maintain a strict separation between tagged and untagged data, instead treating some untagged data (e.g. raw floating point numbers) as possibly-tagged data and so not collecting any Lisp objects that they point to. This has some negative consequences for average time efficiency (though possibly no worse than the negative consequences of trying to implement an exact GC on a processor architecture as register-poor as the X86) and also has potentially unlimited consequences for worst-case memory efficiency. In practice, conservative garbage collectors work reasonably well, not getting anywhere near the worst case. But they can occasionally cause odd patterns of memory usage.

The fork from CMUCL was based on a major rewrite of the system bootstrap process. CMUCL has for many years tolerated a very unusual "build" procedure which doesn't actually build the complete system from scratch, but instead progressively overwrites parts of a running system with new versions. This quasi-build procedure can cause various bizarre bootstrapping hangups, especially when a major change is made to the system. It also makes the connection between the current source code and the current executable more tenuous than in other software systems -- it's easy to accidentally build a CMUCL system containing characteristics not reflected in the current version of the source code.

Other major changes since the fork from CMUCL include:

← ↑ → ↺ λ

3 Starting and Stopping

← ↑ → ↺ λ

3.1 Starting SBCL

← ↑ → ↺ λ

3.1.1 Running from Shell

To run SBCL, type sbcl at the command line.

You should end up in the toplevel REPL (read-eval-print loop), where you can interact with SBCL by typing expressions.

$ sbcl
This is SBCL 0.8.13.60, an implementation of ANSI Common Lisp.
More information about SBCL is available at <http://www.sbcl.org/>.

SBCL is free software, provided as is, with absolutely no warranty.
It is mostly in the public domain; some portions are provided under
BSD-style licenses.  See the CREDITS and COPYING files in the
distribution for more information.
* (+ 2 2)
4
* (exit)
$

Also see Command Line Options and Stopping SBCL.

← ↑ → ↺ λ

3.1.2 Running from Emacs

To run SBCL as an inferior-lisp from Emacs, in your .emacs do something like:

;;; The SBCL binary and command-line arguments
(setq inferior-lisp-program "/usr/local/bin/sbcl --noinform")

For more information on using SBCL with Emacs, see Editor Integration.

← ↑ → ↺ λ

3.1.3 Shebang Scripts

Standard Unix tools that are interpreters follow a common command line protocol that is necessary to work with "shebang scripts". SBCL supports this via the --script command line option (see Command Line Options).

Example file (hello.lisp):

#!/usr/local/bin/sbcl --script
(write-line "Hello, World!")

Usage from the command line:

$ ./hello.lisp
Hello, World!

Note that SBCL skips the shebang line when it reads the file:

$ sbcl --script hello.lisp
Hello, World!

← ↑ → ↺ λ

3.2 Stopping SBCL

← ↑ → ↺ λ

3.2.1 Exit

SBCL can be stopped at any time by calling sb-ext:exit, optionally returning a specified numeric value to the calling process. See Threading for information about terminating individual threads.

← ↑ → ↺ λ

3.2.2 End of File

By default SBCL also exits on end of input, caused either by user pressing Control-D on an attached terminal, or end of input when using SBCL as part of a shell pipeline.

← ↑ → ↺ λ

3.2.3 Saving a Core Image

SBCL has the ability to save its state as a file for later execution. This functionality is important for its bootstrapping process, and is also provided as an extension to the user.

In cases where the standard initialization files have already been loaded into the saved core, and alternative ones should be used (or none at all), SBCL allows customizing the initfile pathname computation.

To facilitate distribution of SBCL applications using external resources, the filesystem location of the SBCL core file being used is available from Lisp.

← ↑ → ↺ λ

3.2.4 Exit on Errors

SBCL can also be configured to exit if an unhandled error occurs, which is mainly useful for acting as part of a shell pipeline; doing so under most other circumstances would mean giving up large parts of the flexibility and robustness of Common Lisp. See Debugger Entry and the command line option --disable-debugger in Runtime Options.

← ↑ → ↺ λ

3.3 Command Line Options

Command line options can be considered an advanced topic; for ordinary interactive use, no command line arguments should be necessary.

In order to understand the command line argument syntax for SBCL, it is helpful to understand that the SBCL system is implemented as two components, a low-level runtime environment written in C and a higher-level system written in Common Lisp itself. Some command line arguments are processed during the initialization of the low-level runtime environment, some command line arguments are processed during the initialization of the Common Lisp system, and any remaining command line arguments are made available to user code via sb-ext:*posix-argv*.

The full, unambiguous syntax for invoking SBCL at the command line is:

sbcl <runtime-option>* --end-runtime-options \
     <toplevel-option>* --end-toplevel-options \
     <user-option>*

For convenience, --end-runtime-options and --end-toplevel-options can be omitted, which can be convenient when you are running the program interactively, and you can see that no ambiguities are possible with the option values you are using. Omitting these elements is probably a bad idea for any batch file where any of the options are under user control, since it makes it impossible for SBCL to detect erroneous command line input, so that erroneous command line arguments will be passed on to the user program even if they was intended for the runtime system or the Lisp system.

← ↑ → ↺ λ

3.3.1 Runtime Options

In the future, runtime options may be added to control behaviour such as lazy allocation of memory.

Runtime options, including any --end-runtime-options option, are stripped out of the command line before the Lisp toplevel logic gets a chance to see it.

← ↑ → ↺ λ

3.3.2 Toplevel Options

The following options are processed and removed by the default toplevel (see sb-ext:save-lisp-and-die).

← ↑ → ↺ λ

3.4 Initialization Files

SBCL processes initialization files with read and eval, not load; hence initialization files can be used to set startup *package* and *readtable*, and for proclaiming a global optimization policy.

Neither initialization file is required.

← ↑ → ↺ λ

3.5 Initialization and Exit Hooks

SBCL provides hooks into the system initialization and exit.

← ↑ → ↺ λ

4 Compiler

This chapter will discuss most compiler issues other than efficiency, including compiler error messages, the SBCL compiler's unusual approach to type safety in the presence of type declarations, the effects of various compiler optimization policies, and the way that inlining and open coding may cause optimized code to differ from a naive translation. Efficiency issues are sufficiently varied and separate that they have their own chapter, Efficiency.

← ↑ → ↺ λ

4.1 Diagnostic Messages

← ↑ → ↺ λ

4.1.1 Controlling Verbosity

The compiler can be quite verbose in its diagnostic reporting, rather more then some users would prefer -- the amount of noise emitted can be controlled, however.

To control emission of compiler diagnostics (of any severity other than error(0 1): Diagnostic Severity) use the sb-ext:muffle-conditions and sb-ext:unmuffle-conditions declarations, specifying the type of condition that is to be muffled (the muffling is done using an associated muffle-warning restart).

Global control:

;;; Muffle compiler-notes globally
(declaim (sb-ext:muffle-conditions sb-ext:compiler-note))

Local control:

;;; Muffle compiler-notes based on lexical scope
(defun foo (x)
  (declare (optimize speed) (fixnum x)
           (sb-ext:muffle-conditions sb-ext:compiler-note))
  (values (* x 5) ; no compiler note from this
    (locally
      (declare (sb-ext:unmuffle-conditions sb-ext:compiler-note))
      ;; this one gives a compiler note
      (* x -5))))

Various details of how the compiler messages are printed can be controlled via the alist sb-ext:*compiler-print-variable-alist*.

For information about muffling warnings signaled outside of the compiler, see Customization Hooks for Users.

← ↑ → ↺ λ

4.1.2 Diagnostic Severity

There are four levels of compiler diagnostic severity:

The first three levels correspond to condition classes which are defined in the ANSI standard for Common Lisp and which have special significance to the compile and compile-file functions. These levels of compiler error severity occur when the compiler handles conditions of these classes.

The fourth level of compiler error severity, note, corresponds to the sb-ext:compiler-note, and is used for problems which are too mild for the standard condition classes, typically hints about how efficiency might be improved. The sb-ext:code-deletion-note, a subtype of sb-ext:compiler-note, is signalled when the compiler deletes user-supplied code after proving that the code in question is unreachable.

Future work for SBCL includes expanding this hierarchy of types to allow more fine-grained control over emission of diagnostic messages.

← ↑ → ↺ λ

4.1.3 Understanding Compiler Diagnostics

The messages emitted by the compiler contain a lot of detail in a terse format, so they may be confusing at first. The messages will be illustrated using this example program:

(defmacro zoq (x)
  `(roq (ploq (+ ,x 3))))

(defun foo (y)
  (declare (symbol y))
  (zoq y))

The main problem with this program is that it is trying to add 3 to a symbol. Note also that the functions roq and ploq aren't defined anywhere.

← ↑ → ↺ λ

Parts of a Compiler Diagnostic

When processing this program, the compiler will produce this warning:

; file: /tmp/foo.lisp
; in: DEFUN FOO
;     (ZOQ Y)
; --> ROQ PLOQ
; ==>
;   (+ Y 3)
;
; caught WARNING:
;   Asserted type NUMBER conflicts with derived type (VALUES SYMBOL &OPTIONAL).

In this example we see each of the six possible parts of a compiler diagnostic:

Note that each part of the message is distinctively marked:

Each part of the message is more specific than the preceding one. If consecutive messages are for nearby locations, then the front part of the messages would be the same. In this case, the compiler omits as much of the second message as in common with the first. For example:

; file: /tmp/foo.lisp
; in: DEFUN FOO
;     (ZOQ Y)
; --> ROQ
; ==>
;   (PLOQ (+ Y 3))
;
; caught STYLE-WARNING:
;   undefined function: PLOQ

; ==>
;   (ROQ (PLOQ (+ Y 3)))
;
; caught STYLE-WARNING:
;   undefined function: ROQ

In this example, the file, definition and original source are identical for the two messages, so the compiler omits them in the second message. If consecutive messages are entirely identical, then the compiler prints only the first message, followed by: [Last message occurs <repeats> times] where <repeats> is the number of times the message was given.

If the source was not from a file, then no file line is printed. If the actual source is the same as the original source, then the processing path and actual source will be omitted. If no forms intervene between the original source and the actual source, then the processing path will also be omitted.

← ↑ → ↺ λ

Original and Actual Source

The original source displayed will almost always be a list. If the actual source for an message is a symbol, the original source will be the immediately enclosing evaluated list form. So even if the offending symbol does appear in the original source, the compiler will print the enclosing list and then print the symbol as the actual source (as though the symbol were introduced by a macro.)

When the actual source is displayed (and is not a symbol), it will always be code that resulted from the expansion of a macro or a source-to-source compiler optimization. This is code that did not appear in the original source program; it was introduced by the compiler.

Keep in mind that when the compiler displays a source form in an diagnostic message, it always displays the most specific (innermost) responsible form. For example, compiling this function

(defun bar (x) (let (a) (declare (fixnum a)) (setq a (foo x)) a))

gives this error message

; file: /tmp/foo.lisp
; in: DEFUN BAR
;     (LET (A)
;     (DECLARE (FIXNUM A))
;     (SETQ A (FOO X))
;     A)
;
; caught WARNING:
;   Asserted type FIXNUM conflicts with derived type (VALUES NULL &OPTIONAL).

This message is not saying that there is a problem somewhere in this let -- it is saying that there is a problem with the let itself. In this example, the problem is that a's nil initial value is not a fixnum.

← ↑ → ↺ λ

Processing Path

The processing path is mainly useful for debugging macros, so if you don't write macros, you can probably ignore it. Consider this example:

(defun foo (n)
  (dotimes (i n *undefined*)))

Compiling results in this error message:

; in: DEFUN FOO
;     (DOTIMES (I N *UNDEFINED*))
; --> DO BLOCK LET TAGBODY RETURN-FROM
; ==>
;   (PROGN *UNDEFINED*)
;
; caught WARNING:
;   undefined variable: *UNDEFINED*

Note that do appears in the processing path. This is because dotimes expands into:

(do ((i 0 (1+ i)) (#:g1 n))
    ((>= i #:g1) *undefined*)
  (declare (type unsigned-byte i)))

The rest of the processing path results from the expansion of do:

(block nil
  (let ((i 0) (#:g1 n))
    (declare (type unsigned-byte i))
    (tagbody (go #:g3)
      #:g2    (psetq i (1+ i))
      #:g3    (unless (>= i #:g1) (go #:g2))
      (return-from nil (progn *undefined*)))))

In this example, the compiler descended into the block, let, tagbody and return-from to reach the progn printed as the actual source. This is a place where the "actual source appears in explanation" rule was applied. The innermost actual source form was the symbol undefined itself, but that also appeared in the explanation, so the compiler backed out one level.

← ↑ → ↺ λ

4.2 Handling of Types

One of the most important features of the SBCL compiler (similar to the original CMUCL compiler, also known as Python) is its fairly sophisticated understanding of the Common Lisp type system and its conservative approach to the implementation of type declarations.

These two features reward the use of type declarations throughout development, even when high performance is not a concern. Also, as discussed in the chapter on performance (see Efficiency), the use of appropriate type declarations can be very important for performance as well.

The SBCL compiler also has a greater knowledge of the Common Lisp type system than other compilers. Support is incomplete only for types involving the satisfies type specifier.

← ↑ → ↺ λ

4.2.1 Declarations as Assertions

The SBCL compiler treats type declarations differently from most other Lisp compilers. Under default compilation policy the compiler doesn't blindly believe type declarations, but considers them assertions about the program that should be checked: all type declarations that have not been proven to always hold are asserted at runtime.

Remaining bugs in the compiler's handling of types unfortunately provide some exceptions to this rule, see Implementation Limitations.

CLOS slot types form a notable exception. Types declared using the :type slot option in defclass are asserted if and only if the class was defined in safe code and the slot access location is in safe code as well. This laxness does not pose any internal consistency issues, as the CLOS slot types are not available for the type inferencer, nor do CLOS slot types provide any efficiency benefits.

There are three type checking policies available in SBCL, selectable via optimize declarations.

← ↑ → ↺ λ

4.2.2 Precise Type Checking

Precise checking means that the check is done as though typep had been called with the exact type specifier that appeared in the declaration.

If a variable is declared to be (integer 3 17), then its value must always be an integer between 3 and 17. If multiple type declarations apply to a single variable, then all the declarations must be correct; it is as though all the types were intersected producing a single and type specifier.

To gain maximum benefit from the compiler's type checking, you should always declare the types of function arguments and structure slots as precisely as possible. This often involves the use of or(0 1), member(0 1), and other list-style type specifiers.

← ↑ → ↺ λ

4.2.3 Getting Existing Programs to Run

Since SBCL's compiler does much more comprehensive type checking than most Lisp compilers, SBCL may detect type errors in programs that have been debugged using other compilers. These errors are mostly incorrect declarations, although compile-time type errors can find actual bugs if parts of the program have never been tested.

Some incorrect declarations can only be detected by run-time type checking. It is very important to initially compile a program with full type checks (high safety optimization) and then test this safe version. After the checking version has been tested, then you can consider weakening or eliminating type checks. This applies even to previously debugged programs because the SBCL compiler does much more type inference than other Common Lisp compilers, so an incorrect declaration can do more damage.

The most common problem is with variables whose constant initial value doesn't match the type declaration. Incorrect constant initial values will always be flagged by a compile-time type error, and they are simple to fix once located. Consider this code fragment:

(prog (foo)
  (declare (fixnum foo))
  (setq foo ...)
  ...)

Here foo is given an initial value of nil but is declared to be a fixnum. Even if it is never read, the initial value of a variable must match the declared type. There are two ways to fix this problem. Change the declaration

(prog (foo)
  (declare (type (or fixnum null) foo))
  (setq foo ...)
  ...)

or change the initial value

(prog ((foo 0))
  (declare (fixnum foo))
  (setq foo ...)
  ...)

It is generally preferable to change to a legal initial value rather than to weaken the declaration, but sometimes it is simpler to weaken the declaration than to try to make an initial value of the appropriate type.

Another declaration problem occasionally encountered is incorrect declarations on defmacro arguments. This can happen when a function is converted into a macro. Consider this macro:

(defmacro my-1+ (x)
  (declare (fixnum x))
  `(the fixnum (1+ ,x)))

Although legal and well-defined Common Lisp code, this meaning of this definition is almost certainly not what the writer intended. For example, this call is illegal:

(my-1+ (+ 4 5))

This call is illegal because the argument to the macro is (+ 4 5), which is a list(0 1), not a fixnum. Because of macro semantics, it is hardly ever useful to declare the types of macro arguments. If you really want to assert something about the type of the result of evaluating a macro argument, then put a the in the expansion:

(defmacro my-1+ (x)
  `(the fixnum (1+ (the fixnum ,x))))

In this case, it would be stylistically preferable to change this macro back to a function and declare it inline.

Some more subtle problems are caused by incorrect declarations that can't be detected at compile time. Consider this code:

(do ((pos 0 (position #a string :start (1+ pos))))
  ((null pos))
  (declare (fixnum pos))
  ...)

Although pos is almost always a fixnum, it is nil at the end of the loop. If this example is compiled with full type checks (the default), then running it will signal a type error at the end of the loop. If compiled without type checks, the program will go into an infinite loop (or perhaps position will complain because (1+ nil) isn't a sensible start.) Why? Because if you compile without type checks, the compiler just quietly believes the type declaration. Since the compiler believes that pos is always a fixnum, it believes that pos is never nil, so (null pos) is never true, and the loop exit test is optimized away. Such errors are sometimes flagged by unreachable code notes, but it is still important to initially compile and test any system with full type checks, even if the system works fine when compiled using other compilers.

In this case, the fix is to weaken the type declaration to (or fixnum null). (Actually, this declaration is unnecessary in SBCL, since it already knows that position returns a non-negative fixnum or nil.)

Note that there is usually little performance penalty for weakening a declaration in this way. Any numeric operations in the body can still assume that the variable is a fixnum, since nil is not a legal numeric argument. Another possible fix would be to say:

(do ((pos 0 (position #a string :start (1+ pos))))
    ((null pos))
  (let ((pos pos))
    (declare (fixnum pos))
    ...))

This would be preferable in some circumstances, since it would allow a non-standard representation to be used for the local pos variable in the loop body.

← ↑ → ↺ λ

4.2.4 Implementation Limitations

If an ftype is placed after the function definition the function won't perform any type checks, and the calls to the function will blindly trust the declared types. (optimize (debug 3)) will not trust any ftype declarations.

← ↑ → ↺ λ

4.3 Compiler Policy

Compiler policy is controlled by the optimize declaration, supporting all ANSI optimization qualities (debug, safety, space, and speed). (A deprecated extension sb-ext:inhibit-warnings is still supported but liable to go away at any time.)

For effects of various optimization qualities on type-safety and debuggability see Declarations as Assertions and Debugger Policy Control.

Ordinarily, when the speed quality is high, the compiler emits notes to notify the programmer about its inability to apply various optimizations. For selective muffling of these notes, see Controlling Verbosity.

The value of space mostly influences the compiler's decision whether to inline operations, which tend to increase the size of programs. Use the value 0 with caution, since it can cause the compiler to inline operations so indiscriminately that the net effect is to slow the program by causing cache misses or even swapping.

← ↑ → ↺ λ

4.4 Compiler Errors

← ↑ → ↺ λ

4.4.1 Type Errors at Compile Time

If the compiler can prove at compile time that some portion of the program cannot be executed without a type error, then it will give a warning at compile time.

It is possible that the offending code would never actually be executed at run-time due to some higher level consistency constraint unknown to the compiler, so a type warning doesn't always indicate an incorrect program.

For example, consider this code fragment:

(defun raz (foo)
  (let ((x (case foo
              (:this 13)
              (:that 9)
              (:the-other 42))))
    (declare (fixnum x))
    (foo x)))

Compilation produces this warning:

; in: DEFUN RAZ
;     (CASE FOO (:THIS 13) (:THAT 9) (:THE-OTHER 42))
; --> LET COND IF COND IF COND IF
; ==>
;   (COND)
;
; caught WARNING:
;   This is not a FIXNUM:
;   NIL

In this case, the warning means that if foo isn't any of :this, :that or :the-other, then x will be initialized to nil, which the fixnum declaration makes illegal. The warning will go away if ecase is used instead of case, or if :the-other is changed to t.

This sort of spurious type warning happens moderately often in the expansion of complex macros and in inline functions. In such cases, there may be dead code that is impossible to correctly execute. The compiler can't always prove this code is dead (could never be executed), so it compiles the erroneous code (which will always signal an error if it is executed) and gives a warning.

← ↑ → ↺ λ

4.4.2 Errors During Macroexpansion

The compiler handles errors that happen during macroexpansion, turning them into compiler errors. If you want to debug the error (to debug a macro), you can set *break-on-signals* to error(0 1). For example, this definition:

(defun foo (e l)
  (do ((current l (cdr current))
       ((atom current) nil))
      (when (eq (car current) e) (return current))))

gives this error:

; in: DEFUN FOO
;     (DO ((CURRENT L (CDR CURRENT))
;        ((ATOM CURRENT) NIL))
;       (WHEN (EQ (CAR CURRENT) E) (RETURN CURRENT)))
;
; caught ERROR:
;   (in macroexpansion of (DO # #))
;   (hint: For more precise location, try *BREAK-ON-SIGNALS*.)
;   DO step variable is not a symbol: (ATOM CURRENT)

← ↑ → ↺ λ

4.4.3 Read Errors

SBCL's compiler does not attempt to recover from read errors when reading a source file, but instead just reports the offending character position and gives up on the entire source file.

← ↑ → ↺ λ

4.5 Open Coding and Inline Expansion

Since Common Lisp forbids the redefinition of standard functions, the compiler can have special knowledge of these standard functions embedded in it. This special knowledge is used in various ways (open coding, inline expansion, source transformation), but the implications to the user are basically the same:

When a function call is open coded, inline code whose effect is equivalent to the function call is substituted for that function call. When a function call is closed coded, it is usually left as is, although it might be turned into a call to a different function with different arguments. As an example, if nthcdr were to be open coded, then

(nthcdr 4 foobar)

might turn into

(cdr (cdr (cdr (cdr foobar))))

or even

(do ((i 0 (1+ i))
  (list foobar (cdr foobar)))
  ((= i 4) list))

If nth is closed coded, then

(nth x l)

might stay the same, or turn into something like

(car (nthcdr x l))

In general, open coding sacrifices space for speed, but some functions (such as car) are so simple that they are always open-coded. Even when not open-coded, a call to a standard function may be transformed into a different function call (as in the last example) or compiled as static call. Static function call uses a more efficient calling convention that forbids redefinition.

← ↑ → ↺ λ

4.6 Interpreter

By default SBCL implements eval by calling the native code compiler.

SBCL also includes an interpreter for use in special cases where using the compiler is undesirable, for example due to compilation overhead. Unlike in some other Lisp implementations, in SBCL interpreted code is not safer or more debuggable than compiled code.

← ↑ → ↺ λ

4.7 Advanced Compiler Use and Efficiency Hints

For more advanced usages of the compiler, please see the chapter of the same name in the CMUCL manual. Many aspects of the compiler have stayed exactly the same, and there is a much more detailed explanation of the compiler's behavior and how to maximally optimize code in their manual. In particular, while SBCL no longer supports byte-code compilation, it does support CMUCL's block compilation facility allowing whole program optimization and increased use of the local call convention.

Unlike CMUCL, SBCL is able to open-code forward-referenced type tests while block compiling. This helps for mutually referential defstructs in particular.

← ↑ → ↺ λ

5 Debugger

This chapter documents the debugging facilities of SBCL, including the debugger, single-stepper and trace(0 1), and the effect of (optimize debug) declarations.

← ↑ → ↺ λ

5.1 Debugger Entry

← ↑ → ↺ λ

5.1.1 Debugger Banner

When you enter the debugger, it looks something like this:

debugger invoked on a TYPE-ERROR in thread 11184:
  The value 3 is not of type LIST.

You can type HELP for debugger help, or (SB-EXT:QUIT) to exit from SBCL.

restarts (invokable by number or by possibly-abbreviated name):
  0: [ABORT   ] Reduce debugger level (leaving debugger, returning to toplevel).
  1: [TOPLEVEL] Restart at toplevel READ/EVAL/PRINT loop.
(CAR 1 3)
0]

The first group of lines describe what the error was that put us in the debugger. In this case car was called on 3, causing a type-error.

This is followed by the "beginner help line", which appears only if sb-debug:*debug-beginner-help-p* is true (default).

Next comes a listing of the active restart names, along with their descriptions -- the ways we can restart execution after this error. In this case, both options return to top-level. Restarts can be selected by entering the corresponding number or name.

The current frame appears right underneath the restarts, immediately followed by the debugger prompt.

← ↑ → ↺ λ

5.1.2 Debugger Invocation

The debugger is invoked when:

When the debugger is invoked by a condition, ANSI mandates that the value of *debugger-hook*, if any, be called with two arguments: the condition that caused the debugger to be invoked and the previous value of *debugger-hook*. When this happens, *debugger-hook* is bound to nil to prevent recursive errors. However, ANSI also mandates that *debugger-hook* not be invoked when the debugger is to be entered by the break function. For users who wish to provide an alternate debugger interface (and thus catch break entries into the debugger), SBCL provides sb-ext:*invoke-debugger-hook*, which is invoked during any entry into the debugger.

← ↑ → ↺ λ

5.2 Debugger Command Loop

The debugger is an interactive read-eval-print loop much like the normal top level, but some symbols are interpreted as debugger commands instead of being evaluated. A debugger command starts with the symbol name of the command, possibly followed by some arguments on the same line. Some commands prompt for additional input. Debugger commands can be abbreviated by any unambiguous prefix: help can be typed as h, he, etc.

The package is not significant in debugger commands; any symbol with the name of a debugger command will work. If you want to show the value of a variable that happens also to be the name of a debugger command you can wrap the variable in a progn to hide it from the command loop.

The debugger prompt is <frame>], where <frame> is the number of the current frame. Frames are numbered starting from zero at the top (most recent call), increasing down to the bottom. The current frame is the frame that commands refer to.

It is possible to override the normal printing behaviour in the debugger by using the sb-ext:*debug-print-variable-alist*.

← ↑ → ↺ λ

5.3 Stack Frames

A stack frame is the run-time representation of a call to a function; the frame stores the state that a function needs to remember what it is doing. Frames have:

← ↑ → ↺ λ

5.3.1 Stack Motion

These commands move to a new stack frame and print the name of the function and the values of its arguments in the style of a Lisp function call:

← ↑ → ↺ λ

5.3.2 How Arguments are Printed

A frame is printed to look like a function call, but with the actual argument values in the argument positions. So the frame for this call in the source:

(myfun (+ 3 4) 'a)

would look like this:

(MYFUN 7 A)

All keyword and optional arguments are displayed with their actual values; if the corresponding argument was not supplied, the value will be the default. So this call:

(subseq "foo" 1)

would look like this:

(SUBSEQ "foo" 1 3)

And this call:

(string-upcase "test case")

would look like this:

(STRING-UPCASE "test case" :START 0 :END NIL)

The arguments to a function call are displayed by accessing the argument variables. Although those variables are initialized to the actual argument values, they can be set inside the function; in this case the new value will be displayed.

&rest arguments are handled somewhat differently. The value of the rest argument variable is displayed as the spread-out arguments to the call, so:

(format t "~A is a ~A." "This" 'test)

would look like this:

(FORMAT T "~A is a ~A." "This" 'TEST)

Rest arguments cause an exception to the normal display of keyword arguments in functions that have both &rest and &key arguments. In this case, the keyword argument variables are not displayed at all; the rest arg is displayed instead. So for these functions, only the keywords actually supplied will be shown, and the values displayed will be the argument values, not values of the (possibly modified) variables.

If the variable for an argument is never referenced by the function, it will be deleted. The variable value is then unavailable, so the debugger prints #<unused-arg> instead of the value. Similarly, if for any of a number of reasons the value of the variable is unavailable or not known to be available (Variable Access), then #<unavailable-arg> will be printed instead of the argument value.

Note that inline expansion and open-coding affect what frames are present in the debugger, see Debugger Policy Control.

← ↑ → ↺ λ

5.3.3 Function Names

If a function is defined by defun it will appear in backtrace by that name. Functions defined by labels and flet will appear as (FLET <name>) and (LABELS <name>) respectively. Anonymous lambdas will appear as (LAMBDA <lambda-list>).

← ↑ → ↺ λ

Entry Point Details

Sometimes the compiler introduces new functions that are used to implement a user function, but are not directly specified in the source. This is mostly done for argument type and count checking.

With recursive or block compiled functions, an additional external frame may appear before the frame representing the first call to the recursive function or entry to the compiled block. This is a consequence of the way the compiler works: there is nothing odd with your program. You may also see cleanup frames during the execution of unwind-protect cleanup code, and optional for variable argument entry points.

← ↑ → ↺ λ

5.3.4 Debug Tail Recursion

The compiler is properly tail recursive. If a function call is in a tail-recursive position, the stack frame will be deallocated at the time of the call, rather than after the call returns. Consider this backtrace:

(BAR ...)
(FOO ...)

Because of tail recursion, it is not necessarily the case that foo directly called bar. It may be that foo called some other function foo2, which then called bar tail-recursively, as in this example:

(defun foo ()
  ...
  (foo2 ...)
  ...)

(defun foo2 (...)
  ...
  (bar ...))

(defun bar (...)
  ...)

Usually the elimination of tail-recursive frames makes debugging more pleasant, since these frames are mostly uninformative. If there is any doubt about how one function called another, it can usually be eliminated by finding the source location in the calling frame. See Source Location Printing.

The elimination of tail-recursive frames can be prevented by disabling tail-recursion optimization, which happens when the debug optimization quality is greater than 2. See Debugger Policy Control.

← ↑ → ↺ λ

5.3.5 Unknown Locations and Interrupts

The debugger operates using special debugging information attached to the compiled code. This debug information tells the debugger what it needs to know about the locations in the code where the debugger can be invoked. If the debugger somehow encounters a location not described in the debug information, then it is said to be unknown. If the code location for a frame is unknown, then some variables may be inaccessible, and the source location cannot be precisely displayed.

There are three reasons why a code location could be unknown:

In the last two cases, the values of argument variables are accessible, but may be incorrect. For more details on when variable values are accessible, see Variable Value Availability.

It is possible for an interrupt to happen when a function call or return is in progress. The debugger may then flame out with some obscure error or insist that the bottom of the stack has been reached, when the real problem is that the current stack frame can't be located. If this happens, return from the interrupt and try again.

← ↑ → ↺ λ

5.4 Variable Access

There are two ways to access the current frame's local variables in the debugger: list-locals and sb-debug:var.

The debugger doesn't really understand lexical scoping; it has just one namespace for all the variables in the current stack frame. If a symbol is the name of multiple variables in the same function, then the reference appears ambiguous, even though lexical scoping specifies which value is visible at any given source location. If the scopes of the two variables are not nested, then the debugger can resolve the ambiguity by observing that only one variable is accessible.

When there are ambiguous variables, the evaluator assigns each one a small integer identifier. The sb-debug:var function uses this identifier to distinguish between ambiguous variables. The list-locals command prints the identifier. In the following example, there are two variables named x. The first one has identifier 0 (which is not printed), the second one has identifier 1.

X  =  1
X#1  =  2

← ↑ → ↺ λ

5.4.1 Variable Value Availability

The value of a variable may be unavailable to the debugger in portions of the program where Lisp says that the variable is defined. If a variable value is not available, the debugger will not let you read or write that variable. With one exception, the debugger will never display an incorrect value for a variable. Rather than displaying incorrect values, the debugger tells you the value is unavailable.

The one exception is this: if you interrupt (e.g. with C-c) or if there is an unexpected hardware error such as a bus error (which should only happen in unsafe code), then the values displayed for arguments to the interrupted frame might be incorrect. This exception applies only to the interrupted frame: any frame farther down the stack will be fine.

Note: Since the location of an interrupt or hardware error will always be an unknown location, non-argument variable values will never be available in the interrupted frame. See Unknown Locations and Interrupts.)

The value of a variable may be unavailable for these reasons:

Since it is especially useful to be able to get the arguments to a function, argument variables are treated specially when the speed optimization quality is less than 3 and the debug quality is at least 1. With this compilation policy, the values of argument variables are almost always available everywhere in the function, even at unknown locations. For non-argument variables, debug must be at least 2 for values to be available, and even then, values are only available at known locations.

← ↑ → ↺ λ

5.4.2 Note On Lexical Variable Access

When the debugger command loop establishes variable bindings for available variables, these variable bindings have lexical scope and dynamic extent. You can close over them, but such closures can't be used as upward function arguments.

Note: The variable bindings are actually created using the Lisp symbol-macrolet special form.

You can also set local variables using setq, but if the variable was closed over in the original source and never set, then setting the variable in the debugger may not change the value in all the functions the variable is defined in. Another risk of setting variables is that you may assign a value of a type that the compiler proved the variable could never take on. This may result in bad things happening.

← ↑ → ↺ λ

5.5 Source Location Printing

One of the debugger's capabilities is source level debugging of compiled code. These commands display the source location for the current frame:

The source form for a location in the code is the innermost list present in the original source that encloses the form responsible for generating that code. If the actual source form is not a list, then some enclosing list will be printed. For example, if the source form was a reference to the variable *some-random-special*, then the innermost enclosing evaluated form will be printed. Here are some possible enclosing forms:

(let ((a *some-random-special*))
  ...)

(+ *some-random-special* ...)

If the code at a location was generated from the expansion of a macro or a source-level compiler optimization, then the form in the original source that expanded into that code will be printed. Suppose the file /usr/me/mystuff.lisp looked like this:

(defmacro mymac ()
  '(myfun))

(defun foo ()
  (mymac)
  ...)

If foo has called myfun, and is waiting for it to return, then the source command would print:

; File: /usr/me/mystuff.lisp

(MYMAC)

Note that the macro use was printed, not the actual function call form, (myfun).

If enclosing source is printed by giving an argument to source or vsource, then the actual source form is marked by wrapping it in a list whose first element is #:***here***. In the previous example, source 1 would print:

; File: /usr/me/mystuff.lisp

(DEFUN FOO ()
  (#:***HERE***
   (MYMAC))
  ...)

← ↑ → ↺ λ

5.5.1 How the Source is Found

If the code was defined from Lisp by compile or eval, then the source can always be reliably located. If the code was defined from a FASL file created by compile-file, then the debugger gets the source forms it prints by reading them from the original source file. This is a potential problem, since the source file might have moved or changed since the time it was compiled.

The source file is opened using the truename of the source file pathname originally given to the compiler. This is an absolute pathname with all logical names and symbolic links expanded. If the file can't be located using this name, then the debugger gives up and signals an error.

If the source file can be found, but has been modified since the time it was compiled, the debugger prints this warning:

; File has been modified since compilation:
;   <filename>
; Using form offset instead of character position.

where <filename> is the name of the source file. It then proceeds using a robust but not foolproof heuristic for locating the source. This heuristic works if:

If the heuristic doesn't work, the displayed source will be wrong, but will probably be near the actual source. If the "shape" of the top-level form in the source file is too different from the original form, then an error will be signaled. When the heuristic is used, the source location commands are noticeably slowed.

Source location printing can also be confused if (after the source was compiled) a read-macro you used in the code was redefined to expand into something different, or if a read-macro ever returns the same eq list twice. If you don't define read macros and don't use ## in perverted ways, you don't need to worry about this.

← ↑ → ↺ λ

5.5.2 Source Location Availability

Source location information is only available when the debug optimization quality is at least 2. If source location information is unavailable, the source commands will give an error message.

If source location information is available, but the source location is unknown because of an interrupt or unexpected hardware error (see Unknown Locations and Interrupts), then the command will print

Unknown location: using block start.

and then proceed to print the source location for the start of the basic block enclosing the code location. It's a bit complicated to explain exactly what a basic block is, but here are some properties of the block start location:

In other words, the true location lies between the printed location and the next conditional (but watch out because the compiler may have changed the program on you.)

← ↑ → ↺ λ

5.6 Debugger Policy Control

The compilation policy specified by optimize declarations affects the behavior seen in the debugger. The debug quality directly affects the debugger by controlling the amount of debugger information dumped. Other optimization qualities have indirect but observable effects due to changes in the way compilation is done.

Unlike the other optimization qualities (which are compared in relative value to evaluate tradeoffs), the debug optimization quality is directly translated to a level of debug information. This absolute interpretation allows the user to count on a particular amount of debug information being available even when the values of the other qualities are changed during compilation. These are the levels of debug information that correspond to the values of the debug quality:

Inlining of local functions is inhibited so that they may be trace(0 1)d.

As you can see, if the speed quality is 3, debugger performance is degraded. This effect comes from the elimination of argument variable special-casing (see Variable Value Availability). Some degree of speed/debuggability tradeoff is unavoidable, but the effect is not too drastic when debug is at least 2.

In addition to inline and notinline declarations, the relative values of the speed and space qualities also change whether functions are inline expanded. If a function is inline expanded, then there will be no frame to represent the call, and the arguments will be treated like any other local variable. Functions may also be semi-inline, in which case there is a frame to represent the call, but the call is to an optimized local version of the function, not to the original function.

← ↑ → ↺ λ

5.7 Exiting Commands

These commands get you out of the debugger.

← ↑ → ↺ λ

5.8 Information Commands

Most of these commands print information about the current frame or function, but a few show general information.

← ↑ → ↺ λ

5.9 Breakpoint Commands

SBCL supports setting of breakpoints inside compiled functions and stepping of compiled code. Breakpoints can only be set at known locations (see Unknown Locations and Interrupts), so these commands are largely useless unless the debug optimize quality is at least 2 (see Debugger Policy Control). These commands manipulate breakpoints:

← ↑ → ↺ λ

5.9.1 Breakpoint Example

Consider this definition of the factorial function:

(defun ! (n)
  (if (zerop n)
      1
      (* n (! (1- n)))))

This debugger session demonstrates the use of breakpoints:

* (break)  ; invoke debugger

debugger invoked on a SIMPLE-CONDITION in thread 11184: break

restarts (invokable by number or by possibly-abbreviated name):
  0: [CONTINUE] Return from BREAK.
  1: [ABORT   ] Reduce debugger level (leaving debugger, returning to toplevel).
  2: [TOPLEVEL] Restart at toplevel READ/EVAL/PRINT loop.
("varargs entry for top level local call BREAK" "break")
0] ll #'!

0-1: (SB-INT:NAMED-LAMBDA ! (N) (BLOCK ! (IF (ZEROP N) 1 (* N (! #)))))
2: (BLOCK ! (IF (ZEROP N) 1 (* N (! (1- N)))))
3: (ZEROP N)
4: (* N (! (1- N)))
5: (1- N)
6: (! (1- N))
7-8: (* N (! (1- N)))
9-10: (IF (ZEROP N) 1 (* N (! (1- N))))
0] br 4

(* N (! (1- N)))
1: 4 in !
added
0] toplevel

> (! 10) ; Call the function

*Breakpoint hit*

Restarts:
  0: [CONTINUE] Return from BREAK.
  1: [ABORT   ] Return to Top-Level.

Debug  (type H for help)

(! 10) ; We are now in first call (arg 10) before the multiply
Source: (* N (! (1- N)))
3] step*

*Step*

(! 10) ; We have finished evaluation of (1- n)
Source: (1- N)
3] step*

*Breakpoint hit*

Restarts:
  0: [CONTINUE] Return from BREAK.
  1: [ABORT   ] Return to Top-Level.

Debug  (type H for help)

(! 9) ; We hit the breakpoint in the recursive call
Source: (* N (! (1- N)))
3]

Note: The step* command differs from the single stepping commands in that it also functions in compiled code which has not been compiled with stepping instrumentation. It simply steps to the next compiled code location. In the future, this form of stepping may be improved enough to subsume the instrumentation based stepping commands, which have much higher overhead.

← ↑ → ↺ λ

5.10 Function Tracing

The tracer causes selected functions to print their arguments and their results whenever they are called. Options allow conditional printing of the trace information and conditional breakpoints on function entry or exit.

In SBCL, tracing can be done either by temporarily redefining the function name (encapsulation), or using breakpoints. When breakpoints are used, the function object itself is destructively modified to cause the tracing action. The advantage of using breakpoints is that tracing works even when the function is anonymously called via funcall, that function object identity is preserved, and that anonymous and local functions can also be traced.

In the case of functions where the known return convention is used to optimize, encapsulation may be necessary in order to make tracing work at all. The symptom of this occurring is an error stating

Error in function FOO: :FUNCTION-END breakpoints are
currently unsupported for the known return convention.

in such cases we recommend using (TRACE FOO :ENCAPSULATE t).

← ↑ → ↺ λ

5.11 Single Stepping

SBCL includes an instrumentation based single-stepper for compiled code, that can be invoked via the step macro, or from within the debugger. See Debugger Policy Control, for details on enabling stepping for compiled code.

The following debugger commands are used for controlling single stepping.

← ↑ → ↺ λ

5.12 Enabling and Disabling the Debugger

In certain contexts (e.g. non-interactive applications), it may be desirable to turn off the SBCL debugger (and possibly re-enable it). The functions here control the debugger.

← ↑ → ↺ λ

6 Efficiency

← ↑ → ↺ λ

6.1 Slot Access

← ↑ → ↺ λ

6.1.1 Structure Object Slot Access

Structure slot accessors are efficient only if the compiler is able to open code them: compiling a call to a structure slot accessor before the structure is defined, declaring one notinline, or passing it as a functional argument to another function causes severe performance degradation.

← ↑ → ↺ λ

6.1.2 Standard Object Slot Access

The most efficient way to access a slot of a standard-object is by using slot-value with a constant slot name argument inside a defmethod body, where the variable holding the instance is a specializer parameter of the method and is never assigned to. The cost is roughly 1.6 times that of an open coded structure slot accessor.

Second most efficient way is to use a CLOS slot accessor, or slot-value with a constant slot name argument, but in circumstances other than specified above. This may be up to 3 times as slow as the method described above.

Example:

(defclass foo () ((bar)))

;; Fast: specializer and never assigned to
(defmethod quux ((foo foo) new)
  (let ((old (slot-value foo 'bar)))
    (setf (slot-value foo 'bar) new)
    old))

;; Slow: not a specializer
(defmethod quux ((foo foo) new)
  (let* ((temp foo)
         (old (slot-value temp 'bar)))
    (setf (slot-value temp 'bar) new)
    old))

;; Slow: assignment to FOO
(defmethod quux ((foo foo) new)
  (let ((old (slot-value foo 'bar)))
    (setf (slot-value foo 'bar) new)
    (setf foo new)
    old))

Note that when profiling code such as this, the first few calls to the generic function are not representative, as the dispatch mechanism is lazily set up during those calls.

← ↑ → ↺ λ

6.2 Stack Allocation

SBCL has fairly extensive support for performing allocations on the stack when a variable or function is declared dynamic-extent. The dynamic-extent declarations are not verified but are simply trusted as long as sb-ext:*stack-allocate-dynamic-extent* is true.

SBCL recognizes any value which a variable declared dynamic-extent can take on as having dynamic extent. This means that, in addition to the value a variable is bound to initially, a value assigned to a variable by setq is also recognized as having dynamic extent when the variable is declared dynamic-extent. Users can thus build complex structures on the stack using iteration and setq.

At present, SBCL implements stack allocation for the following kinds of values when they are recognized as having dynamic extent:

Examples:

;;; Declaiming a structure constructor inline before definition makes
;;; stack allocation possible.
(declaim (inline make-thing))
(defstruct thing obj next)

;;; Stack allocation of various objects bound to DYNAMIC-EXTENT
;;; variables.
(let* ((list (list 1 2 3))
       (nested (cons (list 1 2) (list* 3 4 (list 5))))
       (vector (make-array 3 :element-type 'single-float))
       (thing (make-thing :obj list
                          :next (make-thing :obj (make-array 3))))
       (closure (let ((y ...)) (lambda () y))))
  (declare (dynamic-extent list nested vector thing closure))
  ...)

;;; Stack allocation of objects assigned to DYNAMIC-EXTENT variables.
(let ((x nil))
  (declare (dynamic-extent x))
  (setq x (list 1 2 3))
  (dotimes (i 10)
    (setq x (cons i x)))
  ...)

;;; Stack allocation of arguments to a local function is equivalent
;;; to stack allocation of local variable values.
(flet ((f (x)
         (declare (dynamic-extent x))
         ...))
  ...
  (f (list 1 2 3))
  (f (cons (cons 1 2) (cons 3 4)))
  ...)

;;; Stack allocation of &REST lists
(defun foo (&rest args)
  (declare (dynamic-extent args))
  ...)

As a notable exception to recognizing otherwise inaccessible parts of other recognized dynamic extent values, SBCL does not as of 1.0.48.21 propagate dynamic-extentness through &rest arguments -- but another conforming implementation might, so portable code should not rely on this.

(declaim (inline foo))
(defun foo (fun &rest arguments)
  (declare (dynamic-extent arguments))
  (apply fun arguments))

(defun bar (a)
  ;; SBCL will heap allocate the result of (LIST A), and stack
  ;; allocate only the spine of the &rest list -- so this is
  ;; safe but unportable.
  ;;
  ;; Another implementation, including earlier versions of SBCL
  ;; might consider (LIST A) to be otherwise inaccessible and
  ;; stack-allocate it as well!
  (foo #'car (list a)))

If dynamic extent constraints specified in the Common Lisp standard are violated, the best that can happen is for the program to have garbage in variables and return values; more commonly, the system will crash.

In particular, it is important to realize that this can interact in suprising ways with the otherwise inaccessible parts criterion:

(let* ((a (list 1 2 3))
       (b (cons a a)))
   (declare (dynamic-extent b))
   ;; Unless A is accessed elsewhere as well, SBCL will consider
   ;; it to be otherwise inaccessible -- it can only be accessed
   ;; through B, after all -- and stack allocate it as well.
   ;;
   ;; Hence returning (CAR B) here is unsafe.
   ...)

SBCL also performs sophisticated escape analysis to enable automatic stack allocation of local functions without any bound dynamic extent declarations in many situations where the compiler can prove that no uses escape (traditional Lisp terminology names this situation "all uses are downward funargs"). For example, in the following function, the local function #'predicatep is stack allocated, because the compiler understands that the built-in function position-if only uses its first argument as a downward funarg:

(let ((acc 0))
  (flet ((predicatep (num) (plusp (+ num off))))
    (dotimes (i 10)
      (incf acc (position-if #'predicatep array)))
    (if (plusp off)
        (incf acc (if (positivep acc) 10 3))
        (incf acc (position-if #'predicatep array))))
  acc)

Users can also declare that their own functions take downward funargs by adding bound dynamic extent declarations on the function arguments.

(defun trivial-hof (fun arg)
  (declare (dynamic-extent fun))
  (funcall fun 3 arg))

Currently, such dynamic extent declarations only cause stack allocation of downward funargs at call sites on sufficiently unsafe policy. This is partly because the compiler is currently not able to detect incorrect usage of dynamic extent declarations.

(defun autodxclosure1 (&optional (x 4))
  ;; Calling a higher-order function will only implicitly
  ;; stack-allocate a funarg if the callee is trusted (a CL:
  ;; function) or the caller is unsafe.
  (declare (optimize speed (safety 0) (debug 0)))
  (trivial-hof (lambda (a b) (+ a b x)) 92))

← ↑ → ↺ λ

6.3 Modular Arithmetic

Some numeric functions have a property: n lower bits of the result depend only on n lower bits of (all or some) arguments. If the compiler sees an expression of form (LOGAND <expr> <mask>), where <expr> is a tree of such good functions and <mask> is known to be of type (UNSIGNED-BYTE <w>), where <w> is a good width, all intermediate results will be cut to <w> bits (but it is not done for variables and constants!). This often results in an ability to use simple machine instructions for the functions.

Consider this example:

(defun i (x y)
  (declare (type (unsigned-byte 32) x y))
  (ldb (byte 32 0) (logxor x (lognot y))))

The result of (lognot y) will be negative and of type (signed-byte 33), so a naive implementation on a 32-bit platform is unable to use 32-bit arithmetic here. But modular arithmetic optimizer is able to do it: because the result is cut down to 32 bits, the compiler will replace logxor and lognot with versions cutting results to 32 bits, and because terminals (here, expressions x and y) are also of type (unsigned-byte 32), 32-bit machine arithmetic can be used.

As of SBCL 0.8.5 good functions are +(0 1), -(0 1), logand, logior, logxor, lognot and their combinations; and ash with the positive second argument. Good widths are 32 on 32-bit CPUs and 64 on 64-bit CPUs. While it is possible to support smaller widths as well, currently this is not implemented.

← ↑ → ↺ λ

6.3.1 Signed Modular Arithmetic

Sign-extending the result in the following way will be translated into signed modular arithmetic:

(defun add (a b)
  (declare (type (signed-byte 64) a b))
  (let ((u (ldb (byte 64 0) (+ a b))))
    (logior u (- (mask-field (byte 1 63) u)))))

← ↑ → ↺ λ

6.4 Recognized Idioms

Common Lisp doesn't directly expose all features present in modern hardware. Some code patterns are recognized and turned into more efficient hardware instructions without requiring the use of internal features.

← ↑ → ↺ λ

6.4.1 Count Trailing Zeros

(defun ctz (n)
  (declare (type (unsigned-byte 64) n))
  (integer-length (ldb (byte 64 0) (lognor n (- n)))))

is turned into hardware instructions on arm64 and x86-64. It returns 64 when n is 0. n can also be (signed-byte 64) or fixnum.

← ↑ → ↺ λ

6.5 Global and Always-bound Variables

← ↑ → ↺ λ

6.6 Miscellaneous Efficiency Issues

FIXME: The material in the CMUCL manual about getting good performance from the compiler should be reviewed, reformatted in Texinfo, lightly edited for SBCL, and substituted into this manual. In the meantime, the original CMUCL manual is still 95+% correct for the SBCL version of the Python compiler. See the sections

Besides this information from the CMUCL manual, there are a few other points to keep in mind.

Finally, note that Common Lisp defines many constructs which, in the infamous phrase, "could be compiled efficiently by a sufficiently smart compiler". The phrase is infamous because making a compiler which actually is sufficiently smart to find all these optimizations systematically is well beyond the state of the art of current compiler technology. Instead, they're optimized on a case-by-case basis by hand-written code, or not optimized at all if the appropriate case hasn't been hand-coded. Some cases where no such hand-coding has been done as of SBCL version 0.6.3 include

If your system's performance is suffering because of some construct which could in principle be compiled efficiently, but which the SBCL compiler can't in practice compile efficiently, consider writing a patch to the compiler and submitting it for inclusion in the main sources. Such code is often reasonably straightforward to write; search the sources for the string deftransform to find many examples (some straightforward, some less so).

← ↑ → ↺ λ

7 Beyond the ANSI Standard

SBCL is derived from CMUCL, which implements many extensions to the ANSI standard. SBCL doesn't support as many extensions as CMUCL, but it still has quite a few. See Contributed Modules.

← ↑ → ↺ λ

7.1 Reader Extensions

← ↑ → ↺ λ

7.1.1 Extended Package Prefix Syntax

SBCL supports extended package prefix syntax, which allows specifying an alternate package instead of *package* for the reader to use as the default package for interning symbols:

<package-name>::<form-with-interning-into-package>

Example:

'foo::(bar quux zot) == '(foo::bar foo::quux foo::zot)

*package* is not rebound during the course of reading a form with extended package prefix syntax; if foo::bar would cause a read-time package lock violation, so does foo::(bar).

← ↑ → ↺ λ

7.1.2 Symbol Name Normalization

SBCL also extends the reader to normalize all symbols to Normalization Form KC in builds with Unicode enabled. Whether symbols are normalized is controlled by

Symbols created by intern and similar functions are not affected by this setting. If sb-ext:readtable-normalization is t, symbols that are not normalized are escaped during printing.

← ↑ → ↺ λ

7.1.3 Decimal Syntax for Rationals

SBCL supports a decimal syntax for rationals, modelled after the standard syntax for floating-point numbers. If a number with floating-point syntax has an exponent marker of r or r (rather than one of the standard exponent markers), it is read as the rational with the exact value of the decimal number expressed as a float.

In addition, setting or binding the value of *read-default-float-format* to rational(0 1) around a call to read or read-from-string has the effect that floating-point numbers without exponent markers are read as rational numbers, as if there had been an explicit r or r marker.

Floating point numbers of all types are printed with an exponent marker while the value of *read-default-float-format* is rational; however, rational numbers are printed in their standard syntax, irrespective of the value of *read-default-float-format*.

← ↑ → ↺ λ

7.2 Package-Local Nicknames

SBCL allows giving packages local nicknames: they allow short and easy-to-use names to be used without fear of name conflict associated with normal nicknames.

A local nickname is valid only when inside the package for which it has been specified. Different packages can use same local nickname for different global names, or different local nickname for same global name.

The symbol :package-local-nicknames in *features* denotes the support for this feature.

defpackage options are extended to include

:local-nicknames (<local-nickname> <actual-package-name>)*

with the semantics of adding the package package-local nicknames <local-nickname>s for the corresponding <actual-package-name>s.

Example:

(defpackage :bar (:intern "X"))
(defpackage :foo (:intern "X"))
(defpackage :quux (:use :cl) (:local-nicknames (:bar :foo) (:foo :bar)))
(find-symbol "X" :foo) ; => FOO::X
(find-symbol "X" :bar) ; => BAR::X
(let ((*package* (find-package :quux)))
  (find-symbol "X" :foo))               ; => BAR::X
(let ((*package* (find-package :quux)))
  (find-symbol "X" :bar))               ; => FOO::X

← ↑ → ↺ λ

7.3 Package Variance

defpackage clhs specifies that if the new definition is at variance with the current state of that package, the consequences are undefined. SBCL by default signals a full warning and retains as much of the package state as possible. This can be adjusted with the following variable.

← ↑ → ↺ λ

7.4 Garbage Collection

SBCL provides additional garbage collection functionality not specified by ANSI.

← ↑ → ↺ λ

7.4.1 Finalization

Finalization allows code to be executed after an object has been garbage collected. This is useful for example for releasing foreign memory associated with a Lisp object.

← ↑ → ↺ λ

7.4.2 Weak Pointers

Weak pointers allow references to objects to be maintained without keeping them from being garbage collected: useful for building caches among other things.

Hash tables can also have weak keys and values. See Hash Table Extensions.

← ↑ → ↺ λ

7.4.3 Introspection and Tuning

← ↑ → ↺ λ

7.4.4 Tracing Live Objects Back to Roots

This feature is intended to help expert users diagnose rare low-level issues and should not be needed during normal usage. On top of that, the interface and implementation are experimental and may change at any time without further notice.

It is sometimes important to understand why a given object is retained in the Lisp image instead of being garbage collected. To help with this problem, SBCL provides a mechanism that searches through the different memory spaces, builds a path of references from a root to the object in question and finally reports this paths:

An example of using this could look like this:

* (defvar *my-string* (list 1 2 "my string"))
*MY-STRING*

 * (sb-ext:search-roots (sb-ext:make-weak-pointer (third *my-string*)))
 -> ((SIMPLE-VECTOR 3)) #x10004E9EAF[2] -> (SYMBOL) #x5044100F[1] -> (CONS) #x100181FAE7[1] -> (CONS) #x100181FAF7[1] -> (CONS) #x100181FB07[0] -> #x100181F9AF

The single line of output on *standard-output* shows the path from a root to "my string": the path starts with SBCL's internal package system data structures followed by the symbol (cl-user:*my-string*) followed the three cons cells of the list.

The :print :verbose argument produces similar behavior but describes the path elements in more detail:

* (sb-ext:search-roots (sb-ext:make-weak-pointer (third *my-string*))
                       :print :verbose)
Path to "my string":
 6       10004E9EAF [   2] a (simple-vector 3)
 0         5044100F [   1] COMMON-LISP-USER::*MY-STRING*
 0       100181FAE7 [   1] a cons
 0       100181FAF7 [   1] a cons
 0       100181FB07 [   0] a cons

The :print nil argument is a bit different:

* (sb-ext:search-roots (sb-ext:make-weak-pointer (third *my-string*))
                       :print nil)
(("my string" :STATIC (#(*MY-STRING* 0 0) . 2) (*MY-STRING* . 1)
  ((1 2 "my string") . 1) ((2 "my string") . 1) (("my string") . 0)))

There is no output on *standard-output*, and the return value is a single path for the target object "my string". As before, the path shows the symbol and the three cons cells.

← ↑ → ↺ λ

7.5 Generic Function Dispatch

If a generic function with standard or short method combination is called, and the set of applicable methods does not include any primary methods, then the generic function sb-pcl:no-primary-method will be invoked with the arguments being the invoked generic function and its arguments, similar to the standard function no-applicable-method. As with no-applicable-method, the default method on sb-pcl:no-primary-method signals an error; programmers may define methods on it.

← ↑ → ↺ λ

7.6 Extended Slot Access

The slot access functions slot-value, (setf slot-value), slot-boundp and slot-makunbound are defined to function as expected on conditions (of metaclass sb-pcl::condition-class) and, with some limitations, on structures (of metaclass structure-class).

For structures:

← ↑ → ↺ λ

7.7 Metaobject Protocol

← ↑ → ↺ λ

7.7.1 AMOP Compatibility of Metaobject Protocol

SBCL supports a metaobject protocol which is intended to be compatible with AMOP; present exceptions to this (as distinct from current bugs) are:

← ↑ → ↺ λ

7.7.2 Metaobject Protocol Extensions

In addition, SBCL supports extensions to the Metaobject protocol from AMOP; at present, they are:

← ↑ → ↺ λ

7.8 Extensible Sequences

ANSI Common Lisp has a class sequence with subclasses list(0 1) and vector(0 1), on which the sequence functions like find, subseq, etc. operate. As an extension to the ANSI specification, SBCL allows additional subclasses of sequence to be defined.

A motivation, rationale and additional examples for the design of this extension can be found in the paper Rhodes, Christophe (2007): User-extensible sequences in Common Lisp available for download at http://www.doc.gold.ac.uk/~mas01cr/papers/ilc2007/sequences-20070301.pdf.

Users of this extension just make instances of sequence subclasses and transparently operate on them using sequence functions:

(coerce (subseq (make-instance 'my-sequence) 5 10) 'list)

From this perspective, no distinction between builtin and user-defined sequence subclasses should be necessary.

Providers of the extension, that is of user-defined sequence subclasses, have to adhere to a sequence protocol which consists of a set of generic functions in the sequence package.

A minimal sequence subclass has to specify standard-object and sequence as its superclasses and has to be the specializer of the sequence parameter of methods on at least the following generic functions:

make-sequence-like is needed for functions returning freshly-allocated sequences such as subseq or copy-seq. adjust-sequence is needed for functions which destructively modify their arguments such as delete. In fact, all other sequence functions can be implemented in terms of the above functions and actually are, if no additional methods are defined. However, relying on these generic implementations, in particular not implementing the Iterator Protocol can incur a high performance penalty.

When the sequence protocol is only partially implemented for a given sequence subclass, an attempt to apply one of the missing operations to instances of that class signals the following condition:

In addition to the mandatory functions above, methods on the sequence functions listed below can be defined.

There are some noteworthy irregularities:

The remaining list parallels the Sequence Dictionary, see 17.3 in the ANSI spec.

Counting:

Reversing:

Sorting:

Finding an element:

Finding a position:

Substituting elements:

Removing elements:

Removing duplicates:

← ↑ → ↺ λ

7.8.1 Iterator Protocol

The general iterator protocol allows subsequently accessing some or all elements of a sequence in forward or reverse direction. Users first call sb-sequence:make-sequence-iterator to create an iteration state and receive functions to query and mutate it. These functions allow, among other things, moving to, retrieving or modifying elements of the sequence. The iteration state consists of a state object, a limit object, a from-end indicator and six functions to query or mutate this state.

An iterator is created by calling:

The following convenience macros simplify traversing sequences using iterators:

← ↑ → ↺ λ

7.8.2 Simple Iterator Protocol

For cases in which the full flexibility and performance of the general sequence iterator protocol is not required, there is a simplified sequence iterator protocol consisting of a few generic functions which can be specialized for iterator classes:

Iterator objects implementing the above simple iteration protocol are created by calling the following generic function:

← ↑ → ↺ λ

7.9 Support For Unix

← ↑ → ↺ λ

7.9.1 Running external programs

External programs can be run with sb-ext:run-program.

Note: In SBCL versions prior to 1.0.13, sb-ext:run-program searched for executables in a manner somewhat incompatible with other languages. As of this version, SBCL uses the system library routine execvp(3), and no longer contains the function find-executable-in-search-path, which implemented the old search. Users who need this function may find it in run-program.lisp versions 1.67 and earlier in SBCL's CVS repository here http://sbcl.cvs.sourceforge.net/sbcl/sbcl/src/code/run-program.lisp?view=log. However, we caution such users that this search routine finds executables that system library routines do not.

When sb-ext:run-program is called with :wait nil, an process object is returned. The following functions are available for use with processes:

← ↑ → ↺ λ

7.10 Unicode Support

SBCL provides support for working with Unicode text and querying the standard Unicode database for information about individual codepoints. Unicode-related functions are located in the sb-unicode package.

SBCL also extends ANSI character literal syntax to support Unicode codepoints. You can either specify a character by its Unicode name, with spaces replaced by underscores if a unique name exists or by giving its hexadecimal codepoint preceded by a u, an optional +, and an arbitrary number of leading zeros. You may also input the character directly into your source code if it can be encoded in your file. If a character had an assigned name in Unicode 1.0 that was distinct from its current name, you may also use that name (with spaces replaced by underscores) to specify the character, unless the name is already associated with a codepoint in the latest Unicode standard (such as bell).

Note: Please note that the codepoint u+1f5cf (Page) introduced in Unicode 7.0 is named unicode_page, since the name Page is required to be assigned to form-feed (u+0c) by the ANSI standard.

For example, you can specify the codepoint u+00e1 (_Latin Small Letter A With Acute_) as

← ↑ → ↺ λ

7.10.1 Unicode property access

The following functions can be used to find information about a Unicode codepoint.

← ↑ → ↺ λ

7.10.2 String operations

SBCL can normalize strings using:

SBCL implements the full range of Unicode case operations with the functions

It also extends standard Common Lisp case functions such as string-upcase and string-downcase to support a subset of Unicode's casing behavior. Specifically, a character is both-case-p if its case mapping in Unicode is one-to-one and invertable.

The sb-unicode package also provides functions for collating/sorting strings according to the Unicode Collation Algorithm.

The following functions are provided for detecting visually confusable strings:

← ↑ → ↺ λ

7.10.3 Breaking strings

The sb-unicode package includes several functions for breaking a Unicode string into useful parts.

← ↑ → ↺ λ

7.11 Customization Hooks for Users

The toplevel repl prompt may be customized, and the function that reads user input may be replaced completely. See the :toplevel argument of sb-ext:save-lisp-and-die.

The behaviour of require(0 1) when called with only one argument is implementation-defined. In SBCL, require behaves in the following way:

Although SBCL does not provide a resident editor, the ed function can be customized to hook into user-provided editing mechanisms as follows:

Conditions of type warning and style-warning are sometimes signaled at runtime, especially during execution of Common Lisp defining forms such as defun, defmethod, etc. To muffle these warnings at runtime, SBCL provides a variable sb-ext:*muffled-warnings*:

← ↑ → ↺ λ

7.12 Tools To Help Developers

SBCL provides a profiler and other extensions to the trace(0 1) facility.

The debugger supports a number of options. Its documentation is accessed by typing help at the debugger prompt. See Debugger.

Documentation for the command inspect is accessed by typing help at the inspect prompt.

← ↑ → ↺ λ

7.13 Resolution of Name Conflicts

clhs 11.1.1.2.5 requires that name conflicts in packages be resolvable in favour of any of the conflicting symbols. In the interactive debugger, this is achieved by prompting for the symbol in whose favour the conflict should be resolved; for programmatic use, the sb-ext:resolve-conflict restart should be invoked with one argument, which should be a member of the list returned by the condition accessor sb-ext:name-conflict-symbols.

← ↑ → ↺ λ

7.14 Hash Table Extensions

Hash table extensions supported by SBCL are all controlled by keyword arguments to make-hash-table(0 1).

← ↑ → ↺ λ

7.15 Random Number Generation

The initial value of *random-state* is the same each time SBCL is started. This makes it possible for user code to obtain repeatable pseudo random numbers using only standard-provided functionality. See sb-ext:seed-random-state below for an SBCL extension that allows to seed the random number generator from given data for an additional possibility to achieve this. Non-repeatable random numbers can always be obtained using (make-random-state t).

The sequence of numbers produced by repeated calls to random starting with the same random state and using the same sequence of limit arguments is guaranteed to be reproducible only in the same version of SBCL on the same platform, using the same code under the same evaluator mode and compiler optimization qualities. Just two examples of differences that may occur otherwise: calls to random can be compiled differently depending on how much is known about the limit argument at compile time, yielding different results even if called with the same argument at run time, and the results can differ depending on the machine's word size, for example for limits that are fixnums under 64-bit word size but bignums under 32-bit word size.

Some notes on random floats: The standard doesn't prescribe a specific method of generating random floats. The following paragraph describes SBCL's current implementation and should be taken as purely informational, that is, user code should not depend on any of its specific properties. The method used has been chosen because it is common, conceptually simple and fast.

To generate random floats, SBCL evaluates code that has an equivalent effect as

(* limit
   (float (/ (random (expt 2 23)) (expt 2 23)) 1.0f0))

(for single-floats) and correspondingly (with 52 and 1.0d0 instead of 23 and 1.0f0) for double-floats. Note especially that this means that zero is a possible return value occurring with probability (expt 2 -23) and (expt 2 -52), respectively. Also note that there exist twice as many equidistant floats between 0 and 1 as are generated. For example, the largest number that (random 1.0f0) ever returns is (float (/ (1- (expt 2 23)) (expt 2 23)) 1.0f0) while (float (/ (1- (expt 2 24)) (expt 2 24)) 1.0f0) is the largest single-float less than 1. This is a side effect of the fact that the implementation uses the fastest possible conversion from bits to floats.

SBCL currently uses the Mersenne Twister as its random number generator, specifically the 32-bit version under both 32- and 64-bit word size. The seeding algorithm has been improved several times by the authors of the Mersenne Twister; SBCL uses the third version (from 2002), which is still the most recent as of June 2012. The implementation has been tested to provide output identical to the recommended C implementation.

While the Mersenne Twister generates random numbers of much better statistical quality than other widely used generators, it uses only linear operations modulo 2 and thus fails some statistical tests.

(See chapter 7 Testing widely used RNGs in TestU01: A C Library for Empirical Testing of Random Number Generators by Pierre L'Ecuyer and Richard Simard, ACM Transactions on Mathematical Software, Vol. 33, article 22, 2007.)

For example, the distribution of ranks of (sufficiently large) random binary matrices is much distorted compared to the theoretically expected one when the matrices are generated by the Mersenne Twister. Thus, applications that are sensitive to this aspect should use a different type of generator.

← ↑ → ↺ λ

7.16 Timeouts and Deadlines

SBCL supports three different ways of restricting the execution time available to individual operations or parts of computations:

← ↑ → ↺ λ

7.16.1 Timeout Parameters

Certain operations accept :timeout keyword arguments. These only affect the specific operation and must be specified at each call site by passing a :timeout keyword argument and a corresponding timeout value to the respective operation. Expiration of the timeout before the operation completes results in either a normal return with a return value indicating the timeout or in the signaling of a specialized condition such as sb-thread:join-thread-error.

Example:

(defun join-thread-within-5-seconds (thread)
  (multiple-value-bind (value result)
      (sb-thread:join-thread thread :default nil :timeout 5)
    (when (eq result :timeout)
      (error "Could not join ~A within 5 seconds" thread))
    value))

The above code attempts to join the specified thread for up to five seconds, returning its value in case of success. If the thread is still running after the five seconds have elapsed, sb-thread:join-thread indicates the timeout in its second return value. If a :default value was not provided, sb-thread:join-thread would signal a sb-thread:join-thread-error instead.

To wait for an arbitrary condition, optionally with a timeout, the sb-ext:wait-for macro can be used:

← ↑ → ↺ λ

7.16.2 Synchronous Timeouts

Deadlines, in contrast to timeout parameters, are established for a dynamic scope using the sb-sys:with-deadline macro and indirectly affect operations within that scope. In case of nested uses, the effective deadline is the one that expires first unless an inner use explicitly overrides outer deadlines.

Expiration of deadlines set up this way only has an effect when it happens before or during the execution of a deadline-aware operation (Operations Supporting Timeouts and Deadlines). In this case, a sb-sys:deadline-timeout is signaled. A handler for this condition type may use the sb-sys:defer-deadline or sb-sys:cancel-deadline restarts to defer or cancel the deadline respectively and resume execution of the interrupted operation.

When a thread is executing the debugger, signaling of sb-sys:deadline-timeout conditions for that thread is deferred until it exits the debugger.

Example:

(defun read-input ()
  (list (read-line) (read-line)))

(defun do-it ()
  (sb-sys:with-deadline (:seconds 5))
    (read-input)
    (sleep 2)
    (sb-ext:run-program "my-program"))

The above code establishes a deadline of five seconds within which the body of the do-it function should execute. All calls of deadline-aware functions in the dynamic scope, in this case two read-line calls, a sleep call and a sb-ext:run-program call, are affected by the deadline. If, for example, the first read-line call completes in one second and the second read-line call completes in three seconds, a sb-sys:deadline-timeout condition will be signaled after the sleep call has been executing for one second.

← ↑ → ↺ λ

7.16.3 Asynchronous Timeouts

Asynchronous timeouts are established for a dynamic scope using the sb-ext:with-timeout macro:

Expiration of the timeout will cause the operation being executed at that moment to be interrupted by an asynchronously signaled sb-ext:timeout condition, (almost) irregardless of the operation and its context.

← ↑ → ↺ λ

7.16.4 Operations Supporting Timeouts and Deadlines

| Operation              | Timeout parameter | Affected by deadlines |
|------------------------+-------------------+-----------------------|
| cl:sleep               | -                 | since SBCL 1.4.3      |
| cl:read-line, etc.     | no                | yes                   |
| wait-for               | yes               | yes                   |
| process-wait           | no                | yes                   |
| grab-mutex             | yes               | yes                   |
| condition-wait         | yes               | yes                   |
| wait-on-semaphore      | yes               | yes                   |
| join-thread            | yes               | yes                   |
| receive-message        | yes               | yes?                  |
| wait-on-gate           | yes               | yes?                  |
| frlock-write           | yes               | yes?                  |
| grab-frlock-write-lock | yes               | yes?                  |

← ↑ → ↺ λ

7.17 Miscellaneous Extensions

← ↑ → ↺ λ

7.18 Stale Extensions

SBCL has inherited from CMUCL various hooks to allow the user to tweak and monitor the garbage collection process. These are somewhat stale code, and their interface might need to be cleaned up. If you have urgent need of them, look at the code in src/code/gc.lisp and bring it up on the developers' mailing list.

SBCL has various hooks inherited from CMUCL, like sb-ext:float-denormalized-p, to allow a program to take advantage of IEEE floating point arithmetic properties which aren't conveniently or efficiently expressible using the ANSI standard. These look good, and their interface looks good, but IEEE support is slightly broken due to a stupid decision to remove some support for infinities (because it wasn't in the ANSI spec and it didn't occur to me that it was in the IEEE spec). If you need this stuff, take a look at the code and bring it up on the developers' mailing list.

← ↑ → ↺ λ

7.19 Efficiency Hacks

The sb-ext:purify function (available when #+cheneygc) causes SBCL first to collect all garbage, then to mark all uncollected objects as permanent, never again attempting to collect them as garbage. This can cause a large increase in efficiency when using a primitive garbage collector, or a more moderate increase in efficiency when using a more sophisticated garbage collector which is well suited to the program's memory usage pattern. It also allows permanent code to be frozen at fixed addresses, a precondition for using copy-on-write to share code between multiple Lisp processes. This is less important with modern generational garbage collectors, but not all SBCL platforms use such a garbage collector.

The sb-ext:truly-the special form declares the type of the result of the operations, producing its argument; the declaration is not checked. In short: don't use it.

The sb-ext:freeze-type declaration declares that a type will never change, which can make type testing (e.g. with typep) more efficient for structure types.

← ↑ → ↺ λ

8 External Formats

External formats determine the coding of characters from/to sequences of octets when exchanging data with the outside world. Examples of such exchanges are:

Technically, external formats in SBCL are named objects describing coding of characters as well as policies in case de- or encoding is not possible. Each external format has a canonical name and zero or more aliases. User code mostly interacts with external formats by supplying external format designators to functions that use external formats internally.

← ↑ → ↺ λ

8.1 The Default External Format

← ↑ → ↺ λ

8.2 External Format Designators

In situations where an external format designator is required, such as the :external-format argument in calls to open or with-open-file, users may supply the name of an encoding to denote the external format which is applying that encoding to Lisp characters.

In addition to the basic encoding for an external format, options controlling various special cases may be passed, by using a list (whose first element must be an encoding name and whose rest is a plist) as an external file format designator.

More specifically, external format designators can take the following forms:

Valid options for <options-plist> are:

For example:

(with-open-file (stream pathname :external-format '(:utf-8 :replacement #\?))
  (read-line stream))

will read the first line of pathname, replacing any octet sequence that is not valid in the UTF-8 external format with a question mark character.

← ↑ → ↺ λ

8.3 Character Coding Conditions

De- or encoding characters using a given external format is not always possible:

Unless the external format governing the coding uses the :replacement option, SBCL will signal (continuable) errors under the above circumstances. The types of the condition signaled are not currently exported or documented but will be in future SBCL versions.

← ↑ → ↺ λ

8.4 Converting between Strings and Octet Vectors

To encode Lisp strings as octet vectors and decode octet vectors as Lisp strings, the following SBCL-specific functions can be used:

← ↑ → ↺ λ

8.5 Supported External Formats

The following lists the external formats supported by SBCL in the form of the respective canonical name followed by the list of aliases:

← ↑ → ↺ λ

9 Foreign Function Interface

This chapter describes SBCL's interface to C programs and libraries (and, since C interfaces are a sort of lingua franca of the Unix world, to other programs and libraries in general).

Note: In the modern Lisp world, the usual term for this functionality is Foreign Function Interface, or FFI, where despite the mention of function in this term, FFI also refers to direct manipulation of C data structures as well as functions. The traditional CMUCL terminology is Alien Interface, and while that older terminology is no longer used much in the system documentation, it still reflected in names in the implementation, notably in the name of the sb-alien package.

← ↑ → ↺ λ

9.1 Introduction to the Foreign Function Interface

Because of Lisp's emphasis on dynamic memory allocation and garbage collection, Lisp implementations use non-C-like memory representations for objects. This representation mismatch creates friction when a Lisp program must share objects with programs which expect C data. There are three common approaches to establishing communication:

SBCL, like CMUCL before it, relies primarily on the automatic conversion and direct manipulation approaches. The sb-alien package provides a facility wherein foreign values of simple scalar types are automatically converted and complex types are directly manipulated in their foreign representation. Additionally the lower-level System Area Pointers (or SAPs) can be used where necessary to provide untyped access to foreign memory.

Any foreign objects that can't automatically be converted into Lisp values are represented by objects of type sb-alien-internals:alien-value Since Lisp is a dynamically typed language, even foreign objects must have a run-time type; this type information is provided by encapsulating the raw pointer to the foreign data within an sb-alien-internals:alien-value object.

The type language and operations on foreign types are intentionally similar to those of the C language.

← ↑ → ↺ λ

9.2 Foreign Types

Alien types have a description language based on nested list structure. For example the C type

struct foo {
    int a;
    struct foo *b[100];
};

has the corresponding SBCL FFI type

(struct foo
  (a int)
  (b (array (* (struct foo)) 100)))

← ↑ → ↺ λ

9.2.1 Defining Foreign Types

Types may be either named or anonymous. With structure and union types, the name is part of the type specifier, allowing recursively defined types such as:

(struct foo (a (* (struct foo))))

An anonymous structure or union type is specified by using the name nil. The with-alien macro defines a local scope which captures any named type definitions. Other types are not inherently named, but can be given named abbreviations using the define-alien-type macro.

← ↑ → ↺ λ

9.2.2 Foreign Types and Lisp Types

The foreign types form a subsystem of the SBCL type system. An alien type specifier provides a way to use any foreign type as a Lisp type specifier. For example,

(typep foo '(alien (* int)))

can be used to determine whether foo is a pointer to a foreign int. alien type specifiers can be used in the same ways as ordinary Lisp type specifiers (like string(0 1).) Alien type declarations are subject to the same precise type checking as any other declaration. See Precise Type Checking.

Note that the type identifiers used in the foreign type system overlap with native Lisp type specifiers in some cases. For example, the type specifier (alien single-float) is identical to single-float, since foreign floats are automatically converted to Lisp floats. When type-of is called on an alien value that is not automatically converted to a Lisp value, then it will return an alien type specifier.

← ↑ → ↺ λ

9.2.3 Foreign Type Specifiers

Note: All foreign type names are exported from the sb-alien package. Some foreign type names are also symbols in the common-lisp package, in which case they are reexported from the sb-alien package, so that e.g. it is legal to refer to single-float.

These are the basic foreign type specifiers:

Arrays are accessed using deref, passing the indices as additional arguments. Elements are stored in column-major order (as in C), so the first dimension determines only the size of the memory block, and not the layout of the higher dimensions. An array whose first dimension is variable may be specified by using nil as the first dimension. Fixed-size arrays can be allocated as array elements, structure slots or with-alien variables. Dynamic arrays can only be allocated using make-alien.

← ↑ → ↺ λ

9.3 Operations On Foreign Values

This section describes how to read foreign values as Lisp values, how to coerce foreign values to different kinds of foreign values, and how to dynamically allocate and free foreign variables.

← ↑ → ↺ λ

9.3.1 Accessing Foreign Values

← ↑ → ↺ λ

Untyped memory

As noted at the beginning of the chapter, the System Area Pointer facilities allow untyped access to foreign memory. SAPs can be converted to and from the usual typed foreign values using sap-alien and alien-sap, and also to and from integers (raw machine addresses). They should thus be used with caution; corrupting the Lisp heap or other memory with SAPs is trivial.

Similarly named functions exist for accessing other sizes of word, other comparisons, and other conversions. The reader is invited to use apropos and describe for more details:

(apropos "sap" :sb-sys)

← ↑ → ↺ λ

9.3.2 Coercing Foreign Values

← ↑ → ↺ λ

9.3.3 Foreign Dynamic Allocation

Lisp code can call the C standard library functions malloc and free to dynamically allocate and deallocate foreign variables. The Lisp code uses the same allocator as foreign C code, so it's OK for foreign code to call free on the result of Lisp make-alien, or for Lisp code to call free-alien on foreign objects allocated by C code.

← ↑ → ↺ λ

9.4 Foreign Variables

Both local (stack allocated) and external (C global) foreign variables are supported.

← ↑ → ↺ λ

9.4.1 Local Foreign Variables

← ↑ → ↺ λ

9.4.2 External Foreign Variables

External foreign names are strings, and Lisp names are symbols. When an external foreign value is represented using a Lisp variable, there must be a way to convert from one name syntax into the other. The macros extern-alien, define-alien-variable and define-alien-routine use this conversion heuristic:

← ↑ → ↺ λ

9.5 Foreign Data Structure Examples

Now that we have alien types, operations and variables, we can manipulate foreign data structures. This C declaration

struct foo {
    int a;
    struct foo *b[100];
};

can be translated into the following alien type:

(define-alien-type nil
  (struct foo
    (a int)
    (b (array (* (struct foo)) 100))))

Once the foo alien type has been defined as above, the C expression

struct foo f;
f.b[7].a;

can be translated in this way:

(with-alien ((f (struct foo)))
  (slot (deref (slot f 'b) 7) 'a)
  ;;
  ;; Do something with f...
  )

Or consider this example of an external C variable and some accesses:

struct c_struct {
        short x, y;
        char a, b;
        int z;
        c_struct *n;
};
extern struct c_struct *my_struct;
my_struct->x++;
my_struct->a = 5;
my_struct = my_struct->n;

which can be manipulated in Lisp like this:

(define-alien-type nil
  (struct c-struct
          (x short)
          (y short)
          (a char)
          (b char)
          (z int)
          (n (* c-struct))))
(define-alien-variable "my_struct" (* c-struct))
(incf (slot my-struct 'x))
(setf (slot my-struct 'a) 5)
(setq my-struct (slot my-struct 'n))

← ↑ → ↺ λ

9.6 Loading Shared Object Files

Foreign object files can be loaded into the running Lisp process by calling load-shared-object.

← ↑ → ↺ λ

9.7 Foreign Function Calls

The foreign function call interface allows a Lisp program to call many functions written in languages that use the C calling convention.

Lisp sets up various signal handling routines and other environment information when it first starts up, and expects these to be in place at all times. The C functions called by Lisp should not change the environment, especially the signal handlers: the signal handlers installed by Lisp typically have interesting flags set (e.g to request machine context information, or for signal delivery on an alternate stack) which the Lisp runtime relies on for correct operation. Precise details of how this works may change without notice between versions; the source, or the brain of a friendly SBCL developer, is the only documentation. Users of a Lisp built with the :sb-thread feature should also read the section about threads, Threading.

← ↑ → ↺ λ

9.8 Calling Lisp From C

SBCL supports the calling of Lisp functions using the C calling convention. This is useful for both defining callbacks and for creating an interface for calling into Lisp as a shared library directly from C.

The define-alien-callable macro wraps Lisp code and creates a C foreign function which can be called with the C calling convention. On x86-64 and ARM64, callbacks may receive and return structures by value.

The alien-callable-function function returns the foreign callable value associated with any name defined by define-alien-callable, so that we can, for example, pass the callable value to C as a callback.

The with-alien-callable macro wraps Lisp code and establishes local C foreign functions which can be called with the C calling convention. This macro is handy for passing callbacks which close over Lisp values into C.

Note that the garbage collector moves objects, and won't be able to fix up any references in C variables. There are three mechanisms for coping with this:

← ↑ → ↺ λ

9.8.1 Lisp as a Shared Library

SBCL supports the use of Lisp as a shared library that can be used by C programs using the define-alien-callable interface. See the :callable-exports argument of sb-ext:save-lisp-and-die for how to save the Lisp image in a way that allows a C program to initialize the Lisp runtime and the exported symbols. When SBCL is built as a library, it exposes the symbol initialize_lisp which can be used in conjunction with a core initializing global symbols to foreign callables as function pointers and with object code allocating those symbols to initialize the runtime properly. The arguments to initialize_lisp are the same as the arguments to the main sbcl program.

Note: There is currently no way to run exit hooks or otherwise undo Lisp initialization gracefully from C.

← ↑ → ↺ λ

9.9 Step-By-Step Example of the Foreign Function Interface

This section presents a complete example of an interface to a somewhat complicated C function.

Suppose you have the following C function which you want to be able to call from Lisp in the file test.c:

struct c_struct
{
  int x;
  char *s;
};

struct c_struct *c_function (i, s, r, a)
    int i;
    char *s;
    struct c_struct *r;
    int a[10];
{
  int j;
  struct c_struct *r2;

  printf("i = %dn", i);
  printf("s = %sn", s);
  printf("r->x = %dn", r->x);
  printf("r->s = %sn", r->s);
  for (j = 0; j < 10; j++) printf("a[%d] = %d.n", j, a[j]);
  r2 = (struct c_struct *) malloc (sizeof(struct c_struct));
  r2->x = i + 5;
  r2->s = "a C string";
  return(r2);
};

It is possible to call this C function from Lisp using the file test.lisp containing

(cl:defpackage "TEST-C-CALL" (:use "CL" "SB-ALIEN" "SB-C-CALL"))
(cl:in-package "TEST-C-CALL")

;;; Define the record C-STRUCT in Lisp.
(define-alien-type nil
    (struct c-struct
            (x int)
            (s c-string)))

;;; Define the Lisp function interface to the C routine.  It returns a
;;; pointer to a record of type C-STRUCT.  It accepts four parameters:
;;; I, an int; S, a pointer to a string; R, a pointer to a C-STRUCT
;;; record; and A, a pointer to the array of 10 ints.
;;;
;;; The INLINE declaration eliminates some efficiency notes about heap
;;; allocation of alien values.
(declaim (inline c-function))
(define-alien-routine c-function
    (* (struct c-struct))
  (i int)
  (s c-string)
  (r (* (struct c-struct)))
  (a (array int 10)))

;;; a function which sets up the parameters to the C function and
;;; actually calls it
(defun call-cfun ()
  (with-alien ((ar (array int 10))
               (c-struct (struct c-struct)))
    (dotimes (i 10)                     ; Fill array.
      (setf (deref ar i) i))
    (setf (slot c-struct 'x) 20)
    (setf (slot c-struct 's) "a Lisp string")

    (with-alien ((res (* (struct c-struct))
                      (c-function 5 "another Lisp string" (addr c-struct) ar)))
      (format t "~&amp;back from C function~%")
      (multiple-value-prog1
          (values (slot res 'x)
                  (slot res 's))

        ;; Deallocate result. (after we are done referring to it:
        ;; "Pillage, *then* burn.")
        (free-alien res)))))

To execute the above example, it is necessary to compile the C routine, e.g. with cc -c test.c && ld -shared -o test.so test.o. In order to enable incremental loading with some linkers, you may need to say cc -G 0 -c test.c.

Once the C code has been compiled, you can start up Lisp and load it in: sbcl. Lisp should start up with its normal prompt.

Within Lisp, compile the Lisp file:

(compile-file "test.lisp")

This step can be done separately. You don't have to recompile every time.

Within Lisp, load the foreign object file to define the necessary symbols:

(load-shared-object "test.so")

Now you can load the compiled Lisp (fasl) file into Lisp:

(load "test.fasl")

And once the Lisp file is loaded, you can call the Lisp routine that sets up the parameters and calls the C function:

(test-c-call::call-cfun)

The C routine should print the following information to standard output:

i = 5
s = another Lisp string
r->x = 20
r->s = a Lisp string
a[0] = 0.
a[1] = 1.
a[2] = 2.
a[3] = 3.
a[4] = 4.
a[5] = 5.
a[6] = 6.
a[7] = 7.
a[8] = 8.
a[9] = 9.

After return from the C function, the Lisp wrapper function should print the following output:

back from C function

And upon return from the Lisp wrapper function, before the next prompt is printed, the Lisp read-eval-print loop should print the following return values:

10
"a C string"

← ↑ → ↺ λ

10 Pathnames

← ↑ → ↺ λ

10.1 Lisp Pathnames

There are many aspects of ANSI Common Lisp's pathname support which are implementation-defined and so need documentation.

← ↑ → ↺ λ

10.1.1 Home Directory Specifiers

SBCL accepts the keyword :home and a list of the form (:home "username") as a directory component immediately following :absolute.

:home is represented in namestrings by ~/ and (:home "username") by ~username/ at the start of the namestring. Tilde-characters elsewhere in namestrings represent themselves.

Home directory specifiers are resolved to home directory of the current or specified user by sb-ext:native-namestring, which is used by the implementation to translate pathnames before passing them on to operating system specific routines.

Using (:home "user") form on Windows signals an error.

← ↑ → ↺ λ

10.1.2 The SYS Logical Pathname Host

The logical pathname host named by "SYS" exists in SBCL. Its logical-pathname-translations may be set by the site or the user applicable to point to the locations of the system's sources; in particular, the core system's source files match the logical pathname "SYS:SRC;**;*.*.*", and the contributed modules' source files match "SYS:CONTRIB;**;*.*.*".

← ↑ → ↺ λ

10.2 Native Filenames

In some circumstances, what is wanted is a Lisp pathname object which corresponds to a string produced by the Operating System. In this case, some of the default parsing rules are inappropriate: most filesystems do not have a native understanding of wild pathnames; such functionality is often provided by shells above the OS, often in mutually-incompatible ways.

To allow the user to deal with this, the following functions are provided: sb-ext:parse-native-namestring and sb-ext:native-pathname return the closest equivalent Lisp pathname to a given string (appropriate for the Operating System), while sb-ext:native-namestring converts a non-wild pathname designator to the equivalent native namestring, if possible. Some Lisp pathname concepts (such as the :back directory component) have no direct equivalents in most Operating Systems; the behaviour of sb-ext:native-namestring is unspecified if an inappropriate pathname designator is passed to it. Additionally, note that conversion from pathname to native filename and back to pathname should not be expected to preserve equivalence under equal.

Because some file systems permit the names of directories to be expressed in multiple ways, it is occasionally necessary to parse a native file name as a directory name or to produce a native file name that names a directory as a file. For these cases, PARSE-NATIVE-NAMESTRING accepts the keyword argument :as-directory to force a filename to parse as a directory, and sb-ext:native-namestring accepts the keyword argument :as-file to force a pathname to unparse as a file. For example,

; On Unix, the directory "/tmp/" can be denoted by "/tmp/" or "/tmp".
; Under the default rules for native filenames, these parse and
; unparse differently.
(defvar *p*)
(setf *p* (parse-native-namestring "/tmp/")) => #P"/tmp/"
(pathname-name *p*) => NIL
(pathname-directory *p*) => (:ABSOLUTE "tmp")
(native-namestring *p*) => "/tmp/"

(setf *p* (parse-native-namestring "/tmp")) => #P"/tmp"
(pathname-name *p*) => "tmp"
(pathname-directory *p*) => (:ABSOLUTE)
(native-namestring *p*) => "/tmp"

; A non-NIL AS-DIRECTORY argument to PARSE-NATIVE-NAMESTRING forces
; both the second string to parse the way the first does.
(setf *p* (parse-native-namestring "/tmp"
                                   nil *default-pathname-defaults*
                                   :as-directory t)) => #P"/tmp/"
(pathname-name *p*) => NIL
(pathname-directory *p*) => (:ABSOLUTE "tmp")

; A non-NIL AS-FILE argument to NATIVE-NAMESTRING forces the pathname
; parsed from the first string to unparse as the second string.
(setf *p* (parse-native-namestring "/tmp/")) => #P"/tmp/"
(native-namestring *p* :as-file t) => "/tmp"

← ↑ → ↺ λ

11 Streams

Streams which read or write Lisp character data from or to the outside world -- files, sockets or other external entities -- require the specification of a conversion between the external, binary data and the Lisp characters. In ANSI Common Lisp, this is done by specifying the :external-format argument when the stream is created. The major information required is an encoding, specified by a keyword naming that encoding; however, it is also possible to specify refinements to that encoding as additional options to the external format designator.

In addition, SBCL supports various extensions of ANSI Common Lisp streams:

← ↑ → ↺ λ

11.1 Stream External Formats

The function stream-external-format returns the canonical name of the external format (See External Formats) used by the stream for character-based input and/or output.

When constructing file streams, for example using open or with-open-file, the external format to use is specified via the :external-format argument which accepts an external format designator (see External Format Designators).

← ↑ → ↺ λ

11.2 Bivalent Streams

A bivalent stream can be used to read and write both character(0 1) and (unsigned-byte 8) values. A bivalent stream is created by calling open with the argument :element-type :default. On such a stream, both binary and character data can be read and written with the usual input and output functions.

Streams are not created bivalent by default for performance reasons. Bivalent streams are incompatible with fast-read-char, an internal optimization in SBCL's stream machinery that bulk-converts octets to characters and implements a fast path through read-char.

← ↑ → ↺ λ

11.3 Gray Streams

The Gray Streams interface is a widely supported extension that provides for definition of CLOS-extensible stream classes. Gray stream classes are implemented by adding methods to generic functions analogous to Common Lisp's standard I/O functions. Instances of Gray stream classes may be used with any I/O operation where a non-Gray stream can, provided that all required methods have been implemented suitably.

← ↑ → ↺ λ

11.3.1 Gray Streams classes

The defined Gray Stream classes are these:

The function input-stream-p will return true of any generalized instance of sb-gray:fundamental-input-stream.

The function output-stream-p will return true of any generalized instance of sb-gray:fundamental-output-stream.

Note that instantiable subclasses of sb-gray:fundamental-binary-stream should provide (or inherit) an applicable method for the generic function stream-element-type.

← ↑ → ↺ λ

11.3.2 Methods common to all streams

These generic functions can be specialized on any generalized instance of fundamental-stream.

← ↑ → ↺ λ

11.3.3 Input stream methods

These generic functions may be specialized on any generalized instance of fundamental-input-stream.

← ↑ → ↺ λ

11.3.4 Character input stream methods

These generic functions are used to implement subclasses of sb-gray:fundamental-input-stream:

← ↑ → ↺ λ

11.3.5 Output stream methods

These generic functions are used to implement subclasses of sb-gray:fundamental-output-stream:

← ↑ → ↺ λ

11.3.6 Character output stream methods

These generic functions are used to implement subclasses of sb-gray:fundamental-character-output-stream:

← ↑ → ↺ λ

11.3.7 Binary stream methods

The following generic functions are available for subclasses of sb-gray:fundamental-binary-stream:

← ↑ → ↺ λ

11.3.8 Gray Streams Examples

Below are two classes of stream that can be conveniently defined as wrappers for Common Lisp streams. These are meant to serve as examples of minimal implementations of the protocols that must be followed when defining Gray streams. Realistic uses of the Gray Streams API would implement the various methods that can do I/O in batches, such as sb-gray:stream-read-line, sb-gray:stream-write-string, sb-gray:stream-read-sequence, and sb-gray:stream-write-sequence.

← ↑ → ↺ λ

Character Counting Input Stream

It is occasionally handy for programs that process input files to count the number of characters and lines seen so far, and the number of characters seen on the current line, so that useful messages may be reported in case of parsing errors, etc. Here is a character input stream class that keeps track of these counts. Note that all character input streams must implement sb-gray:stream-read-char and sb-gray:stream-unread-char.

(defclass wrapped-stream (fundamental-stream)
  ((stream :initarg :stream :reader stream-of)))

(defmethod stream-element-type ((stream wrapped-stream))
  (stream-element-type (stream-of stream)))

(defmethod close ((stream wrapped-stream) &key abort)
  (close (stream-of stream) :abort abort))

(defclass wrapped-character-input-stream
    (wrapped-stream fundamental-character-input-stream)
  ())

(defmethod stream-read-char ((stream wrapped-character-input-stream))
  (read-char (stream-of stream) nil :eof))

(defmethod stream-unread-char ((stream wrapped-character-input-stream)
                               char)
  (unread-char char (stream-of stream)))

(defclass counting-character-input-stream
    (wrapped-character-input-stream)
  ((char-count :initform 1 :accessor char-count-of)
   (line-count :initform 1 :accessor line-count-of)
   (col-count :initform 1 :accessor col-count-of)
   (prev-col-count :initform 1 :accessor prev-col-count-of)))

(defmethod stream-read-char ((stream counting-character-input-stream))
  (with-accessors ((inner-stream stream-of) (chars char-count-of)
                   (lines line-count-of) (cols col-count-of)
                   (prev prev-col-count-of)) stream
      (let ((char (call-next-method)))
        (cond ((eql char :eof)
               :eof)
              ((char= char #Newline)
               (incf lines)
               (incf chars)
               (setf prev cols)
               (setf cols 1)
               char)
              (t
               (incf chars)
               (incf cols)
               char)))))

(defmethod stream-unread-char ((stream counting-character-input-stream)
                               char)
  (with-accessors ((inner-stream stream-of) (chars char-count-of)
                   (lines line-count-of) (cols col-count-of)
                   (prev prev-col-count-of)) stream
      (cond ((char= char #Newline)
             (decf lines)
             (decf chars)
             (setf cols prev))
            (t
             (decf chars)
             (decf cols)
             char))
      (call-next-method)))

The default methods for sb-gray:stream-read-char-no-hang, sb-gray:stream-peek-char, sb-gray:stream-listen, sb-gray:stream-clear-input, sb-gray:stream-read-line, and sb-gray:stream-read-sequence should be sufficient (though the last two will probably be slower than methods that forwarded directly).

Here's a sample use of this class:

(with-input-from-string (input "1 2
 3 :foo  ")
  (let ((counted-stream (make-instance 'counting-character-input-stream
                         :stream input)))
    (loop for thing = (read counted-stream) while thing
       unless (numberp thing) do
         (error "Non-number ~S (line ~D, column ~D)" thing
                (line-count-of counted-stream)
                (- (col-count-of counted-stream)
                   (length (format nil "~S" thing))))
       end
       do (print thing))))

Output:

1
2
3
Non-number :FOO (line 2, column 5)
  [Condition of type SIMPLE-ERROR]

← ↑ → ↺ λ

Output Prefixing Character Stream

One use for a wrapped output stream might be to prefix each line of text with a timestamp, e.g. for a logging stream. Here's a simple stream that does this, though without any fancy line-wrapping. Note that all character output stream classes must implement sb-gray:stream-write-char and sb-gray:stream-line-column.

(defclass wrapped-stream (fundamental-stream)
  ((stream :initarg :stream :reader stream-of)))

(defmethod stream-element-type ((stream wrapped-stream))
  (stream-element-type (stream-of stream)))

(defmethod close ((stream wrapped-stream) &key abort)
  (close (stream-of stream) :abort abort))

(defclass wrapped-character-output-stream
    (wrapped-stream fundamental-character-output-stream)
  ((col-index :initform 0 :accessor col-index-of)))

(defmethod stream-line-column ((stream wrapped-character-output-stream))
  (col-index-of stream))

(defmethod stream-write-char ((stream wrapped-character-output-stream)
                              char)
  (with-accessors ((inner-stream stream-of) (cols col-index-of)) stream
    (write-char char inner-stream)
    (if (char= char #Newline)
        (setf cols 0)
        (incf cols))))

(defclass prefixed-character-output-stream
    (wrapped-character-output-stream)
  ((prefix :initarg :prefix :reader prefix-of)))

(defgeneric write-prefix (prefix stream)
  (:method ((prefix string) stream) (write-string prefix stream))
  (:method ((prefix function) stream) (funcall prefix stream)))

(defmethod stream-write-char ((stream prefixed-character-output-stream)
                              char)
  (with-accessors ((inner-stream stream-of) (cols col-index-of)
                   (prefix prefix-of)) stream
    (when (zerop cols)
      (write-prefix prefix inner-stream))
    (call-next-method)))

As with the example input stream, this implements only the minimal protocol. A production implementation should also provide methods for at least sb-gray:stream-write-string, sb-gray:stream-write-sequence.

And here's a sample use of this class:

(flet ((format-timestamp (stream)
         (apply #'format stream "[~2@*~2,' D:~1@*~2,'0D:~0@*~2,'0D] "
                (multiple-value-list (get-decoded-time)))))
  (let ((output (make-instance 'prefixed-character-output-stream
                               :stream *standard-output*
                               :prefix #'format-timestamp)))
    (loop for string in '("abc" "def" ")ghi") do
         (write-line string output)
         (sleep 1))))

Output:

[ 0:30:05] abc
[ 0:30:06] def
[ 0:30:07] ghi
NIL

← ↑ → ↺ λ

11.4 Simple Streams

Simple streams are an extensible streams protocol that avoids some problems with Gray Streams.

Documentation about simple streams is available at:

http://www.franz.com/support/documentation/6.2/doc/streams.htm

The implementation should be considered Alpha-quality; the basic framework is there, but many classes are just stubs at the moment.

See SYS:CONTRIB;SB-SIMPLE-STREAMS;SIMPLE-STREAM-TEST.LISP for things that should work.

Known differences to the ACL behaviour:

← ↑ → ↺ λ

12 Package Locks

None of the following sections apply to SBCL built without package locking support.

The interface described here is experimental: incompatible changes in future SBCL releases are possible, even expected: the concept of implementation packages and the associated operators may be renamed; more operations (such as naming restarts or catch tags) may be added to the list of operations violating package locks.

← ↑ → ↺ λ

12.1 Package Lock Concepts

Package locks protect against unintentional modifications of a package: they provide similar protection to user packages as is mandated to common-lisp package by the ANSI specification. They are not, and should not be used as, a security measure.

Newly created packages are by default unlocked (see the :lock option to defpackage).

The package common-lisp and SBCL internal implementation packages are locked by default, including sb-ext.

It may be beneficial to lock common-lisp-user as well, to ensure that various libraries don't pollute it without asking, but this is not currently done by default.

← ↑ → ↺ λ

12.1.1 Implementation Packages

Each package has a list of associated implementation packages. A locked package, and the symbols whose home package it is, can be modified without violating package locks only when *package* is bound to one of the implementation packages of the locked package.

Unless explicitly altered by defpackage, sb-ext:add-implementation-package, or sb-ext:remove-implementation-package each package is its own (only) implementation package.

← ↑ → ↺ λ

12.1.2 Package Lock Violations

← ↑ → ↺ λ

Lexical Bindings and Declarations

Lexical bindings or declarations that violate package locks cause a compile-time warning, and a runtime program-error when the form that violates package locks would be executed.

A complete listing of operators affect by this is: let, let*, flet, labels, macrolet, and symbol-macrolet, declare.

Package locks affecting both lexical bindings and declarations can be disabled locally with the sb-ext:disable-package-locks declaration, and re-enabled with the sb-ext:enable-package-locks declaration.

Example:

(in-package :locked)

(defun foo () ...)

(defmacro with-foo (&body body)
  `(locally (declare (disable-package-locks locked:foo))
     (flet ((foo () ...))
       (declare (enable-package-locks locked:foo)) ; re-enable for body
       ,@body)))

← ↑ → ↺ λ

Other Operations

If an non-lexical operation violates a package lock, a continuable error that is of a subtype of sb-ext:package-lock-violation (subtype of package-error) is signalled when the operation is attempted.

Additional restarts may be established for continuable package lock violations for interactive use.

The actual type of the error depends on circumstances that caused the violation: operations on packages signal errors of type sb-ext:package-locked-error, and operations on symbols signal errors of type sb-ext:symbol-package-locked-error.

← ↑ → ↺ λ

12.1.3 Package Locks in Compiled Code

If file-compiled code contains interned symbols, then loading that code into an image without the said symbols will not cause a package lock violation, even if the packages in question are locked.

With the exception of interned symbols, behaviour is unspecified if package locks affecting compiled code are not the same during loading of the code or execution.

Specifically, code compiled with packages unlocked may or may not fail to signal package-lock-violations even if the packages are locked at runtime, and code compiled with packages locked may or may not signal spurious package-lock-violations at runtime even if the packages are unlocked.

In practice all this means that package-locks have a negligible performance penalty in compiled code as long as they are not violated.

← ↑ → ↺ λ

12.1.4 Operations Violating Package Locks

← ↑ → ↺ λ

Operations on Packages

The following actions cause a package lock violation if the package operated on is locked, and *package* is not an implementation package of that package, and the action would cause a change in the state of the package (so e.g. exporting already external symbols is never a violation). Package lock violations caused by these operations signal errors of type sb-ext:package-locked-error.

← ↑ → ↺ λ

Operations on Symbols

Following actions cause a package lock violation if the home package of the symbol operated on is locked, and *package* is not an implementation package of that package. Package lock violations caused by these action signal errors of type sb-ext:symbol-package-locked-error.

These actions cause only one package lock violation per lexically apparent violated package.

Example:

;;; Packages FOO and BAR are locked.
;;;
;;; Two lexically apparent violated packages: exactly two
;;; package-locked-errors will be signalled.

(defclass foo:point ()
  ((x :accessor bar:x)
   (y :accessor bar:y)))

← ↑ → ↺ λ

12.2 Package Lock Dictionary

The defpackage options are extended to include the following:

Example:

(defpackage "FOO" (:export "BAR") (:lock t) (:implement))
(defpackage "FOO-INT" (:use "FOO") (:implement "FOO" "FOO-INT"))

;;; is equivalent to

(defpackage "FOO") (:export "BAR"))
(lock-package "FOO")
(remove-implementation-package "FOO" "FOO")

(defpackage "FOO-INT" (:use "BAR"))
(add-implementation-package "FOO-INT" "FOO")

← ↑ → ↺ λ

13 Threading

SBCL supports a fairly low-level threading interface that maps onto the host operating system's concept of threads or lightweight processes. This means that threads may take advantage of hardware multiprocessing on machines that have more than one CPU, but it does not allow Lisp control of the scheduler. This is found in the sb-thread package.

Threads are part of the default build on x86[-64]/ARM64 Linux and Windows.

They are also supported on: x86[-64] Darwin (Mac OS X), x86[-64] FreeBSD, x86 SunOS (Solaris), PPC Linux, ARM64 Linux, RISC-V Linux. On these platforms threads must be explicitly enabled at build-time, see install for directions.

← ↑ → ↺ λ

13.1 Threading Basics

(make-thread (lambda () (write-line "Hello, world")))

← ↑ → ↺ λ

13.1.1 Thread Objects

← ↑ → ↺ λ

13.1.2 Running Threads

← ↑ → ↺ λ

13.1.3 Asynchronous Operations

← ↑ → ↺ λ

13.1.4 Miscellaneous Operations

← ↑ → ↺ λ

13.1.5 Error Conditions

← ↑ → ↺ λ

13.2 Special Variables

The interaction of special variables with multiple threads is mostly as one would expect, with behaviour very similar to other implementations.

The last point means that

(defparameter *x* 0)
(let ((*x* 1))
  (sb-thread:make-thread (lambda () (print *x*))))

prints 0 and not 1.

Note, however, that there is a hard limit on the number of distinct symbols that can be bound dynamically in threaded builds (see --tls-limit in Runtime Options). Exceeding this limit triggers the low-level error Thread local storage exhausted.

← ↑ → ↺ λ

13.3 Atomic Operations

Following atomic operations are particularly useful for implementing lockless algorithms.

Our sb-ext:compare-and-swap is user-extensible by defining functions named (cas <place>), allowing users to add CAS support to new places.

← ↑ → ↺ λ

13.4 Mutex Support

Mutexes are used for controlling access to a shared resource. One thread is allowed to hold the mutex, others which attempt to take it will be made to wait until it's free. Threads are woken in the order that they go to sleep.

(defpackage :demo (:use "CL" "SB-THREAD" "SB-EXT"))

(in-package :demo)

(defvar *a-mutex* (make-mutex :name "my lock"))

(defun thread-fn ()
  (format t "Thread ~A running ~%" *current-thread*)
  (with-mutex (*a-mutex*)
    (format t "Thread ~A got the lock~%" *current-thread*)
    (sleep (random 5)))
  (format t "Thread ~A dropped lock, dying now~%" *current-thread*))

(make-thread #'thread-fn)
(make-thread #'thread-fn)

← ↑ → ↺ λ

13.5 Semaphores

Semaphores are among other things useful for keeping track of a countable resource, e.g. messages in a queue, and sleep when the resource is exhausted.

← ↑ → ↺ λ

13.6 Waitqueue/condition variables

These are based on the POSIX condition variable design, hence the annoyingly CL-conflicting name. For use when you want to check a condition and sleep until it's true. For example: you have a shared queue, a writer process checking queue is empty and one or more readers that need to know when queue is not empty. It sounds simple but is astonishingly easy to deadlock if another process runs when you weren't expecting it to.

There are three components:

Important stuff to be aware of:

← ↑ → ↺ λ

13.7 Barriers

These are based on the Linux kernel barrier design, which is in turn based on the Alpha CPU memory model. They are presently implemented for x86, x86-64, PPC, ARM64, and RISC-V systems, and behave as compiler barriers on all other CPUs.

In addition to explicit use of the sb-thread:barrier macro, the following functions and macros also serve as :memory barriers:

← ↑ → ↺ λ

13.8 Sessions/Debugging

If the user has multiple views onto the same Lisp image (for example, using multiple terminals, or a windowing system, or network access) they are typically set up as multiple sessions such that each view has its own collection of foreground, background, and stopped threads. A thread which wishes to create a new session can use sb-thread:with-new-session to remove itself from the current session (which it shares with its parent and siblings) and create a fresh one.

Within a single session, threads arbitrate between themselves for the user's attention. A thread may be in one of three notional states: foreground, background, or stopped. When a background process attempts to print a repl prompt or to enter the debugger, it will stop and print a message saying that it has stopped. The user at his leisure may switch to that thread to find out what it needs. If a background thread enters the debugger, selecting any restart will put it back into the background before it resumes. Arbitration for the input stream is managed by calls to sb-thread:get-foreground (which may block) and sb-thread:release-foreground.

← ↑ → ↺ λ

13.9 Foreign threads

Direct calls to pthread_create(3) (instead of sb-thread:make-thread) create threads that SBCL is not aware of, these are called foreign threads. Currently, it is not possible to run Lisp code in such threads. This means that the Lisp side signal handlers cannot work. The best solution is to start foreign threads with signals blocked, but since third party libraries may create threads, it is not always feasible to do so. As a workaround, upon receiving a signal in a foreign thread, SBCL changes the thread's sigmask to block all signals that it wants to handle and resends the signal to the current process which should land in a thread that does not block it, that is, a Lisp thread.

The resignalling trick cannot work for synchronously triggered signals (sigsegv and co), take care not to trigger any. Resignalling for synchronously triggered signals in foreign threads is subject to --lose-on-corruption, see Runtime Options.

← ↑ → ↺ λ

13.10 Implementation on Linux x86oids

Threading is implemented using pthreads and some Linux specific bits like futexes.

On x86, the per-thread local bindings for special variables is achieved using the %fs segment register to point to a per-thread storage area. This may cause interesting results if you link to foreign code that expects threading or creates new threads, and the thread library in question uses %fs in an incompatible way. On x86-64 the r12 register has a similar role.

Queues require the futex(2) system call to be available: this is the reason for the NPTL requirement. We test at runtime that this system call exists.

Garbage collection is done with the existing Conservative Generational GC. Allocation is done in small (typically 8k) regions: each thread has its own region so this involves no stopping. However, when a region fills, a lock must be obtained while another is allocated, and when a collection is required, all processes are stopped. This is achieved by sending them signals, which may make for interesting behaviour if they are interrupted in system calls. The streams interface is believed to handle the required system call restarting correctly, but this may be a consideration when making other blocking calls e.g. from foreign library code.

Large amounts of the SBCL library have not been inspected for thread-safety. Some of the obviously unsafe areas have large locks around them, so compilation and fasl loading, for example, cannot be parallelized. Work is ongoing in this area.

A new thread by default is created in the same POSIX process group and session as the thread it was created by. This has an impact on keyboard interrupt handling: pressing your terminal's intr key (typically Control-C) will interrupt all processes in the foreground process group, including Lisp threads that SBCL considers to be notionally background. This is undesirable, so background threads are set to ignore the sigint signal.

sb-thread:make-listener-thread in addition to creating a new Lisp session makes a new POSIX session, so that pressing Control-C in one window will not interrupt another listener - this has been found to be embarrassing.

← ↑ → ↺ λ

14 Timers

SBCL supports a system-wide event scheduler implemented on top of setitimer(2) that also works with threads but does not require a separate scheduler thread.

The following example schedules a timer that writes Hello, world after two seconds.

(schedule-timer (make-timer (lambda ()
                              (write-line "Hello, world")
                              (force-output)))
                2)

It should be noted that writing timer functions requires special care, as the dynamic environment in which they run is unpredictable: dynamic variable bindings, locks held, etc, all depend on whatever code was running when the timer fired. The following example should serve as a cautionary tale:

(defvar *foo* nil)

(defun show-foo ()
  (format t "~&foo=~S~%" *foo*)
  (force-output t))

(defun demo ()
  (schedule-timer (make-timer #'show-foo) 0.5)
  (schedule-timer (make-timer #'show-foo) 1.5)
  (let ((*foo* t))
    (sleep 1.0))
  (let ((*foo* :surprise!))
    (sleep 2.0)))

← ↑ → ↺ λ

15 Networking

The sb-bsd-sockets module provides a thinly disguised BSD socket API for SBCL. Ideas have been stolen from the BSD socket API for C and Graham Barr's IO::Socket classes for Perl.

Sockets are represented as CLOS objects, and the API naming conventions attempt to balance between the BSD names and good lisp style.

← ↑ → ↺ λ

15.1 Sockets Overview

Most of the functions are modelled on the BSD socket API. BSD sockets are widely supported, portably (by Unix standards, at least) available on a variety of systems, and documented. There are some differences in approach where we have taken advantage of some of the more useful features of Common Lisp -- briefly:

← ↑ → ↺ λ

15.2 General Sockets

← ↑ → ↺ λ

15.3 Socket Options

A subset of socket options are supported, using a fairly general framework which should make it simple to add more as required -- see SYS:CONTRIB;SB-BSD-SOCKETS:SOCKOPT.LISP for details. The name mapping from C is fairly straightforward: SO_RCVLOWAT becomes sb-bsd-sockets:sockopt-receive-low-water and (setf sb-bsd-sockets:sockopt-receive-low-water).

← ↑ → ↺ λ

15.4 INET Domain Sockets

The TCP and UDP sockets that you know and love. Some representation issues:

← ↑ → ↺ λ

15.5 Local Domain Sockets

Local domain (AF_LOCAL) sockets are also known as Unix-domain sockets but were renamed by POSIX presumably on the basis that they may be available on other systems too.

A local socket address is a string, which is used to create a node in the local filesystem. This means of course that they cannot be used across a network.

A local abstract socket address is also a string the scope of which is the local machine. However, in contrast to a local socket address, there is no corresponding filesystem node.

← ↑ → ↺ λ

15.6 Name Service

Presently name service is implemented by calling out to the getaddrinfo(3) and gethostinfo(3), or to gethostbyname(3) and gethostbyaddr(3) on platforms where the preferred functions are not available. The exact details of the name resolving process (for example the choice of whether DNS or a hosts file is used for lookup) are platform dependent.

← ↑ → ↺ λ

16 Profiling

SBCL includes both a deterministic profiler, that can collect statistics on individual functions, and a more "modern", statistical profiler.

Inlined functions do not appear in the results reported by either.

← ↑ → ↺ λ

16.1 Deterministic Profiler

The package sb-profile provides a classic, per-function-call profiler.

Warning: When profiling code executed by multiple threads in parallel, the consing attributed to each function is inaccurate.

← ↑ → ↺ λ

16.2 Statistical Profiler

The sb-sprof module, loadable by

(require :sb-sprof)

provides an alternate profiler which works by taking samples of the program execution at regular intervals, instead of instrumenting functions as sb-profile:profile does. You might find sb-sprof more useful than the deterministic profiler when profiling functions in the common-lisp package, SBCL internals, or code where the instrumenting overhead is excessive.

Additionally sb-sprof includes a limited deterministic profiler which can be used for reporting the amounts of calls to some functions during

Example usage:

(in-package :cl-user)

(require :sb-sprof)

(declaim (optimize speed))

(defun cpu-test-inner (a i)
  (logxor a
          (* i 5)
          (+ a i)))

(defun cpu-test (n)
  (let ((a 0))
    (dotimes (i (expt 2 n) a)
      (setf a (cpu-test-inner a i)))))

;;;; CPU profiling

;;; Take up to 1000 samples of running (CPU-TEST 26), and give a flat
;;; table report at the end. Profiling will end one the body has been
;;; evaluated once, whether or not 1000 samples have been taken.
(sb-sprof:with-profiling (:max-samples 1000
                          :report :flat
                          :loop nil)
  (cpu-test 26))

;;; Record call counts for functions defined on symbols in the CL-USER
;;; package.
(sb-sprof:profile-call-counts "CL-USER")

;;; Take 1000 samples of running (CPU-TEST 24), and give a flat
;;; table report at the end. The body will be re-evaluated in a loop
;;; until 1000 samples have been taken. A sample count will be printed
;;; after each iteration.
(sb-sprof:with-profiling (:max-samples 1000
                          :report :flat
                          :loop t
                          :show-progress t)
  (cpu-test 24))

;;;; Allocation profiling

(defun foo (&rest args)
  (mapcar (lambda (x) (float x 1d0)) args))

(defun bar (n)
  (declare (fixnum n))
  (apply #'foo (loop repeat n collect n)))

(sb-sprof:with-profiling (:max-samples 10000
                          :mode :alloc
                          :report :flat)
  (bar 1000))

Output:

The flat report format will show a table of all functions that the profiler encountered on the call stack during sampling, ordered by the number of samples taken while executing that function.

           Self        Total        Cumul
  Nr  Count     %  Count     %  Count     %    Calls  Function
------------------------------------------------------------------------
   1     69  24.4     97  34.3     69  24.4 67108864  CPU-TEST-INNER
   2     64  22.6     64  22.6    133  47.0        -  SB-VM::GENERIC-+
   3     39  13.8    256  90.5    172  60.8        1  CPU-TEST
   4     31  11.0     31  11.0    203  71.7        -  SB-KERNEL:TWO-ARG-XOR

For each function, the table will show three absolute and relative sample counts. The Self column shows samples taken while directly executing that function. The Total column shows samples taken while executing that function or functions called from it (sampled to a platform-specific depth). The Cumul column shows the sum of all Self columns up to and including that line in the table.

Additionally the Calls column will record the amount of calls that were made to the function during the profiling run. This value will only be reported for functions that have been explicitly marked for call counting with sb-sprof:profile-call-counts.

The profiler also hooks into the disassembler such that instructions which have been sampled are annotated with their relative frequency of sampling. This information is not stored across different sampling runs.

;      6CF:       702E             JO L4              ; 6/242 samples
;      6D1:       D1E3             SHL EBX, 1
;      6D3:       702A             JO L4
;      6D5: L2:   F6C303           TEST BL, 3         ; 2/242 samples
;      6D8:       756D             JNE L8
;      6DA:       8BC3             MOV EAX, EBX       ; 5/242 samples
;      6DC: L3:   83F900           CMP ECX, 0         ; 4/242 samples

Platform support

Allocation profiling is only supported on SBCL builds that use the generational garbage collector. Tracking of call stacks at a depth of more than two levels is only supported on x86 and x86-64.

Macros

Functions

Variables

Credits

sb-sprof is an SBCL port, with enhancements, of Gerd Moellmann's statistical profiler for CMUCL.

← ↑ → ↺ λ

17 Contributed Modules

SBCL comes with a number of modules that are not part of the core system. These are loaded via (require :<modulename>) (see Customization Hooks for Users). This section contains documentation (or pointers to documentation) for some of the contributed modules.

← ↑ → ↺ λ

17.1 sb-aclrepl

The sb-aclrepl module offers an Allegro CL-style Read-Eval-Print Loop for SBCL, with integrated inspector. Adding a debugger interface is planned.

Allegro CL is a registered trademark of Franz Inc.

← ↑ → ↺ λ

17.1.1 Usage

To start sb-aclrepl as your read-eval-print loop, put the form

(require 'sb-aclrepl)

in your ~/.sbclrc, one of your Initialization Files.

← ↑ → ↺ λ

17.1.2 Customization

The following customization variables are available:

← ↑ → ↺ λ

17.1.3 Example Initialization

Here's a longer example of a ~/.sbclrc file that shows off some of the features of sb-aclrepl:

(ignore-errors (require 'sb-aclrepl))

(when (find-package 'sb-aclrepl)
  (push :aclrepl cl:*features*))
#+aclrepl
(progn
  (setq sb-aclrepl:*max-history* 100)
  (setf (sb-aclrepl:alias "asdc")
       #'(lambda (sys) (asdf:operate 'asdf:compile-op sys)))
  (sb-aclrepl:alias "l" (sys) (asdf:operate 'asdf:load-op sys))
  (sb-aclrepl:alias "t" (sys) (asdf:operate 'asdf:test-op sys))
  ;; The 1 below means that two characaters ("up") are required
  (sb-aclrepl:alias ("up" 1 "Use package") (package) (use-package package))
  ;; The 0 below means only the first letter ("r") is required,
  ;; such as ":r base64"
  (sb-aclrepl:alias ("require" 0 "Require module") (sys) (require sys))
  (setq cl:*features* (delete :aclrepl cl:*features*)))

Questions, comments, or bug reports should be sent to Kevin Rosenberg (kevin@rosenberg.net).

← ↑ → ↺ λ

17.2 sb-concurrency

Additional data structures, synchronization primitives and tools for concurrent programming. Similiar to Java's java.util.concurrent package.

← ↑ → ↺ λ

17.2.1 Queue

sb-concurrency:queue is a lock-free, thread-safe FIFO queue datatype.

The implementation is based on An Optimistic Approach to Lock-Free FIFO Queues by Edya Ladan-Mozes and Nir Shavit.

Before SBCL 1.0.38, this implementation resided in its own contrib (see sb-queue), which is still provided for backwards-compatibility, but which has since been deprecated.

← ↑ → ↺ λ

17.2.2 Mailbox (lock-free)

sb-concurrency:mailbox is a lock-free message queue where one or multiple ends can send messages to one or multiple receivers. The difference to Queue is that the receiving end may block until a message arrives.

Built on top of the Queue implementation.

← ↑ → ↺ λ

17.2.3 Gates

sb-concurrency:gate is a synchronization object suitable for when multiple threads must wait for a single event before proceeding.

← ↑ → ↺ λ

17.2.4 Frlocks, aka Fast Read Locks

← ↑ → ↺ λ

17.3 sb-cover

The sb-cover module provides a code coverage tool for SBCL. The tool has support for expression coverage, and for some branch coverage. Coverage reports are only generated for code compiled using compile-file with the value of the sb-cover:store-coverage-data optimization quality set to 3.

As of SBCL 1.0.6, sb-cover is still experimental, and the interfaces documented here might change in later versions.

How to use it:

;;; Load SB-COVER
(require :sb-cover)

;;; Turn on generation of code coverage instrumentation in the compiler
(declaim (optimize sb-cover:store-coverage-data))

;;; Load some code, ensuring that it's recompiled with the new optimization
;;; policy.
(asdf:oos 'asdf:load-op :cl-ppcre-test :force t)

;;; Run the test suite.
(cl-ppcre-test:test)

;;; Produce a coverage report
(sb-cover:report "/tmp/report/")

;;; Turn off instrumentation
(declaim (optimize (sb-cover:store-coverage-data 0)))

← ↑ → ↺ λ

17.4 sb-grovel

The sb-grovel module helps in generation of foreign function interfaces. It aids in extracting constants' values from the C compiler and in generating sb-alien structure and union types, Defining Foreign Types.

The ASDF (http://www.cliki.net/ASDF) component type GROVEL-CONSTANTS-FILE has its asdf:perform operation defined to write out a C source file, compile it, and run it. The output from this program is Lisp, which is then itself compiled and loaded.

sb-grovel is used in a few contributed modules, and it is currently compatible only to SBCL. However, if you want to use it, here are a few directions.

← ↑ → ↺ λ

17.4.1 Using sb-grovel in your own ASDF System

Make sure to specify the package you chose in step 1.

← ↑ → ↺ λ

17.4.2 Contents of a grovel-constants-file

The grovel-constants-file, typically named constants.lisp, comprises lisp expressions describing the foreign things that you want to grovel for. A constants.lisp file contains two sections:

There are two types of things that sb-grovel can sensibly extract from the C compiler: constant integers and structure layouts. It is also possible to define foreign functions in the constants.lisp file, but these definitions don't use any information from the C program; they expand directly to sb-alien:define-alien-routine forms.

Here's how to use the grovel clauses:

Note that c-string and sb-grovel::c-string-pointer do not have the same meaning. If you declare that an element is of type c-string, it will be treated as if the string is a part of the structure, whereas if you declare that the element is of type sb-grovel::c-string-pointer, a pointer to a string will be the structure member.

← ↑ → ↺ λ

17.4.3 Programming with sb-grovel's structure types

Let us assume that you have a grovelled structure definition:

(:structure mystruct ("struct my_structure"
                      (integer myint "int" "st_int")
                      (c-string mystring "char[]" "st_str")))

What can you do with it? Here's a short interface document:

← ↑ → ↺ λ

17.4.4 Traps and Pitfalls

Basically, you can treat functions and data structure definitions that sb-grovel spits out as if they were alien routines and types. This has a few implications that might not be immediately obvious (especially if you have programmed in a previous version of sb-grovel that didn't use alien types):

← ↑ → ↺ λ

17.5 sb-introspect

The sb-introspect module is about finding definitions, as well as querying their properties and relationships in the running image.

← ↑ → ↺ λ

17.5.1 Finding Definitions

← ↑ → ↺ λ

17.5.2 Special Variables

← ↑ → ↺ λ

17.5.3 Functions

← ↑ → ↺ λ

17.5.4 Types and Classes

← ↑ → ↺ λ

17.5.5 Allocation

← ↑ → ↺ λ

17.6 sb-manual

The sb-manual module has the SBCL user manual in forms mimicking pax:defsection:

(defsection @example (:title "Example")
  "This is an example, but see the real @SB-MANUAL."
  (print function)
  (@subexample section))

The names of the variables holding the documentation are exported from the sb-manual package. Since sections are basically variables, in Slime, M-. on "@SB-MANUAL", "print", or on "@subexample" will take you to the respective definition. This makes it easy to navigate the documentation. Normal Lisp definition docstrings and section docstrings reference sections following the usual convention of uppercasing the name. Docstrings are in a subset of Markdown and use very little markup in general, so they are easy to read directly in the source.

The official manual in Info, HTML and PDF formats is generated via Texinfo generated from these definitions.

← ↑ → ↺ λ

17.6.1 Using PAX

However, sb-manual::defsection is but a dummy implementation of pax:defsection to avoid a hard dependency on PAX.

See the mgl-pax asdf:system or https://github.com/melisgl/mgl-pax/.

When PAX is loaded, the dummy defsection definitions are made real, so that PAX can work with them.

← ↑ → ↺ λ

17.6.2 Browsing Live with PAX

With PAX, you can browse the manual live. The documentation of this feature is available at online.

If you are browsing this manual live right now, here is the equivalent live link: Browsing Live Documentation.

Notable features:

Live browsing can greatly reduce the latency of Edit-Compile-View Loop, when working on documentation.

← ↑ → ↺ λ

17.6.3 Fancy Documentation with PAX

PAX can generate dead documentation, too. In the SBCL sources, contrib/sb-manual/make-pax-docs.sh generates the manual in plain text, Markdown, PDF, and HTML formats. These differ from those generated via Texinfo in that they are autolinked (like when Browsing Live with PAX).

Also, you can generate documentation yourself with e.g.

(pax:document sb-manual:@sbcl-manual :format :markdown)

← ↑ → ↺ λ

17.7 sb-md5

The sb-md5 module implements the RFC1321 MD5 Message Digest Algorithm.

The implementation for CMUCL was largely done by Pierre Mai, with help from members of the cmucl-help mailing list. Since CMUCL and SBCL are similar in many respects, it was not too difficult to extend the low-level implementation optimizations for CMUCL to SBCL. Following this, SBCL's compiler was extended to implement efficient compilation of modular arithmetic (Modular Arithmetic), which enabled the implementation to be expressed in portable arithmetical terms, apart from the use of sb-rotate-byte for bitwise rotation.

← ↑ → ↺ λ

17.8 sb-posix

Sb-posix is the supported interface for calling out to the operating system.

Note: The functionality contained in the package sb-unix is for SBCL internal use only; its contents are likely to change from version to version.

The scope of this interface is "operating system calls on a typical Unixlike platform". This is section 2 of the Unix manual, plus section 3 calls that are (a) typically found in libc, but (b) not part of the C standard. For example, we intend to provide support for opendir(3) and readdir(3) but not for printf(3). That said, if your favourite system call is not included yet, you are encouraged to submit a patch to the SBCL mailing list.

Some facilities are omitted where they offer absolutely no additional use over some portable function, or would be actively dangerous to the consistency of Lisp. Not all functions are available on all platforms.

Sb-posix functions do not implicitly take measures to provide thread-safety or reentrancy beyond whatever the underlying C library does, except in cases where doing so is necessary to maintain the consistency of the Lisp image. For example, the bindings to the user and group database accessing functions are neither thread-safe nor reentrant unless the underlying libc happens to make them so (but see Extensions to POSIX).

← ↑ → ↺ λ

17.8.1 Lisp names for C names

All symbols are in the sb-posix package. This package contains a Lisp function for each supported Unix system call or function, a variable or constant for each supported Unix constant, an object type for each supported Unix structure type, and a slot name for each supported Unix structure member. A symbol name is derived from the C binding's name, by (a) uppercasing, then (b) removing leading underscores (#\_) then replacing remaining underscore characters with the hyphen (#\-). The requirement to uppercase is so that in a standard upcasing reader the user may write sb-posix:creat instead of sb-posix:|creat| as would otherise be required.

No other changes to "Lispify" symbol names are made, so creat becomes CREAT, not CREATE.

The user is encouraged not to (use-package :sb-posix) but instead to use the sb-posix: prefix on all references, as some of the symbols symbols contained in the sb-posix package have the same name as CL symbols (e.g. open, close(0 1), signal). Also, see Package-Local Nicknames.

← ↑ → ↺ λ

17.8.2 Types

Generally, marshalling between Lisp and C data types is done using SBCL's FFI. See Foreign Function Interface.

Some functions accept objects such as filenames or file descriptors. In the C binding to POSIX, these are represented as strings and small integers respectively. For the Lisp programmer's convenience we introduce designators such that CL pathnames or open streams can be passed to these functions. For example, sb-posix:rename accepts both pathnames and strings as its arguments.

← ↑ → ↺ λ

File-descriptors

← ↑ → ↺ λ

Filenames

← ↑ → ↺ λ

17.8.3 Function Parameters

The calling convention is modelled after that of CMUCL's unix package: in particular, it's like the C interface except that:

← ↑ → ↺ λ

17.8.4 Function Return Values

The return value is usually the same as for the C binding, except in error cases: where the C function is defined as returning some sentinel value and setting errno on error, we instead signal an error of type sb-posix:syscall-error. The actual error value (errno) is stored in this condition and can be accessed with sb-posix:syscall-errno.

We do not automatically translate the returned value into lispy objects -- for example, sb-posix:open returns a small integer, not a stream. Exception: boolean-returning functions (or, more commonly, macros) do not return a C integer but instead a Lisp boolean.

← ↑ → ↺ λ

17.8.5 Lisp Objects and C structures

Sb-posix provides various Lisp object types to stand in for C structures in the POSIX library. Lisp bindings to C functions that accept, manipulate, or return C structures accept, manipulate, or return instances of these Lisp types instead of instances of alien types.

The names of the Lisp types are chosen according to the general rules described above. For example Lisp objects of type sb-posix:stat stand in for C structures of type struct stat.

Accessors are provided for each standard field in the structure. These are named <structure-name>-<field-name> where the two components are chosen according to the general name conversion rules, with the exception that in cases where all fields in a given structure have a common prefix, that prefix is omitted. For example, stat.st_dev in C becomes stat-dev in Lisp.

Because sb-posix might not support all semi-standard or implementation-dependent members of all structure types on your system (patches welcome), here is an enumeration of all supported Lisp objects corresponding to supported POSIX structures, and the supported slots for those structures.

← ↑ → ↺ λ

17.8.6 Functions with Idiosyncratic Bindings

A few functions in sb-posix don't correspond directly to their C counterparts.

← ↑ → ↺ λ

17.8.7 Extensions to POSIX

Some of POSIX's standardized operators are not safe to use on their own, so sb-posix exports a few helpers that do not correspond exactly to functionality present in the POSIX standard.

The user and group database accessing routines are not required to be thread-safe or reentrant and so can only be used safely if all clients coordinate around their use. Since it would be logically impossible for independently developed programs to coordinate, sb-posix exports two iteration macros, sb-posix:do-passwds and sb-posix:do-groups, each of which iterates over the respective database while preventing the keyed accesses (sb-posix:getpwnam, sb-posix:getpwuid, sb-posix:getgrnam, sb-posix:getgrgid) from running until iteration completes.

← ↑ → ↺ λ

17.9 sb-queue

Since SBCL 1.0.38, the sb-queue module has been merged into the sb-concurrency module. See sb-concurrency.

← ↑ → ↺ λ

17.10 sb-rotate-byte

The sb-rotate-byte module offers an interface to bitwise rotation, with an efficient implementation for operations which can be performed directly using the platform's arithmetic routines. It implements the specification at http://www.cliki.net/ROTATE-BYTE.

Bitwise rotation is a component of various cryptographic or hashing algorithms: MD5, SHA-1, etc.; often these algorithms are specified on 32-bit rings.

← ↑ → ↺ λ

17.11 sb-simd

The sb-simd module provides a convenient interface for SIMD programming in SBCL. It provides one package per SIMD instruction set, plus functions and macros for querying whether an instruction set is available and what functions and data types it exports.

← ↑ → ↺ λ

17.11.1 Data Types

The central data type in sb-simd is the SIMD pack. A SIMD pack is very similar to a specialized vector, except that its length must be a particular power of two that depends on its element type and the underlying hardware. The set of element types that are supported for SIMD packs is similar to that of SBCL's specialized array element types, except that there is currently no support for SIMD packs of complex numbers or characters.

The supported scalar types are f32, f64, s<n>, and u<n>, where <n> is either 8, 16, 32, or 64. These scalar types are abbreviations for the Common Lisp types single-float, double-float, signed-byte, and unsigned-byte, respectively. For each scalar data type x, there exists one or more SIMD data type x.y with y elements. For example, in AVX there are two supported SIMD data types with element type f64, namely f64.2 (128 bit) and f64.4 (256 bit).

SIMD packs are regular Common Lisp objects that have a type, a class, and can be passed as function arguments. The price for this is that SIMD packs have both a boxed and an unboxed representation. The unboxed representation of a SIMD pack has zero overhead and fits into a CPU register but can only be used within a function and when the compiler can statically determine the SIMD pack's type. Otherwise, the SIMD pack is boxed, i.e. spilled to the heap together with its type information. In practice, boxing of SIMD packs can usually be avoided via inlining, or by loading and storing them to specialized arrays instead of passing them around as function arguments.

← ↑ → ↺ λ

17.11.2 Casts

For each scalar data type x, there is a function named x that is equivalent to (lambda (v) (coerce v 'x)). For each SIMD data type x.y, there is a function named x.y that ensures that its argument is of type x.y, or, if the argument is a number, calls the cast function of x and broadcasts the result.

All functions provided by sb-simd (apart from the casts themselves) implicitly cast each argument to its expected type. So, to add the number five to each single float in a SIMD pack x of type f32.8, it is sufficient to write (f32.8+ x 5). We don't mention this implicit conversion explicitly in the following sections, so if any function description states that an argument must be of type x.y, the argument can actually be of any type that is a suitable argument of the cast function named x.y.

← ↑ → ↺ λ

17.11.3 Constructors

For each SIMD data type x.y, there is a constructor named make-x.y that takes y arguments of type x and returns a SIMD pack whose elements are the supplied values.

← ↑ → ↺ λ

17.11.4 Unpackers

For each SIMD data type x.y, there is a function named x.y-values that returns, as y multiple values, the elements of the supplied SIMD pack of type x.y.

← ↑ → ↺ λ

17.11.5 Reinterpret Casts

For each SIMD data type x.y, there is a function named x.y! that takes any SIMD pack or scalar datum and interprets its bits as a SIMD pack of type x.y. If the supplied datum has more bits than the resulting value, the excess bits are discarded. If the supplied datum has less bits than the resulting value, the missing bits are assumed to be zero.

← ↑ → ↺ λ

17.11.6 Associatives

For each associative binary function, e.g. two-arg-x.y-op, there is a function x.y-op that takes any number of arguments and combines them with this binary function in a tree-like fashion. If the binary function has an identity element, it is possible to call the function with zero arguments, in which case the identity element is returned. If there is no identity element, the function must receive at least one argument.

Examples of associative functions are sb-simd-avx:f32.8+, for summing any number of 256 bit packs of single floats, and sb-simd-fma:u8.32-max, for computing the element-wise maximum of one or more 256 bit packs of 8 bit integers.

← ↑ → ↺ λ

17.11.7 Reducers

For binary functions two-arg-x.y-op that are not associative but have a neutral element, there are functions x.y-op that take any positive number of arguments and return the reduction of all arguments with the binary function. In the special case of a single supplied argument, the binary function is invoked on the neutral element and that argument. Reducers have been introduced to generate Lisp-style subtraction and division functions.

Examples of reducers are sb-simd-avx:f32.8/, for successively dividing a pack of 32 bit single floats by all further supplied packs of 32 bit single floats, or sb-simd-fma:u32.8- for subtracting any number of supplied packs of 32 bit unsigned integers from the first supplied one, except in the case of a single argument, where sb-simd-fma:u32.8- simply negates all values in the pack.

← ↑ → ↺ λ

17.11.8 Rounding

For each floating-point SIMD data type x.y, there are several functions that round the values of a supplied SIMD pack to nearby floating-point values whose fractional digits are all zero. Those functions are x.y-round, x.y-floor, x.y-ceiling, and x.y-truncate, and they have the same semantics as the one argument versions of cl:round, cl:floor, cl:ceiling, and cl:truncate, respectively.

← ↑ → ↺ λ

17.11.9 Comparisons

For each SIMD data type x.y, there exist conversion functions x.y<, x.y<=, x.y>, x.y>=, and x.y= that check whether the supplied arguments are strictly monotonically increasing, monotonically increasing, strictly monotonically decreasing, monotonically decreasing, equal, or nowhere equal, respectively. In contrast to the Common Lisp functions <, <=, >, >=, =, and /=, the SIMD comparison functions don't return a generalized boolean but a SIMD pack of unsigned integers with y elements. The bits of each unsigned integer are either all one, if the values of the arguments at that position satisfy the test, or all zero, if they don't. We call a SIMD packs of such unsigned integers a mask.

← ↑ → ↺ λ

17.11.10 Conditionals

The SIMD paradigm is inherently incompatible with fine-grained control flow. A piece of code containing an if special form cannot be vectorized in a straightforward way, because doing so would require as many instruction pointers and processor states as there are values in the desired SIMD data type. Instead, most SIMD instruction sets provide an operator for selecting values from one of two supplied SIMD packs based on a mask. The mask is a SIMD pack with as many elements as the other two arguments, but whose elements are unsigned integers whose bits must be either all zeros or all ones. This selection mechanism can be used to emulate the effect of an if special form, at the price that both operands have to be computed each time.

In sb-simd, all conditional operations and comparisons emit suitable mask fields, and there is a x.y-if function for each SIMD data type with element type x and number of elements y whose first arguments must be a suitable mask, whose second and third argument must be objects that can be converted to the SIMD data type x.y, and that returns a value of type x.y where each element is from the second operand if the corresponding mask bits are set, and from the third operand if the corresponding mask bits are not set.

← ↑ → ↺ λ

17.11.11 Loads and Stores

In practice, a SIMD pack x.y is usually not constructed by calling its constructor but by loading y consecutive elements from a specialized array with element type x. The functions for doing so are called x.y-aref and x.y-row-major-aref, and have similar semantics as Common Lisp's aref and row-major-aref. In addition to that, some instruction sets provide the functions x.y-non-temporal-aref and x.y-non-temporal-row-major-aref, for accessing a memory location without loading the referenced values into the CPU's cache.

For each function x.y-foo for loading SIMD packs from an array, there also exists a corresponding function (setf x.y-foo) for storing a SIMD pack in the specified memory location. An exception to this rule is that some instruction sets (e.g., SSE) only provide functions for non-temporal stores but not for the corresponding non-temporal loads.

One difficulty when treating the data of a Common Lisp array as a SIMD pack is that some hardware instructions require a particular alignment of the address being referenced. Luckily, most architectures provide instructions for unaligned loads and stores that are, at least on modern CPUs, not slower than their aligned equivalents. So by default we translate all array references as unaligned loads and stores. An exception are the instructions for non-temporal loads and stores, that always require a certain alignment. We do not handle this case specially, so without special handling by the user, non-temporal loads and stores will only work on certain array indices that depend on the actual placement of that array in memory.

← ↑ → ↺ λ

17.11.12 Specialized Scalar Operations

Finally, for each SIMD function x.y-op that applies a certain operation op element-wise to the y elements of type x, there exists also a functions x-op for applying that operation only to a single element. For example, the SIMD function f64.4+ has a corresponding function f64+ that differs from cl:+(0 1) in that it only accepts arguments of type double float, and that it adds its supplied arguments in a fixed order that is the same as the one used by f64.4.

There are good reasons for exporting scalar functions from a SIMD library, too. The most obvious one is that they obey the same naming convention and hence make it easier to locate the correct functions. Another benefit is that the semantics of each scalar operation is precisely the same as that of the corresponding SIMD function, so they can be used to write reference implementations for testing. A final reason is that these scalar functions can be used to simplify the life of tools for automatic vectorization.

← ↑ → ↺ λ

17.11.13 Instruction Set Dispatch

One challenge that is unique to image-based programming systems such as Lisp is that a program can run on one machine, be dumped as an image, and then resumed on another machine. While nobody expects this feature to work across machines with different architectures, it is quite likely that the machine where the image is dumped and the one where execution is resumed provide different instruction set extensions.

As a practical example, consider a game developer that develops software on an x86-64 machine with all SIMD extensions up to AVX2, but then dumps it as an image and ships it to a customer whose machine only supports SIMD extensions up to SSE2. Ideally, the image should contain multiple optimized versions of all crucial functions, and dynamically select the most appropriate version based on the instruction set extensions that are actually available.

This kind of run time instruction set dispatch is explicitly supported by means of the SB-SIMD-INTERNALS:INSTRUCTION-SET-CASE macro. The code resulting from an invocation of this macro compiles to an efficient jump table whose index is recomputed on each startup of the Lisp image.

← ↑ → ↺ λ

18 Deprecation

In order to support evolution of interfaces in SBCL as well as in user code, SBCL allows declaring functions, variables and types as deprecated. Users of deprecated things are notified by means of warnings while the deprecated thing in question is still available.

This chapter documents the interfaces for being notified when using deprecated thing and declaring things as deprecated, the deprecation process used for SBCL interfaces, and lists legacy interfaces in various stages of deprecation.

Deprecation in this context should not be confused with those things the ANSI Common Lisp standard calls deprecated: the entirety of ANSI CL is supported by SBCL, and none of those interfaces are subject to censure.

← ↑ → ↺ λ

18.1 Why Deprecate?

While generally speaking we try to keep SBCL changes as backwards compatible as feasible, there are situations when existing interfaces are deprecated:

← ↑ → ↺ λ

18.2 The Deprecation Pipeline

SBCL uses a deprecation pipeline with multiplestages: as time time goes by, deprecated things move from earlier stages of deprecation to later stages before finally being removed. The intention is making users aware of necessary changes early but allowing a migration to new interfaces at a reasonable pace.

Deprecation proceeds in three stages, each lasting approximately a year. In some cases it might move slower or faster, but one year per stage is what we aim at in general. During each stage warnings (and errors) of increasing severity are signaled, which note that the interface is deprecated, and point users towards any replacements when applicable.

← ↑ → ↺ λ

18.3 Deprecation Conditions

sb-ext:deprecation-condition is the superclass of all deprecation-related warning and error conditions. All common slots and readers are defined in this condition class.

← ↑ → ↺ λ

18.4 Introspecting Deprecation Information

The deprecation status of functions and variables can be inspected using the sb-cltl2:function-information and sb-cltl2:variable-information functions provided by the sb-cltl2 contributed module.

← ↑ → ↺ λ

18.5 Deprecation Declaration

The sb-ext:deprecated declaration can be used to declare objects in various namespaces as deprecated.

Note: See the namespace clhs glossary entry in the glossary of the Common Lisp Hyperspec.)

← ↑ → ↺ λ

18.6 Deprecation Examples

Marking functions as deprecated:

(defun foo ())
(defun bar ())
(declaim (deprecated :early ("my-system" "1.2.3")
                     (function foo :replacement bar)))

;; Remember: do not define the actual function or variable in case of
;; :final deprecation:
(declaim (deprecated :final ("my-system" "1.2.3")
                     (function fez :replacement whoop)))

Attempting to use the deprecated functions:

(defun baz ()
  (foo))
| STYLE-WARNING: The function CL-USER::FOO has been deprecated...
=> BAZ
(baz)
=> NIL ; no error

(defun danger ()
  (fez))
| WARNING: The function CL-USER::FEZ has been deprecated...
=> DANGER
(danger)
|- ERROR: The function CL-USER::FEZ has been deprecated...

← ↑ → ↺ λ

18.7 Deprecated Interfaces in SBCL

This sections lists legacy interfaces in various stages of deprecation.

← ↑ → ↺ λ

18.7.1 List of Deprecated Interfaces

← ↑ → ↺ λ

Early Deprecation

← ↑ → ↺ λ

Late Deprecation

← ↑ → ↺ λ

Final Deprecation

No interfaces are currently in final deprecation.

← ↑ ↺ λ

18.7.2 Historical Interfaces

The following is a partial list of interfaces present in historical versions of SBCL, which have since then been deleted.