Table of Contents
Coma provides a uniform configuration interface to items. Through Coma, parameters can be defined, queried, set and validated by the item itself or even by other items thanks to the simple object persistency employed.
An item has slots (i.e. attributes, parameters) that can be set concrete values. The item can then use these slot values to control arbitrary aspects of its behaviour such as building and running.
One can also define kits that contain other items and set their parameters.
A very simple item definition may look like this:
(defitem simple-item
(log-level
c-compiler)
(:config-files "Makefile"))This says that simple-item has two slots, log-level and c-compiler and the Makefile shall be created by Coma. One way to set a slot is through the command line. Let's set log-level to 3 and tell the item to use gcc by issuing coma.sh set-slot log-level 3 c-compiler \"gcc\".
To create Makefile Coma needs the template file Makefile.coma:
all:
@c-compiler@ -DLOG_LEVEL=@log-level@ my-prog.cNow, if the item is told to activate its configuration by coma.sh configure then the template file is processed the values of the appropriate slots are substitited for @c-compiler@ and @log-level@:
all:
gcc -DLOG_LEVEL=3 my-prog.c
Usually, products consist of several items that must be configured properly to work together. By defining kits it is possible to encapsulate complete configurations in one item.
Suppose our product called test consists of two items: simple-item and complex-item. The following defines a kit named test-kit.
(defkit test-kit)
By itself this definition does not say much, the bulk of the information is in the kit-items file:
(item1 #S(hoc-module :name "simple-item" :version-or-tag #v0.0.0) ((log-level 3) (c-compiler "gcc"))) (another-item #S(hoc-module :name "complex-item" :version-or-tag #v1.2.3) ((debug t) (lib item1)))
Here, two items are defined: item1 and another-item.
When the configuration of this kit is activated for the first time the kit looks at the kit-items file and fetches the items defined in it. In this example it checks out version 0.0.0 of module simple-item and version 1.2.3 of module complex-item from HOC. The checked out items contain their own instances of Coma and item definitions.
The kit then sets slots of these items: log-level to 3, c-compiler to the string "gcc". In the other item the debug slot is set to t while the lib slot is set to an object representing the configuration of item1.
Of course, there is much more to Coma, this is no more than a sketch, a quick, skeletal view of the concepts involved. The later chapters shall flesh out these concepts and provide detailed descriptions of kits, template processing, validation and customization of behaviour.
To make an item use Coma for managing its configuration first we must include it in the item. As with autoconf, a Coma instance is included in every single item that uses it.
The installation is simple. Suppose Coma is in /coma-dir and the item's root directory is /item-dir.
$ cd /coma-dir $ make $ mkdir /item-dir/cm/ $ ./coma.sh install /item-dir/cm/
This installs three files into /item-dir/cm/: coma.sh, coma.lisp and .coma-sources.bundle. These files should be included in the distribution, added to version control, etc.
Now Coma is installed but there isn't much we can do with it until the item definition is provided.
Note that not all Comas are created equal: the exact set of features (such as HOC and kit support) varies and can be controlled by the --with and --without install options. See coma.sh help install and coma.sh version.
Table of Contents
For Coma to be able to manage an item's configuration it must first know what kind of slots the item has, their names, their possible values, what is to be saved and which files it shall generate. Coma reads item definitions from a file named item.lisp in the same directory as itself. Let's take an example:
(defitem simple-item
(log-level
c-compiler))This can be read as a class definition: the item's name is simple-item and it has two slots: log-level and c-compiler.
Documentation can be attached to items and slots. These are presented to the user in some cases.
(defitem simple-item
((log-level :documentation "Controls the verbosity of logging. Values: 0-3")
c-compiler)
(:documentation "This is a simple example item."))In this example the default value of log-level is 3 as is other-log-level's. The difference is that initform can be combined with other specifiers such as documentation while no specifiers are allowed with the short form.
It is possible to give default values (lisp: initform) for slots:
(defitem simple-item
((log-level 3)
(other-log-level :default 3)
c-compiler))In this example the default value of log-level is 3 as is other-log-level's. Using :initform is also supported for backward compatibilty.
Basically, this is all there is to defining an item. However, for convenience, it is possible to parameterize some aspects of an item's behaviour directly form the item definition.
(defitem simple-item
((log-level 3)
c-compiler)
(:config-files "Makefile" "config.cfg")
(:rules ()
(integerp log-level))
(:transient c-compiler))See section activating configuration and processing templates for discussion of :config-files, validation for :rules, and saving items for :transient.
The defitem is not the only thing that can occur in the item definition file. This file is loaded as Common Lisp code and consequently it can contain macros, variables, function definitions, etc. See this example.
It is possible to access functions in a different item definition with the tell-item macro:
(tell-item my-item greet "World")
where my-item is defined like this:
(defun greet (target) (format t "Hello, ~A" target)) (defitem my-item)
Kits are specialized items. They perform everything a normal item does, but they also do more: see saving, activating a kit configuration.
Kits are defined by the defkit macro that is otherwise identical to defitem.
Often, kits have no user defined slots, configuration files. Such kits are defined like this:
(defkit my-kit)
Kits have a slot named items that contains the kit item definitions. This slot is almost never seen in or accessed explicitly from the item definition file, but it is the value of this slot that is in the file kit-items.lisp. This file contains multiple kit item definitions.
A kit item definition is a list of three elements:
the item symbol to which the object representing the item's configuration shall be bound
the item specification, an object with information on how construct/fetch the item
the item configuration, a list of (slot-name value-form) lists
The item symbol is an arbitrary symbol (lisp: symbol) used to refer to a kit item by name. Typically, the symbol is the same as the module name.
The item specification defines how an item is to be constructed. The only type of item specifier supported at present is hoc-module. One has to supply at least the name and the version of the module:
#S(hoc-module :name "my-module" :version-or-tag #v0.0.0)
It is possible to specify versions with tags:
#S(hoc-module :name "my-module" :version-or-tag "MAIN")
Note that the version of the item specified by a branch tag can change in time. Reproducability of kits with items specified with branch tags is not guaranteed.
An item configuration is a list of (slot-name value-form) lists. For each element in the list Coma sets the item's slot-name slot to the value obtained by evaluating value-form. See Activating a Kit Configuration.
When value-form is evaluated the symbol self is bound to the object representing the configuration of the kit in question. Item symbols are also available.
In practice, quite often slot-name and value-form are identical:
(numeric-lib #S(hoc-module :name "numeric-lib-module" :version-or-tag #v0.0.0) ()) (complex-item #S(hoc-module :name "complex-item" :version-or-tag #v0.0.0) ((numeric-lib numeric-lib)))
In this case, for convenience, (numeric-lib numeric-lib) can be abbreviated to numeric-lib:
(numeric-lib #S(hoc-module :name "numeric-lib-module" :version-or-tag #v0.0.0) ()) (complex-item #S(hoc-module :name "complex-item" :version-or-tag #v0.0.0) (numeric-lib))
By default configuration of an item is saved to the file config.lisp. It is simply a list of slot-name value pairs. Kits do the same, but save some of their slots to different files: items to kit-items.lisp and item-instance-map to map.lisp. The latter is not yet part of the public interface of Coma.
We might find ourselves in a similar situation when some of the slots are to be saved in a different manner or not saved at all. In this case we should declare these slots transient in the item definition:
(:transient "slot1" "another-slot")
Item 3rd-party-lib has a slot with a default value
(defitem 3rd-party-lib
((vendor-version #v1.0.0)))that is not be set and it should not be saved either because we expect to change its value by changing the default value and the saved value would override it.
(defitem 3rd-party-lib
((vendor-version #v1.0.0))
(:transient vendor-version))
When the modification is done from the command line by either set-slot or configureing a kit, the effected items are saved automatically.
Slots of an item hold values. These values together make up configuration of the item. Reading values from slots is a common need that arises in rules, templates and supporting code.
Items have predefined slots. Standard item slots are available in all items. HOC item slots are available in items which are HOC modules and kit slots are only available in kits.
Standard item slots
name (transient, read-only) is the name of the item as a string (usually all uppercase, "SIMPLE-ITEM" in the example)
path (transient, read-only) is an absolute path to the root directory of the item.
config-path (transient, read-write) is name of the file where the configuration is stored. If it is relative it is relative to the root of the item (path). The default value is "cm/config.lisp".
Kit slots
items (transient, read-only) contains a list of kit item definitions.
items-path (transient, read-only) is the name of the file where the items slot is stored. If it is relative it is relative to the root of the item (path). The default value is "cm/kit-items.lisp".
item-instance-map (transient, read-only) is a hash table of kit item symbol item instance mapping. contains a list of kit item definitions. Not intended for public consumption.
item-instance-map-path (transient, read-only) is the name of the file where the item-instance-map slot is stored. If it is relative it is relative to the root of the item (path). The default value is "cm/map.lisp".
With the -> macro slots of an item can be accessed. Suppose self is an item that has a slot named log-level. To get the value of this slot one writes:
(-> self log-level)
And to set the value:
(setf (-> self log-level) 3)
Which is the ordinary lisp idiom for setting a value.
It is worth noting that setting a slot this way does not automatically save the item.
Nested calls to -> can simply be chained:
(-> (-> self slot1) slot2)
is equivalent to:
(-> self slot1 slot2)
Frequently, it is the kits that set the configuration of items, but sometimes one finds the need either to configure an item without a kit or to set a slot of an item in a kit explicitly.
See the command line reference.
Table of Contents
The command coma.sh configure activates the configuration. The default behaviour for an item is to process its template files. In addition, a kit also fetches its items, configures them (by setting some of their slots) and finally activates their configurations.
With the template processing mechanism one can generate files with contents that depend on the configuration. A template file usually contains the skeleton of a makefile or property file plus special markup for parts that vary, called variant sections.
In the :config-files section of the item definition we list all files that should be produced by processing their corresponding template files. The name of the template file is the name of the output file with ".coma" appended to it.
An item definition can have zero or more :config-files sections. Each section contains zero or more filenames:
(:config-files "file1" "another.txt")
A variant section is a string of characters in a template file between two @ chars. When it is processed the Common Lisp code in the variant section is evaluated and its value inserted (printed by the princ function to be precise) at the point of the variant section. Also, everything printed to the stream coma-stream goes to the output file. When the @ character itself is to be written to the output, then it needs be escaped by doubling it.
Template files are processed with the current directory set to the root of the item.
In variant sections the following things are available:
every slot of the item is directly available by its name.
the variable SELF is bound to the object that represents the configuration of the item
everything defined in the item definition file including functions, variables, constants
everything defined by ANSI specification of Common Lisp
The template file is read one character at a time. If it is not a @, then it is written to the output file. If it is a @, then the file as read until another @. If there were no characters between the @'s (double @), then a single @ is written to the output stream. Otherwise the enclosed string is taken as a lisp form and is evaluated in the given environment. Any output to coma-stream during the evaluation of the form is written at once to the output file and the value of the form is finally printed (by the princ function) to it.
This algorithm continues until the end of the template file or until an error is encountered.
Note that the output file is not written if an error occurs or it already exists and has identical content to the one just generated. When this is not desirable, the output file must be removed or the template should generate different output each time it is run, for example by inserting the time at the time of processing into the output:
# Ensure this file gets written every time # the template is processed by inserting the time in # seconds here: @(get-universal-time)@
There is nothing like a few examples to clear things up.
Consider this example:
The loglevel is @(+ 1 2)@ and debug is@" not"@ enabled. my-email@@address.com
When this template file is processed the lisp expression (+ 1 2) is evaluated and the result (the number 3) is printed to the result file. The same happens with the string " not", it is printed without the quotes. Also, notice the escaped @ in the email address. The output looks like this:
The loglevel is 3 and debug is not enabled. my-email@address.com
The template in the previous example is not overly useful, since it only inserts a few constants into the file. Most template files use the values in the configuration of the item. Consider this item:
(defitem my-item
(log-level
debug
printer)
(:config-files "static.cfg"))MY-ITEM has three slots and a config file whose template looks like this:
log_level=@log-level@ debug_enabled=@debug@ use_printer=@printer@
This template file uses the values of slots in the item definition to generate an output like this (depending on the actual values, naturally):
log_level=3 debug_enabled=NIL use_printer=T
The log_level is OK, but it is unlikely that our program that reads this file will be happy with this output. It may want "true" and "false" instead of T and NIL. Fortunately, this is easy to do:
log_level=@log-level@ debug_enabled=@(if debug "true" "false")@ use_printer=@(if printer "true" "false")@
Now, we have a simple file, but that little conversion code occurs twice. This, too, can be solved easily. Let's define a function next to the definition:
(defun ->bool (v)
"Convert the boolean value V to a string."
(if v
"true"
"false"))and modify the the template file:
log_level=@log-level@ debug_enabled=@(->bool debug)@ use_printer=@(->bool printer)@
Kits process the contents item definitions found in the kit-items.lisp file. The abstract algorithm is the following.
For each item definition Coma fetches the item according to the item specification. If it has already been fetched it may be updated. Then objects representing the item configurations are constructed and bound to the appropriate item symbols. Finally, item configurations are set.
Let's examine the treatment of HOC modules in detail.
If the kit item is a HOC module, then the kit checks it out. If it is already checked out it is updated to the version in the item specification. So, activating the configuration of a kit synchronizes the items with the kit according to the contents of the kit-items.lisp file.
Coma tries its best to keep the versions in item specifications and actual module versions in sync. To this end it registers a HOC extension at each HOC module in the kit that allows it to be notified of version changes and update the item specification accordingly.
If the item specification names a branch instead of a version, then Coma only changes the it if the module switches to another branch. Even then, it only uses the new branch name in the specification.
Table of Contents
Slots can hold arbitrary values, but typically not all values are valid for the item. One slot may only take a boolean value, another a string. It may be that the a slot should be a string or a boolean depending on the value of another slot.
It is useful to define these rules explicitly, so that configurations can be validated automatically. To define rules one has to add a :RULES section to the item definition:
(defitem simple-item
(log-level
c-compiler)
(:rules ()
(integerp log-level)
(stringp c-compiler))
(:config-files "Makefile"))This defines two rules: one that says log-level should be an integer, and one that says c-compiler should be a string.
When the command coma.sh validate is issued, these two rules evaluated. If all rules return true (lisp: T), then the configuration is deemed valid.
A rule is said to have failed if it returns false (lisp: NIL). It is said to have failed with an error if evaluation of the rule has not finished normally, possibly due to an error condition.
Failed rules are listed by Coma and the error if any is printed.
Sometimes it is desirable to test whether the configuration is suitable to perform a certain operation, such as compilation, running or testing. To this end rules can be associated with rule sets.
(defitem simple-item
(log-level
c-compiler)
(:rules (:run)
(integerp log-level))
(:rules (:build)
(stringp c-compiler))
(:config-files "Makefile"))In this example the rule (integerp log-level) is associated with the rule set :run, while (stringp c-compiler) is associated with the rule set :build.
To test whether the configuration is OK for building the item, we issue the coma.sh validate :build command. This runs only the rules associated with the rule set :build.
We might have rules that should be included in many/all rule sets.
(defitem simple-item
(debug
log-level
c-compiler)
(:rules (:run :build)
(member debug '(t nil)))
(:rules (:run)
(integerp log-level))
(:rules (:build)
(stringp c-compiler))
(:config-files "Makefile"))The rule (member debug '(t nil)) says that DEBUG is either T or NIL. Furthermore, this rule is in both the :run and :build rule sets and is evaluated accordingly.
Rule set names can be arbitrary objects, the only restriction is that they can be tested for equality (lisp: equal).
As a matter of style, though, it is strongly encouraged to use keyword symbols (lisp: keywords) to name rule sets.
Rules can be arbitrary lisp forms. They can use everything in Common Lisp in addition to what's defined in the item definition file.
(defun boolean? (v)
(member v '(t nil)))
(defitem simple-item
(debug)
(:rules ()
(boolean? debug)))
That's how the boolean? predicate could be defined, if it wasn't defined for us already. As a matter of fact it is one of the few predefined helper functions/macros.
The function boolean? tests if its one argument is either T or NIL:
(boolean? t) => t (boolean? nil) => t (boolean? 1) => nil
The function compatible-item? tests if an object is an item of with a given name and performs version compatibility checking.
(defitem complex-item
(simple-item
other-item)
(:rules ()
;; the value of simple-item should be a version of
;; item "simple-item" compatible with #v1.2.3
(compatible-item? simple-item "simple-item" #v1.2.3)
;; same rule in a different form
(compatible? simple-item #v1.2.3)
;; other-item should be compatible with #v2.3.4 on the
;; patch level
(compatible-item? other-item "item-name" #v2.3.4 :patch)))The fourth, optional parameter is one of :major, :minor (default) and :patch, it controls the compatibility checking.
The convenience macro compatible? is very similar to compatible-item?. It assumes that the name of the item is the same as the name of the symbol given as its firts argument.
Table of Contents
First, we create an item that is a HOC module called empty-lib. Suppose this module already exists, but does not use Coma, yet. If it doesn't exists it can be created by the hoc create command.
So, the module is checked out in directory empty-lib and contains three files: Makefile:
all:
@gcc -DNO_DEBUG src/empty-lib.csrc/config.h:
#define LOG_LEVEL 0
and src/empty-lib.c that includes config.h.
Let's create a directory cm under it, and install Coma into this directory:
$ mkdir cm/ $ path-to-coma/coma.sh install empty-lib/cm/
If all goes well, we got a message that a number of files were generated for us. Add these files to the module
hoc add cm/coma.sh cm/coma.lisp cm/.coma-sources.bundle
and see if it works:
cm/coma.sh help
It should only list a few generic commands, because we haven't provided an item definition, yet. So, create a file name cm/item.lisp with the following contents:
(defitem empty-lib
((debug nil)
(log-level 0)
(c-compiler "cc"))
(:rules (:build)
(boolean? debug)
(integerp log-level)
(<= 0 log-level 3)
(stringp c-compiler))
(:config-files "Makefile" "src/config.h"))We have defined empty-lib to have three slots: debug, log-level and c-compiler all with suitable default values and rules to check for their correctness. Futhermore, we have declared that Makefile and src/config.h should be generated by Coma. We have to create template files for them. The easiest way to start is renaming them:
$ hoc mv Makefile Makefile.coma $ hoc mv src/config.h src/config.h/coma
Now, edit Makefile.coma to take slot values from the configuration:
all:
@@@c-compiler@ @(if debug "" "-DNO_DEBUG")@ src/empty-lib.cNote that the starting @ was escaped by doubling it. Also edit src/config.h.coma:
#define LOG_LEVEL @log-level@
We are ready to roll. Set slot values by issuing coma.sh set-slot log-level 3 c-compiler \"gcc\" and activate the configuration with coma.sh. This processes the template files and generates Makefile and src/config.h.
To validate the configuration type coma.sh validate, then set log-level to 4 and validate it again.
Creating a kit is very similar, the only notable difference is in the item definition (item.lisp): kits are defined by defkit instead of defitem.
(defkit test-kit)
In most cases it is enough, but kits can also have user defined slots, rules, config files. In fact defkit has much the same parameters as defitem, all of which are omitted here.
There is one more thing. We need to create kit-items.lisp that lists the items this kit is made of.
(empty-lib #S(hoc-module :name "empty-lib" :version-or-tag #v2.0.0) ((debug t) (log-level 3) (c-compiler "gcc"))) (other-item #S(hoc-module :name "other-item" :version-or-tag #v5.0.0) ((lib empty-lib)))
This means that version #v2.0.0 of hoc module empty-lib is part of this kit and can be referenced as empty-lib. We also say that debug and emacs slots are to be set to t and c-compiler to "gcc".
Another hoc module "other-item" is specified whose lib slot is to be set to empty-lib.
There can be kits that consist of the same items but their item configurations are not identical. These kits can be unified by adding new kit level parameters.
Take two kits that both use empty-lib. One kit defines the product shipped to testers with debug enabled:
(empty-lib #S(hoc-module :name "empty-lib" :version-or-tag #v2.0.0) ((debug t) (log-level 3) (c-compiler "gcc")))
and the other is the final product shipped with debug turned off:
(empty-lib #S(hoc-module :name "empty-lib" :version-or-tag #v2.0.0) ((debug nil) (log-level 0) (c-compiler "gcc")))
These two kits always have the same items (with the same versions), but configure them a bit differently.
Let's introduce a new slot name final? to the kit:
(defkit test-kit
((final? nil)))and modify the kit item definitions to use its value:
(empty-lib #S(hoc-module :name "empty-lib" :version-or-tag #v2.0.0) ((debug (not (-> self final?))) (log-level (if (-> self final?) 0 3)) (c-compiler "gcc")))
When building the final release the slot final? is set to t (coma.sh set-slot 'final?' t) and the kit is reconfigured (coma.sh).
No, set-slot doesn't work because it is transient. It must be transient since reading its value from the file it denotes would be circular. So, how can it be customized? By providing a default value for it in the item definition.
(defitem item-with-customized-config-path
((config-path "cm/.config.db")
(debug nil)
(log-level 0)
(c-compiler "cc")))
In this appendix the most notable files and their purpose are described.
.coma-sources.bundle - the bulk of the code packaged into one file
config.lisp - the saved configuration. Default value of the config-path slot.
kit-items.lisp - kit item definitions. Default value of the items-path slot.
map.lisp - kit item symbol to persistent object mapping, not public yet. Default value of the item-instance-map-path slot.
Coma is written in Common Lisp that also serves as its extension language. Some very basic Lisp knowledge is necessary for doing non-trivial template processing or writing extensions.
A bundle of stuff (code, doc, ...) that is the basic unit of configuration management (version handling, dependency tracking, ...). It is called configuration item in CM terminology.
A kind of item that defines a set of items that make it up. It is called configuration in CM terminology.
A parameter of an item. In OO terms, it is an attribute of the object that represents the configuration of an item.