Design Philosophy and Features for v3+¶
Here we describe in broad terms the design philosophy of the new
and some of its new features.
This is useful to get an initial bearing of how to go about using
most likely the tutorials will be better for that.
It is also useful for those who have used the “old”
21cmFAST (versions 2.1 and less)
and want to know why they should use this new version (and how to convert).
In doing so, we’ll go over some of the key features of
To get a more in-depth view of all the options and features available, look at the
very thorough API Reference.
The goal of v3 of
21cmFAST is to provide the same computational efficiency and
scientific features of the previous generations, but packaged in a form that adopts the
best modern programming standards, including:
comprehensive test suite
more modular code
standardised code formatting
truly open-source and collaborative design (via Github)
Partly to enable these standards, and partly due to the many extra benefits it brings, v3 also has the major distinction of being wrapped entirely in Python. The extra benefits brought by this include:
a native python library interface (eg. get your output box directly as a
better file-writing, into the HDF5 format, which saves metadata along with the box data.
a caching system so that the same data never has to be calculated twice.
reproducibility: know which exact version of
21cmFAST, with what parameters, produced a given dataset.
significantly improved warnings and error checking/propagation.
simplicity for adding new additional effects, and inserting them in the calculation pipeline.
We hope that additional features and benefits will be found by the growing community
21cmFAST developers and users.
How it Works¶
v3 is not a complete rewrite of
21cmFAST. Most of the C-code of previous versions
is kept, though it has been modularised and modified in many places. The fundamental
routines are the same (barring bugfixes!).
The major programs of the original version (
ionize etc.) have
been converted into modular functions in C. Furthermore, most of the global parameters
(and, more often than not, global
#define options) have been modularised and converted
into a series of input “parameter”
structs. These get passed into the functions.
Furthermore, each C function, instead of writing a bunch of files, returns an output
struct containing all the stuff it computed.
Each of these functions and structs are wrapped in Python using the
CFFI compiles the C code once upon installation. Due to the fact that parameters are
now passed around to the different functions, rather than being global defines, we no
longer need to re-compile every time an option is changed. Python itself can handle
changing the parameters, and can use the outputs in whatever way the user desires.
To maintain continuity with previous versions, a CLI interface is provided (see below) that acts in a similar fashion to previous versions.
21cmFAST is installed, it automatically creates a configuration directory in
the user’s home:
~/.21cmfast. This houses a number of important configuration
options; usually default values of parameters. At this stage, the location of this
directory is not itself configurable. The config directory contains example
configuration files for the CLI interface (see below), which can also be copied anywhere
on disk and modified. Importantly, the
config.yml file in this directory specifies
some of the more important options for how
21cmFAST behaves by default.
One such option is
boxdir, which specifies the directory in which
cache results (see below for details). Finally, the config directory houses several data
tables which are used to accelerate several calculations. In principle
these files are over-writeable, but they should only be touched if one knows very well
what they are doing.
21cmFAST contains a more robust cataloguing/caching method. Instead of
saving data with a selection of the dependent parameters written into the filename –
a method which is prone to error if a parameter which is not part of that selection is
21cmFAST writes all data into a configurable central directory with a hash
filename unique to all parameters upon which the data depends. Each kind of dataset has
attached methods which efficiently search this central directory for matching data to be
read when necessary.
Several arguments are available for all library functions which produce such datasets
that control this output. In this way, the data that is being retrieved is always
reliably produced with the desired parameters, and users need not concern themselves
with how and where the data is saved – it can be retrieved merely by creating an empty
object with the desired parameters and calling
.read(), or even better, by calling
the function to produce the given dataset, which will by default just read it in if
The CLI interface always starts with the command
21cmfast, and has a number of
subcommands. To list the available subcommands, use:
$ 21cmfast --help
To get help on any subcommand, simply use:
$ 21cmfast <subcommand> --help
Any subcommand which runs some aspect of
21cmFAST will have a
which specifies a configuration file (by default this is
~/.21cmfast/runconfig_example.yml). This config file specifies the parameters of the
run. Furthermore, any particular parameter that can be specified in the config file can
be alternatively specified on the command line by appending the command with the
parameter name, eg.:
$ 21cmfast init --config=my_config.yml --HII_DIM=40 hlittle=0.7 --DIM 100 SIGMA_8 0.9
The above command shows the numerous ways in which these parameters can be specified (with or without leading dashes, and with or without “=”).
The CLI interface, while simple to use, does have the limitation that more complex
arguments than can be passed to the library functions are disallowed. For example,
one cannot pass a previously calculated initial conditions box to the
command. However, if such a box has been calculated with the default option to write it
to disk, then it will automatically be found and used in such a situation, i.e. the
following will not re-calculate the init box:
$ 21cmfast init $ 21cmfast perturb redshift=8.0
This means that almost all of the functionality provided in the library is accessible via the CLI.