.. _fmrib_configuration_profile:

FMRIB configuration profile
===========================


FUNPACK comes with a built-in configuration profile (the `"FMRIB"
<https://www.win.ox.ac.uk/about/locations/fmrib/>`_ profile), containing a
range of processing rules for a large number of UK BioBank data fields. These
rules can be applied by running::

    fmrib_unpack -cfg fmrib <output.tsv> <input.csv>


The ``fmrib`` configuration profile is installed alongside the FUNPACK source
code - it can be viewed online `here
<https://git.fmrib.ox.ac.uk/fsl/funpack-fmrib-config/-/tree/master/funpack/configs>`_,
or found in your local FUNPACK installation within
``<python-env>/lib/python<X.Y>/site-packages/funpack/configs/`` (replacing
``<python-env>`` with the location of your Python environment, and ``<X.Y>``
with the Python version).

.. note:: The ``fmrib`` configuration profile is managed independently from
          the FUNPACK source code at
          https://git.fmrib.ox.ac.uk/fsl/funpack-fmrib-config/, but is always
          installed alongside FUNPACK.


The ``fmrib`` configuration profile is split across several files, each of
which are described below. Click on the arrow to the left of each section to
view the contents of that file.

..
  |cfgdir| is defined in conf.py

.. details::
   .. summary:: ``funpack/configs/fmrib.cfg``: Top-level configuration file,
                containing general settings, and references to the other
                configuration files.
   .. includesub:: |cfgdir|/fmrib.cfg
      :literal:

.. details::
   .. summary:: ``funpack/configs/local.cfg``: Included by ``fmrib.cfg``.
                Contains some miscellaneous settings related to performance
                and error-checking.
   .. includesub:: |cfgdir|/local.cfg
      :literal:

.. details::
   .. summary:: ``funpack/configs/fmrib/categories.tsv``: Definition of FMRIB
                datafield categories - groups of related data-fields.
                Categories can be selected with the ``-c/--category`` option.
   .. includesub:: |cfgdir|/fmrib/categories.tsv
      :literal:

.. details::
   .. summary:: ``funpack/configs/fmrib/datacodings_navalues.tsv``: NA value
                replacement rules. The ``ID`` column specifies the UKB
                data-coding, and the ``NAValues`` column is a comma-separated
                list of values which will be removed. Each rule is applied to
                every data-field that uses the corresponding data-coding. Only
                the first 20 rules are shown here.
   .. includesub:: |cfgdir|/fmrib/datacodings_navalues.tsv
      :literal:
      :end-line: 21

.. details::
   .. summary:: ``funpack/configs/fmrib/datacodings_recoding.tsv``:
                Categorical recoding rules. The ``ID`` column specifies the
                UKB data-coding, the ``RawLevels`` column is a comma-separated
                list of values to be replaced, and the ``NewLevels`` column is
                a comma-separated list of values to replace the ``RawLevels``
                with. Each rule is applied to every data-field that uses the
                corresponding data-coding. Only the first 20 rules are shown
                here.
   .. includesub:: |cfgdir|/fmrib/datacodings_recoding.tsv
      :literal:
      :end-line: 21

.. details::
   .. summary:: ``funpack/configs/fmrib/variables_clean.tsv``: Custom cleaning
                functions. The ``ID`` column specifies the UKB data-field, and
                the ``Clean`` column specifies the cleaning function to apply
                (see :ref:`here <cleaning_functions>` for an overview of all
                built-in cleaning functions). Only the first 20 rules are
                shown here.
   .. includesub:: |cfgdir|/fmrib/variables_clean.tsv
      :literal:
      :end-line: 21

.. details::
   .. summary:: ``funpack/configs/fmrib/datetime_formatting.tsv``: Output
                format for all date/time data-fields. These functions are
                defined in the built-in :mod:`funpack.plugins.fmrib` plugin
                module.
   .. includesub:: |cfgdir|/fmrib/datetime_formatting.tsv
      :literal:

.. details::
   .. summary:: ``funpack/configs/fmrib/variables_parentvalues.tsv``: Child
                value replacement rules applied to data fields. The ``ID``
                column specifies the UKB data-fiuld, the ``ParentValues``
                column is an exprssion which is evaluated on the parent
                data-fields, and the ``ChildValues`` column is a value to set
                the data-field to when the expression evaluates to true.  Only
                the first 20 rules are shown here.
   .. includesub:: |cfgdir|/fmrib/variables_parentvalues.tsv
      :literal:
      :end-line: 21

.. details::
   .. summary:: ``funpack/configs/fmrib/processing.tsv``: Processing rules
                applied to the data set after all cleaning stages have been
                performed. See :ref:`here <processing_functions>` for an
                overview of all of the built-in processing functions.
   .. includesub:: |cfgdir|/fmrib/processing.tsv
      :literal:

|


Some additional configuration profiles enhance the ``fmrib`` profile with
some extra options.

The ``fmrib_standard`` profile incorporates all options from the ``fmrib``
profile, however it only load datafields from the categories listed in
``fmrib_cats.cfg``, and it also configures ``funpack`` to output verbose
logging information and additional summary files.

.. details::
  .. summary:: ``fmrib_standard.cfg``
  .. includesub:: |cfgdir|/fmrib_standard.cfg
     :literal:
.. details::
  .. summary:: ``fmrib_logs.cfg``
  .. includesub:: |cfgdir|/fmrib_logs.cfg
     :literal:
.. details::
  .. summary:: ``fmrib_cats.cfg``
  .. includesub:: |cfgdir|/fmrib_cats.cfg
     :literal:

|


The ``fmrib_new_release`` profile is the same as the ``fmrib_standard``
profile, but also load and process any unknown or uncategorised
datafields. This is useful when processing a new data set which may contain
newly added data fields that you have not yet encountered.

.. details::
   .. summary:: ``fmrib_new_release.cfg``
   .. includesub:: |cfgdir|/fmrib_new_release.cfg
      :literal: