funpack.exporting_hdf5

This module contains functions for exporting data to HDF5 files.

funpack.exporting_hdf5.HDF5_KEY = 'funpack'

Default group key to use when exporting to a HDF5 file.

funpack.exporting_hdf5.HDF5_STYLE = 'pandas'

Default style to use when exporting to a HDF5 file.

funpack.exporting_hdf5.HDF5_STYLES = ['pandas', 'funpack']

Available styles to use when saving to HDF5, specified via the style argument passed to the exportHDF5() function.

funpack.exporting_hdf5.exportFunpackStyle(dtable, outfile, key, **kwargs)[source]

Save the data to a funpack-style HDF5 file.

Each column is saved as a separate data set, and named according to the column name. All columns are saved under a single group named according to the key argument.

Parameters:

  • dtable – DataTable containing the data

  • outfile – File to output to

  • key – Name to give the HDF5 group.

funpack.exporting_hdf5.exportHDF5(dtable, outfile, key=None, style=None, dropNaRows=False, dateFormat=None, timeFormat=None, formatters=None, **kwargs)[source]

Export data to a HDF5 file.

The style argument determins the internal format of the resulting HDF5 file. Available styles are 'pandas' and 'funpack' - see the exportPandasStyle() and exportFunpackStyle() functions for details.

Note

Both functions assume that every column in dtable has either numeric or string type. Storage of other types is not supported, and columns with an unsupported type will not be exported.

Parameters:

  • dtable – DataTable containing the data

  • outfile – File to output to

  • key – Name to give the HDF5 group. Defaults to HDF5_KEY.

  • style – HDF5 style to use (see above). Defaults to HDF5_STYLE.

  • dropNaRows – If True, rows which do not contain data for any columns are not exported.

  • dateFormat – Name of formatter to use for date columns.

  • timeFormat – Name of formatter to use for time columns.

  • formatters – Dict of { [vid|column] : formatter } mappings, specifying custom formatters to use for specific variables.

funpack.exporting_hdf5.exportPandasStyle(dtable, outfile, key, numRows=None, **kwargs)[source]

Save the data to a pandas-style HDF5 file.

The entire data frame is saved using the pandas.to_hdf function, under a single group named according to the key argument.

The data can be reloaded via the pandas.read_hdf function.

Parameters:

  • dtable – DataTable containing the data

  • outfile – File to output to

  • key – Name to give the HDF5 group.

  • numRows – Number of rows to write at time (only used for pandas-style files).

funpack.exporting_hdf5.importFunpackStyle(infile, idcol, key=None)[source]

Load a funpack-style HDF5 file as a pandas.DataFrame. See the exportFunpackStyle() function.

Parameters:

  • infile – File to load

  • idocol – Name of index column.

  • key – HDF5 group key containing the data. If not provided, and the file only has one group, that group is assumed to contain the data. If not provided, and the file contains more than one group, a ValueError is raised.

Returns:

A pandas.DataFrame containing the data.