ftag.hdf5.h5reader#

Classes#

H5SingleReader

H5Reader

Reads data from multiple HDF5 files.

Module Contents#

class ftag.hdf5.h5reader.H5SingleReader#
fname: pathlib.Path | str#
batch_size: int = 100000#
jets_name: str = 'jets'#
precision: str | None = None#
shuffle: bool = True#
do_remove_inf: bool = False#
transform: ftag.transform.Transform | None = None#
__post_init__() → None#
property num_jets: int#
get_attr(name, group=None)#
empty(ds: h5py.Dataset, variables: list[str]) → numpy.ndarray#
read_chunk(ds: h5py.Dataset, array: numpy.ndarray, low: int) → numpy.ndarray#
remove_inf(data: dict) → dict#
stream(variables: dict | None = None, num_jets: int | None = None, cuts: ftag.cuts.Cuts | None = None, start: int = 0, skip_batches: int = 0) → collections.abc.Generator#
class ftag.hdf5.h5reader.H5Reader#

Reads data from multiple HDF5 files.

Parameters:

  • fname (Path | str | list[Path | str]) – Path to the HDF5 file or list of paths

  • batch_size (int, optional) – Number of jets to read at a time, by default 100_000

  • jets_name (str, optional) – Name of the jets dataset, by default “jets”

  • precision (str | None, optional) – Cast floats to given precision, by default None

  • shuffle (bool, optional) – Read batches in a shuffled order, by default True

  • weights (list[float] | None, optional) – Weights for different input datasets, by default None

  • do_remove_inf (bool, optional) – Remove jets with inf values, by default False

  • transform (Transform | None, optional) – Transform to apply to data, by default None

  • equal_jets (bool, optional) – Take the same number of jets (weighted) from each sample, by default True. This is useful when you specify a list of DSIDs for the sample and they are qualitatively different, and you want to ensure that you always return batches with jets from all DSIDs. This is used for example in the QCD resampling for Xbb. If False, use all jets in each sample, allowing for the full available statistics to be used. Useful for example if you have multiple ttbar samples and you want to use all available jets from each sample.

fname: pathlib.Path | str | list[pathlib.Path | str]#
batch_size: int = 100000#
jets_name: str = 'jets'#
precision: str | None = None#
shuffle: bool = True#
weights: list[float] | None = None#
do_remove_inf: bool = False#
transform: ftag.transform.Transform | None = None#
equal_jets: bool = False#
__post_init__() → None#
property num_jets: int#
property files: list[pathlib.Path]#
dtypes(variables: dict[str, list[str]] | None = None) → dict[str, numpy.dtype]#
shapes(num_jets: int, groups: list[str] | None = None) → dict[str, tuple[int, Ellipsis]]#
stream(variables: dict | None = None, num_jets: int | None = None, cuts: ftag.cuts.Cuts | None = None, start: int = 0, skip_batches: int = 0) → collections.abc.Generator#

Generate batches of selected jets.

Parameters:

  • variables (dict | None, optional) – Dictionary of variables to for each group, by default use all jet variables.

  • num_jets (int | None, optional) – Total number of selected jets to generate, by default all.

  • cuts (Cuts | None, optional) – Selection cuts to apply, by default None

  • start (int, optional) – Starting index of the first jet to read, by default 0

  • skip_batches (int, optional) – Number of batches to skip, by default 0

Yields:

Generator – Generator of batches of selected jets.

load(variables: dict | None = None, num_jets: int | None = None, cuts: ftag.cuts.Cuts | None = None) → dict#

Load multiple batches of selected jets into memory.

Parameters:

  • variables (dict | None, optional) – Dictionary of variables to for each group, by default use all jet variables.

  • num_jets (int | None, optional) – Total number of selected jets to load, by default all.

  • cuts (Cuts | None, optional) – Selection cuts to apply, by default None

Returns:

Dictionary of arrays for each group.

Return type:

dict

estimate_available_jets(cuts: ftag.cuts.Cuts, num: int = 1000000) → int#

Estimate the number of jets available after selection cuts.

Parameters:

  • cuts (Cuts) – Selection cuts to apply.

  • num (int, optional) – Number of jets to use for the estimation, by default 1_000_000.

Returns:

Estimated number of jets available after selection cuts, rounded down.

Return type:

int