dataeval.utils.metadata.merge#

dataeval.utils.metadata.merge(metadata, ignore_lists=False, fully_qualified=False, as_numpy=False)#

Merges a collection of metadata dictionaries into a single flattened dictionary of keys and values.

Nested dictionaries are flattened, and lists are expanded. Nested lists are dropped as the expanding into multiple hierarchical trees is not supported.

Parameters:

  • metadata (Iterable[Mapping[str, Any]]) – Iterable collection of metadata dictionaries to flatten and merge

  • ignore_lists (bool, default False) – Option to skip expanding lists within metadata

  • fully_qualified (bool, default False) – Option to return dictionary keys full qualified instead of minimized

  • as_numpy (bool, default False) – Option to return results as lists or NumPy arrays

Returns:

  • dict[str, list[Any]] or dict[str, NDArray[Any]] – A single dictionary containing the flattened data as lists or NumPy arrays

  • NDArray[np.int_] – Array defining where individual images start, helpful when working with object detection metadata

Return type:

tuple[dict[str, list[Any]] | dict[str, numpy.typing.NDArray[Any]], numpy.typing.NDArray[numpy.int_]]

Note

Nested lists of values and inconsistent keys are dropped in the merged metadata dictionary

Example

>>> list_metadata = [{"common": 1, "target": [{"a": 1, "b": 3, "c": 5}, {"a": 2, "b": 4}], "source": "example"}]
>>> reorganized_metadata, image_indicies = merge(list_metadata)
>>> reorganized_metadata
{'common': [1, 1], 'a': [1, 2], 'b': [3, 4], 'source': ['example', 'example']}
>>> image_indicies
array([0])