npstreams.pload¶

npstreams.pload(files, load_func, processes=1, **kwargs)¶

Create a stream of arrays from files, which are loaded lazily from multiple processes.

This function should be preferred to iload() in cases where the consumer function is much faster than the data can be loaded.

Parameters

pattern (iterable of str or str) – Either an iterable of filenames or a glob-like pattern str.
load_func (callable, optional) – Function taking a filename as its first arguments
processes (int or None, optional) – Number of processes to use. If None, maximal number of processes is used. Default is one.
kwargs – Keyword arguments are passed to load_func.

Yields

arr (~numpy.ndarray) – Loaded data.