npstreams.pload¶
-
npstreams.
pload
(files, load_func, processes=1, **kwargs)¶ Create a stream of arrays from files, which are loaded lazily from multiple processes.
This function should be preferred to
iload()
in cases where the consumer function is much faster than the data can be loaded.Parameters: - pattern (iterable of str or str) – Either an iterable of filenames or a glob-like pattern str.
- load_func (callable, optional) – Function taking a filename as its first arguments
- processes (int or None, optional) – Number of processes to use. If None, maximal number of processes is used. Default is one.
- kwargs – Keyword arguments are passed to
load_func
.
Yields: arr (~numpy.ndarray) – Loaded data.
See also
iload()
- load files lazily