Reference/API

Click on any function below to see detailed information.

Creation of Streams

Decorator for streaming functions which guarantees that the stream elements will be converted to arrays.

array_stream(func)

Decorates streaming functions to make sure that the stream is a stream of ndarrays.

The array_stream() decorator wraps iterables into an ArrayStream iterator. This is not required to use the functions defined here, but it provides some nice guarantees.

ArrayStream(stream)

Iterator of arrays.

Statistical Functions

imean(arrays[, axis, ignore_nan])

Streaming mean of arrays.

iaverage(arrays[, axis, weights, ignore_nan])

Streaming (weighted) average of arrays.

istd(arrays[, axis, ddof, weights, ignore_nan])

Streaming standard deviation of arrays.

ivar(arrays[, axis, ddof, weights, ignore_nan])

Streaming variance of arrays.

isem(arrays[, axis, ddof, weights, ignore_nan])

Streaming standard error in the mean (SEM) of arrays.

ihistogram(arrays, bins[, range, weights])

Streaming histogram calculation.

The following functions consume entire streams. By avoiding costly intermediate steps, they can perform much faster than their generator versions.

mean(arrays[, axis, ignore_nan])

Mean of a stream of arrays.

average(arrays[, axis, weights, ignore_nan])

Average (weighted) of a stream of arrays.

std(arrays[, axis, ddof, weights, ignore_nan])

Total standard deviation of arrays.

var(arrays[, axis, ddof, weights, ignore_nan])

Total variance of a stream of arrays.

sem(arrays[, axis, ddof, weights, ignore_nan])

Standard error in the mean (SEM) of a stream of arrays.

average_and_var(arrays[, axis, ddof, ...])

Calculate the simultaneous average and variance of a stream of arrays.

Numerics

isum(arrays[, axis, dtype, ignore_nan])

Streaming sum of array elements.

iprod(arrays[, axis, dtype, ignore_nan])

Streaming product of array elements.

isub(arrays[, axis, dtype])

Subtract elements in a reduction fashion.

sum(arrays[, axis, dtype, ignore_nan])

Sum of arrays in a stream.

prod(arrays[, axis, dtype, ignore_nan])

Product of arrays in a stream.

Linear Algebra

idot(arrays)

Yields the cumulative array inner product (dot product) of arrays.

iinner(arrays)

Cumulative inner product of all arrays in a stream.

itensordot(arrays[, axes])

Yields the cumulative array inner product (dot product) of arrays.

ieinsum(arrays, subscripts, **kwargs)

Evaluates the Einstein summation convention on the operands.

Control Flow

ipipe(*args, **kwargs)

Pipe arrays through a sequence of functions.

iload(files, load_func, **kwargs)

Create a stream of arrays from files, which are loaded lazily.

pload(files, load_func[, processes])

Create a stream of arrays from files, which are loaded lazily from multiple processes.

Comparisons

iany(arrays[, axis])

Test whether any array elements along a given axis evaluate to True.

iall(arrays[, axis])

Test whether all array elements along a given axis evaluate to True

imax(arrays, axis[, ignore_nan])

Maximum of a stream of arrays along an axis.

imin(arrays, axis[, ignore_nan])

Minimum of a stream of arrays along an axis.

Parallelization

pmap(func, iterable[, args, kwargs, ...])

Parallel application of a function with keyword arguments.

pmap_unordered(func, iterable[, args, ...])

Parallel application of a function with keyword arguments in no particular order.

preduce(func, iterable[, args, kwargs, ...])

Parallel application of the reduce function, with keyword arguments.

Stacking

stack(arrays[, axis])

Stack of all arrays from a stream.

Iterator Utilities

last(stream)

Retrieve the last item from a stream/iterator, consuming iterables in the process.

cyclic(iterable)

Yields cyclic permutations of an iterable.

itercopy(iterable[, copies])

Split iterable into 'copies'.

chunked(iterable, chunksize)

Generator yielding multiple iterables of length 'chunksize'.

linspace(start, stop, num[, endpoint])

Generate linear space.

multilinspace(start, stop, num[, endpoint])

Generate multilinear space, for joining the values in two iterables.

peek(iterable)

Peek ahead in an iterable.

primed(gen)

Decorator that primes a generator function, i.e. runs the function until the first yield statement.

length_hint(obj[, default])

Return an estimate of the number of items in obj.

Array Utilities

nan_to_num(array[, fill_value, copy])

Replace NaNs with another fill value.

Benchmarking

benchmark([funcs, ufuncs, shapes, file])

Benchmark npstreams against numpy and print the results.