Helper Functions

Sparse Hessians

class vittles.sparse_hessian_lib.SparseBlockHessian(objective_function, sparsity_array)

Efficiently calculate block-sparse Hessians.

The objective function is expected to be of the form

\[ \begin{align}\begin{aligned}x = (x_1 , ... , x_G) \textrm{ (or some permutation thereof)}\\f(x) = \sum_{g=1}^{G} f_g(x_g)\end{aligned}\end{align} \]

Each \(x_g\) is expected to have the same dimension. Consequently, the Hessian matrix of f with respect to x, is block diagonal with G blocks, up to a permutation of the order of x. The purpose of this class is to efficiently calculate this Hessian when the block structure (i.e., the partition of x) is known.

__init__(self, objective_function, sparsity_array)

In terms of the class description, objective_function = f, opt_par = x, and sparsity_array describes the partition of x into \((x_1, ..., x_G)\).

Parameters:

objective_function : callable

An objective function of which to calculate a Hessian. The argument should be

• opt_par: numpy.ndarray (N,) The optimization parameter.

sparsity_array : numpy.ndarray (G, M)

An array containing the indices of rows and columns of each block. The Hessian should contain G dense blocks, each of which is M by M. Each row of sparsity_array should contain the indices of the corresponding block. There must be no repeated indices, and each block must be the same size.

get_block_hessian(self, opt_par, print_every=0)

Get the block Hessian at opt_par and weights.

Returns:

hessian : scipy.sparse.coo_matrix (N, N)

The block-sparse Hessian given by and sparsity_array.

get_global_hessian(self, opt_par, global_inds=None, print_every=0)

Get the dense Hessian terms for the global parameters, which are, by default, indexed by any indices not in _sparsity_array.

System solvers

vittles.solver_lib.get_cg_solver(mat_times_vec, dim, cg_opts={})

Return a function solving \(h^{-1} v\) for a matrix h using conjugate gradient.

Parameters:

mat_times_vec : callable

A function of a single vector argument, v that returns the product h v for some invertible matrix h.

dim : int

The dimension of the vector v.

cg_opts : dict

Optional, a dictionary of named arguments for the solver sp.sparse.linalg.cg.

Returns:

solve : callable

A function of a single vector argument, v that returns the conjugate gradient approximation to \(h^{-1} v\).

vittles.solver_lib.get_cholesky_solver(h)

Return a function solving \(h^{-1} v\) for a matrix h.

Parameters:

h : A dense or sparse matrix.

Returns:

solve : callable

A function of a single vector argument, v that returns \(h^{-1} v\).

vittles.solver_lib.get_dense_cholesky_solver(h, h_chol=None)

Return a function solving \(h^{-1} v\) with a dense Cholesky factorization.

Parameters:

h : numpy.ndarray

A dense symmetric positive definite matrix.

h_chol : A Cholesky factorization

Optional, a Cholesky factorization created with sp.linalg.cho_factor. If this is specified, the argument h is ignored. If None (the default), the factoriztion of h is calculated.

Returns:

solve : callable

A function of a single vector argument, v that returns \(h^{-1} v\).

vittles.solver_lib.get_sparse_cholesky_solver(h)

Return a function solving \(h^{-1} v\) for a sparse h.

Parameters:

h : A sparse invertible matrix.

Returns:

solve : callable

A function of a single vector argument, v that returns \(h^{-1} v\).