Mathematical background

Fisher information matrix

The Fisher information measures the amount of information that a set of observables \(X_i\) carries about unknown parameters \(\theta_i\), upon which the probability distribution of the observables depends.

When there are \(N\) parameters, i.e., \(\theta = (\theta_1, \ldots, \theta_N)\), the Fisher information takes the form of an \(N \times N\) matrix. The Fisher Information Matrix (FIM) \(I(\theta)\) for \(X_a\) has elements given by

\[I_{ij}(\theta) = \mathrm{E} \left[ \left( \frac{\partial \log p(X_a | \theta)}{\partial \theta_i} \right) \left( \frac{\partial \log p(X_a | \theta)}{\partial \theta_j} \right) \middle| \theta \right],\]

where \(p(X_a | \theta)\) is the probability of the observable \(X_a\) given the parameters \(\theta\), and \(\mathrm{E}\) is the expectation value over the distribution of \(X_a\) (here, evaluated at the true parameter values).

Note that, under the regularity conditions, the FIM can also be expressed as

\[I_{ij}(\theta) = - \mathrm{E} \left[ \frac{\partial^2 \log p(X_a | \theta)}{\partial \theta_i \partial \theta_j} \middle| \theta \right].\]

Note that these expressions are valid for any probability distribution of the observables; however, they involve the expectation value over the distribution of the observables, which is often not known analytically. In practice, the FIM is often estimated by sampling from the distribution of the observables, for example using Monte Carlo methods.

Matched-filter Gaussian-noise likelihood

Consider a set of observables \(X_a\) defined as

\[X_a = h_a(\theta) + n_a,\]

where \(h_a(\theta)\) is a template waveform (including the instrument response) and \(n_a\) is a noise term. If this noise is Gaussian, and under the regularity conditions, the components of the FIM no longer involve the expectation value over the distribution of the observables; but instead can be directly evaluated as

\[I_{ij}(\theta) = \left( \frac{\partial h}{\partial \theta_i} \middle| \frac{\partial h}{\partial \theta_j} \right),\]

where the frequency-domain noise-weighted inner product is defined as

\[(a|b) = 4 \Re \int_{f_{\min}}^{f_{\max}} \tilde{a}(f) \Sigma^{-1}(f) \tilde{b}^+(f) \, \mathrm{d}f,\]

with \(\tilde{a}(f)\) and \(\tilde{b}(f)\) the Fourier transforms of the signals \(a(t)\) and \(b(t)\), respectively, and \(\Sigma(f)\) the noise covariance matrix.

Here, we have assumed that the noise is independent of the model parameters \(\theta\), and that the noise is stationary, such that there is no correlation between the noise at different frequencies (i.e., the noise covariance matrix is frequency-dependent and only characterize the noise correlations between the various observables channels).

The inner product is implemented in wejax.fim.inner_product(), and the matched-filter Gaussian-noise FIM is implemented in wejax.fim.matched_filter_gaussian_fim().

Error estimation

The inverse of the FIM provides an estimate of the covariance matrix \(\Sigma\) of the parameters, which can be used to estimate the errors on the parameters. The diagonal elements of the covariance matrix provide the variance of the parameters, while the off-diagonal elements provide the covariance between the parameters,

\[\mathrm{Cov}(\theta_i, \theta_j) = \Sigma_{ij} = \left( I^{-1} \right)_{ij}.\]

The standard deviation of the parameters is given by the square root of the diagonal elements of the covariance matrix,

\[\sigma(\theta_i) = \sqrt{\Sigma_{ii}} = \sqrt{\left( I^{-1} \right)_{ii}}.\]

The correlation between the parameters is given by the off-diagonal elements of the covariance matrix divided by the square root of the product of the diagonal elements,

\[\rho(\theta_i, \theta_j) = \frac{\Sigma_{ij}}{\sqrt{\Sigma_{ii} \Sigma_{jj}}} = \frac{\left( I^{-1} \right)_{ij}}{\sqrt{\left( I^{-1} \right)_{ii} \left( I^{-1} \right)_{jj}}}.\]

The correlation matrix \(P\) is usually defined as the matrix containing the correlation coefficients \(\rho(\theta_i, \theta_j)\) as the off-diagonal elements. Here we will also include the standard deviations \(\sigma(\theta_i)\) as the diagonal elements instead of the usual unit entries. This gives a matrix of the form:

\[\begin{split}P_{ij} = \begin{cases} \sigma(\theta_i) & \text{if } i = j, \\ \rho(\theta_i, \theta_j) & \text{if } i \neq j. \end{cases}\end{split}\]

This form combines both the scale (via standard deviations) and the correlation structure between parameters, and can be helpful for visualization or uncertainty reporting.

To convert the FIM to a covariance or a correlation matrix, use wejax.fim.fim_to_covariance() or wejax.errors.fim_to_correlation(). You can plot the latter using wejax.errors.plot_correlation_matrix().