Skip to content

Preconditioner #1588

Description

@epapoutsellis

In #1586 , I described the new PGA class which implements the following iteration
$$x_{k+1} = prox_{\gamma_{k}g}(x_{k} - \gamma_{k}D(x_{k})\nabla f(x_{k}))$$
where $D(x_{k})$ is a (diagonal) Preconditioner. For this, I implemented thee base class called Preconditioner.

It is an ABC class with an abstractmethod called update. There are two classes that inherit from this class 1) Sensitivity, 2) AdaptiveSensitivity

  1. This computes that standard $D(x_{k}) = \frac{1}{A^{T}1}$. So it is constant for all $x_{k}$ and computed when the class is instantiated. Then, the update method will change the output the f.gradient. It works for both CIL and SIRF.
  2. This is $D(x_{k}) = \frac{x_{k} + \delta}{A^{T}1}$, where $\delta>=0$. If $\delta=0$, we have the standard preconditioner used in MLEM. The $\delta>0$ was recently used here for stochastic optimisation.

Screen Shot 2023-11-22 at 20 10 45

In the AdaptiveSensitivity there are freezing_point and iterations arguments, which are used for specific cases of this preconditioner:
"We investigated the impact of different preconditioner inputs on the convergence of SAGA and SVRG. These included
D(xk), freezing the preconditioner at D(xOSEM), and freezing the preconditioner after 5 and 10 epochs, D(x5M) and D(x10M) respectively.
"

Since the Preconditioner acts on self, i.e., the actual algorithm, our users can define their own preconditioner.

Example: SIRT is equivalent to a Preconditioned Projected Gradient Descent on Weighted LS.

Screen Shot 2023-11-22 at 20 20 27

Example: MLEM for PET using a SIRF function and a CIL algorithm

Screen Shot 2023-11-22 at 20 22 19

Note In MLEM for PET, the SIRF Objective class make_Poisson_log_likelihood(d) maximizes the negative log, so we need to take step_size = -1 to simulate a gradient ascent. For CT (CIL) and using the KullbackLeibler function, we can keep step_size = 1 and have MLEM for CT application for low counts.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions