[mif]

The graph named “non-linear growth”, is actually showing linear growth. I know, it’s confusing, but as long as the factor is constant (10), growth is linear.

A quick way to check if something grows linearly is to put it on a log-scale and to see whether it’s a straight line.

Nice explanation, though. We should talk about logs more often.

[kccqzy]

I think you and the author are using terminology differently. That graph is absolutely a logarithmic axis to me. The five ticks on the axis are equidistant but each represents a number 10x of the previous. That's nonlinear to me. My definition of linear growth is that it is bounded above by a linear function. Its first derivative would therefore be bounded above by a constant.

If something is a straight line when you plot it in log scale, you are plotting exponential growth.

[Bimos]

Good point. Also "linear convergence" means the residual reduces "linearly", or |r_{k+1}| = \lambda |r_k| with \lambda \in (0, 1). So it is somehow exponentially converging, and an algorithm with linear convergence is neat and fast.

[nerdponx]

"Linear" here doesn't mean "a linear transformation is applied at each time step", it means "a constant rate of change over all time".

Multiplicative growth by a constant factor is an increasing rate of change over time.