The Remez Exchange Algorithm
Overview
The Remez exchange algorithm (also called the Remez algorithm) is an iterative numerical method for computing the best uniform (minimax) approximation of a continuous function \( f(x) \) on a closed interval \( [a, b] \) using functions from a finite-dimensional Chebyshev system (also known as a Haar subspace).
It is most famous for finding the polynomial \( p(x) \) of degree at most \( n \) that minimizes the maximum deviation:
However, the algorithm is far more general and applies to any linear approximating family satisfying the Haar condition, including:
- Rational functions (fixed numerator/denominator degrees)
- Exponential sums
- Trigonometric polynomials
- Splines with fixed knots
- Other Chebyshev systems
This makes Remez the standard method for solving general minimax approximation problems.
Key Property: The Equioscillation Theorem
The solution is uniquely characterized by the de la Vallée-Poussin / Chebyshev equioscillation theorem:
This equioscillation of the error at \( n+2 \) points is both necessary and sufficient for optimality.
How the Algorithm Works
- Initialize a set of \( n+2 \) reference points \( \{x_0, x_1, \dots, x_{n+1}\} \) in \( [a, b] \) (often Chebyshev nodes or equally spaced).
-
Solve the leveled-reference equation:
Find coefficients of \( p(x) \) and deviation \( E \) such that: $$ f(x_i) - p(x_i) = (-1)^i E \quad \text{for} \quad i = 0, \dots, n+1 $$ This forms a nonlinear system (or linear after fixing the sign pattern). -
Update reference set:
Find the extrema of the current error \( e(x) = f(x) - p(x) \).
Replace the old reference points with new ones where the error achieves local maxima/minima (typically using the current extrema). - Repeat steps 2–3 until the error equioscillates (or changes are negligible).
Convergence: The algorithm typically exhibits quadratic convergence near the solution.
| f1: | \(\displaystyle f(x)=\begin{cases} x^{2} & for\ x<\dfrac{1}{\sqrt{2}} \\ -x^{2}+2\sqrt{2}\,x-1 & for\ x>\dfrac{1}{\sqrt{2}} \end{cases}\) | |
| f2: | \(\displaystyle f(x)=|x|\sqrt{|x|}\) | |
| f3: | \(\displaystyle f(x)=x^{3} + \dfrac{x^{1/3}e^{-x^{2}}}{8}\) | |
| f4: | \(\displaystyle \frac{100\pi \left(x^{2}-0.36 \right)}{sinh\left(100\pi \left(x^{2}-0.36 \right) \right)}\) | |
| f5: | \(\displaystyle f(x)=-\dfrac{1}{log|x|}\) | |
| f6: | \(\displaystyle f(x)=\sqrt{|x|}\) |
| f1: | \(\displaystyle f(x)=tanh(x+0.5)-tanh(x-0.5)\) | |
| f2: | \(\displaystyle f(x)=sin(exp(x))\) | |
| f3: | \(\displaystyle f(x)=\sqrt{x+1}\) | |
| f4: | \(\displaystyle f(x)=\sqrt{|x-0.1}|\) | |
| f5: | \(\displaystyle f(x)=1-sin(5|x-0.5|)\) | |
| f6: | \(\displaystyle f(x)=min(sech(3sin(10x)), sin(9x))\) | |
| f7: | \(\displaystyle f(x)=max(sin(20x), e^{x-1})\) | |
| f8: | \(\displaystyle f(x)=sech(10(0.5x+0.3))^{2}+sech(100(0.5x+0.1))^{4}+sech(1000(0.5x-0.1))^{6}\) | |
| f9: | \(\displaystyle f(x)=log(1.0001+x)\) |
-
c.f. 5Best \(L_{\infty}\) Approximation of a polynomial of degree n by polynomials of degree < n\[\begin{align*}The\:best\:approximation\:to\:f(x)=\sum_{k=0}^{n}a_{k}x^{k}\:,a_{0}\lt \gt 0,\:by\:a\:polynomial\:p(x) \:of\:degree\:at\:most\:n−1\:in\:[−1, 1]\:is\end{align*}\] \[\begin{align*}p(x)=f(x)−a_{0}2^{−n+1}T_{n}(x)\:with\:maximum\:norm\:error\:a_{0}2^{−n+1}.\end{align*}\]
-
Differentiable f(x)If f is differentiable, the solution of one of eight equation systems yields the best approximation: \[\begin{aligned} 1.\;& f(x_i) - p(x_i) = (-1)^i E, \quad f'(x_i) - p'(x_i) = 0, \quad x_i \in (a,b),\; 0 \le i \le n+1 \\[4pt] 2.\;& f(x_i) - p(x_i) = -(-1)^i E, \quad f'(x_i) - p'(x_i) = 0, \quad x_i \in (a,b),\; 0 \le i \le n+1 \\[4pt] 3.\;& f(a) - p(a) = E, \quad f(x_i) - p(x_i) = (-1)^i E, \quad f'(x_i) - p'(x_i) = 0, \quad 0 < i \le n+1 \\[4pt] 4.\;& f(a) - p(a) = -E, \quad f(x_i) - p(x_i) = -(-1)^i E, \quad f'(x_i) - p'(x_i) = 0, \quad 0 < i \le n+1 \\[4pt] 5.\;& f(b) - p(b) = (-1)^n E, \quad f(x_i) - p(x_i) = (-1)^i E, \quad f'(x_i) - p'(x_i) = 0, \quad 0 \le i < n+1 \\[4pt] 6.\;& f(b) - p(b) = -(-1)^n E, \quad f(x_i) - p(x_i) = -(-1)^i E, \quad f'(x_i) - p'(x_i) = 0, \quad 0 \le i < n+1 \\[4pt] 7.\;& f(a) - p(a) = E, \quad f(b) - p(b) = (-1)^n E, \quad f(x_i) - p(x_i) = (-1)^i E, \quad f'(x_i) - p'(x_i) = 0, \quad 0 < i < n \\[4pt] 8.\;& f(a) - p(a) = -E, \quad f(b) - p(b) = -(-1)^n E, \quad f(x_i) - p(x_i) = -(-1)^i E, \quad f'(x_i) - p'(x_i) = 0, \quad 0 < i < n \end{aligned}\] Any solution of one of these equations yields an alternant of sufficient length and therefore is the unique solution of the approximation problem. Of course, these equations become very complex with increasing n. That's why Remez invented his algorithm!
-
Best \(L_{\infty}\) Approximation of abs(x) on [-1,1] by polynomials of degree 2n\[\begin{align*}Let\:p_{2n}(x)\:be\:the\:best\:2n-th\:grade\:polynomial\:approximation\:on\:[-1,1].\end{align*}\] \[\begin{align*}Then\:p_{2n}(0)=E,\:p_{2n}(x_{i})=x_{i}+(-1)^{i}E\:for\:0\lt i\lt n\:and\:p_{2n}(1)=1-(-1)^{n}E,\:where\:0\lt x_{i}\lt x_{i+1}\lt ...\lt x_{n}\lt 1.\end{align*}\] \[\begin{align*}Note:\:the\:alternant\:is\:1\:point\:longer\:than\:required.\end{align*}\] \[\begin{align*}n=1:\:p_{2}(x)=x^{2}-\frac{1}{8}; E=\frac{1}{8}.\end{align*}\] \[\begin{align*}n=2:\:p_{4}(x)=ax^{4}+bx^{2}+E\end{align*}\] \[\begin{align*}ax_{1}^{4}+bx_{1}^{2}+E=x_{1}-E\end{align*}\] \[\begin{align*}ax_{2}^{4}+bx_{2}^{2}+E=x_{2}+E\end{align*}\] \[\begin{align*}a+b+E=1-E\end{align*}\] \[\begin{align*}4ax_{1}^{3}+2bx_{1}-1=0\end{align*}\] \[\begin{align*}4ax_{2}^{3}+2bx_{2}-1=0\end{align*}\] \[\begin{align*}The\:system\:has\:the\:following\:solution:\end{align*}\] \[\begin{align*}x_{1} = \frac{5 - 3\sqrt{3} - 2\sqrt{3\sqrt{3} - 5}}{3(\sqrt{3} - 3)}\cong0.28443158314308692545\end{align*}\] \[\begin{align*}a = \frac{5 - 3\sqrt{3}}{8x_{1}^3}\cong-1.0655411683005148005\end{align*}\] \[\begin{align*}b = \frac{3\sqrt{3} - 3}{4x_{1}}\cong1.9302993697449462500\end{align*}\] \[\begin{align*}x_{2} = (1 + \sqrt{3})x_{1}\cong0.7770815364241649003\end{align*}\] \[\begin{align*}E= \frac{1}{2}(1-a-b)\cong0.06762089927778427525\end{align*}\] \[\begin{align*}Setting\:\phi := 3\sqrt{3} - 5\cong0.19615242270663188058\:this\:becomes:\end{align*}\] \[\begin{align*}x_{1} = \frac{2\sqrt{\phi} - \phi}{3(3 - \sqrt{3})}\end{align*}\] \[\begin{align*}a = \frac{-\phi}{8x_{1}^3}\end{align*}\] \[\begin{align*}b = \frac{\phi + 2}{4x_{1}}\end{align*}\] \[\begin{align*}x_{2} = \frac{8 + \phi}{3}x_{1}\end{align*}\] \[\begin{align*}E= \frac{1}{2}(1-a-b)\end{align*}\] \[\begin{align*}\end{align*}\]
-
Best \(L_{\infty}\) Approximation of \(abs(x^{2}-\alpha),0\lt\alpha\lt1\) on [-1,1] by polynomials of degree 4Let \(-1=-x_{0}\lt -x_{1}\lt -x_{2}\lt x_{3}=0\lt x_{2}\lt x_{1}\lt x_{0}=1,\:p_{4}(x)=ax^{4}+bx^{2}+c\).
We try to solve these 5 equations with the unknowns a, b, c, E, \(x_{1}\): \[\begin{align*}p_{4}(-1)=a+b+c=1-\alpha+E\end{align*}\] \[\begin{align*}p_{4}(-x_{1})=ax_{1}^{4}+bx_{1}^{2}+c=x_{1}^{2}-\alpha-E\end{align*}\] \[\begin{align*}p'_{4}(-x_{1})=-4ax_{1}^{3}-2bx_{1}=-2x_{1}\Rightarrow 4ax_{1}^{2}=2-2b\end{align*}\] \[\begin{align*}p_{4}(\sqrt{\alpha})=a\alpha^{2}+b\alpha+c=E\end{align*}\] \[\begin{align*}p'_{4}(\sqrt{\alpha})=4a\sqrt{\alpha}^{3}+2b\sqrt{\alpha}=0\Rightarrow b=-2a\alpha\end{align*}\] -
Approximation of the zero function by monic (normalized) polynomials
The monic polynomial of degree n, which represents the best approximation of the zero function on [-1,1] in the maximum norm, is the normalized Chebyshev polynomial: T̃_n(x) = (1/2^(n-1)) · T_n(x) T_n(x) is the Chebyshev polynomial of the first kind of degree n. Maximum error: ‖T̃_n‖_∞ = 1/2^(n-1) Examples: n=1: T̃₁(x) = x, error = 1 n=2: T̃₂(x) = x² - 1/2, error = 1/2 n=3: T̃₃(x) = x³ - 3x/4, error = 1/4 n=4: T̃₄(x) = x⁴ - x² + 1/8, error = 1/8 -
Functions with steps.It is easy to see that for any piecewise continuos function f with maximum step hight h \(\parallel f-p_{n}\parallel ^{\infty }_{[a,b]}\ge\frac{h}{2}\) holds.
It follows that for any piecewise continuous function with exactly n+1 steps,which all are of height h the best polynomial approximation of degree n must pass through the center of the steps and can be achieved by interpolation in a unique way.
This gives us a method, to construct for any given polynomial functions, which have that polynomial as best approximations.
Here is an example, that uses the floor-function: -
c.f. 6Best approximation of \(\frac {1}{r-x}\), r>1, on [-1, 1]
The function \( f(x) = \frac{1}{r - x} \) with \( r > 1 \) is optimally approximated in the interval \( [-1, 1] \) for each \( n \in \mathbb{N}_0 \)
by the polynomial \( p_n(x) := \frac{1}{r - x} - E_n V_n \) of degree \( n \).
Here, \( E_n := \frac{\left( r - \sqrt{r^2 - 1} \right)^n}{r^2 - 1} \) and \( V_n(x) := \cos(n \varphi + \psi) \), where \( \varphi \) is defined implicitly by \( \cos \varphi := x \) and \( \psi \) by \( \tan \frac{\psi}{2} := \sqrt{\frac{r + 1}{r - 1}} \tan \frac{\varphi}{2} \).
Proof Sketch
Polynomial Property: Through transformations, including those involving Chebyshev polynomials of the first and second kind, the function \( q(x) := 1 - (r - x) E_n V_n \) in the numerator of \( p_n(x) = \frac{1}{r - x} - E_n V_n \) is shown to be a polynomial of degree \( n + 1 \). Thus, \( p_n(x) \) is initially a rational function. Furthermore, \( q(x) \) has a zero at \( x = r \), so the factor \( (x - r) \) can be factored out of \( q(x) \) and canceled with the denominator \( (r - x) \) of \( p_n(x) \). Ultimately, \( p_n(x) \) is a polynomial of degree \( n \).
Best Approximation: The given relations define a monotone and bijective mapping
\[\begin{align*} ]-1,\ 1[\ \ \to\ \ ]0,\ (n + 1) \pi[\end{align*}\]
\[\begin{align*}\quad x \mapsto n \varphi + \psi\end{align*}\]
Because of the monotony this maps the extrema of the error function \( f(x) - p_n(x) = E_n \cos(n \varphi + \psi) \) onto the extrema of the mapped function, which are attained where \(n \varphi + \psi\) is a multiple of \( \pi \).
By adding the interval boundaries \( x_0 = -1 \) with \( n \varphi + \psi = 0 \) and \( x_{n+1} = 1 \) with \( n \varphi + \psi = (n + 1) \pi \), (using the main branches of the arc-functions) one obtains the \( n + 2 \) alternants \( -1=x_0 < \dots < x_i < \dots < x_{n+1}=1 \), for which the error function takes on the extremal value \( (-1)^i E_n \) exactly \( n + 2 \) times with alternating sign.
Example: First 4 Best Approximations for r = 37/12
n Best Polynomial pn Extrema Points Max Error En 0 444/1225 1, -1 144/1225 ≈ 0.1176 1 (144/1225)x + 12/35 1, 1/6, -1 24/1225 ≈ 0.0196 2 (48/1225)x² + (4/35)x + 396/105 1, 7/12, -5/12, -1 4/1225 ≈ 0.0033 3 (16/1225)x³ + (4/105)x² + (128/1225)x + 34/105 1, 3/4, 1/12, -2/3, -1 2/3675 ≈ 0.0005
- Polynomials on any interval: \[\begin{align*}\left\{1,x,\cdots,x^{n}\right\}\end{align*}\]
-
Polynomials with even exponents on any interval [a, b] where a > 0: \[\begin{align*}\left\{1,x^{2},\cdots,x^{2n}\right\}\end{align*}\]
2
-
Exponential functions on any interval: \[\begin{align*}\left\{1,e^{\alpha_{0}x},\cdots,e^{\alpha_{n}x}\right\},\alpha_{i}\in\Re\end{align*}\]
2
-
Trigonomertric functions: \[\begin{align*}\left\{1,sin\left(x\right),\cdots,sin\left(nx\right)\right\};\:\left\{1,cos\left(x\right),\cdots,cos\left(nx\right)\right\};\:\left\{1,sin\left(x\right),cos\left(x\right),\cdots,sin\left(nx\right),cos\left(nx\right)\right\}\:on\:\left[0,2\pi\right]\end{align*}\]
3
-
On any interval:\[\begin{align*}\:\left\{1,x,\cdots,x^{n},f(x)\right\},\:if\:f^{\left(n\right)}\left(x\right)\lt\gt0\end{align*}\] (proof by induction using the mean value theorem)
2
-
The Faber–Schauder system on any interval:
\[\begin{align*}\left\{ \begin{cases}2^n(t-\frac{k}{2^n})&\text{for }t\in I_{n,k}\text{ left half }\\2^n(\frac{k+1}{2^n}-t)&\text{for }t\in I_{n,k}\text{ right half }\\0&\text{otherwise}\end{cases}\\f(t)=a_0s_0(t)+a_1s_1(t)+\sum_{m=0}^{n-1}\sum_{k=0}^{2^m-1}a_{m,k}s_{m,k}(t) \right\}\end{align*}\] 4
2 PETER B. BORWEIN: ON SYLVESTER'S PROBLEM AND HAAR SPACES, PACIFIC JOURNAL OF MATHEMATICS Vol 109, No. 2, 1983
3 Wikipedia Haar-Raum
4 lolremez
5 Mudde, M.H. (2017) Chebyshev approximation. Master's Thesis
6 Wikipedia Alternantensatz
7 ALGEBRAICALLY SOLVABLE CHEBYSHEV APPROXIMATION PROBLEMS
8 RATIONAL MINIMAX APPROXIMATION VIA ADAPTIVE BARYCENTRIC REPRESENTATIONS
9 Ricardo Pachón · Lloyd N. Trefethen: Barycentric-Remez algorithms for best polynomial approximation in the chebfun system