.

e o

587?
I
S

Meshfree Approximation
Methods with MATLAB

1

INTERDISCIPLINARY MATHEMATICAL SCIENCES

Series Editor: Jinqiao Duan (Illinois Inst, of Tech., USA)

Editorial Board: Ludwig Arnold, Roberto Camassa, Peter Constantin,
Charles Doering, Paul Fischer, Andrei V. Fursikov, Fred R. McMorris,
Daniel Schertzer, Bjorn Schmalfuss, Xiangdong Ye, and
Jerzy Zabczyk

Published
V o l . 1: G l o b a l A t t r a c t o r s o f N o n a u t o n o m o u s D i s s i p a t i v e D y n a m i c a l Systems
David N. Cheban

V o l . 2: Stochastic D i f f e r e n t i a l E q u a t i o n s : T h e o r y a n d Applications
A V o l u m e i n H o n o r o f Professor B o r i s L . R o z o v s k i i
eds. Peter H. Baxendale & Sergey V. Lototsky

V o l . 3: A m p l i t u d e E q u a t i o n s f o r Stochastic P a r t i a l D i f f e r e n t i a l E q u a t i o n s
Dirk Blomker

V o l . 4: Mathematical Theory of Adaptive Control

Vladimir G. Sragovich

V o l . 5: T h e H i l b e r t - H u a n g T r a n s f o r m and Its A p p l i c a t i o n s
Norden E. Huang & Samuel S. P. Shen

Vol. 6: Meshfree A p p r o x i m a t i o n Methods w i t h MATLAB

Gregory E. Fasshauer
Meshfree Approximation
Methods with M A T L A B

G r e g o r y E . F a s s h a u e r r _ s

Illinois Institute of Technology, U S A - ^

W r i f l r l ^ H o n t i f i n
. lb. /
 Published by
W o r l d
fif-?, Y~ Scientific Publishing Co. Pte. Ltd.
5 Toh Tuck Link, Singapore 596224
U S A c e : 2 7 W a r r e n
/ t>&^~ ^ Street, Suite 401-402, Hackensack, NJ 07601
/] UK office: 57 Shelton Street, Covent Garden, London W C 2 H 9 H E

British Library Cataloguing-in-Publication Data

A catalogue record for this book is available from the British Library.

M E S H F R E E APPROXIMATION M E T H O D S W I T H MATLAB
(With CD-ROM)
Interdisciplinary Mathematical Sciences Vol. 6

Copyright 2007 by World Scientific Publishing Co. Pte. Ltd.

All rights reserved. This book, or parts thereof, may not be reproduced in any form or by any means, electronic or
mechanical, including photocopying, recording or any information storage and retrieval system now known or to
be invented, without written permission from the Publisher.

For photocopying of material in this volume, please pay a copying fee through the Copyright Clearance Center,
Inc., 222 Rosewood Drive, Danvers, M A 01923, U S A . In this case permission to photocopy is not required from
the publisher.

ISBN-13 978-981-270-633-1
ISBN-10 981-270-633-X
ISBN-13 978-981-270-634-8 (pbk)
ISBN-10 981-270-634-8 (pbk)

Printed by Mainland Press Pte Ltd

 This book is dedicated to
Inge, Conny, Marc and Patrick.
 Preface 34

T r a d i t i o n a l n u m e r i c a l m e t h o d s , s u c h as finite e l e m e n t , finite difference, o r finite v o l -

u m e m e t h o d s , w e r e m o t i v a t e d m o s t l y b y e a r l y one- a n d t w o - d i m e n s i o n a l s i m u l a t i o n s
of engineering problems v i a p a r t i a l differential equations ( P D E s ) . T h e discretiza-
t i o n i n v o l v e d i n a l l o f these m e t h o d s r e q u i r e s some s o r t o f u n d e r l y i n g c o m p u t a t i o n a l
m e s h , e.g., a t r i a n g u l a t i o n o f t h e r e g i o n o f i n t e r e s t . C r e a t i o n o f these meshes ( a n d
possible r e - m e s h i n g ) becomes a r a t h e r d i f f i c u l t t a s k i n t h r e e d i m e n s i o n s , a n d v i r t u -
a l l y i m p o s s i b l e for h i g h e r - d i m e n s i o n a l p r o b l e m s . T h i s is w h e r e meshfree methods
enter t h e p i c t u r e . Meshfree m e t h o d s are o f t e n b u t b y n o m e a n s h a v e t o be
r a d i a l l y s y m m e t r i c i n n a t u r e . T h i s is a c h i e v e d b y c o m p o s i n g some u n i v a r i a t e basic
function w i t h a (Euclidean) n o r m , a n d therefore t u r n i n g a p r o b l e m i n v o l v i n g m a n y
space d i m e n s i o n s i n t o o n e t h a t is v i r t u a l l y o n e - d i m e n s i o n a l . S u c h radial basis func-
tions are a t t h e h e a r t o f t h i s b o o k . Some p e o p l e have a r g u e d t h a t t h e r e are t h r e e
" b i g t e c h n o l o g i e s " for t h e n u m e r i c a l s o l u t i o n o f P D E s , n a m e l y finite difference, fi-
n i t e element, a n d s p e c t r a l m e t h o d s . W h i l e these t e c h n o l o g i e s c a m e i n t o t h e i r o w n
r i g h t i n successive decades, n a m e l y finite difference m e t h o d s i n t h e 1950s, finite e l -
e m e n t m e t h o d s i n t h e 1960s, a n d s p e c t r a l m e t h o d s i n t h e 1970s, meshfree methods
s t a r t e d t o a p p e a r i n t h e m a t h e m a t i c s l i t e r a t u r e i n t h e 1980s, a n d t h e y are n o w o n
their w a y t o b e c o m i n g " b i g technology" n u m b e r four. I n fact, we w i l l demonstrate
i n l a t e r p a r t s o f t h i s b o o k h o w d i f f e r e n t t y p e s o f meshfree m e t h o d s c a n b e v i e w e d
as g e n e r a l i z a t i o n s o f t h e t r a d i t i o n a l " b i g t h r e e " .
M u l t i v a r i a t e meshfree a p p r o x i m a t i o n m e t h o d s are b e i n g s t u d i e d b y m a n y re-
searchers. T h e y exist i n m a n y flavors a n d are k n o w n u n d e r m a n y n a m e s , e.g.,
diffuse e l e m e n t m e t h o d , element-free G a l e r k i n m e t h o d , g e n e r a l i z e d finite element
m e t h o d , / i p - c l o u d s , meshless l o c a l P e t r o v - G a l e r k i n m e t h o d , m o v i n g least squares
method, partition of unity finite e l e m e n t m e t h o d , r a d i a l basis f u n c t i o n method,
reproducing kernel particle m e t h o d , smooth particle hydrodynamics m e t h o d .
I n t h i s b o o k w e are c o n c e r n e d m o s t l y w i t h t h e m o v i n g least squares ( M L S ) a n d
r a d i a l basis f u n c t i o n ( R B F ) m e t h o d s . W e w i l l c o n s i d e r a l l d i f f e r e n t k i n d s o f aspects
o f these meshfree approximation methods: H o w to construct them? A r e these
c o n s t r u c t i o n s m a t h e m a t i c a l l y j u s t i f i a b l e ? H o w a c c u r a t e are t h e y ? A r e theEg~wavs
t o i m p l e m e n t t h e m e f f i c i e n t l y w i t h s t a n d a r d m a t h e m a t i c a l software-packages such

vii
viii Meshfree Approximation Methods with M A T L A B

as M A T L A B ? H O W d o t h e y c o m p a r e w i t h t r a d i t i o n a l m e t h o d s ? H o w d o t h e v a r i o u s
flavors o f meshfree m e t h o d s differ f r o m one a n o t h e r , a n d h o w are t h e y s i m i l a r t o one
a n o t h e r ? Is t h e r e a g e n e r a l f r a m e w o r k t h a t c a p t u r e s a l l o f these m e t h o d s ? What
s o r t o f a p p l i c a t i o n s are t h e y e s p e c i a l l y w e l l s u i t e d for?
W h i l e w e d o p r e s e n t m u c h o f t h e u n d e r l y i n g t h e o r y for R B F a n d M L S ap-
proximation methods, t h e e m p h a s i s i n t h i s b o o k is n o t o n p r o o f s . For read-
ers w h o are interested i n all the mathematical details and intricacies of the
t h e o r y we r e c o m m e n d t h e t w o e x c e l l e n t recent monographs [ B u h m a n n (2003);
W e n d l a n d (2005a)]. I n s t e a d , o u r o b j e c t i v e is t o m a k e t h e t h e o r y accessible t o a
w i d e a u d i e n c e t h a t i n c l u d e s g r a d u a t e s t u d e n t s a n d p r a c t i t i o n e r s i n a l l s o r t s o f sci-
ence a n d e n g i n e e r i n g fields. W e w a n t t o p u t t h e m a t h e m a t i c a l t h e o r y i n t h e c o n t e x t
o f a p p l i c a t i o n s a n d p r o v i d e M A T L A B i m p l e m e n t a t i o n s w h i c h g i v e t h e r e a d e r a n easy
e n t r y i n t o meshfree a p p r o x i m a t i o n m e t h o d s . T h e s k i l l e d r e a d e r s h o u l d t h e n easily
be able t o m o d i f y t h e p r o g r a m s p r o v i d e d here for h i s / h e r specific p u r p o s e s .
I n a c e r t a i n sense t h e present b o o k was i n s p i r e d b y t h e b e a u t i f u l l i t t l e b o o k [ T r e -
f e t h e n ( 2 0 0 0 ) ] . W h i l e t h e present b o o k is m u c h m o r e e x p a n s i v e ( f i l l i n g m o r e t h a n
1 2
five h u n d r e d pages w i t h f o r t y - s e v e n M A T L A B p r o g r a m s , one M a p l e p r o g r a m , one
h u n d r e d figures, m o r e t h a n f i f t y t a b l e s , a n d m o r e t h a n five h u n d r e d references), i t is
o u r a i m t o p r o v i d e t h e reader w i t h r e l a t i v e l y s i m p l e M A T L A B c o d e t h a t i l l u s t r a t e s
j u s t a b o u t e v e r y aspect discussed i n t h e b o o k .
A l l M A T L A B p r o g r a m s p r i n t e d i n t h e t e x t (as w e l l as a few m o d i f i c a t i o n s dis-
cussed) are also i n c l u d e d o n t h e enclosed C D . T h e f o l d e r MATLAB c o n t a i n s M - f i l e s
a n d d a t a files o f t y p e M A T t h a t h a v e b e e n w r i t t e n a n d t e s t e d w i t h M A T L A B 7. F o r
t h o s e readers w h o d o n o t have access t o M A T L A B 7, t h e folder MATLAB6 c o n t a i n s
versions o f these files t h a t are c o m p a t i b l e w i t h t h e o l d e r M A T L A B release. The
m a i n difference b e t w e e n t h e t w o v e r s i o n s is t h e use o f a n o n y m o u s f u n c t i o n s i n t h e
M A T L A B 7 code as c o m p a r e d t o i n l i n e f u n c t i o n s i n t h e M A T L A B 6 v e r s i o n . T w o
packages f r o m t h e M A T L A B C e n t r a l F i l e E x c h a n g e [ M C F E ] are used t h r o u g h o u t t h e
b o o k : t h e f u n c t i o n h a l t o n s e q w r i t t e n b y D a n i e l D o u g h e r t y a n d used t o g e n e r a t e
sequences o f H a l t o n p o i n t s ; t h e /cd-tree l i b r a r y ( g i v e n as a set o f M A T L A B M E X - f i l e s )
w r i t t e n b y G u y Shechter a n d used t o g e n e r a t e t h e kd-tvee data structure underlying
o u r sparse m a t r i c e s based o n c o m p a c t l y s u p p o r t e d basis f u n c t i o n s . B o t h o f these
packages are discussed i n A p p e n d i x A a n d need t o be d o w n l o a d e d separately. The
folder M a p l e o n t h e C D c o n t a i n s t h e one M a p l e file m e n t i o n e d a b o v e .
T h e m a n u s c r i p t for t h i s b o o k a n d some o f i t s e a r l i e r i n c a r n a t i o n s h a v e b e e n
used i n g r a d u a t e l e v e l courses a n d s e m i n a r s a t N o r t h w e s t e r n U n i v e r s i t y , V a n d e r b i l t
U n i v e r s i t y , a n d t h e I l l i n o i s I n s t i t u t e o f T e c h n o l o g y . S p e c i a l t h a n k s are d u e t o J o n

1
M A T L A B is a trademark of T h e MathWorks, Inc. and is used with permission. T h e Math-
Works does not warrant the accuracy of the text or exercises in this book. T h i s book's use or
discussion of M A T L A B software or related products does not constitute endorsement or sponsor-
ship by T h e MathWorks of a particular pedagogical approach or particular use of the M A T L A B
software.
2
M a p l e is a registered trademark of Waterloo Maple Inc.
 Preface ix

C h e r r i e , J o h n E r i c k s o n , P a r i t o s h M o k h a s i , L a r r y S c h u m a k e r , a n d J a c k Z h a n g for
reading various p o r t i o n s of t h e m a n u s c r i p t a n d / o r M A T L A B code a n d p r o v i d i n g
h e l p f u l feedback. F i n a l l y , t h a n k s are d u e t o a l l t h e p e o p l e a t W o r l d Scientific
P u b l i s h i n g C o . w h o h e l p e d m a k e t h i s p r o j e c t a success: R a j e s h B a b u , Y i n g O i
Chiew, L i n d a K w a n , R o k T i n g Tan, and Yubing Zhai.

Greg Fasshauer
Chicago, I L , J a n u a r y 2007
 Contents

Preface vii

1. Introduction 1
s
1.1 M o t i v a t i o n : Scattered D a t a I n t e r p o l a t i o n i n K 2
1.1.1 The Scattered D a t a I n t e r p o l a t i o n P r o b l e m 2
1.1.2 Example: I n t e r p o l a t i o n w i t h Distance Matrices 4
1.2 Some H i s t o r i c a l R e m a r k s 13

2. R a d i a l Basis F u n c t i o n I n t e r p o l a t i o n i n M A T L A B 17

2.1 R a d i a l (Basis) F u n c t i o n s 17
2.2 R a d i a l Basis F u n c t i o n I n t e r p o l a t i o n 19

3. Positive Definite Functions 27

3.1 Positive Definite Matrices and Functions 27

3.2 I n t e g r a l C h a r a c t e r i z a t i o n s for ( S t r i c t l y ) P o s i t i v e D e f i n i t e
Functions 31
3.2.1 Bochner's Theorem 31
3.2.2 Extensions t o Strictly Positive Definite Functions 32
3.3 Positive Definite Radial Functions 33

4. Examples of Strictly Positive Definite Radial Functions 37

4.1 E x a m p l e 1: G a u s s i a n s 37
4.2 E x a m p l e 2: L a g u e r r e - G a u s s i a n s 38
4.3 E x a m p l e 3: P o i s s o n R a d i a l F u n c t i o n s 39
4.4 E x a m p l e 4: M a t e r n F u n c t i o n s 41
4.5 E x a m p l e 5: G e n e r a l i z e d Inverse M u l t i q u a d r i c s 41
4.6 E x a m p l e 6: T r u n c a t e d P o w e r F u n c t i o n s 42
4.7 E x a m p l e 7: P o t e n t i a l s a n d W h i t t a k e r R a d i a l F u n c t i o n s 43
4.8 E x a m p l e 8: I n t e g r a t i o n A g a i n s t S t r i c t l y P o s i t i v e
Definite Kernels 45

xi
xii Meshfree Approximation Methods with MATLAB

4.9 Summary 45

5. Completely Monotone and M u l t i p l y Monotone Functions 47

5.1 Completely Monotone Functions 47

5.2 M u l t i p l y Monotone Functions 49

6. Scattered D a t a I n t e r p o l a t i o n w i t h P o l y n o m i a l Precision 53

6.1 Interpolation w i t h Multivariate Polynomials 53

6.2 Example: Reproduction of Linear Functions Using
Gaussian R B F s 55
6.3 Scattered D a t a Interpolation w i t h M o r e General
Polynomial Precision 57
6.4 Conditionally Positive Definite Matrices and Reproduction
of Constant Functions 59

7. Conditionally Positive Definite Functions 63

7.1 Conditionally Positive Definite Functions Defined 63

7.2 C o n d i t i o n a l l y Positive Definite Functions and Generalized
Fourier Transforms 65

8. Examples of Conditionally Positive Definite Functions 67

8.1 E x a m p l e 1: G e n e r a l i z e d M u l t i q u a d r i c s 67
8.2 E x a m p l e 2: R a d i a l P o w e r s 69
8.3 E x a m p l e 3: T h i n P l a t e Splines 70

9. Conditionally Positive Definite Radial Functions 73

9.1 Conditionally Positive Definite R a d i a l Functions and Completely

Monotone Functions 73
9.2 Conditionally Positive Definite R a d i a l Functions and
M u l t i p l y Monotone Functions 75
9.3 Some S p e c i a l P r o p e r t i e s o f C o n d i t i o n a l l y P o s i t i v e D e f i n i t e
Functions of Order One 76

10. Miscellaneous T h e o r y : Other N o r m s a n d Scattered D a t a F i t t i n g

on Manifolds 79

10.1 C o n d i t i o n a l l y P o s i t i v e D e f i n i t e F u n c t i o n s a n d p-Norms 79
10.2 Scattered D a t a F i t t i n g o n Manifolds 83
10.3 Remarks 83

11. C o m p a c t l y S u p p o r t e d R a d i a l Basis F u n c t i o n s 85

11.1 O p e r a t o r s for R a d i a l F u n c t i o n s a n d D i m e n s i o n W a l k s 85
11.2 Wendland's Compactly Supported Functions 87
 Contents xiii

11.3 Wu's Compactly Supported Functions 88

11.4 Oscillatory Compactly Supported Functions 90
11.5 O t h e r C o m p a c t l y S u p p o r t e d R a d i a l Basis Functions 92

12. Interpolation w i t h Compactly Supported RBFs in M A T L A B 95

12.1 A s s e m b l y o f t h e Sparse I n t e r p o l a t i o n M a t r i x 95
12.2 Numerical Experiments w i t h CSRBFs 99

13. R e p r o d u c i n g K e r n e l H i l b e r t Spaces a n d N a t i v e Spaces for

Strictly Positive Definite Functions 103

13.1 R e p r o d u c i n g K e r n e l H i l b e r t Spaces 103

13.2 N a t i v e Spaces for S t r i c t l y P o s i t i v e D e f i n i t e F u n c t i o n s 105
13.3 E x a m p l e s o f N a t i v e Spaces for P o p u l a r R a d i a l B a s i c F u n c t i o n s . . 108

14. T h e P o w e r F u n c t i o n a n d N a t i v e Space E r r o r E s t i m a t e s 111

14.1 F i l l Distance and A p p r o x i m a t i o n Orders I l l

14.2 Lagrange F o r m of the Interpolant and Cardinal
Basis Functions 112
14.3 T h e Power F u n c t i o n 115
14.4 G e n e r i c E r r o r E s t i m a t e s for F u n c t i o n s i n Af$(l) 117
14.5 E r r o r Estimates i n Terms of the F i l l Distance 119

15. Refined and I m p r o v e d E r r o r B o u n d s 125

15.1 N a t i v e Space E r r o r B o u n d s f o r Specific B a s i s F u n c t i o n s 125

15.1.1 I n f i n i t e l y S m o o t h Basis Functions 125
15.1.2 Basis F u n c t i o n s w i t h F i n i t e Smoothness 126
15.2 I m p r o v e m e n t s for N a t i v e Space E r r o r B o u n d s 127
15.3 E r r o r B o u n d s for F u n c t i o n s O u t s i d e t h e N a t i v e Space 128
15.4 E r r o r B o u n d s for S t a t i o n a r y A p p r o x i m a t i o n 130
15.5 Convergence w i t h Respect t o the Shape P a r a m e t e r 132
15.6 P o l y n o m i a l I n t e r p o l a t i o n as t h e L i m i t o f R B F I n t e r p o l a t i o n . . . 133

16. Stability and Trade-Off Principles 135

16.1 S t a b i l i t y and C o n d i t i o n i n g o f R a d i a l Basis F u n c t i o n I n t e r p o l a n t s . 135

16.2 Trade-Off Principle I : A c c u r a c y vs. S t a b i l i t y 138
16.3 T r a d e - O f f P r i n c i p l e I I : A c c u r a c y a n d S t a b i l i t y vs. P r o b l e m Size . 140
16.4 T r a d e - O f f P r i n c i p l e I I I : A c c u r a c y vs. E f f i c i e n c y 140

17. N u m e r i c a l E v i d e n c e for A p p r o x i m a t i o n O r d e r R e s u l t s 141

17.1 I n t e r p o l a t i o n for e > 0 141

17.1.1 C h o o s i n g a G o o d Shape P a r a m e t e r v i a T r i a l a n d E r r o r . . 142
xiv Meshfree Approximation Methods with MATLAB

17.1.2 T h e P o w e r F u n c t i o n as I n d i c a t o r for a G o o d S h a p e
Parameter 142
17.1.3 Choosing a G o o d Shape P a r a m e t e r v i a Cross V a l i d a t i o n . 1 4 6
17.1.4 The Contour-Pade A l g o r i t h m 151
17.1.5 Summary 152
17.2 Non-stationary Interpolation 153
17.3 Stationary Interpolation 155

18. The O p t i m a l i t y of R B F Interpolation 159

18.1 T h e Connection to O p t i m a l Recovery 159

18.2 O r t h o g o n a l i t y i n R e p r o d u c i n g K e r n e l H i l b e r t Spaces 160
18.3 Optimality Theorem I 162
18.4 Optimality Theorem I I 163
18.5 Optimality Theorem I I I 164

19. L e a s t Squares R B F A p p r o x i m a t i o n w i t h M A T L A B 165

19.1 O p t i m a l Recovery Revisited 165

19.2 R e g u l a r i z e d L e a s t Squares A p p r o x i m a t i o n 166
19.3 L e a s t Squares A p p r o x i m a t i o n W h e n R B F C e n t e r s D i f f e r from
D a t a Sites 168
19.4 L e a s t Squares S m o o t h i n g o f N o i s y D a t a 170

20. T h e o r y for L e a s t Squares A p p r o x i m a t i o n 177

20.1 Well-Posedness o f R B F L e a s t Squares A p p r o x i m a t i o n 177

20.2 E r r o r B o u n d s for L e a s t Squares A p p r o x i m a t i o n 179

21. A d a p t i v e L e a s t Squares A p p r o x i m a t i o n 181

21.1 A d a p t i v e L e a s t Squares u s i n g K n o t I n s e r t i o n 181

21.2 A d a p t i v e L e a s t Squares u s i n g K n o t R e m o v a l 184
21.3 Some N u m e r i c a l E x a m p l e s 188

22. M o v i n g L e a s t Squares A p p r o x i m a t i o n 191

22.1 Discrete W e i g h t e d Least Squares A p p r o x i m a t i o n 191

22.2 Standard Interpretation of M L S A p p r o x i m a t i o n 192
22.3 The Backus-Gilbert Approach to M L S A p p r o x i m a t i o n 194
22.4 Equivalence of the T w o Formulations of M L S A p p r o x i m a t i o n . . . 198
22.5 D u a l i t y a n d B i - O r t h o g o n a l Bases 199
22.6 S t a n d a r d M L S A p p r o x i m a t i o n as a C o n s t r a i n e d Quadratic
Optimization Problem 202
22.7 Remarks 202

23. Examples of M L S Generating Functions 205

 Contents xv

23.1 Shepard's M e t h o d 205

23.2 M L S Approximation w i t h Nontrivial Polynomial Reproduction . . 207

24. MLS Approximation with M A T L A B 211

24.1 A p p r o x i m a t i o n w i t h Shepard's M e t h o d 211

24.2 M L S A p p r o x i m a t i o n w i t h Linear Reproduction 216
24.3 Plots of Basis-Dual Basis Pairs 222

25. E r r o r B o u n d s for M o v i n g L e a s t Squares A p p r o x i m a t i o n 225

25.1 A p p r o x i m a t i o n O r d e r o f M o v i n g Least Squares 225

26. A p p r o x i m a t e M o v i n g Least Squares A p p r o x i m a t i o n 229

26.1 High-order Shepard M e t h o d s via M o m e n t Conditions 229

26.2 Approximate Approximation .. . 230
26.3 C o n s t r u c t i o n o f G e n e r a t i n g F u n c t i o n s for A p p r o x i m a t e M L S
Approximation 232

27. N u m e r i c a l E x p e r i m e n t s for A p p r o x i m a t e M L S A p p r o x i m a t i o n 237

27.1 Univariate Experiments 237

27.2 Bivariate Experiments 241

28. Fast F o u r i e r T r a n s f o r m s 243

28.1 NFFT 243

28.2 A p p r o x i m a t e M L S A p p r o x i m a t i o n v i a N o n - u n i f o r m Fast Fourier
Transforms 245

29. Partition of Unity Methods 249

29.1 Theory 249

29.2 Partition of Unity Approximation w i t h M A T L A B 251

30. Approximation of Point Cloud D a t a i n 3D 255

30.1 A G e n e r a l A p p r o a c h v i a I m p l i c i t Surfaces 255

30.2 A n Illustration in 2D 257
30.3 A Simplistic Implementation i n 3 D via Partition of U n i t y
Approximation in M A T L A B 260

31. Fixed Level Residual Iteration 265

31.1 Iterative Refinement 265

31.2 Fixed Level Iteration 267
31.3 Modifications o f the Basic F i x e d Level I t e r a t i o n A l g o r i t h m . . . . 269
31.4 Iterated Approximate M L S Approximation i n M A T L A B 270
31.5 Iterated Shepard A p p r o x i m a t i o n 274
xvi Meshfree Approximation Methods with M A T L A B

32. Multilevel Iteration 277

32.1 Stationary Multilevel Interpolation 277

32.2 A M A T L A B Implementation of Stationary Multilevel
Interpolation 279
32.3 Stationary Multilevel A p p r o x i m a t i o n 283
32.4 Multilevel Interpolation w i t h Globally Supported RBFs 287

33. Adaptive Iteration 291

33.1 A Greedy Adaptive A l g o r i t h m 291

33.2 T h e Faul-Powell A l g o r i t h m 298

34. I m p r o v i n g the C o n d i t i o n N u m b e r of the I n t e r p o l a t i o n M a t r i x 303

34.1 Preconditioning: T w o Simple Examples 304

34.2 Early Preconditioners 305
34.3 Preconditioned G M R E S via A p p r o x i m a t e Cardinal Functions . . 309
34.4 Change o f Basis 311
34.5 Effect o f t h e " B e t t e r " B a s i s o n t h e C o n d i t i o n N u m b e r o f t h e
Interpolation M a t r i x 314
34.6 Effect o f t h e " B e t t e r " B a s i s o n t h e A c c u r a c y o f t h e I n t e r p o l a n t . 316

35. O t h e r Efficient N u m e r i c a l M e t h o d s 321

35.1 T h e Fast M u l t i p o l e M e t h o d 321

35.2 F a s t Tree C o d e s 327
35.3 Domain Decomposition 331

36. Generalized H e r m i t e I n t e r p o l a t i o n 333

36.1 T h e Generalized H e r m i t e I n t e r p o l a t i o n P r o b l e m 333

36.2 M o t i v a t i o n for t h e S y m m e t r i c F o r m u l a t i o n 335

37. R B F Hermite Interpolation in M A T L A B 339

38. Solving Elliptic Partial Differential Equations via R B F Collocation 345

38.1 Kansa's Approach 345

38.2 A n Hermite-based Approach 348
38.3 E r r o r B o u n d s for S y m m e t r i c C o l l o c a t i o n 349
38.4 O t h e r Issues 350

39. Non-Symmetric R B F Collocation in M A T L A B 353

39.1 Kansa's Non-Symmetric Collocation M e t h o d 353

40. Symmetric R B F Collocation i n M A T L A B 365

 Contents xvii

40.1 Symmetric Collocation M e t h o d 365

40.2 Summarizing Remarks on the Symmetric and Non-Symmetric
Collocation Methods 372

41. Collocation w i t h CSRBFs i n M A T L A B 375

41.1 Collocation w i t h Compactly Supported RBFs 375

41.2 Multilevel R B F Collocation 380

42. U s i n g R a d i a l Basis Functions i n Pseudospectral M o d e 387

42.1 Differentiation Matrices 388

42.2 P D E s w i t h B o u n d a r y Conditions via Pseudospectral Methods . . 390
42.3 A Non-Symmetric RBF-based Pseudospectral M e t h o d 391
42.4 A Symmetric RBF-based Pseudospectral M e t h o d 394
42.5 A Unified Discussion 396
42.6 Summary 398

43. RBF-PS Methods in M A T L A B 401

43.1 C o m p u t i n g the RBF-Differentiation M a t r i x i n M A T L A B . . . . . . 401

43.1.1 S o l u t i o n o f a 1-D T r a n s p o r t E q u a t i o n 403
43.2 Use o f t h e C o n t o u r - P a d e A l g o r i t h m w i t h t h e P S A p p r o a c h . . . . 405
43.2.1 Solution of the I D Transport Equation Revisited 405
43.3 C o m p u t a t i o n of Higher-Order Derivatives 407
43.3.1 Solution of the Allen-Cahn Equation 409
43.4 Solution of a 2D Helmholtz Equation 411
43.5 S o l u t i o n o f a 2 D Laplace E q u a t i o n w i t h Piecewise B o u n d a r y
Conditions 415
43.6 Summary 416

44. R B F Galerkin Methods 419

44.1 A n Elliptic P D E w i t h Neumann Boundary Conditions 419

44.2 A Convergence E s t i m a t e 420
44.3 A Multilevel R B F Galerkin Algorithm 421

45. R B F Galerkin Methods in M A T L A B 423

Appendix A Useful Facts f r o m Discrete M a t h e m a t i c s 427

A.l H a l t o n Points 427

A. 2 kd-Tvees 428

Appendix B U s e f u l Facts f r o m A n a l y s i s 431

B. l Some I m p o r t a n t C o n c e p t s f r o m M e a s u r e T h e o r y 431
B.2 A Brief S u m m a r y of Integral Transforms 432
xviii Meshfree Approximation Methods with MATLAB

B. 3 T h e S c h w a r t z Space a n d t h e G e n e r a l i z e d F o u r i e r T r a n s f o r m . . . 433

Appendix C Additional Computer Programs 435

C. l M A T L A B Programs 435
C. 2 Maple Programs 440

Appendix D Catalog of R B F s w i t h Derivatives 443

D. l Generic Derivatives 443

D.2 F o r m u l a s for Specific B a s i c F u n c t i o n s 444
D.2.1 Globally Supported, Strictly Positive Definite Functions . 444
D.2.2 Globally Supported, Strictly Conditionally Positive
Definite Functions of Order 1 445
D.2.3 Globally Supported, Strictly Conditionally Positive
Definite Functions o f Order 2 446
D.2.4 Globally Supported, Strictly Conditionally Positive
Definite Functions o f Order 3 446
D.2.5 Globally Supported, Strictly Conditionally Positive
Definite Functions of Order 4 447
D.2.6 Globally Supported, Strictly Positive Definite and
Oscillatory Functions 447
D.2.7 Compactly Supported, Strictly Positive Definite
Functions 448

Bibliography 451

Index 491
 Chapter 1

Introduction

Meshfree m e t h o d s have g a i n e d m u c h a t t e n t i o n i n recent years, n o t o n l y i n t h e

m a t h e m a t i c s b u t also i n t h e e n g i n e e r i n g c o m m u n i t y . T h u s , m u c h o f t h e w o r k c o n -
cerned w i t h meshfree a p p r o x i m a t i o n m e t h o d s is i n t e r d i s c i p l i n a r y a t t h e interface
b e t w e e n m a t h e m a t i c s a n d n u m e r o u s a p p l i c a t i o n areas (see t h e p a r t i a l l i s t b e l o w ) .
M o r e o v e r , c o m p u t a t i o n w i t h h i g h - d i m e n s i o n a l d a t a is a n i m p o r t a n t issue i n m a n y
areas o f science a n d e n g i n e e r i n g . M a n y t r a d i t i o n a l n u m e r i c a l m e t h o d s c a n e i t h e r
n o t h a n d l e such p r o b l e m s a t a l l , o r are l i m i t e d t o v e r y special ( r e g u l a r ) s i t u a t i o n s .
Meshfree m e t h o d s are o f t e n b e t t e r s u i t e d t o cope w i t h changes i n t h e g e o m e t r y
o f t h e d o m a i n o f i n t e r e s t (e.g., free surfaces a n d large d e f o r m a t i o n s ) t h a n classical
d i s c r e t i z a t i o n techniques s u c h as finite differences, finite elements o r finite v o l u m e s .
A n o t h e r o b v i o u s a d v a n t a g e o f meshfree d i s c r e t i z a t i o n s is o f course t h e i r i n -
dependence f r o m a mesh. M e s h g e n e r a t i o n is s t i l l t h e m o s t t i m e c o n s u m i n g p a r t
o f a n y mesh-based n u m e r i c a l s i m u l a t i o n . Since meshfree d i s c r e t i z a t i o n t e c h n i q u e s
are based o n l y o n a set o f i n d e p e n d e n t p o i n t s , these costs o f m e s h g e n e r a t i o n are
eliminated. Meshfree a p p r o x i m a t i o n m e t h o d s c a n be seen t o p r o v i d e a n e w gen-
e r a t i o n o f n u m e r i c a l t o o l s . O t h e r t r a d i t i o n a l n u m e r i c a l m e t h o d s such as t h e finite
element, finite difference o r finite v o l u m e m e t h o d s are u s u a l l y l i m i t e d t o p r o b l e m s
i n v o l v i n g t w o o r t h r e e p a r a m e t e r s (space d i m e n s i o n s ) . H o w e v e r , i n m a n y a p p l i c a -
t i o n s t h e n u m b e r o f p a r a m e t e r s c a n easily r a n g e i n t h e h u n d r e d s o r even t h o u s a n d s .
M u l t i v a r i a t e a p p r o x i m a t i o n m e t h o d s present one w a y t o address these issues.
A p p l i c a t i o n s o f meshfree m e t h o d s c a n be f o u n d

i n m a n y different areas o f science a n d e n g i n e e r i n g v i a scattered data mod-

eling (e.g., fitting o f p o t e n t i a l energy surfaces i n c h e m i s t r y ; c o u p l i n g o f
e n g i n e e r i n g m o d e l s w i t h sets o f i n c o m p a t i b l e p a r a m e t e r s ; mapping prob-
lems i n geodesy, geophysics, m e t e o r o l o g y ) ;
i n m a n y different areas o f science a n d e n g i n e e r i n g v i a solution of partial
differential equations (e.g., s o l u t i o n o f gas d y n a m i c s e q u a t i o n s , B o l t z m a n n
a n d F o k k e r - P l a n c k e q u a t i o n s i n s i x - d i m e n s i o n a l phase space; p r o b l e m s i n -
v o l v i n g m o v i n g d i s c o n t i n u i t i e s such as cracks a n d shocks, m u l t i - s c a l e resolu-
t i o n , large m a t e r i a l d i s t o r t i o n s ; e l a s t i c i t y studies i n p l a t e a n d s h e l l b e n d i n g

l
2 Meshfree Approximation Methods with MATLAB

problems; applications i n nanotechnology);

i n non-uniform sampling (e.g., medical imaging, tomographic reconstruc-
tion);
i n mathematical finance (e.g., option pricing);
i n computer graphics (e.g., r e p r e s e n t a t i o n o f surfaces f r o m p o i n t i n f o r m a t i o n
such as laser r a n g e scan d a t a , i m a g e w a r p i n g ) ;
i n learning theory, neural networks a n d data mining (e.g., kernel approxi-
m a t i o n , support vector machines);
in optimization.

Since m a n y o f these a p p l i c a t i o n s e i t h e r c o m e d o w n t o a f u n c t i o n a p p r o x i m a t i o n
p r o b l e m , o r i n c l u d e f u n c t i o n a p p r o x i m a t i o n as a f u n d a m e n t a l c o m p o n e n t , we w i l l
b e g i n o u r discussion w i t h a n d i n fact base a l a r g e p a r t o f t h e c o n t e n t s o f t h i s
book on the multivariate scattered data i n t e r p o l a t i o n problem.

s
1.1 Motivation: Scattered Data Interpolation in M

W e w i l l n o w describe t h e g e n e r a l process o f s c a t t e r e d d a t a f i t t i n g , w h i c h is one o f

t h e f u n d a m e n t a l p r o b l e m s i n a p p r o x i m a t i o n t h e o r y a n d d a t a m o d e l i n g i n general.
O u r desire t o have a w e l l - p o s e d p r o b l e m f o r m u l a t i o n w i l l n a t u r a l l y l e a d t o a n i n -
t r o d u c t o r y e x a m p l e based o n t h e use o f so-called distance matrices. I n the next
c h a p t e r s we w i l l generalize t h i s a p p r o a c h b y i n t r o d u c i n g t h e c o n c e p t o f a r a d i a l
basis f u n c t i o n .

1.1.1 The Scattered Data Interpolation Problem

I n m a n y scientific d i s c i p l i n e s one faces t h e f o l l o w i n g p r o b l e m : W e are g i v e n a set o f

d a t a ( m e a s u r e m e n t s , a n d l o c a t i o n s a t w h i c h these m e a s u r e m e n t s w e r e o b t a i n e d ) ,
a n d we w a n t t o f i n d a r u l e w h i c h a l l o w s us t o deduce i n f o r m a t i o n a b o u t t h e process
we are s t u d y i n g also at l o c a t i o n s different f r o m t h o s e a t w h i c h we o b t a i n e d o u r
m e a s u r e m e n t s . T h u s , we are t r y i n g t o f i n d a f u n c t i o n Vf w h i c h is a " g o o d " f i t t o
t h e g i v e n d a t a . T h e r e are m a n y w a y s t o decide w h a t w e m e a n b y " g o o d " , a n d t h e
o n l y c r i t e r i o n we w i l l consider n o w is t h a t w e w a n t t h e f u n c t i o n Vf to e x a c t l y m a t c h
the given measurements at the corresponding locations. T h i s a p p r o a c h is c a l l e d
interpolation, a n d i f t h e l o c a t i o n s at w h i c h t h e m e a s u r e m e n t s are t a k e n do n o t lie
o n a u n i f o r m o r r e g u l a r g r i d , t h e n t h e process is c a l l e d scattered data interpolation.
T o give a precise d e f i n i t i o n we assume t h a t t h e m e a s u r e m e n t l o c a t i o n s (or data
sites) are l a b e l e d xj, j = 1 , . . . , j V , a n d t h e c o r r e s p o n d i n g m e a s u r e m e n t s (or data
values) are c a l l e d yj. W e w i l l use X t o d e n o t e t h e set o f d a t a sites a n d assume
s
t h a t X C fl for some r e g i o n O i n R . T h r o u g h o u t t h i s b o o k we w i l l r e s t r i c t o u r
discussion t o s c a l a r - v a l u e d d a t a , i.e., V j e R. However, m u c h of the following can
be g e n e r a l i z e d easily t o p r o b l e m s w i t h v e c t o r - v a l u e d d a t a . I n m a n y of our later

i . .ft
 1. Introduction 3

discussions we w i l l assume t h a t t h e d a t a are o b t a i n e d b y s a m p l i n g some ( u n k n o w n )

f u n c t i o n / at t h e d a t a sites, i.e., yj = f(xj), j = 1,... ,N. O u r n o t a t i o n Vf for t h e
i n t e r p o l a t i n g f u n c t i o n emphasizes t h e c o n n e c t i o n b e t w e e n t h e i n t e r p o l a n t a n d t h e
d a t a f u n c t i o n / . W e are n o w r e a d y for a precise f o r m u l a t i o n o f t h e s c a t t e r e d d a t a
interpolation problem.

P r o b l e m 1.1 ( S c a t t e r e d D a t a I n t e r p o l a t i o n ) . Given data (xj,yj), j =

s
1,...,N, with xj e R ; yj G R, find a (continuous) function Vf such that
x
^f( o) = yj, j = i,...,N.
s
T h e fact t h a t we a l l o w Xj t o l i e i n a n a r b i t r a r y s - d i m e n s i o n a l space R means
t h a t t h e f o r m u l a t i o n o f P r o b l e m 1.1 allows us t o cover m a n y different t y p e s o f ap-
plications. I f s = 1 t h e d a t a c o u l d , e.g., b e a series o f m e a s u r e m e n t s t a k e n over
a c e r t a i n t i m e p e r i o d , t h u s t h e " d a t a sites" Xj w o u l d c o r r e s p o n d t o c e r t a i n t i m e
instances. F o r s = 2 we c a n t h i n k o f t h e d a t a b e i n g o b t a i n e d over a p l a n a r r e g i o n ,
a n d so xj c o r r e s p o n d s t o t h e t w o c o o r d i n a t e s i n t h e p l a n e . F o r instance, w e m i g h t
w a n t t o p r o d u c e a m a p t h a t shows t h e r a i n f a l l i n t h e s t a t e we l i v e i n based o n t h e
d a t a c o l l e c t e d at w e a t h e r s t a t i o n s l o c a t e d t h r o u g h o u t t h e s t a t e . F o r s 3 w e m i g h t
t h i n k o f a s i m i l a r s i t u a t i o n i n space. O n e p o s s i b i l i t y is t h a t w e c o u l d be i n t e r e s t e d
i n t h e t e m p e r a t u r e d i s t r i b u t i o n i n s i d e some s o l i d b o d y . H i g h e r - d i m e n s i o n a l e x a m -
ples m i g h t n o t be t h a t i n t u i t i v e , b u t a m u l t i t u d e o f t h e m e x i s t , e.g., i n finance,
o p t i m i z a t i o n , economics or s t a t i s t i c s , b u t also i n a r t i f i c i a l i n t e l l i g e n c e or l e a r n i n g
theory.
A c o n v e n i e n t a n d c o m m o n a p p r o a c h t o s o l v i n g t h e s c a t t e r e d d a t a p r o b l e m is t o
m a k e t h e a s s u m p t i o n t h a t t h e f u n c t i o n Vf is a l i n e a r c o m b i n a t i o n o f c e r t a i n basis
functions B,k i.e.,
N
s
Vf{x) = c B (x),
k k x e R . (1.1)
A;=l
S o l v i n g t h e i n t e r p o l a t i o n p r o b l e m u n d e r t h i s a s s u m p t i o n leads t o a s y s t e m o f
linear equations of the f o r m
Ac = y,
w h e r e t h e entries o f t h e interpolation matrix A are g i v e n b y Ajk = Bk(xj), j ,k =
T T
1,...,N, c= [a,... ,c ] ,
N a n d y = [y ,... 1 ,y ] .
N

P r o b l e m 1.1 w i l l be well-posed, i.e., a s o l u t i o n t o t h e p r o b l e m w i l l exist a n d be

u n i q u e , i f a n d o n l y i f t h e m a t r i x A is n o n - s i n g u l a r .
I n t h e u n i v a r i a t e s e t t i n g i t is w e l l k n o w n t h a t one c a n i n t e r p o l a t e t o a r b i t r a r y
d a t a at N d i s t i n c t d a t a sites u s i n g a p o l y n o m i a l o f degree Nl. For the m u l t i v a r i a t e
s e t t i n g , however, t h e r e is t h e f o l l o w i n g n e g a t i v e r e s u l t (see [ M a i r h u b e r ( 1 9 5 6 ) ; C u r t i s
(1959)]).

T h e o r e m 1.1 ( M a i r h u b e r - C u r t i s ) . If Q, C R, s
s > 2 , contains an interior
point, then there exist no Haar spaces of continuous functions except for one-
dimensional ones.
4 Meshfree Approximation Methods with MATLAB

I n o r d e r t o u n d e r s t a n d t h i s t h e o r e m w e need

D e f i n i t i o n 1.1. L e t t h e f i n i t e - d i m e n s i o n a l l i n e a r f u n c t i o n space B C C ( f 2 ) have a

basis {Bi,..., Bjy}. T h e n B is a Haar space o n fl i f
det A ^ O
for a n y set o f d i s t i n c t X i , . . . , CCJV i n f2. H e r e A is t h e m a t r i x w i t h entries Ajk =
B ( ).
k Xj

N o t e t h a t existence o f a H a a r space g u a r a n t e e s i n v e r t i b i l i t y o f t h e i n t e r p o l a t i o n
m a t r i x A, i.e., existence a n d uniqueness o f a n i n t e r p o l a n t o f t h e f o r m (1.1) t o
d a t a specified a t XI,...,XN f r o m t h e space B. A s m e n t i o n e d above, u n i v a r i a t e
p o l y n o m i a l s o f degree N 1 f o r m a n A ^ - d i m e n s i o n a l H a a r space for d a t a g i v e n a t
X i , . . . , X N .

T h e M a i r h u b e r - C u r t i s t h e o r e m t e l l s us t h a t i f w e w a n t t o have a w e l l - p o s e d
m u l t i v a r i a t e scattered d a t a i n t e r p o l a t i o n p r o b l e m we can no longer fix i n advance
t h e set o f basis f u n c t i o n s w e p l a n t o use for i n t e r p o l a t i o n o f a r b i t r a r y s c a t t e r e d d a t a .
For e x a m p l e , i t is n o t possible t o p e r f o r m u n i q u e i n t e r p o l a t i o n w i t h ( m u l t i v a r i a t e )
2
p o l y n o m i a l s o f degree N t o d a t a g i v e n a t a r b i t r a r y l o c a t i o n s i n M . Instead, the
basis s h o u l d d e p e n d o n t h e d a t a l o c a t i o n s . W e w i l l g i v e a s i m p l e e x a m p l e o f s u c h
a n i n t e r p o l a t i o n scheme i n t h e n e x t s u b s e c t i o n .

Proof. [of T h e o r e m 1.1] L e t s > 2 a n d assume t h a t B is a H a a r space w i t h basis

{Bi,..., B N } w i t h N > 2. W e need t o s h o w t h a t t h i s leads t o a c o n t r a d i c t i o n .
W e l e t x\,..., XN be a set o f d i s t i n c t p o i n t s i n ft C M s
and A the m a t r i x w i t h
entries Ajk = Bk(xj), j , k = 1 , . . . , N. T h e n , b y t h e d e f i n i t i o n o f a H a a r space, w e
have
det A ^ O . (1.2)
Now, consider a closed p a t h P i n f2 c o n n e c t i n g o n l y x a n d X2- T h i s is p o s s i b l e
since b y a s s u m p t i o n fl c o n t a i n s a n i n t e r i o r p o i n t . W e c a n exchange the
p o s i t i o n s o f X \ a n d X2 b y m o v i n g t h e m c o n t i n u o u s l y a l o n g t h e p a t h P (without
interfering w i t h any of the other X j ) . T h i s means, however, t h a t rows 1 a n d 2 o f
t h e d e t e r m i n a n t ( 1 . 2 ) have been e x c h a n g e d , a n d so t h e d e t e r m i n a n t has changed
sign.
Since t h e d e t e r m i n a n t is a c o n t i n u o u s f u n c t i o n o f X \ a n d X2 w e m u s t have h a d
d e t = 0 a t some p o i n t a l o n g P. T h i s c o n t r a d i c t s ( 1 . 2 ) .

1.1.2 Example: Interpolation with Distance Matrices

In order t o o b t a i n data dependent a p p r o x i m a t i o n spaces, as suggested by the

M a i r h u b e r - C u r t i s t h e o r e m w e n o w consider a s i m p l e e x a m p l e . A s a " t e s t f u n c t i o n "
we e m p l o y t h e f u n c t i o n

f (x)
a
s
= 4 ]^[a:d(l - x d ) , X = ( X ! , . . . , x a ) e [o, i ] .s

d=l
 1. Introduction 5

S
T h i s f u n c t i o n is zero o n t h e b o u n d a r y o f t h e u n i t c u b e i n R a n d has a m a x i m u m
v a l u e o f one at t h e center o f t h e c u b e . A s i m p l e M A T L A B s c r i p t d e f i n i n g f s is g i v e n
as P r o g r a m C . l i n A p p e n d i x C .
We w i l l use a set o f u n i f o r m l y s c a t t e r e d d a t a sites i n t h e u n i t c u b e a t w h i c h
w e sample o u r t e s t f u n c t i o n f . T h i s w i l l be a c c o m p l i s h e d here ( a n d i n m a n y o t h e r
s

examples l a t e r o n ) b y r e s o r t i n g t o t h e so-called Halton points. T h e s e are u n i f o r m l y

s
d i s t r i b u t e d r a n d o m p o i n t s i n (0, l ) . A set o f 289 H a l t o n p o i n t s i n t h e u n i t s q u a r e
2
i n M. is s h o w n i n F i g u r e 1.1. M o r e d e t a i l s o n H a l t o n p o i n t s are p r e s e n t e d i n
A p p e n d i x A . I n our c o m p u t a t i o n a l experiments we generate H a l t o n points using
the program h a l t o n s e q . m w r i t t e n by Daniel Dougherty. T h i s function can be
d o w n l o a d e d f r o m t h e M A T L A B C e n t r a l F i l e E x c h a n g e (see [ M C F E ] ) .

2
Fig. 1.1 289 Halton points in the unit square in R .

A s e x p l a i n e d i n t h e p r e v i o u s s u b s e c t i o n we are i n t e r e s t e d i n c o n s t r u c t i n g a ( c o n -
t i n u o u s ) f u n c t i o n Vf t h a t interpolates t h e samples o b t a i n e d f r o m f s a t t h e set o f
H a l t o n p o i n t s , i.e., s u c h t h a t
Vf(xj) f (xj),
s Xj a Halton point.
A s p o i n t e d o u t above, i f s = 1, t h e n t h i s p r o b l e m is o f t e n s o l v e d u s i n g u n i v a r i a t e
p o l y n o m i a l s or splines. F o r a s m a l l n u m b e r o f d a t a sites p o l y n o m i a l s m a y w o r k
s a t i s f a c t o r i l y . H o w e v e r , i f t h e n u m b e r o f p o i n t s increases, i.e., t h e p o l y n o m i a l degree
g r o w s , t h e n i t is w e l l k n o w n t h a t one s h o u l d use splines ( o r piecewise p o l y n o m i a l s )
t o a v o i d o s c i l l a t i o n s . T h e s i m p l e s t s o l u t i o n is t o use a c o n t i n u o u s piecewise l i n e a r
spline, i.e., t o " c o n n e c t t h e d o t s " . I t is also w e l l k n o w n t h a t one possible basis f o r
t h e space o f piecewise l i n e a r splines i n t e r p o l a t i n g d a t a at a g i v e n set o f p o i n t s i n
[0,1] consists o f t h e shifts o f t h e a b s o l u t e v a l u e f u n c t i o n t o t h e d a t a sites. I n o t h e r
w o r d s , we c a n c o n s t r u c t t h e piecewise l i n e a r s p l i n e i n t e r p o l a n t b y a s s u m i n g Vf is
of the form
N
V (x)
f = ^Tc \x k - x \,
k are [0,1],
fc=i
6 Meshfree Approximation Methods with MATLAB

a n d t h e n d e t e r m i n e t h e coefficients c k by satisfying the interpolation conditions

V {xj)
f = h{ ), Xj j = l,...,N.

C l e a r l y , t h e basis f u n c t i o n s B k = |- x \ k are d e p e n d e n t o n t h e d a t a sites as suggested

by the Mairhuber-Curtis theorem. T h e points x k t o w h i c h t h e basic f u n c t i o n B(x)
\x\ is s h i f t e d are u s u a l l y referred t o as centers. W h i l e t h e r e m a y be circumstances
t h a t suggest c h o o s i n g these centers d i f f e r e n t f r o m t h e d a t a sites one g e n e r a l l y p i c k s
t h e centers t o c o i n c i d e w i t h t h e d a t a sites. T h i s simplifies the analysis of the
m e t h o d , a n d is sufficient for m a n y a p p l i c a t i o n s . Since t h e f u n c t i o n s B k are ( r a d i a l l y )
s y m m e t r i c a b o u t t h e i r centers x k t h i s c o n s t i t u t e s t h e first e x a m p l e o f radial basis
functions. We w i l l formally introduce the n o t i o n of a radial function i n the next
chapter.
O f course, one c a n i m a g i n e m a n y o t h e r w a y s t o c o n s t r u c t a n i V - d i m e n s i o n a l
d a t a - d e p e n d e n t basis for t h e p u r p o s e o f s c a t t e r e d d a t a i n t e r p o l a t i o n . H o w e v e r , t h e
use o f shifts o f one single basic f u n c t i o n m a k e s t h e r a d i a l basis f u n c t i o n a p p r o a c h
p a r t i c u l a r l y elegant.
N o t e t h a t we d i s t i n g u i s h b e t w e e n basis f u n c t i o n s B k a n d t h e basic f u n c t i o n B.
W e use t h i s t e r m i n o l o g y t o e m p h a s i z e t h a t t h e r e is one basic f u n c t i o n B which
generates t h e basis v i a shifts t o t h e v a r i o u s centers.
C o m i n g b a c k t o t h e s c a t t e r e d d a t a p r o b l e m , w e find t h e coefficients c k by solving
the linear system

\xi - X i | X2\ \Xi - X\

N ' Cl " "/l(*l)~
|x - Xi\ \X2 X2\ \X2 X\ /l(*2)

N
2

(1.3)

\X N - ml \X N - X2\ \x N - XN -CN _

A s m e n t i o n e d earlier, for h i g h e r space d i m e n s i o n s s s u c h a d a t a d e p e n d e n t basis

is r e q u i r e d . T h u s , even t h o u g h t h e c o n s t r u c t i o n o f piecewise l i n e a r splines i n h i g h e r
space d i m e n s i o n s is a different one ( t h e y are closely a s s o c i a t e d w i t h a n u n d e r l y i n g
c o m p u t a t i o n a l m e s h ) , t h e idea j u s t p r e s e n t e d suggests a v e r y s i m p l e g e n e r a l i z a t i o n
o f u n i v a r i a t e piecewise l i n e a r splines t h a t w o r k s for a n y space d i m e n s i o n .
T h e m a t r i x i n (1.3) above is a n e x a m p l e o f a distance matrix. S u c h m a t r i c e s have
been s t u d i e d i n g e o m e t r y a n d a n a l y s i s i n t h e c o n t e x t o f i s o m e t r i c e m b e d d i n g s o f
m e t r i c spaces for a l o n g t i m e (see, e.g., [ B a x t e r ( 1 9 9 1 ) ; B l u m e n t h a l ( 1 9 3 8 ) ; B o c h n e r
(1941); M i c c h e l l i ( 1 9 8 6 ) ; S c h o e n b e r g ( 1 9 3 8 a ) ; W e l l s a n d W i l l i a m s (1975)] a n d also
C h a p t e r 10). I t is k n o w n t h a t t h e d i s t a n c e m a t r i x based o n t h e E u c l i d e a n d i s t a n c e
S
b e t w e e n a set o f d i s t i n c t p o i n t s i n R is a l w a y s n o n - s i n g u l a r (see S e c t i o n 9.3 for
m o r e d e t a i l s ) . T h e r e f o r e , we c a n solve t h e s c a t t e r e d d a t a i n t e r p o l a t i o n p r o b l e m we
s
posed o n [0, l ] b y assuming

Vf( ) x
= Y2 W Ck x
~ x
k h , xe[0,iy (1.4)
fc=i
 1. Introduction 7

a n d t h e n d e t e r m i n e t h e coefficients c k by solving the linear system

1*1 Xi\\2 11*1 352112 1*1 - X \\

N 2 Cl /s(*l)
\x ~
2 Xi\\ 2 11*2 *2|| 2 \x 2 X \\
N 2 C2 fs{x2)

\XN - * 1 2 \\XN - X2\\2 \XN N\\2.

X
JS(XN)

T h i s is precisely t h e i n t e r p o l a t i o n m e t h o d w e w i l l choose t o i l l u s t r a t e w i t h o u r first

M A T L A B s c r i p t D i s t a n c e M a t r i x F i t .m (see P r o g r a m 1.2 b e l o w ) a n d t h e s u p p o r t i n g
figures a n d t a b l e s . A t y p i c a l basis f u n c t i o n for t h e E u c l i d e a n d i s t a n c e m a t r i x fit,
B {x)
k \\x Efc||2, is s h o w n i n F i g u r e 1.2 for t h e case xk 0 a n d s = 2.

Fig. 1.2 A typical basis function for the Euclidean distance matrix centered at the origin in R .

Before we discuss t h e a c t u a l i n t e r p o l a t i o n p r o g r a m we f i r s t l i s t a subroutine

used i n m a n y o f o u r l a t e r e x a m p l e s . I t is c a l l e d D i s t a n c e M a t r i x . m a n d w e use i t
t o compute the m a t r i x o f pairwise E u c l i d e a n distances o f t w o (possibly different)
s
sets o f p o i n t s i n M . I n t h e c o d e these t w o sets are d e n o t e d b y d s i t e s a n d c t r s . I n
m o s t o f o u r e x a m p l e s b o t h o f these sets w i l l c o i n c i d e w i t h t h e set X o f d a t a sites.

Program 1.1. D i s t a n c e M a t r i x . m

% DM = DistanceMatrix(dsites,ctrs)
% Forms t h e d i s t a n c e m a t r i x of two s e t s o f p o i n t s in R"s,
% i.e., DM(i,j) = II datasite_i - center_j I I_2.
% Input
% dsites: Mxs m a t r i x r e p r e s e n t i n g a set of M data s i t e s in R~s
7, (i.e., each row c o n t a i n s one s-dimensional point)
7o ctrs: Nxs m a t r i x r e p r e s e n t i n g a set of N centers in R~s
7o (one center per row)
7. O u t p u t
7. DM: MxN m a t r i x w h o s e i , j p o s i t i o n c o n t a i n s the Euclidean
7o d i s t a n c e between the i - t h data site and j - t h c e n t e r
8 Meshfree Approximation Methods with MATLAB

1 f u n c t i o n DM = D i s t a n c e M a t r i x ( d s i t e s , c t r s )
2 [M,s] = s i z e ( d s i t e s ) ; [N,s] = size(ctrs);
3 DM = zeros(M,N);
% Accumulate sum of s q u a r e s of c o o r d i n a t e differences
% The n d g r i d command produces two MxN matrices:
'/ . d r , c o n s i s t i n g of N i d e n t i c a l columns ( e a c h containing
% the d-th coordinate of t h e M d a t a sites)
7o c c , c o n s i s t i n g of M i d e n t i c a l rows ( e a c h containing
% the d-th coordinate of t h e N c e n t e r s )
4 f o r d=l:s
5 [[Link]] = n d g r i d ( d s i t e s ( : , d ) , c t r s ( : , d ) ) ;
6 DM = DM + ( d r - c c ) . " 2 ;
7 end
8 DM = sqrt(DM);

N o t e t h a t t h i s s u b r o u t i n e c a n easily b e m o d i f i e d t o p r o d u c e a p - n o r m d i s t a n c e
m a t r i x b y m a k i n g t h e o b v i o u s changes t o lines 6 a n d 8 o f t h e code i n P r o g r a m 1.1.
W e w i l l come back t o t h i s idea i n C h a p t e r 10.
O u r first m a i n s c r i p t is P r o g r a m 1.2. T h i s s c r i p t c a n be u s e d t o c o m p u t e the
distance m a t r i x interpolant to d a t a sampled f r o m the test function / s provided by
P r o g r a m C l . W e use H a l t o n p o i n t s a n d are a b l e t o select t h e space d i m e n s i o n
s and number of points N b y e d i t i n g lines 1 a n d 2 o f t h e code. The subrou-
t i n e MakeSDGrid.m w h i c h w e use t o c o m p u t e t h e e q u a l l y spaced p o i n t s i n t h e s-
dimensional u n i t cube o n line 6 o f D i s t a n c e M a t r i x F i t .mis p r o v i d e d i n A p p e n d i x C.
T h e s e e q u a l l y spaced p o i n t s are used as e v a l u a t i o n p o i n t s a n d t o c o m p u t e errors.
N o t e t h a t since t h e d i s t a n c e m a t r i x i n t e r p o l a n t is o f t h e f o r m ( 1 . 4 ) i t s s i m u l t a n e o u s
e v a l u a t i o n a t t h e e n t i r e set o f e v a l u a t i o n p o i n t s a m o u n t s t o a m a t r i x - v e c t o r p r o d u c t
o f t h e e v a l u a t i o n m a t r i x EM a n d t h e coefficients c. H e r e t h e e v a l u a t i o n m a t r i x has
t h e same s t r u c t u r e as t h e i n t e r p o l a t i o n m a t r i x a n d c a n also be c o m p u t e d u s i n g t h e
subroutine Distancematrix.m ( o n l y u s i n g e v a l u a t i o n p o i n t s i n place o f t h e d a t a
sites, see l i n e 9 o f D i s t a n c e M a t r i x F i t .m). T h e coefficient v e c t o r c is s u p p l i e d d i -
r e c t l y as s o l u t i o n o f t h e l i n e a r s y s t e m Ac = f (see ( 1 . 3 ) a n d t h e M A T L A B e x p r e s s i o n
IM\rhs o n l i n e 1 0 o f t h e p r o g r a m ) . T h e e v a l u a t i o n p o i n t s are s u b s e q u e n t l y used
for t h e e r r o r c o m p u t a t i o n i n lines 1 1 - 1 3 a n d are also u s e d for p l o t t i n g p u r p o s e s i n
t h e last p a r t o f t h e p r o g r a m (lines 1 6 - 3 5 ) . N o t e t h a t for t h i s e x a m p l e w e k n o w t h e
function f s t h a t g e n e r a t e d t h e d a t a , a n d t h e r e f o r e are a b l e t o c o m p u t e t h e e r r o r i n
o u r r e c o n s t r u c t i o n . T h e s u b r o u t i n e s t h a t p r o d u c e t h e 2 D a n d 3 D p l o t s o n lines 2 4
3 2 are p r o v i d e d i n A p p e n d i x C . N o t e t h a t t h e use o f r e s h a p e o n lines 2 2 - 2 3 and
2 7 - 2 9 c o r r e s p o n d s t o t h e use o f meshgrid for p l o t t i n g p u r p o s e s .

P r o g r a m 1.2. D i s t a n c e M a t r i x F i t .m

% DistanceMatrixFit
% S c r i p t that uses E u c l i d e a n d i s t a n c e matrices t o perform
 1. Introduction

% s c a t t e r e d d a t a i n t e r p o l a t i o n f o r a r b i t r a r y space dimensions
% C a l l s on: D i s t a n c e M a t r i x , MakeSDGrid, t e s t f u n c t i o n
/ Uses:
0 h a l t o n s e q ( w r i t t e n by D a n i e l Dougherty from MATLAB
% C e n t r a l F i l e Exchange)
1 s = 3;
2 k = 2; N = ( 2 ~ k + l ) ~ s ;
3 n e v a l = 10; M = n e v a l ' s ;
% Use Halton p o i n t s a s d a t a s i t e s and c e n t e r s
4 d s i t e s = haltonseq(N,s);
5 ctrs = dsites;
% Create n e v a l ~ s e q u a l l y spaced e v a l u a t i o n l o c a t i o n s i n t h e
% s-dimensional u n i t cube
6 e p o i n t s = MakeSDGrid(s,neval);
% Create right-hand side vector,
% i . e . , evaluate the t e s t function a t the data s i t e s
7 rhs= testfunction(s,dsites);
% Compute d i s t a n c e m a t r i x f o r t h e d a t a s i t e s and c e n t e r s
8 IM = D i s t a n c e M a t r i x ( d s i t e s , c t r s ) ;
% Compute d i s t a n c e m a t r i x f o r e v a l u a t i o n p o i n t s and c e n t e r s
9 EM = D i s t a n c e M a t r i x ( e p o i n t s , c t r s ) ;
% E v a l u a t e t h e i n t e r p o l a n t on e v a l u a t i o n p o i n t s
/ ( e v a l u a t i o n m a t r i x * s o l u t i o n of i n t e r p o l a t i o n system)
0

10 Pf = EM * ( I M \ r h s ) ;
% Compute exact s o l u t i o n ,
% i . e . , e v a l u a t e t e s t f u n c t i o n on e v a l u a t i o n p o i n t s
11 exact = t e s t f u n c t i o n ( s , e p o i n t s ) ;
% Compute maximum and RMS e r r o r s on e v a l u a t i o n g r i d
12 maxerr = n o r m ( P f - e x a c t , i n f ) ;
13 [Link] = n o r m ( P f - e x a c t ) / s q r t ( M ) ;
J
14 f p r i n t f ( R M S e r r o r : 7 e\n', r m s _ e r r )
0

0
15 f p r i n t f ('Maximum e r r o r : / e\n', maxerr)
0

16 s w i t c h s
17 case 1
18 plot(epoints, Pf)
19 figure; plot(epoints, abs(Pf-exact))
20 case 2
21 fview = [-30,30];
22 xe = r e s h a p e ( e p o i n t s ( : , 2 ) , n e v a l , n e v a l ) ;
23 ye = r e s h a p e ( e p o i n t s ( : , 1 ) , n e v a l , n e v a l ) ;
24 PlotSurf(xe,ye,Pf,neval,[Link],fview);
25 PlotError2D(xe,ye,Pf,exact,maxerr,neval,fview);
26 case 3
10 Meshfree Approximation Methods with M A T L A B

27 xe = reshapeCepoints(:,2),neval,neval,neval);
38 ye = reshape(epoints(:,1),neval,neval,neval);
29 ze = reshape(epoints(:,3),neval,neval,neval);
30 xslice = .25:.25:1; yslice = 1; zslice = [0,0.5];
31 PlotSIices(xe,ye,ze,Pf,neval,xslice,yslice,zslice);
32a PlotErrorSlices(xe,ye,ze,Pf,exact,neval,...
32b xslice,yslice,zslice);
33 otherwise
34 disp('Cannot display plots for s>3')
35 end

I n T a b l e s 1.1 a n d 1.2 as w e l l as F i g u r e s 1.3 a n d 1.4 w e p r e s e n t some e x a m p l e s

c o m p u t e d w i t h P r o g r a m 1.2. The number M of evaluation points (determined by
n e v a l o n l i n e 3 o f t h e code) w e used for t h e cases s = 1, 2 , . . . , 6, was 1000, 1600,
1000, 256, 1024, a n d 4096, r e s p e c t i v e l y {i.e., neval = 1000, 40, 10, 4, 4 , a n d 4,
r e s p e c t i v e l y ) . N o t e t h a t , as t h e space d i m e n s i o n s increases, m o r e a n d m o r e o f t h e
e v a l u a t i o n p o i n t s lie o n t h e b o u n d a r y o f t h e d o m a i n , w h i l e t h e d a t a sites ( w h i c h are
g i v e n as H a l t o n p o i n t s ) are l o c a t e d i n t h e i n t e r i o r o f t h e d o m a i n . T h e v a l u e k l i s t e d
i n Tables 1.1 a n d 1.2 is t h e same as t h e k i n l i n e 2 o f P r o g r a m 1.2. T h e f o r m u l a for
t h e r o o t - m e a n - s q u a r e e r r o r ( R M S - e r r o r ) is g i v e n b y

RMS-error (1.5)

w h e r e t h e j, j = 1 , . . . , M are t h e evaluation points. F o r m u l a (1.5) is used o n

l i n e 13 o f P r o g r a m 1.2.
T h e basic M A T L A B code for t h e s o l u t i o n o f a n y k i n d o f R B F i n t e r p o l a t i o n p r o b -
l e m w i l l be v e r y s i m i l a r t o P r o g r a m 1.2. N o t e i n p a r t i c u l a r t h a t t h e d a t a u s e d
even for t h e d i s t a n c e m a t r i x i n t e r p o l a t i o n c o n s i d e r e d h e r e c a n also be "real"
data. I n t h a t case one s i m p l y needs t o r e p l a c e lines 4 a n d 7 o f t h e p r o g r a m b y
a p p r o p r i a t e code t h a t generates t h e d a t a sites a n d d a t a values for t h e r i g h t - h a n d
side.
T h e p l o t s o n t h e left o f F i g u r e s 1.3 a n d 1.4 d i s p l a y t h e g r a p h s o f t h e d i s t a n c e
m a t r i x fits for space d i m e n s i o n s s = 1, 2, a n d 3, r e s p e c t i v e l y , w h i l e t h o s e o n t h e
right depict the corresponding errors. F o r t h e I D p l o t s ( i n F i g u r e 1.3) w e u s e d
5 H a l t o n p o i n t s t o i n t e r p o l a t e t h e t e s t f u n c t i o n / i . T h e piecewise l i n e a r n a t u r e o f
t h e i n t e r p o l a n t is c l e a r l y v i s i b l e a t t h i s r e s o l u t i o n . I f w e use m o r e p o i n t s t h e n t h e
fit becomes m o r e a c c u r a t e see T a b l e 1.1 b u t t h e n i t is n o l o n g e r p o s s i b l e t o
d i s t i n g u i s h t h e piecewise l i n e a r n a t u r e o f t h e i n t e r p o l a n t . T h e 2 D p l o t ( t o p left o f
F i g u r e 1.4) i n t e r p o l a t e s t h e t e s t f u n c t i o n f 2 a t 289 H a l t o n p o i n t s . T h e g r a p h o f Vf is
false-colored a c c o r d i n g t o t h e a b s o l u t e e r r o r ( i n d i c a t e d b y t h e c o l o r b a r a t t h e r i g h t
o f t h e p l o t ) . T h e b o t t o m p l o t i n F i g u r e 1.4 shows a slice p l o t o f t h e d i s t a n c e m a t r i x
i n t e r p o l a n t t o fa b a s e d o n 729 H a l t o n p o i n t s . For this p l o t the colors represent
f u n c t i o n values ( a g a i n i n d i c a t e d b y t h e c o l o r b a r o n t h e r i g h t ) .
 1. Introduction 11

s
Table 1.1 Distance matrix fit to N Halton points in [0, l ] , s 1, 2, 3.

ID 2D 3D

k N RMS-error N RMS-error N RMS-error

1 3 5.896957e-001 9 1.937341e-001 27 9.721476e-002

2 5 3.638027e-001 25 6.336315e-002 125 6.277141e-002
3 9 1.158328e-001 81 2.349093e-002 729 2.759452e-002
4 17 3.981270e-002 289 1.045010e-002
5 33 1.406188e-002 1089 4.326940e-003
6 65 5.068541e-003 4225 1.797430e-003
7 129 1.877013e-003
8 257 7.264159e-004
9 513 3.016376e-004
10 1025 1.381896e-004
11 2049 6.907386e-005
12 4097 3.453179e-005

X X

Fig. 1.3 F i t (left) and absolute error (right) for 5 point distance matrix interpolation in I D .

I n t h e r i g h t h a l f o f F i g u r e s 1.3 a n d 1.4 w e s h o w a b s o l u t e e r r o r s for t h e d i s t a n c e

m a t r i x i n t e r p o l a n t s d i s p l a y e d i n t h e left c o l u m n . W e use a n a l o g o u s c o l o r schemes,
i.e., t h e 2 D p l o t ( t o p p a r t o f F i g u r e 1.4) is false-colored a c c o r d i n g t o t h e a b s o l u t e
e r r o r , a n d so is t h e 3 D p l o t ( b o t t o m ) since n o w t h e " f u n c t i o n v a l u e " corresponds
to the absolute error. W e c a n see c l e a r l y t h a t m o s t o f t h e e r r o r is c o n c e n t r a t e d
near t h e b o u n d a r y o f t h e d o m a i n . I n f a c t , t h e a b s o l u t e e r r o r is a b o u t one o r d e r o f
m a g n i t u d e l a r g e r n e a r t h e b o u n d a r y t h a n i t is i n t h e i n t e r i o r o f t h e d o m a i n . T h i s
is n o s u r p r i s e since t h e d a t a sites are l o c a t e d i n t h e i n t e r i o r . H o w e v e r , even f o r
u n i f o r m l y spaced d a t a sites ( i n c l u d i n g p o i n t s o n t h e b o u n d a r y ) t h e m a i n e r r o r i n
r a d i a l basis f u n c t i o n i n t e r p o l a t i o n is u s u a l l y l o c a t e d near t h e b o u n d a r y .
F r o m t h i s f i r s t s i m p l e e x a m p l e w e c a n observe a n u m b e r o f o t h e r f e a t u r e s . Most
o f t h e m are c h a r a c t e r i s t i c f o r t h e r a d i a l basis f u n c t i o n i n t e r p o l a n t s w e w i l l be s t u d y -
i n g l a t e r o n . F i r s t , t h e basis f u n c t i o n s e m p l o y e d , B k = || a?&||2> are r a d i a l l y s y m -
12 Meshfree Approximation Methods with MATLAB

y 0 0 x y 0 0

Fig. 1.4 F i t s (left) and errors (right) for distance matrix interpolation with 289 points in 2D
(top), and 729 points in 3D (bottom).

s
Table 1.2 Distance matrix fit to N Halton points in [0, l ] , s 4, 5, 6.

4D 5D 6D

k N RMS-error N RMS-error N RMS-error

1 81 1.339581e-001 243 9.558350e-002 729 5.097600e-002

2 625 6.817424e-002 3125 3.118905e-002

metric. Second, as t h e M A T L A B s c r i p t s s h o w , t h e m e t h o d is e x t r e m e l y s i m p l e t o
i m p l e m e n t for a n y space d i m e n s i o n s. For example, no u n d e r l y i n g c o m p u t a t i o n a l
m e s h is r e q u i r e d t o c o m p u t e t h e i n t e r p o l a n t . T h e process o f m e s h g e n e r a t i o n is
a m a j o r f a c t o r w h e n w o r k i n g i n h i g h e r space d i m e n s i o n s w i t h polynomial-based
m e t h o d s such as splines o r finite e l e m e n t s . A l l t h a t is r e q u i r e d for o u r m e t h o d is
t h e p a i r w i s e d i s t a n c e b e t w e e n t h e d a t a sites. T h e r e f o r e , w e h a v e w h a t is k n o w n as
a meshfree (or meshless) method.
T h i r d , t h e a c c u r a c y o f t h e m e t h o d i m p r o v e s i f w e a d d m o r e d a t a sites. I n f a c t ,
i t seems t h a t t h e R M S - e r r o r i n T a b l e s 1.1 a n d 1.2 is r e d u c e d b y a f a c t o r o f a b o u t
s
t w o f r o m one r o w t o t h e n e x t . Since w e use (2* + l ) uniformly distributed random
 1. Introduction 13

d a t a p o i n t s i n r o w k t h i s i n d i c a t e s a convergence r a t e o f r o u g h l y O(h), where h can

be v i e w e d as s o m e t h i n g l i k e t h e average d i s t a n c e o r meshsize o f t h e set X o f d a t a
sites (we w i l l be m o r e precise l a t e r o n ) .
A n o t h e r t h i n g t o n o t e is t h a t t h e s i m p l e d i s t a n c e f u n c t i o n i n t e r p o l a n t used here
(as w e l l as m a n y o t h e r r a d i a l basis f u n c t i o n i n t e r p o l a n t s used l a t e r ) r e q u i r e s the
s o l u t i o n o f a s y s t e m o f l i n e a r e q u a t i o n s w i t h a dense N x N m a t r i x . T h i s makes
i t v e r y c o s t l y t o a p p l y t h e m e t h o d i n i t s s i m p l e f o r m t o l a r g e d a t a sets. M o r e o v e r ,
as we w i l l see l a t e r , these m a t r i c e s also t e n d t o be r a t h e r i l l - c o n d i t i o n e d . T h e s e are
t h e reasons w h y we c a n o n l y present r e s u l t s for r e l a t i v e l y s m a l l d a t a sets i n h i g h e r
space d i m e n s i o n s u s i n g t h i s s i m p l e a p p r o a c h .
I n t h e r e m a i n d e r o f t h i s b o o k i t is o u r g o a l t o present a l t e r n a t i v e s t o t h i s basic
i n t e r p o l a t i o n m e t h o d t h a t address t h e p r o b l e m s m e n t i o n e d a b o v e such as l i m i t a t i o n
t o s m a l l d a t a sets, i l l - c o n d i t i o n i n g , l i m i t e d a c c u r a c y a n d l i m i t e d s m o o t h n e s s o f t h e
interpolant.

1.2 Some Historical R e m a r k s

O r i g i n a l l y , t h e m o t i v a t i o n for t h e basic meshfree approximation methods (ra-

d i a l basis f u n c t i o n a n d m o v i n g least squares m e t h o d s ) c a m e f r o m a p p l i c a t i o n s i n
geodesy, geophysics, m a p p i n g , o r m e t e o r o l o g y . L a t e r , applications were f o u n d i n
m a n y o t h e r areas such as i n t h e n u m e r i c a l s o l u t i o n o f P D E s , c o m p u t e r graph-
ics, artificial intelligence, statistical learning theory, neural networks, signal a n d
i m a g e processing, sampling theory, statistics (kriging), finance, and optimiza-
tion. I t s h o u l d be p o i n t e d o u t t h a t meshfree l o c a l regression m e t h o d s have b e e n
used i n d e p e n d e n t l y i n s t a t i s t i c s for w e l l over 100 years (see, e.g., [Cleveland and
L o a d e r (1996)] a n d t h e references t h e r e i n ) . I n fact, t h e basic m o v i n g least squares
m e t h o d ( k n o w n also as l o c a l regression i n t h e s t a t i s t i c s l i t e r a t u r e ) c a n be t r a c e d
back at least t o t h e w o r k o f [ G r a m ( 1 8 8 3 ) ; W o o l h o u s e ( 1 8 7 0 ) ; D e Forest ( 1 8 7 3 ) ;
D e Forest ( 1 8 7 4 ) ] .
I n t h e l i t e r a t u r e o n a p p r o x i m a t i o n t h e o r y a n d r e l a t e d a p p l i c a t i o n s areas some
h i s t o r i c a l l a n d m a r k c o n t r i b u t i o n s have come f r o m

D o n a l d S h e p a r d , w h o as a n u n d e r g r a d u a t e s t u d e n t a t H a r v a r d U n i v e r s i t y ,
suggested t h e use o f w h a t are n o w c a l l e d Shepard functions in the late
1960s (see C h a p t e r 22). T h e p u b l i c a t i o n [Shepard (1968)] discusses t h e
basic inverse d i s t a n c e w e i g h t e d S h e p a r d m e t h o d a n d some m o d i f i c a t i o n s
thereof. T h e m e t h o d was a t t h e t i m e i n c o r p o r a t e d i n t o a c o m p u t e r p r o -
g r a m , S Y M A P , for m a p m a k i n g .
R o l l a n d H a r d y , w h o was a geodesist at I o w a S t a t e U n i v e r s i t y . H e i n t r o -
d u c e d t h e so-called multiquadrics ( M Q s ) i n t h e e a r l y 1970s (see, e.g., [ H a r d y
(1971)] or C h a p t e r 8 ) . H a r d y ' s w o r k was p r i m a r i l y c o n c e r n e d w i t h a p p l i -
c a t i o n s i n geodesy a n d m a p p i n g .
 Meshfree Approximation Methods with MATLAB

R o b e r t L . H a r d e r a n d R o b e r t N . D e s m a r a i s , w h o w e r e aerospace engineers
at M a c N e a l - S c h w e n d l e r C o r p o r a t i o n ( M S C S o f t w a r e ) , a n d N A S A ' s L a n g l e y
Research C e n t e r . T h e y i n t r o d u c e d t h e s o - c a l l e d thin plate splines (TPSs)
i n 1972 (see, e.g., [ H a r d e r a n d D e s m a r a i s ( 1 9 7 2 ) ] o r C h a p t e r 8 ) . T h e i r w o r k
was c o n c e r n e d m o s t l y w i t h a i r c r a f t d e s i g n .
Jean Duchon, a m a t h e m a t i c i a n at the Universite Joseph Fourier i n Greno-
ble, France. D u c h o n suggested a v a r i a t i o n a l a p p r o a c h minimizing the
2 2
integral of V / in R w h i c h also leads t o t h e t h i n p l a t e splines. This
w o r k was d o n e i n t h e m i d 1970s a n d is c o n s i d e r e d t o b e t h e foundation
o f t h e v a r i a t i o n a l a p p r o a c h t o r a d i a l basis f u n c t i o n s (see [ D u c h o n ( 1 9 7 6 ) ;
D u c h o n (1977); D u c h o n (1978); D u c h o n (1980)]) or C h a p t e r 13).
Jean Meinguet, a mathematican a t U n i v e r s i t e C a t h o l i q u e de L o u v a i n i n
L o u v a i n , B e l g i u m . M e i n g u e t i n t r o d u c e d w h a t he c a l l e d surface splines in
t h e l a t e 1970s. Surface splines a n d t h i n p l a t e splines f a l l u n d e r w h a t w e
w i l l refer t o as polyharmonic splines (see, e.g., [ M e i n g u e t ( 1 9 7 9 a ) ; M e i n g u e t
(1979b); M e i n g u e t (1979c); M e i n g u e t (1984)] or C h a p t e r 8).
P e t e r L a n c a s t e r a n d K e s Salkauskas, m a t h e m a t i c i a n s at t h e U n i v e r s i t y o f
Calgary, Canada. T h e y p u b l i s h e d [ L a n c a s t e r a n d Salkauskas ( 1 9 8 1 ) ] i n -
t r o d u c i n g t h e moving least squares method (a g e n e r a l i z a t i o n o f S h e p a r d
functions).
R i c h a r d Franke, a m a t h e m a t i c i a n at the N a v a l Postgraduate School i n M o n -
terey, C a l i f o r n i a . I n [ F r a n k e (1982a)] he c o m p a r e d v a r i o u s s c a t t e r e d d a t a
i n t e r p o l a t i o n methods, a n d concluded M Q s a n d T P S s were the best. Franke
also c o n j e c t u r e d t h a t t h e i n t e r p o l a t i o n m a t r i x for M Q s is i n v e r t i b l e .
W o l o d y m y r ( W a l l y ) M a d y c h , a m a t h e m a t i c i a n at the U n i v e r s i t y of C o n -
necticut, and Stuart A l a n Nelson, a m a t h e m a t i c i a n from Iowa State Univer-
sity. I n 1983 t h e y c o m p l e t e d t h e i r m a n u s c r i p t [ M a d y c h a n d N e l s o n (1983)]
i n w h i c h t h e y p r o v e d Franke's conjecture ( a n d m u c h more) based o n a varia-
t i o n a l approach. However, t h i s m a n u s c r i p t was never published. O t h e r fun-
d a m e n t a l p a p e r s b y these t w o a u t h o r s are, e.g., [ M a d y c h a n d N e l s o n ( 1 9 8 8 ) ;
M a d y c h and Nelson (1990a); M a d y c h and Nelson (1992)].
Charles Micchelli, a m a t h e m a t i c i a n at the I B M W a t s o n Research Center.
M i c c h e l l i p u b l i s h e d t h e p a p e r [ M i c c h e l l i ( 1 9 8 6 ) ] . H e also p r o v e d F r a n k e ' s
conjecture. H i s p r o o f s are r o o t e d i n t h e w o r k o f [ B o c h n e r ( 1 9 3 2 ) ; B o c h n e r
(1933)] a n d [Schoenberg ( 1 9 3 7 ) ; S c h o e n b e r g ( 1 9 3 8 a ) ; S c h o e n b e r g ( 1 9 3 8 b ) ]
on positive definite a n d c o m p l e t e l y m o n o t o n e functions. T h i s is also t h e
approach we w i l l follow t h r o u g h o u t m u c h of this book.
G r a c e W a h b a , a s t a t i s t i c i a n a t t h e U n i v e r s i t y o f W i s c o n s i n . She s t u d i e d t h e
use o f t h i n p l a t e splines for s t a t i s t i c a l p u r p o s e s i n t h e c o n t e x t o f s m o o t h i n g
noisy data a n d d a t a o n spheres, a n d i n t r o d u c e d t h e A N O V A a n d cross
v a l i d a t i o n a p p r o a c h e s t o t h e r a d i a l basis f u n c t i o n s e t t i n g ( s e e , e.g., [Wahba
 1. Introduction 15

(1979); W a h b a ( 1 9 8 1 ) ; W a h b a a n d W e n d e l b e r g e r ( 1 9 8 0 ) ] ) . O n e o f t h e first
m o n o g r a p h s o n t h e s u b j e c t is [ W a h b a ( 1 9 9 0 b ) ] .
R o b e r t Schaback, a m a t h e m a t i c i a n a t t h e U n i v e r s i t y o f G o t t i n g e n , Ger-
many. Compactly supported radial basis functions ( C S R B F s ) were i n t r o -
d u c e d i n [Schaback ( 1 9 9 5 a ) ] , a n d a v e r y p o p u l a r f a m i l y o f C S R B F s was
presented b y H o l g e r W e n d l a n d (also a m a t h e m a t i c i a n i n G o t t i n g e n ) i n his
P h . D . thesis (see also [ W e n d l a n d (1995)] a n d C h a p t e r 1 1 ) . B o t h o f these
a u t h o r s have c o n t r i b u t e d e x t e n s i v e l y t o t h e field o f r a d i a l basis f u n c t i o n s .
W e m e n t i o n p a r t i c u l a r l y t h e recent m o n o g r a p h [ W e n d l a n d ( 2 0 0 5 a ) ] .
 Chapter 2

Radial Basis Function Interpolation

in MATLAB

Before we discuss a n y o f t h e t h e o r e t i c a l f o u n d a t i o n o f r a d i a l basis f u n c t i o n s w e w a n t

t o get a feel for w h a t t h e y are a l l a b o u t . W e saw i n t h e i n t r o d u c t o r y c h a p t e r t h a t
i t is easy t o use E u c l i d e a n d i s t a n c e m a t r i c e s t o c o m p u t e a s o l u t i o n t o t h e s c a t t e r e d
d a t a i n t e r p o l a t i o n p r o b l e m . H o w e v e r , we also p o i n t e d o u t a n u m b e r o f l i m i t a t i o n s
t o t h a t a p p r o a c h such as t h e l i m i t e d a c c u r a c y a n d l i m i t e d s m o o t h n e s s . I t t u r n s o u t
t h a t we c a n m a i n t a i n t h e u n d e r l y i n g s t r u c t u r e presented b y t h e d i s t a n c e m a t r i x
a p p r o a c h a n d address these l i m i t a t i o n s b y c o m p o s i n g t h e d i s t a n c e f u n c t i o n w i t h
certain "good" univariate functions.

2.1 Radial (Basis) Functions

A s a first e x a m p l e we p i c k a f u n c t i o n w e l l - r e p r e s e n t e d i n m a n y branches o f m a t h e -
matics, namely the Gaussian
2
^( )= -(^) ,
r e r e i
2
O u r shape parameter e is r e l a t e d t o t h e v a r i a n c e a of the normal distribution
2 2
function by e = l/(2o~ ). I f we c o m p o s e t h e G a u s s i a n w i t h t h e E u c l i d e a n d i s t a n c e
s
f u n c t i o n 11 - j12 we o b t a i n for a n y f i x e d center x k GR a multivariate function
3
$ (sc) = e - " -
fc
e a a ! a ? f c
ll2, x e R .

Obviously, the connection between a n d (p is g i v e n b y

^fc(x) = (p(\\x - x \\ ).
k 2

I t is t h i s c o n n e c t i o n t h a t gives rise t o t h e n a m e radial basis function (RBF). The

f o l l o w i n g is a f o r m a l d e f i n i t i o n o f a r a d i a l f u n c t i o n .

S
D e f i n i t i o n 2.1. A f u n c t i o n $ : R > R is c a l l e d radial p r o v i d e d t h e r e exists a
univariate f u n c t i o n <p : [0, 00) > R s u c h t h a t

&(x) = (f(r), where r \\x\\,

a n d || || is some n o r m o n M u s u a l l y t h e E u c l i d e a n n o r m .
s

17
18 Meshfree Approximation Methods with MATLAB

D e f i n i t i o n 2.1 says t h a t for a r a d i a l f u n c t i o n <&

s
||a;i|| = ||x2|| $(JCI) = $(x ), 2 x u x eR .
2

I n other words, the value o f a t a n y p o i n t a t a c e r t a i n fixed d i s t a n c e f r o m t h e o r i g i n

(or a n y o t h e r fixed center p o i n t ) is c o n s t a n t . T h u s , <3> is r a d i a l l y (or s p h e r i c a l l y )
s y m m e t r i c a b o u t i t s center. D e f i n i t i o n 2 . 1 s h o w s t h a t t h e E u c l i d e a n d i s t a n c e func-
t i o n we used i n t h e i n t r o d u c t i o n is j u s t a s p e c i a l case o f a r a d i a l (basis) f u n c t i o n .
N a m e l y , w i t h ip(r) = r.
F i g u r e 2.1 shows t h e g r a p h s o f t w o G a u s s i a n r a d i a l basis f u n c t i o n s , one w i t h
shape p a r a m e t e r e 1 (left) a n d one w i t h e 3 ( r i g h t ) ( b o t h c e n t e r e d a t t h e o r i g i n
2
i n R ) . A s m a l l e r v a l u e o f e (i.e., l a r g e r v a r i a n c e ) causes t h e f u n c t i o n t o b e c o m e
" f l a t t e r " , w h i l e i n c r e a s i n g e leads t o a m o r e p e a k e d R B F , a n d t h e r e f o r e localizes i t s
influence. W e w i l l see s o o n t h a t t h e choice o f e has a p r o f o u n d i n f l u e n c e o n b o t h
the accuracy and numerical s t a b i l i t y o f the s o l u t i o n t o our i n t e r p o l a t i o n p r o b l e m .

u is

2
Fig. 2.1 Gaussian with e = 1 (left) and e = 3 (right) centered at the origin in R .

D e f i n i t i o n 2.1 a n d t h e d i s c u s s i o n l e a d i n g u p t o i t s h o w a g a i n w h y i t m a k e s sense
t o c a l l ip t h e basic f u n c t i o n , a n d &k(\\ H2) (centered at x ) k a r a d i a l basis f u n c t i o n .
O n e single basic f u n c t i o n generates a l l o f t h e basis f u n c t i o n s t h a t are u s e d i n t h e
expansion (1.1).
R a d i a l f u n c t i o n i n t e r p o l a n t s h a v e t h e nice p r o p e r t y o f b e i n g i n v a r i a n t u n d e r a l l
Euclidean transformations (i.e., translations, rotations, a n d reflections). B y this
w e m e a n t h a t i t does n o t m a t t e r w h e t h e r w e first c o m p u t e t h e R B F i n t e r p o l a n t
a n d t h e n a p p l y a E u c l i d e a n t r a n s f o r m a t i o n , o r i f w e first t r a n s f o r m t h e d a t a a n d
t h e n c o m p u t e t h e i n t e r p o l a n t . T h i s is a n i m m e d i a t e consequence o f t h e fact t h a t
E u c l i d e a n t r a n s f o r m a t i o n s are c h a r a c t e r i z e d by orthogonal transformation matri-
ces a n d are t h e r e f o r e 2 - n o r m - i n v a r i a n t . I n v a r i a n c e u n d e r t r a n s l a t i o n , r o t a t i o n a n d
r e f l e c t i o n is o f t e n d e s i r a b l e i n a p p l i c a t i o n s .
Moreover, the application of radial functions to the solution of the scattered data
i n t e r p o l a t i o n p r o b l e m (as w e l l as m a n y o t h e r m u l t i v a r i a t e a p p r o x i m a t i o n p r o b l e m s )
benefits f r o m t h e fact t h a t t h e i n t e r p o l a t i o n p r o b l e m b e c o m e s i n s e n s i t i v e t o the
 2. Radial Basis Function Interpolation in M A T L A B 19

d i m e n s i o n s o f t h e space i n w h i c h t h e d a t a sites l i e . I n s t e a d o f h a v i n g t o d e a l w i t h
a multivariate function $ (whose c o m p l e x i t y w i l l increase w i t h i n c r e a s i n g space
d i m e n s i o n s) w e c a n w o r k w i t h t h e same u n i v a r i a t e f u n c t i o n ip for a l l choices o f s.

2.2 Radial Basis Function Interpolation

I n s t e a d o f u s i n g s i m p l e d i s t a n c e m a t r i c e s as w e d i d earlier, w e n o w use a r a d i a l
s
basis f u n c t i o n e x p a n s i o n t o solve t h e s c a t t e r e d d a t a i n t e r p o l a t i o n p r o b l e m i n M. b y
assuming
N
s
V (x)
f = J2ck<p(\\x-x \\ ), k 2 x e R . (2.1)
fc=i
T h e coefficients c k are f o u n d b y e n f o r c i n g t h e i n t e r p o l a t i o n c o n d i t i o n s , a n d t h u s
solving the linear system

p(\\x X - Xi|| ) 2 (p (\\x 1 - X \\ )

2 2 <p(\\Xi - X \\ )
N 2

tp (\\x 2 - SC1U2) <P (\\x 2 - x \\ ) 2 2 ... ip(\\x 2 - X \\ )

N 2 C2 /(*2)
=

_ip(\\x N - xi\\ ) 2 (p(\\x

N - X2W2) f(\\x N - x \\ )
N 2

A s t h e s o l u t i o n o f t h e s c a t t e r e d d a t a i n t e r p o l a t i o n p r o b l e m hinges e n t i r e l y o n t h e
solution of this system of linear equations we w i l l devote t h e next chapter t o the
q u e s t i o n o f w h e n (i.e., for w h a t t y p e o f basic f u n c t i o n s <p) t h e s y s t e m m a t r i x is
non-singular.
F o r t h e n u m e r i c a l e x a m p l e p r e s e n t e d b e l o w w e r e s t r i c t ourselves t o t h e t w o -
d i m e n s i o n a l case s = 2. A s basic f u n c t i o n ip w e w i l l use b o t h Gaussians a n d t h e
l i n e a r f u n c t i o n <p(r) = r w h i c h gives rise t o t h e E u c l i d e a n d i s t a n c e m a t r i x a p p r o a c h
used earlier.
T h e code o f t h e MATLAB script RBFInterpolation2D.m (see P r o g r a m 2.1)
w e use t o p e r f o r m R B F i n t e r p o l a t i o n i n 2 D is v e r y s i m i l a r t o t h e earlier s c r i p t
D i s t a n c e M a t r i x F i t .m. I t also m a k e s use o f t h e s u b r o u t i n e D i s t a n c e M a t r i x . m .
W h i l e i t is easy t o w r i t e a v e r s i o n o f t h e i n t e r p o l a t i o n s c r i p t t h a t w o r k s for a n y
space d i m e n s i o n s ( j u s t as w e d i d i n D i s t a n c e M a t r i x F i t .m) w e w i l l s t i c k w i t h a
basic 2 D v e r s i o n here.
I n l i n e 1 we define t h e G a u s s i a n R B F as a M A T L A B a n o n y m o u s f u n c t i o n t h a t
accepts a m a t r i x a r g u m e n t ( n a m e l y t h e o u t p u t f r o m D i s t a n c e M a t r i x ) a l o n g w i t h
its shape p a r a m e t e r . N o t e t h a t t h i s f e a t u r e is o n l y a v a i l a b l e since M A T L A B Re-
lease 7. For older M A T L A B versions w e suggest u s i n g a n i n l i n e f u n c t i o n i n s t e a d
(see t h e p r o g r a m s i n t h e f o l d e r M a t l a b 6 o f t h e enclosed C D ) . I f e x e c u t i o n speed
is i m p o r t a n t , t h e n one s h o u l d e x p l i c i t l y p r o v i d e t h e f u n c t i o n ( e i t h e r h a r d c o d e d d i -
r e c t l y w h e r e needed, o r as a n M - f i l e ) . T h i s l a t t e r a p p r o a c h w i l l a l w a y s be m o r e
efficient t h a n t h e i n l i n e o r e v e n a n o n y m o u s f u n c t i o n a p p r o a c h . H o w e v e r , t h e n t h e
i n t e r p o l a t i o n p r o g r a m is n o l o n g e r as generic.
20 Meshfree Approximation Methods with MATLAB

W e c a n replace t h e d e f i n i t i o n o f t h e G a u s s i a n o n l i n e 1 b y t h e d e f i n i t i o n o f t h e
l i n e a r f u n c t i o n <p(r) = r o r a n y o t h e r a d m i s s i b l e R B F w e w i l l e n c o u n t e r l a t e r . In
lines 2 - 6 w e define a t e s t f u n c t i o n t h a t w e w i l l s a m p l e s i m i l a r l y t o t h e function
f
s used i n t h e i n t r o d u c t o r y e x a m p l e . H e r e ( a n d i n m a n y l a t e r e x a m p l e s ) w e use
Franke's function

ffay) = ^ - l / 4 ( ( 9 . r - 2 ) + (9s/-2) )
e
2 2
+ 3 _(l/49)(9a +l) -(l/10)(9 /+l)
e ;
2
!
2

2 2 2 2
+ I - l / 4 ( ( 9 x - 7 ) + (9j/-3) ) _ l _ (
e e 9 a ; -4) -(9y-7)
2 5

w h i c h is a s t a n d a r d t e s t f u n c t i o n f o r 2 D s c a t t e r e d d a t a fitting. N o t e t h a t we used
2
(x,y) t o d e n o t e t h e t w o c o m p o n e n t s o f a; G M . T h e g r a p h o f Franke's function
over t h e u n i t s q u a r e is s h o w n i n F i g u r e 2.2.

Fig. 2.2 Franke's test function.

For m a n y o f o u r e x a m p l e s w e use d a t a l o c a t i o n s t h a t h a v e b e e n saved i n files

0
n a m e d Data2D_/ d / s w h e r e t h e n u m b e r o f p o i n t s (%d) is t a k e n f r o m t h e p r o g r e s s i o n
0 0

f c 2
{(2 + l) } = { 9 , 2 5 , 8 1 , 2 8 9 , 1 0 8 9 , 4 2 2 5 , . . . } . T h e characters u or h ( i n place o f
/ s) are used t o d e n o t e e i t h e r u n i f o r m l y s p a c e d p o i n t s , o r H a l t o n p o i n t s i n t h e u n i t
0

s q u a r e . T h e set o f d a t a p o i n t s is d e f i n e d a n d l o a d e d i n lines 7 a n d 8. A s i n t h e e a r l i e r
e x a m p l e w e consider here o n l y t h e case w h e r e t h e centers f o r t h e R B F s c o i n c i d e
w i t h the d a t a locations (line 9 ) .
A g r i d o f e v a l u a t i o n p o i n t s used t o e v a l u a t e o u r i n t e r p o l a n t for t h e p u r p o s e s
o f r e n d e r i n g a n d e r r o r c o m p u t a t i o n is d e f i n e d i n l i n e s 10 a n d 1 1 . T h e t e s t data
( r i g h t - h a n d side o f t h e i n t e r p o l a t i o n e q u a t i o n s ) are c o m p u t e d o n l i n e 12 w h e r e t h e
t e s t f u n c t i o n is s a m p l e d a t t h e d a t a sites.
T h e m a i n p a r t o f t h e code is g i v e n b y lines 1 3 - 1 7 . N o t e t h a t t h i s p a r t is v e r y
similar t o t h e corresponding segment ( l i n e s 7 - 9 ) i n D i s t a n c e M a t r i x F i t .m. The
o n l y difference is t h a t w e n o w a p p l y t h e basic f u n c t i o n <p t o t h e e n t i r e distance
matrices i n order to o b t a i n the i n t e r p o l a t i o n a n d evaluation matrices.
 2. Radial Basis Function Interpolation in MATLAB 21

P r o g r a m 2.1. RBFInterpolation2D.m

% RBFInterpolation2D
S c r i p t t h a t performs b a s i c 2D RBF i n t e r p o l a t i o n
% C a l l s on: D i s t a n c e M a t r i x
/ Define t h e G a u s s i a n RBF and shape parameter
1 r b f = @(e,r) e x p ( - ( e * r ) . ~ 2 ) ; ep = 21.1;
% Define Franke's f u n c t i o n as t e s t f u n c t i o n
2 f l = @(x,y) 0 . 7 5 * e x p ( - ( ( 9 * x - 2 ) . ~ 2 + ( 9 * y - 2 ) . " 2 ) / 4 ) ;
3 f 2 = <3(x,y) 0 . 7 5 * e x p ( - ( ( 9 * x + l ) . ~ 2 / 4 9 + ( 9 * y + l ) . ~ 2 / 1 0 ) ) ;
4 f 3 = @(x,y) 0 . 5 * e x p ( - ( ( 9 * x - 7 ) . ~ 2 + ( 9 * y - 3 ) . ~ 2 ) / 4 ) ;
5 f 4 = @(x,y) 0 . 2 * e x p ( - ( ( 9 * x - 4 ) . ~ 2 + ( 9 * y - 7 ) . " 2 ) ) ;
6 t e s t f u n c t i o n = @(x,y) f l ( x , y ) + f 2 ( x , y ) + f 3 ( x , y ) - f 4 ( x , y ) ;
7 N = 1089; g r i d t y p e = 'h';
% Load d a t a p o i n t s
0
8 name = s p r i n t f ( 'Data2D_y d / s' ,N,gridtype) ; load(name)
o 0

9 ctrs = dsites;
10 n e v a l = 40; g r i d = l i n s p a c e ( 0 , 1 , n e v a l ) ;
11 [xe,ye] = m e s h g r i d ( g r i d ) ; e p o i n t s = [ x e ( : ) y e ( : ) ] ;
% Evaluate the t e s t function a t the data points
12 r h s = t e s t f u n c t i o n ( d s i t e s ( : , 1 ) , d s i t e s ( : , 2 ) ) ;
% Compute d i s t a n c e m a t r i x between t h e d a t a s i t e s and c e n t e r s
13 DM_data = D i s t a n c e M a t r i x ( d s i t e s , c t r s ) ;
% Compute i n t e r p o l a t i o n m a t r i x
14 IM = rbf(ep,DM_data);
% Compute d i s t a n c e m a t r i x between e v a l u a t i o n p o i n t s and c e n t e r s
15 DM_eval = D i s t a n c e M a t r i x ( e p o i n t s , c t r s ) ;
% Compute e v a l u a t i o n m a t r i x
16 EM = rbf(ep,DM_eval);
% Compute RBF i n t e r p o l a n t
% ( e v a l u a t i o n m a t r i x * s o l u t i o n of i n t e r p o l a t i o n system)
17 Pf = EM * ( I M \ r h s ) ;
% Compute exact s o l u t i o n , i . e . ,
% e v a l u a t e t e s t f u n c t i o n on e v a l u a t i o n p o i n t s
18 exact = t e s t f u n c t i o n ( e p o i n t s ( : , 1 ) , e p o i n t s ( : , 2 ) ) ;
0/ Compute e r r o r s on e v a l u a t i o n g r i d
19 maxerr = n o r m ( P f - e x a c t , i n f ) ;
20 [Link] = n o r m ( P f - e x a c t ) / n e v a l ;
21 f p r i n t f ( ' R M S e r r o r : /,e\n', r m s . e r r )
22 f p r i n t f ('Maximum e r r o r : /e\n', maxerr)
0

23 fview = [160,20]; % f o r Franke's f u n c t i o n

24 P l o t S u r f ( x e , y e , P f , n e v a l , e x a c t , m a x e r r , f v i e w ) ;
25 PlotError2D(xe,ye,Pf,exact,maxerr,neval,fview);
22 Meshfree Approximation Methods with MATLAB

I n T a b l e 2.1 w e r e p o r t t h e results o f a series o f e x p e r i m e n t s i n w h i c h w e c o m p u t e

G a u s s i a n R B F a n d d i s t a n c e m a t r i x i n t e r p o l a n t s t o i n c r e a s i n g l y l a r g e r sets o f d a t a .
W e use one fixed v a l u e o f e for a l l o f t h e e x p e r i m e n t s w i t h t h e G a u s s i a n s . This
t y p e o f a p p r o x i m a t i o n is k n o w n as non-stationary approximation. Its counterpart
is k n o w n as stationary approximation.
E v e n t h o u g h we do n o t p e r f o r m s t a t i o n a r y i n t e r p o l a t i o n i n t h i s e x p e r i m e n t we
t a k e a m i n u t e t o e x p l a i n t h e essential difference b e t w e e n t h e t w o a p p r o a c h e s . I n t h e
s t a t i o n a r y s e t t i n g w e w o u l d scale t h e s h a p e p a r a m e t e r e a c c o r d i n g t o t h e fill distance
(or meshsize) h so t h a t we e n d u p u s i n g " p e a k e d " basis f u n c t i o n s for densely spaced
d a t a a n d " f l a t " basis f u n c t i o n s for coarsely s p a c e d d a t a . W e w i l l use t h e fill d i s t a n c e
as a m e a s u r e o f t h e d a t a d i s t r i b u t i o n . T h e fill d i s t a n c e is u s u a l l y d e f i n e d as

h = h ,n x s u p m i n ||aj (2.3)

a n d i t i n d i c a t e s h o w w e l l t h e d a t a i n t h e set X fill o u t t h e d o m a i n Q. A g e o m e t r i c
i n t e r p r e t a t i o n o f t h e fill d i s t a n c e is g i v e n b y t h e r a d i u s o f t h e largest possible e m p t y
b a l l t h a t c a n be p l a c e d a m o n g t h e d a t a l o c a t i o n s i n s i d e fl (see F i g u r e 2 . 3 ) . Some-
t i m e s t h e s y n o n y m covering radius is used. I n o u r M A T L A B code w e c a n estimate
t h e fill d i s t a n c e v i a

hX = m a x C m i n C D M - e v a l ' ) ) (2.4)
w h e r e DM_eval is t h e m a t r i x c o n s i s t i n g o f p a i r w i s e d i s t a n c e s b e t w e e n t h e e v a l u a t i o n
p o i n t s ( p l a c e d o n a fine u n i f o r m g r i d i n Q) a n d t h e d a t a sites X (c.f. l i n e 15 o f
P r o g r a m 2 . 1 ) . N o t e t h a t we t r a n s p o s e t h e n o n - s y m m e t r i c e v a l u a t i o n m a t r i x . T h i s
corresponds t o finding for each e v a l u a t i o n p o i n t t h e d i s t a n c e t o t h e c o r r e -
s p o n d i n g closest d a t a site, a n d t h e n s e t t i n g hx,n as t h e w o r s t o f t h o s e d i s t a n c e s .
F i g u r e 2.3 i l l u s t r a t e s t h e fill d i s t a n c e for a set o f 25 H a l t o n p o i n t s . Note that in
t h i s case t h e largest "hole" i n t h e d a t a is n e a r t h e b o u n d a r y .

0.8

0.6

0.4

0.2

0 0.2 0.4 0.6 0.8 1

X

Fig. 2.3 T h e fill distance for N = 25 Halton points (h ,n. x 0.2667).

We w i l l take a closer l o o k a t t h e differences between stationary and non-

stationary interpolation i n later chapters of this book.
 2. Radial Basis Function Interpolation in MATLAB 23

In t h e f o l l o w i n g e x a m p l e s w e w i l l c l e a r l y see t h e effects t h e s h a p e parame-

t e r has o n t h e c o n d i t i o n n u m b e r o f t h e i n t e r p o l a t i o n m a t r i x ( a n d t h e r e f o r e the
numerical stability) of our computations. I n o r d e r t o be able t o use o u r s c r i p t
R B F I n t e r p o l a t i o n 2 D . m i n c o n j u n c t i o n w i t h G a u s s i a n s t o p r o d u c e a m e a n i n g f u l se-
quence o f n o n - s t a t i o n a r y e x p e r i m e n t s , i.e., w i t h a f i x e d v a l u e o f t h e s h a p e p a r a m e t e r
s, we are r e q u i r e d t o t a k e t h e f a i r l y l a r g e v a l u e e 21.1. Otherwise computation
w i t h t h e r e l a t i v e l y densely spaced p o i n t set o f N = 4225 H a l t o n p o i n t s r e s u l t s i n
M A T L A B warnings of ill-conditioning. T h i s m e a n s t h a t for t h e n o n - s t a t i o n a r y
a p p r o a c h t h e basis f u n c t i o n s are t o o l o c a l i z e d o n t h e s m a l l e r p o i n t sets, a n d t h e
a p p r o x i m a t i o n is v e r y p o o r (see F i g u r e 2.4).
T h e t e s t r e s u l t s for a n o n - s t a t i o n a r y i n t e r p o l a t i o n e x p e r i m e n t u s i n g G a u s s i a n s
a n d E u c l i d e a n d i s t a n c e m a t r i c e s for F r a n k e ' s f u n c t i o n are s h o w n i n T a b l e 2.1. As
j u s t p o i n t e d o u t , w e n o t e t h a t a fit w i t h G a u s s i a n s a n d a s m a l l shape p a r a m e t e r
s u c h as s = 1 w o u l d q u i c k l y l e a d t o a n u m e r i c a l b r e a k d o w n . F o r as few as N = 25
d a t a p o i n t s a n d e = 1 M A T L A B issues a " m a t r i x close t o s i n g u l a r " w a r n i n g w i t h a n
e s t i m a t e d r e c i p r o c a l c o n d i t i o n n u m b e r o f RCOND=3.986027e-020.

Table 2.1 Non-stationary R B F interpolation to Franke's function using

Gaussians (e 21.1) and Euclidean distance matrices.

Gaussian distance matrix

k N RMS-error max-error RMS-error max-error

1 9 3.647169e-001 1.039682e+000 1.323106e-001 4.578028e-001

2 25 3.203404e-001 9.670980e-001 6.400558e-002 2.767871e-001
3 81 2.152222e-001 8.455161e-001 1.343780e-002 6.733130e-002
4 289 7.431729e-002 7.219253e-001 3.707360e-003 3.057540e-002
5 1089 1.398297e-002 3.857234e-001 1.143589e-003 1.451950e-002
6 4225 4.890709e-004 1.940675e-002 4.002749e-004 8.022336e-003

Fig. 2.4 Gaussian R B F interpolant with e = 21.1 at N = 289 (left) and at N = 1089 Halton
points (right).
24 Meshfree Approximation Methods with M A T L A B

I f we l o o k a t t h e e n t r i e s i n T a b l e 2 . 1 t h e n w e see t h a t c o n t r a r y t o w h a t w e
a n n o u n c e d e a r l i e r t h e r e s u l t s b a s e d o n t h e d i s t a n c e m a t r i x f i t are m o r e a c c u r a t e
t h a n those o b t a i n e d w i t h Gaussians. T h i s w i l l change, however, i f we t r y t o o p t i m i z e
our choice o f t h e shape p a r a m e t e r e (see t h e r e s u l t s o f t h e n e x t e x p e r i m e n t i n
Table 2.2).
I n a second e x p e r i m e n t w e c o n s i d e r t h e same t e s t f u n c t i o n a n d G a u s s i a n basis
functions. N o w , h o w e v e r , w e w a n t t o s t u d y t h e effects o f t h e s h a p e parameter.
T h e r e f o r e , i n F i g u r e 2.5 w e d i s p l a y b o t h t h e m a x i m u m a n d R M S e r r o r s as a f u n c t i o n
o f t h e s h a p e p a r a m e t e r e f o r four f i x e d d a t a sets ( 8 1 , 289, 1089 a n d 4225 H a l t o n
points). T h e s e curves r e v e a l s o m e o f t h e p r o b l e m s a s s o c i a t e d w i t h r a d i a l basis
function i n t e r p o l a t i o n especially w h e n w o r k i n g w i t h globally s u p p o r t e d basis
f u n c t i o n s , i.e., dense m a t r i c e s . W e see t h a t t h e e r r o r s decrease w i t h d e c r e a s i n g e
(of course, t h e y also decrease w i t h d e c r e a s i n g f i l l d i s t a n c e b u t t h a t is n o t w h a t
we are c o n c e r n e d w i t h n o w ) . H o w e v e r , t h e e r r o r c u r v e s are n o t m o n o t o n i c . W e c a n
i d e n t i f y a n o p t i m a l v a l u e o f e for w h i c h b o t h e r r o r s are m i n i m a l ( t h e m i n i m a o f
t h e t w o e r r o r c u r v e s o c c u r a t a l m o s t t h e s a m e p l a c e ) . M o r e o v e r , t h e r e is a v a l u e o f
e at w h i c h t h e c o m p u t a t i o n a l results become u n p r e d i c t a b l e , a n d t h e error curves
b e c o m e e r r a t i c . T h i s p o i n t is a s s o c i a t e d w i t h severe i l l - c o n d i t i o n i n g o f t h e s y s t e m
m a t r i x . Since M A T L A B issues a w a r n i n g w h e n a t t e m p t i n g t o solve a n i l l - c o n d i t i o n e d
l i n e a r s y s t e m , we refer t o t h e s m a l l e s t v a l u e o f e for w h i c h w e d o n o t see a M A T L A B
w a r n i n g as t h e "safe" v a l u e o f e ( f o r a g i v e n set X o f d a t a sites a n d basic f u n c t i o n
<p>). T h e i n t e r e s t i n g fact a b o u t t h e f o u r p l o t s d i s p l a y e d i n F i g u r e 2.5 is t h a t for t h e
s m a l l e r d a t a sets (N = 81 and N = 289) t h e m i n i m u m e r r o r s are o b t a i n e d f o r a
"safe" e, w h i l e for t h e l a r g e r sets ( i V = 1089 a n d N = 4225) the m i n i m u m errors
are o b t a i n e d i n t h e "unsafe" r a n g e . T h e r e f o r e , w e are c o m p u t i n g i n a c e r t a i n " g r a y
zone". W e are o b t a i n i n g h i g h l y a c c u r a t e s o l u t i o n s f r o m severely i l l - c o n d i t i o n e d
l i n e a r systems. W e w i l l c o m e b a c k l a t e r t o t h i s i n t e r e s t i n g f e a t u r e o f r a d i a l basis
f u n c t i o n i n t e r p o l a t i o n ( c a l l e d uncertainty o r trade-off principle). I t is c o n c e i v a b l e
(and i n fact possible [ F o r n b e r g a n d W r i g h t ( 2 0 0 4 ) ] ) t o o b t a i n even m o r e a c c u r a t e
r e s u l t s b y u s i n g a m o r e s t a b l e w a y t o e v a l u a t e t h e r a d i a l basis f u n c t i o n i n t e r p o l a n t
(see t h e d i s c u s s i o n i n C h a p t e r s 16 a n d 1 7 ) .

I n T a b l e 2.2 w e l i s t t h e "best p o s s i b l e " r e s u l t s f o r s t a t i o n a r y G a u s s i a n i n t e r p o -

l a t i o n . W e v i e w "best" i n t w o d i f f e r e n t w a y s . F o r t h e r e s u l t s p r e s e n t e d i n c o l u m n s
3 - 5 w e select for e a c h choice o f N t h e s m a l l e s t p o s s i b l e v a l u e o f e s u c h t h a t M A T L A B
does n o t issue a w a r n i n g . W e refer t o t h i s case as t h e "safe" e case i n T a b l e 2.2.
M o s t o f these e r r o r s are n o w c o m p a r a b l e ( o r s m a l l e r ) t h a n t h o s e for s i m p l e d i s t a n c e
m a t r i x i n t e r p o l a t i o n (c.f. T a b l e 2 . 1 ) .
These results, however, do n o t always represent t h e smallest achievable error.
T h e r e f o r e , we p r e s e n t ( i n c o l u m n s 6 - 8 ) r e s u l t s for t h o s e " o p t i m a l " values o f e w h i c h
y i e l d t h e s m a l l e s t R M S - e r r o r . T h e s e r e s u l t s are o b t a i n e d i n t h e " g r a y zone" m e n -
t i o n e d a b o v e . F o r e x a m p l e , i f w e use N = 1089 H a l t o n p o i n t s a n d s h a p e p a r a m e t e r
e = 6.2 t h e n M A T L A B issues a w a r n i n g w i t h RC0ND = 2 . 6 8 3 5 2 7 e - 0 2 0 . However,
 2. Radial Basis Function Interpolation in MATLAB 25

10

10 jl

10

10"

10"

10
10 15 20

Fig. 2.5 Maximum (dashed/top curve) and R M S (solid/bottom curve) errors vs. e for 81 (top
left), 289 (top right), 1089 (bottom left), and 4225 Halton points (bottom right).

Table 2.2 "Optimal" R B F interpolation to Franke's function using Gaussians.

smallest "safe" e smallest RMS-error

k N RMS-error max-error RMS-error max-error

1 9 0.02 3.658421e-001 1.580259e+000 2.23 1.118026e-001 3.450275e-001

2 25 0.32 3.629342e-001 2.845554e+000 3.64 4.032550e-002 2.996488e-001
3 81 1.64 1.743059e-001 2.398284e+000 4.28 1.090601e-002 1.579465e-001
4 289 4.73 2.785388e-003 5.472502e-002 5.46 4.610079e-004 7.978283e-003
5 1089 10.5 4.945428e-004 1.812246e-002 6.2 2.498848e-006 8.779119e-005
6 4225 21.1 4.890709e-004 1.940675e-002 6.3 4.269292e-008 8.889552e-007

as we c a n see i n T a b l e 2.2 a n d F i g u r e 2.5, t h e c o r r e s p o n d i n g e r r o r s are n o w much

s m a l l e r t h a n t h o s e p r e v i o u s l y o b t a i n e d . M o r e o v e r , t h e e r r o r s decrease a t a r a t e t h a t
is faster t h a n t h e 0(h) w e o b s e r v e d e a r l i e r for t h e d i s t a n c e m a t r i x fit e x a m p l e .
O f course, i f t h e d a t a w e are t r y i n g t o fit are n o t s a m p l e d f r o m a k n o w n t e s t
f u n c t i o n t h e n w e w i l l n o t be a b l e t o choose a n " o p t i m a l " s h a p e p a r a m e t e r b y m o n -
itoring the R M S error. T h e a s s o c i a t e d issues o f i l l - c o n d i t i o n i n g , p r e c o n d i t i o n i n g ,
o p t i m a l shape p a r a m e t e r selection, a n d a l t e r n a t e stable e v a l u a t i o n m e t h o d s v i a a
C o n t o u r - P a d e a l g o r i t h m are s t u d i e d l a t e r i n C h a p t e r s 16 a n d 17.
 Chapter 3

Positive Definite Functions

We noted i n the previous chapters t h a t the solution of the scattered d a t a interpola-

t i o n p r o b l e m w i t h R B F s boils d o w n t o the solution of a system of linear equations

Ac = y,

w h e r e t h e s y s t e m m a t r i x A has e n t r i e s <p(||Cj JCfclh), j , k = 1 , . . . , N. We know

f r o m linear a l g e b r a t h a t t h i s s y s t e m w i l l h a v e a u n i q u e s o l u t i o n w h e n e v e r t h e m a t r i x
A is n o n - s i n g u l a r . W h i l e n o one has y e t succeeded i n c h a r a c t e r i z i n g t h e class o f a l l
basic f u n c t i o n s <p t h a t g e n e r a t e a n o n - s i n g u l a r s y s t e m m a t r i x for a n y set X =
{xi,..., XN} o f d i s t i n c t d a t a sites, t h e s i t u a t i o n is m u c h b e t t e r i f we consider
positive definite matrices. *
I n t h i s c h a p t e r we p r e s e n t t h e m a i n t h e o r e t i c a l r e s u l t s u n d e r l y i n g t h i s a p p r o a c h
a l o n g w i t h some o f t h e i r p r o o f s . A series o f e x a m p l e s are p r e s e n t e d i n t h e next
c h a p t e r . A c o m p r e h e n s i v e t r e a t m e n t o f t h e m a t h e m a t i c a l t h e o r y needed f o r scat-
t e r e d d a t a i n t e r p o l a t i o n w i t h s t r i c t l y p o s i t i v e d e f i n i t e f u n c t i o n s (see Def. 3.2 b e l o w )
is p r e s e n t e d i n t h e recent m o n o g r a p h [ W e n d l a n d ( 2 0 0 5 a ) ] .

3.1 Positive Definite M a t r i c e s a n d Functions

D e f i n i t i o n 3.1. A r e a l s y m m e t r i c m a t r i x A is c a l l e d positive semi-definite i f its

associated q u a d r a t i c f o r m is n o n - n e g a t i v e , i.e.,

N N

Y^CjC A >Q
k jk (3.1)
3= 1 k=l

T N
for c = [ c i , . . .,c ] N G R .
I f t h e q u a d r a t i c f o r m ( 3 . 1 ) is zero o n l y for c = 0, t h e n A is c a l l e d positive
definite.

A n i m p o r t a n t p r o p e r t y o f p o s i t i v e d e f i n i t e m a t r i c e s is t h a t a l l t h e i r eigenvalues
are p o s i t i v e , a n d t h e r e f o r e a p o s i t i v e d e f i n i t e m a t r i x is n o n - s i n g u l a r ( b u t c e r t a i n l y
n o t vice v e r s a ) .

27
28 Meshfree Approximation Methods with MATLAB

I f we therefore h a d basis f u n c t i o n s B k i n the expansion (1.1) t h a t generate a

p o s i t i v e d e f i n i t e i n t e r p o l a t i o n m a t r i x , we w o u l d a l w a y s have a w e l l - p o s e d i n t e r p o -
l a t i o n p r o b l e m . T o t h i s e n d we i n t r o d u c e t h e c o n c e p t o f a positive definite function
f r o m classical analysis.
P o s i t i v e d e f i n i t e f u n c t i o n s w e r e first c o n s i d e r e d i n classical a n a l y s i s e a r l y i n t h e
2 0 t h century. [ M a t h i a s (1923)] seems t o h a v e b e e n t h e first t o define a n d study
positive definite functions. A n overview of the development of positive definite
f u n c t i o n s u p t o t h e m i d 1970s c a n b e f o u n d i n [ S t e w a r t ( 1 9 7 6 ) ] . H o w e v e r , as w e
see f r o m t h e d e f i n i t i o n b e l o w , p o s i t i v e d e f i n i t e f u n c t i o n s w e r e u n f o r t u n a t e l y
defined i n a n a l o g y t o p o s i t i v e s e m i - d e f i n i t e m a t r i c e s . Therefore, i n order to meet
o u r g o a l o f h a v i n g a w e l l - p o s e d i n t e r p o l a t i o n p r o b l e m , i t is necessary t o s h a r p e n t h e
classical n o t i o n o f a p o s i t i v e d e f i n i t e f u n c t i o n t o t h a t o f a strictly positive definite
one. T h i s c o n c e p t does n o t seem t o h a v e b e e n s t u d i e d u n t i l [ M i c c h e l l i (1986)] m a d e
the connection between scattered d a t a i n t e r p o l a t i o n and positive definite functions.
T h i s leads t o a n u n f o r t u n a t e difference i n t e r m i n o l o g y used i n t h e c o n t e x t o f m a t r i -
ces a n d f u n c t i o n s . Instead of r e w r i t i n g h i s t o r y we w i l l adhere t o this t e r m i n o l o g y
here. W e w o u l d l i k e t o p o i n t o u t t h a t w h e n r e a d i n g recent a r t i c l e s (especially i n
t h e r a d i a l basis f u n c t i o n l i t e r a t u r e ) d e a l i n g w i t h ( s t r i c t l y ) p o s i t i v e d e f i n i t e f u n c t i o n s
one has t o be a w a r e o f t h e fact t h a t some a u t h o r s h a v e t r i e d t o " c o r r e c t " h i s t o r y ,
a n d n o w refer t o s t r i c t l y p o s i t i v e d e f i n i t e f u n c t i o n s as p o s i t i v e d e f i n i t e f u n c t i o n s .

D e f i n i t i o n 3.2. A c o m p l e x - v a l u e d c o n t i n u o u s f u n c t i o n <fr : R s
- C is c a l l e d positive
definite on K. i fs

N N
(3.2)
3= 1 k=l

XN &
s N
for a n y N p a i r w i s e different p o i n t s x i , . . . , R , and c = [ c i , . . . , CJV] T
<C .
s
T h e f u n c t i o n <E> is c a l l e d strictly positive definite on M. i f t h e q u a d r a t i c form
(3.2) is zero o n l y for c = 0 .

W e n o t e t h a t even t h o u g h we are i n t e r e s t e d i n problems w i t h real data and

r e a l coefficients, a n e x t e n s i o n o f t h e n o t i o n o f p o s i t i v e deffniteness t o cover c o m p l e x
coefficients c a n d c o m p l e x - v a l u e d f u n c t i o n s <E> as d o n e i n D e f i n i t i o n 3.2 w i l l be h e l p -
f u l w h e n d e r i v i n g some p r o p e r t i e s o f ( s t r i c t l y ) p o s i t i v e d e f i n i t e f u n c t i o n s l a t e r o n .
M o r e o v e r , t h e c e l e b r a t e d Bochner's theorem (see T h e o r e m 3.3) c h a r a c t e r i z e s e x a c t l y
t h e p o s i t i v e d e f i n i t e f u n c t i o n s o f D e f i n i t i o n 3.2. I n a l l p r a c t i c a l c i r c u m s t a n c e s , h o w -
ever, we w i l l be c o n c e r n e d w i t h r e a l - v a l u e d f u n c t i o n s o n l y , a n d a characterization
o f such f u n c t i o n s a p p e a r s b e l o w as T h e o r e m 3.2. I t s h o u l d also be n o t e d t h a t Def-
i n i t i o n 3.2 i m p l i e s t h a t o n l y f u n c t i o n s w h o s e q u a d r a t i c f o r m is r e a l are c a n d i d a t e s
for ( s t r i c t l y ) p o s i t i v e d e f i n i t e f u n c t i o n s .

E x a m p l e 3.1. H e r e , a n d t h r o u g h o u t t h i s b o o k , we w i l l d e n o t e t h e s t a n d a r d i n n e r
t x y
product of x and y i n R s
b y x y. W i t h t h i s n o t a t i o n t h e f u n c t i o n <fr(cc) = e ,
 3. Positive Definite Functions 29

for y G R s
fixed, is p o s i t i v e d e f i n i t e o n R s
since t h e q u a d r a t i c f o r m i n D e f i n i t i o n 3.2
becomes
N N N N

j=l k=l j=l k=l

N N

3=1 k=l
2
N
> 0.
3= 1

D e f i n i t i o n 3.2 a n d t h e discussion p r e c e d i n g i t suggest t h a t we s h o u l d use s t r i c t l y

p o s i t i v e d e f i n i t e f u n c t i o n s as basis f u n c t i o n s i n ( 1 . 1 ) , i.e., Bk(x) $>(x X k ) , or
N
Xk), a; G R . (3.3)
fc=i
N o t e t h a t at t h i s p o i n t w e d o n o t r e q u i r e $ t o be a r a d i a l f u n c t i o n . I n fact,
t h e f u n c t i o n Vf o f (3.3) w i l l y i e l d a n i n t e r p o l a n t t h a t is translation invariant, i.e.,
t h e i n t e r p o l a n t t o t r a n s l a t e d d a t a is t h e same as t h e t r a n s l a t e d i n t e r p o l a n t t o t h e
o r i g i n a l d a t a . I n o r d e r t o o b t a i n i n v a r i a n c e also u n d e r r o t a t i o n s a n d reflections we
w i l l l a t e r specialize t o s t r i c t l y p o s i t i v e d e f i n i t e f u n c t i o n s t h a t are also r a d i a l o n R . s

W e w i l l n o w discuss some o f t h e m o s t i m p o r t a n t p r o p e r t i e s a n d characterizations

of ( s t r i c t l y ) p o s i t i v e d e f i n i t e f u n c t i o n s . F o r t h e sake o f completeness we present a l i s t
of some basic p r o p e r t i e s o f ( s t r i c t l y ) p o s i t i v e d e f i n i t e f u n c t i o n s a n d some e x a m p l e s .

T h e o r e m 3.1. Some basic properties of positive definite functions are

(1) Non-negative finite linear combinations of positive definite functions are

positive definite. i/$i,...,$ n are positive definite on R s
and Cj > 0,
j 1 , . . . , n, then
n
$(as) = ^2cj$j(x), xGR , s

3= 1

is also positive definite. Moreover, if at least one of the <&j is strictly positive
definite and the corresponding c - > 0, then $ is strictly
3 positive definite.
(2) $ ( 0 ) > 0.
(3) $ ( - J T ) - $(aj).
(4) Any positive definite function is bounded. In fact,

\(x)\ < $(0).

(5) If is positive definite with $ ( 0 ) = 0 then $ = 0.

(6) The product of (strictly) positive definite functions is (strictly) positive def-
inite.
30 Meshfree Approximation Methods with MATLAB

Proof. P r o p e r t i e s (1) a n d (2) f o l l o w i m m e d i a t e l y f r o m D e f i n i t i o n 3.2.

T o s h o w (3) w e l e t N = 2, x\ = 0, x 2 = x, a n d choose c\ = 1 a n d c 2 = c. T h e n
t h e q u a d r a t i c f o r m i n D e f i n i t i o n 3.2 becomes
2 2
2
^ cjcT^ixj - x) k = (1 + | c | ) 3 > ( 0 ) + c&(x) + c$(-x) > 0
i = i fc=i

for e v e r y c G C. T a k i n g c = 1 a n d c = z ( w h e r e i = y/ 1 ) , r e s p e c t i v e l y , w e c a n see
t h a t b o t h <3?(a;) + <&(ic) a n d z ( $ ( x ) <&(cc)) m u s t be r e a l . T h i s , h o w e v e r , is o n l y
possible i f <E>(x) = <fr(x).
F o r t h e p r o o f o f (4) w e l e t N = 2, X\ 0, x 2 = x, a n d choose c\ = | $ ( a ; ) | a n d
c 2 = Q(x). T h e n t h e q u a d r a t i c f o r m i n D e f i n i t i o n 3.2 is

2 2
^ r ^ T c j - c ^ ^ - x) k = 2<Z>(0)\$(x)\ 2 2
- $ ( - a j ) $ ( x ) | $ ( x ) | - $ (x)\(x)\ > 0.
j=ik=i

Since <&(a:) = &(x) b y P r o p e r t y ( 3 ) , t h i s gives

2 3
2$(0)\$(x)\ - 2\(x)\ > 0.

2
I f | $ ( x ) | > 0, w e d i v i d e b y |<3?(;E)| a n d t h e s t a t e m e n t f o l l o w s i m m e d i a t e l y . I n case
|<E>(a?)| = 0 t h e s t a t e m e n t h o l d s t r i v i a l l y .
P r o p e r t y (5) f o l l o w s i m m e d i a t e l y f r o m ( 4 ) , a n d P r o p e r t y (6) is a consequence o f a
t h e o r e m b y Schur i n t h e field o f l i n e a r a l g e b r a w h i c h s t a t e s t h a t t h e e l e m e n t w i s e (or
H a d a m a r d ) p r o d u c t o f p o s i t i v e ( s e m i - ) d e f i n i t e m a t r i c e s is p o s i t i v e ( s e m i - ) d e f i n i t e .
For m o r e d e t a i l s w e refer t h e r e a d e r t o [ C h e n e y a n d L i g h t (1999)] o r [ W e n d l a n d
(2005a)].

E x a m p l e 3.2. T h e cosine f u n c t i o n is p o s i t i v e d e f i n i t e o n K since, for x G R, w e

xx lx
h a v e cos a; = ^ (e + e~ ). N o w P r o p e r t y (1) a n d E x a m p l e 3 . 1 c a n be i n v o k e d .

P r o p e r t y (3) shows t h a t a n y r e a l - v a l u e d ( s t r i c t l y ) p o s i t i v e d e f i n i t e f u n c t i o n has

t o be even. H o w e v e r , i t is also possible t o c h a r a c t e r i z e r e a l - v a l u e d ( s t r i c t l y ) p o s i t i v e
d e f i n i t e f u n c t i o n s u s i n g o n l y real coefficients (see [ W e n d l a n d (2005a)] for d e t a i l s ) ,
i.e.,

T h e o r e m 3.2. A real-valued continuous function <fr is positive definite on R s

if and
only if it is even and

N N
c c
^2 j k<&(xj - x k ) > 0 (3.4)
3 = 1 fc=l
s T N
for any N pairwise different points Xi,..., x^ G M. , and c = [ c i , . . . , C j v ] G M. .
The function $ is strictly positive definite on R s
if the quadratic form (3.4) is
zero only for c = 0.
 3. Positive Definite Functions 31

3.2 I n t e g r a l C h a r a c t e r i z a t i o n s for ( S t r i c t l y ) P o s i t i v e Definite

Functions

W e w i l l n o w s u m m a r i z e some facts a b o u t i n t e g r a l c h a r a c t e r i z a t i o n s o f p o s i t i v e def-

inite functions. T h e y were e s t a b l i s h e d i n t h e 1930s b y B o c h n e r a n d S c h o e n b e r g .
H o w e v e r , we w i l l also m e n t i o n t h e m o r e recent e x t e n s i o n s t o s t r i c t l y p o s i t i v e defi-
n i t e a n d s t r i c t l y c o m p l e t e l y / m u l t i p l y m o n o t o n e f u n c t i o n s t h a t are essential t o t h e
application of the theory t o the scattered d a t a i n t e r p o l a t i o n problem. A much
m o r e d e t a i l e d discussion o f t h i s m a t e r i a l is p r e s e n t e d i n t h e recent b o o k [ W e n d -
l a n d (2005a)]. Some f r e q u e n t l y used i n t e g r a l t r a n s f o r m s are l i s t e d i n A p p e n d i x B .
I n t e g r a l c h a r a c t e r i z a t i o n s o f t h e closely r e l a t e d c o m p l e t e l y a n d m u l t i p l y monotone
f u n c t i o n s are presented i n C h a p t e r 5.

3.2.1 Bochner''s Theorem

O n e o f t h e m o s t c e l e b r a t e d r e s u l t s o n p o s i t i v e d e f i n i t e f u n c t i o n s is t h e i r c h a r a c t e r -
i z a t i o n i n t e r m s o f F o u r i e r t r a n s f o r m s e s t a b l i s h e d b y B o c h n e r i n 1932 (for s = 1)
a n d 1933 (for g e n e r a l s).

S
T h e o r e m 3 . 3 ( B o c h n e r ) . A (complex-valued) function <E> e C ( I R ) is positive def-
s
inite on R if and only if it is the Fourier transform of a finite non-negative Borel
s
measure fi on R , i.e.

<2>(cc) = jl(x)
1
J ,ixy x e

Proof. T h e r e are m a n y p r o o f s o f t h i s t h e o r e m . B o c h n e r ' s o r i g i n a l p r o o f c a n be

f o u n d i n [ B o c h n e r (1933)]. O t h e r p r o o f s c a n be f o u n d , e.g., i n t h e b o o k s [ C u p p e n s
(1975)] or [ G e l ' f a n d a n d V i l e n k i n ( 1 9 6 4 ) ] . A p r o o f u s i n g t h e Riesz r e p r e s e n t a t i o n
t h e o r e m t o i n t e r p r e t t h e B o r e l m e a s u r e as a d i s t r i b u t i o n , a n d t h e n t a k i n g a d v a n t a g e
of d i s t r i b u t i o n a l F o u r i e r t r a n s f o r m s c a n be f o u n d i n t h e b o o k [ W e n d l a n d ( 2 0 0 5 a ) ] .
W e w i l l p r o v e o n l y t h e one (easy) d i r e c t i o n . I t is t h i s p a r t o f t h e s t a t e m e n t t h a t
is i m p o r t a n t for t h e a p p l i c a t i o n t o s c a t t e r e d d a t a i n t e r p o l a t i o n . W e assume <fr is
t h e F o u r i e r t r a n s f o r m o f a f i n i t e n o n - n e g a t i v e B o r e l m e a s u r e a n d show <E> is p o s i t i v e
definite. Thus,
N N N N
^^TcjCTQiXj
1=1 k=l
- x)
k =
:
EE
j=l k=X
CjC k J ,-i(xj-x )-y
k
d^(y)

N N
1
J E Cj e -zxj-y
E
fc=i
Cite
ix y
k
dLi(y)
Jut*
J=l

J E N
Cje
-txj-y dfj,(y) > 0.
32 Meshfree Approximation Methods with MATLAB

T h e last i n e q u a l i t y h o l d s because o f t h e c o n d i t i o n s i m p o s e d o n t h e m e a s u r e \x.

t x y
R e m a r k 3 . 1 . W e c a n see f r o m T h e o r e m 3.3 t h a t t h e f u n c t i o n $(x) = e of
E x a m p l e 3.1 c a n be c o n s i d e r e d as t h e fundamental positive definite function since
a l l o t h e r p o s i t i v e d e f i n i t e f u n c t i o n s are o b t a i n e d as ( i n f i n i t e ) l i n e a r c o m b i n a t i o n s o f
t h i s f u n c t i o n . W h i l e P r o p e r t y (1) o f T h e o r e m 3.1 i m p l i e s t h a t l i n e a r c o m b i n a t i o n s
o f <& w i l l a g a i n b e p o s i t i v e d e f i n i t e , t h e r e m a r k a b l e c o n t e n t o f B o c h n e r ' s Theorem
is t h e fact t h a t i n d e e d all p o s i t i v e d e f i n i t e f u n c t i o n s are g e n e r a t e d b y

3.2.2 Extensions to Strictly Positive Definite Functions

I n order t o accomplish our goal o f guaranteeing a well-posed i n t e r p o l a t i o n p r o b l e m

we have t o e x t e n d ( i f possible) B o c h n e r ' s c h a r a c t e r i z a t i o n t o strictly positive definite
functions.
W e b e g i n w i t h a sufficient c o n d i t i o n for a f u n c t i o n t o be s t r i c t l y p o s i t i v e d e f i n i t e
s
on R .
F o r t h i s r e s u l t we r e q u i r e t h e n o t i o n o f t h e carrier of a (non-negative) Borel
m e a s u r e d e f i n e d o n some t o p o l o g i c a l space X (see also A p p e n d i x B ) . T h i s set is
given by

X \ \ J { 0 : O is o p e n a n d fi(0) = 0}.

s
T h e o r e m 3 . 4 . Let /J, be a non-negative finite Borel measure on R whose carrier
is a set of nonzero Lebesgue measure. Then the Fourier transform of /j, is strictly
s
positive definite on R .

Proof. A s i n the p r o o f o f Bochner's t h e o r e m we have

N N N N
d^{y)
3= 1 k=l VW) j = 1 k =i
N N
= _ L _ / c e i X j y e i X k dfi(y)
j 2 3 ~ Y l ^
3=1 k=l
2
\/(2jr) A t
d(y) > 0.
3= 1

N o w let v/(2ir) h

AT
c e i X i v
g{y) = Y. i ~ ' '
3= 1

a n d assume t h a t t h e p o i n t s xj are a l l d i s t i n c t a n d c ^ O . I n t h i s case t h e f u n c t i o n s

lXj y
y i* e~ ' are l i n e a r l y i n d e p e n d e n t so t h a t g ^ 0. Since g is a n e n t i r e f u n c t i o n i t s
s
zero set, i.e., {y R : g(y) = 0 } can have no accumulation p o i n t and therefore
i t has Lebesgue m e a s u r e zero (see, e.g., [ C h e n e y a n d L i g h t ( 1 9 9 9 ) ] ) . N o w , t h e o n l y
 3. Positive Definite Functions 33

r e m a i n i n g w a y t o m a k e t h e a b o v e i n e q u a l i t y a n e q u a l i t y is i f t h e c a r r i e r o f fi is
c o n t a i n e d i n t h e zero set o f g, i.e., has Lebesgue m e a s u r e zero. T h i s , however, is
ruled out i n the hypothesis o f the theorem.

W o r k t o w a r d a n a n a l o g o f B o c h n e r ' s t h e o r e m , i.e., a c o m p l e t e i n t e g r a l charac-

s
t e r i z a t i o n o f f u n c t i o n s t h a t are s t r i c t l y p o s i t i v e d e f i n i t e o n Wt , is g i v e n i n [ C h a n g
(1996)] for t h e case s = 1.
T h e f o l l o w i n g c o r o l l a r y gives us a w a y t o construct s t r i c t l y positive definite
functions.

s
C o r o l l a r y 3 . 1 . Let f be a continuous non-negative function in Li(R ) which is not
s
identically zero. Then the Fourier transform of f is strictly positive definite on M. .

Proof. T h i s is a special case o f t h e p r e v i o u s t h e o r e m i n w h i c h t h e m e a s u r e fj, has

Lebesgue d e n s i t y / . T h u s , w e use t h e m e a s u r e \i defined for a n y B o r e l set B b y

//() = ( f(x)dx.
JB
T h e n t h e c a r r i e r o f fi is e q u a l t o t h e (closed) s u p p o r t o f / . H o w e v e r , since / is
n o n - n e g a t i v e a n d n o t i d e n t i c a l l y e q u a l t o zero, i t s s u p p o r t has p o s i t i v e L e b e s g u e
measure, a n d hence t h e F o u r i e r t r a n s f o r m o f / is s t r i c t l y p o s i t i v e d e f i n i t e b y t h e
preceding theorem.

F i n a l l y , a c r i t e r i o n t o check w h e t h e r a g i v e n f u n c t i o n is s t r i c t l y p o s i t i v e d e f i n i t e
is g i v e n i n [ W e n d l a n d ( 2 0 0 5 a ) ] .

s
T h e o r e m 3.5. Let <& be a continuous function in L i ( R ) . <& is strictly positive
definite if and only if $ is bounded and its Fourier transform is non-negative and
not identically equal to zero.

T h e o r e m 3.5 is o f f u n d a m e n t a l i m p o r t a n c e a n d we w i l l c o m e b a c k t o t h i s t h e o r e m
several t i m e s l a t e r o n . I n f a c t , t h e p r o o f o f T h e o r e m 3.5 i n [ W e n d l a n d (2005a)] shows
t h a t i f <& ^ 0 ( w h i c h i m p l i e s t h a t t h e n also <E ^ 0) w e need t o ensure o n l y
t h a t <l> be n o n - n e g a t i v e i n o r d e r for <3> t o be s t r i c t l y p o s i t i v e d e f i n i t e .

3.3 Positive Definite R a d i a l Functions

W e n o w t u r n o u r a t t e n t i o n t o p o s i t i v e d e f i n i t e radial functions. Recall t h a t Def-

i n i t i o n 3.2 characterizes ( s t r i c t l y ) p o s i t i v e d e f i n i t e f u n c t i o n s in terms of multi-
v a r i a t e f u n c t i o n s <. H o w e v e r , w h e n we are d e a l i n g w i t h r a d i a l f u n c t i o n s , i.e.,
<&(x) = (p(\\x\\), t h e n i t w i l l be c o n v e n i e n t t o also refer t o t h e u n i v a r i a t e f u n c t i o n
<p as a positive definite radial function. W h i l e t h i s does present a s l i g h t abuse o f
our t e r m i n o l o g y for p o s i t i v e d e f i n i t e f u n c t i o n s t h i s is w h a t is c o m m o n l y d o n e i n t h e
literature.
A n i m m e d i a t e consequence o f t h i s n o t a t i o n a l c o n v e n t i o n is
34 Meshfree Approximation Methods with MATLAB

L e m m a 3 . 1 . If ^ p(\\ ||) is (strictly) positive definite and radial on R s

then <fr
CT
is also (strictly) positive definite and radial on R for any a < s.

We now r e t u r n to integral characterizations and begin w i t h a theorem due t o

Schoenberg (see, e.g., [Schoenberg ( 1 9 3 8 a ) ] , p . 8 1 6 , o r [ W e l l s a n d W i l l i a m s ( 1 9 7 5 ) ] ,
p.27).

T h e o r e m 3 . 6 . A continuous function ip : [0, oo) * 3R is positive definite and radial

s
on R if and only if it is the Bessel transform of a finite non-negative Borel measure
/ i on [0, o o ) , i.e.

Here

for s = 1,
for s>2,

5
and J ( _ 2 ) / 2 ^ the classical
s Bessel function of the first kind of order (s 2 ) / 2 .

A s above, n o w t h e f u n c t i o n <E>(a;) = cos(a:) f r o m E x a m p l e 3.2 c a n be v i e w e d

as t h e f u n d a m e n t a l positive definite r a d i a l f u n c t i o n o n R . W e w i l l see b e l o w ( i n
E x a m p l e 3 o f C h a p t e r 4) t h a t t h e c h a r a c t e r i z a t i o n o f T h e o r e m 3.6 immediately
suggests a class o f (even s t r i c t l y ) p o s i t i v e d e f i n i t e r a d i a l f u n c t i o n s . A s for t h e basic
I D e x a m p l e , t h e m e a s u r e fi w i l l s i m p l y be a p o i n t e v a l u a t i o n m e a s u r e .
A Fourier t r a n s f o r m characterization of s t r i c t l y positive definite r a d i a l functions
s
on R c a n be f o u n d i n [ W e n d l a n d ( 2 0 0 5 a ) ] . I t is e s s e n t i a l l y a c o m b i n a t i o n o f T h e o -
r e m 3.5 a n d t h e f o r m u l a i n T h e o r e m B . l o f A p p e n d i x B for t h e F o u r i e r t r a n s f o r m
of a radial function:

T h e o r e m 3.7. A continuous function <p : [0, oo) * R such that r t> r ~ cp(r) s 1
e
s
L\[0, oo) is strictly positive definite and radial on R if and only if the s-dimensional
Fourier transform

is non-negative and not identically equal to zero.

Since L e m m a 3.1 states t h a t a n y f u n c t i o n t h a t is ( s t r i c t l y ) p o s i t i v e d e f i n i t e a n d

s CT
radial on R is also ( s t r i c t l y ) p o s i t i v e d e f i n i t e a n d r a d i a l o n R for a n y a < s,
s
t h o s e f u n c t i o n s w h i c h are ( s t r i c t l y ) p o s i t i v e d e f i n i t e a n d r a d i a l o n R for all s are
s
of p a r t i c u l a r i n t e r e s t . T h e class o f f u n c t i o n s t h a t are p o s i t i v e d e f i n i t e o n R for a l l
s was also c h a r a c t e r i z e d b y S c h o e n b e r g ( [ S c h o e n b e r g (1938a)], pp. 817-821). An
e x t e n s i o n t o t h e s t r i c t l y p o s i t i v e d e f i n i t e case c a n be f o u n d i n [ M i c c h e l l i ( 1 9 8 6 ) ] :
 3. Positive Definite Functions 35

T h e o r e m 3.8 ( S c h o e n b e r g ) . A continuous function ip : [ 0 , o o ) R is strictly

S
positive definite and radial on R for all s if and only if it is of the form

Jo
where fi is a finite non-negative Borel measure on [0, oo) not concentrated at the
origin.

A s suggested for T h e o r e m 3.6 above, l e t t i n g fi be a p o i n t e v a l u a t i o n m e a s u r e i n

T h e o r e m 3.8 we o b t a i n t h a t t h e G a u s s i a n is s t r i c t l y p o s i t i v e d e f i n i t e a n d r a d i a l o n
S
R for a l l s (c.f Example 1 of Chapter 4).
T h e Schoenberg c h a r a c t e r i z a t i o n o f ( s t r i c t l y ) p o s i t i v e d e f i n i t e r a d i a l f u n c t i o n s o n
S
R for a l l s ( T h e o r e m 3.8) i m p l i e s t h a t we have a f i n i t e n o n - n e g a t i v e B o r e l m e a s u r e
o n [0, oo) such t h a t

I f we w a n t t o find a zero r n o f cp t h e n we have t o solve

Since t h e e x p o n e n t i a l f u n c t i o n is p o s i t i v e a n d t h e m e a s u r e is n o n - n e g a t i v e , i t f o l l o w s
t h a t /J, m u s t be t h e zero m e a s u r e . H o w e v e r , t h e n cp is i d e n t i c a l l y e q u a l t o zero.
S
T h e r e f o r e , a n o n - t r i v i a l f u n c t i o n cp t h a t is p o s i t i v e d e f i n i t e a n d r a d i a l o n R for a l l
s c a n have n o zeros. T h i s i m p l i e s i n p a r t i c u l a r t h a t

T h e o r e m 3.9. There are no oscillatory univariate continuous functions that are

S
strictly positive definite and radial on R for all s. Moreover, there are no com-
pactly supported univariate continuous functions that are strictly positive definite
S
and radial on R for all s.

A n e q u i v a l e n t a r g u m e n t for t h e o s c i l l a t o r y case is g i v e n i n T h e o r e m 2.3 o f [ F o r n -

b e r g et al. ( 2 0 0 4 ) ] .
 Chapter 4

Examples of Strictly Positive Definite

Radial Functions

W e n o w present a n u m b e r o f f u n c t i o n s t h a t are covered b y t h e t h e o r y p r e s e n t e d

t h u s far. W h i l e i t is possible t o i n c l u d e a shape parameter e for a l l o f t h e f u n c t i o n s
presented i n the examples below b y rescaling x t o ex, we a v o i d i t s use i n t h e
f o r m u l a t i o n o f a l l b u t t h e G a u s s i a n e x a m p l e t o keep t h e f o r m u l a s as s i m p l e as
possible. W e d o , however, use a shape p a r a m e t e r w h e n p l o t t i n g some o f t h e basis
functions.
O u r use o f t h e shape p a r a m e t e r does n o t a l w a y s m a t c h i t s " t r a d i t i o n a l " use.
F o r e x a m p l e , H a r d y i n t r o d u c e d his inverse m u l t i q u a d r i c s (see E x a m p l e 5 b e l o w )
2 2
i n t h e f o r m <&(||a:||) = l/\/c + \\x\\ w i t h shape p a r a m e t e r c. I t is, o f course,
straightforward to transform this representation t o t h e one suggested above, i.e.,
x = 2 2 2 2
^Kll II) + ||cc|| , by setting c = 1/e a n d scaling the result by
O u r use o f t h e shape p a r a m e t e r as a f a c t o r a p p l i e d d i r e c t l y t o x has t h e a d v a n -
t a g e o f p r o v i d i n g a u n i f i e d t r e a t m e n t i n w h i c h a decrease o f t h e shape p a r a m e t e r
always has t h e effect o f p r o d u c i n g " f i a t " basis f u n c t i o n s , w h i l e i n c r e a s i n g e leads t o
m o r e p e a k e d (or l o c a l i z e d ) basis f u n c t i o n s .

4.1 E x a m p l e 1: Gaussians

W e c a n n o w show t h a t t h e Gaussian

2 x | 1 2
&(x) = e- H , > 0 , (4.1)

is s t r i c t l y p o s i t i v e d e f i n i t e ( a n d r a d i a l ) o n K s
for a n y s. T h i s is d u e t o t h e fact t h a t
t h e F o u r i e r t r a n s f o r m o f a G a u s s i a n is essentially a G a u s s i a n . I n fact,

, 1

a n d t h i s is p o s i t i v e i n d e p e n d e n t o f t h e space d i m e n s i o n s. I n p a r t i c u l a r , for e =
w e have <E> = P l o t s o f G a u s s i a n R B F s were p r e s e n t e d i n F i g . 2 . 1 . C l e a r l y , t h e
Gaussians are i n f i n i t e l y d i f f e r e n t i a b l e . S o m e o f i t s d e r i v a t i v e s (as w e l l as t h o s e o f
m a n y o t h e r R B F s ) are c o l l e c t e d i n A p p e n d i x D .

37
38 Meshfree Approximation Methods with MATLAB

A n o t h e r a r g u m e n t t o s h o w t h a t G a u s s i a n s are s t r i c t l y p o s i t i v e d e f i n i t e a n d r a d i a l
S
on R for a n y s t h a t avoids d e a l i n g w i t h F o u r i e r t r a n s f o r m s w i l l b e c o m e a v a i l a b l e
l a t e r . I t w i l l m a k e use o f c o m p l e t e l y m o n o t o n e f u n c t i o n s .
R e c a l l t h a t P r o p e r t y ( 1 ) o f T h e o r e m 3.1 shows t h a t a n y finite n o n - n e g a t i v e l i n -
ear c o m b i n a t i o n o f ( s t r i c t l y ) p o s i t i v e d e f i n i t e f u n c t i o n s is a g a i n ( s t r i c t l y ) p o s i t i v e
d e f i n i t e . M o r e o v e r , w e j u s t saw t h a t G a u s s i a n s are s t r i c t l y p o s i t i v e d e f i n i t e a n d r a -
S
d i a l o n a l l R . N o w , t h e S c h o e n b e r g c h a r a c t e r i z a t i o n o f f u n c t i o n s t h a t are ( s t r i c t l y )
S
p o s i t i v e d e f i n i t e a n d r a d i a l o n a n y R , T h e o r e m 3.8, s t a t e s t h a t all s u c h f u n c t i o n s
are g i v e n as i n f i n i t e l i n e a r c o m b i n a t i o n s o f G a u s s i a n s . T h e r e f o r e , t h e G a u s s i a n s c a n
be v i e w e d as t h e f u n d a m e n t a l m e m b e r o f t h e f a m i l y o f f u n c t i o n s t h a t are s t r i c t l y
S
positive definite and radial on R for a l l s.
Since Gaussians p l a y a c e n t r a l r o l e i n s t a t i s t i c s t h i s is a g o o d p l a c e t o m e n t i o n
t h a t p o s i t i v e d e f i n i t e f u n c t i o n s are u p t o a n o r m a l i z a t i o n $ ( 0 ) = 1 i d e n t i c a l
w i t h characteristic functions of d i s t r i b u t i o n functions i n statistics.

4.2 E x a m p l e 2: L a g u e r r e - G a u s s i a n s

I n o r d e r t o o b t a i n a g e n e r a l i z a t i o n o f G a u s s i a n s w e s t a r t w i t h t h e generalized La-
2
guerre polynomials L % / o f degree n a n d o r d e r s/2 d e f i n e d b y t h e i r R o d r i g u e s f o r m u l a
(see, e.g., f o r m u l a (6.2.1) i n [ A n d r e w s et al. ( 1 9 9 9 ) ] )
tj. s/2 JTl . .

L / 2 ( i ) = i e < n + / 2
V ^ ( " ' ' ) ' "
A n e x p l i c i t f o r m u l a for t h e g e n e r a l i z e d L a g u e r r e p o l y n o m i a l s is

W e t h e n define t h e Laguerre- Gaussians

2
= c-H-II'L^dlasH ), (4.2)

a n d list t h e i r F o u r i e r t r a n s f o r m s as
_ H"ll 2
n n | | 2 l '

<KuO = ^ V M ^ > 0 . (4.3)

N o t e t h a t t h e d e f i n i t i o n o f t h e L a g u e r r e - G a u s s i a n s d e p e n d s o n t h e space d i m e n s i o n
S
s. T h e r e f o r e t h e y are s t r i c t l y p o s i t i v e d e f i n i t e a n d r a d i a l o n R ( a n d b y L e m m a 3.1
C T
also o n R for a n y a < s).
Laguerre-Gaussian f u n c t i o n s for some s p e c i a l choices o f s a n d n are l i s t e d i n
T a b l e 4 . 1 . F i g u r e 4 . 1 shows a L a g u e r r e - G a u s s i a n for s l , n = 2, a n d for s =
2 , n = 2 d i s p l a y e d w i t h a shape p a r a m e t e r e = 3 a n d scaled so t h a t <&(0) = 1.
M o r e o v e r , t h e L a g u e r r e - G a u s s i a n s are i n f i n i t e l y s m o o t h for a l l choices o f n a n d s.
N o t e t h a t t h e L a g u e r r e - G a u s s i a n s ( w h i l e b e i n g s t r i c t l y p o s i t i v e d e f i n i t e func-
t i o n s ) are n o t p o s i t i v e . Since t h e L a g u e r r e - G a u s s i a n s are o s c i l l a t o r y f u n c t i o n s w e

^ 1
 4- Examples of Strictly Positive Definite Radial Functions 39

k n o w f r o m T h e o r e m 3.9 t h a t t h e y c a n n o t be s t r i c t l y p o s i t i v e d e f i n i t e a n d r a d i a l o n
s
R for a l l s. W e w i l l e n c o u n t e r these f u n c t i o n s l a t e r i n t h e c o n t e x t o f a p p r o x i m a t e
m o v i n g least squares a p p r o x i m a t i o n (c.f. Chapter 26).

Table 4.1 Laguerre-Gaussians for various choices of s and n.

s n = 1 n = 2

Fig. 4.1 Laguerre-Gaussians with s = l,n = 2 (left) and s = 2, n = 2 (right) centered at the
origin.

4.3 E x a m p l e 3: P o i s s o n R a d i a l Functions

A n o t h e r class o f o s c i l l a t o r y f u n c t i o n s t h a t are s t r i c t l y p o s i t i v e d e f i n i t e a n d r a d i a l
s CT
on R (and all R for a < s) w e r e r e c e n t l y s t u d i e d b y F o r n b e r g a n d c o - w o r k e r s (see
[ F o r n b e r g et al. (2004)] a n d also [ F l y e r ( 2 0 0 6 ) ] ) . T h e s e f u n c t i o n s are o f t h e f o r m

*(*> = ^ P . (*<)

where J u is t h e Bessel f u n c t i o n o f t h e first k i n d o f o r d e r v. W h i l e these f u n c t i o n s

are n o t defined a t t h e o r i g i n t h e y c a n be e x t e n d e d t o be i n f i n i t e l y d i f f e r e n t i a b l e i n
s
allofR .
T h e f u n c t i o n s (4.4) were a l r e a d y s t u d i e d b y S c h o e n b e r g (see t h e d i s c u s s i o n s u r -
r o u n d i n g T h e o r e m 3.6) w h o s u g g e s t e d c a l l i n g t h e m Poisson functions. I n fact, t h e
s _ 2 2
f u n c t i o n s i n (4.4) are ( u p t o t h e scale f a c t o r 2 ^ ^ r ( s / 2 ) ) t h e f u n c t i o n s fl s of

^1. t
40 Meshfree Approximation Methods with MATLAB

T h e o r e m 3.6 a n d t h e r e f o r e c a n be v i e w e d as t h e f u n d a m e n t a l m e m b e r o f t h e f a m i l y
S
o f f u n c t i o n s t h a t are s t r i c t l y p o s i t i v e d e f i n i t e a n d r a d i a l o n R for f i x e d s.
S c h o e n b e r g s h o w e d t h a t t h e f u n c t i o n s f 2 are g i v e n b y
s

Q (x) s = _L_ f e dcr,ixa

1 S
w h e r e co -i
s denotes t h e area o f t h e u n i t s p h e r e S**" i n R , a n d da d e n o t e s t h e
s - 1
usual measure o n S ' .
T h e Poisson f u n c t i o n s are a n o t h e r g e n e r a l i z a t i o n o f G a u s s i a n s ( t h e f u n d a m e n t a l
S
s t r i c t l y positive definite radial function o n R for a l l s) since t h e f o l l o w i n g l i m i t -
i n g r e l a t i o n due t o J o h n v o n N e u m a n n h o l d s (see t h e d i s c u s s i o n i n [Schoenberg
(1938a)]):

- 7 - 2
l i m n (rV2s)
s = e .
s>oo
Since t h e P o i s s o n r a d i a l f u n c t i o n s are d e f i n e d i n t e r m s o f Bessel f u n c t i o n s t h e y are
also band-limited, i.e., t h e i r F o u r i e r t r a n s f o r m has c o m p a c t s u p p o r t . I n fact, t h e
F F
F o u r i e r t r a n s f o r m o f <E> i n R , a < s, is g i v e n b y (see [ F l y e r ( 2 0 0 6 ) ] )

Some o f these P o i s s o n f u n c t i o n s are l i s t e d i n T a b l e 4.2 a n d d i s p l a y e d i n F i g u r e 4.2

( w h e r e a shape p a r a m e t e r e 10 was used for t h e p l o t s ) .

Table 4.2 Poisson functions for various choices of s.

s = 2 s = 3 s = 4 s = 5

r2 s i n (11*11) MINI) /TsindlccH) - ll^ll c o s M )

Jo(IMI)
V n | X \\ 11*11 V 7T \\ \\
x 3

2
Fig. 4.2 Poisson functions with s = 2 (left) and s 3 (right) centered at the origin in R .
 4- Examples of Strictly Positive Definite Radial Functions 41

4.4 E x a m p l e 4: M a t e r n Functions

A f o u r t h e x a m p l e o f s t r i c t l y p o s i t i v e d e f i n i t e f u n c t i o n s is g i v e n b y t h e class o f
Matern functions w h i c h are q u i t e c o m m o n i n t h e s t a t i s t i c s l i t e r a t u r e (see, e.g.,
[ M a t e r n (1986)] or [Stein ( 1 9 9 9 ) ] )

Here K v is t h e modified Bessel function of the second kind ( s o m e t i m e s also c a l l e d

m o d i f i e d Bessel f u n c t i o n o f t h e t h i r d k i n d , or M a c D o n a l d ' s f u n c t i o n ) of order v.
T h e F o u r i e r t r a n s f o r m o f t h e M a t e r n f u n c t i o n s is g i v e n b y t h e Bessel kernels
2
<l(uO = ( i + M ) ^ > o.
S
T h e r e f o r e t h e M a t e r n f u n c t i o n s are s t r i c t l y p o s i t i v e d e f i n i t e o n IR for a l l s < 23.
Schaback calls these f u n c t i o n s Sobolev splines (see, e.g., [Schaback (1995a)] o r his
earlier discussion i n [Schaback ( 1 9 9 3 ) ] ) since t h e y are n a t u r a l l y r e l a t e d t o S o b o l e v
spaces (see C h a p t e r 13). T h e s e f u n c t i o n s are also discussed i n t h e r e l a t i v e l y e a r l y
paper [ D i x and Ogden (1994)].
Some s i m p l e r e p r e s e n t a t i v e s o f t h e f a m i l y o f M a t e r n f u n c t i o n s are l i s t e d ( u p t o
a d i m e n s i o n - d e p e n d e n t scale f a c t o r ) i n T a b l e 4.3. N o t e t h a t t h e scaled f u n c t i o n s
l i s t e d i n T a b l e 4.3 d o n o t d e p e n d o n s. Since t h e m o d i f i e d Bessel f u n c t i o n s are
p o s i t i v e , so are t h e M a t e r n f u n c t i o n s . T w o e x a m p l e s are d i s p l a y e d i n F i g u r e 4.3.
T h e f u n c t i o n o n t h e left is d i s p l a y e d u s i n g a shape p a r a m e t e r e 3. The plot
o n t h e r i g h t is scaled so t h a t t h e value a t t h e o r i g i n equals one a n d uses a shape
1
p a r a m e t e r e = 10. N o t e t h a t t h e f u n c t i o n o n t h e left ( c o r r e s p o n d i n g t o 3 = ^ r )
2
is n o t d i f f e r e n t i a b l e a t t h e o r i g i n . T h e M a t e r n f u n c t i o n for 3 = is C smooth,
4 S
a n d t h a t for 3 = is i n C ( R ) .

Table 4.3 Matern functions for various choices of (3.

P=sk (3= its 0 = *

2
(1+11*11)6-11*11 (3 +311*11+ ||a || )e3

4.5 E x a m p l e 5: G e n e r a l i z e d I n v e r s e Multiquadrics

Since b o t h $ a n d <E> i n t h e p r e v i o u s e x a m p l e are p o s i t i v e r a d i a l f u n c t i o n s we c a n

use t h e H a n k e l i n v e r s i o n t h e o r e m (see A p p e n d i x B ) t o reverse t h e i r roles a n d see
t h a t t h e so-called generalized inverse multiquadrics
2 S
*(x) = (1 + Hxll )-^, 3 > - , (4.6)
s
are s t r i c t l y p o s i t i v e d e f i n i t e o n R for s < 23. G e n e r a l i z e d inverse m u l t i q u a d r i c s are
i n f i n i t e l y d i f f e r e n t i a b l e . B y u s i n g a n o t h e r a r g u m e n t based o n c o m p l e t e l y monotone
42 Meshfree Approximation Methods with MATLAB

2
Fig. 4.3 Matern functions with B = i (left) and B = ^5 (right) centered at the origin in R .

f u n c t i o n s w e w i l l be able t o s h o w t h a t i n f a c t w e n e e d t o r e q u i r e o n l y 8 > 0, a n d
S
t h e r e f o r e t h e g e n e r a l i z e d inverse m u l t i q u a d r i c s are s t r i c t l y p o s i t i v e d e f i n i t e o n ]R
for a n y s.
T h e " o r i g i n a l " inverse m u l t i q u a d r i c was i n t r o d u c e d b y H a r d y i n t h e e a r l y 1970s
a n d c o r r e s p o n d s t o t h e value 8 1/2. T h e s p e c i a l choice 8=1 was r e f e r r e d t o as
inverse quadratic i n v a r i o u s p a p e r s o f F o r n b e r g a n d c o - w o r k e r s (see, e.g., [Fornberg
a n d W r i g h t ( 2 0 0 4 ) ] ) . T h e s e t w o f u n c t i o n s are d i s p l a y e d i n F i g u r e 4.4 u s i n g a s h a p e
p a r a m e t e r e = 5.

1-. r , 1

Fig. 4.4 Inverse multiquadric (/3 = ^, left) and inverse quadratic (8=1, right) centered at the
2
origin in R .

4.6 E x a m p l e 6: T r u n c a t e d P o w e r Functions

W e n o w present a n e x a m p l e o f a f a m i l y o f s t r i c t l y p o s i t i v e d e f i n i t e f u n c t i o n s w i t h
compact support. N o t e t h a t d u e t o t h e o b s e r v a t i o n m a d e i n T h e o r e m 3.9 a t t h e e n d
S
of t h e p r e v i o u s c h a p t e r , t h e y c a n n o t be s t r i c t l y p o s i t i v e d e f i n i t e o n R for a l l s.
 4- Examples of Strictly Positive Definite Radial Functions 43

T h e truncated power functions

ipi(r) = ( 1 - r ) i+ (4.7)
s
give rise t o s t r i c t l y p o s i t i v e d e f i n i t e a n d r a d i a l f u n c t i o n s o n M p r o v i d e d I satisfies
t > L f J + 1- F i n d i n g t h e F o u r i e r t r a n s f o r m o f t h e t r u n c a t e d p o w e r f u n c t i o n is
r a t h e r i n v o l v e d . F o r d e t a i l s we refer t o [ W e n d l a n d ( 2 0 0 5 a ) ] . W e w i l l l a t e r use a
s i m p l e r t e s t based o n m u l t i p l y m o n o t o n e f u n c t i o n s t o e s t a b l i s h t h e s t r i c t p o s i t i v e
definiteness o f t h e t r u n c a t e d p o w e r f u n c t i o n s . I n (4.7) we used t h e cutoff function
() + w h i c h is defined b y

x, for x > 0,
0, for x < 0.

T h e c u t o f f f u n c t i o n c a n be i m p l e m e n t e d c o n v e n i e n t l y i n M A T L A B u s i n g t h e max
f u n c t i o n , i.e., i f f x is a v e c t o r o f f u n c t i o n values o f / for d i f f e r e n t choices o f x, t h e n
m a x ( f x , 0 ) c o m p u t e s (f(x)) . + W e also p o i n t o u t t h a t t h e expressions o f t h e f o r m
( 1 r) e
+ are t o be i n t e r p r e t e d as ( ( 1 r)+)*, i.e., we first a p p l y t h e c u t o f f f u n c t i o n ,
and then the power.
T w o different t r u n c a t e d p o w e r f u n c t i o n s ( w i t h i = 2 , 4 ) are d i s p l a y e d i n F i g -
u r e 4.5. W h i l e n o n e o f t h e t r u n c a t e d p o w e r f u n c t i o n s are d i f f e r e n t i a b l e a t t h e o r i g i n ,
t h e s m o o t h n e s s a t t h e b o u n d a r y o f t h e s u p p o r t increases w i t h I .

Fig. 4.5 Truncated power function with i = 2 (left) and = 4 (right) centered at the origin in

4.7 E x a m p l e 7: P o t e n t i a l s a n d W h i t t a k e r R a d i a l F u n c t i o n s

L e t / G C[0, oo) be n o n - n e g a t i v e a n d n o t i d e n t i c a l l y e q u a l t o zero, a n d define t h e

f u n c t i o n cp b y

(4.8)

^1 it.
44 Meshfree Approximation Methods with MATLAB

s
T h e n <& = ip(\\ ||) is s t r i c t l y p o s i t i v e d e f i n i t e a n d r a d i a l o n E p r o v i d e d k > |_|J + 2
(see also T h e o r e m 5.5 b e l o w ) . T h i s c a n be v e r i f i e d b y c o n s i d e r i n g t h e quadratic
form
N N oo N N
- 1
EEw^ *!^ /
3= 1 k=l J
3= 1
c
^2^2 jCkVk-i(t\\xj
k=l
-x \\)f(t)dt
k

w h i c h is n o n - n e g a t i v e since t h e t r u n c a t e d p o w e r f u n c t i o n ^ _ i ( | | | | ) is s t r i c t l y
p o s i t i v e d e f i n i t e b y E x a m p l e 6, a n d / is n o n - n e g a t i v e . Since / is also a s s u m e d t o
be n o t i d e n t i c a l l y e q u a l t o zero, t h e o n l y w a y for t h e q u a d r a t i c f o r m t o e q u a l zero
is i f c = 0 , a n d therefore ip is s t r i c t l y p o s i t i v e d e f i n i t e .
For e x a m p l e , i f we t a k e f(t) = t@, 8 > 0, t h e n we get
- r(fc)r(/3 + 1 )
{ X } ( 4 9 )
~ T{k + 8 + l)\\x\\^- -
W h i l e these f u n c t i o n s are s t r i c t l y p o s i t i v e d e f i n i t e a n d r a d i a l t h e y are also s i n g u l a r
a t t h e o r i g i n a n d therefore n o t useful for o u r p u r p o s e s . H o w e v e r , these f u n c t i o n s
are u p t o s c a l i n g g e n e r a l i z a t i o n s o f t h e Coulomb potential (for 3 = 0 ) , a n d
c a n therefore be g i v e n a p h y s i c a l i n t e r p r e t a t i o n .
a 1
A n o t h e r p o s s i b i l i t y is t o t a k e f(t) = t e~^ , a > 0,/5 > 0. T h e n we get
$ f x = |Ml t f c Q )
- ^r(i+a)r fc) -^ ( e x u 1 Q )

* W /3i+(fc+)/2r(fc+<*+2) e x
i *-- ; L U

(kM ^ ,(k+a+l)/2
{a k)/2 (|Tij) + ( l + o ) M i _ ( f c _ )/
a 2 i ( f c + Q + i)/2 (|NT)) "
Here M M ) I / is t h e W h i t t a k e r - M f u n c t i o n , a c o n f l u e n t h y p e r g e o m e t r i c f u n c t i o n (see,
e.g., Chapter 13 o f [ A b r a m o w i t z a n d S t e g u n (1972)]). W h e n v is a h a l f - i n t e g e r
( w h i c h is, e.g., t h e case for integer k a n d a) f o r m u l a (4.10) s i m p l i f i e s s i g n i f i c a n t l y .
E x a m p l e s for v a r i o u s integer values o f k a n d a are l i s t e d i n T a b l e 4.4. N o t e t h a t
these f u n c t i o n s are n o t defined a t t h e o r i g i n . H o w e v e r , t h e y c a n be m a d e ( o n l y )
c o n t i n u o u s a t t h e o r i g i n . P l o t s o f t w o o f these f u n c t i o n s are p r o v i d e d i n F i g u r e 4.6.
N o t e t h a t o n l y t h e f u n c t i o n s for k > 3 are g u a r a n t e e d t o be s t r i c t l y p o s i t i v e d e f i n i t e
3
and r a d i a l on IR .

Table 4.4 Whittaker radial functions <I> for various choices of k and a.

a fc = 2 fc = 3

2 2

/ ? - |*|| + | | * | | e ~ A -2/3||* | +2||*|| - 2||*|| e~T^f

0
3
P

P- 2||a;| + (/? + 2 | | * | | ) e ~ T W 2
P - 4/3j *|| + 6 | x\\
2
- (2/3\\x\\ + 6 | | * | | ) e " T i r 2

1
3 i
P P

Equation (4.8) a m o u n t s t o another integral transform of / (not listed i n

A p p e n d i x B ) w i t h t h e c o m p a c t l y s u p p o r t e d t r u n c a t e d p o w e r f u n c t i o n as i n t e g r a -
t i o n k e r n e l . W e w i l l t a k e a n o t h e r l o o k a t these f u n c t i o n s i n t h e c o n t e x t o f m u l t i p l y
monotone functions below.
 4- Examples of Strictly Positive Definite Radial Functions 45

Fig. 4.6 Whittaker radial functions for a = 0 and 3 = 1 with k = 2 (left) and k = 3 (right)
2
centered at the origin in R .

4.8 E x a m p l e 8: I n t e g r a t i o n A g a i n s t S t r i c t l y P o s i t i v e
Definite K e r n e l s

I n fact, i n [ W e n d l a n d (2005a)] i t is s h o w n t h a t i n t e g r a t i o n o f a n y non-negative

f u n c t i o n / t h a t is n o t i d e n t i c a l l y e q u a l t o zero against a f u n c t i o n K(t,-) t h a t is
S
s t r i c t l y p o s i t i v e definite o n R leads t o a n o t h e r f u n c t i o n t h a t is s t r i c t l y p o s i t i v e
S
definite o n R , i.e.,

<p{r)= / K(t,r)f(t)dt
Jo
S
gives rise t o $ = tp(\\ ||) b e i n g s t r i c t l y p o s i t i v e definite o n R . B y c h o o s i n g / and
K a p p r o p r i a t e l y we can o b t a i n b o t h g l o b a l l y s u p p o r t e d a n d c o m p a c t l y supported
functions.
For example, t h e m u l t i p l y m o n o t o n e f u n c t i o n s i n W i l l i a m s o n ' s c h a r a c t e r i z a t i o n
1
T h e o r e m 5.4 are covered b y t h i s general t h e o r e m b y t a k i n g K(t, r) = (l rt)^T and
/ a n a r b i t r a r y p o s i t i v e f u n c t i o n i n L \ so t h a t dfi(t) = f(t)dt. Also, functions t h a t
S
are s t r i c t l y p o s i t i v e definite a n d r a d i a l o n R for a l l s (or e q u i v a l e n t l y c o m p l e t e l y
rt
m o n o t o n e f u n c t i o n s ) are covered b y c h o o s i n g K(t,r) = e~ .

4.9 Summary

T o s u m m a r i z e t h e t h e o r y s u r v e y e d t h u s far w e c a n say t h a t a n y m u l t i v a r i a t e ( r a d i a l )
f u n c t i o n $ whose F o u r i e r t r a n s f o r m is n o n - n e g a t i v e can be used t o g e n e r a t e a basis
for t h e s c a t t e r e d d a t a i n t e r p o l a t i o n p r o b l e m b y s h i f t i n g i t t o t h e d a t a sites. The
f u n c t i o n $ can be p o s i t i v e , o s c i l l a t o r y , o r have c o m p a c t s u p p o r t . H o w e v e r , i f $ has
S
any zeros t h e n i t c a n n o t be s t r i c t l y p o s i t i v e d e f i n i t e o n R for a l l choices of s.
 Chapter 5

Completely Monotone and

Multiply Monotone Functions

Since F o u r i e r t r a n s f o r m s are n o t always easy t o c o m p u t e , we n o w present t w o alter-

n a t i v e c r i t e r i a t h a t a l l o w us t o decide w h e t h e r a f u n c t i o n is s t r i c t l y p o s i t i v e d e f i n i t e
s
and radial o n M (one for t h e case o f a l l s, a n d one for o n l y l i m i t e d choices o f s ) .

5.1 Completely Monotone Functions

W e b e g i n w i t h t h e f o r m e r case. T o t h i s e n d we n o w i n t r o d u c e a class o f f u n c t i o n s
t h a t is v e r y closely r e l a t e d t o p o s i t i v e d e f i n i t e r a d i a l f u n c t i o n s a n d leads t o a s i m p l e
c h a r a c t e r i z a t i o n o f such f u n c t i o n s .

Definition 5 . 1 . A f u n c t i o n <p : [0, oo) > M t h a t is i n C [ 0 , o o ) D C ( 0 , o o ) and

satisfies
)
(-l)V (r) > 0 , r >0, = 0,1,2,...,

is called completely monotone on [0, o o ) .

E x a m p l e 5 . 1 . T h e f u n c t i o n cp(r) = e, s > 0, is c o m p l e t e l y m o n o t o n e o n [0, o o ) .

_ e r
E x a m p l e 5 . 2 . T h e f u n c t i o n </?(r) = e , e > 0, is c o m p l e t e l y m o n o t o n e o n [ 0 , o o )
since
W r
(-l)V ( r ) = e e- > 0, t = 0,1,2,....

E x a m p l e 5 . 3 . T h e f u n c t i o n <p(r) = t j ~ ~ y g > /5 > 0, is c o m p l e t e l y m o n o t o n e o n

[0, oo) since

2 1 3 1
( - l ) y < > ( ) = ( - l ) ' / 3 ( / 3 + 1) (3 + i - 1 ) ( 1 + r y
r - > 0, = 0,1, 2,....

Some p r o p e r t i e s o f c o m p l e t e l y m o n o t o n e f u n c t i o n s t h a t c a n be f o u n d i n [Cheney
a n d L i g h t (1999); Feller ( 1 9 6 6 ) ; W i d d e r (1941)] are:

(1) A non-negative f i n i t e l i n e a r c o m b i n a t i o n o f c o m p l e t e l y m o n o t o n e f u n c t i o n s is
completely monotone.
(2) T h e p r o d u c t o f t w o c o m p l e t e l y m o n o t o n e f u n c t i o n s is c o m p l e t e l y m o n o t o n e .

47
48 Meshfree Approximation Methods with M A T L A B

(3) I f <p is c o m p l e t e l y m o n o t o n e a n d ip is a b s o l u t e l y m o n o t o n e (i.e., ip^ > 0 for a l l

i > 0 ) , t h e n ip o (p is c o m p l e t e l y m o n o t o n e .
(4) I f <p is c o m p l e t e l y m o n o t o n e a n d ip is a p o s i t i v e f u n c t i o n such t h a t i t s d e r i v a t i v e
is c o m p l e t e l y m o n o t o n e , t h e n <p o ip is c o m p l e t e l y m o n o t o n e .

N o t e t h a t t h e f u n c t i o n s i n t h e second a n d t h i r d e x a m p l e above are, except for

2
a v a r i a b l e s u b s t i t u t i o n r t> r , s i m i l a r t o t h e G a u s s i a n a n d inverse m u l t i q u a d r i c s
m e n t i o n e d earlier. I n o r d e r t o see h o w c o m p l e t e l y m o n o t o n e f u n c t i o n s are r e l a t e d
t o s t r i c t l y p o s i t i v e definite r a d i a l f u n c t i o n s w e r e q u i r e a n i n t e g r a l c h a r a c t e r i z a t i o n
of completely monotone functions.

T h e o r e m 5.1 ( H a u s d o r f f - B e r n s t e i n - W i d d e r ) . A function ip : [0, oo) > R is

completely monotone on [0, oo) if and only if it is the Laplace transform of a finite
non-negative Borel measure \x on [ 0 , o o ) i.e., <p is of the
; form
rOO
rt
<p(r) = /x(r) = / e~ d{t).
Jo

Proof. W i d d e r ' s p r o o f o f t h i s t h e o r e m c a n be f o u n d i n [ W i d d e r ( 1 9 4 1 ) ] , p . 160,

where he reduces t h e p r o o f o f t h i s t h e o r e m t o a n o t h e r t h e o r e m b y H a u s d o r f f o n
c o m p l e t e l y m o n o t o n e sequences. A d e t a i l e d p r o o f c a n also be f o u n d i n t h e b o o k s
[Cheney a n d L i g h t (1999); W e n d l a n d (2005a)].

T h e o r e m 5.1 shows t h a t , i n t h e s p i r i t o f o u r earlier r e m a r k s , t h e f u n c t i o n p(r) =

er
e~ c a n be v i e w e d as t h e f u n d a m e n t a l c o m p l e t e l y m o n o t o n e f u n c t i o n .
T h e following connection between positive definite radial a n d completely mono-
t o n e f u n c t i o n s was first p o i n t e d o u t b y S c h o e n b e r g i n 1938.

T h e o r e m 5.2. A function <p is completely monotone on [0, oo) if and only if <> =
2 s
<p(\\ \\ ) is positive definite and radial on R for all s.

N o t e t h a t t h e f u n c t i o n <3> is n o w defined v i a t h e square o f t h e n o r m . T h i s differs

f r o m o u r d e f i n i t i o n o f r a d i a l f u n c t i o n s (see D e f i n i t i o n 2 . 1 ) .

Proof. O n e p o s s i b i l i t y is t o use a change o f v a r i a b l e s t o c o m b i n e Schoenberg's

s
c h a r a c t e r i z a t i o n o f f u n c t i o n s t h a t are p o s i t i v e d e f i n i t e a n d r a d i a l o n a n y R , T h e o -
r e m 3.8, w i t h t h e H a u s d o r f f - B e r n s t e i n - W i d d e r c h a r a c t e r i z a t i o n o f c o m p l e t e l y m o n o -
t o n e f u n c t i o n s . T o get m o r e i n s i g h t w e present a n a l t e r n a t i v e p r o o f o f t h e c l a i m t h a t
t h e c o m p l e t e l y m o n o t o n e f u n c t i o n <p gives rise t o a $ t h a t is p o s i t i v e d e f i n i t e a n d
s
r a d i a l o n a n y R . D e t a i l s for t h e o t h e r d i r e c t i o n c a n be f o u n d , e.g., i n [ W e n d l a n d
(2005a)].
T h e H a u s d o r f f - B e r n s t e i n - W i d d e r t h e o r e m i m p l i e s t h a t we c a n w r i t e <p as
/OO
r t
tp(r) = / c- d/i(0
Jo
 5. Completely Monotone and Multiply Monotone Functions 49

2
w i t h a finite n o n - n e g a t i v e B o r e l m e a s u r e / i . T h e r e f o r e , <&(x) = <^(||cc|j ) has the
representation
/OO

Jo
s
T o see t h a t t h i s f u n c t i o n is p o s i t i v e d e f i n i t e o n a n y R w e consider t h e q u a d r a t i c
form

N N r oo N N

j=l k=l J
j=l k=l

Since we saw earlier t h a t t h e Gaussians are s t r i c t l y p o s i t i v e d e f i n i t e a n d r a d i a l o n

s
any R i t follows t h a t t h e q u a d r a t i c f o r m is n o n - n e g a t i v e .
W e c a n see f r o m t h e p r e v i o u s p r o o f t h a t i f t h e measure LL is n o t c o n c e n t r a t e d
s
at t h e o r i g i n , t h e n <fr is even s t r i c t l y p o s i t i v e d e f i n i t e a n d r a d i a l o n a n y R . This
c o n d i t i o n o n t h e measure is e q u i v a l e n t w i t h ip n o t b e i n g c o n s t a n t . W i t h t h i s a d d i -
t i o n a l r e s t r i c t i o n o n <p we c a n a p p l y t h e n o t i o n o f a c o m p l e t e l y m o n o t o n e f u n c t i o n
t o t h e scattered d a t a i n t e r p o l a t i o n p r o b l e m . T h e f o l l o w i n g interpolation theorem
o r i g i n a t e s i n t h e w o r k o f Schoenberg ([Schoenberg (1938a)], p . 823) w h o s h o w e d
t h a t c o m p l e t e m o n o t o n i c i t y i m p l i e s s t r i c t p o s i t i v e definiteness, t h u s p r o v i d i n g a
v e r y s i m p l e test for v e r i f y i n g t h e well-posedness o f m a n y s c a t t e r e d d a t a i n t e r p o -
l a t i o n p r o b l e m s . A p r o o f t h a t t h e converse also h o l d s c a n be f o u n d i n [ W e n d l a n d
(2005a)].
T h e o r e m 5.3. A function <p : [0, oo) > R is completely monotone but not constant
2 s
if and only if <p{\\ || ) is strictly positive definite and radial on R for any s.

_ e r
E x a m p l e 5.4. Since we showed above t h a t t h e f u n c t i o n s <p(r) = e , s > 0, a n d
p(r) = 1 / ( 1 + r)@, 8 > 0, are c o m p l e t e l y m o n o t o n e o n [0, o o ) , a n d since t h e y are
2
also n o t c o n s t a n t we k n o w f r o m T h e o r e m 5.3 t h a t t h e Gaussians <&(cc) = <>(||cc|| ) =
2 2
e - ^ I M I ^ e > 0, a n d inverse m u l t i q u a d r i c s $(cc) = ^ ( | | a ; | | ) = 1 / ( 1 + | | c c | | ) ^ , 8 > 0,
s
are s t r i c t l y p o s i t i v e d e f i n i t e a n d r a d i a l o n R. for a l l s. N o t o n l y is t h e test for
c o m p l e t e m o n o t o n i c i t y a s i m p l e r one t h a n c a l c u l a t i o n o f t h e F o u r i e r t r a n s f o r m s ,
b u t we also are able t o v e r i f y s t r i c t p o s i t i v e definiteness o f t h e inverse m u l t i q u a d r i c s
w i t h o u t a n y dependence o f s o n 8.

5.2 Multiply Monotone Functions

W e can also use m o n o t o n i c i t y t o test for s t r i c t p o s i t i v e definiteness o f r a d i a l func-

S
t i o n s o n 1R for some fixed v a l u e o f s. T o t h i s e n d we i n t r o d u c e t h e concept o f a
multiply monotone function.

D e f i n i t i o n 5 . 2 . A f u n c t i o n p : (0, oo) R w h i c h is i n C ~ ( 0 , o o ) , k > 2, a n d for

f c 2

w h i c h ( !)'(/?(') ( r ) is n o n - n e g a t i v e , n o n - i n c r e a s i n g , a n d convex for I = 0 , 1 , 2 , . . . , k

50 Meshfree Approximation Methods with M A T L A B

2 is c a l l e d k-times monotone on ( 0 , o o ) . I n case fc = 1 w e o n l y r e q u i r e p C ( 0 , oo)

t o be n o n - n e g a t i v e a n d n o n - i n c r e a s i n g .

r r 2 r i r
Since c o n v e x i t y o f tp m e a n s t h a t < ^ ( i + ) < ^ ( ) + ^ ( 2 ) ; o r s i p l y p"(r)
m > 0 if
ip" exists, a m u l t i p l y m o n o t o n e f u n c t i o n is i n essence j u s t a c o m p l e t e l y m o n o t o n e
f u n c t i o n whose m o n o t o n i c i t y is " t r u n c a t e d " .

E x a m p l e 5 . 5 . T h e t r u n c a t e d p o w e r f u n c t i o n (c.f. (4.7))

e
<Pi(r) = (1 - r) +

is ^-times m o n o t o n e for a n y since

(-l)Vf(r) = { - ! ) . . . { - 1 + 1)(1 - rf~ l

> 0, I = 0,1,2,..., L

W e saw i n S e c t i o n 4.6 t h a t t h e t r u n c a t e d p o w e r f u n c t i o n s l e a d t o r a d i a l f u n c t i o n s
s
t h a t are s t r i c t l y p o s i t i v e d e f i n i t e o n R p r o v i d e d > [s/2\ + 1.

E x a m p l e 5 . 6 . I f w e define t h e i n t e g r a l o p e r a t o r I b y
/OO

(//)(r)= / f(t)dt, r > 0 , (5.1)

Jr
a n d / is ^-times m o n o t o n e , t h e n If is + 1-times m o n o t o n e . T h i s follows i m m e d i -
a t e l y f r o m t h e f u n d a m e n t a l t h e o r e m o f c a l c u l u s . A s w e w i l l see l a t e r , t h e o p e r a t o r
/ p l a y s a n i m p o r t a n t r o l e i n t h e c o n s t r u c t i o n o f c o m p a c t l y s u p p o r t e d r a d i a l basis
functions.

To make the connection t o s t r i c t l y positive definite r a d i a l functions we require

a n i n t e g r a l r e p r e s e n t a t i o n for t h e class o f m u l t i p l y m o n o t o n e f u n c t i o n s . T h i s was
g i v e n i n [ W i l l i a m s o n (1956)] b u t a p p a r e n t l y a l r e a d y k n o w n t o S c h o e n b e r g i n 1940.

T h e o r e m 5 . 4 ( W i l l i a m s o n ) . A continuous function ip : ( 0 , o o ) R is k-times

monotone on ( 0 , oo) if and only if it is of the form
/OO

1
p(r)= / (l-r*)*- ^*), (5.2)

where IL is a non-negative Borel measure on ( 0 , o o ) .

Proof. T o see t h a t a f u n c t i o n o f t h e f o r m (5.2) is i n d e e d m u l t i p l y m o n o t o n e we
j u s t need t o d i f f e r e n t i a t e u n d e r t h e i n t e g r a l (since d e r i v a t i v e s u p t o o r d e r k 2
_ 1
of (1 r ) k are c o n t i n u o u s a n d b o u n d e d ) . T h e o t h e r d i r e c t i o n c a n be f o u n d i n
[ W i l l i a m s o n (1956)].

W i l l i a m s o n ' s c h a r a c t e r i z a t i o n shows us t h a t j u s t like t h e t r u n c a t e d p o w e r

f u n c t i o n s t h e W h i t t a k e r r a d i a l f u n c t i o n s ( 4 . 1 0 ) i n S e c t i o n 4.7 are b a s e d o n m u l -
t i p l y monotone functions.
For k > oo t h e W i l l i a m s o n c h a r a c t e r i z a t i o n c o r r e s p o n d s t o t h e Hausdorff-
B e r n s t e i n - W i d d e r c h a r a c t e r i z a t i o n T h e o r e m 5.1 o f c o m p l e t e l y m o n o t o n e f u n c t i o n s
 5. Completely Monotone and Multiply Monotone Functions 51

( a n d is e q u i v a l e n t p r o v i d e d we e x t e n d W i l l i a m s o n ' s w o r k t o i n c l u d e c o n t i n u i t y a t
the origin).
W e c a n see f r o m Sections 4.6 a n d 4.7 t h a t m u l t i p l y m o n o t o n e f u n c t i o n s give rise
to positive definite r a d i a l functions. S u c h a c o n n e c t i o n was first n o t e d i n [ A s k e y
(1973)] ( a n d i n t h e o n e - d i m e n s i o n a l case b y P o l y a ) u s i n g t h e t r u n c a t e d p o w e r func-
t i o n s o f S e c t i o n 4.6.
I n t h e R B F l i t e r a t u r e t h e f o l l o w i n g t h e o r e m was s t a t e d i n [ M i c c h e l l i ( 1 9 8 6 ) ] ,
a n d t h e n refined i n [ B u h m a n n (1993a)]:

T h e o r e m 5.5 ( M i c c h e l l i ) . Let k = [s/2\+2 be a positive integer. If p : [0, oo)

R , p (E C [ 0 , o o ) , is k-times monotone on (0, oo) but not constant, then ip is strictly
positive definite and radial on R s
for any s such that \_s/2\ < k 2 .

We w o u l d like t o m e n t i o n t h a t several versions o f T h e o r e m 5.5 c o n t a i n m i s -

p r i n t s i n t h e l i t e r a t u r e . T h e c o r r e c t f o r m s h o u l d be as s t a t e d above (c.f. also t h e
g e n e r a l i z a t i o n for s t r i c t l y c o n d i t i o n a l l y p o s i t i v e d e f i n i t e f u n c t i o n s , T h e o r e m 9.3).
U s i n g T h e o r e m 5.5 we c a n n o w v e r i f y t h e s t r i c t p o s i t i v e definiteness of the
t r u n c a t e d p o w e r f u n c t i o n s a n d W h i t t a k e r r a d i a l f u n c t i o n s o f Sections 4.6 a n d 4.7
w i t h o u t t h e use o f F o u r i e r t r a n s f o r m s . A g a i n , as for Gaussians a n d t h e Poisson
r a d i a l f u n c t i o n s , w e can v i e w t h e t r u n c a t e d p o w e r f u n c t i o n as t h e fundamental
c o m p a c t l y s u p p o r t e d s t r i c t l y p o s i t i v e d e f i n i t e r a d i a l f u n c t i o n since i t is o b t a i n e d
u s i n g t h e p o i n t e v a l u a t i o n measure i n W i l l i a m s o n ' s c h a r a c t e r i z a t i o n o f a m u l t i p l y
monotone function.
I t is i n t e r e s t i n g t o observe a c e r t a i n l a c k o f s y m m e t r y i n t h e t h e o r y for c o m p l e t e l y
monotone and m u l t i p l y monotone functions. First, i n the completely monotone
case we c a n use T h e o r e m 5.3 t o c o n c l u d e t h a t i f (p is c o m p l e t e l y m o n o t o n e a n d n o t
2 s
c o n s t a n t t h e n ip{ ) is s t r i c t l y p o s i t i v e d e f i n i t e o n R for a n y s. I n the m u l t i p l y
m o n o t o n e case (see T h e o r e m 5.5) t h e square is m i s s i n g . N o w i t is clear t h a t w e
c a n n o t expect t h e s t a t e m e n t w i t h a square t o be t r u e i n t h e m u l t i p l y monotone
case. T o see t h i s w e consider t h e t r u n c a t e d p o w e r f u n c t i o n ip{r) = ( 1 r ) + ( w h i c h
we k n o w a c c o r d i n g t o E x a m p l e 5.1 above t o be ^-times m u l t i p l y m o n o t o n e for
2 e
a n y ) . However, t h e f u n c t i o n tp(r) = ( 1 r ) + is n o t s t r i c t l y p o s i t i v e d e f i n i t e a n d
s
radial on R for a n y s since i t is n o t even s t r i c t l y p o s i t i v e d e f i n i t e a n d r a d i a l o n R
( a n d therefore even m u c h less so o n a n y h i g h e r - d i m e n s i o n a l space). W e c a n see t h i s
f r o m t h e u n i v a r i a t e r a d i a l F o u r i e r t r a n s f o r m o f ijj (see T h e o r e m B . l o f A p p e n d i x B
w i t h s = 1)

2 e
F^(r) = - 7 = f / (1 - t ) t^J_ (rt)dt
+ 1/2

vi Jo 1

= \/f J
1
(1 - t )2 e
cos(rt)dt
52 Meshfree Approximation Methods with M A T L A B

Here we used t h e c o m p a c t s u p p o r t o f ip a n d t h e fact t h a t J_i/ (r) 2 = y/2/ivr cosr.

T h e f u n c t i o n T\ip is o s c i l l a t o r y , a n d therefore ip c a n n o t be s t r i c t l y p o s i t i v e d e f i n i t e
(c.f. T h e o r e m 3.5). I n fact, t h e F o u r i e r t r a n s f o r m T\ip is closely r e l a t e d t o t h e
Poisson r a d i a l f u n c t i o n s o f S e c t i o n 4.3.
M o r e o v e r , i n t h e c o m p l e t e l y m o n o t o n e case w e have a n equivalence b e t w e e n
c o m p l e t e l y m o n o t o n e a n d s t r i c t l y p o s i t i v e d e f i n i t e f u n c t i o n s t h a t are r a d i a l o n a n y
s
R (see T h e o r e m 5.3). A g a i n , we c a n n o t e x p e c t such a n equivalence t o h o l d i n t h e
m u l t i p l y m o n o t o n e case, i.e., t h e converse o f T h e o r e m 5.5 c a n n o t be t r u e . This
is clear since w e have a l r e a d y seen a n u m b e r o f f u n c t i o n s t h a t are s t r i c t l y p o s i t i v e
definite a n d r a d i a l , b u t n o t m o n o t o n e a t a l l n a m e l y t h e o s c i l l a t o r y L a g u e r r e -
Gaussians o f S e c t i o n 4.2 a n d t h e Poisson r a d i a l f u n c t i o n s o f S e c t i o n 4.3.
However, i t is i n t e r e s t i n g t o c o m b i n e t h e Schoenberg T h e o r e m 5.3 a n d T h e -
o r e m 5.5 based o n W i l l i a m s o n ' s c h a r a c t e r i z a t i o n . I f one s t a r t s w i t h t h e s t r i c t l y
2 2
e r
p o s i t i v e d e f i n i t e r a d i a l Gaussian <p(r) = e~ , t h e n T h e o r e m 5.3 tells us t h a t
2
r
4>{r) = <p(y/r) = e~~ is c o m p l e t e l y m o n o t o n e . N o w , a n y f u n c t i o n t h a t is c o m -
p l e t e l y m o n o t o n e is also m u l t i p l y m o n o t o n e o f a n y o r d e r , so t h a t we c a n use T h e o -
2
r
r e m 5.5 a n d c o n c l u d e t h a t t h e f u n c t i o n (p{r) = e~ is also s t r i c t l y p o s i t i v e d e f i n i t e
s
a n d r a d i a l o n M for a l l s. O f course, n o w we c a n r e p e a t t h e a r g u m e n t a n d c o n c l u d e
2 /
y / r s
t h a t ip(r) = e~ is s t r i c t l y p o s i t i v e d e f i n i t e a n d r a d i a l o n R for a l l s, a n d so o n
(see [ W e n d l a n d (2005c)]). T h i s r e s u l t was a l r e a d y k n o w n t o Schoenberg (at least
s
i n t h e n o n - s t r i c t case).
As a f i n a l r e m a r k i n t h i s c h a p t e r we m e n t i o n t h a t we are a l o n g w a y f r o m
h a v i n g a c o m p l e t e c h a r a c t e r i z a t i o n o f ( r a d i a l ) f u n c t i o n s for w h i c h t h e scattered
d a t a i n t e r p o l a t i o n p r o b l e m has a u n i q u e s o l u t i o n . A s we w i l l see l a t e r , such a n (as
of n o w u n k n o w n ) c h a r a c t e r i z a t i o n w i l l i n v o l v e also f u n c t i o n s w h i c h are n o t s t r i c t l y
positive definite. For e x a m p l e , we w i l l m e n t i o n a r e s u l t o f M i c c h e l l i ' s a c c o r d i n g
t o w h i c h conditionally p o s i t i v e d e f i n i t e f u n c t i o n s o f o r d e r one c a n be used for t h e
s c a t t e r e d d a t a i n t e r p o l a t i o n p r o b l e m . F u r t h e r m o r e , a l l o f t h e results d e a l t w i t h so
far i n v o l v e r a d i a l basis f u n c t i o n s t h a t are centered a t t h e g i v e n d a t a sites. There
are o n l y l i m i t e d results addressing t h e s i t u a t i o n i n w h i c h t h e centers for t h e basis
f u n c t i o n s a n d t h e d a t a sites m a y differ.
 Chapter 6

Scattered Data Interpolation with

Polynomial Precision

6.1 Interpolation with Multivariate Polynomials

As we m e n t i o n e d i n t h e i n t r o d u c t i o n i t is n o t a n easy m a t t e r t o use p o l y n o m i a l s
t o p e r f o r m m u l t i v a r i a t e s c a t t e r e d d a t a i n t e r p o l a t i o n . O n l y i f t h e d a t a sites are i n
c e r t a i n special l o c a t i o n s c a n we g u a r a n t e e well-posedness o f m u l t i v a r i a t e p o l y n o m i a l
i n t e r p o l a t i o n . W e n o w address t h i s p r o b l e m .

S
D e f i n i t i o n 6 . 1 . W e c a l l a set o f p o i n t s X = {x\,... ,x^} C R m-unisolvent if
t h e o n l y p o l y n o m i a l o f t o t a l degree at m o s t m i n t e r p o l a t i n g zero d a t a o n X is t h e
zero p o l y n o m i a l .

T h i s d e f i n i t i o n guarantees a u n i q u e s o l u t i o n for i n t e r p o l a t i o n t o g i v e n d a t a at a
subset o f c a r d i n a l i t y M = ( * ) m s
f t n e
p o i n t s x \ , . . . , XN b y a p o l y n o m i a l o f degree
m. Here M is t h e d i m e n s i o n o f t h e linear space o f p o l y n o m i a l s o f t o t a l degree
less t h a n or equal t o m i n s variables.
S
For p o l y n o m i a l i n t e r p o l a t i o n a t N d i s t i n c t d a t a sites i n R t o be a w e l l - p o s e d
p r o b l e m , t h e p o l y n o m i a l degree needs t o be chosen a c c o r d i n g l y , i.e., we need M =
N, a n d t h e d a t a sites need t o f o r m a n m - u n i s o l v e n t set. T h i s is r a t h e r r e s t r i c t i v e .
2
For example, t h i s i m p l i e s t h a t p o l y n o m i a l i n t e r p o l a t i o n at N = 7 points i n R
can n o t be done i n a u n i q u e w a y since we c o u l d either a t t e m p t t o use b i v a r i a t e
q u a d r a t i c p o l y n o m i a l s (for w h i c h M = 6 ) , or b i v a r i a t e c u b i c p o l y n o m i a l s ( w i t h
M = 10). T h e r e exists n o space o f b i v a r i a t e p o l y n o m i a l s for w h i c h M = 7.
W e w i l l see i n t h e n e x t c h a p t e r t h a t m - u n i s o l v e n t sets p l a y a n i m p o r t a n t role i n
t h e c o n t e x t o f c o n d i t i o n a l l y p o s i t i v e definite f u n c t i o n s . T h e r e , however, even t h o u g h
we w i l l be interested i n i n t e r p o l a t i n g N pieces o f d a t a , t h e p o l y n o m i a l degree w i l l
be s m a l l ( u s u a l l y m = 1, 2, 3 ) , a n d t h e r e s t r i c t i o n s i m p o s e d o n t h e l o c a t i o n s o f t h e
d a t a sites b y t h e u n i s o l v e n c y c o n d i t i o n s w i l l be r a t h e r m i l d .
A sufficient c o n d i t i o n ( t o be f o u n d i n [ C h u i (1988)], C h . 9) o n t h e points
X \ , . . . , xpj t o f o r m a n m - u n i s o l v e n t set i n R 2
is

2
T h e o r e m 6 . 1 . Suppose {Lo,..., L m } is a set o / m + 1 distinct lines in R , and that
U = . . . , UM} is a set of M = ( m + l ) ( m + 2 ) / 2 distinct points such that the

53
54 Meshfree Approximation Methods with M A T L A B

first point lies on L Q , the next two points lie on L \ but not on L Q , and so on, so that
the last m + 1 points lie on L m but not on any of the previous lines LQ, . . . , L m _ i .
Then there exists a unique interpolation polynomial of total degree at most m to
arbitrary data given at the points in U. Furthermore, if the data sites {x,..., x^}
2
contain U as a subset then they form an m-unisolvent set on R .

Proof. W e use i n d u c t i o n o n m . F o r m = 0 t h e r e s u l t is t r i v i a l . T a k e R t o be t h e
m a t r i x a r i s i n g f r o m p o l y n o m i a l i n t e r p o l a t i o n a t t h e p o i n t s i n IA, i.e.,

Rjk=Pk(uj), j,k = l,...,M,

w h e r e t h e pk f o r m a basis o f LT^. W e w a n t t o s h o w t h a t t h e o n l y possible s o l u t i o n

t o Rc = 0 is c = 0. T h i s is e q u i v a l e n t t o s h o w i n g t h a t i f p TL^ satisfies

p(t*i)=0, i = l,...,M,

t h e n p is t h e zero p o l y n o m i a l .
For each i = 1 , . . . , m, let t h e e q u a t i o n o f t h e l i n e Li be g i v e n b y

OHX + fay = 7 i ,
2
w h e r e x = (x, y) R .
Suppose n o w t h a t p i n t e r p o l a t e s zero d a t a at a l l t h e p o i n t s U i as s t a t e d above.
Since p reduces t o a u n i v a r i a t e p o l y n o m i a l o f degree m o n L m w h i c h vanishes a t
m + 1 distinct points on L m , i t follows t h a t p vanishes i d e n t i c a l l y o n L m , a n d so

p(x, y) = (a xm + 3y
m - ~/m)q(x, y),

w h e r e q is a p o l y n o m i a l o f degree m 1. B u t n o w q satisfies t h e h y p o t h e s i s o f t h e
x
t h e o r e m w i t h m r e p l a c e d b y m 1 a n d U r e p l a c e d b y U c o n s i s t i n g o f t h e first ( " ^ )
p o i n t s o f U. B y i n d u c t i o n , therefore q = 0, a n d t h u s p = 0. T h i s establishes t h e
uniqueness o f t h e i n t e r p o l a t i o n p o l y n o m i a l . T h e l a s t s t a t e m e n t o f t h e t h e o r e m is
obvious.

A s i m i l a r t h e o r e m was a l r e a d y p r o v e d i n [ C h u n g a n d Y a o ( 1 9 7 7 ) ] . T h e o r e m 6.1
S
c a n be generalized t o R b y using hyperplanes. T h e p r o o f is c o n s t r u c t e d w i t h t h e
h e l p o f a n a d d i t i o n a l i n d u c t i o n o n s. C h u i also gives a n e x p l i c i t expression for t h e
d e t e r m i n a n t o f t h e m a t r i x associated w i t h ( p o l y n o m i a l ) i n t e r p o l a t i o n a t t h e set o f
p o i n t s U.

R e m a r k 6.1. F o r l a t e r reference w e n o t e t h a t ( m l ) - u n i s o l v e n c y o f t h e p o i n t s
x\,..., XN is e q u i v a l e n t t o t h e fact t h a t t h e m a t r i x P w i t h

Pji = Pi{xj), j = l,...,N, I = 1,..., M,

has f u l l ( c o l u m n - ) r a n k . F o r N = M t h i s is t h e p o l y n o m i a l i n t e r p o l a t i o n m a t r i x .

2
Example 6.1. A s can easily be v e r i f i e d , t h r e e c o l l i n e a r p o i n t s i n R are n o t 1-
u n i s o l v e n t , since a linear i n t e r p o l a n t , i.e., a p l a n e t h r o u g h t h r e e a r b i t r a r y h e i g h t s
a t these t h r e e c o l l i n e a r p o i n t s is n o t u n i q u e l y d e t e r m i n e d . O n t h e o t h e r h a n d , i f a
2
set o f p o i n t s i n R c o n t a i n s t h r e e n o n - c o l l i n e a r p o i n t s , t h e n i t is 1-unisolvent.

x 4-
 6. Scattered Data Interpolation with Polynomial Precision 55

W e used t h e difficulties associated w i t h m u l t i v a r i a t e p o l y n o m i a l i n t e r p o l a t i o n

as one o f t h e m o t i v a t i o n s for t h e use o f r a d i a l basis f u n c t i o n s . H o w e v e r , s o m e t i m e s
i t is desirable t o have a n i n t e r p o l a n t t h a t e x a c t l y reproduces c e r t a i n t y p e s o f func-
t i o n s . For e x a m p l e , i f t h e d a t a are c o n s t a n t , or come f r o m a l i n e a r f u n c t i o n , t h e n
i t w o u l d be nice i f o u r i n t e r p o l a n t were also c o n s t a n t or linear, respectively. U n f o r -
t u n a t e l y , t h e m e t h o d s we have presented t h u s far (except for t h e d i s t a n c e m a t r i x
fit i n t h e s = 1 case) do n o t r e p r o d u c e these s i m p l e p o l y n o m i a l f u n c t i o n s . More-
over, later o n we w i l l be i n t e r e s t e d i n a p p l y i n g o u r i n t e r p o l a t i o n m e t h o d s t o t h e
n u m e r i c a l s o l u t i o n o f p a r t i a l d i f f e r e n t i a l equations, a n d p r a c t i t i o n e r s (especially o f
finite element m e t h o d s ) o f t e n j u d g e a n i n t e r p o l a t i o n m e t h o d b y i t s a b i l i t y t o pass
t h e so-called patch test. A n i n t e r p o l a t i o n m e t h o d passes t h e s t a n d a r d p a t c h test i f
i t can reproduce linear f u n c t i o n s . I n engineering a p p l i c a t i o n s t h i s t r a n s l a t e s i n t o
exact c a l c u l a t i o n o f c o n s t a n t stress a n d s t r a i n . W e w i l l see later t h a t i n o r d e r t o
prove e r r o r estimates for meshfree a p p r o x i m a t i o n m e t h o d s i t is n o t necessary t o
be able t o r e p r o d u c e p o l y n o m i a l s g l o b a l l y ( b u t l o c a l p o l y n o m i a l r e p r o d u c t i o n is a n
essential i n g r e d i e n t ) . T h u s , i f we are o n l y concerned w i t h t h e a p p r o x i m a t i o n p o w e r
of a n u m e r i c a l m e t h o d t h e r e is r e a l l y n o need for t h e s t a n d a r d p a t c h test t o h o l d .

6.2 E x a m p l e : R e p r o d u c t i o n of L i n e a r F u n c t i o n s U s i n g
Gaussian RBFs

I f we do insist o n r e p r o d u c t i o n o f l i n e a r f u n c t i o n s t h e n t h e t o p p a r t o f F i g u r e 6.1
shows a Gaussian R B F i n t e r p o l a n t (e = 6) t o t h e b i v a r i a t e l i n e a r f u n c t i o n f(x,y) =
(x + y)/2 based o n 1089 u n i f o r m l y spaced p o i n t s i n t h e u n i t square a l o n g w i t h
t h e absolute error. C l e a r l y t h e i n t e r p o l a n t is n o t c o m p l e t e l y p l a n a r n o t even t o
m a c h i n e precision.
F o r t u n a t e l y , t h e r e is a s i m p l e r e m e d y for t h i s p r o b l e m . A l l we need t o d o
is a d d t h e p o l y n o m i a l f u n c t i o n s x i> 1, x i x, and x t> y t o the- basis
_ e - X l _ e - X J V
{e H' H ,...,e H' H } we have t h u s far been u s i n g t o o b t a i n o u r i n t e r -
polant. However, n o w we have N + 3 u n k n o w n s , n a m e l y t h e coefficients c,
k

k = 1 , . . . , N + 3, i n t h e e x p a n s i o n
N
e 2 x 2 2
V (x)
f = ^ 2 c e - k ^ - ^ + c N + 1 + c N + 2 x + c N + 3 y, x = {x,y) e M ,
fc=i
a n d we have o n l y N c o n d i t i o n s t o d e t e r m i n e t h e m , n a m e l y t h e i n t e r p o l a t i o n con-
ditions
V {x )
S 3 = f( ) Xj = ( Xj + )/2,Vj j = 1 , . . . , N.
W h a t can we do t o o b t a i n a ( n o n - s i n g u l a r ) square system? A s we w i l l see b e l o w ,
we can a d d t h e f o l l o w i n g t h r e e c o n d i t i o n s :
N
^ c f c = 0,
fc=i
56 Meshfree Approximation Methods with M A T L A B

^CkXk = 0,
fc=l
N
y^c ?/fc = o.
fc

fc=i
How do we have t o m o d i f y o u r e x i s t i n g M A T L A B p r o g r a m for s c a t t e r e d data
i n t e r p o l a t i o n t o i n c o r p o r a t e these m o d i f i c a t i o n s ? I f we p r e v i o u s l y d e a l t w i t h t h e
solution of

Ac = y,
T
with A jk = e -e*\\ - \\^
Xj Xk j k = i , . . . c = [ C l ,... ) C 7 v] , and y =
t
[ / ( c c i ) , . . . , / ( C J V ) ] , t h e n we n o w have t o solve t h e augmented system

' A P~ c V
(6.1)
P T
O d 0

w h e r e A, c, a n d y are as before, a n d Pji = pi(xj), j = 1,..., N, I = 1,..., 3, w i t h

Pi(x) = 1, p {x) 2 = x, a n d pz{x) = y. M o r e o v e r , 0 is a zero v e c t o r o f l e n g t h 3, a n d
O is a zero m a t r i x o f size 3 x 3 .
The MATLAB s c r i p t R B F I n t e r p o l a t i o n 2 D l i n e a r .m shows a n i m p l e m e n t a t i o n
o f t h i s a p p r o a c h for Gaussians ( a l t h o u g h t h e y c a n easily be r e p l a c e d b y a n y o t h e r
RBF) a n d test f u n c t i o n f(x,y) = (x + y)/2. T h e resulting interpolant using iV = 9
e q u a l l y spaced d a t a p o i n t s a n d s = 6 is s h o w n i n t h e b o t t o m p a r t o f F i g u r e 6.1.
Now, w h i l e s t i l l n o t p e r f e c t l y l i n e a r , t h e e r r o r is o n t h e level o f m a c h i n e accuracy.

P r o g r a m 6 . 1 . R B F I n t e r p o l a t i o n 2 D l i n e a r .m

/ R B F I n t e r p o l a t i o n 2 D l i n e a r
0

% S c r i p t t h a t performs 2D RBF i n t e r p o l a t i o n w i t h r e p r o d u c t i o n of
% l i n e a r functions
% C a l l s on: D i s t a n c e M a t r i x
% Define t h e Gaussian RBF and shape parameter
1 r b f = @(e,r) e x p ( - ( e * r ) . ~ 2 ) ; ep = 6;
% Define l i n e a r t e s t f u n c t i o n
2 t e s t f u n c t i o n = @(x,y) (x+y)/2;
% Number and type of d a t a p o i n t s
5
3 N = 9; g r i d t y p e = ' u ;
% Load d a t a p o i n t s
5
4 name = s p r i n t f ( Data2D_y d%s' ,N, g r i d t y p e ) ; l o a d (name)
o

5 ctrs = dsites;
6 neval = 4 0 ; M = neval~2; g r i d = l i n s p a c e ( 0 , 1 , n e v a l ) ;
7 [[Link]] = m e s h g r i d ( g r i d ) ; e p o i n t s = [ x e ( : ) y e ( : ) ] ;
% Evaluate the t e s t function a t the data p o i n t s .
8 rhs = testfunction(dsites(:,1).dsites(:,2));
 6. Scattered Data Interpolation with Polynomial Precision 57

/ Add z e r o s f o r l i n e a r (2D) r e p r o d u c t i o n
0

9 rhs = [rhs; z e r o s ( 3 , l ) ] ;
% Compute d i s t a n c e m a t r i x between t h e d a t a s i t e s and c e n t e r s
10 DM_data = D i s t a n c e M a t r i x ( d s i t e s , c t r s ) ;
% Compute i n t e r p o l a t i o n m a t r i x
11 IM = rbf(ep,DM_data);
% Define 3-column m a t r i x P f o r l i n e a r r e p r o d u c t i o n
12 PM = [ones(N.l) d s i t e s ] ;
% Augment i n t e r p o l a t i o n m a t r i x
J
13 IM = [IM PM; [PM z e r o s ( 3 , 3 ) ] ] ;
% Compute d i s t a n c e m a t r i x between e v a l u a t i o n p o i n t s and c e n t e r s
14 DM_eval = D i s t a n c e M a t r i x ( e p o i n t s , c t r s ) ;
% Compute e v a l u a t i o n m a t r i x
15 EM = rbf(ep,DM_eval);
% Add column f o r constant r e p r o d u c t i o n
16 PM = [ones(M,l) e p o i n t s ] ; EM = [EM PM];
/ Compute RBF i n t e r p o l a n t
0

% ( e v a l u a t i o n matrix * s o l u t i o n of i n t e r p o l a t i o n system)
17 Pf = EM * ( I M \ r h s ) ;
/ Compute maximum e r r o r on e v a l u a t i o n g r i d
0

18 exact = t e s t f u n c t i o n ( e p o i n t s ( : , 1 ) , e p o i n t s ( : , 2 ) ) ;
19 maxerr = n o r m ( P f - e x a c t , i n f ) ;
20 rms_err = norm(Pf-exact)/neval;
21 fprintf('RMS e r r o r : %e\n', rms_err)
5
22 fprintf('Maximum e r r o r : % e \ n , maxerr)
23 fview = [-30,30];
24 plotsurf(xe,ye,Pf,neval,exact,maxerr,fview);
25 ploterror2D(xe,ye,Pf,exact,maxerr,neval,fview);

N o t e t h a t P r o g r a m 6.1 is a l m o s t t h e same as P r o g r a m 2.1. T h e o n l y difference

are lines 9, 12, 13, a n d 16 t h a t have been a d d e d t o d e a l w i t h t h e a u g m e n t e d p r o b l e m .
I n P r o g r a m 6.1 we also m o d i f i e d t h e d e f i n i t i o n o f t h e test f u n c t i o n .

6.3 Scattered D a t a Interpolation with More General Polynomial

Precision

As we j u s t saw for a specific e x a m p l e , we m a y w a n t t o m o d i f y t h e a s s u m p t i o n on

the f o r m (1.1) o f t h e s o l u t i o n t o t h e s c a t t e r e d d a t a i n t e r p o l a t i o n P r o b l e m 1.1 b y
a d d i n g c e r t a i n p o l y n o m i a l s t o t h e e x p a n s i o n , i.e., Vf is n o w assumed t o be o f t h e
form
N M
(6.2)
58 Meshfree Approximation Methods with M A T L A B

Fig. 6.1 Top: Gaussian interpolant to bivariate linear function with N = 1089 (left) and as-
sociated abolute error (right). Bottom: Interpolant based on linearly augmented Gaussians to
bivariate linear function with N = 9 (left) and associated abolute error (right).

w h e r e pi,... ,PM f o r m a basis for t h e M = ( " ^ " l " } " ) - d i m e n s i o n a l l i n e a r space

5
H _i
m

of p o l y n o m i a l s o f t o t a l degree less t h a n o r e q u a l t o m 1 i n s v a r i a b l e s . I t seems

a w k w a r d t o formulate t h i s setup w i t h p o l y n o m i a l s i n n ^ _ l 1 i n s t e a d o f degree m
p o l y n o m i a l s . H o w e v e r , i n l i g h t o f o u r discussion o f c o n d i t i o n a l l y p o s i t i v e d e f i n i t e
f u n c t i o n s i n t h e n e x t c h a p t e r t h i s choice is q u i t e n a t u r a l .
Since e n f o r c i n g t h e i n t e r p o l a t i o n c o n d i t i o n s Vf(xj) = f{xj), j = 1, , N, leads
t o a system of N linear equations i n the N + M unknowns c k a n d di one u s u a l l y
adds t h e M a d d i t i o n a l c o n d i t i o n s
N
c
J2 kPi(xk) = 0, Z = 1,...,M,
fc=i
t o ensure a u n i q u e s o l u t i o n . T h e e x a m p l e i n t h e p r e v i o u s s e c t i o n represents t h e
p a r t i c u l a r case s = m = 2.
W h i l e t h e use o f p o l y n o m i a l s is s o m e w h a t a r b i t r a r y ( a n y o t h e r set o f M l i n e a r l y
i n d e p e n d e n t f u n c t i o n s c o u l d also be u s e d ) , i t is o b v i o u s t h a t t h e a d d i t i o n o f p o l y -
n o m i a l s o f t o t a l degree at m o s t m 1 g u a r a n t e e s p o l y n o m i a l p r e c i s i o n p r o v i d e d t h e
p o i n t s i n X f o r m a n ( m l ) - u n i s o l v e n t set. I n o t h e r w o r d s , i f t h e d a t a c o m e f r o m a
 6. Scattered Data Interpolation with Polynomial Precision 59

p o l y n o m i a l o f t o t a l degree less t h a n o r e q u a l t o m 1, t h e n t h e y are f i t t e d e x a c t l y

by the expansion (6.2).
I n general, s o l v i n g t h e i n t e r p o l a t i o n p r o b l e m based o n t h e e x t e n d e d expansion
(6.2) n o w a m o u n t s t o s o l v i n g a s y s t e m o f l i n e a r e q u a t i o n s o f t h e f o r m

" A P~ c y
(6.3)
d 0

where t h e pieces are g i v e n b y Ajk = <p(\\xj - Xk\\), j,k = 1 , . . . , N, Pj\ = pi(xj),

j = 1 , . . . , N, I = 1 , . . . ,M, c = [ c i , . . . ,c ] ,
N
T
d = [di,.. .,d ] ,
M
T
V = [yi, -,VN] ,
T

0 is a zero vector o f l e n g t h M, a n d O is a n M x M zero m a t r i x . B e l o w we w i l l

s t u d y t h e i n v e r t i b i l i t y o f t h i s m a t r i x i n t w o steps. F i r s t for t h e case m = 1 i n
T h e o r e m 6.2, a n d t h e n for t h e case o f general m i n T h e o r e m 7.2.
N o t e t h a t we c a n easily m o d i f y t h e M A T L A B p r o g r a m l i s t e d above t o deal w i t h
r e p r o d u c t i o n o f p o l y n o m i a l s o f o t h e r degrees. F o r e x a m p l e , i f we w a n t t o r e p r o d u c e
constants t h e n we need t o replace lines 9, 12, 13, a n d 16 b y

9 rhs = [rhs; 0 ] ;
12 PM = o n e s ( N , l ) ;
13 IM = [IM PM; [PM' 0 ] ] ;
16 PM = ones(M,l); EM = [EM PM];

a n d for r e p r o d u c t i o n o f b i v a r i a t e q u a d r a t i c p o l y n o m i a l s we c a n use

9 rhs = [rhs; zeros(6,l)];

12a PM = [ones(N,l) d s i t e s d s i t e s ( : , 1 ) . " 2 ...
12b dsites(:,2)."2 dsites(:,1).*dsites(:,2)];
13 IM = [IM PM; [PM' z e r o s ( 6 , 6 ) ] ] ;
16a PM = [ones(M,l) e p o i n t s e p o i n t s ( : , 1 ) . " 2 ...
16b epoints(:,2).~2 epoints(:,1).*epoints(:,2)];
16c EM = [EM PM] ;

O f course, these specific examples w o r k o n l y for t h e case s = 2. T h e generaliza-

t i o n t o higher dimensions, however, is o b v i o u s b u t m o r e c u m b e r s o m e .

6.4 Conditionally Positive Definite Matrices and Reproduction of

Constant Functions

W e n o w need t o i n v e s t i g a t e w h e t h e r t h e a u g m e n t e d s y s t e m m a t r i x i n (6.3) is n o n -
singular. T h e special case m = 1 ( i n a n y space d i m e n s i o n s), i.e., r e p r o d u c t i o n o f
constants, is covered b y s t a n d a r d results f r o m l i n e a r algebra, a n d w e discuss i t f i r s t .

D e f i n i t i o n 6 . 2 . A r e a l s y m m e t r i c m a t r i x A is c a l l e d conditionally positive semi-

60 Meshfree Approximation Methods with M A T L A B

definite of order one i f its associated q u a d r a t i c f o r m is n o n - n e g a t i v e , i.e.

N N
^2^2c j C k A j k >0 (6.4)
3= 1 k=l

for a l l c = [ c i , . . . , C N ] T
R N
t h a t satisfy

3= 1

I f c ^ 0 i m p l i e s s t r i c t i n e q u a l i t y i n (6.4) t h e n A is c a l l e d conditionally positive

definite of order one.
I n t h e l i n e a r algebra l i t e r a t u r e t h e d e f i n i t i o n u s u a l l y is f o r m u l a t e d u s i n g " < "
i n ( 6 . 4 ) , a n d t h e n A is referred t o as ( c o n d i t i o n a l l y or a l m o s t ) negative definite.
O b v i o u s l y , c o n d i t i o n a l l y p o s i t i v e d e f i n i t e m a t r i c e s o f o r d e r one exist o n l y for N > 1.
W e can i n t e r p r e t a m a t r i x A t h a t is c o n d i t i o n a l l y p o s i t i v e definite o f o r d e r one
as one t h a t is p o s i t i v e d e f i n i t e o n t h e space o f vectors c s u c h t h a t

3= 1

T h u s , i n t h i s sense, A is p o s i t i v e d e f i n i t e o n t h e space o f v e c t o r s c " p e r p e n d i c u l a r "

t o c o n s t a n t functions.
Now we are r e a d y t o f o r m u l a t e a n d p r o v e
Theorem 6.2. Let A be a real symmetric N x N matrix that is conditionally
T
positive definite of order one, and let P = [ 1 , . . . , 1 ] be an N x 1 matrix (column
vector). Then the system of linear equations
' A P~ c y~
T
P 0 d 0
is uniquely solvable.

T
Proof. A s s u m e [c, d] is a s o l u t i o n o f t h e homogeneous l i n e a r system, i.e., with
T T
y = 0 . W e show t h a t [c, d] = 0 is t h e o n l y possible s o l u t i o n .
T
M u l t i p l i c a t i o n o f t h e t o p b l o c k o f t h e (homogeneous) l i n e a r s y s t e m b y c yields
T T
c Ac + dc P = 0.
T T
F r o m t h e b o t t o m b l o c k o f t h e s y s t e m we k n o w P c = cP = 0, a n d therefore
T
c Ac = 0.

Since t h e m a t r i x A is c o n d i t i o n a l l y p o s i t i v e d e f i n i t e o f o r d e r one b y a s s u m p t i o n we
get t h a t c = 0 . F i n a l l y , t h e t o p b l o c k o f t h e h o m o g e n e o u s linear s y s t e m u n d e r
c o n s i d e r a t i o n states t h a t

Ac + dP = 0,

so t h a t c = 0 a n d t h e fact t h a t P is a v e c t o r o f ones i m p l y d = 0.
 6. Scattered Data Interpolation with Polynomial Precision 61

Since Gaussians ( a n d a n y o t h e r s t r i c t l y p o s i t i v e definite r a d i a l f u n c t i o n ) give rise

t o p o s i t i v e d e f i n i t e m a t r i c e s , a n d since p o s i t i v e d e f i n i t e m a t r i c e s are also c o n d i t i o n -
a l l y p o s i t i v e definite o f o r d e r one, T h e o r e m 6.2 establishes t h e n o n s i n g u l a r i t y o f t h e
( a u g m e n t e d ) r a d i a l basis f u n c t i o n i n t e r p o l a t i o n m a t r i x for c o n s t a n t r e p r o d u c t i o n .
I n order t o cover r a d i a l basis f u n c t i o n i n t e r p o l a t i o n w i t h r e p r o d u c t i o n o f h i g h e r -
order p o l y n o m i a l s we w i l l n o w i n t r o d u c e ( s t r i c t l y ) c o n d i t i o n a l l y p o s i t i v e d e f i n i t e
functions o f o r d e r m .
I
i
 Chapter 7

Conditionally Positive Definite Functions

7.1 Conditionally Positive Definite Functions Defined

I n a n a l o g y t o o u r earlier discussion o f i n t e r p o l a t i o n w i t h p o s i t i v e definite f u n c t i o n s

we w i l l n o w i n t r o d u c e c o n d i t i o n a l l y p o s i t i v e d e f i n i t e a n d s t r i c t l y c o n d i t i o n a l l y pos-
i t i v e definite f u n c t i o n s o f o r d e r m. W e w i l l realize t h a t these f u n c t i o n s p r o v i d e t h e
n a t u r a l g e n e r a l i z a t i o n o f R B F i n t e r p o l a t i o n w i t h p o l y n o m i a l r e p r o d u c t i o n discussed
i n the previous chapter. Examples of s t r i c t l y c o n d i t i o n a l l y positive definite (radial)
f u n c t i o n s are p r e s e n t e d i n t h e n e x t c h a p t e r .

Definition 7 . 1 . A c o m p l e x - v a l u e d c o n t i n u o u s f u n c t i o n <& is c a l l e d conditionally

S
positive definite of order m on R if
N N

E ^CjCk&ixj - x k ) > 0 (7.1)

3= 1 k=l

for a n y N p a i r w i s e d i s t i n c t p o i n t s x \ , . . . , XN R , a n d c = [ c i , . . . , c y v ]
S T
& N

satisfying
N
^2cjP(xj) = 0,
3= 1
for any c o m p l e x - v a l u e d p o l y n o m i a l p o f degree at m o s t m 1. T h e f u n c t i o n <E> is
S
called strictly conditionally positive definite of order m on 1R. i f t h e q u a d r a t i c f o r m
(7.1) is zero o n l y for c = 0.

A n i m m e d i a t e o b s e r v a t i o n is

L e m m a 7 . 1 . A function that is (strictly) conditionally positive definite of order

S
m on R is also (strictly) conditionally positive definite of any higher order. In
particular, a (strictly) positive definite function is always (strictly) conditionally
positive definite of any order.

Proof. T h e first s t a t e m e n t f o l l o w s i m m e d i a t e l y f r o m D e f i n i t i o n 7 . 1 . T h e second

s t a t e m e n t is t r u e since t h e case m = 0 y i e l d s t h e class o f ( s t r i c t l y ) p o s i t i v e d e f i n i t e
f u n c t i o n s , i.e., ( s t r i c t l y ) c o n d i t i o n a l l y p o s i t i v e definite f u n c t i o n s o f o r d e r zero are
(strictly) positive definite.

63
64 Meshfree Approximation Methods with M A T L A B

A s for p o s i t i v e definite f u n c t i o n s earlier, w e c a n r e s t r i c t ourselves t o r e a l - v a l u e d ,

even f u n c t i o n s <fr a n d real coefficients. A d e t a i l e d discussion is p r e s e n t e d i n [ W e n d -
l a n d (2005a)].

Theorem 7 . 1 . A real-valued continuous even function <3> is called conditionally

s
p o s i t i v e definite o f order m o n M if
N N
E ^2cjC ^(xj k - x k ) > 0 (7.2)
j=i k=i

for any N pairwise distinct points x \ , . . . , x^ R , s

and c = [ c i , . . . , C N ] T
R N

satisfying
N

^r p( ) Cj Xj = 0,
3= 1

for any real-valued polynomial p of degree at most m 1. The function <fr is called
s
s t r i c t l y conditionally positive definite of order m on M if the quadratic form (7.2)
is zero only for c = 0.

T h e m a t r i x A w i t h entries A j k = {xj x) k c o r r e s p o n d i n g t o a r e a l a n d even

s t r i c t l y c o n d i t i o n a l l y p o s i t i v e d e f i n i t e f u n c t i o n $ o f o r d e r m c a n also be i n t e r p r e t e d
as b e i n g p o s i t i v e d e f i n i t e o n t h e space o f v e c t o r s c s u c h t h a t
N
8
^2c (x )jP j = o, p e u ^ .
3= 1

T h u s , i n t h i s sense, A is p o s i t i v e d e f i n i t e o n t h e space o f v e c t o r s c "perpendicular"

t o s-variate p o l y n o m i a l s o f degree a t m o s t m 1.
W e c a n n o w generalize t h e i n t e r p o l a t i o n T h e o r e m G.2 t o t h e case o f g e n e r a l
polynomial reproduction:

Theorem 7 . 2 . If the real-valued even function $ is strictly conditionally positive

definite of order m s
on M. and the points x,..., XN form an ( m l)-unisolvent
set, then the system of linear equations (6.3) is uniquely solvable.

T
Proof. T h e p r o o f is a l m o s t i d e n t i c a l t o t h e p r o o f o f T h e o r e m 6.2. A s s u m e [c, d]
is a s o l u t i o n o f t h e homogeneous l i n e a r s y s t e m , i.e., w i t h y = 0. W e show that
T
[c, d] = 0 is t h e o n l y possible s o l u t i o n .
T
Multiplication of the top block by c yields
T T
c Ac + c Pd = 0.
T T T
F r o m t h e b o t t o m b l o c k o f (6.3) we k n o w P c 0. T h i s implies c P = 0 , and
therefore

T
c Ac = 0. (7.3)
 7. Conditionally Positive Definite Functions 65

Since t h e f u n c t i o n $ is s t r i c t l y c o n d i t i o n a l l y p o s i t i v e d e f i n i t e o f o r d e r m b y a s s u m p -
T
t i o n we k n o w t h a t t h e q u a d r a t i c f o r m o f A ( w i t h coefficients such t h a t P c = 0)
above is zero o n l y for c = 0. T h e r e f o r e (7.3) tells us t h a t c = 0. T h e u n i s o l v e n c y o f
t h e d a t a sites, i.e., t h e l i n e a r independence o f t h e c o l u m n s o f P (c.f. R e m a r k 6.1),
a n d t h e fact t h a t c = 0 g u a r a n t e e d = 0 f r o m t h e t o p b l o c k

Ac + Pd = 0

of (6.3).

7.2 C o n d i t i o n a l l y Positive Definite Functions and Generalized

Fourier Transforms

As before, i n t e g r a l c h a r a c t e r i z a t i o n s h e l p us i d e n t i f y f u n c t i o n s t h a t are s t r i c t l y con-

s
d i t i o n a l l y p o s i t i v e definite o f o r d e r m o n I . A n integral characterization o f con-
d i t i o n a l l y p o s i t i v e definite f u n c t i o n s o f o r d e r m, i.e., a g e n e r a l i z a t i o n o f B o c h n e r ' s
t h e o r e m , can be f o u n d i n t h e p a p e r [Sun ( 1 9 9 3 b ) ] . However, since t h e s u b j e c t m a t -
ter is r a t h e r c o m p l i c a t e d , a n d since i t does n o t r e a l l y h e l p us solve t h e s c a t t e r e d
d a t a i n t e r p o l a t i o n p r o b l e m , we d o n o t m e n t i o n a n y details here.
T h e Fourier transform characterization o f s t r i c t l y conditionally positive definite
s
functions o f order m o n M. also makes use o f some a d v a n c e d t o o l s f r o m analy-
sis. However, since t h i s c h a r a c t e r i z a t i o n is relevant for o u r purposes we s t a t e t h e
result (due t o [iske (1994)]) a n d collect some o f t h e m o s t relevant concepts f r o m
distribution theory in Appendix B .
T h i s d i s t r i b u t i o n a l approach originated i n the manuscript [ M a d y c h and Nelson
(1983)]. M a n y m o r e details c a n be f o u n d i n t h e o r i g i n a l papers m e n t i o n e d above as
w e l l as i n t h e b o o k [ W e n d l a n d (2005a)].

T h e o r e m 7.3. Suppose the complex-valued function <& G B possesses a generalized

s
Fourier transform $ of order m which is continuous on M. \ { 0 } . Then $ is strictly
conditionally positive definite of order m if and only if $ is non-negative and non-
vanishing.

T h e o r e m 7.3 states t h a t s t r i c t l y c o n d i t i o n a l l y p o s i t i v e d e f i n i t e f u n c t i o n s o n W
are characterized b y t h e o r d e r o f t h e s i n g u l a r i t y o f t h e i r generalized F o u r i e r t r a n s -
f o r m at t h e o r i g i n , p r o v i d e d t h a t t h i s generalized F o u r i e r t r a n s f o r m is n o n - n e g a t i v e
a n d non-zero.
Since i n t e g r a l c h a r a c t e r i z a t i o n s s i m i l a r t o Schoenberg's T h e o r e m s 3.6 a n d 3.8
are so c o m p l i c a t e d i n t h e c o n d i t i o n a l l y p o s i t i v e d e f i n i t e case we do n o t p u r s u e t h e
concept o f a c o n d i t i o n a l l y p o s i t i v e definite r a d i a l f u n c t i o n here. The interested
reader is referred t o [ G u o et al. (1993a)] for details. W e w i l l discuss some e x a m -
ples o f r a d i a l f u n c t i o n s v i a t h e F o u r i e r t r a n s f o r m a p p r o a c h i n t h e n e x t c h a p t e r ,
a n d i n C h a p t e r 9 we w i l l e x p l o r e t h e c o n n e c t i o n between c o m p l e t e l y a n d m u l t i p l y
m o n o t o n e f u n c t i o n s a n d c o n d i t i o n a l l y p o s i t i v e definite r a d i a l f u n c t i o n s .
1
 Chapter 8

Examples of Conditionally Positive

Definite Functions

We n o w present a n u m b e r o f s t r i c t l y c o n d i t i o n a l l y p o s i t i v e definite ( r a d i a l ) func-

t i o n s t h a t are covered b y t h e F o u r i e r t r a n s f o r m c h a r a c t e r i z a t i o n T h e o r e m 7.3. T h e
generalized F o u r i e r t r a n s f o r m s for these examples are e x p l i c i t l y c o m p u t e d i n [ W e n d -
l a n d (2005a)]. W e w i l l e s t a b l i s h t h e s t r i c t c o n d i t i o n a l p o s i t i v e definiteness o f these
functions a g a i n i n d e t a i l i n t h e n e x t c h a p t e r w i t h t h e h e l p o f c o m p l e t e l y m o n o t o n e
functions. I n c l u d e d i n t h e examples b e l o w are several o f t h e best k n o w n r a d i a l basic
functions such as t h e m u l t i q u a d r i c due t o [ H a r d y (1971)] a n d t h e t h i n p l a t e spline
due t o [ D u c h o n (1976)].

8.1 E x a m p l e 1: G e n e r a l i z e d M u l t i q u a d r i c s

T h e generalized multiquadrics
2 0 s
$(x) = ( l + WxW ) , x e R, 3 e R \ N ,
0 (8.1)
have generalized F o u r i e r t r a n s f o r m s
9I+/3
/ 3 s / 2
<&M = f 7z^ii^ir - ^ / 3 + s /2(ii^ii)> # ,

of order m = m a x ( 0 , [/?]), w h e r e \3~\ denotes t h e smallest integer greater t h a n o r

equal t o 3. H e r e t h e K v are a g a i n t h e m o d i f i e d Bessel f u n c t i o n s o f t h e second k i n d
o f order v (c.f. Section 4 . 5 ) . N o t e t h a t we need t o exclude p o s i t i v e integer values
o f 3 since t h i s w o u l d lead t o p o l y n o m i a l s o f even degree (see t h e r e l a t e d discussion
in Example 2 below).
Since t h e generalized F o u r i e r t r a n s f o r m s are p o s i t i v e w i t h a s i n g u l a r i t y o f o r d e r
m at t h e o r i g i n , T h e o r e m 7.3 tells us t h a t t h e f u n c t i o n s
f / 3 ] 2
$(x) = (-l) ( 1 + \\x\\ f, 0 < 3 N,
are s t r i c t l y c o n d i t i o n a l l y p o s i t i v e d e f i n i t e o f o r d e r m \8~\ ( a n d h i g h e r ) .
For 3 < 0 t h e F o u r i e r t r a n s f o r m is a classical one a n d we are b a c k t o t h e
generalized inverse m u l t i q u a d r i c s o f S e c t i o n 4.5. These f u n c t i o n s are a g a i n s h o w n
t o be s t r i c t l y c o n d i t i o n a l l y p o s i t i v e definite o f o r d e r m = 0, i.e., s t r i c t l y p o s i t i v e
definite.

67
68 Meshfree Approximation Methods with M A T L A B

Fig. 8.1 Hardy's multiquadric with 3 = | (left) and a generalized multiquadric with 3 = |
2
(right) centered at the origin in R .

F i g u r e 8.1 shows H a r d y ' s " o r i g i n a l " m u l t i q u a d r i c ( w i t h 3 = 1/2, i.e., s t r i c t l y

conditionally positive definite of order 1) a n d a generalized multiquadric with
3 = 5 / 2 (i.e., s t r i c t l y conditionally positive definite of order 3). Note that the
generalized m u l t i q u a d r i c s are no longer " b u m p " f u n c t i o n s (as m o s t o f t h e s t r i c t l y
p o s i t i v e definite f u n c t i o n s w e r e ) , b u t f u n c t i o n s t h a t g r o w w i t h t h e d i s t a n c e f r o m
the origin.
T h e a r g u m e n t s above t o g e t h e r w i t h T h e o r e m 7.2 s h o w t h a t we c a n use H a r d y ' s
multiquadrics i n the form
N
C k 1 + x X f c 2 + d s
V {x)
f = Y^ ^ W - H > x e R ,
k=l
together w i l l the constraint
N
c
E* =
fc=i
t o solve t h e s c a t t e r e d d a t a i n t e r p o l a t i o n p r o b l e m . The resulting interpolant will
be exact for c o n s t a n t d a t a . A s i n o u r earlier discussions we can scale t h e basis
f u n c t i o n s w i t h a shape p a r a m e t e r e b y r e p l a c i n g ||sr|| b y ||||cc||. T h i s does n o t
affect t h e well-posedness o f t h e i n t e r p o l a t i o n p r o b l e m . H o w e v e r , a s m a l l value o f e
gives rise t o " f l a t " basis f u n c t i o n s , whereas a large value o f e p r o d u c e s v e r y steep
functions. A s before, t h e accuracy o f t h e fit w i l l i m p r o v e w i t h decreasing e w h i l e
t h e s t a b i l i t y w i l l decrease, a n d t h e n u m e r i c a l results w i l l become i n c r e a s i n g l y less
reliable. For F i g u r e 8.1 we used t h e shape p a r a m e t e r e = 1.
B y T h e o r e m 9.7 b e l o w we c a n also solve t h e s c a t t e r e d d a t a i n t e r p o l a t i o n p r o b l e m
using t h e s i m p l e r e x p a n s i o n
N
2 s
V (x)
f = J2ckV^ + \\x-Xk\\ , x e R .
k=i
T h i s is w h a t H a r d y p r o p o s e d t o d o i n his w o r k i n t h e e a r l y 1970s (see, e.g., [ H a r d y
(1971)]).
 8. Examples of Conditionally Positive Definite Functions 69

8.2 E x a m p l e 2: R a d i a l P o w e r s

T h e radial powers
S
{x) = \\x\f, x e 3R , 0 < 3 2 N , (8.2)
have generalized F o u r i e r t r a n s f o r m s
20+S/2Y(S0\
= L_2j.|| ,||-/3-*
a u=AO
{ J 11 11
T(-3/2) ' ^ '
of order m = \3/2~\. Therefore, the functions
$(x) = ( - 1 ) ^ / 2 1 \\xf, 0</32N,
are s t r i c t l y c o n d i t i o n a l l y p o s i t i v e d e f i n i t e o f o r d e r m = \8/2~\ ( a n d h i g h e r ) .
T h i s shows t h a t t h e basic f u n c t i o n $(cc) = ||a?||2 used for t h e d i s t a n c e m a t r i x
fits i n t h e i n t r o d u c t o r y c h a p t e r are s t r i c t l y c o n d i t i o n a l l y p o s i t i v e d e f i n i t e o f o r d e r
one. A c c o r d i n g t o T h e o r e m 7.2 we s h o u l d have used these basic f u n c t i o n s t o g e t h e r
w i t h an a p p e n d e d c o n s t a n t . H o w e v e r , T h e o r e m 9.7 b e l o w p r o v i d e s t h e j u s t i f i c a t i o n
for t h e i r use as a p u r e distance m a t r i x .
I n F i g u r e 8.2 we show r a d i a l cubics {3 = 3, i.e., s t r i c t l y c o n d i t i o n a l l y p o s i t i v e
definite of o r d e r 2) a n d q u i n t i c s (8 = 5, i.e., s t r i c t l y c o n d i t i o n a l l y p o s i t i v e d e f i n i t e
of order 3 ) .
N o t e t h a t we h a d t o exclude even powers i n ( 8 . 2 ) . T h i s is clear since a n even
power c o m b i n e d w i t h t h e square r o o t i n t h e d e f i n i t i o n o f t h e E u c l i d e a n n o r m results
i n a p o l y n o m i a l a n d we have a l r e a d y d e c i d e d t h a t p o l y n o m i a l s c a n n o t be used
for i n t e r p o l a t i o n at a r b i t r a r i l y s c a t t e r e d m u l t i v a r i a t e sites.
N o t e t h a t r a d i a l powers are n o t affected b y a s c a l i n g o f t h e i r a r g u m e n t . I n o t h e r
w o r d s , r a d i a l powers are shape parameter free. T h i s has t h e advantage t h a t t h e
user need n o t w o r r y a b o u t f i n d i n g a " g o o d " value o f e. O n t h e o t h e r h a n d , w e w i l l
see below t h a t r a d i a l powers w i l l n o t be able t o achieve t h e s p e c t r a l convergence
rates t h a t are possible w i t h some o f t h e o t h e r basic f u n c t i o n s such as Gaussians a n d
generalized (inverse) m u l t i q u a d r i c s .

2
Fig. 8.2 Radial cubic (left) and quintic (right) centered at the origin in R .
70 Meshfree Approximation Methods with M A T L A B

8.3 E x a m p l e 3: T h i n P l a t e Splines

I n t h e p r e v i o u s e x a m p l e we h a d t o r u l e o u t even powers. H o w e v e r , i f t h e even r a d i a l

powers are m u l t i p l i e d b y a l o g t e r m , t h e n w e are b a c k i n business.
D u c h o n ' s thin plate splines (or M e i n g u e t ' s surface splines)
2/3 s
&(x) = ||x|| l o g ||x||, x e R , 3eN, (8.3)

have generalized F o u r i e r t r a n s f o r m s
0 + 1 2 l 3 1 + s 2 s 213
= {-l) 2 - / r(B + s/2)8\\\u>\\- -

of o r d e r m = 3 + 1. T h e r e f o r e , t h e f u n c t i o n s
+ 1 2
$(x) = (-l)0 ||x|| 01og||a;||, /?GN,

are s t r i c t l y c o n d i t i o n a l l y p o s i t i v e d e f i n i t e o f o r d e r m = 3 + 1. I n p a r t i c u l a r , we can
use
N
2 2
Vf(x) = "Y^CkHx - c c | | l o g ||cc - x \\
fc k + di + d x 2 + d y, 3 x = (x,y) G R,
fc=i
together w i l l the constraints

N N N

k=l fc=l k=l

2
t o solve t h e s c a t t e r e d d a t a i n t e r p o l a t i o n p r o b l e m i n R p r o v i d e d t h e d a t a sites are
not a l l c o l l i n e a r . T h e r e s u l t i n g i n t e r p o l a n t w i l l be e x a c t for d a t a c o m i n g f r o m a
bivariate linear function.

Fig. 8.3 "Classical" thin plate spline (left) and order 3 thin plate spline (right) centered at the
2
origin in R .

F i g u r e 8.3 shows t h e "classical" t h i n p l a t e spline ( w i t h 3 = 1, i.e., strictly

4
c o n d i t i o n a l l y p o s i t i v e d e f i n i t e o f o r d e r 2) a n d t h e o r d e r 3 s p l i n e <>(x) = ||a?|| l o g ||cc||.
N o t e t h a t t h e t h i n p l a t e spline basic f u n c t i o n s are n o t m o n o t o n e . A l s o , b o t h graphs
d i s p l a y e d i n F i g u r e 8.3 c o n t a i n a p o r t i o n w i t h n e g a t i v e f u n c t i o n values.
 8. Examples of Conditionally Positive Definite Functions 71

A s w i t h r a d i a l powers, use o f a shape p a r a m e t e r i n c o n j u n c t i o n w i t h t h i n p l a t e

splines is pointless. F i n a l l y , w e n o t e t h a t t h e families o f r a d i a l powers a n d t h i n p l a t e
splines are o f t e n referred t o c o l l e c t i v e l y as polyharmonic splines.
T h e r e is n o r e s u l t t h a t states t h a t i n t e r p o l a t i o n w i t h t h i n p l a t e splines (or a n y
o t h e r s t r i c t l y c o n d i t i o n a l l y p o s i t i v e d e f i n i t e f u n c t i o n o f o r d e r m > 2) w i t h o u t t h e
a d d i t i o n o f a n a p p r o p r i a t e degree m 1 p o l y n o m i a l is w e l l - p o s e d . T h e o r e m 9.7
q u o t e d several t i m e s before covers o n l y t h e case m = 1.
J
 Chapter 9

Conditionally Positive Definite

Radial Functions

A s for s t r i c t l y p o s i t i v e definite r a d i a l f u n c t i o n s , we w i l l be able t o connect s t r i c t l y

c o n d i t i o n a l l y p o s i t i v e definite r a d i a l f u n c t i o n s t o c o m p l e t e l y m o n o t o n e a n d m u l t i p l y
m o n o t o n e functions, a n d t h u s be able t o o b t a i n a c r i t e r i o n for checking c o n d i t i o n a l
p o s i t i v e definiteness o f r a d i a l f u n c t i o n s t h a t is easier t o use t h a n t h e generalized
Fourier t r a n s f o r m i n t h e p r e v i o u s c h a p t e r s .

9.1 Conditionally Positive Definite R a d i a l Functions and

Completely Monotone Functions

I n analogy t o t h e discussion i n S e c t i o n 3.3 we n o w focus o n c o n d i t i o n a l l y p o s i t i v e

S
definite functions t h a t are r a d i a l o n R for a l l s. T h e p a p e r [ G u o et al. (1993a)]
b y G u o , H u a n d Sun c o n t a i n s a n i n t e g r a l c h a r a c t e r i z a t i o n for such f u n c t i o n s . This
c h a r a c t e r i z a t i o n is t o o t e c h n i c a l t o be i n c l u d e d here.
A n o t h e r i m p o r t a n t r e s u l t i n [ G u o et al. (1993a)] is a c h a r a c t e r i z a t i o n o f c o n -
S
d i t i o n a l l y p o s i t i v e definite r a d i a l f u n c t i o n s o n R for a l l s i n t e r m s o f c o m p l e t e l y
monotone functions.

2
T h e o r e m 9 . 1 . Let p> G C [ 0 , oo) n C ( 0 , o o ) . Then the function $ = p(\\ | | ) is
S
conditionally positive definite of order m and radial on R for all s if and only if
m m
(l) <>( ) is completely monotone on ( 0 , o o ) .

Proof. T h e fact t h a t c o m p l e t e m o n o t o n i c i t y i m p l i e s c o n d i t i o n a l p o s i t i v e d e f i n i t e -
ness was p r o v e d i n [ M i c c h e l l i (1986)]. M i c c h e l l i also c o n j e c t u r e d t h a t t h e converse
holds a n d gave a s i m p l e p r o o f for t h i s i n t h e case m = 1. For m = 0 t h i s is Schoen-
S
berg's c h a r a c t e r i z a t i o n o f p o s i t i v e definite r a d i a l functions o n R for a l l s i n t e r m s o f
c o m p l e t e l y m o n o t o n e f u n c t i o n s ( T h e o r e m 5.2). T h e r e m a i n i n g p a r t o f t h e t h e o r e m
is s h o w n i n [Guo et al. (1993a)].

I n order t o get s t r i c t c o n d i t i o n a l p o s i t i v e definiteness we need t o generalize

T h e o r e m 5.3, i.e., t h e fact t h a t ip n o t be c o n s t a n t . T h i s leads t o (see [Wendland
(2005a)])

73
74 Meshfree Approximation Methods with M A T L A B

T h e o r e m 9.2. If <p is as in Theorem 9.1 and not a polynomial of degree at most

5
m, then <E> is strictly conditionally positive definite of order m and radial on M for
all s.

W e c a n n o w m o r e easily v e r i f y t h e c o n d i t i o n a l p o s i t i v e definiteness o f t h e func-

tions listed i n the previous chapter.

E x a m p l e 9.1. T h e f u n c t i o n s

p(r) = ( - 1 ) ^ 1 ( 1 + 7-)", 0 < / 3 N

imply
r / 3 1 e
^)(r) = (-l) W - 1) [8 - I + 1 ) ( 1 + rf-

so t h a t
(_i)r/3i^(r/3i) ( r ) ( / 5 _ ^ + 1 ) ( i + r ) /3-r/3i

is c o m p l e t e l y m o n o t o n e . M o r e o v e r , m \8~\ is t h e smallest possible m s u c h t h a t

m m
( l ) t p ( ) is c o m p l e t e l y m o n o t o n e . Since 8 N we k n o w t h a t <p is n o t a p o l y n o -
m i a l , a n d therefore t h e generalized m u l t i q u a d r i c s (c./. ( 8 . 1 ) )

*(NI) = ( - i ) ^ ( i + NIV, /5>o,

S
are s t r i c t l y c o n d i t i o n a l l y p o s i t i v e d e f i n i t e o f o r d e r m > and radial on R for a l l
values o f s.

E x a m p l e 9.2. T h e f u n c t i o n s
2 2
<p(r) = ( - 1 ) ^ / V ^ , 0<8<2N,

imply

^>(r) = ( - 1 ) ^ 1 f - l ) - . . r ^ 2
^
2
so t h a t ( l ) r ^ / l ^ ( r / 3 / 2 l ) j s c o m p l e t e l y m o n o t o n e a n d m = \B/2~\ is t h e smallest
7 7 1
possible m such t h a t ( l ) ^ ) is c o m p l e t e l y m o n o t o n e . Since /? is n o t a n even
integer ip is n o t a p o l y n o m i a l , a n d therefore, t h e r a d i a l powers (c.f. ( 8 . 2 ) )
W 2 ]
*(||*||) = (-l) \\xf, 3>0, 82N,
S
are s t r i c t l y c o n d i t i o n a l l y p o s i t i v e d e f i n i t e o f o r d e r m > \8/2\ and radial on R for
a l l s.

E x a m p l e 9.3. T h e t h i n p l a t e splines (c.f. ( 8 . 3 ) )

3 + 1 2 / 3
*(||x||) = ( - l ) ' |||| log||x||, PeN,
S
are s t r i c t l y c o n d i t i o n a l l y p o s i t i v e d e f i n i t e o f o r d e r m > 8 + 1 a n d r a d i a l o n R for
a l l s. T o see t h i s we observe t h a t
1 2 3 2
2$(||o ||) = ; (-l)^ || r|| / log(||a || ).
; ;
 9. Conditionally Positive Definite Radial Functions 75

Therefore, we l e t

<p{r) = (-l^+Vlogr, 8eN,

w h i c h is o b v i o u s l y n o t a p o l y n o m i a l . D i f f e r e n t i a t i n g ip we get

pW(r) +1
= {-lf (3{(3 - l ) . . . ( 8 - e + iy-< logr + p {r),
e 1<<P,

w i t h pe a p o l y n o m i a l o f degree (3 1. T h e r e f o r e , t a k i n g i = (3 we have

<pW(r) = BWogr + C

and

.09+D( ) = ( - l ) / 3 + i [
r >

w h i c h is c o m p l e t e l y m o n o t o n e o n (0, o o ) .

9.2 Conditionally Positive Definite R a d i a l Functions a n d M u l t i p l y

Monotone Functions

F i n a l l y , [ M i c c h e l l i (1986)] p r o v e d a m o r e general v e r s i o n o f T h e o r e m 5.5 r e l a t i n g

s
c o n d i t i o n a l l y p o s i t i v e definite r a d i a l f u n c t i o n s o f o r d e r m o n R ( f o r some fixed
value o f s) a n d m u l t i p l y m o n o t o n e f u n c t i o n s . W e state a stronger version due t o
[ B u h m a n n (1993a)] w h i c h ensures s t r i c t c o n d i t i o n a l p o s i t i v e definiteness.

T h e o r e m 9 . 3 . Let k = [s/2j m + 2 be a positive integer, and suppose ip

m _ 1
C [ 0 , oo) is not a polynomial of degree at most m. If (1) k-times
monotone on (0, oo) but not constant, then ip is strictly conditionally positive definite
s
of order m and radial on M for any s such that [s/2] < k + m 2.

J u s t as we showed earlier t h a t c o m p a c t l y s u p p o r t e d r a d i a l f u n c t i o n s c a n n o t b e
s
s t r i c t l y p o s i t i v e definite o n R for a l l s, i t is i m p o r t a n t t o n o t e t h a t t h e r e are n o
t r u l y c o n d i t i o n a l l y p o s i t i v e definite f u n c t i o n s w i t h c o m p a c t s u p p o r t . M o r e precisely
(see [ W e n d l a n d (2005a)]),

S
T h e o r e m 9 . 4 . Assume that the complex-valued function $ G C ( R ) has compact
support. If & is strictly conditionally positive definite of (minimal) order m, then
m is necessarily zero, i.e., $ is already strictly positive definite.

Proof. T h e hypotheses o n $ ensure t h a t i t is i n t e g r a b l e , a n d therefore i t pos-

sesses a classical F o u r i e r t r a n s f o r m ^> w h i c h is c o n t i n u o u s . F o r i n t e g r a b l e f u n c t i o n s
t h e generalized F o u r i e r t r a n s f o r m coincides w i t h t h e classical F o u r i e r transform.
s
T h e o r e m 7.3 ensures t h a t is n o n - n e g a t i v e o n R \ { 0 } a n d not identically equal
t o zero. B y c o n t i n u i t y we also get <&(0) > 0, a n d T h e o r e m 3.5 shows t h a t $ is
s t r i c t l y p o s i t i v e definite.
76 Meshfree Approximation Methods with M A T L A B

T h e o r e m 9.3 t o g e t h e r w i t h T h e o r e m 9.4 i m p l i e s t h a t i f we p e r f o r m ra-fold a n t i -

d i f f e r e n t i a t i o n o n a n o n - c o n s t a n t A;-times m o n o t o n e f u n c t i o n , t h e n we o b t a i n a func-
s
t i o n t h a t is s t r i c t l y p o s i t i v e d e f i n i t e a n d r a d i a l o n R for \_s/2\ < k + m 2.

? r r
Example 9.4. T h e f u n c t i o n v fe( ) = ( 1 ~ )+ is fc-times m o n o t o n e (see E x -
a m p l e 5.5 i n S e c t i o n 5.2). To avoid the i n t e g r a t i o n constant for t h e compactly
s u p p o r t e d t r u n c a t e d p o w e r f u n c t i o n we c o m p u t e t h e a n t i - d e r i v a t i v e v i a t h e i n t e g r a l
o p e r a t o r I of E x a m p l e 5.6 i n S e c t i o n 5.2, i.e.,
oo poo / i \fc

/
Mt)dt = j ( l ~ t ) l d t = ) ~ ^ ( l - r ) k
+
+ 1
.
I f we a p p l y m - f o l d a n t i - d i f f e r e n t i a t i o n we get
(
I-Mr) = / / - W ) = ( , + 1 ) ( f c ; 2 > , , ( + m ) ( l- r ) ^ .

T h e r e f o r e , b y T h e o r e m 9.3, t h e f u n c t i o n

<p(r) = ( 1 - r ) * +
s
is s t r i c t l y c o n d i t i o n a l l y p o s i t i v e d e f i n i t e o f o r d e r m a n d r a d i a l o n R for [s/2\ <
s
k + rn 2, a n d b y T h e o r e m 9.4 i t is even s t r i c t l y p o s i t i v e d e f i n i t e a n d r a d i a l o n M. .
T h i s was also observed i n E x a m p l e 6 o f C h a p t e r 4. I n fact, we saw t h e r e t h a t <p is
s
s t r i c t l y p o s i t i v e d e f i n i t e a n d r a d i a l o n R. for [s/2j < k + m 1.

W e see t h a t we c a n c o n s t r u c t s t r i c t l y p o s i t i v e d e f i n i t e c o m p a c t l y supported
radial functions by anti-differentiating the t r u n c a t e d power f u n c t i o n . T h i s is es-
sentially the approach taken by Wendland to construct his p o p u l a r compactly
s u p p o r t e d r a d i a l basis f u n c t i o n s . W e p r o v i d e m o r e d e t a i l s o f his c o n s t r u c t i o n i n
Chapter 11.

9.3 Some Special Properties of Conditionally Positive Definite

Functions of O r d e r O n e

Since a n N x N m a t r i x t h a t is c o n d i t i o n a l l y p o s i t i v e d e f i n i t e o f o r d e r one is p o s i t i v e
d e f i n i t e o n a subspace o f d i m e n s i o n N 1 i t has t h e i n t e r e s t i n g p r o p e r t y that
at least N 1 o f i t s eigenvalues are p o s i t i v e . T h i s follows i m m e d i a t e l y f r o m the
C o u r a n t - F i s c h e r t h e o r e m o f l i n e a r a l g e b r a (see e.g., p . 550 o f [ M e y e r ( 2 0 0 0 ) ] ) :

T h e o r e m 9 . 5 ( C o u r a n t - F i s c h e r ) . Let A be a real symmetric N x N matrix with

eigenvalues A i > A2 > > \N, then
T
Afc = max min x Ax
dimV=fc ^GV
II a, 11=1

and
T
Afc = min max x Ax.
dimV=iV-fc+l <*ev
II <= | | = i
 9. Conditionally Positive Definite Radial Functions 77

W i t h an a d d i t i o n a l a s s u m p t i o n o n A we c a n m a k e a n even s t r o n g e r s t a t e m e n t .

Theorem 9 . 6 . An N x N matrix A which is conditionally positive definite of or-

der one and has a non-positive trace possesses one negative and N 1 positive
eigenvalues.

Proof. L e t A i > A2 > > \ N d e n o t e t h e eigenvalues o f A. F r o m the Courant-

Fischer t h e o r e m we get

XN-I = max m i n x Ax T
> min T
c Ac > 0,
dimV=iV-l =>=V c: E--k=
II a> 11 = 1 l|c||=l

so t h a t A has at least N 1 p o s i t i v e eigenvalues. B u t since tr(A) Ylk=i ^ 0'

A also m u s t have at least one n e g a t i v e eigenvalue.

N o t e t h a t t h e a d d i t i o n a l h y p o t h e s i s o f T h e o r e m 9.6 is satisfied for t h e i n t e r p o -

l a t i o n m a t r i x r e s u l t i n g f r o m ( t h e negative) o f R B F s such as H a r d y ' s multiquadric
or t h e linear r a d i a l f u n c t i o n ip(r) = r since i t s d i a g o n a l elements c o r r e s p o n d t o t h e
value o f t h e basic f u n c t i o n at t h e o r i g i n .
M o r e o v e r , we w i l l n o w use T h e o r e m 9.6 t o conclude t h a t we c a n use radial
functions t h a t are s t r i c t l y c o n d i t i o n a l l y p o s i t i v e definite o f o r d e r one (such as t h e
m u l t i q u a d r i c , 0 < 3 < 1, a n d t h e n o r m basic f u n c t i o n ) without a p p e n d i n g t h e con-
s t a n t t e r m t o solve t h e s c a t t e r e d d a t a i n t e r p o l a t i o n p r o b l e m . T h i s was first p r o v e d
by [ M i c c h e l l i (1986)] a n d m o t i v a t e d b y H a r d y ' s earlier w o r k w i t h multiquadrics
a n d Franke's conjecture t h a t t h e m a t r i x A is n o n - s i n g u l a r i n t h i s case (see [Franke
(1982a)]).

T h e o r e m 9 . 7 ( I n t e r p o l a t i o n ) . Suppose <fr is strictly conditionally positive defi-

nite of order one and that <&(0) < 0. Then for any distinct points x \ , . . . , XN G l s

the matrix A with entries Ajk = &{xj x)k has N 1 positive and one negative
eigenvalue, and is therefore non-singular.

Proof. Clearly, t h e m a t r i x A is c o n d i t i o n a l l y p o s i t i v e definite o f o r d e r one. M o r e -

over, t h e t r a c e o f A is g i v e n b y ti(A) = N<&(0) < 0. Therefore, T h e o r e m 9.6 applies
a n d t h e s t a t e m e n t follows.

A s m e n t i o n e d above, t h i s t h e o r e m covers t h e generalized m u l t i q u a d r i c s <E>(a?) =

(1-f- H ^ l l ) ^ w i t h 0 < 3 < 1 ( w h i c h includes t h e H a r d y m u l t i q u a d r i c ) . T h e t h e o r e m
also covers t h e r a d i a l powers <&(x) = \\x\\P for 0 < 3 < 2 ( i n c l u d i n g t h e E u c l i d e a n
distance f u n c t i o n ) .
A n o t h e r special p r o p e r t y o f a c o n d i t i o n a l l y p o s i t i v e definite f u n c t i o n o f o r d e r
one is

Lemma 9 . 1 . If C is an arbitrary real constant and the real even function $ is

(strictly) conditionally positive definite of order one, then & + C is also (strictly)
conditionally positive definite of order one.
78 Meshfree Approximation Methods with M A T L A B

Proof. S i m p l y consider
N N N N N N

Yl J2 j k[$(xj
c c
- x) k + C] = E ^CjCk&ixj - Xk) + E ^CjCkC.
3=1 k=l 3=1 fc=l j=l fc=l
T h e second t e r m o n t h e r i g h t is zero since <3> is c o n d i t i o n a l l y p o s i t i v e d e f i n i t e o f
c = a n
o r d e r one, i.e., X ^ y L i j 0' d thus the statement follows.
 Chapter 10

Miscellaneous Theory: Other Norms and

Scattered Data Fitting on Manifolds

10.1 Conditionally Positive Definite Functions and p-Norms

I n C h a p t e r 1 we used i n t e r p o l a t i o n w i t h d i s t a n c e m a t r i c e s as a m u l t i v a r i a t e g e n e r a l -
i z a t i o n o f t h e piecewise l i n e a r a p p r o a c h . O u r choice o f t h e distance m a t r i x a p p r o a c h
was m o t i v a t e d b y t h e fact t h a t t h e associated basis f u n c t i o n s , &j{x) = \\x X j \ \
w o u l d satisfy t h e dependence o n t h e d a t a sites i m p o s e d o n a m u l t i v a r i a t e i n t e r p o -
lation m e t h o d by the M a i r h u b e r - C u r t i s theorem. We made the (natural?) choice
o f u s i n g t h e E u c l i d e a n ( 2 - n o r m ) d i s t a n c e f u n c t i o n , a n d t h e n showed i n subsequent
chapters t h a t t h e f u n c t i o n &(x) = \\x\\2 is s t r i c t l y c o n d i t i o n a l l y p o s i t i v e d e f i n i t e
s
of o r d e r one a n d r a d i a l o n R , a n d t h u s o u r distance m a t r i x a p p r o a c h was i n d e e d
well-posed v i a M i c c h e l l i ' s T h e o r e m 9.7.
W e n o w b r i e f l y consider s o l v i n g t h e s c a t t e r e d d a t a i n t e r p o l a t i o n p r o b l e m w i t h
r a d i a l f u n c t i o n s based o n o t h e r p - n o r m s . These n o r m s are defined as u s u a l as

1 < p < oo.

T h e c o n t e n t o f t h i s s e c t i o n is m o s t l y t h e s u b j e c t o f t h e p a p e r [ B a x t e r (1991)].
I f we consider o n l y distance m a t r i c e s , i.e., i n t e r p o l a t i o n m a t r i c e s g e n e r a t e d b y
t h e basic f u n c t i o n <&(cc) = ||cc|| , t h e n i t was s h o w n i n [ D y n et al.
p (1989)] t h a t
t h e choice p = 1 leads t o a s i n g u l a r m a t r i x a l r e a d y for v e r y s i m p l e sets o f d i s t i n c t
interpolation points. For e x a m p l e , i f X = { ( 0 , 0 ) , ( 1 , 0 ) , ( 1 , 1 ) , ( 0 , 1 ) } t h e n t h e 1-
n o r m distance m a t r i x is g i v e n b y

"0 1 2 1 "
10 12
2 10 1
12 10

a n d i t is easy t o v e r i f y t h a t t h i s m a t r i x is s i n g u l a r . T h i s r e s u l t has d i s c o u r a g e d
people f r o m u s i n g 1 - n o r m r a d i a l basis f u n c t i o n s .
However, i f we use, e.g., N H a l t o n p o i n t s , t h e n we have never e n c o u n t e r e d a
s i n g u l a r 1-norm distance m a t r i x i n a l l o f o u r n u m e r i c a l e x p e r i m e n t s . I n fact, t h e

79
80 Meshfree Approximation Methods with M A T L A B

m a t r i x seems t o have N 1 negative a n d one p o s i t i v e eigenvalue (just as p r e d i c t e d

b y T h e o r e m 9.7 for t h e 2 - n o r m case).
F i g u r e 10.2 shows various i n t e r p o l a n t s t o t h e l i n e a r f u n c t i o n f(x,y) = (x + y)/2
on t h e u n i t square. T h e i n t e r p o l a n t is false c o l o r e d a c c o r d i n g t o t h e m a x i m u m
e r r o r . I n t h e t o p r o w o f t h e figure we used a 1-norm d i s t a n c e m a t r i x based o n 1089
H a l t o n p o i n t s . T h e M A T L A B code for g e n e r a t i n g a p - n o r m distance m a t r i x f i t is
v i r t u a l l y i d e n t i c a l t o o u r earlier code i n P r o g r a m s 1.1 a n d 1.2. T h e o n l y change
r e q u i r e d is t h e replacement o f lines 6 a n d 8 o f P r o g r a m 1.1 b y

6 DM = DM + a b s ( d r - c c ) . " p ;
8 DM = DM."(l/p);

W e can also use t h i s m o d i f i c a t i o n o f P r o g r a m 1.1 t o p r o d u c e m o r e general R B F

i n t e r p o l a n t s (see t h e e x a m p l e w i t h p - n o r m Gaussians i n t h e b o t t o m r o w o f F i g -
u r e 10.2 b e l o w ) .
S i m i l a r t o t h e 1-norm result f r o m [ D y n et al. (1989)] q u o t e d above i t was s h o w n
in [ B a x t e r (1991)] t h a t for p > 2 we c a n n o t i n general g u a r a n t e e n o n - s i n g u l a r
distance m a t r i c e s , either. O n t h e o t h e r h a n d , a n u m b e r o f n u m e r i c a l e x p e r i m e n t s
showed t h e p - n o r m m a t r i c e s t o be n o n - s i n g u l a r p r o v i d e d u n i f o r m l y spaced o r H a l t o n
2
p o i n t s i n [0, l ] were used. T h e second r o w o f F i g u r e 10.2 shows d i s t a n c e m a t r i x
i n t e r p o l a n t s t o f(x,y) = (x + y)/2 o n t h e u n i t square u s i n g a p - n o r m distance
m a t r i x for p = 10 a n d p = 100 based o n 25 u n i f o r m l y spaced p o i n t s .
These examples show t h a t c e r t a i n l y n o t a l l is lost w h e n u s i n g p - n o r m r a d i a l
basis f u n c t i o n s . T h e s i t u a t i o n is s i m i l a r as w i t h t h e use o f K a n s a ' s m e t h o d for t h e
c o l l o c a t i o n s o l u t i o n o f e l l i p t i c P D E s (see C h a p t e r 3 8 ) . T h e r e d o exist c o n f i g u r a t i o n s
of d a t a p o i n t s for w h i c h t h e i n t e r p o l a t i o n m a t r i x becomes s i n g u l a r . H o w e v e r , these
c o n f i g u r a t i o n s m a y be rare, a n d therefore t h e use o f p - n o r m r a d i a l basis f u n c t i o n s
m a y be j u s t i f i e d i n m a n y cases. W e p o i n t o u t t h a t we used n o r m s for p > 2 even
t h o u g h t h e B a x t e r result m e n t i o n e d above guarantees existence o f d a t a sets X for
w h i c h t h e i n t e r p o l a t i o n m a t r i x w i l l be s i n g u l a r . For o u r examples t h e i n t e r p o l a t i o n
m a t r i x was far f r o m s i n g u l a r . U s i n g 25 u n i f o r m l y spaced d a t a sites t h e m a t r i c e s
a g a i n e x h i b i t e d 24 n e g a t i v e a n d one p o s i t i v e eigenvalue. T h i s use o f p - n o r m r a d i a l
basis f u n c t i o n s c e r t a i n l y deserves f u r t h e r i n v e s t i g a t i o n .
T h e case 1 < p < 2, however, is m u c h b e t t e r u n d e r s t o o d . I n [ B a x t e r (1991)] we
find

T h e o r e m 1 0 . 1 . Suppose 1 < p < 2 and let A be the p-norm distance matrix with
entries

Ajk Xk\\p, j,k = l,...,N.

Then the matrix A is conditionally positive definite of order one. Moreover,

it is strictly conditionally positive definite of order one if N > 2 and the points
X i , . . . , XN are distinct.
 10. Miscellaneous Theory: Other Norms and Scattered Data Fitting on Manifolds 81

T h i s t h e o r e m is d e r i v e d f r o m a m u c h earlier t h e o r e m b y Schoenberg r e l a t i n g
c o n d i t i o n a l l y p o s i t i v e definite m a t r i c e s o f o r d e r one a n d E u c l i d e a n d i s t a n c e m a t r i -
ces. W h e n Schoenberg first s t u d i e d c o n d i t i o n a l l y p o s i t i v e d e f i n i t e m a t r i c e s o f o r d e r
one t h i s was i n c o n n e c t i o n w i t h i s o m e t r i c e m b e d d i n g s . Based o n earlier w o r k b y
K a r l M e n g e r [Menger (1928)] Schoenberg d e r i v e d t h e f o l l o w i n g result c h a r a c t e r i z i n g
c e r t a i n c o n d i t i o n a l l y p o s i t i v e definite m a t r i c e s as E u c l i d e a n distance m a t r i c e s (see
[Schoenberg (1937)]).

T h e o r e m 10.2 ( S c h o e n b e r g - M e n g e r ) . Let A be a real symmetric N x N ma-

trix with all diagonal entries zero and all other elements positive. Then A is
conditionally positive semi-definite of order one if and only if there exist N points
2/1, , VN R N
for which

Ajk = I I ^ -Vk\\l-
N
These points are the vertices of a simplex in ~R .

I n t h e t h i r d r o w o f F i g u r e 10.2 we d i s p l a y t h e i n t e r p o l a n t s t o t h e test f u n c t i o n
2
f(x, y) = (x+y)/2 o n [0, l ] u s i n g distance m a t r i x i n t e r p o l a t i o n based o n 25 e q u a l l y
spaced p o i n t s a n d p - n o r m s w i t h p = 1.001 a n d p = 2. Since we use a p l a i n d i s t a n c e
i n t e r p o l a n t , i.e., $(x) = ||aj|| p i t is r e m a r k a b l e t h a t t h e e r r o r u s i n g t h e p = 1.001-
n o r m is a b o u t t w o orders o f m a g n i t u d e smaller t h a n t h e n e x t best p - n o r m d i s t a n c e
m a t r i x fit a m o n g o u r e x p e r i m e n t s ( w h i c h we o b t a i n e d for p = 100, c.f. F i g u r e 10.2).
T h e use o f different p - n o r m s for different a p p l i c a t i o n s has n o t been s t u d i e d
carefully i n t h e l i t e r a t u r e .
T w o o t h e r results r e g a r d i n g i n t e r p o l a t i o n w i t h p - n o r m r a d i a l basis f u n c t i o n s
can also be f o u n d i n t h e l i t e r a t u r e . I n [ W e n d l a n d (2005a)] we find a reference t o
[ Z a s t a v n y i (1993)] a c c o r d i n g t o w h i c h for space dimensions s > 3 t h e o n l y
s
f u n c t i o n t h a t is p o s i t i v e d e f i n i t e a n d p - n o r m r a d i a l o n M is t h e zero f u n c t i o n .
A g a i n , s o m e w h a t d i s c o u r a g i n g news. H o w e v e r , t h e r e is also g o o d news. T h e f o l l o w -
i n g r a t h e r p o w e r f u l t h e o r e m comes f r o m [ B a x t e r (1991)]. B a x t e r calls t h e m a t r i x
A o f T h e o r e m 10.2 a n almost negative definite m a t r i x (c.f. the remarks following
D e f i n i t i o n 6.2).

T h e o r e m 1 0 . 3 . Let A be an N x N matrix that is conditionally positive semi-

2
definite of order one with all diagonal entries zero, and let ip( ) be a function that
s
is conditionally positive definite of order one and radial on M . Then the matrix
defined by

B jk =-<p(Ajk), j,k = l,...,N,

is conditionally positive semi-definite of order one. Moreover, if N > 2 and no

off-diagonal elements of A vanish, then B is strictly conditionally positive definite
2
of order one whenever <p( ) is strictly conditionally positive definite of order one.
82 Meshfree Approximation Methods with M A T L A B

r a
Proof. B y Schoenberg's T h e o r e m 10.2 w e c a n w r i t e Ajk = \\yj f P"
N 2
p r o p r i a t e p o i n t s yj ~R B y a s s u m p t i o n p{ ) is c o n d i t i o n a l l y p o s i t i v e d e f i n i t e o f
o r d e r one a n d r a d i a l , a n d therefore B is c o n d i t i o n a l l y p o s i t i v e s e m i - d e f i n i t e o f o r d e r
one. M o r e o v e r , i f Ajk ^ 0 for a l l o f f - d i a g o n a l elements, t h e n j / i , . . . , ? / A T are d i s t i n c t ,
2
a n d therefore B is s t r i c t l y c o n d i t i o n a l l y p o s i t i v e d e f i n i t e o f o r d e r one p r o v i d e d (p( )
is s t r i c t l y c o n d i t i o n a l l y p o s i t i v e d e f i n i t e o f o r d e r one.

Since B a x t e r also shows t h a t i f A is a 1 - n o r m d i s t a n c e m a t r i x , t h e n A is a

c o n d i t i o n a l l y p o s i t i v e s e m i - d e f i n i t e m a t r i x o f o r d e r one, T h e o r e m 10.3 g u a r a n t e e s
t h a t we c a n use m a n y " s t a n d a r d " r a d i a l basic f u n c t i o n s i n c o n j u n c t i o n w i t h t h e 1-
n o r m for R B F i n t e r p o l a t i o n . For e x a m p l e , t h e use o f 1 - n o r m Gaussians is j u s t i f i e d
by T h e o r e m 10.3. I n t h e l i t e r a t u r e one c a n also find a n a n a l o g o f B o c h n e r ' s t h e o r e m
for p o s i t i v e d e f i n i t e 1 - n o r m r a d i a l f u n c t i o n s d u e t o [ C a m b a n i s at al. (1983)] (see
also [ W e n d l a n d ( 2 0 0 5 a ) ] ) .
s x
F i g u r e 10.1 shows p - n o r m Gaussians <3>(:E) = e~ " " p for p = 1 a n d p = 10. A
shape p a r a m e t e r e = 3 was used. I n t e r p o l a n t s t o t h e f u n c t i o n f(x, y) = (x + y)/2 at
2
25 e q u a l l y spaced p o i n t s i n [0, l ] u s i n g these basic f u n c t i o n s w i t h e = 1 are s h o w n
i n t h e b o t t o m r o w o f F i g u r e 10.2.

2
Fig. 10.1 p-norm Gaussians for p = 1 (left) and p = 10 (right) centered at the origin in R .

A n o t h e r , closely r e l a t e d t h e o r e m b y B a x t e r is

2 2
T h e o r e m 1 0 . 4 . Suppose p( ) and ip( ) are functions that are conditionally pos-
S 2
itive definite of order one and radial on ~R with ip(0) = 0. Then ip o ip( ) is also
S 2
conditionally positive definite of order one and radial on 1R . Indeed, if ip(- ) is
strictly conditionally positive definite of order one and radial and tp vanishes only
2
at zero, then ip o ip( ) is strictly conditionally positive definite of order one and
radial.

T h i s t h e o r e m is a g e n e r a l i z a t i o n o f a classical r e s u l t i n l i n e a r a l g e b r a b y Schur
(see, e.g., [ H o r n a n d J o h n s o n ( 1 9 9 1 ) ; M i c c h e l l i ( 1 9 8 6 ) ] , w h e r e Schur's t h e o r e m was
e x t e n d e d t o cover s t r i c t n e s s ) .
 10. Miscellaneous Theory: Other Norms and Scattered Data Fitting on Manifolds 83

10.2 Scattered D a t a Fitting on Manifolds

T h e r e exists a sizeable b o d y o f l i t e r a t u r e o n t h e t o p i c o f s c a t t e r e d d a t a i n t e r p o l a t i o n
- 1 s
on m a n i f o l d s , especially t h e sphere S ^ i n M . W e w i l l n o t m e n t i o n a n y specific
results here. I n s t e a d w e refer t h e reader t o t h e b o o k [Freeden et al. (1998)], t h e
survey papers [Cheney (1995a); Fasshauer a n d S c h u m a k e r (1998)], as w e l l as m a n y
o r i g i n a l papers such as [ B a x t e r a n d H u b b e r t (2001); B i n g h a m (1973); Fasshauer
(1995a); Fasshauer (1999b); H u b b e r t a n d M o r t o n (2004a); H u b b e r t a n d M o r t o n
(2004b); Levesley et al. (1999); M e n e g a t t o ( 1 9 9 4 b ) ; N a r c o w i c h a n d W a r d ( 2 0 0 2 ) ;
R a g o z i n a n d Levesley (1996); R o n a n d S u n ( 1 9 9 6 ) ; Schoenberg (1942); Schreiner
(1997); W a h b a (1981); W a h b a (1982); X u a n d C h e n e y ( 1 9 9 2 b ) ] .
R a d i a l basis functions o n m o r e general R i e m a n n i a n m a n i f o l d s are s t u d i e d i n ,
e.g., [ D y n et al. (1997); D y n et al. (1999); Levesley a n d R a g o z i n (2002); N a r c o w i c h
(1995); N a r c o w i c h et al. (2003); S c h i m m i n g a n d Belger (1991)].
T h e r e is also a " p o o r m a n ' s s o l u t i o n " t o i n t e r p o l a t i o n o n m a n i f o l d s , especially
the sphere. O n e can use t h e E u c l i d e a n r a d i a l basis f u n c t i o n m e t h o d s discussed t h u s
far, a n d s i m p l y r e s t r i c t t h e i r e v a l u a t i o n t o t h e m a n i f o l d . T h i s a p p r o a c h is o u t l i n e d
i n Section 6 o f [Fasshauer a n d S c h u m a k e r (1998)].
W e w i l l discuss a n o t h e r , r e l a t e d , i n t e r p o l a t i o n p r o b l e m l a t e r . N a m e l y , i n t e r p o -
3
l a t i o n t o p o i n t c l o u d d a t a i n R . I n t h i s case, t h e u n d e r l y i n g m a n i f o l d is u n k n o w n ,
a n d a n o t h e r a p p r o a c h needs t o be t a k e n . See C h a p t e r 30 for details.

10.3 Remarks

M a n y o f t h e results g i v e n i n t h e p r e v i o u s c h a p t e r s c a n be generalized t o v e c t o r -
v a l u e d or even m a t r i x - v a l u e d f u n c t i o n s . Some results a l o n g these lines c a n be f o u n d
i n [ L o w i t z s c h (2002); L o w i t z s c h (2005); M y e r s (1992); N a r c o w i c h a n d W a r d (1994a);
Schaback (1995a)].
We point out that the approach to solving the interpolation problems taken i n
t h e p r e v i o u s chapters a l w a y s assumes t h a t t h e centers, i.e., the points x , k k =
1,...,N, at w h i c h t h e basis f u n c t i o n s are centered, coincide w i t h t h e d a t a sites.
T h i s is a f a i r l y severe r e s t r i c t i o n , a n d i t has been s h o w n i n examples i n t h e c o n t e x t
o f least squares a p p r o x i m a t i o n o f s c a t t e r e d d a t a (see, e.g., [Franke et al. (1994);
F r a n k e et al. (1995)] or [Fasshauer (1995a)]) t h a t b e t t e r r e s u l t s can be achieved
i f t h e centers are chosen different f r o m t h e d a t a sites. Theoretical results i n this
d i r e c t i o n are v e r y l i m i t e d , a n d are r e p o r t e d i n [ Q u a k et al. (1993)] a n d i n [Sun
(1993a)].
84 Meshfree Approximation Methods with M A T L A B

z
0.5

zo.5

2
0.5-

2
Fig. 10.2 p-norm distance matrix fits to f(x, y) = (x + y)/2 on a 5 X 5 grid in [0, l ] unless noted
otherwise. Top: p = 1 (1089 Halton points). 2nd row: p = 10 (left), p = 100 (right). 3rd row:
p = 1.001 (left), p = 2 (right). Bottom: p-norm Gaussian fits for p = 1 (left) and p = 10 (right).
 Chapter 11

Compactly Supported
Radial Basis Functions

A s we saw earlier (see T h e o r e m 9.4), c o m p a c t l y s u p p o r t e d f u n c t i o n s <& t h a t are

t r u l y s t r i c t l y c o n d i t i o n a l l y p o s i t i v e d e f i n i t e o f order m > 0 d o n o t exist. The
c o m p a c t s u p p o r t a u t o m a t i c a l l y ensures t h a t <E> is s t r i c t l y p o s i t i v e d e f i n i t e . A n o t h e r
o b s e r v a t i o n (see T h e o r e m 3.9) was t h a t c o m p a c t l y s u p p o r t e d r a d i a l f u n c t i o n s c a n
s
be s t r i c t l y p o s i t i v e definite o n K o n l y for a fixed m a x i m a l s-value. I t is n o t possible
s
for a f u n c t i o n t o be s t r i c t l y p o s i t i v e d e f i n i t e a n d r a d i a l o n R for a l l s a n d also have
a compact support. T h e r e f o r e we focus o u r a t t e n t i o n o n t h e c h a r a c t e r i z a t i o n and
c o n s t r u c t i o n o f f u n c t i o n s t h a t are c o m p a c t l y s u p p o r t e d , s t r i c t l y p o s i t i v e d e f i n i t e
s
a n d r a d i a l o n M for some fixed s.
A c c o r d i n g t o o u r earlier w o r k ( B o c h n e r ' s t h e o r e m a n d g e n e r a l i z a t i o n s t h e r e o f ) , a
s
f u n c t i o n is s t r i c t l y p o s i t i v e d e f i n i t e a n d r a d i a l o n R i f i t s s-variate F o u r i e r t r a n s f o r m
is non-negative. T h e o r e m B . l i n t h e A p p e n d i x gives t h e F o u r i e r t r a n s f o r m o f t h e
r a d i a l f u n c t i o n <fr = ip(\\ ||) as a n o t h e r r a d i a l f u n c t i o n

/>oo
2
*(x) = F,<p{\\x\\) = Ija-H-C/ <p(t)r' J - (t\\x\\)dt,
(a 2)/2

Jo

where J v is t h e Bessel f u n c t i o n o f t h e first k i n d o f order v.

11.1 O p e r a t o r s for R a d i a l F u n c t i o n s a n d D i m e n s i o n W a l k s

A certain integral operator a n d i t s inverse d i f f e r e n t i a l o p e r a t o r were defined i n

[Schaback a n d W u (1996)]. I n t h a t p a p e r a n e n t i r e calculus was developed for h o w
these o p e r a t o r s act o n r a d i a l f u n c t i o n s . I n fact, a c c o r d i n g t o [ G n e i t i n g ( 2 0 0 2 ) ] ,
these o p e r a t o r s c a n be t r a c e d b a c k t o JjMatheron (1965)] w h o called t h e i n t e g r a l
o p e r a t o r montee a n d t h e d i f f e r e n t i a l o p e r a t o r descente motivated by an application
related to m i n i n g .
I n t h e f o l l o w i n g we define these o p e r a t o r s a n d show h o w t h e y f a c i l i t a t e t h e
construction of compactly supported r a d i a l functions.

85
86 Meshfree Approximation Methods with M A T L A B

Definition 11.1.

(1) L e t (fi be such t h a t t i t<p(t) G L i [ 0 , o o ) . T h e n we define t h e integral operator

X via

(T<p)(r) = / f<p(t)dt, r > 0.

J r

2
(2) For even <p G C(M) we define t h e differential operator V via

(XV)0\) = (p'(r), r > 0.

I n b o t h cases t h e r e s u l t i n g f u n c t i o n s are t o be i n t e r p r e t e d as even functions

u s i n g even extensions.

N o t e t h a t t h e i n t e g r a l o p e r a t o r X differs f r o m t h e o p e r a t o r / i n t r o d u c e d earlier
(see (5.1)) b y a f a c t o r t i n t h e i n t e g r a n d .
T h e m o s t i m p o r t a n t p r o p e r t i e s o f t h e m o n t e e a n d descente o p e r a t o r s are (see,
e.g., [Schaback a n d W u (1996)] o r [ W e n d l a n d ( 1 9 9 5 ) ] ) :

Theorem 11.1.

(1) Both T> andX preserve compact support, i.e., if if has compact support, then so
do T>p and Xp.
(2) IfpE C(R) and t ^ t(f>(t) G L i [ 0 , o o ) , then VX<p = (p.
2
(3) Ifpe C (R) (<p^l) is even and <p' G L [0, o o ) , then XVp x = ip.
s x
(4) J / t H t ~ p{t) G L i [ 0 , o o ) and s > 3, then F (<p) s = JF _ (X^).
S 2

2 s
(5) Ifpe C (R) is even and 11-+ t <p'(t) G L i [ 0 , o o ) , then F (ip) 8 = T i{pp).
s+

T h e o p e r a t o r s X a n d V a l l o w us t o express s - v a r i a t e F o u r i e r t r a n s f o r m s as (s2)-
o r ( s + 2 ) - v a r i a t e F o u r i e r t r a n s f o r m s , respectively. I n p a r t i c u l a r , a d i r e c t consequence
o f t h e above p r o p e r t i e s a n d t h e c h a r a c t e r i z a t i o n o f s t r i c t l y p o s i t i v e d e f i n i t e r a d i a l
f u n c t i o n s ( T h e o r e m 3.6) is

Theorem 11.2.

s l
(1) Suppose cp G C(R). Ift ^ t ~ p(t) G Z a [ 0 , o o ) and s > 3, then <p is strictly
s
positive definite and radial on R if and only if X<p is strictly positive definite
s 2
and radial on R~.
2 s
(2) If p e C (R) is even and t H- t <p'(t) L i [ 0 , o o ) , then p is strictly positive
s
definite and radial on R if and only if T>p is strictly positive definite and
s + 2
radial on R .

T h i s a l l o w s us t o c o n s t r u c t n e w s t r i c t l y p o s i t i v e d e f i n i t e r a d i a l f u n c t i o n s from
g i v e n ones b y a " d i m e n s i o n - w a l k " t e c h n i q u e t h a t steps t h r o u g h m u l t i v a r i a t e E u -
c l i d e a n space i n even i n c r e m e n t s . T h e e x a m p l e s p r e s e n t e d i n t h e f o l l o w i n g sections
illustrate this technique.

.1,
 11. Compactly Supported Radial Basis Functions 87

11.2 Wendland's Compactly Supported Functions

p r o b a b l y the most popular family of compactly supported r a d i a l functions presently

i n use was c o n s t r u c t e d i n [ W e n d l a n d (1995)]. W e n d l a n d s t a r t s w i t h t h e t r u n c a t e d
e
power f u n c t i o n <pe(r) = ( 1 r) + ( w h i c h we k n o w t o be s t r i c t l y p o s i t i v e d e f i n i t e a n d
s
r a d i a l o n M for > |_J + 1 a c c o r d i n g t o S e c t i o n 4 . 6 ) , a n d t h e n he w a l k s t h r o u g h
dimensions b y r e p e a t e d l y a p p l y i n g t h e o p e r a t o r X.

e
Definition 11.2. W i t h <pe(r) = ( 1 r) + we define

k(
Vs,k = 1 P\_s/2\+k+l-

I t t u r n s o u t t h a t t h e f u n c t i o n s (p k St are a l l s u p p o r t e d o n [0,1] a n d have a p o l y -

n o m i a l r e p r e s e n t a t i o n t h e r e . M o r e precisely,

s
T h e o r e m 1 1 . 3 . The functions (p kSj are strictly positive definite and radial on R
and are of the form

s k K
^ ' ' \0, r > 1,

2 f e
with a univariate polynomial pk 3t of degree [s/2\ + Sk + 1. Moreover, (p k
Sj C (IR)
are unique up to a constant factor, and the polynomial degree is minimal for given
space dimension s and smoothness 2k.

2k
T h i s t h e o r e m states t h a t a n y o t h e r c o m p a c t l y s u p p o r t e d C p o l y n o m i a l func-
t i o n t h a t is s t r i c t l y p o s i t i v e d e f i n i t e a n d r a d i a l o n W w i l l n o t have a smaller p o l y -
n o m i a l degree. O u r o t h e r e x a m p l e s b e l o w ( W u ' s f u n c t i o n s , G n e i t i n g ' s f u n c t i o n s )
i l l u s t r a t e t h i s fact. T h e s t r i c t p o s i t i v e definiteness of Wendland's functions ip k
Sj

s t a r t i n g w i t h non-integer values o f i n D e f i n i t i o n 11.2 was established i n [ G n e i t i n g

(1999)]. N o t e , however, t h a t t h e n t h e f u n c t i o n s are no longer g u a r a n t e e d t o be
polynomials on their support.
W e n d l a n d gave recursive f o r m u l a s for t h e f u n c t i o n s <p ,k for a l l s, k. W e i n s t e a d s

list the e x p l i c i t f o r m u l a s o f [Fasshauer (1999a)].

T h e o r e m 1 1 . 4 . The functions (p ,k,

s k = 0 , 1 , 2 , 3 , have the form

+ 1
>,,i(r) = ( l - r [(* + l ) r + l ] ,
e 2 2 2
Vs,2{r) = ( 1 - r) + [( + 4 + 3 ) r + (3 + 6 ) r + 3] ,
3 3 2 3 2 2
ip (r)
at3 = (1 - rY+ [{ + 9 + 23 + 1 5 ) r + {U + 3Q + 4 5 ) r
+ (15^ + 4 5 ) r + 1 5 ] ,

where = [s/2\ + k + 1, and the symbol = denotes equality up to a multiplicative

positive constant.
88 Meshfree Approximation Methods with M A T L A B

Proof. T h e case k = 0 follows d i r e c t l y f r o m t h e d e f i n i t i o n . Application of the

d e f i n i t i o n for t h e case k = 1 y i e l d s
oo
t<p (t)dt
/
e

oo
- l
t(l -
t) dt +

e
= f t(l-t) dt
J r
+ 1
= ( , + 1 )(, + 2 ) ( i - r ) ' [ ( ^ i ) r + i ] ,

w h e r e t h e c o m p a c t s u p p o r t o f <f reduces t h e i m p r o p e r i n t e g r a l t o a d e f i n i t e i n t e g r a l
w h i c h c a n be e v a l u a t e d u s i n g i n t e g r a t i o n b y p a r t s . T h e o t h e r t w o cases are o b t a i n e d
similarly b y repeated application o f X.

Example 1 1 . 1 . F o r s = 3 we l i s t some o f t h e m o s t c o m m o n l y used f u n c t i o n s i n

s
T a b l e 1 1 . 1 . T h e s e f u n c t i o n s are s t r i c t l y p o s i t i v e d e f i n i t e a n d r a d i a l o n M. for s < 3.
W e also l i s t t h e i r degree o f s m o o t h n e s s 2k. T h e f u n c t i o n s were d e t e r m i n e d using
t h e f o r m u l a s f r o m T h e o r e m 11.4, i.e., f o r k = 1, 2 , 3 t h e y m a t c h D e f i n i t i o n 11.2 o n l y
u p t o a positive constant factor.
For t h e M A T L A B i m p l e m e n t a t i o n i n t h e n e x t c h a p t e r i t is b e t t e r t o express
t h e c o m p a c t l y s u p p o r t e d f u n c t i o n s i n a s h i f t e d f o r m since w e w i l l be u s i n g a m a t r i x
v e r s i o n o f 1er i n place o f t h e code used earlier i n D i s t a n c e M a t r i x . m for r . T h u s w e
also list t h e a p p r o p r i a t e f u n c t i o n s <p ^ = <> ,/c(l ) so t h a t > fc(l er) =
s s S) p> ,k{^f).
s

For c l a r i f i c a t i o n purposes w e r e i t e r a t e t h a t expressions o f t h e f o r m (x)+ are t o be

e
i n t e r p r e t e d as ((x)+) , i.e., w e first a p p l y t h e c u t o f f f u n c t i o n , a n d t h e n t h e p o w e r .

a n
Table 11.1 Wendland's compactly supported radial functions v'[Link] d *?s,k Vs,fc(l " )
for various choices offcand s 3.

k <P3,k( ) r
&3,k(r) smoothness
r C
0 i
2
1 (1 - r)\ (4r + 1) r\ (5 - 4 r ) C
2 2 4
2 (1 - r)\ ( 3 5 r + 18r + 3) r\ (56 - 88r + 3 5 r ) C
3 2 2 3 6
3 (1 - r)\ ( 3 2 r + 2 5 r + 8r + l ) r \ (66 - 154r + 1 2 1 r - 3 2 r ) C

11.3 Wu's Compactly Supported Functions

I n [ W u (1995b)] w e f i n d a n o t h e r w a y t o c o n s t r u c t s t r i c t l y p o s i t i v e d e f i n i t e r a d i a l
functions w i t h compact support. W u starts w i t h t h e f u n c t i o n
2 e
i>(r) = (1 - r ) , + eN,
 11. Compactly Supported Radial Basis Functions 89

w h i c h i n itself is n o t p o s i t i v e d e f i n i t e (see t h e discussion at t h e e n d o f C h a p t e r 5 ) .

However, W u t h e n uses c o n v o l u t i o n t o c o n s t r u c t a n o t h e r f u n c t i o n t h a t is s t r i c t l y
p o s i t i v e definite a n d r a d i a l o n R , i.e.,

ipi(r) = (ip * ip)(2r)

2 e 2
( 1 - t ) (l + - (2r - t) Y dt
+

-oo

2
= f ( l - t Y { l - { 2 r - t Y Y + d t .

T h i s f u n c t i o n is s t r i c t l y p o s i t i v e d e f i n i t e since i t s F o u r i e r t r a n s f o r m is essentially
the square o f t h e F o u r i e r t r a n s f o r m o f ip a n d therefore n o n - n e g a t i v e . J u s t like t h e
W e n d l a n d functions, t h i s f u n c t i o n is a p o l y n o m i a l o n i t s s u p p o r t . I n fact, t h e degree
2e
of t h e p o l y n o m i a l is 4 + 1, a n d ip e e C (R).
N o w , a f a m i l y o f s t r i c t l y p o s i t i v e d e f i n i t e r a d i a l f u n c t i o n s is c o n s t r u c t e d b y a
d i m e n s i o n w a l k u s i n g t h e T> o p e r a t o r .

2 2
D e f i n i t i o n 11.3. W i t h ip {r)e = ( ( 1 - - Y+ * ( 1 - - ) + ) ( 2 r ) we define

k
iP t K = V ip . t

s
T h e f u n c t i o n s ipk,e are s t r i c t l y p o s i t i v e d e f i n i t e a n d r a d i a l o n R for s < 2k + 1,
2 k
are p o l y n o m i a l s o f degree A 2k+l o n t h e i r s u p p o r t a n d i n C ^~ ^ i n the interior
2 k
of t h e s u p p o r t . O n t h e b o u n d a r y t h e smoothness increases t o C ~.

E x a m p l e 1 1 . 2 . For = 3 we c a n c o m p u t e t h e four f u n c t i o n s

k k 2 2
W ) = V ip {r) 3 = V ((l - -f + * (1 - - ) ) ( 2 r ) , + k = 0,1,2,3.

T h e y are l i s t e d i n T a b l e 11.2 a l o n g w i t h t h e i r smoothness. T h e m a x i m a l space

d i m e n s i o n s for w h i c h these f u n c t i o n s are s t r i c t l y p o s i t i v e definite a n d r a d i a l o n
s
R is also l i s t e d . J u s t as w i t h the W e n d l a n d f u n c t i o n s , t h e f u n c t i o n s i n T a b l e 11.2
m a t c h t h e d e f i n i t i o n o n l y u p t o a p o s i t i v e m u l t i p l i c a t i v e c o n s t a n t . A g a i n , we also list
the functions ip ,i k = ipk,(l ) used i n o u r M A T L A B i m p l e m e n t a t i o n i n C h a p t e r 12.
T h i s r e p r e s e n t a t i o n o f t h e W u f u n c t i o n s is g i v e n i n T a b l e 11.3.

Table 11.2 Wu's compactly supported radial functions ipk,e for various choices of
fc and I = 3.

fc smoothness s

0 2
( l - r ) + ( 5 - r 35r + 101r + 147r + 101r + 3 5 r + 5 r )
3 4 5 6
1
1 2
( 1 - r ) ( 6 + 36r + 8 2 r + 7 2 r + 3 0 r + 5 r )
+
3 4 5
c 4
3
2
2 (1 - r ) + ( 8 + 40r + 4 8 r + 2 5 r + 5 r )
2 3 4
c 5
3 4
(1 - r ) (16 + 29r + 2 0 r + 5 r ) 2 3
c 7
90 Meshfree Approximation Methods with M A T L A B

Table 11.3 Shifted version ipk,e of Wu's compactly supported radial functions tpk,e
for various choices of k and = 3.
r
k ^k,zi ) smoothness s
2 3 4 5 6
0 r ( 4 2 9 - 1287r + 1573r - l O O l r + 3 5 1 r - 6 5 r + 5 r )
+ C 6
1
2 3 4 5
1 r ( 2 3 1 - 561r + 528r - 242r + 5 5 r - 5 r )
+ C 4
3
2 3 4
2 r ( 1 2 6 - 231r + 153r - 4 5 r + 5 r )
+ C 2
5
4 2 3
3 r (70 - 84r + 3 5 r - 5 r ) C 7

Fig. 11.1 Plot of Wendland's functions from Example 11.1 (left) and Wu's functions from E x a m -
ple 11.2 (right).

As predicted by Theorem 11.3, for a p r e s c r i b e d smoothness the polynomial

degree o f W e n d l a n d ' s f u n c t i o n s is l o w e r t h a n t h a t o f W u ' s f u n c t i o n s . F o r e x a m p l e ,
4
b o t h W e n d l a n d ' s f u n c t i o n (p 3j2 a n d W u ' s f u n c t i o n ipi j3 are C smooth and strictly
3
p o s i t i v e definite a n d r a d i a l o n I R . H o w e v e r , t h e p o l y n o m i a l degree o f W e n d l a n d ' s
f u n c t i o n is 8, whereas t h a t o f W u ' s f u n c t i o n is 1 1 . A n o t h e r c o m p a r a b l e f u n c t i o n
4
is G n e i t i n g ' s o s c i l l a t o r y f u n c t i o n a 2 (see T a b l e 11.5), w h i c h is a C polynomial of
3
degree 9 t h a t is s t r i c t l y p o s i t i v e d e f i n i t e a n d r a d i a l o n R .
W h i l e t h e t w o families o f s t r i c t l y p o s i t i v e d e f i n i t e c o m p a c t l y s u p p o r t e d functions
discussed above are b o t h c o n s t r u c t e d v i a d i m e n s i o n w a l k , W e n d l a n d uses i n t e g r a t i o n
( a n d t h u s o b t a i n s a f a m i l y o f i n c r e a s i n g l y s m o o t h e r f u n c t i o n s ) , whereas W u needs
t o s t a r t w i t h a f u n c t i o n o f sufficient s m o o t h n e s s , a n d t h e n o b t a i n s successively less
s m o o t h functions (via differentiation).

11.4 Oscillatory Compactly Supported Functions

O t h e r s t r i c t l y p o s i t i v e d e f i n i t e c o m p a c t l y s u p p o r t e d r a d i a l f u n c t i o n s have b e e n p r o -
posed b y G n e i t i n g (see, e.g., [ G n e i t i n g ( 2 0 0 2 ) ] ) . H e s h o w e d t h a t a f a m i l y o f o s c i l l a -
t o r y c o m p a c t l y s u p p o r t e d f u n c t i o n s c a n be c o n s t r u c t e d u s i n g t h e so-called turning
 11. Compactly Supported Radial Basis Functions 91

bands operator o f [ M a t h e r o n ( 1 9 7 3 ) ] . S t a r t i n g w i t h a f u n c t i o n ip s t h a t is s t r i c t l y
s
p o s i t i v e definite a n d r a d i a l o n M. for s > 3 t h e t u r n i n g b a n d s o p e r a t o r p r o d u c e s

^ - ( r ) = ^ (r) + ^i^
2 s (11.1)

s _ 2
w h i c h is s t r i c t l y p o s i t i v e d e f i n i t e a n d r a d i a l o n R .

Example 1 1 . 3 . O n e such f a m i l y o f f u n c t i o n s is g e n e r a t e d is we s t a r t w i t h the

W e n d l a n d functions (p +2,i(f) s = ( 1 r ) ^ " [( + l ) r + 1] ( n o n - i n t e g e r
1
allowed).
A p p l i c a t i o n of the t u r n i n g bands operator results i n the functions

( \ fi V (-\ ^ o ( l + l ) ( l + 2 + s) 2 \

s
w h i c h are s t r i c t l y p o s i t i v e d e f i n i t e a n d r a d i a l o n M p r o v i d e d > (see [ G n e i t i n g
(2002)]). Some specific f u n c t i o n s f r o m t h i s f a m i l y are l i s t e d i n T a b l e 11.4. A l l o f
2
t h e f u n c t i o n s are i n C ( 1 R ) . I f we w a n t s m o o t h e r f u n c t i o n s , t h e n we need t o s t a r t
w i t h a s m o o t h e r W e n d l a n d f a m i l y as d e s c r i b e d b e l o w i n E x a m p l e 11.4.

Table 11.4 Gneiting's compactly supported radial

functions r for various choices of and s = 2.
s

T r
^ 2,e( ) smoothness

7 i | 5
7/2 (1 - r) ^ (1 + \r - r 2 )
C 2

2 2
5 (1 - r)\ (1 + 5r - 2 7 r ) C
15/2 ( l - r ^ l + f r - _3|i r 2 ) C 2

12 (i - 0 + ( i + 1 2 r
- 104r )
2
C 2

T h e f u n c t i o n s o f T a b l e 11.4 are s h o w n i n t h e left p l o t o f F i g u r e 11.2 w i t h

increasing f r o m t h e o u t s i d e i n (as v i e w e d near t h e o r i g i n ) .

Fig. 11.2 Oscillatory functions of Table 11.4 (left) and Table 11.5 (right).
92 Meshfree Approximation Methods urith M A T L A B

E x a m p l e 1 1 . 4 . A l t e r n a t i v e l y , we c a n o b t a i n a set o f o s c i l l a t o r y f u n c t i o n s t h a t are
3
s t r i c t l y positive definite and r a d i a l o n M b y applying the t u r n i n g bands operator
t o t h e W e n d l a n d f u n c t i o n s </?5,fc t h a t are s t r i c t l y p o s i t i v e d e f i n i t e a n d r a d i a l o n R 5

for different choices o f A:. T h e n t h e r e s u l t i n g f u n c t i o n s a k w i l l have t h e same degree

o f s m o o t h n e s s 2k as t h e o r i g i n a l f u n c t i o n s a n d t h e y w i l l be s t r i c t l y p o s i t i v e d e f i n i t e
3
a n d r a d i a l o n R . T h e r e s u l t s for k = 1,2,3 are l i s t e d i n T a b l e 11.5 a n d d i s p l a y e d
i n t h e r i g h t p l o t o f F i g u r e 11.2.

Table 11.5 Oscillatory compactly supported functions that are

3
strictly positive definite and radial on R parametrized by smooth-
ness.

k Ok(r) smoothness

1 (1 - r)\ (1 + 4 r - 1 5 r ) 2
C 2

2 3 4
2 (1 - r)\ (3 + 18r + 3 r - 1 9 2 r ) C
2 3 4 6
3 (1 - r ) \ (15 + 120r + 2 1 0 r - 8 4 0 r - 3 4 6 5 r ) C

G n e i t i n g also suggests t h e c o n s t r u c t i o n o f s t r i c t l y p o s i t i v e d e f i n i t e r a d i a l func-

t i o n s b y t a k i n g t h e p r o d u c t o f t h e ( a p p r o p r i a t e l y scaled) Poisson f u n c t i o n s Q, s (see
e i t h e r T h e o r e m 3.6 or S e c t i o n 4.3) w i t h a c e r t a i n c o m p a c t l y s u p p o r t e d non-negative
f u n c t i o n (see [ G n e i t i n g (2002)] for m o r e d e t a i l s ) . B y P r o p e r t y (6) o f T h e o r e m 3.1
t h e r e s u l t i n g f u n c t i o n w i l l be s t r i c t l y p o s i t i v e d e f i n i t e .

11.5 Other Compactly Supported Radial Basis Functions

T h e r e are m a n y o t h e r w a y s i n w h i c h one c a n c o n s t r u c t c o m p a c t l y s u p p o r t e d func-

s
t i o n s t h a t are s t r i c t l y p o s i t i v e d e f i n i t e a n d r a d i a l o n M . I n [Schaback (1995a)]
several such p o s s i b i l i t i e s are d e s c r i b e d .

E x a m p l e 1 1 . 5 . Euclid's hat f u n c t i o n s are c o n s t r u c t e d i n a n a l o g y t o S - s p l i n e s . I t is

w e l l k n o w n t h a t t h e u n i v a r i a t e f u n c t i o n (3(r) = ( 1 | r | ) + is a second-order jE?-spline
w i t h k n o t s a t 1 , 0 , 1 , a n d i t is o b t a i n e d as t h e c o n v o l u t i o n o f t h e characteristic
f u n c t i o n o f t h e i n t e r v a l [ 1 / 2 , 1 / 2 ] w i t h itself. E u c l i d ' s h a t f u n c t i o n s are n o w o b -
tained by convolving the characteristic function of the s-dimensional Euclidean u n i t
b a l l w i t h itself. T h e r e s u l t i n g f u n c t i o n s c a n be w r i t t e n for r G [ 0 , 1 ] i n t h e f o r m

f 27ry 2 f c _ (2r)-r(l-r )
1
2 f c
r. _ -i o q
^ f c + 1 (2r) = K - 1 , 2 , 6 , . . . ,

12(1-r) fc = 0,
for o d d space d i m e n s i o n s s = 2k + 1, a n d as

[ 2 ry 7 2 f c (2r)-TV(l-r2)(l-r ) 2 f c
, _
>2fc+ (2r) = I
2 ^+2 * - 1, A 6,
2 ( a r c c o s r ry/l r) 2
k = 0,
 11. Compactly Supported Radial Basis Functions 93

for even space d i m e n s i o n s s = 2k. N o t e t h a t these f u n c t i o n s are zero o u t s i d e the

interval [0,2].
W e have l i s t e d several o f these f u n c t i o n s i n T a b l e 11.6 w h e r e w e have e m p l o y e d
a s u b s t i t u t i o n 2r > r a n d a n o r m a l i z a t i o n f a c t o r such t h a t t h e f u n c t i o n s a l l have
a v a l u e o f one a t t h e o r i g i n . T h e f u n c t i o n s are also d i s p l a y e d i n t h e left p l o t o f
F i g u r e 11.3.

Table 11.6 Euclid's hat functions (defined for 0 < r < 2) for
different values of s.

s V>s(r) smoothness

1 1_ r C
1
2
2
2 ^ ^4arccos (^) r \ / 4 r ^ C
3 1
- sh (( + * ) ~ ) 4 1 6 r r3 c
4 I arccos ( r ) _ _1_^4 - r (20r + r ) 2 3
c
5 1
- 64^ ( t 1 2
+ 8 7 r
+ 3 2 7 r 2
) r 3
~ ( + 2 ? r
) r 3
) c

Fig. 11.3 Euclid's hat functions (left) of Table 11.6 and Buhmann's function of Example 11.6
(right).

A n o t h e r c o n s t r u c t i o n d e s c r i b e d i n [Schaback (1995a)] is t h e r a d i a l i z a t i o n o f t h e
s-fold tensor p r o d u c t o f u n i v a r i a t e 5 - s p l i n e s o f even o r d e r 2m w i t h u n i f o r m k n o t s .
T h e s e f u n c t i o n s d o n o t seem t o have a s i m p l e r e p r e s e n t a t i o n t h a t lends i t s e l f t o
numerical computations. A s c a n be seen f r o m i t s r a d i a l i z e d F o u r i e r t r a n s f o r m , t h e
5
r a d i a l i z e d S - s p l i n e i t s e l f is n o t s t r i c t l y p o s i t i v e d e f i n i t e a n d r a d i a l o n a n y R with
s > 1. For s = 1 o n l y t h e S - s p l i n e s o f even o r d e r are s t r i c t l y p o s i t i v e d e f i n i t e (see,
e.g., [ S c h o l k o p f a n d S m o l a ( 2 0 0 2 ) ] ) .
T h e last f a m i l y o f c o m p a c t l y s u p p o r t e d s t r i c t l y p o s i t i v e d e f i n i t e r a d i a l f u n c t i o n s
we w o u l d like t o m e n t i o n is due t o [ B u h m a n n ( 1 9 9 8 ) ] . B u h m a n n ' s f u n c t i o n s c o n t a i n
94 Meshfree Approximation Methods with M A T L A B

a l o g a r i t h m i c t e r m i n a d d i t i o n t o a p o l y n o m i a l . H i s f u n c t i o n s have t h e g e n e r a l f o r m
/OO

<p(r)= / (l-r /t)+* (l 2 a

Jo
H e r e 0 < 5 < ^, p > 1, a n d i n o r d e r t o o b t a i n f u n c t i o n s t h a t are s t r i c t l y p o s i t i v e
s
d e f i n i t e a n d r a d i a l o n M for s < 3 t h e c o n s t r a i n t s for t h e r e m a i n i n g p a r a m e t e r s are
A > 0, a n d - 1 < a < =.

Example 11.6. A n example w i t h a = < 5 = | , p = l and A = 2 is l i s t e d i n

[ B u h m a n n (2000)]:
4 4 3 2
(p(r) = 12r logr - 21r + 32r - 12r + 1, 0 < r < 1.
2 S
T h i s f u n c t i o n is i n C ( R ) a n d s t r i c t l y p o s i t i v e d e f i n i t e a n d r a d i a l o n IR for s < 3.
I t is d i s p l a y e d i n t h e r i g h t p l o t o f F i g u r e 11.3.

W h i l e i t is s t a t e d i n [ B u h m a n n (2000)] t h a t t h e c o n s t r u c t i o n t h e r e encompasses
b o t h W e n d l a n d ' s a n d W u ' s f u n c t i o n s , a n even m o r e g e n e r a l t h e o r e m t h a t shows t h a t
integration of a positive function / L i [ 0 , o o ) against a s t r i c t l y positive definite
k e r n e l K r e s u l t s i n a s t r i c t l y p o s i t i v e d e f i n i t e f u n c t i o n c a n be f o u n d i n [ W e n d l a n d
(2005a)] (see also S e c t i o n 4 . 8 ) . M o r e specifically,
roc
p(r) = / K(t,r)f(t)dt
Jo
is s t r i c t l y p o s i t i v e d e f i n i t e . B u h m a n n ' s c o n s t r u c t i o n t h e n c o r r e s p o n d s t o c h o o s i n g
a s 2
f(t) = t ( l - t )^ a n d K(t, r) = ( 1 - r /t)\.
 Chapter 12

Interpolation with Compactly Supported

R B F s in M A T L A B

We n o w have a n a l t e r n a t i v e w a y t o c o n s t r u c t a n R B F i n t e r p o l a n t t o scattered
s
d a t a i n M. . I f we use t h e c o m p a c t l y s u p p o r t e d r a d i a l f u n c t i o n s o f t h e p r e v i o u s
chapter t h e n t h e m a i n difference t o o u r p r e v i o u s i n t e r p o l a n t s is t h a t n o w t h e i n -
t e r p o l a t i o n m a t r i x c a n be m a d e sparse b y s c a l i n g t h e s u p p o r t o f t h e basic f u n c t i o n
a p p r o p r i a t e l y . T o achieve t h i s w e use as w e d i d earlier t h e basic f u n c t i o n s
(p (r)
e = <p{er). T h u s , a large v a l u e o f e corresponds t o a s m a l l s u p p o r t . I n o t h e r
w o r d s , i f t h e s u p p o r t o f <p is t h e i n t e r v a l [ 0 , 1 ] , t h e n t h e s u p p o r t r a d i u s p o f <p is

g i v e n b y p = 1/e so t h a t ip (r) e = 0 for r > p = 1/e.

Since we k n o w t h a t t h e i n t e r p o l a t i o n m a t r i x w i l l be a sparse m a t r i x , we w a n t
t o w r i t e M A T L A B code t o efficiently assemble t h e m a t r i x . O n c e we have defined a
sparse m a t r i x , M A T L A B w i l l a u t o m a t i c a l l y use s t a t e - o f - t h e - a r t sparse m a t r i x t e c h -
niques t o solve t h e l i n e a r s y s t e m . O b v i o u s l y , we d o n o t w a n t t o c o m p u t e t h e m a t r i x
entries for a l l p a i r s o f p o i n t s since we k n o w a l l o f t h e entries for far away p o i n t s
w i l l be zero. T h e r e f o r e , a n efficient d a t a s t r u c t u r e is needed. W e use A;c?-trees
( i m p l e m e n t e d i n a set o f M A T L A B M E X - f i l e s w r i t t e n b y G u y Shechter t h a t c a n be
d o w n l o a d e d f r o m t h e M A T L A B C e n t r a l F i l e E x c h a n g e , see [ M C F E ] ) . Some i n f o r -
m a t i o n o n kd-tvees is p r o v i d e d i n A p p e n d i x A . D a t a s t r u c t u r e s for t h e use w i t h
meshfree a p p r o x i m a t i o n m e t h o d s are also discussed i n [ W e n d l a n d (2005a)].

12.1 Assembly o f the Sparse I n t e r p o l a t i o n M a t r i x

We have s t r u c t u r e d t h e s c a t t e r e d d a t a i n t e r p o l a t i o n p r o g r a m i n t h e c o m p a c t l y sup-
p o r t e d case analogous t o t h e code for t h e g l o b a l i n t e r p o l a n t s , i.e., first c o n s t r u c t a
distance m a t r i x , a n d t h e n a p p l y t h e a n o n y m o u s f u n c t i o n r b f t o o b t a i n t h e i n t e r p o -
l a t i o n / e v a l u a t i o n m a t r i x (as o n lines 1 3 - 1 4 a n d 1 5 - 1 6 o f P r o g r a m 2.1). H o w e v e r , i t
t u r n s o u t t h a t i t is easier t o d e a l w i t h t h e c o m p a c t s u p p o r t i f we c o m p u t e t h e "dis-
tance m a t r i x " c o r r e s p o n d i n g t o t h e ( 1 s r ) + t e r m since o t h e r w i s e those entries o f
t h e distance m a t r i x t h a t are zero (since t h e m u t u a l distance between t w o i d e n t i c a l
p o i n t s is zero) w o u l d be "lost" i n t h e sparse r e p r e s e n t a t i o n o f t h e m a t r i x .
T h e M A T L A B code DistanceMatrixCSRBF.m ( P r o g r a m 12.1) c o n t a i n s t w o s i m i -

95
96 Meshfree Approximation Methods with M A T L A B

lar blocks that will be used depending on whether we have more centers than data
sites or vice versa. For example, if there are more data sites than centers (cf. lines 7
16), then we build a kd-txee for the data sites and find for each center x - those 3

data sites within the support of the basis function centered at Xj, i.e., we construct
the (sparse) matrix column by column. In the other case (cf. lines 18-27) we start
with a tree for the centers and build the matrix row by row. This is accomplished by
determining for each data site Xi all centers whose associated^ basis function
covers data site Xi.
The functions kdtree and kdrangequery are provided by the kd-tree library
mentioned above. T h e call in line 7 (respectively 18) of Program 12.1 generates the
kd-txee of all the centers (data sites), and with the call to kdrangequery in line 9
(respectively 20) we find all centers (data sites) that lie within a distance support
of the jth center point (data site). T h e actual distances are returned in the vector
d i s t and the indices into the list of all data sites are provided in idx. T h e distances
for these points only are stored in the matrix DM. For maximum efficiency (in order
to avoid dynamic memory allocation) it is important to have a good estimate of
the number of nonzero entries in the matrix for the allocation statement in lines 4
and 5. The version of the code presented here has the best performance for larger
problems since s p a r s e is only invoked once.

P r o g r a m 12.1. DistanceMatrixCSRBF.m

% DM = D i s t a n c e M a t r i x C S R B F ( d s i t e s , c t r s , e p )
% Forms t h e d i s t a n c e m a t r i x of two s e t s of p o i n t s i n R~s
% f o r compactly supported r a d i a l b a s i s f u n c t i o n s , i . e . ,
7. DM(i,j) = I I d a t a s i t e _ i - c e n t e r _ j I I _2.
7o The CSRBF used with t h i s code must be g i v e n i n s h i f t e d form
7. rbf2(u) = r b f ( r ) , u=l-e*r.
% F o r example, t h e Wendland C2
7o rbf = @(e,r) max(l-e*r , 0 ) . ~ 4 . * ( 4 * e * r + l ) ;
7 becomes
7. r b f 2 = ffl(u) u. "4.*(4*u+5) ;
7o Input
% d s i t e s : Nxs matrix r e p r e s e n t i n g a s e t of N d a t a s i t e s
% i n R"s ( i . e . , each row c o n t a i n s one
7o s-dimensional p o i n t )
% ctrs: Mxs matrix r e p r e s e n t i n g a s e t of M c e n t e r s f o r
7o RBFs i n R~s ( a l s o one c e n t e r p e r row)
7 ep: determines s i z e of support of b a s i s f u n c t i o n .
7o Small ep y i e l d s wide f u n c t i o n ,
7o i . e . , s u p p o r t s i z e = 1/ep
7. Output
7, DM: NxM SPARSE m a t r i x t h a t c o n t a i n s t h e E u c l i d e a n
 12. Interpolation with Compactly Supported RBFs in M A T L A B 97

/
0 u - d i s t a n c e ( u = l - e * r ) between t h e i - t h d a t a
% s i t e and t h e j - t h c e n t e r i n t h e i , j p o s i t i o n
% Uses: k-D t r e e package by Guy S h e c h t e r from
'/, MATLAB C e n t r a l F i l e Exchange
1 f u n c t i o n DM = D i s t a n c e M a t r i x C S R B F ( d s i t e s , c t r s , e p )
2 N = size(dsites,1); M = size(ctrs,1);
7 B u i l d k-D t r e e f o r d a t a s i t e s
% F o r each c e n t e r ( b a s i s f u n c t i o n ) , f i n d t h e d a t a s i t e s
% i n i t s support along w i t h u - d i s t a n c e
3 support = 1/ep;
4 nzmax = 25*N; rowidx = zeros(1,nzmax); c o l i d x = zeros(1,nzmax);
5 v a l i d x = zeros(1,nzmax); i s t a r t = 1; i e n d = 0;
6 i f M > N / f a s t e r i f more c e n t e r s than d a t a s i t e s
7 [tmp,tmp,Tree] = k d t r e e ( c t r s , [] ) ;
8 f o r i = 1:N
9 [pts,dist,idx] = kdrangequery(Tree,dsites(i,:),support);
10 newentries = l e n g t h ( i d x ) ;
11 iend = iend + n e w e n t r i e s ;
12 rowidx(istart:iend) = repmat(i,1,newentries);
13 c o l i d x ( i s t a r t : i e n d ) = idx';
14 validx(istart:iend) = l-ep*dist';
15 i s t a r t = i s t a r t + newentries;
16 end
17 e l s e
18 [tmp,tmp,Tree] = k d t r e e ( d s i t e s , [ ] ) ;
19 f o r j = 1:M
20 [[Link],idx] = kdrangequery(Tree,ctrs(j,:),support);
21 newentries = l e n g t h ( i d x ) ;
22 iend = iend + n e w e n t r i e s ;
23 rowidx(istart:iend) = idx';
24 c o l i d x ( i s t a r t : i e n d ) = repmat(j,[Link]);
25 validx(istart:iend) = l-ep*dist';
26 i s t a r t = i s t a r t + newentries;
27 end
f
28 end
29 idx = f i n d ( r o w i d x ) ;
30 DM = s p a r s e ( r o w i d x ( i d x ) , c o l i d x ( i d x ) , v a l i d x ( i d x ) , N , M ) ;
7. Free t h e k-D Tree from memory.
31 kdtree ( [ ] , , Tree) ;
T h e reason for c o d i n g D i s t a n c e M a t r i x C S R B F . m i n t w o different w a y s is so t h a t
we w i l l be able t o speed u p t h e p r o g r a m w h e n d e a l i n g w i t h n o n - s q u a r e ( e v a l u a t i o n )
m a t r i c e s (for e x a m p l e i n t h e c o n t e x t o f M L S a p p r o x i m a t i o n (c.f. Chapter 24).

-3*, *
98 Meshfree Approximation Methods with M A T L A B

O n e c o u l d also i m p l e m e n t t h e d i s t a n c e m a t r i x r o u t i n e for sparse m a t r i c e s as

follows:

1 f u n c t i o n DM = D i s t a n c e M a t r i x C S R B F ( d s i t e s , c t r s , e p )
2 N = size(dsites,l); M = size(ctrs,1);
% B u i l d k-D t r e e f o r d a t a sites
% F o r each c e n t e r ( b a s i s f u n c t i o n ) , f i n d t h e d a t a sites
% i n i t s support along w i t h u - d i s t a n c e
3 support = 1/ep; nzmax = 25*N; DM = spalloc(N,M,nzmax);
4 [tmp, tmp,Tree] = k d t r e e ( d s i t e s , [ ] ) ;
5 f o r j = 1:M
6 Cpts,dist,idx] = kdrangequery(Tree,ctrs(j,:),support);
7 DM(idx.j) = l - e p * d i s t ;
8 end
% F r e e the k-D Tree from memory.
9 kdtree ( [ ] , [ ] , Tree) ;

T h i s code is c e r t a i n l y easier t o f o l l o w , b u t n o t as efficient as t h e one l i s t e d i n

P r o g r a m 12.1. N o t e t h a t we l i s t e d o n l y one v e r s i o n o f t h e code here. C l e a r l y , t h e
a l t e r n a t i v e v e r s i o n can be a d d e d a n a l o g o u s l y t o t h e p r e v i o u s p r o g r a m . j
T h e i n t e r p o l a t i o n p r o g r a m is v i r t u a l l y i d e n t i c a l t o P r o g r a m 2.1. The only
changes are t o replace lines 13 a n d 15 b y t h e c o r r e s p o n d i n g lines

13 DM_data = D i s t a n c e M a t r i x C S R B F ( d s i t e s , c t r s , e p ) ;
15 DM_eval = D i s t a n c e M a t r i x C S R B F ( e p o i n t s , c t r s , e p ) ;
2
a n d t o define t h e R B F i n s h i f t e d f o r m , i.e., i n s t e a d o f r e p r e s e n t i n g , e.g., t h e C
W e n d l a n d f u n c t i o n c / ^ i o n line 1 b y

1 r b f = @(e,r) m a x ( l - e * r , 0 ) . ~ 4 . * ( 4 * e * r + l ) ; ep=0.7;

we n o w w r i t e

1 r b f = @(e,r) r . " 4 . * ( 5 * s p o n e s ( r ) - 4 * r ) ; ep=0.7;

N o t e t h e use o f t h e sparse m a t r i x o f ones spones. H a d we used 5-4*r i n s t e a d , t h e n

a full m a t r i x w o u l d have been g e n e r a t e d ( w i t h m a n y a d d i t i o n a l a n d u n w a n t e d
ones).
I n o r d e r t o speed u p t h e s o l u t i o n o f t h e ( s y m m e t r i c p o s i t i v e d e f i n i t e ) sparse
linear s y s t e m we c o u l d use t h e p r e c o n d i t i o n e d c o n j u g a t e g r a d i e n t a l g o r i t h m (peg
in MATLAB) i n s t e a d o f t h e basic backslash \ (or m a t r i x left d i v i s i o n m l d i v i d e )
o p e r a t i o n , i.e., we c o u l d replace line 17 of P r o g r a m 2.1 b y

17 c = p c g ( I M , r h s ) ; Pf = EM * c;

N o t e , however, t h a t t h e \ o p e r a t o r also e m p l o y s s t a t e - o f - t h e - a r t direct sparse solvers

by first a p p l y i n g a m i n i m u m degree p r e o r d e r i n g .
 12. Interpolation with Compactly Supported RBFs in M A T L A B 99

12-2 Numerical Experiments with CSRBFs

-yVe now present t w o sets o f i n t e r p o l a t i o n e x p e r i m e n t s w i t h c o m p a c t l y supported

RBFs. I n T a b l e 12.2 we use t h e non-stationary approach to interpolation, i.e.,
the s u p p o r t size r e m a i n s fixed for i n c r e a s i n g l y denser sets X o f d a t a sites. I n t h i s
approach we w i l l be able t o observe convergence. However, the matrices become
increasingly denser, a n d therefore t h e n o n - s t a t i o n a r y a p p r o a c h is v e r y inefficient.
I n Table 1 2 . 1 , o n t h e o t h e r h a n d , we use t h e stationary a p p r o a c h , i.e., we scale t h e
support size o f t h e basis f u n c t i o n s p r o p o r t i o n a l l y t o t h e fill distance hx,o. (defined
in (2.3)). N o w t h e " b a n d w i d t h " o f t h e i n t e r p o l a t i o n m a t r i x A is c o n s t a n t . This
theoretically results i n O(N) c o m p u t a t i o n a l c o m p l e x i t y , i.e., a v e r y efficient i n t e r -
p o l a t i o n m e t h o d . T h e s t a t i o n a r y i n t e r p o l a t i o n m e t h o d is also n u m e r i c a l l y stable,
but there w i l l be essentially n o convergence (see T a b l e 12.1).
We use W e n d l a n d ' s c o m p a c t l y s u p p o r t e d f u n c t i o n </?3,i(r) = ( 1 r ) + ( 4 r + 1)
to i n t e r p o l a t e Franke's f u n c t i o n (2.2) o n g r i d s o f e q u a l l y spaced p o i n t s i n t h e u n i t
2
square [0, l ] . I n t h e s t a t i o n a r y case ( T a b l e 12.1) t h e s u p p o r t o f t h e basis f u n c t i o n
starts o u t w i t h a n i n i t i a l scale p a r a m e t e r e = 0.7 w h i c h is subsequently multiplied
by a factor o f t w o whenever t h e fill distance is h a l v e d , i.e., w h e n we r e p e a t t h e
experiment o n t h e n e x t finer g r i d . T h i s corresponds t o k e e p i n g a c o n s t a n t number
of r o u g h l y 25 d a t a sites w i t h i n t h e s u p p o r t o f a n y basis f u n c t i o n . T h e r e f o r e , the
" b a n d w i d t h " o f t h e i n t e r p o l a t i o n m a t r i x A is k e p t c o n s t a n t ( a t 2 5 ) , so t h a t A is v e r y
sparse for finer grids. W e c a n observe nice convergence for t h e first few i t e r a t i o n s ,
3
but once an R M S - e r r o r o f a p p r o x i m a t e l y 5 x 1 0 ~ is reached, t h e r e is n o t m u c h
further i m p r o v e m e n t . T h i s b e h a v i o r is n o t y e t f u l l y u n d e r s t o o d . H o w e v e r , i t is
similar t o w h a t happens i n t h e approximate approximation m e t h o d o f M a z ' y a (see,
e.g., [ M a z ' y a a n d S c h m i d t (2001)] a n d o u r discussion i n C h a p t e r 2 6 ) . The rate
r&te
listed i n t h e t a b l e is t h e e x p o n e n t o f t h e observed R M S - c o n v e r g e n c e r a t e 0(h ).
I t is c o m p u t e d u s i n g t h e f o r m u l a

e
r a t '"fa--/ *> fc = 2,3,..., (12.1)
ln(/ifc_i/ftfc)

where e k is t h e e r r o r for e x p e r i m e n t n u m b e r k, a n d h k is t h e fill distance o f t h e

kth c o m p u t a t i o n a l mesh. N o t e , t h a t for u n i f o r m l y spaced p o i n t s t h e r a t i o o f f i l l
distances o f t w o consecutive meshes w i l l always be t w o , w h i l e for r a n d o m p o i n t s
(such as H a l t o n p o i n t s ) we e s t i m a t e t h e fill distance v i a ( 2 . 4 ) . The % nonzero
c o l u m n indicates t h e s p a r s i t y o f t h e i n t e r p o l a t i o n m a t r i c e s , a n d t h e t i m e is m e a s u r e d
in seconds. E r r o r s are c o m p u t e d o n a n e v a l u a t i o n g r i d o f 40 x 40 e q u a l l y spaced
2
points i n [0, l ] .
I n t h e n o n - s t a t i o n a r y case ( T a b l e 12.2) we use basis f u n c t i o n s w i t h o u t a d j u s t i n g
their s u p p o r t size, i.e., e = 0.7 is k e p t fixed for a l l e x p e r i m e n t s . W e have convergence
a l t h o u g h i t is n o t o b v i o u s w h a t t h e r a t e m i g h t be. However, t h e m a t r i c e s b e c o m e
increasingly dense a n d c o m p u t a t i o n requires l o t s o f s y s t e m m e m o r y . T h e r e f o r e , we
left o u t t h e s o l u t i o n for t h e N = 16641 a n d N = 66049 cases i n T a b l e 12.2. T h e t i m e
100 Meshfree Approximation Methods with M A T L A B

Table 12.1 Stationary interpolation at N equally

2
spaced points in [0, l ] (constant 25 points in support)
with Wendland's function <p(r) = (1 r ) ( 4 r + 1). +

N RMS-error rate % nonzero time

9 1.562729e-001 100 0.23

25 2.690350e-002 2.5382 57.8 0.31
81 1.027881e-002 1.3881 23.2 0.33
289 6.589552e-003 0.6414 7.47 0.41
1089 3.891263e-003 0.7599 2.13 0.63
4225 3.726913e-003 0.0623 0.57 1.23
16641 2.638296e-003 0.4984 0.15 3.75
66049 2.467867e-003 0.0963 0.04 15.48

Table 12.2 Non-stationary interpolation

at N equally spaced points in [0,1] 2

(e = 0.7 fixed) with Wendland's! function

4
<p(r) = (1 - r ) (4r + 1)

N RMS-error rate time

9 1.562729e-001 0.03
25 2.807706e-002 2.4766 0.04
81 4.853006e-003 2.5324 0.12
289 2.006041e-004 4.5965 0.45
1089 1.288000e-005 3.9611 2.75
4225 1.382497e-006 3.2198 47.92

c o m p a r i s o n b e t w e e n t h e entries i n T a b l e 12.1 a n d T a b l e 12.2 is n o t a s t r a i g h t f o r w a r d

one since we used t h e (dense) code P r o g r a m 2.1 t o do t h e e x p e r i m e n t s for T a b l e 12.2
since t h e r e is no sparseness t o be e x p l o i t e d a n d t h e kd-tvees actually introduce
a d d i t i o n a l overhead.
T h e i n t e r p l a y b e t w e e n c o m p u t a t i o n a l efficiency a n d non-convergence i n t h e sta-
t i o n a r y case a n d convergence a n d c o m p u t a t i o n a l inefficiency i n t h e n o n - s t a t i o n a r y
case is a g a i n a trade-off principle similar t o the i n t e r p l a y between accuracy and
i l l - c o n d i t i o n i n g for g l o b a l l y s u p p o r t e d R B F s (c.f. C h a p t e r 2 ) . These t r a d e - o f f p r i n -
ciples were e x p l a i n e d t h e o r e t i c a l l y as w e l l as i l l u s t r a t e d w i t h n u m e r i c a l e x p e r i m e n t s
i n [Schaback ( 1 9 9 7 b ) ] , a n d we w i l l consider t h e m i n C h a p t e r 16.
For c o m p a r i s o n purposes we r e p e a t t h e e x p e r i m e n t s w i t h t h e o s c i l l a t o r y basic
function
2 3
>(r) = o (r)
2 = ( 1 - r)% (3 + 1 8 r + 3 r - 192r ) ,
4 s
w h i c h is also C s m o o t h a n d s t r i c t l y p o s i t i v e d e f i n i t e a n d r a d i a l o n M for s < 3
(see T a b l e 11.5). T h e results are l i s t e d i n T a b l e 12.3 for t h e s t a t i o n a r y case a n d i n
T a b l e 12.4 for t h e n o n - s t a t i o n a r y case. N o t e t h a t t h e f u n c t i o n is i m p l e m e n t e d as

rbf = @(e,r) -r."6.*(168*spones(r)-552*r+573*r.~2-192*r."3);

 12. Interpolation with Compactly Supported RBFs in M A T L A B 101

Table 12.3 Stationary interpolation at N equally spaced

2
points in [0, l ] (constant 25 points in support) with the os-
2 3
cillatory function <p(r) = ( l - r ) (3 + 18r + 3 r - 1 9 2 r ) .
+

N RMS-error rate % nonzero time

9 1.655969e-001 100 0.28

25 3.941226e-002 2.0710 57.8 0.34
81 2.978973e-002 0.4038 23.2 0.36
289 2.914215e-002 0.0317 7.47 0.42
1089 3.063424e-002 -0.0720 2.13 0.64
4225 3.094308e-002 -0.0145 0.57 1.31
16641 3.089882e-002 0.0021 0.15 4.13
66049 3.086639e-002 0.0015 0.04 16.81

Table 12.4 Non-stationary interpolation

2
at N equally spaced points in [0,1]
(e = 0.7 fixed) with the oscillatory function
<p(r) = 2
( 3 + 1 8 r + 3 r - 192r ). 3

N RMS-error rate time

9 1.655969e-001 0.03
25 3.097850e-002 2.4183 0.06
81 4.612941e-003 2.7475 0.20
289 1.305297e-004 5.1432 0.72
1089 4.780575e-006 4.7711 4.06
4225 2.687479e-007 4.1529 55.09

i n t h e sparse s e t t i n g a n d as

rbf = @(e,r) m a x ( l - e * r , 0 ) . " 6 . * ( 3 + 1 8 * e * r + 3 * ( e * r ) . " 2 - 1 9 2 * ( e * r ) . " 3 ) ;

for t h e dense code.

W h i l e t h e p e r f o r m a n c e o f t h e o s c i l l a t o r y f u n c t i o n s for t h e s t a t i o n a r y e x p e r i -
m e n t is even m o r e d i s a p p o i n t i n g t h a n t h a t o f W e n d l a n d ' s f u n c t i o n s , t h e s i t u a t i o n
is reversed i n t h e n o n - s t a t i o n a r y case. I n fact, t h e e r r o r s o b t a i n e d w i t h t h e o s c i l l a -
t o r y basis f u n c t i o n s are a l m o s t as g o o d as those achieved w i t h " o p t i m a l l y " scaled
Gaussians (c.f. T a b l e 2.2).
I n o r d e r t o overcome t h e p r o b l e m s d u e t o t h e t r a d e - o f f p r i n c i p l e t h a t are ap-
parent i n b o t h the stationary and non-stationary approach t o i n t e r p o l a t i o n w i t h
c o m p a c t l y s u p p o r t e d r a d i a l f u n c t i o n s we w i l l l a t e r consider u s i n g a m u l t i l e v e l sta-
t i o n a r y scheme (see C h a p t e r 3 2 ) .
 Chapter 13

Reproducing Kernel Hilbert Spaces and

Native Spaces for Strictly Positive
Definite Functions

I n t h e n e x t few c h a p t e r s w e w i l l present some o f t h e t h e o r e t i c a l w o r k o n e r r o r

b o u n d s for a p p r o x i m a t i o n a n d i n t e r p o l a t i o n w i t h r a d i a l basis f u n c t i o n s . Since t h e
discussion for s t r i c t l y p o s i t i v e d e f i n i t e f u n c t i o n s w i l l a l r e a d y be t e c h n i c a l e n o u g h ,
we focus o n t h i s case, a n d o n l y m e n t i o n a few r e s u l t s for t h e c o n d i t i o n a l l y p o s i t i v e
definite case. T h e f o l l o w i n g discussion f o l l o w s m o s t l y t h e p r e s e n t a t i o n i n [ W e n d l a n d
(2005a)] w h e r e t h e i n t e r e s t e d reader c a n f i n d m a n y m o r e d e t a i l s .

13.1 Reproducing Kernel Hilbert Spaces

O u r first set o f e r r o r b o u n d s w i l l c o m e r a t h e r n a t u r a l l y once w e associate w i t h each

( s t r i c t l y p o s i t i v e d e f i n i t e ) r a d i a l basic f u n c t i o n a c e r t a i n space o f f u n c t i o n s c a l l e d i t s
native space. W e w i l l t h e n be able t o e s t a b l i s h a c o n n e c t i o n t o r e p r o d u c i n g k e r n e l
H i l b e r t spaces, w h i c h i n t u r n w i l l g i v e us t h e desired e r r o r b o u n d s as w e l l as c e r t a i n
o p t i m a l i t y results for r a d i a l basis f u n c t i o n i n t e r p o l a t i o n (see C h a p t e r 18).
R e p r o d u c i n g kernels are a classical c o n c e p t i n analysis i n t r o d u c e d b y N a c h m a n
A r o n s z a j n (see [ A r o n s z a j n ( 1 9 5 0 ) ] ) . W e b e g i n w i t h

S
D e f i n i t i o n 1 3 . 1 . L e t H be a r e a l H i l b e r t space o f f u n c t i o n s / : Q(Q R ) > R w i t h
i n n e r p r o d u c t (-, - ) ^ - A f u n c t i o n K : Q x ft R is called reproducing kernel for Ji
if

(1) K(-,x) e H for a l l x Q,

(2) f(x) = (f,K(-,x)) n for a l l / e W a n d all x fl.

T h e n a m e reproducing kernel is i n s p i r e d b y t h e r e p r o d u c i n g p r o p e r t y (2) i n

D e f i n i t i o n 1 3 . 1 . I t is k n o w n t h a t t h e r e p r o d u c i n g k e r n e l o f a H i l b e r t space is u n i q u e ,
a n d t h a t existence o f a r e p r o d u c i n g k e r n e l is e q u i v a l e n t t o t h e fact t h a t t h e p o i n t
evaluation functionals 5 X are b o u n d e d l i n e a r f u n c t i o n a l s o n Cl, i.e., t h e r e exists a
positive constant M = M x such t h a t

\8 f\
x = \f(x)\<--M\\f\\ n

103
104 Meshfree Approximation Methods with M A T L A B

for a l l / G 7i a n d a l l x G Cl. T h i s l a t t e r fact is d u e t o t h e Riesz representation

theorem.
O t h e r p r o p e r t i e s o f r e p r o d u c i n g kernels are g i v e n b y

Theorem 1 3 . 1 . Suppose H is a Hilbert space of functions f : Cl * K. with repro-

ducing kernel K. Then we have

(1) K(x, y) = (K(-,y), K(; x)) n forx,yeCl.

(2) K(x, y) = K(y, x) for x,yeCl.
(3) Convergence in Hilbert space norm implies pointwise convergence, i.e., if we
have ||/ fnWn 0 for n > oo then
>
\f(x) f {x)\
n 0 for all x G Cl.

Proof. B y P r o p e r t y (1) o f D e f i n i t i o n 13.1 K(-,y) G 7i f o r e v e r y y G Cl. T h e n t h e

r e p r o d u c i n g p r o p e r t y ( 2 ) o f t h e d e f i n i t i o n gives us

K(x,y) = (K(;y),K(;x)) n

for a l l x, y G Cl. T h i s establishes ( 1 ) . P r o p e r t y (2) follows f r o m ( 1 ) b y t h e s y m m e t r y

o f t h e H i l b e r t space inner p r o d u c t . F o r ( 3 ) w e use t h e r e p r o d u c i n g p r o p e r t y o f K
along w i t h the Cauchy-Schwarz inequality:

\f(x) - f {x)\
n = \(f - f ,K(-,x)) \
n n < ||/ - f \\ \\K(;x)\\ .
n n n D

N o w i t is i n t e r e s t i n g for us t h a t t h e r e p r o d u c i n g k e r n e l K is k n o w n t o b e p o s i t i v e
definite. Here w e use a s l i g h t g e n e r a l i z a t i o n o f t h e n o t i o n o f a p o s i t i v e d e f i n i t e func-
t i o n t o a p o s i t i v e d e f i n i t e k e r n e l . Essentially, w e replace &(xj X k ) i n D e f i n i t i o n 3.2
b y K(xj, X k ) . A t t h i s p o i n t w e r e m i n d t h e reader t h a t t h e space o f b o u n d e d l i n e a r
f u n c t i o n a l s o n 7i is k n o w n as i t s dual, a n d d e n o t e d b y 7i*.

Theorem 1 3 . 2 . Suppose Ji is a reproducing kernel Hilbert function space with

reproducing kernel K : Cl x Cl E . Then K is positive definite. Moreover, K is
strictly positive definite if and only if the point evaluation functionals S x are linearly
independent in 7i*.

Proof. Since t h e k e r n e l is r e a l - v a l u e d w e c a n r e s t r i c t ourselves t o a quadratic

N
f o r m w i t h real coefficients. F o r d i s t i n c t p o i n t s X \ , . . . , x^ a n d n o n z e r o c G M. w e
have
N N N N

^^CjCkKixj^Xk) = ^Y2cjC (K{-,Xj),K{-,Xk))n

k

j=l k=l j=l k=l

N n
c K x
= ( j (-> j)>^CkK(;X ))<H k

j=l k=l
N
2
= ||^c -K(-,x )|| ,>0.
J J

j=i

T h u s K is p o s i t i v e d e f i n i t e . T o e s t a b l i s h t h e second c l a i m w e assume K is n o t
s t r i c t l y p o s i t i v e d e f i n i t e a n d show t h a t t h e p o i n t e v a l u a t i o n f u n c t i o n a l s m u s t b e
 13. Reproducing Kernel Hilbert Spaces for Strictly Positive Definite Functions 105

linearly dependent. I f K is n o t s t r i c t l y p o s i t i v e d e f i n i t e t h e n t h e r e e x i s t d i s t i n c t
p o i n t s Xi,..., XN a n d n o n z e r o coefficients c - such t h a t 3

N N
c c
Y2Y2 j kK{xj,x ) k = 0.
3=1 k=l
T h e first p a r t o f t h e p r o o f t h e r e f o r e i m p l i e s

3= 1

N o w we t a k e t h e H i l b e r t space i n n e r p r o d u c t w i t h a n a r b i t r a r y f u n c t i o n / 6 H a n d
use t h e r e p r o d u c i n g p r o p e r t y o f K t o o b t a i n

3= 1
N
c K x
= ^2 3(fi (-, 3))n
3= 1
N
3= 1
N

3=1
T h i s , however, i m p l i e s t h e l i n e a r dependence o f t h e p o i n t e v a l u a t i o n f u n c t i o n a l s
S Xj ( / ) f( j)i
x
j 1, ) AT, since t h e coefficients Cj were assumed t o be n o t a l l
zero. A n analogous a r g u m e n t c a n be used t o e s t a b l i s h t h e converse.

T h i s t h e o r e m p r o v i d e s one d i r e c t i o n o f t h e c o n n e c t i o n b e t w e e n s t r i c t l y p o s i t i v e
definite f u n c t i o n s a n d r e p r o d u c i n g kernels. H o w e v e r , we are also i n t e r e s t e d i n t h e
o t h e r d i r e c t i o n . Since t h e R B F s w e have b u i l t o u r i n t e r p o l a t i o n m e t h o d s f r o m are
strictly positive definite functions, we w a n t t o k n o w how t o construct a r e p r o d u c i n g
k e r n e l H i l b e r t space associated w i t h those s t r i c t l y p o s i t i v e d e f i n i t e basic f u n c t i o n s .

13.2 N a t i v e S p a c e s for S t r i c t l y P o s i t i v e D e f i n i t e Functions

I n t h i s section we w i l l show t h a t e v e r y s t r i c t l y p o s i t i v e d e f i n i t e r a d i a l basic f u n c t i o n

can indeed be associated w i t h a r e p r o d u c i n g k e r n e l H i l b e r t space i t s native space.
F i r s t , we n o t e t h a t D e f i n i t i o n 13.1 t e l l s us t h a t Ti. c o n t a i n s a l l f u n c t i o n s o f t h e
form
N
K X
f = ^2c3 (-i 3)
3=1
106 Meshfree Approximation Methods with M A T L A B

p r o v i d e d Xj Cl. A s a consequence o f T h e o r e m 13.1 we have t h a t

N N

11/11* = (f,f)n = ( Y , ^ K { . , X j ) ^ c K { . , x ) )
k k u

3= 1 k=l
N N
S c
= Z2^C3 k{K(-,x ),K(-,x ))n j k

3= 1 k=l
N N

C C K X X
= ^2Yl i k ( 3' k)-
3=1 k=l
Therefore, we define t h e (possibly i n f i n i t e - d i m e n s i o n a l ) space

H (Cl)
K = s p a n { K ( - , y) : y Cl} (13.1)

w i t h a n associated b i l i n e a r f o r m (, )#- g i v e n b y

=
N K N K N K N K

C^Z 3 i^ j)^Yl ^' ^

C K x
dkK yk K
Y2^2 j k ( 3^Vk), C d K X

3= 1 k=l j=l k=l

w h e r e NK = oo is also a l l o w e d .

T h e o r e m 1 3 . 3 . If K : Cl x Cl > E is a symmetric strictly positive definite kernel,

then the bilinear form {-,-)K defines an inner product on HK(CI). Furthermore,
HK(CI) is a pre-Hilbert space with reproducing kernel K .

Proof. (-, -)K is o b v i o u s l y b i l i n e a r a n d s y m m e t r i c . W e j u s t need t o show t h a t

(/> f ) K > 0 for nonzero / G HK(CI). A n y such / c a n be w r i t t e n i n t h e f o r m
N K

f = ^2cjK(;Xj), XjeCl.
3= 1

Then
N K N K

(f, f)K = Y2Y2c c K(x ,x ) j k j k >0

3= 1 k=l
since K is s t r i c t l y p o s i t i v e d e f i n i t e . T h e r e p r o d u c i n g p r o p e r t y follows f r o m
N K

{f, K(; X)) K = CjK(x, Xj) = f(x).

3= 1

Since we j u s t showed t h a t HK{CI) is a p r e - H i l b e r t space, i.e., need n o t be c o m -

plete, we n o w define t h e native space JVK(CI) o f K t o be t h e c o m p l e t i o n o f HK(CI)
G
w i t h respect t o t h e i f - n o r m || \ \ K so t h a t \\f\\K = H/HA/W^) ^ O R A 1 1
f H {Cl).
K

T h e t e c h n i c a l details concerned w i t h t h i s c o n s t r u c t i o n are discussed i n [ W e n d l a n d

(2005a)].
I n t h e special case w h e n we are d e a l i n g w i t h s t r i c t l y p o s i t i v e d e f i n i t e ( t r a n s l a t i o n
i n v a r i a n t ) f u n c t i o n s &(x y) = K(x, y) a n d w h e n Cl = R s
we get a c h a r a c t e r i z a t i o n
of n a t i v e spaces i n t e r m s o f F o u r i e r t r a n s f o r m s .
 13. Reproducing Kernel Hilbert Spaces for Strictly Positive Definite Functions 107

S s
T h e o r e m 1 3 . 4 . Suppose $ G C ( R ) n L i ( R ) is a real-valued strictly positive def-
inite function. Define

S S 3
g = {fe L (R ) n C(R ):
2 -= e -MR )}

7
ana egwip /ns space with the bilinear form

i ,/ 9 , i / / M I R do;.

Then Q is a real Hilbert space with inner product (, -)g and reproducing kernel
s S
<&( ) . Hence, Q is the native space of & on R , i.e., Q = A / $ ( R ) and both
S
inner products coincide. In particular, every f G A / $ ( R ) can be recovered from its
s S
Fourier transform f G Za(R ) fl L ( R ) . 2

A n o t h e r c h a r a c t e r i z a t i o n o f t h e n a t i v e space is g i v e n i n t e r m s o f t h e eigenfunc-
t i o n s o f a linear o p e r a t o r associated w i t h t h e r e p r o d u c i n g k e r n e l . This operator,
T $ : L 2 ( P ) > L (fl),2 is g i v e n b y

T*(v)(x)= &(x,y)v(y)dy, v G L (fl),2 ^ G fi.

in
For t h e eigenvalues Afc, k = 1, 2 , . . ., a n d eigenfunctions (p o f t h i s o p e r a t o r M e r c e r ' s k

t h e o r e m [Riesz a n d Sz.-Nagy (1955)] states

T h e o r e m 1 3 . 5 ( M e r c e r ) . Let <&(, ) be a continuous positive definite kernel that

satisfies

/ $(x,y)v(x)v(y)dxdy > 0, for all v G L (fl), 2 x,y E fl. (13.2)

Then <fr can be represented by

oo
$(x,y) = ^X (t> {x)4> {y),
k k k (13.3)
fc=i
where X k are the (non-negative) eigenvalues and 4> are the (L2-orthonormal) k eigen-
functions of T<&. Moreover, this representation is absolutely and uniformly conver-
gent.

W e can i n t e r p r e t c o n d i t i o n (13.2) as a t y p e o f integral positive definiteness. As

usual, t h e eigenvalues a n d eigenfunctions satisfy T<s>(f> = \ <p k k k or

/ $(x,y)(p (y)dy
k = \ <f) (x),
k k A; = 1 , 2 , . . . .
Jn
I n general, M e r c e r ' s t h e o r e m allows us t o c o n s t r u c t a r e p r o d u c i n g k e r n e l H i l b e r t
space 7i b y representing t h e f u n c t i o n s i n 7i as i n f i n i t e linear c o m b i n a t i o n s o f t h e
eigenfunctions, i.e.,

{
oo
/: / = ^ C f c 0 *
108 Meshfree Approximation Methods with M A T L A B

I t is clear t h a t t h e k e r n e l <3> i t s e l f is i n 7i since i t has t h e e i g e n f u n c t i o n expansion

( 1 3 . 3 ) . T h e i n n e r p r o d u c t for 7i is g i v e n b y
oo oo oo ^
(f,g)n = C ^ C j ^ j ^ Y ^ d k ^ n = ^ '
3=1
k
k=l k=l

w h e r e we used t h e 7 i - o r t h o g o n a l i t y

(<t>j,4>k)H =
y/Xj^/Xk

of t h e eigenfunctions.
W e n o t e t h a t $ is indeed t h e r e p r o d u c i n g k e r n e l o f Ti since t h e eigenfunction
e x p a n s i o n (13.3) o f <E> a n d t h e o r t h o g o n a l i t y o f t h e eigenfunctions imply
oo oo
(/,$(, x)) n = (Y2cj<pj,^2Xk(f)k<Pk(x))n
j=i k=i

_ CkXk4>k{x)

oo
- ^c <f> ix)
k k = fix).
k=i

F i n a l l y , one has (c./. [ W e n d l a n d (2005a)]) t h a t t h e n a t i v e space A/$(f2) is g i v e n

by

A / - * ( f i ) = j / L (n) 2 : J-|(/,0 f c ) i 2 ( o ) |2 < o o |

a n d t h e n a t i v e space i n n e r p r o d u c t c a n be w r i t t e n as
oo ^
(f,g)rt* = J^T-(/i0fc)L (n)(^,0fc>L (n), a 2 f,g e jV&iQ,).
A k
k=i

Since A/$(f2) is a subspace o f L (fl) 2 t h i s c o r r e s p o n d s t o t h e i d e n t i f i c a t i o n Ck =

(/) 4>k)L (Q) 3 f ^he generalized F o u r i e r coefficients i n t h e discussion above.

13.3 E x a m p l e s o f N a t i v e S p a c e s for P o p u l a r R a d i a l B a s i c
Functions

T h e o r e m 13.4 shows t h a t n a t i v e spaces o f t r a n s l a t i o n i n v a r i a n t f u n c t i o n s c a n be

v i e w e d as a g e n e r a l i z a t i o n o f s t a n d a r d Sobolev spaces. I n d e e d , for m > s/2 the
Sobolev space W c a n be defined as (see, e.g., [ A d a m s (1975)])
S S S 2 m/2 S
W^{R ) = { / G L (R ) 2 n C(R ) : / ( - ) ( 1 + || \\ ) 2 G L (R )}.
2 (13.4)

O n e also f r e q u e n t l y sees t h e d e f i n i t i o n
m a s
W 2 ( f l ) = { / G L ( f i ) n C(Q)
2 : Df G L {Q)
2 for a l l \a\ < m, a GN }, (13.5)
 13. Reproducing Kernel Hilbert Spaces for Strictly Positive Definite Functions 109

S
w h i c h applies whenever Q, C R is a b o u n d e d d o m a i n . This interpretation will
make clear t h e c o n n e c t i o n b e t w e e n t h e n a t i v e s spaces o f Sobolev splines a n d t h o s e
3
of p o l y h a r m o n i c splines t o be discussed b e l o w . The n o r m of W ^ R ) is u s u a l l y
given by

I|/IIW7(R-) = { H j D a
/lli (R ) 2
S

||<m
A c c o r d i n g t o ( 1 3 . 4 ) , any s t r i c t l y p o s i t i v e d e f i n i t e f u n c t i o n 3> whose F o u r i e r t r a n s -
f o r m decays o n l y a l g e b r a i c a l l y has a Sobolev space as i t s n a t i v e space. I n p a r t i c u l a r ,
the M a t e r n functions
K -t(\\x\\)\\x\\f>-*
0 8
= ' ^ 2'

of Section 4.4 w i t h F o u r i e r t r a n s f o r m

* () = (i + H | ) -
/9
2 / 9

8
can i m m e d i a t e l y be seen t o have n a t i v e space JV* (R ) P = Wg(R) with B > s/2
( w h i c h is w h y some people refer t o t h e M a t e r n f u n c t i o n s as Sobolev splines).
W e n d l a n d ' s c o m p a c t l y s u p p o r t e d f u n c t i o n s <& fc = <>s,fc(|| l b ) o f C h a p t e r 11 c a n
S]

s 2 + k + 1 2 s
be s h o w n t o have n a t i v e spaces A / $ a k (R ) = W ^ 2 ^ (R ) (where the r e s t r i c t i o n
s > 3 is r e q u i r e d for t h e special case k = 0 ) .
N a t i v e spaces for s t r i c t l y c o n d i t i o n a l l y p o s i t i v e d e f i n i t e f u n c t i o n s c a n also be
c o n s t r u c t e d . H o w e v e r , since t h i s is m o r e t e c h n i c a l , we l i m i t e d t h e discussion a b o v e
t o s t r i c t l y p o s i t i v e d e f i n i t e f u n c t i o n s , a n d refer t h e i n t e r e s t e d reader t o t h e b o o k
[ W e n d l a n d (2005a)] o r t h e p a p e r s [Schaback (1999a); Schaback (2000a)]. W i t h t h e
extension of the theory t o s t r i c t l y c o n d i t i o n a l l y positive definite functions the native
spaces o f t h e r a d i a l powers a n d t h i n p l a t e (or surface) splines o f Sections 8.2 a n d
8.3 c a n be s h o w n t o be t h e so-called Beppo-Levi spaces o f o r d e r k
S S a S s
BL F C ( R ) = { / C{R ) : Df G L (R )
2 for a l l | a | = k, a e N } ,
a
where D denotes a generalized derivative o f o r d e r at (defined i n t h e same s p i r i t as
t h e generalized F o u r i e r t r a n s f o r m , see A p p e n d i x B ) . I n fact, t h e i n t e r s e c t i o n o f a l l
S 3
B e p p o - L e v i spaces B L f c ( R ) o f o r d e r k < m y i e l d s t h e Sobolev space W ^ R ) . In
S
t h e l i t e r a t u r e t h e B e p p o - L e v i spaces B L f c ( R ) are sometimes referred t o as homo-
s
geneous Sobolev spaces of order k. A l t e r n a t i v e l y , t h e B e p p o - L e v i spaces o n R are
defined as
S S S
BL F C ( R ) = { / G C{R ) : /(-)|| | | ? G L (R )},
2

a n d t h e f o r m u l a s g i v e n i n C h a p t e r 8 for t h e F o u r i e r t r a n s f o r m s o f r a d i a l powers a n d
t h i n p l a t e splines show i m m e d i a t e l y t h a t t h e i r n a t i v e spaces are B e p p o - L e v i spaces.
T h e s e m i - n o r m o n BLfc is g i v e n b y

mBL* = { t ^ ^ i i ^ / i i i 2 ( R . > } , (13.6)

110 Meshfree Approximation Methods with M A T L A B

a n d i t s k e r n e l is t h e p o l y n o m i a l space n | _ . x F o r m o r e details see [Wendland

(2005a)]. B e p p o - L e v i spaces were a l r e a d y s t u d i e d i n t h e e a r l y papers [ D u c h o n
(1976); D u c h o n (1977); D u c h o n (1978); D u c h o n ( 1 9 8 0 ) ] .
T h e n a t i v e spaces for Gaussians a n d (inverse) m u l t i q u a d r i c s are r a t h e r s m a l l .
For e x a m p l e , a c c o r d i n g t o T h e o r e m 13.4, for Gaussians t h e F o u r i e r t r a n s f o r m o f
/ J\f<i>(fl) m u s t decay faster t h a n t h e F o u r i e r t r a n s f o r m o f t h e G a u s s i a n ( w h i c h is
itself a G a u s s i a n ) . I t is k n o w n t h a t , even t h o u g h t h e n a t i v e space o f Gaussians is
s m a l l , i t does c o n t a i n t h e i m p o r t a n t class o f so-called band-limited functions, i.e.,
functions whose F o u r i e r t r a n s f o r m is c o m p a c t l y s u p p o r t e d . These f u n c t i o n s p l a y
an i m p o r t a n t role i n sampling theory w h e r e S h a n n o n ' s famous s a m p l i n g t h e o r e m
[Shannon (1949)] states t h a t a n y b a n d - l i m i t e d f u n c t i o n c a n be c o m p l e t e l y recovered
f r o m i t s discrete samples p r o v i d e d t h e f u n c t i o n is s a m p l e d at a s a m p l i n g r a t e at
least t w i c e i t s b a n d w i d t h . T h e c o n t e n t o f t h i s t h e o r e m was a l r e a d y k n o w n m u c h
earlier (see [ W h i t t a k e r ( 1 9 1 5 ) ] ) .

T h e o r e m 1 3 . 6 ( S h a n n o n S a m p l i n g ) . Suppose f C ( J R ) n L i ( I R ) such that

5 s
its
s
Fourier transform vanishes outside the cube Q = [ | , | ] . Then f can be uniquely
reconstructed from its values on Z s
; i.e.,

s
Here t h e sine f u n c t i o n is defined for a n y x = (xi,... ,x )
s M as sine a; =
m o r e d e t a i l s o n S h a n n o n ' s s a m p l i n g t h e o r e m see, e.g., Chap-
ter 29 i n t h e b o o k [Cheney a n d L i g h t (1999)] or t h e p a p e r [Unser ( 2 0 0 0 ) ] .
 Chapter 14

The Power Function and Native Space

Error Estimates

14.1 Fill Distance and Approximation Orders

Our goal i n t h i s s e c t i o n is t o p r o v i d e e r r o r e s t i m a t e s for s c a t t e r e d d a t a i n t e r p o l a t i o n

w i t h strictly (conditionally) positive definite functions. A s i n the previous chapter
we w i l l p r o v i d e m o s t o f t h e d e t a i l s for t h e s t r i c t l y p o s i t i v e d e f i n i t e case, a n d o n l y
m e n t i o n t h e e x t e n s i o n t o t h e c o n d i t i o n a l l y p o s i t i v e d e f i n i t e case i n t h e e n d . I n t h e i r
final f o r m we w i l l w a n t o u r e s t i m a t e s t o d e p e n d o n some k i n d o f measure o f t h e
d a t a d i s t r i b u t i o n . T h e measure t h a t is u s u a l l y used i n a p p r o x i m a t i o n t h e o r y is t h e
so-called fill distance

h = h ,n x = sup m i n \\x Xj\\ 2

already i n t r o d u c e d i n (2.3) i n C h a p t e r 2. T h e fill distance i n d i c a t e s h o w w e l l t h e

d a t a fill o u t t h e d o m a i n Q, a n d i t t h e r e f o r e denotes t h e r a d i u s o f t h e largest e m p t y
b a l l t h a t can be p l a c e d a m o n g t h e d a t a l o c a t i o n s . W e w i l l be i n t e r e s t e d i n w h e t h e r
the e r r o r

tends t o zero as h > 0, a n d i f so, h o w fast. H e r e {V^}h presents a sequence

of i n t e r p o l a t i o n (or, m o r e generally, p r o j e c t i o n ) o p e r a t o r s t h a t v a r y w i t h t h e fill
n s
distance h. For e x a m p l e , c o u l d d e n o t e i n t e r p o l a t i o n t o d a t a g i v e n at ( 2 + l ) ,
S n
n = 1 , 2 , . . . , e q u a l l y spaced p o i n t s i n t h e u n i t cube i n JR ( w i t h h = 2~ ) as w e
used i n some o f o u r earlier e x a m p l e s . O f course, t h e d e f i n i t i o n o f t h e fill d i s t a n c e
also covers s c a t t e r e d d a t a such as sets o f H a l t o n p o i n t s . I n fact, since H a l t o n p o i n t s
n
are q u a s i - u n i f o r m l y d i s t r i b u t e d (see A p p e n d i x A ) we c a n assume h ^ 2 ~ for a set
n s S
of ( 2 + l) H a l t o n p o i n t s i n JR . T h i s e x p l a i n s t h e specific sizes o f t h e p o i n t sets
we used i n earlier examples.
Since we w a n t t o e m p l o y t h e m a c h i n e r y o f r e p r o d u c i n g k e r n e l H i l b e r t spaces
presented i n t h e p r e v i o u s c h a p t e r w e w i l l c o n c e n t r a t e o n e r r o r e s t i m a t e s for f u n c t i o n s
/ G A / $ . I n t h e n e x t c h a p t e r w e w i l l also m e n t i o n some m o r e general estimates.
T h e t e r m t h a t is o f t e n used t o m e a s u r e t h e speed o f convergence t o zero is approx-
imation order. W e say t h a t t h e a p p r o x i m a t i o n o p e r a t o r has L -approximation
p

ill
112 Meshfree Approximation Methods with M A T L A B

order k i f
h) k
\\f-V$ \\ p = 0(h ) for/i-O.
h) k w
M o r e o v e r , i f we c a n also show t h a t | | / - V \\ f p / o(h ), then V has exact
L -approximation
p order k. W e w i l l c o n c e n t r a t e m o s t l y o n t h e case p = oo {i.e.,
p o i n t w i s e estimates), b u t a p p r o x i m a t i o n o r d e r i n o t h e r n o r m s c a n also be s t u d i e d .
I n order t o keep t h e f o l l o w i n g discussion as t r a n s p a r e n t as possible we w i l l r e s t r i c t
ourselves t o s t r i c t l y p o s i t i v e definite f u n c t i o n s . W i t h ( c o n s i d e r a b l y ) m o r e t e c h n i c a l
d e t a i l s t h e f o l l o w i n g can also be f o r m u l a t e d for s t r i c t l y c o n d i t i o n a l l y p o s i t i v e d e f i n i t e
f u n c t i o n s (see [ W e n d l a n d (2005a)] for d e t a i l s ) .

14.2 L a g r a n g e F o r m of t h e I n t e r p o l a n t a n d Cardinal
Basis Functions

T h e key idea for t h e f o l l o w i n g discussion is t o express t h e i n t e r p o l a n t i n Lagrange

form, i.e., u s i n g so-called cardinal basis functions. F o r r a d i a l basis f u n c t i o n a p p r o x -
i m a t i o n t h i s idea is due t o [ W u a n d Schaback ( 1 9 9 3 ) ] . I n t h e p r e v i o u s c h a p t e r s we
established t h a t , for a n y s t r i c t l y p o s i t i v e d e f i n i t e f u n c t i o n <E>, t h e l i n e a r s y s t e m

Ac = y
T
with Aij = (xi - Xj), i,j = 1,...,N, c = [ci,..., c],
N and y =
T
[f(xi),...,f(xN)] has a u n i q u e s o l u t i o n . I n t h e f o l l o w i n g we w i l l consider the
m o r e general s i t u a t i o n w h e r e $ is a s t r i c t l y p o s i t i v e d e f i n i t e k e r n e l , i.e., t h e entries
of A are g i v e n b y A^ = &(xi,Xj). T h e uniqueness r e s u l t holds i n t h i s case also.
I n order t o o b t a i n t h e c a r d i n a l basis f u n c t i o n s u* , j = 1,... 3 ,N, w i t h the prop-
e r t y u*(xi) = Sij, i.e.,

Uj{Xi) =

we consider t h e linear s y s t e m

Au*(x) = b(x), (14.1)

T
where t h e m a t r i x A is as above ( a n d therefore i n v e r t i b l e ) , u* = [u\,..., u* } ,
N and
6 = [ ^ ( ; X 1 ) , . . . , ^ ( ; X N ) ] T
. Thus,

S
T h e o r e m 1 4 . 1 . Suppose <fr is a strictly positive definite kernel on I R . Then, for any
distinct points x\,..., XN, there exist functions u* e span{<E>(-, Xj),j = 1,..., N}
such that Uj(xi) = 5ij.

T h e r e f o r e , we can w r i t e t h e i n t e r p o l a n t Vf t o / a t x\,..., XN m t h e c a r d i n a l
form
 14- The Power Function and Native Space Error Estimates 113

I t is o f interest t o n o t e t h a t t h e c a r d i n a l f u n c t i o n s d o n o t d e p e n d o n t h e d a t a
values o f t h e i n t e r p o l a t i o n p r o b l e m . O n c e t h e d a t a sites are fixed a n d t h e basic
f u n c t i o n is chosen w i t h a n a p p r o p r i a t e s h a p e p a r a m e t e r (whose o p t i m a l v a l u e w i l l
d e p e n d o n t h e d a t a ) , t h e n t h e c a r d i n a l f u n c t i o n s are d e t e r m i n e d b y t h e l i n e a r s y s t e m
(14.1). W e have p l o t t e d v a r i o u s c a r d i n a l f u n c t i o n s based o n t h e G a u s s i a n basic
f u n c t i o n w i t h shape p a r a m e t e r e = 5 i n F i g u r e s 1 4 . 1 - 1 4 . 3 . T h e dependence o n t h e
d a t a l o c a t i o n s is c l e a r l y a p p a r e n t w h e n c o m p a r i n g t h e different d a t a d i s t r i b u t i o n s
( u n i f o r m l y spaced i n F i g u r e 1 4 . 1 , t e n s o r - p r o d u c t C h e b y s h e v i n F i g u r e 14.2, and
H a l t o n p o i n t s i n F i g u r e 14.3). T h e d a t a sets c a n be seen i n F i g u r e 14.5 b e l o w .

Fig. 14.1 Cardinal functions for Gaussian interpolation (with e = 5) on 81 uniformly spaced
2
points in [0, l ] . Centered at an edge point (left) and at an interior point (right).

Fig. 14.2 Cardinal functions for Gaussian interpolation (with e = 5) on 81 tensor-product Cheby-
2
shev points in [0, l ] . Centered at an edge point (left) and at an interior point (right).

Basic f u n c t i o n s t h a t g r o w w i t h i n c r e a s i n g d i s t a n c e f r o m t h e center p o i n t (such as

m u l t i q u a d r i c s ) are s o m e t i m e s c r i t i c i z e d for b e i n g " c o u n t e r - i n t u i t i v e " for s c a t t e r e d
d a t a a p p r o x i m a t i o n . H o w e v e r , as F i g u r e 14.4 shows, t h e c a r d i n a l f u n c t i o n s are j u s t
as l o c a l i z e d as those for t h e G a u s s i a n basic f u n c t i o n s , a n d t h u s t h e f u n c t i o n space
114 Meshfree Approximation Methods with M A T L A B

Fig. 14.3 Cardinal functions for Gaussian interpolation (with e = 5) on 81 Halton points in
2
[0, l ] . Centered at an edge point (left) and at an interior point (right).

spanned by multiquadrics is a "good" local space.

Fig. 14.4 Cardinal functions for multiquadric interpolation (with e = 5) on 81 Halton points in
2
[0, l ] . Centered at an edge point (left) and at an interior point (right).

The M A T L A B program RBFCardinalFunction.m used to produce the plots in

Figures 14.1-14.3 is provided in Program 14.1. Note that we use the pseudo-inverse
(via the MATLABcommand pinv) to stably compute the inverse of the interpolation
matrix (see line 13 of Program 14.1). A specific cardinal function is then chosen in
line 15.

P r o g r a m 14.1. RBFCardinalFunction.m

7o R B F C a r d i n a l F u n c t i o n
/ Computes and p l o t s c a r d i n a l f u n c t i o n f o r 2D RBF i n t e r p o l a t i o n
/ C a l l s on: D i s t a n c e M a t r i x
0

1 r b f = @(e,r) e x p ( - ( e * r ) . " 2 ) ; ep = 5 ;
2 N = 81; g r i d t y p e = 'u';
3 neval = 8 0 ; M = neval~2;
 14- The Power Function and Native Space Error Estimates 115

/ Load data p o i n t s
0

4 name = s p r i n t f ( 'Data2D_%d/ s' ,N,gridtype) ; load(name)

0

5 ctrs = dsites; 7 c e n t e r s c o i n c i d e w i t h data s i t e s

6 g r i d = l i n s p a c e (0 ,1, n e v a l ) ; [xe,ye] = [Link] i d ( g r i d ) ;
7 epoints = [xe(:) y e ( : ) ] ;
7o Compute d i s t a n c e m a t r i x between e v a l u a t i o n p o i n t s and c e n t e r s
8 DM_eval = D i s t a n c e M a t r i x ( e p o i n t s , c t r s ) ;
% Compute d i s t a n c e m a t r i x between t h e data s i t e s and c e n t e r s
9 DM_data = D i s t a n c e M a t r i x ( d s i t e s , c t r s ) ;
% Compute i n t e r p o l a t i o n m a t r i x
10 IM = rbf(ep,DM_data);
% Compute e v a l u a t i o n m a t r i x
11 EM = rbf(ep,DM_eval);
% Compute c a r d i n a l f u n c t i o n s a t e v a l u a t i o n p o i n t s
12 invIM = p i n v ( I M ) ;
% centered a t d a t a s i t e ( 5 0 )
13 f o r j=l:M
14 cardvec = ( i n v I M * E M ( j , : ) ' ) ' ;
15 cardfun(j) = cardvec(50);
16 end
17 figure
18 RBFplot = s u r f ( x e , y e , r e s h a p e ( c a r d f u n , n e v a l , n e v a l ) ) ;
19 set(RBFplot,'FaceColor','interp','EdgeColor','none')
20 colormap autumn; v i e w ( [ 1 4 5 4 5 ] ) ; camlight; l i g h t i n g gouraud

14.3 The Power Function

A n o t h e r i m p o r t a n t i n g r e d i e n t needed for o u r e r r o r estimates is t h e so-called power

s
function. T o t h i s end, we consider a d o m a i n fl C M . T h e n for any s t r i c t l y p o s i t i v e
definite k e r n e l <& G C(fl x fl), a n y set o f d i s t i n c t p o i n t s X = {x\,..., XN} ^ ^l, a n d
any vector u G JR^, we define t h e q u a d r a t i c form
N N N

Q(u) = $(x,x) - 2^>2uj$(x,Xj) + y^ y^ UiUj$(x ,x ).

j j i j

j=l i=l j = l

Then

s
D e f i n i t i o n 1 4 . 1 . Suppose fiCl a n d $ G C(fl x fl) is s t r i c t l y p o s i t i v e d e f i n i t e
S
on JR . For a n y d i s t i n c t p o i n t s X = {x,..., XN} C fl t h e power function is defined
by

w h e r e u* is t h e v e c t o r o f c a r d i n a l f u n c t i o n s f r o m T h e o r e m 1 4 . 1 .
116 Meshfree Approximation Methods with M A T L A B

U s i n g t h e d e f i n i t i o n o f t h e n a t i v e space n o r m f r o m t h e p r e v i o u s c h a p t e r we c a n
r e w r i t e t h e q u a d r a t i c f o r m Q(u) as

^2 Y2 Yl
N N N

Q{u) = <fr(cc, x) 2 Uj$(x, Xj) + u

i u
3 ^ { x
i i X
j )

j=l i=l j=l

N
2 x $ x
= ($(, x), $ ( - , W n } - )> ("' i))wsi)

N N

+ i j{(-i
u u x
i), H-> x
j))rt*(n)
i=i j=i

N N

3=1 3=1
2
N

(14.2)
3= 1

The name power function was chosen b y Schaback based o n i t s c o n n e c t i o n t o t h e

power function of a statistical decision f u n c t i o n (originally i n t r o d u c e d i n [ N e y m a n
a n d P e a r s o n ( 1 9 3 6 ) ] ) . I n t h e p a p e r [ W u a n d S c h a b a c k (1993)] t h e p o w e r f u n c t i o n
was referred t o as kriging function. T h i s t e r m i n o l o g y comes f r o m geostatistics (see,
e.g., [Myers (1992)]).
U s i n g t h e l i n e a r s y s t e m n o t a t i o n e m p l o y e d earlier, i.e., A\j = &(xi,Xj), i,j =
T
1 , . . . , N, u = [ i i i , . . . , w y v ] , a n d b = [<&(, xi),..., $(, XJV)] , T
we n o t e t h a t we c a n
also r e w r i t e t h e q u a d r a t i c f o r m Q(u) as

T T
Q(u) = (x, x) - 2u b(x) 4- u Au. (14.3)

T h i s suggests t w o a l t e r n a t i v e r e p r e s e n t a t i o n s o f t h e p o w e r f u n c t i o n . Using the

m a t r i x - v e c t o r n o t a t i o n for Q ( u ) , t h e p o w e r f u n c t i o n is g i v e n as

T T
P* (x)
jX = y/Q(u*(x)) = \J${x,x) - 2(u*(x)) b(x) + (u*(x)) Au*(x).

H o w e v e r , b y t h e d e f i n i t i o n o f t h e c a r d i n a l f u n c t i o n s Au*(x) = b(x), and therefore

w e have t h e t w o n e w v a r i a n t s

x x x T
P*,x{x) = \/$( , )-(u*( )) b(x)

T
= \J$(x,x) - (u*(x)) Au*(x).

These f o r m u l a s c a n be used for t h e n u m e r i c a l e v a l u a t i o n o f t h e p o w e r f u n c t i o n

a t x. T o t h i s e n d one has t o f i r s t find t h e v a l u e o f t h e c a r d i n a l f u n c t i o n s u*(x) by
s o l v i n g t h e s y s t e m Au*(x) = b(x). T h i s results i n

P*,x(x) = y/${x,x) - {b(x)) A- b(x). T l

(14.4)
 14- The Power Function and Native Space Error Estimates 117

Since A is a p o s i t i v e definite m a t r i x whenever $ is a s t r i c t l y p o s i t i v e d e f i n i t e k e r n e l

we see t h a t t h e p o w e r f u n c t i o n satisfies t h e b o u n d s

0<P*,*(aO < y / $ ( x , x ) .

P l o t s o f t h e p o w e r f u n c t i o n for t h e G a u s s i a n w i t h e 6 o n t h r e e different p o i n t
sets w i t h N = 81 i n t h e u n i t square are p r o v i d e d i n F i g u r e 14.5. T h e sets o f d a t a
p o i n t s are displayed o n t h e left, w h i l e t h e p l o t s o f t h e power f u n c t i o n are d i s p l a y e d
in the right column. Dependence o f t h e p o w e r f u n c t i o n o n t h e d a t a l o c a t i o n s is
clearly v i s i b l e . I n fact, t h i s c o n n e c t i o n was used i n a recent p a p e r [De M a r c h i et al.
(2005)] t o i t e r a t i v e l y o b t a i n a n o p t i m a l set o f d a t a l o c a t i o n s t h a t are independent
of t h e d a t a values.
A t t h i s p o i n t t h e power f u n c t i o n is m o s t l y a t h e o r e t i c a l t o o l t h a t helps us b e t t e r
u n d e r s t a n d e r r o r estimates since we c a n decouple t h e effects due t o t h e d a t a f u n c t i o n
/ f r o m those due t o t h e basic f u n c t i o n <& a n d t h e d a t a l o c a t i o n s X (see t h e f o l l o w i n g
T h e o r e m 14.2).
T h e power f u n c t i o n is defined i n a n analogous w a y for s t r i c t l y c o n d i t i o n a l l y
p o s i t i v e definite functions.

14.4 G e n e r i c E r r o r E s t i m a t e s for F u n c t i o n s i n Af<p(fl)

N o w we c a n give a first generic e r r o r e s t i m a t e .

s s
T h e o r e m 14.2. Let C I , $ e C ( f 2 x f2) be strictly positive definite on R,
and suppose that the points X = {xi,... ,xjy} are distinct. Denote the interpolant
to f G A/$(f2) on X by Vf. Then for every x G O

\f(x)-V (x)\<P*, (x)\\f\U .

f x m

Proof. Since / is assumed t o lie i n t h e n a t i v e space o f <& t h e r e p r o d u c i n g p r o p e r t y

of $ y i e l d s

f(x) = (/,$(", 2 0 ) ^ ( 0 ) .

W e express t h e i n t e r p o l a n t i n i t s c a r d i n a l f o r m a n d a p p l y t h e r e p r o d u c i n g p r o p e r t y
o f 3>. T h i s gives us

N
Vf(x) = J2f(*j)uj(x)
3= 1
N

3= 1
N
118 Meshfree Approximation Methods with M A T L A B

5
z-m-

0 0

1* o o

o o
0.8

0.6
o o

0.4
o o o o

0.2
o o o o o

o o o o o
0*
0.2 0.4 0.6 0.8 0 0
X

0 0

Fig. 14.5 Data sites and power function for Gaussian interpolant with e = 6 based on N = 81
uniformly distributed points (top), tensor-product Chebyshev points (middle), and Halton points
(bottom).

N o w a l l t h a t r e m a i n s t o be done is t o c o m b i n e t h e t w o f o r m u l a s j u s t d e r i v e d a n d
apply the Cauchy-Schwarz inequality. T h u s ,

N
\f(x)-V (x)\
f = (f,&(-,x)-Y2 j( )*(-i j))M-*(n)
u x x

3= 1
 14- The Power Function and Native Space Error Estimates 119

<
i=i
= H/II.V(n)-P*,*(aO,

where we have a p p l i e d (14.2) a n d t h e d e f i n i t i o n o f t h e power f u n c t i o n .

One o f t h e m a i n benefits o f T h e o r e m 14.2 is t h a t we are n o w able t o e s t i m a t e
the i n t e r p o l a t i o n e r r o r b y c o n s i d e r i n g t w o i n d e p e n d e n t phenomena:

the smoothness o f t h e d a t a ( m e a s u r e d i n t e r m s o f t h e n a t i v e space n o r m o f /

w h i c h is independent o f t h e d a t a l o c a t i o n s , b u t does d e p e n d o n <&),
a n d t h e c o n t r i b u t i o n due t o t h e use o f t h e specific k e r n e l (i.e., basic f u n c t i o n )
$ and the d i s t r i b u t i o n o f the d a t a (measured i n terms of the power function
independent o f t h e a c t u a l d a t a values).

T h i s is analogous t o t h e s t a n d a r d e r r o r e s t i m a t e for p o l y n o m i a l i n t e r p o l a t i o n c i t e d
i n most n u m e r i c a l analysis t e x t s . N o t e , however, t h a t , for a n y g i v e n basic f u n c t i o n
$ , a change o f t h e shape p a r a m e t e r e w i l l have a n effect o n b o t h t e r m s i n t h e e r r o r
b o u n d i n T h e o r e m 14.2 since t h e n a t i v e space n o r m o f / varies w i t h e.

14.5 Error Estimates i n Terms of the Fill Distance

T h e next step is t o refine t h i s e r r o r b o u n d b y expressing t h e influence o f t h e d a t a

locations i n t e r m s o f t h e fill distance. A n d t h e n , o f course, t h e b o u n d needs t o be
specialized t o v a r i o u s choices o f basic f u n c t i o n s <E>.
T h e most c o m m o n s t r a t e g y for o b t a i n i n g e r r o r b o u n d s i n n u m e r i c a l analysis is
t o take advantage o f t h e p o l y n o m i a l p r e c i s i o n o f a m e t h o d (at least l o c a l l y ) , a n d
t h e n t o a p p l y a T a y l o r e x p a n s i o n . W i t h t h i s i n m i n d we observe

S
T h e o r e m 1 4 . 3 . Let fl C JR , and suppose 3> C(fl x fl) is strictly positive definite
on R . s
Let X = {x\,..., XN} be a set of distinct points in fl, and define the
quadratic form Q(u) as in (14-2). The minimum of Q(u) is given for the vector
u = u*(x) from Theorem 14-1, i-e.,

N
Q(u*(x)) < Q(u) for all u e R .

Proof. W e showed above (see (14.3)) t h a t

T T
Q{u) = $(x, x) - 2u b(x) + u Au.

T h e m i n i m u m o f t h i s q u a d r a t i c f o r m is g i v e n b y t h e s o l u t i o n o f t h e linear s y s t e m

Au = b(x).

T h i s , however, yields t h e c a r d i n a l f u n c t i o n s u = u*(x).

120 Meshfree Approximation Methods with MATLAB

I n t h e p r o o f b e l o w we w i l l use a special coefficient v e c t o r u w h i c h p r o v i d e s t h e

p o l y n o m i a l p r e c i s i o n desired for t h e p r o o f o f t h e r e f i n e d e r r o r e s t i m a t e . I t s existence
is g u a r a n t e e d b y t h e f o l l o w i n g t h e o r e m o n local polynomial reproduction proved i n
[ W e n d l a n d (2005a)]. T h i s t h e o r e m r e q u i r e s t h e n o t i o n o f a d o m a i n t h a t satisfies a n
i n t e r i o r cone c o n d i t i o n .

S
D e f i n i t i o n 1 4 . 2 . A r e g i o n fl C I R satisfies a n interior cone condition i f t h e r e exists
a n angle 6 G ( 0 , 7 r / 2 ) a n d a r a d i u s r > 0 s u c h t h a t for every x fl t h e r e exists a
u n i t v e c t o r (x) such t h a t t h e cone

a T
C={x + Xy: y G E , | | y | | = 1 , y (x)
2 >cosd, AG[0,r]}

is c o n t a i n e d i n fl.

A consequence o f t h e i n t e r i o r cone c o n d i t i o n is t h e fact t h a t a d o m a i n that

satisfies t h i s c o n d i t i o n c o n t a i n s b a l l s o f a c o n t r o l l a b l e r a d i u s . I n p a r t i c u l a r , t h i s w i l l
be i m p o r t a n t w h e n b o u n d i n g t h e r e m a i n d e r o f t h e T a y l o r e x p a n s i o n s b e l o w . For
m o r e d e t a i l s see [ W e n d l a n d (2005a)].
E x i s t e n c e o f a n a p p r o x i m a t i o n scheme w i t h l o c a l p o l y n o m i a l p r e c i s i o n is g u a r -
anteed by

S
Theorem 1 4 . 4 . Suppose fl C I R is bounded and satisfies an interior cone condi-
tion, and let be a non-negative integer. Then there exist positive constants ho, c\,
and C2 such that for all X = {x,..., Ejv} C fl with h ,nx < ho and every x G fl
there exist numbers ui(x),..., ujv(x) with

N
(1) ^^Uj(x)p(xj) = p(x) for all polynomials p GII|,
i=i
N
(2) X>;(a:)| <ci,
3= 1
(3) Uj{x) = 0 if \\x - xj\\ 2 > c h Q.
2 Xj

P r o p e r t y (1) y i e l d s t h e p o l y n o m i a l p r e c i s i o n , a n d p r o p e r t y (3) shows t h a t the

scheme is l o c a l . T h e b o u n d i n p r o p e r t y ( 2 ) is essential for c o n t r o l l i n g t h e g r o w t h
o f e r r o r e s t i m a t e s a n d t h e q u a n t i t y o n t h e l e f t - h a n d side o f (2) is k n o w n as the
Lebesgue constant at x.
I n t h e f o l l o w i n g t h e o r e m a n d i t s p r o o f w e w i l l m a k e r e p e a t e d use o f m u l t i - i n d e x
n o t a t i o n and m u l t i v a r i a t e T a y l o r expansions. F o r j3 = ( / 5 i , . . . , B)
s G NQ w i t h
\(3\ = YH=i Pi w e
define t h e d i f f e r e n t i a l o p e r a t o r D 13
as

9 l m

(dxi)h (dx )0*'

s

a n d t h e n o t a t i o n D^^iw, ) used b e l o w i n d i c a t e s t h a t t h e o p e r a t o r is a p p l i e d t o
$ ( i y , ) v i e w e d as a f u n c t i o n o f t h e second v a r i a b l e .
 14- The Power Function and Native Space Error Estimates 121

T h e m u l t i v a r i a t e T a y l o r e x p a n s i o n o f t h e f u n c t i o n $ ( t o , ) centered a t w is g i v e n
by

$(w,z)= J2 3 i \ z - w f + R(w,z)
\/3\<2k P
'

w i t h remainder

R(w,z)= 2^ /3!
\0\=2k P
'

where ^ , 2 lies somewhere o n t h e l i n e segment c o n n e c t i n g w a n d z.

T h e generic error e s t i m a t e o f T h e o r e m 14.2 c a n n o w be f o r m u l a t e d i n t e r m s o f
t h e f i l l distance.

S
T h e o r e m 14.5. Suppose fl C JR is bounded and satisfies an interior cone condi-
2k
tion. Suppose <& G C (fl x fl) is symmetric and strictly positive definite. Denote
the interpolant to f G A/$(f2) on the set X byVf. Then there exist positive constants
ho and C (independent of x, f and such that

\f(x) - V ( )\
f X < Ch y/C^x)\\f\U^ ,
XtCi a)

provided h ,n
x < ho. Here

C${x) ~ max max l-Off <E(it;, z ) |

\(3\=2k iv,zenr\B(x,C2h ,n) x

with B(x,C2hx,vi) denoting the ball of radius ch n

2 Xt centered at x.

Proof. B y T h e o r e m 14.2 we k n o w

\f(x)-Vf(x)\<P*, (*)\\f\U.M- x

Therefore, we n o w derive t h e b o u n d

P*M*) < c h k
x ^ c * { x )

for t h e power f u n c t i o n i n t e r m s o f t h e f i l l distance.

W e k n o w t h a t t h e p o w e r f u n c t i o n is defined b y
2
[P^ (x)}
x =Q(u*(x)).

M o r e o v e r , we k n o w f r o m T h e o r e m 14.3 t h a t t h e q u a d r a t i c f o r m Q(u) is m i n i m i z e d
by u u*(x). Therefore, a n y o t h e r coefficient v e c t o r u w i l l y i e l d a n u p p e r b o u n d
on t h e power f u n c t i o n . W e t a k e u = u{x) f r o m T h e o r e m 14.4 so t h a t we are ensured
t o have p o l y n o m i a l p r e c i s i o n o f degree i > 2k 1.
For t h i s specific choice o f coefficients we have

[P^ {x)\
jX
2
< Q(u) = $(x, x) - 2 ^2 Uj$(x : xj) + Y2Y2 u u
i i( iix x
i)i
3 i 3
122 Meshfree Approximation Methods with M A T L A B

w h e r e t h e sums are over those indices j w i t h uj ^ 0. N o w we a p p l y t h e T a y l o r

e x p a n s i o n c e n t e r e d a t x t o <&(cc, ) a n d c e n t e r e d a t Xi t o &(xi, ), and evaluate b o t h
f u n c t i o n s a t Xj. T h i s yields

Q(u) = ${x,x)-2Y J m ( g
i - x
f R
+ (> x x
i)
\/3\<2k
,/3
D%$(xi,Xi)
W
+ EE * 3
E
\0\<2k
{xj Xi)^ ~\~ R{x>i, Xj)

N e x t , we i d e n t i f y p(z) = [z x)& so t h a t p(x) 0 unless /3 = 0. T h e r e f o r e the

p o l y n o m i a l p r e c i s i o n p r o p e r t y o f t h e coefficient v e c t o r u s i m p l i f i e s t h i s expression
to

Q(u) (x, x) 2(x, x) 2 23 UjR{x, Xj)

3

+* E P
*(* ~ *^ + E E ( i 4 . 5 )
i |/3|<2fc ' i j

Now we c a n a p p l y t h e T a y l o r e x p a n s i o n a g a i n a n d m a k e t h e o b s e r v a t i o n that

D^(xi,Xj)
E
\/3\<2k
0!
(x - Xi) 13
= <E>(xi, x) - R(xi, x). (14.6)

I f we use (14.6) a n d r e a r r a n g e t h e t e r m s i n (14.5) we get

Q(u) = Q(x, x) 23 u
2R(x,
j Xj) ^3 UiR(x{,Xj)

+ 23 U
i [( ii x x
) - R( i, x x
)\ (14.7)

O n e f i n a l T a y l o r e x p a n s i o n we need is ( u s i n g t h e s y m m e t r y o f <E)

$(xi,x) = &(x,Xi)= 2^ ^ ^ ~ ^ - ( x i - x ) p
+ R(x,Xi). (14.8)
\/3\<2k

I f we i n s e r t (14.8) i n t o (14.7) a n d once m o r e t a k e a d v a n t a g e o f t h e p o l y n o m i a l

p r e c i s i o n p r o p e r t y of t h e coefficient v e c t o r u we are left w i t h

Q( )u
= _
U
J
R(x, Xj) + R(xj,x) 23 UiR(x Xj)i:

Now T h e o r e m 14.4 a l l o w s us t o b o u n d ]3 \uj\ < c\. M o r e o v e r , \\x Xj H2 < C2hx,

and II H2 < 2c2^A",r2- T h e r e f o r e , a l l t h r e e r e m a i n d e r t e r m s c a n be bounded
by a n expression o f t h e f o r m C / I ^ Q C $ ( C C ) . H e r e we m a d e use o f t h e i n t e r i o r cone
p r o p e r t y o f Q e n a b l i n g us t o define t h e t e r m C^(x). C o m b i n i n g these b o u n d s a n d
t a k i n g t h e square r o o t y i e l d s t h e s t a t e d b o u n d for t h e p o w e r f u n c t i o n .
 14- The Power Function and Native Space Error Estimates 123

2 k
T h e o r e m 14.5 says t h a t i n t e r p o l a t i o n w i t h a C s m o o t h k e r n e l <E> has approx-
i m a t i o n order k. T h u s , for i n f i n i t e l y s m o o t h s t r i c t l y p o s i t i v e definite functions
such as t h e Gaussians, L a g u e r r e - G a u s s i a n s , P o i s s o n r a d i a l f u n c t i o n s , a n d t h e gen-
eralized inverse m u l t i q u a d r i c s we see t h a t t h e a p p r o x i m a t i o n o r d e r k is a r b i t r a r i l y
h i g h . For s t r i c t l y p o s i t i v e d e f i n i t e f u n c t i o n s w i t h l i m i t e d smoothness such as t h e
M a t e r n f u n c t i o n s , t h e W h i t t a k e r r a d i a l f u n c t i o n s , as w e l l as a l l o f t h e compactly
s u p p o r t e d functions, t h e a p p r o x i m a t i o n o r d e r is l i m i t e d b y t h e smoothness o f t h e
basic f u n c t i o n .
T h e e s t i m a t e i n T h e o r e m 14.5 is s t i l l generic. I t does n o t f u l l y a c c o u n t for t h e
p a r t i c u l a r basic f u n c t i o n <& b e i n g used for t h e i n t e r p o l a t i o n since t h e factor C$(x)
s t i l l depends o n <E. M o r e o v e r , we p o i n t o u t t h a t t h e t e r m C$(x) may include a
h i d d e n dependence o n hx,n- For m o s t basic f u n c t i o n s i t w i l l be possible t o use
C<s>(x) t o "squeeze o u t " a d d i t i o n a l powers o f h. T h i s is t h e reason for s p l i t t i n g t h e
c o n s t a n t i n f r o n t o f t h e fo-power i n t o a generic C a n d a C$(x).
T h e s t a t e m e n t o f T h e o r e m 14.5 c a n be generalized for s t r i c t l y c o n d i t i o n a l l y pos-
i t i v e definite functions a n d also t o cover t h e error for a p p r o x i m a t i n g t h e d e r i v a t i v e s
of / b y derivatives o f Vf. W e state this general theorem w i t h o u t comment (c.f.
[ W e n d l a n d (2005a)] for d e t a i l s ) .

S
Theorem 14.6. Suppose Q C ] R is open and bounded and satisfies an interior
2k
cone condition. Suppose <& C (Q x Q.) is symmetric and strictly conditionally
positive definite of order m on W. Denote the interpolant to f j\f<&(Q) on the
(m 1 )-unisolvent set X by Vf. Fix ex NQ with \ot\ < k. Then there exist positive
constants ho and C (independent of x, f and such that
a a
\D f(x) - D V (x)\ f < C h ^ V C ^ l f U ^ ,

provided h ,n
x < ho- Here

C<f,(x) = max max |>f D7&(w, z)\.

|/3| + |-y|=2fc

N o t e t h a t for c o n d i t i o n a l l y p o s i t i v e definite f u n c t i o n s we have o n l y a n a t i v e

space s e m i - n o r m instead o f a n o r m .
1
 Chapter 15

Refined and Improved Error Bounds

15.1 N a t i v e S p a c e E r r o r B o u n d s for S p e c i f i c B a s i s F u n c t i o n s

For t h e first p a r t o f t h i s c h a p t e r we discuss t h e n o n - s t a t i o n a r y s e t t i n g . T h e ad-

d i t i o n a l refinement of t h e e r r o r e s t i m a t e o f T h e o r e m 14.6 for specific functions
is r a t h e r t e c h n i c a l (for d e t a i l s see, e.g., the b o o k [Wendland (2005a)]). A large
b o d y o f l i t e r a t u r e exists o n t h i s t o p i c such as, e.g., [ B u h m a n n a n d D y n (1991);
L i g h t (1996); L i g h t a n d W a y n e ( 1 9 9 5 ) ; L i g h t a n d W a y n e (1998); M a d y c h (1992);
M a d y c h a n d N e l s o n ( 1 9 9 2 ) ; N a r c o w i c h a n d W a r d (2004); N a r c o w i c h et al. ( 2 0 0 3 ) ;
N a r c o w i c h et al. ( 2 0 0 5 ) ; Schaback ( 1 9 9 5 b ) ; Schaback (1996); W e n d l a n d (1998);
W e n d l a n d (1997); W u a n d Schaback (1993); Y o o n ( 2 0 0 3 ) ] ) . W e n o w l i s t some o f
t h e results t h a t can be o b t a i n e d .

15.1.1 Infinitely Smooth Basis Functions

A s m e n t i o n e d before, a n a p p l i c a t i o n o f T h e o r e m 14.6 t o i n f i n i t e l y s m o o t h func-

t i o n s such as Gaussians o r generalized (inverse) m u l t i q u a d r i c s i m m e d i a t e l y y i e l d s
a r b i t r a r i l y h i g h algebraic convergence rates, i.e., for every N a n d | a | < we
have
a a
\D f(x) - D T (x)\ f < CV/-ll|/|^ ( n ) . (15.1)
whenever / Af$>(fl). A c o n s i d e r a b l e a m o u n t o f w o r k has gone i n t o i n v e s t i g a t i n g
t h e dependence o f t h e c o n s t a n t Ci o n (see, e.g., [ W e n d l a n d ( 2 0 0 1 b ) ] ) .
U s i n g different p r o o f t e c h n i q u e s (see, e.g., [ M a d y c h a n d N e l s o n (1988)] o r [ W e n d -
l a n d (2005a)] for d e t a i l s ) i t is possible t o d e r i v e m o r e precise e r r o r b o u n d s for Gaus-
sians a n d (inverse) m u l t i q u a d r i c s . T h e r e s u l t i n g t h e o r e m f r o m [ W e n d l a n d (2005a)]
is

T h e o r e m 1 5 . 1 . Let fl be a cube in M . . Suppose S

that <E> = tp(|| ||) is a strictly con-
ditionally positive definite radial function such that ip = <^(\A) satisfies \ip^(r)\ <
\M L
for all integers > Q and all r > 0, where M is a fixed positive constant.
Then there exists a constant c such that for any f j\f$>(fl)

\\f--P \\
f Loo(Q) <e*^|/Ufc ( n ) > (15.2)

125
126 Meshfree Approximation Methods with M A T L A B

for all data sites X with sufficiently small fill distance hx,a-
e
Moreover, if ip satisfies even \ipW(r)\ < M, then

e l n l
Wf-PfU^n) <e ^n' ||/|k. n) ( (15-3)

provided hx,o, is sufficiently small.

_ e 2 x 2 e T
Example 15.1. For Gaussians &(x) = e H H , e > 0 fixed, we have ip{r) = e~ * ',
e 2 i 2 r 2
so t h a t ipW(r) = (-l) e e- for t > = 0. T h u s , M = e , a n d t h e e r r o r b o u n d
0

(15.3) applies. T h i s k i n d o f e x p o n e n t i a l a p p r o x i m a t i o n o r d e r is u s u a l l y r e f e r r e d t o
as spectral (or even s u p e r - s p e c t r a l ) a p p r o x i m a t i o n o r d e r . W e emphasize t h a t t h i s
nice p r o p e r t y holds o n l y i n t h e n o n - s t a t i o n a r y s e t t i n g a n d for d a t a f u n c t i o n s / t h a t
are i n t h e n a t i v e space of t h e Gaussians such as b a n d - l i m i t e d f u n c t i o n s .

2
Example 15.2. For generalized (inverse) m u l t i q u a d r i c s $(x) = ( 1 + |Ja5|| )^, (3 < 0,
e 6
or 0 < 3 N , we have ip(r) = ( 1 + r ) ^ . I n t h i s case one c a n show t h a t \ip (r)\ < \M
w h e n e v e r ^ > \3~\. H e r e M = 1 + 1/3+11. T h e r e f o r e , t h e e r r o r e s t i m a t e (15.2) applies,
i.e., i n t h e n o n - s t a t i o n a r y s e t t i n g generalized (inverse) m u l t i q u a d r i c s have s p e c t r a l
a p p r o x i m a t i o n order.

/2 2 2 2
E x a m p l e 1 5 . 3 . For L a g u e r r e - G a u s s i a n s $(cc) = L (\\ex\\ )e- ^ , n e > 0 fixed,
2 2 e2r e
w e have ip(r) = Ln (e r)e~ a n d t h e d e r i v a t i v e s ip^ w i l l be b o u n d e d b y ip^ \0) =
2i
Pn{) , where p n is a p o l y n o m i a l o f degree n . T h u s , t h e a p p r o x i m a t i o n p o w e r o f
L a g u e r r e - G a u s s i a n s falls b e t w e e n (15.3) a n d (15.2) a n d L a g u e r r e - G a u s s i a n s have at
least s p e c t r a l a p p r o x i m a t i o n p o w e r .

15.1.2 Basis Functions with Finite Smoothness

For f u n c t i o n s w i t h f i n i t e s m o o t h n e s s (such as t h e M a t e r n f u n c t i o n s , r a d i a l powers,

t h i n p l a t e splines, a n d W e n d l a n d ' s c o m p a c t l y s u p p o r t e d f u n c t i o n s ) i t is possible t o
b o u n d t h e c o n s t a n t C$(x) b y some a d d i t i o n a l p o w e r s o f h, a n d t h e r e b y t o i m p r o v e
k
t h e o r d e r p r e d i c t e d b y T h e o r e m 14.6. I n p a r t i c u l a r , for C f u n c t i o n s t h e factor
Cq>(x) can be expressed as

C*(x) = max \\D^\\ L o o { B ( 0 j 2 c h x n ) )

i n d e p e n d e n t o f x (see [ W e n d l a n d ( 2 0 0 5 a ) ] ) . T h e r e f o r e , t h i s results i n t h e f o l l o w i n g
e r r o r e s t i m a t e s (see, e.g., [ W e n d l a n d ( 2 0 0 5 a ) ] , o r t h e m u c h earlier [ W u a n d Schaback
(1993)] w h e r e o t h e r p r o o f t e c h n i q u e s were used).

K0 > w e e t
E x a m p l e 1 5 . 4 . For t h e M a t e r n f u n c t i o n s <I>(CE) = ~^-^vlt) ' @ '

a
\D~f{x)-D V,{x)\ < Ch -*-\f\^ . x m (15.4)

p r o v i d e d \a\ < 3 f ^ ! ) hx,u 5

is sufficiently s m a l l , a n d / G J\f<s>(Q).
 15. Refined and Improved Error Bounds 127

Example 1 5 . 5 . For t h e c o m p a c t l y s u p p o r t e d W e n d l a n d functions <fr fc(cc)

Si =
35
^[Link] !!) f h i s first r e f i n e m e n t leads t o
a | t t |
\D f{x) - DV (x)\ f < C^*- ||/||^ ( n ) . (15.5)

p r o v i d e d | a | < k, hx,n is s u f f i c i e n t l y s m a l l , a n d / A/$(S1).

2
E x a m p l e 1 5 . 6 . For t h e r a d i a l p o w e r s $(sc) = ( - 1 ) ^ / 1 ||a:||0, 0 < 3 2 N , w e get
a
\D f(x) - D<*V {x)\ } < C h ^ l f U ^ . (15.6)
x
p r o v i d e d \a\ < ^ ~ , hx,o,
2 is s u f f i c i e n t l y s m a l l , a n d / G A / $ ( f 2 ) .
f e + 1 2 / i :
E x a m p l e 1 5 . 7 . For t h i n p l a t e splines <&(x) = ( - l ) ||cc|| l o g ||cc||, w e get
a a k
\D f{x) - D V (x)\ f < Ch -^\fW, .x {Cl) (15.7)

p r o v i d e d \ot\ < k 1, hx,n is s u f f i c i e n t l y s m a l l , a n d / G j\f<s>(fl).

15.2 I m p r o v e m e n t s for N a t i v e S p a c e E r r o r Bounds

R a d i a l powers a n d t h i n p l a t e splines c a n be i n t e r p r e t e d as a g e n e r a l i z a t i o n o f u n i -
v a r i a t e n a t u r a l splines. T h e r e f o r e , w e k n o w t h a t t h e a p p r o x i m a t i o n o r d e r e s t i m a t e s
o b t a i n e d v i a t h e n a t i v e space a p p r o a c h are n o t o p t i m a l . For e x a m p l e , for i n t e r -
p o l a t i o n w i t h u n i v a r i a t e piecewise l i n e a r splines ( i . e . , $(cc) = ||cc|| i n x G R ) w e
k n o w t h e a p p r o x i m a t i o n o r d e r t o b e O(h), whereas t h e e s t i m a t e (15.6) y i e l d s o n l y
2 2
a p p r o x i m a t i o n o r d e r 0{h}/ ). S i m i l a r l y , for t h i n p l a t e splines &(x) = ||cc|| l o g ||ic||
2
one w o u l d expect o r d e r 0(h ) i n t h e case o f p u r e f u n c t i o n a p p r o x i m a t i o n . H o w e v e r ,
t h e e s t i m a t e (15.7) yields o n l y 0(h). These t w o e x a m p l e s suggest t h a t i t s h o u l d be
possible t o " d o u b l e " t h e a p p r o x i m a t i o n o r d e r s o b t a i n e d t h u s far.
O n e c a n i m p r o v e t h e e s t i m a t e s for f u n c t i o n s w i t h finite smoothness ( i . e . , M a t e r n
f u n c t i o n s , W e n d l a n d f u n c t i o n s , r a d i a l powers, a n d t h i n p l a t e splines) b y e i t h e r (or
b o t h ) o f t h e f o l l o w i n g t w o ideas:

b y r e q u i r i n g t h e d a t a f u n c t i o n / t o be even s m o o t h e r t h a n w h a t t h e n a t i v e
space prescribes, i.e., b y b u i l d i n g c e r t a i n b o u n d a r y c o n d i t i o n s i n t o t h e n a t i v e
space;
b y u s i n g weaker n o r m s t o m e a s u r e t h e e r r o r .

T h e idea t o localize t h e d a t a b y a d d i n g b o u n d a r y c o n d i t i o n s was i n t r o d u c e d i n

t h e p a p e r [ L i g h t a n d W a y n e ( 1 9 9 8 ) ] . T h i s " t r i c k " a l l o w s us t o "square" t h e a p p r o x -
i m a t i o n order, a n d t h u s r e a c h t h e e x p e c t e d a p p r o x i m a t i o n orders. T h e second i d e a
can a l r e a d y be f o u n d i n t h e e a r l y p a p e r [ D u c h o n ( 1 9 7 8 ) ] .
A f t e r a p p l y i n g b o t h o f these t e c h n i q u e s t h e f i n a l a p p r o x i m a t i o n o r d e r e s t i m a t e
for i n t e r p o l a t i o n w i t h t h e c o m p a c t l y s u p p o r t e d f u n c t i o n s <& fc is (see [ W e n d l a n d S)

(1997)])
2k 1+s
11/ - P / I U n ) < Ch + \\f\\ , r s
a ( w k+ + {R3)1 (15.8)
128 Meshfree Approximation Methods with M A T L A B

2 f c + 1 + S S s
w h e r e / is assumed t o lie i n t h e subspace W 2 ( R ) o f Af<j,(R ) = W^"^ . For
r =
e x a m p l e , for t h e p o p u l a r basic f u n c t i o n ^3,1 ( ) 0- ~~ r)+(4r + 1) w e have
6
\\f~V \\ ^ f L <Ch \\f\\ e . w iRs)

N o t e t h a t t h e n u m e r i c a l e x p e r i m e n t s i n T a b l e 12.2 p r o d u c e d R M S - c o n v e r g e n c e r a t e s
o n l y as h i g h as 4.5.
For r a d i a l powers a n d t h i n p l a t e splines o n e o b t a i n s L 2 - e r r o r e s t i m a t e s o f o r -
f3+s 2 k + s
der 0(h ) and 0 ( h ), respectively. T h e s e e s t i m a t e s a r e o p t i m a l , i.e., e x a c t
a p p r o x i m a t i o n orders, as s h o w n i n [ B e j a n c u ( 1 9 9 9 ) ] .
M o r e w o r k o n i m p r o v e d e r r o r b o u n d s c a n b e f o u n d i n , e.g., [ J o h n s o n (2004a)]
or [Schaback ( 1 9 9 9 b ) ] .

15.3 E r r o r B o u n d s for F u n c t i o n s O u t s i d e t h e N a t i v e Space

T h e e r r o r b o u n d s m e n t i o n e d so f a r were a l l v a l i d u n d e r t h e a s s u m p t i o n t h a t t h e
f u n c t i o n / p r o v i d i n g t h e d a t a c a m e f r o m ( a subspace o f ) t h e n a t i v e space o f t h e
RBF employed i n the interpolation. W e n o w m e n t i o n a f e w recent r e s u l t s that
provide error bounds for i n t e r p o l a t i o n o f functions / not i n t h e n a t i v e space o f
t h e basic f u n c t i o n . I n p a r t i c u l a r , t h e case w h e n / lies i n some S o b o l e v space is o f
great i n t e r e s t . A r a t h e r general t h e o r e m w a s r e c e n t l y g i v e n i n [ N a r c o w i c h et al.
(2005)]. I n t h i s t h e o r e m N a r c o w i c h , W a r d a n d W e n d l a n d p r o v i d e S o b o l e v b o u n d s
for f u n c t i o n s w i t h m a n y zeros. H o w e v e r , since t h e i n t e r p o l a t i o n e r r o r f u n c t i o n is
j u s t such a f u n c t i o n , these b o u n d s have a d i r e c t a p p l i c a t i o n t o o u r s i t u a t i o n . W e
p o i n t o u t t h a t t h i s t h e o r e m a g a i n applies t o t h e n o n - s t a t i o n a r y s e t t i n g .

T h e o r e m 15.2. Let k be a positive integer, 0 < a < l , l < p < 00, 1 < q < 0 0
and let a be a multi-index satisfying k > \ct\ + s/p or, for p = 1, k > \a\ -f- s.
Let X C fl be a discrete set with fill distance h hx,n where fl is a compact set
k+a
with Lipschitz boundary which satisfies an interior cone condition. If u e W (fl)
satisfies u\x = 0, then
k+a a s 1 1
1 , < rh ~\ \- ( /P- /l)+1 | 7/ t ,

where c is a constant independent of u and h, and {x) + is the cutoff function.

k + a
Suppose we have a n i n t e r p o l a t i o n process V : W ' (fl) > V t h a t m a p s S o b o l e v
k + a
f u n c t i o n s t o a f i n i t e - d i m e n s i o n a l subspace V o f W (fl) w i t h t h e a d d i t i o n a l p r o p -
f e + C T
e r t y \pf\ k+a^ w < |/lvK ( f i ) ' t h e n T h e o r e m 15.2 i m m e d i a t e l y y i e l d s t h e e r r o r
estimate

f c + C T s
T h e a d d i t i o n a l p r o p e r t y [Pf\ k+a^ w < |/lw ( n ) * c e r t a i n l y satisfied p r o v i d e d
t h e n a t i v e space o f t h e basic f u n c t i o n is a Sobolev space. T h u s , T h e o r e m 15.2
p r o v i d e s a n a l t e r n a t i v e t o t h e p o w e r f u n c t i o n a p p r o a c h discussed i n t h e p r e v i o u s
 15. Refined and Improved Error Bounds 129

chapter i f we base V o n l i n e a r c o m b i n a t i o n s o f shifts o f t h e basic f u n c t i o n <E>. T h i s

new a p p r o a c h has t h e a d v a n t a g e t h a t t h e t e r m C<j>(x) w h i c h m a y d e p e n d o n b o t h
<& a n d X no longer needs t o be d e a l t w i t h .
I n p a r t i c u l a r , t h e a u t h o r s o f [ N a r c o w i c h et al. (2005)] s h o w t h a t i f t h e F o u r i e r
t r a n s f o r m o f <E> satisfies

ci(l + ||u>||l)- T
< $(w) < c (l + 2 ||a;||l)- , T
- oo, u e R,
s
(15.9)

t h e n t h e above e r r o r e s t i m a t e h o l d s w i t h r = k + a a n d p = 2 p r o v i d e d t h e f i l l
distance is sufficiently s m a l l . E x a m p l e s o f basic f u n c t i o n s w i t h a n a p p r o p r i a t e l y
decaying F o u r i e r t r a n s f o r m are p r o v i d e d b y t h e families o f W e n d l a n d or M a t e r n
functions. I n a d d i t i o n , N a r c o w i c h , W a r d a n d W e n d l a n d s h o w t h a t analogous e r r o r
b o u n d s h o l d for r a d i a l powers a n d t h i n p l a t e splines (whose n a t i v e spaces are B e p p o -
L e v i spaces).
For f u n c t i o n s / outside t h e n a t i v e space o f a basic f u n c t i o n <E> whose F o u r i e r
t r a n s f o r m satisfies (15.9) N a r c o w i c h , W a r d a n d W e n d l a n d p r o v e

T h e o r e m 15.3. Let k and n be integers with 0 < n < k < r and k > s/2, and let
k
f 6 C (fl). Also suppose that X { x i , . . . ,x^} C fl satisfies diam(A') < 1 with
sufficiently small fill distance. Then for any 1 < q < oo we have

1/ - Pflw^in) < cPx fc

C (fi)'

where px = is the mesh ratio for X and qx is the separation distance

\xmm^j \\xi - Xj\\ . 2

We r e m i n d t h e reader t h a t t h e f i l l distance c o r r e s p o n d s t o t h e r a d i u s o f t h e
largest possible e m p t y b a l l t h a t c a n be p l a c e d b e t w e e n t h e p o i n t s i n X. T h e sep-
a r a t i o n distance (c.f. C h a p t e r 16), o n t h e o t h e r h a n d , c a n be i n t e r p r e t e d as t h e
r a d i u s o f t h e largest b a l l t h a t c a n be p l a c e d a r o u n d every p o i n t i n X such t h a t n o
t w o balls o v e r l a p . T h u s , t h e m e s h r a t i o is a measure o f t h e n o n - u n i f o r m i t y o f t h e
d i s t r i b u t i o n o f the points i n X.
S i m i l a r results were o b t a i n e d earlier i n [ B r o w n l e e a n d L i g h t (2004)] (for r a d i a l
powers a n d t h i n p l a t e splines o n l y ) , a n d i n [ Y o o n (2003)] (for shifted surface splines,
see b e l o w ) .

Example 1 5 . 8 . I f we consider p o l y h a r m o n i c splines, t h e n t h e decay c o n d i t i o n

(15.9) for t h e F o u r i e r t r a n s f o r m is satisfied w i t h r = 23 for t h i n p l a t e splines a n d
w i t h T 3 for r a d i a l powers. I f we t a k e k r , n = 0, a n d q = oo i n T h e o r e m 15.3
t h e n we a r r i v e at t h e b o u n d

s 2
\f-r \ f L o o <ch^- / \\f\\ c 0 { m

for r a d i a l powers $ ( x ) = | | x | | ^ . These b o u n d s i m m e d i a t e l y c o r r e s p o n d t o t h e "op-

t i m a l " n a t i v e space b o u n d s o b t a i n e d earlier o n l y after t h e i m p r o v e m e n t s discussed
130 Meshfree Approximation Methods with M A T L A B

i n t h e p r e v i o u s subsection. For d a t a f u n c t i o n s / w i t h less smoothness t h e a p p r o x i -

m a t i o n order is reduced a c c o r d i n g l y .

L o w e r b o u n d s o n t h e a p p r o x i m a t i o n o r d e r for a p p r o x i m a t i o n b y p o l y h a r m o n i c
splines were r e c e n t l y p r o v i d e d i n [ M a i o r o v (2005)]. M a i o r o v studies for a n y 1 <
p, q < oo a n d 3/s > (1/p l/q)+ the error E of //^-approximation of functions
b y p o l y h a r m o n i c splines. M o r e precisely,
a s
E(W^{0, l] ),n {p ,3),L {^,lY))>cN-^ ,
N p q

w h e r e 71^(^/3,0) denotes t h e l i n e a r space f o r m e d b y a l l possible linear c o m b i n a t i o n s

of N p o l y h a r m o n i c (or t h i n - p l a t e t y p e ) splines

a n d m u l t i v a r i a t e p o l y n o m i a l s o f degree at m o s t 3 1. N o t e t h a t these b o u n d s are

i n t e r m s o f t h e n u m b e r N o f d a t a sites i n s t e a d o f t h e u s u a l fill distance h.
For t h e special cases p q = oo a n d p = 2, 1 < g < 2 t h e above lower b o u n d is
s h o w n t o be a s y m p t o t i c a l l y exact.

15.4 E r r o r B o u n d s for S t a t i o n a r y Approximation

T h e s t a t i o n a r y s e t t i n g is a n a t u r a l a p p r o a c h for use w i t h l o c a l basis f u n c t i o n s . T h e

m a i n m o t i v a t i o n comes f r o m t h e c o m p u t a t i o n a l p o i n t o f v i e w . W e are i n t e r e s t e d i n
m a i n t a i n i n g sparse i n t e r p o l a t i o n m a t r i c e s as t h e d e n s i t y o f t h e d a t a increases. T h i s
can be achieved b y scaling t h e basis f u n c t i o n s p r o p o r t i o n a l t o t h e d a t a density.
I n p r i n c i p l e we c a n t a k e a n y o f o u r basic f u n c t i o n s a n d a p p l y a s c a l i n g o f t h e
variable, i.e., we replace x b y ex, e > 0. A s m e n t i o n e d several t i m e s earlier, t h i s
scaling results i n "peaked" or " n a r r o w " basis f u n c t i o n s for large values o f e, a n d
"flat" basis functions for e * 0. W e w i l l n o w discuss w h a t h a p p e n s i f we choose e
inversely p r o p o r t i o n a l t o t h e fill distance, i.e.,

for some fixed base scale eo a n d s t u d y t h e a p p r o x i m a t i o n e r r o r based o n t h e R B F

interpolant
N
Vf(x) = c ^ e h ( | |X - Xj\\),

3= 1

where

Ve
h = <p(h-)-

E x a m p l e 1 5 . 9 . A r a t h e r d i s a p p o i n t i n g fact is t h a t Gaussians do not provide any

positive approximation order, i.e., t h e a p p r o x i m a t i o n process is saturated. T h i s was
 15. Refined and Improved Error Bounds 131

s t u d i e d b y [ B u h m a n n (1989a)] o n i n f i n i t e l a t t i c e s . H o w e v e r , for q u a s i - i n t e r p o l a t i o n
the approximate approximation a p p r o a c h o f M a z ' y a shows t h a t i t is possible t o
choose eo i n such a w a y t h a t t h e level at w h i c h t h e s a t u r a t i o n occurs can be c o n -
t r o l l e d (see, e.g., [ M a z ' y a a n d S c h m i d t ( 1 9 9 6 ) ] ) . T h e r e f o r e , Gaussians m a y v e r y
w e l l be used for s t a t i o n a r y i n t e r p o l a t i o n p r o v i d e d a n a p p r o p r i a t e i n i t i a l shape pa-
rameter is chosen. W e w i l l i l l u s t r a t e t h i s b e h a v i o r i n t h e n e x t c h a p t e r . T h e same
k i n d o f a r g u m e n t also applies t o t h e L a g u e r r e - G a u s s i a n s o f S e c t i o n 4.2.

Example 1 5 . 1 0 . Basis f u n c t i o n s w i t h c o m p a c t s u p p o r t such as t h e W e n d l a n d

functions also do n o t p r o v i d e a n y p o s i t i v e a p p r o x i m a t i o n o r d e r i n t h e station-
ary case. T h i s can be seen b y l o o k i n g at t h e p o w e r f u n c t i o n for t h e scaled ba-

sic f u n c t i o n & h = 3>( h-) w h i c h is o f t h e f o r m P$ ,x(x) Sh = P<$,x e {EHX) where
X h = {eh,x\,... ,ehXN} denotes t h e scaled d a t a set. M o r e o v e r , t h e fill distances o f
the sets X h a n d X satisfy hx e ,JI = hh x SL- Therefore, the power function ( w h i c h
can be b o u n d e d i n t e r m s o f t h e fill distance, c.f. t h e p r o o f o f T h e o r e m 14.5) satisfies

P<b, ,x{<x)
h < C (e h s^J
h x

for some a > 0. T h i s , however, does n o t go t o zero i f eh is chosen as i n ( 1 5 . 1 0 ) .

If, o n t h e o t h e r h a n d , we w o r k i n t h e a p p r o x i m a t e a p p r o x i m a t i o n r e g i m e , t h e n
we can o b t a i n g o o d convergence i n m a n y cases (see t h e n e x t c h a p t e r for some
numerical experiments).

E x a m p l e 1 5 . 1 1 . S t a t i o n a r y i n t e r p o l a t i o n w i t h (inverse) m u l t i q u a d r i c s , r a d i a l p o w -
ers a n d t h i n p l a t e splines presents no difficulties. I n fact, [Schaback (1995c)] shows
t h a t t h e n a t i v e space e r r o r b o u n d for t h i n p l a t e splines a n d r a d i a l powers is i n v a r i a n t
under a s t a t i o n a r y scaling. T h e r e f o r e , t h e n o n - s t a t i o n a r y b o u n d o f T h e o r e m 15.3
applies i n t h e s t a t i o n a r y case also. T h e a d v a n t a g e o f scaling t h i n p l a t e splines or
r a d i a l powers comes f r o m t h e a d d e d s t a b i l i t y one can g a i n b y p r e v e n t i n g t h e sepa-
r a t i o n distance f r o m b e c o m i n g t o o s m a l l (see C h a p t e r 16 a n d t h e w o r k o f Iske o n
local p o l y h a r m o n i c spline a p p r o x i m a t i o n , e.g., [Iske ( 2 0 0 4 ) ] ) .
Y o o n p r o v i d e s e r r o r e s t i m a t e s for s t a t i o n a r y a p p r o x i m a t i o n o f r o u g h f u n c t i o n s
(i.e., f u n c t i o n s t h a t are n o t i n t h e n a t i v e space o f t h e basic f u n c t i o n ) b y so-called
shifted surface splines. S h i f t e d surface splines are o f t h e f o r m
I r / 3 S / 2 1 2 7 2
*M = J (- ) " (1 + I N I ) " - , * odd,
1 } 2 s 2 2 1/2
\ ( - l y - ^ + ^ l + \\x\ f~ l log(l + ||a:|| ) > s even,
where s/2 < 6 G N . These f u n c t i o n s i n c l u d e a l l o f t h e (inverse) m u l t i q u a d r i c s ,
r a d i a l powers a n d t h i n p l a t e splines.
Y o o n has t h e f o l l o w i n g t h e o r e m (see [ Y o o n (2003)] for t h e L p case, a n d [ Y o o n
(2001)] for Loo b o u n d s o n l y ) .

T h e o r e m 15.4. Let $ be a shifted surface spline with shape parameter e inversely

proportional to the fill distance hx,ci- Then there exists a positive constant C (in-
dependent of X) such that for every f in the Sobolev space W - f ( f ) ) D W ^ , ( f i ) we
132 Meshfree Approximation Methods with M A T L A B

have

11/ - V \ \
f L p i a ) < Ch^\f\ , w { R 3 ) , 1 < p < oo,
with

7 p = mm{3,3-s/2 + s/p}.

Furthermore, if f G W?(fl) D w i t / i m a x { 0 , s / 2 - s/p} < a < 3, then

7 P _ / 3 + Q
I I / - ^ / I I M ) = ^ ) -

Y o o n ' s estimates address t h e q u e s t i o n o f h o w w e l l t h e i n f i n i t e l y s m o o t h (inverse)

m u l t i q u a d r i c s a p p r o x i m a t e f u n c t i o n s t h a t are less s m o o t h t h a n those i n t h e i r n a t i v e
space. F o r example, T h e o r e m 15.4 states t h a t / ^ - a p p r o x i m a t i o n t o f u n c t i o n s i n
s 2 2
W | ( f 2 ) , Q C E , b y m u l t i q u a d r i c s $ (x) e = y/l + ||ea;|| is o f t h e o r d e r 0{h ).
However, we emphasize once m o r e t h a t t h i s refers t o s t a t i o n a r y a p p r o x i m a t i o n o f
r o u g h f u n c t i o n s , i.e., is scaled inversely p r o p o r t i o n a l t o t h e f i l l distance a n d /
need n o t lie i n t h e n a t i v e space o f <E>, whereas t h e s p e c t r a l order g i v e n i n (15.2)
corresponds t o a p p r o x i m a t i o n o f f u n c t i o n s i n t h e n a t i v e space i n t h e n o n - s t a t i o n a r y
case w i t h fixed e.
For t h i n p l a t e splines a n d r a d i a l powers t h e a p p r o x i m a t i o n orders i n T h e o -
rem 15.4 are equivalent t o those o f T h e o r e m 15.3 a n d t h e results o f B r o w n l e e a n d
L i g h t m e n t i o n e d above. T h i s is t o be e x p e c t e d due t o t h e i n v a r i a n c e o f these basic
f u n c t i o n s w i t h respect t o scaling.
T h e second p a r t o f Y o o n ' s r e s u l t is a step t o w a r d exact approximation orders as
is t h e w o r k o f [ M a i o r o v (2005)] a n d [ B e j a n c u (1999)] m e n t i o n e d above.

15.5 C o n v e r g e n c e w i t h R e s p e c t to the S h a p e P a r a m e t e r

N o n e o f t h e e r r o r b o u n d s discussed t h u s far have t a k e n i n t o account t h e pos-

s i b i l i t y o f v a r y i n g t h e shape p a r a m e t e r e for a fixed d a t a set X. However, i n
t h e l i t e r a t u r e t h e i n f i n i t e l y s m o o t h basic f u n c t i o n s s u c h as t h e Gaussians a n d
(inverse) m u l t i q u a d r i c s are u s u a l l y f o r m u l a t e d i n c l u d i n g t h e shape p a r a m e t e r e
(or another parameter equivalent t o i t ) and one m a y wonder how a change
i n t h i s shape p a r a m e t e r affects t h e convergence properties of the R B F inter-
polant. I n fact, quite a bit of work has been spent on the quest for the
" o p t i m a l " shape p a r a m e t e r (see, e.g., [ C a r l s o n a n d F o l e y (1991); F o l e y (1994);
H a g a n a n d K a n s a (1994); K a n s a a n d C a r l s o n (1992); R i p p a (1999); T a r w a t e r (1985);
W e r t z et al. ( 2 0 0 6 ) ] ) .
Convergence o f t h e i n f i n i t e l y s m o o t h Gaussians a n d (inverse) m u l t i q u a d r i c s w i t h
respect t o t h e shape p a r a m e t e r was s t u d i e d e a r l y o n i n [ M a d y c h (1991)]. M a d y c h
showed t h a t for these basic f u n c t i o n s t h e r e exists a p o s i t i v e c o n s t a n t A < 1 such
that
l ehx
\f(x) - V/(x)\ < C\ ^ >^ (15.11)
 15. Refined and Improved Error Bounds 133

p r o v i d e d / is i n t h e n a t i v e space o f <E>. T h i s e s t i m a t e shows t h a t t a k i n g e i t h e r t h e

shape p a r a m e t e r e or t h e f i l l d i s t a n c e h ,a x t o zero r e s u l t s i n e x p o n e n t i a l conver-
gence.

15.6 P o l y n o m i a l I n t e r p o l a t i o n as t h e L i m i t of R B F I n t e r p o l a t i o n

Recently, a n u m b e r o f a u t h o r s (see, e.g., [ D r i s c o l l a n d F o r n b e r g ( 2 0 0 2 ) ; F o r n b e r g a n d

F l y e r (2005); F o r n b e r g a n d W r i g h t ( 2 0 0 4 ) ; L a r s s o n a n d F o r n b e r g ( 2 0 0 5 ) ; Schaback
(2005); Schaback ( 2 0 0 6 b ) ] ) have s t u d i e d t h e l i m i t i n g case as e * 0 o f scaled r a d i a l
basis f u n c t i o n i n t e r p o l a t i o n w i t h i n f i n i t e l y s m o o t h basic f u n c t i o n s such as Gaussians
a n d generalized (inverse) m u l t i q u a d r i c s . I t t u r n s o u t t h a t t h e r e is a n i n t e r e s t i n g
connection t o p o l y n o m i a l interpolation.
I n [ D r i s c o l l a n d F o r n b e r g (2002)] u n i v a r i a t e (s 1) i n t e r p o l a t i o n w i t h e-scaled
i n f i n i t e l y s m o o t h r a d i a l basic f u n c t i o n s is s t u d i e d . D r i s c o l l a n d F o r n b e r g s h o w t h a t
the R B F interpolant
N

f( )
V x
= Y, M\\ ( - 3)\\)>
C X X x e
W,b]cM,

t o f u n c t i o n values a t N d i s t i n c t d a t a sites t e n d s t o t h e L a g r a n g e i n t e r p o l a t i n g
p o l y n o m i a l o f / as e 0.
T h e m u l t i v a r i a t e case is m o r e c o m p l i c a t e d . However, the l i m i t i n g R B F inter-
p o l a n t ( p r o v i d e d i t exists) is g i v e n b y a low-degree m u l t i v a r i a t e p o l y n o m i a l (see
[Larsson a n d F o r n b e r g ( 2 0 0 5 ) ; Schaback ( 2 0 0 5 ) ; Schaback ( 2 0 0 6 b ) ] ) . F o r e x a m p l e ,
if t h e d a t a sites are l o c a t e d s u c h t h a t t h e y g u a r a n t e e a u n i q u e p o l y n o m i a l i n t e r -
p o l a n t , t h e n t h e l i m i t i n g R B F i n t e r p o l a n t is g i v e n b y t h i s p o l y n o m i a l . I f p o l y n o -
m i a l i n t e r p o l a t i o n is n o t u n i q u e , t h e n t h e R B F l i m i t depends o n t h e choice o f basic
function. H o w e v e r , these s t a t e m e n t s r e q u i r e t h e R B F s t o satisfy a n (unproven)
c o n d i t i o n o n c e r t a i n coefficient m a t r i c e s A j. Pt I n [ L a r s s o n a n d F o r n b e r g (2005)]
t h e a u t h o r s also p r o v i d e a n e x p l a n a t i o n for t h e e r r o r b e h a v i o r for s m a l l values o f
t h e shape p a r a m e t e r , a n d for t h e existence o f a n o p t i m a l ( p o s i t i v e ) value o f e g i v -
ing rise t o a g l o b a l m i n i m u m o f t h e e r r o r f u n c t i o n . F o r t h e special case o f scaled
Gaussians Schaback [Schaback (2005)] shows t h a t t h e R B F i n t e r p o l a n t converges
t o t h e d e B o o r a n d R o n least p o l y n o m i a l i n t e r p o l a n t (see [ d e B o o r a n d R o n ( 1 9 9 0 ) ;
d e B o o r a n d R o n (1992a)] a n d also [ d e B o o r ( 2 0 0 6 ) ] ) as e - > 0.
I n [ F o r n b e r g a n d W r i g h t (2004)] t h e a u t h o r s describe a so-called Contour-Pade
a l g o r i t h m t h a t makes i t possible (for d a t a sets o f r e l a t i v e l y m o d e s t size) t o c o m p u t e
t h e R B F i n t e r p o l a n t for a l l values o f t h e shape p a r a m e t e r e i n c l u d i n g t h e l i m i t i n g
case e -> 0. W e present some n u m e r i c a l r e s u l t o b t a i n e d w i t h G r a d y W r i g h t ' s
M A T L A B toolbox in Chapter 17.
We w i l l later exploit the connection between R B F and p o l y n o m i a l interpolants
t o design n u m e r i c a l solvers for p a r t i a l d i f f e r e n t i a l e q u a t i o n s .
 Chapter 16

Stability and Trade-Off Principles

16.1 S t a b i l i t y a n d C o n d i t i o n i n g of R a d i a l B a s i s Function
Interpolants

A s t a n d a r d c r i t e r i o n for m e a s u r i n g t h e n u m e r i c a l s t a b i l i t y o f a n approximation
m e t h o d is i t s c o n d i t i o n n u m b e r . I n p a r t i c u l a r , for r a d i a l basis f u n c t i o n i n t e r p o l a t i o n
we need t o l o o k at t h e c o n d i t i o n n u m b e r o f t h e i n t e r p o l a t i o n m a t r i x A w i t h entries
Aij = $(xi Xj). For a n y m a t r i x A i t s -condition
2 number is g i v e n b y

1
cond(A) = | | A | | | | A - | |
2 2 = ^ ,
^min
where c r m a x a n d c r i n are t h e largest a n d smallest s i n g u l a r values o f A.
m I f we
c o n c e n t r a t e o n p o s i t i v e d e f i n i t e m a t r i c e s A, t h e n t h e c o n d i t i o n n u m b e r o f A can
also t a k e be c o m p u t e d as t h e r a t i o
Amax
Amin
of t h e largest a n d smallest eigenvalues.
W h a t do we k n o w a b o u t these eigenvalues? F i r s t , G e r s h g o r i n ' s t h e o r e m (see,
e . g . , [Meyer (2000)]) says t h a t
JV
| Amax An | 5: ^ ^ | Aij |
J= I

for some i { 1 , . . . , N}. Therefore,

Amax < N max \Aij \ = N max \$(xi - Xj)\,

x,j=l,...,N Xi,XjX
w h i c h , since $ is s t r i c t l y p o s i t i v e definite, becomes

A m a x < JV$(0)

b y t h e p r o p e r t i e s o f p o s i t i v e d e f i n i t e f u n c t i o n s ( P r o p e r t y (4) i n T h e o r e m 3.1). N o w ,
s
as l o n g as t h e d a t a are n o t t o o w i l d l y d i s t r i b u t e d , N w i l l g r o w as h x Q which
is acceptable. Therefore, t h e m a i n w o r k i n e s t a b l i s h i n g a b o u n d for t h e c o n d i t i o n
n u m b e r o f A lies i n finding lower b o u n d s for A i m n (or c o r r e s p o n d i n g l y u p p e r b o u n d s

135
136 Meshfree Approximation Methods with M A T L A B

_ 1
for t h e n o r m o f t h e inverse | | A | | 2 ) . T h i s is t h e s u b j e c t o f several papers b y B a l l ,
N a r c o w i c h , S i v a k u m a r a n d W a r d [ B a l l et al. (1992); N a r c o w i c h et al. (1994);
N a r c o w i c h a n d W a r d (1991a); N a r c o w i c h a n d W a r d ( 1 9 9 1 b ) ; N a r c o w i c h a n d W a r d
(1992)] w h o m a k e use o f a r e s u l t f r o m [ B a l l (1992)] o n eigenvalues o f distance
m a t r i c e s . B a l l ' s result follows f r o m t h e Rayleigh quotient (or t h e C o u r a n t - F i s c h e r
T h e o r e m 9.5), w h i c h gives t h e smallest eigenvalue o f a s y m m e t r i c p o s i t i v e d e f i n i t e
m a t r i x as
T
c Ac
Amin = mm .
w 1
cK \0 C C

T h i s can be used t o p r o v e t h e f o l l o w i n g b o u n d for t h e n o r m o f t h e inverse o f A

w h i c h covers also t h e case o f c o n d i t i o n a l p o s i t i v e ( n e g a t i v e ) definiteness o f o r d e r
one.

s s
L e m m a 1 6 . 1 . Let X\,..., x^ be distinct points in R and let <3> : R > R be either
strictly positive definite, or strictly conditionally negative definite of order one with
<&(0) < 0. Also, let A be the interpolation matrix with entries A^ = $>(xi Xj). If
the inequality

N N
2
CiCj-Ay > 9\\c\\ 2

i=l j=l

c =
is satisfied whenever the components of c satisfy X^jLi j > then

l
\\A- \\ <6-\ 2

N o t e t h a t for p o s i t i v e d e f i n i t e m a t r i c e s t h e R a y l e i g h q u o t i e n t i m p l i e s 6 = A i n m

w h i c h shows w h y lower b o u n d s o n t h e smallest eigenvalue c o r r e s p o n d t o u p p e r

b o u n d s o n t h e n o r m o f t h e inverse o f A. I n o r d e r t o o b t a i n t h e b o u n d for con-
d i t i o n a l l y negative definite m a t r i c e s t h e C o u r a n t - F i s c h e r t h e o r e m 9.5 needs t o be
employed.
T h e b o u n d i n L e m m a 16.1 is t o o generic t o give us a n y i n f o r m a t i o n for specific
basic f u n c t i o n s <3>. T h i s e x t e n s i o n was a c c o m p l i s h e d i n some o f t h e o t h e r p a p e r s
m e n t i o n e d above. N a r c o w i c h a n d W a r d e s t a b l i s h b o u n d s o n t h e n o r m o f t h e inverse
of A i n t e r m s o f t h e separation distance o f t h e d a t a sites

qx = \ m i n |jac< - Xj\\ . 2

We c a n p i c t u r e qx as t h e r a d i u s o f t h e largest b a l l t h a t c a n be p l a c e d a r o u n d e v e r y
p o i n t i n X such t h a t no t w o balls o v e r l a p (see F i g u r e 16.1). T h e s e p a r a t i o n d i s t a n c e
is sometimes also referred t o as t h e packing radius. I n our MATLAB code we c a n
c o m p u t e t h e s e p a r a t i o n distance v i a

qX = m i n ( m i n ( D M _ d a t a + e y e ( s i z e ( D M _ d a t a ) ) ) ) / 2
 16. Stability and Trade-Off Principles 137

0 0.2 0.4 0.6 0.8 1

X

Fig. 16.1 The separation distance for N = 25 Halton points (qx ~ 0.0597).

w h e r e DM_data is t h e m a t r i x o f p a i r w i s e distances a m o n g t h e d a t a sites X. The

i d e n t i t y m a t r i x is a d d e d o n l y t o a v o i d c o u n t i n g t h e d i s t a n c e o f a p o i n t Xj t o i t s e l f
as a p o t e n t i a l m i n i m u m .
T h e d e r i v a t i o n o f these b o u n d s is r a t h e r t e c h n i c a l , a n d for details we refer t o
e i t h e r t h e o r i g i n a l papers b y N a r c o w i c h , W a r d a n d co-workers l i s t e d above, the
m o r e recent p a p e r [Schaback ( 2 0 0 2 ) ] , or t h e b o o k [ W e n d l a n d (2005a)]. B y p r o v i d i n g
_ 1
m a t c h i n g lower b o u n d s for | | A | | 2 (i.e., upper b o u n d s for A i ) m n Schaback s h o w e d
t h a t t h e u p p e r b o u n d s o n t h e n o r m o f t h e inverse o b t a i n e d b y N a r c o w i c h , W a r d
a n d o t h e r s are near o p t i m a l (see [Schaback ( 1 9 9 4 b ) ] ) .
W e n o w l i s t several ( l o w e r ) b o u n d s for A i m n as d e r i v e d i n [ W e n d l a n d ( 2 0 0 5 a ) ] .
I n t h e examples b e l o w t h e e x p l i c i t (space-dependent) c o n s t a n t s

M , = 1 2 ( L i i j < 6.38s and C. = ^ ( ^ j

are used. T h e u p p e r b o u n d for M s c a n be o b t a i n e d u s i n g S t i r l i n g ' s f o r m u l a (see

[ W e n d l a n d (2005a)]).
Since t h e b o u n d s i n t h e l i t e r a t u r e for Gaussians a n d m u l t i q u a d r i c s also i n c l u d e
t h e influence o f t h e shape p a r a m e t e r e we present t h e basic f u n c t i o n s i n t h e i r scaled
v e r s i o n here.

e 2 2
Example 16.1. For Gaussians &(x) = e~ ^ one o b t a i n s

s 4 0 7 1 s 2
A m i n > C ( v ^ ) - e -
s - / ^ V -
W e see t h a t , for a fixed shape p a r a m e t e r e, t h e lower b o u n d for A i m n goes e x p o -
n e n t i a l l y t o zero as t h e s e p a r a t i o n d i s t a n c e qx decreases. Since we observed above
t h a t t h e c o n d i t i o n n u m b e r o f t h e i n t e r p o l a t i o n m a t r i x A depends o n t h e r a t i o o f
its largest a n d smallest eigenvalues a n d t h e g r o w t h o f A m a x is o f o r d e r N we see
t h a t t h e c o n d i t i o n n u m b e r g r o w s e x p o n e n t i a l l y w i t h decreasing s e p a r a t i o n d i s t a n c e .
T h i s shows t h a t , i f one adds m o r e i n t e r p o l a t i o n p o i n t s i n o r d e r t o i m p r o v e t h e ac-
c u r a c y o f t h e i n t e r p o l a n t ( w i t h i n t h e same d o m a i n fl), t h e n t h e p r o b l e m becomes
138 Meshfree Approximation Methods with M A T L A B

increasingly ill-conditioned. O f course one w o u l d a l w a y s expect t h i s t o h a p p e n ,

b u t here t h e i l l - c o n d i t i o n i n g grows p r i m a r i l y d u e t o t h e decrease i n t h e s e p a r a t i o n
d i s t a n c e qx, a n d n o t t o t h e increase i n t h e n u m b e r N o f d a t a p o i n t s . W e w i l l come
b a c k t o t h i s o b s e r v a t i o n w h e n w e discuss a possible change o f basis i n S e c t i o n 34.4.
O n t h e o t h e r h a n d , i f one keeps t h e n u m b e r o f p o i n t s ( o r a t least t h e s e p a r a t i o n
distance) fixed a n d i n s t e a d decreases t h e v a l u e o f e, t h e n t h e c o n d i t i o n n u m b e r o f
A suffers i n a l m o s t t h e same e x p o n e n t i a l m a n n e r . O f course, a n increase i n e c a n
be used t o i m p r o v e t h e c o n d i t i o n n u m b e r o f A (however, as we saw earlier, a t t h e
expense o f a c c u r a c y o f t h e f i t ) .

Example 16.2. F o r scaled generalized (inverse) m u l t i q u a d r i c s <$>(x) =

2
(l + I M I ) ^ , 8 e R \ N 0 one o b t a i n s

i + i 2 M e
A m i n > C(a, A e ) 4 " e- '/<* )

w i t h another explicitly k n o w n constant C(s,0,e).

T h e same c o m m e n t s as i n t h e p r e v i o u s e x a m p l e a p p l y .

/ 3 + 1 2 / 3
E x a m p l e 1 6 . 3 . F o r t h i n p l a t e splines $(cc) = ( - l ) ||cc|| l o g \\x\\, 0 G N , one
obtains
s 2
A m i n > C s C / 3 (2M )- - ^^s

w i t h another explicitly k n o w n constant c p .

I n t h i s case t h e lower b o u n d also goes t o zero w i t h decreasing s e p a r a t i o n distance.
However t h e decay is o n l y o f p o l y n o m i a l o r d e r .

2 p
E x a m p l e 1 6 . 4 . F o r t h e r a d i a l powers $ ( J C ) = ( - l ) ^ / ! \\x\\ , 0 < Q 2 N , one
obtains
s / 3
A m i n > C c s / 3 (2M )- - s 4

w i t h a n o t h e r e x p l i c i t l y k n o w n c o n s t a n t c p (different f r o m c p i n E x a m p l e 3 ) . A g a i n ,
t h e decay is o f p o l y n o m i a l order.

E x a m p l e 1 6 . 5 . F o r t h e c o m p a c t l y s u p p o r t e d f u n c t i o n s o f W e n d l a n d $> k(x) Sj
(p k(||x||)
St one o b t a i n s
k+1
A m i n > C(s,k)q%

w i t h a c o n s t a n t C(s, k) d e p e n d i n g o n s a n d k. T h e lower b o u n d goes t o zero w i t h

t h e s e p a r a t i o n distance a t a p o l y n o m i a l r a t e .

16.2 Trade-Off Principle I : Accuracy vs. Stability

T h e observations m a d e i n E x a m p l e s 16.1 a n d 16.2 above set u p t h e first trade-

off principle. T h i s p r i n c i p l e states t h a t i f w e use t h e s t a n d a r d a p p r o a c h t o t h e
R B F i n t e r p o l a t i o n p r o b l e m (i.e., s o l u t i o n o f t h e l i n e a r s y s t e m (6.3)) t h e n t h e r e is
 16. Stability and Trade-Off Principles 139

a conflict between t h e o r e t i c a l l y achievable a c c u r a c y a n d n u m e r i c a l s t a b i l i t y . For

example, t h e e r r o r b o u n d s for n o n - s t a t i o n a r y i n t e r p o l a t i o n u s i n g i n f i n i t e l y s m o o t h
basis f u n c t i o n s show t h a t t h e e r r o r decreases ( e x p o n e n t i a l l y ) as t h e fill distance
decreases. For w e l l - d i s t r i b u t e d d a t a a decrease i n t h e fill distance also i m p l i e s a
decrease o f t h e s e p a r a t i o n distance. B u t n o w t h e c o n d i t i o n e s t i m a t e s o f t h e p r e v i o u s
subsection i m p l y t h a t t h e c o n d i t i o n n u m b e r o f A g r o w s e x p o n e n t i a l l y . T h i s leads
to numerical instabilities w h i c h make i t v i r t u a l l y impossible to o b t a i n the h i g h l y
accurate results p r o m i s e d b y t h e t h e o r e t i c a l e r r o r b o u n d s .
S i m i l a r l y , i f we use t h e shape p a r a m e t e r t o ( e x p o n e n t i a l l y ) increase a c c u r a c y
as g u a r a n t e e d b y M a d y c h ' s e r r o r b o u n d (15.11), t h e n t h e c o n d i t i o n n u m b e r a g a i n
grows e x p o n e n t i a l l y . T h i s is t o be e x p e c t e d since for s m a l l values o f e t h e basic
functions m o r e a n d m o r e resemble a c o n s t a n t f u n c t i o n , a n d therefore t h e rows (as
w e l l as c o l u m n s ) o f t h e m a t r i x A become m o r e a n d m o r e a l i k e , so t h a t t h e m a t r i x
becomes a l m o s t s i n g u l a r even for w e l l s e p a r a t e d d a t a sites.
I n t h e l i t e r a t u r e t h i s p h e n o m e n o n has been referred t o as trade-off or [uncer-
tainty) principle (see, e.g., t h e papers [Schaback ( 1 9 9 5 b ) ; Schaback ( 1 9 9 5 c ) ] ) .
Schaback l o o k e d at t h e p o w e r f u n c t i o n P<&,x a n d showed t h a t i t c a n always be
b o u n d e d f r o m above b y a f u n c t i o n F$> d e p e n d i n g o n t h e fill distance. O n t h e o t h e r
h a n d , he showed t h a t t h e R a y l e i g h q u o t i e n t c a n always be b o u n d e d f r o m b e l o w b y a
f u n c t i o n G$ d e p e n d i n g o n t h e s e p a r a t i o n distance. F u r t h e r m o r e , Schaback showed
that

G*(q )x < F*(h ,n),

x

a n d therefore, for w e l l - d i s t r i b u t e d d a t a ( w i t h q x ~ hx,n), a small error b o u n d (i.e.,

s m a l l F$(hx,n.)) w i l l necessarily r e s u l t i n a s m a l l lower b o u n d (i.e., s m a l l G<f>(qx))
for t h e R a y l e i g h q u o t i e n t , a n d therefore for t h e smallest eigenvalue. T h i s however
i m p l i e s a large c o n d i t i o n n u m b e r .
We have seen evidence o f t h e first t r a d e - o f f p r i n c i p l e i n v a r i o u s n u m e r i c a l ex-
periments. T h i s t r a d e - o f f has l e d a n u m b e r o f people t o search for a n " o p t i m a l "
value o f t h e shape p a r a m e t e r , i.e., a value t h a t yields m a x i m a l accuracy, w h i l e s t i l l
maintaining numerical stability.
I n p a r t i c u l a r , m u l t i q u a d r i c s have a t t r a c t e d t h e best p a r t o f t h i s a t t e n t i o n . F o r
2
e x a m p l e , i n his o r i g i n a l w o r k o n (inverse) m u l t i q u a d r i c i n t e r p o l a t i o n i n M Hardy
[ H a r d y (1971)] suggested u s i n g e = 1 / ( 0 . 8 1 5 d ) , w h e r e d = j^J2iL\di, a n d di is
t h e distance f r o m Xi t o i t s nearest n e i g h b o r . L a t e r , i n [Franke (1982a)], one c a n
8
find t h e r e c o m m e n d e d value e = ' ^ , w h e r e D is t h e d i a m e t e r o f t h e smallest
circle c o n t a i n i n g a l l d a t a p o i n t s . A n o t h e r s t r a t e g y for f i n d i n g a g o o d value for e is
based o n t h e o b s e r v a t i o n t h a t such a value seems t o be s i m i l a r for m u l t i q u a d r i c s a n d
inverse m u l t i q u a d r i c s (see [Foley ( 1 9 9 4 ) ] ) . O t h e r studies were r e p o r t e d i n [ C a r l s o n
a n d Foley (1992); C a r l s o n a n d N a t a r a j a n (1994)]. W e w i l l consider a m o r e recent
a l g o r i t h m p r o p o s e d i n [ R i p p a (1999)] i n t h e n e x t chapter.
140 Meshfree Approximation Methods with M A T L A B

16.3 Trade-Off Principle I I : A c c u r a c y a n d Stability vs. Problem

Size

M o r e recently, F o r n b e r g a n d co-workers h a v e i n v e s t i g a t e d t h e dependence o f t h e

s t a b i l i t y o n t h e values o f t h e shape p a r a m e t e r e i n a series o f p a p e r s (e.g., [Driscoll
a n d F o r n b e r g (2002); F o r n b e r g a n d W r i g h t ( 2 0 0 4 ) ; L a r s s o n a n d F o r n b e r g (2005);
P l a t t e a n d D r i s c o l l ( 2 0 0 5 ) ] ) . T h e y suggest a w a y t o s t a b l y c o m p u t e v e r y a c c u r a t e
generalized (inverse) m u l t i q u a d r i c a n d G a u s s i a n i n t e r p o l a n t s w i t h e x t r e m e values
o f e 0 b y u s i n g a c o m p l e x C o n t o u r - P a d e i n t e g r a t i o n a l g o r i t h m . T h u s , t h i s ap-
p r o a c h allows us t o overcome t h e first t r a d e - o f f p r i n c i p l e m e n t i o n e d i n t h e p r e v i o u s
section. H o w e v e r , t h e r e is a n o t h e r k i n d o f t r a d e - o f f associated w i t h t h e C o n t o u r -
P a d e a p p r o a c h . N a m e l y i t is l i m i t e d t o o n l y r a t h e r s m a l l d a t a sets ( r o u g h l y A" = 20
for s = 1 a n d N = 80 for s = 2 ) .
I n spite o f these l i m i t a t i o n s t h e C o n t o u r - P a d e a l g o r i t h m has been used t o g a i n
a n u m b e r o f t h e o r e t i c a l i n s i g h t s such as t h e c o n n e c t i o n b e t w e e n R B F i n t e r p o l a t i o n
a n d p o l y n o m i a l i n t e r p o l a t i o n m e n t i o n e d i n S e c t i o n 15.6. W e present some n u m e r i c a l
e x p e r i m e n t s based o n t h e C o n t o u r - P a d e a p p r o a c h i n t h e n e x t c h a p t e r .

16.4 Trade-Off Principle I I I : A c c u r a c y vs. Efficiency

T h e r e is also a t r a d e - o f f p r i n c i p l e for c o m p a c t l y s u p p o r t e d f u n c t i o n s . T h i s was ex-

p l a i n e d t h e o r e t i c a l l y as w e l l as i l l u s t r a t e d w i t h n u m e r i c a l e x p e r i m e n t s i n [Schaback
( 1 9 9 7 b ) ] . T h e consequences are as follows. I n t h e case o f s t a t i o n a r y i n t e r p o l a t i o n ,
i.e., i f we scale t h e s u p p o r t size o f t h e basis f u n c t i o n s p r o p o r t i o n a l t o t h e f i l l dis-
t a n c e hx,n, t h e " b a n d w i d t h " o f t h e i n t e r p o l a t i o n m a t r i x A is k e p t c o n s t a n t . T h i s
means we c a n a p p l y n u m e r i c a l a l g o r i t h m s (e.g., t h e c o n j u g a t e g r a d i e n t m e t h o d ) for
t h e s o l u t i o n o f t h e i n t e r p o l a t i o n s y s t e m t h a t c a n be p e r f o r m e d w i t h 0(N) compu-
t a t i o n a l c o m p l e x i t y . T h e m e t h o d is n u m e r i c a l l y stable, b u t t h e r e w i l l be essentially
n o convergence (see o u r earlier n u m e r i c a l e x p e r i m e n t s i n T a b l e 12.1). I n t h e n o n -
s t a t i o n a r y case, i.e., w i t h f i x e d s u p p o r t size, t h e b a n d w i d t h o f A increases as hx,n
decreases. T h i s results i n convergence (i.e., t h e e r r o r decreases) as we s h o w e d w i t h
o u r e x p e r i m e n t s i n T a b l e 12.2 a n d t h e e r r o r b o u n d s i n S e c t i o n 15.1.2. However,
t h e i n t e r p o l a t i o n m a t r i c e s w i l l b e c o m e m o r e a n d m o r e densely p o p u l a t e d as w e l l as
i l l - c o n d i t i o n e d . T h e r e f o r e , t h i s a p p r o a c h is n o t v e r y efficient.
 Chapter 17

Numerical Evidence for Approximation

Order Results

17.1 I n t e r p o l a t i o n for e 0

W e b e g i n b y c o n s i d e r i n g t h e choice o f t h e shape p a r a m e t e r for a fixed d a t a set.

T h i s is p r o b a b l y t h e s i t u a t i o n t h a t w i l l arise m o s t f r e q u e n t l y i n p r a c t i c a l s i t u a t i o n s .
I n o t h e r w o r d s , we assume we are g i v e n a set o f d a t a (xj, fj), j 1 , . . . , N, with
s
d a t a sites Xj R ( w i t h s 1 or s 2 for t h e p u r p o s e o f o u r e x p e r i m e n t s ) , and
f u n c t i o n values fj f{xj) e R . O u r g o a l is t o use a n R B F i n t e r p o l a n t
N
v x c x x
f( ) = Yl w(W - i\\)
3= 1
t o m a t c h these d a t a e x a c t l y , i.e., t o satisfy Vf(xi) = f(xi), i = 1 , . . . , N. The two
m o s t i m p o r t a n t questions n o w seem t o be:

W h i c h basic f u n c t i o n <p s h o u l d we use?

H o w s h o u l d we scale t h e basis f u n c t i o n s (fj </?(|| Xj\\)?

T h e e r r o r b o u n d s we r e v i e w e d i n p r e v i o u s c h a p t e r s give us some i n s i g h t i n t o
t h e first issue. I f we k n o w t h a t t h e d a t a come f r o m a v e r y s m o o t h f u n c t i o n , t h e n
a p p l i c a t i o n o f one o f t h e s m o o t h e r basic f u n c t i o n s is called for. O t h e r w i s e , t h e r e
is n o t m u c h t o be g a i n e d f r o m d o i n g so. I n fact, these f u n c t i o n s m a y a d d t o o
m u c h smoothness t o t h e i n t e r p o l a n t . A first a t t e m p t at p r o v i d i n g guidelines for t h e
selection o f a p p r o p r i a t e basic f u n c t i o n s (or kernels) c a n be f o u n d i n [Schaback a n d
W e n d l a n d (2006)]. W e w i l l n o t p u r s u e t h i s issue a n y f u r t h e r .
I n s t e a d we w a n t t o focus o u r a t t e n t i o n o n t h e second q u e s t i o n , i.e., t h e choice o f
t h e shape p a r a m e t e r e. A n u m b e r o f strategies c a n be used t o g u i d e us i n m a k i n g a
decision. W e w i l l assume t h r o u g h o u t t h a t a (fixed) basic f u n c t i o n has been chosen,
a n d t h a t we w i l l use o n l y one value t o scale a l l basis f u n c t i o n s u n i f o r m l y . Clearly,
one can also follow o t h e r strategies such as u s i n g a shape p a r a m e t e r t h a t varies
w i t h j , or even basic f u n c t i o n s t h a t v a r y w i t h j . W h i l e some w o r k has b e e n d o n e
i n these d i r e c t i o n s (see, e.g., [ B o z z i n i et al. ( 2 0 0 2 ) ; K a n s a a n d C a r l s o n (1992);
Schaback a n d W e n d l a n d ( 2 0 0 0 b ) ; F o r n b e r g a n d Z u e v ( 2 0 0 6 ) ] ) , n o t m u c h c o n c r e t e
c a n be said i n these cases.

141
142 Meshfree Approximation Methods with M A T L A B

W e n o w discuss four strategies for c h o o s i n g a " g o o d " value o f s.

17.1.1 Choosing a Good Shape Parameter via Trial and Error

T h e s i m p l e s t s t r a t e g y is t o p e r f o r m a series o f i n t e r p o l a t i o n e x p e r i m e n t s w i t h v a r y -
i n g shape p a r a m e t e r , a n d t h e n t o p i c k t h e "best" one. T h i s s t r a t e g y c a n be used i f
we k n o w t h e f u n c t i o n / t h a t g e n e r a t e d t h e d a t a , a n d t h e r e f o r e c a n c a l c u l a t e some
s o r t o f e r r o r for t h e i n t e r p o l a n t . O f course, i f we a l r e a d y k n o w / , t h e n t h e exercise
o f f i n d i n g a n i n t e r p o l a n t Vf m a y be m o s t l y p o i n t l e s s . H o w e v e r , t h i s is t h e s t r a t e g y
we used for t h e "academic" examples i n C h a p t e r 2.
I f we do n o t have a n y k n o w l e d g e o f / , t h e n i t becomes v e r y d i f f i c u l t t o decide
w h a t "best" means. O n e ( n o n - o p t i m a l ) c r i t e r i o n w e used i n C h a p t e r 2 was based
o n t h e t r a d e - o f f p r i n c i p l e , i.e., t h e fact t h a t for s m a l l e t h e e r r o r i m p r o v e s w h i l e
t h e c o n d i t i o n n u m b e r g r o w s . W e t h e n defined "best" t o be t h e smallest e for w h i c h
M A T L A B d i d n o t issue a n e a r - s i n g u l a r w a r n i n g .
I n m a n y cases selection o f a n o p t i m a l shape p a r a m e t e r v i a trial and error will
e n d u p b e i n g a r a t h e r s u b j e c t i v e process. H o w e v e r , t h i s m a y p r e s e n t l y be the
approach taken b y most practitioners.
For c o m p a r i s o n w i t h t h e o t h e r s e l e c t i o n m e t h o d s f e a t u r e d b e l o w we present t h r e e
o n e - d i m e n s i o n a l test cases for w h i c h w e k n o w t h e d a t a f u n c t i o n / . W e use

Fi(x) = sine (a;),

2 2 2 2
JP ( )2 X = | ^ -(9x-2) /4
e + e -(9x+l) /49^ + ^ ~(Qx-7) /4
e _ _Lg-(9x-4) ^

F (x)
3 = ( l - \x - \ \ j (\ + 5\x - \ \ - 27\x - i | 2 ) .

T h e first o f these f u n c t i o n s is t h e classical b a n d l i m i t e d f u n c t i o n ( a n d t h u s i n t h e

n a t i v e space o f Gaussians). T h e second f u n c t i o n is a o n e - d i m e n s i o n a l v a r i a n t o f
2
Franke's f u n c t i o n , a n d t h e t h i r d f u n c t i o n is one o f G n e i t i n g ' s C oscillatory com-
p a c t l y s u p p o r t e d R B F s s h i f t e d t o t h e p o i n t ( 1 / 2 , 1 / 2 ) (see T a b l e 11.4).
For these f u n c t i o n s w e l i s t m a x i m u m e r r o r s a n d o p t i m a l shape p a r a m e t e r s e i n
T a b l e 1 7 . 1 . M a x i m u m e r r o r s for a l a r g e r a n g e o f e values a n d t h e different values
o f N used i n T a b l e 17.1 are d i s p l a y e d i n F i g u r e 1 7 . 1 . T h e o p t i m a l e values l i s t e d
i n T a b l e 17.1 corresponds t o t h e lowest p o i n t for each o f t h e curves i n t h e figure.
Clearly, t h e o p t i m a l value o f t h e shape p a r a m e t e r is s t r o n g l y d e p e n d e n t on the
function t h a t generated the test data.

17.1.2 The Power Function as Indicator for a Good Shape

Parameter

A n o t h e r s t r a t e g y is suggested b y t h e e r r o r a n a l y s i s o f C h a p t e r 14. W e s h o w e d t h e r e
i n T h e o r e m 14.2 t h a t

\f(x)-Vf{x)\<P* (x)\\f\\ ,
tX Wn)
 17. Numerical Evidence for Approximation Order Results 143

Table 17.1 Optimal e values based on Gaussian interpolation to various test func-
tions in I D for various choices of N uniform points.

Fi F 2 F 3

N max-error optimal max-error optimal e max-error optimal e

3 2.1403e-003 1.12 4.9722e-001 2.20 3.7087e-002 5.68

5 2.3260e-005 0.68 6.9380e-002 5.44 2.5253e-002 5.20
9 4.8764e-009 0.64 1.7555e-002 5.20 2.5360e-003 8.84
17 1.8922e-010 1.00 2.1928e-004 5.80 1.4380e-003 9.52
33 1.5250e-010 2.04 1.6536e-007 6.08 3.4189e-004 13.24
65 4.1307e-010 2.04 3.6260e-009 7.48 8.6431e-005 21.84

Fig. 17.1 Optimal e curves based on Gaussian interpolation in I D for various choices of N uniform
2
points. Data sampled from sine function Fx (left) and C oscillatory function F3 (right).

where P,x denotes t h e p o w e r f u n c t i o n . T h i s e s t i m a t e decouples t h e i n t e r p o l a t i o n

e r r o r i n t o a c o m p o n e n t i n d e p e n d e n t o f t h e d a t a f u n c t i o n / a n d one d e p e n d i n g o n
/. Once we have decided o n a basic f u n c t i o n <> a n d a d a t a set X we c a n use t h e
power f u n c t i o n based o n scaled versions o f <& t o o p t i m i z e t h e e r r o r c o m p o n e n t t h a t
is i n d e p e n d e n t o f / . ' W h i l e t h i s a p p r o a c h has t h e a d v a n t a g e over t h e p r e v i o u s l y
m e n t i o n e d t r i a l a n d e r r o r a p p r o a c h t h a t i t is o b j e c t i v e a n d does n o t d e p e n d o n
a n y k n o w l e d g e o f t h e d a t a f u n c t i o n , u n f o r t u n a t e l y , t h i s a p p r o a c h w i l l n o t be a n
o p t i m a l one since t h e second c o m p o n e n t o f t h e e r r o r b o u n d also d e p e n d s o n t h e
basic f u n c t i o n v i a t h e n a t i v e space n o r m ( w h i c h changes w h e n <I> is scaled).
W e said earlier (see ( 1 4 . 4 ) ) t h a t t h e p o w e r f u n c t i o n c a n be c o m p u t e d v i a

1
P*,x{x) = ^ ( x , ^ ) - ^ ) ) ^ - ^ ) ,

T
where A is t h e i n t e r p o l a t i o n m a t r i x a n d 6 = [ $ ( - , x i ) , . . . , $ ( - , C j v ) ] . T h i s f o r m u l a
is i m p l e m e n t e d o n lines 1 5 - 1 8 i n t h e M A T L A B p r o g r a m P o w e r f u n c t i o n 2 D .m. W e
c o m p u t e t h e inverse o f A u s i n g t h e f u n c t i o n p i n v w h i c h is based o n t h e s i n g u l a r
value d e c o m p o s i t i o n o f A a n d t h e r e f o r e g u a r a n t e e s m a x i m u m s t a b i l i t y . A l s o , due t o
r o u n d o f f some o f t h e a r g u m e n t s o f t h e s q r t f u n c t i o n o n l i n e 18 come o u t n e g a t i v e .
144 Meshfree Approximation Methods with M A T L A B

This explains the use of the r e a l command. The vectors b(x) are just rows of the
evaluation matrix if x is taken from the grid of evaluation points we used earlier
for error computations and plotting purposes. Except for the loop over the shape
parameter e (lines 12-20) the rest of the program is similar to earlier code.

P r o g r a m 1 7 . 1 . Powerfunction2D.m

% Powerfunction2D
S c r i p t t h a t f i n d s "optimal" shape parameter by computing t h e power
'/, f u n c t i o n f o r the 2D RBF i n t e r p o l a t i o n approach w i t h v a r y i n g e p s i l o n
/, C a l l s on: DistanceMatrix
1 r b f = @(e,r) e x p ( - ( e * r ) . ~ 2 ) ; 7. Define t h e Gaussian RBF
% Parameters f o r shape parameter loop below
2 mine = 0; maxe = 20;
3 ne = 500; ep = linspace(mine,maxe,ne);
'/, Number and type of data p o i n t s
4 N = 81; g r i d t y p e = 'u';
7, R e s o l u t i o n of g r i d f o r power f u n c t i o n norm computation
5 n e v a l = 2 0 ; M = neval"2;
% Load data p o i n t s
, ,
6 name = s p r i n t f ( Data2D_7od7oS ,N,gridtype) ; load(name)
7 ctrs = dsites; % c e n t e r s c o i n c i d e w i t h data s i t e s
8 g r i d = l i n s p a c e ( 0 , 1 , n e v a l ) ; [xe,ye] = m e s h g r i d ( g r i d ) ;
9 epoints = [ x e ( : ) y e ( : ) ] ;
7o Compute d i s t a n c e m a t r i x between e v a l u a t i o n p o i n t s and c e n t e r s
10 DM_eval = D i s t a n c e M a t r i x ( e p o i n t s , c t r s ) ;
7. Compute d i s t a n c e m a t r i x between t h e data s i t e s and c e n t e r s
11 DM_data = D i s t a n c e M a t r i x ( d s i t e s , c t r s ) ;
12 for i=l:length(ep)
7, Compute i n t e r p o l a t i o n m a t r i x
13 IM = r b f ( e p ( i ) , D M _ d a t a ) ;
7o Compute e v a l u a t i o n matrix
14 EM = r b f ( e p ( i ) , D M _ e v a l ) ;
7o Compute power f u n c t i o n a t e v a l u a t i o n p o i n t s
15 invIM = i n v ( I M ) ; phiO = r b f ( e p ( i ) , 0 ) ;
16 f o r j=l:M
17 powfun(j) = r e a l ( s q r t ( p h i O - ( i n v I M * E M ( j , : ) ' ) ' * E M ( j , : ) ' ) ) ;
18 end
7o Compute max. norm of power f u n c t i o n on e v a l u a t i o n g r i d
19 maxPF(i) = max(powfun);
20 end
21 f p r i n t f (' S m a l l e s t maximum norm: 7oe\n' , min(maxPF))
22 f p r i n t f ('at e p s i l o n = 7.f \n' , ep(maxPF==min(maxPF) ) )

&4
 17. Numerical Evidence for Approximation Order Results 145

23 f p r i n t f ( ' w i t h cond (A) = 7 e \ n ' , ...

condest(rbf(ep(find(maxPF==min(maxPF))),DM_data)))
7, P l o t p o w e r f u n c t i o n n o r m
24 figure; semilogy(ep,maxPF,'b');

I n F i g u r e 17.2 w e show p l o t s o f t h e m a x i m u m n o r m s o f t h e p o w e r function

vs. e for a o n e - d i m e n s i o n a l experiment (left) a n d a 2 D e x p e r i m e n t (right). Each
p l o t shows several curves c o r r e s p o n d i n g t o different choices o f N (set o n l i n e 4 o f
P o w e r f u n t i o n 2 D . m ) . T h e o p t i m a l e values a l o n g w i t h t h e c o r r e s p o n d i n g c o n d i t i o n
numbers of the interpolation m a t r i x (computed using the c o n d e s t command) are
l i s t e d i n T a b l e 17.2. T h e g r a p h s o f t h e m a x i m u m n o r m o f t h e p o w e r f u n c t i o n c a n
a l l be i n c l u d e d i n a single p l o t b y a d d i n g a n o t h e r l o o p t o t h e p r o g r a m w h i c h varies
N. T h e p r o g r a m for t h e o n e - d i m e n s i o n a l case is a l m o s t i d e n t i c a l t o t h e one p r i n t e d
a n d therefore o m i t t e d .

0 5 10 15 20 0 5 10 15 20
e e

Fig. 17.2 Optimal e curves based on power functions for Gaussians in I D (left) and 2D (right)
for various choices of N uniform points.

Clearly, even for t h e s m a l l d a t a sets considered here, t h e n u m e r i c a l i n s t a b i l i t y ,

i.e., large c o n d i t i o n n u m b e r o f t h e i n t e r p o l a t i o n m a t r i x A, plays a s i g n i f i c a n t r o l e .

Table 17.2 Optimal e values based on power functions for Gaus-

sians in I D and 2D for various choices of N uniform points.

ID 2D
N optimal e cond(A) N optimal e cond(A)

3 0.04 1.8749e+007 9 0.16 5.3534e+009

5 0.44 5.7658e+007 25 0.84 1.0211e+011
9 1.72 6.5682e+008 81 0.04 2.0734e+019
17 4.48 6.1306e+009 289 0.56 1.2194e+020
33 9.60 5.4579e+010
65 19.52 1.2440e+011
146 Meshfree Approximation Methods with M A T L A B

17.1.3 Choosing a Good Shape Parameter via Cross Validation

A t h i r d s t r a t e g y for f i n d i n g a n " o p t i m a l " shape p a r a m e t e r is t o use a cross valida-

tion a p p r o a c h . I n [ R i p p a (1999)] a n a l g o r i t h m is d e s c r i b e d t h a t corresponds t o a
v a r i a n t o f cross v a l i d a t i o n k n o w n as "leave-one-out" cross validation. This method
is r a t h e r p o p u l a r i n t h e statistics l i t e r a t u r e w h e r e i t is also k n o w n as P R E S S (Pre-
d i c t i v e R E s i d u a l S u m o f Squares) p r o v i d e d t h e 2 - n o r m is used. I n t h i s a l g o r i t h m
a n " o p t i m a l " value o f e is selected b y m i n i m i z i n g t h e (least squares) e r r o r for a f i t
t o t h e d a t a based o n an i n t e r p o l a n t for w h i c h one o f t h e centers was "left o u t " . A
m a j o r advantage over t h e p r e v i o u s m e t h o d is t h a t n o w t h e dependence o f t h e e r r o r
on t h e d a t a f u n c t i o n is also t a k e n i n t o a c c o u n t . T h e r e f o r e , t h e p r e d i c t e d " o p t i m a l "
shape p a r a m e t e r is closer t o t h e one we f o u n d v i a t h e t r i a l a n d e r r o r a p p r o a c h (for
w h i c h we h a d t o assume k n o w l e d g e o f t h e exact s o l u t i o n ) .
A s i m i l a r s t r a t e g y was p r o p o s e d earlier i n [ G o l b e r g et al. (1996)] for t h e s o l u t i o n
of e l l i p t i c p a r t i a l differential e q u a t i o n s v i a t h e d u a l r e c i p r o c i t y m e t h o d based o n
multiquadric interpolation.
Specifically, i f Vf is t h e r a d i a l basis function interpolant to the data
{fi, , fk-i, fk+i, , / A T } , i-e-,

p}k*) = i>?V(ii*-*jii),
cf

such t h a t

Vfixi) = f it i = 1 , . . . , k - 1, k + 1 , . . . , N,

a n d i f Ek is t h e e r r o r

E k = fk-Vf{x )
k

at t h e one p o i n t Xk n o t used t o d e t e r m i n e t h e i n t e r p o l a n t , t h e n t h e q u a l i t y o f t h e
T
fit is d e t e r m i n e d b y t h e n o r m o f t h e v e c t o r o f errors E \E\,..., Ejy} obtained
by r e m o v i n g i n t u r n one o f t h e d a t a p o i n t s a n d c o m p a r i n g t h e r e s u l t i n g f i t w i t h
t h e ( k n o w n ) value at t h e r e m o v e d p o i n t . I n [ R i p p a (1999)] t h e a u t h o r presented
examples based o n use o f t h e l\ and i 2 n o r m s . W e w i l l m o s t l y use t h e m a x i m u m
n o r m (see line 14 i n t h e code b e l o w ) .
B y a d d i n g a l o o p over e we c a n c o m p a r e t h e e r r o r n o r m s for different values o f
t h e shape p a r a m e t e r , a n d choose t h a t value o f e t h a t y i e l d s t h e m i n i m a l e r r o r n o r m
as t h e o p t i m a l one.
W h i l e a naive i m p l e m e n t a t i o n o f t h e leave-one-out a l g o r i t h m is r a t h e r expensive
4
(on t h e order o f N o p e r a t i o n s ) , R i p p a showed t h a t t h e c o m p u t a t i o n o f t h e e r r o r
c o m p o n e n t s can be s i m p l i f i e d t o a single f o r m u l a

Ek = (17-1)
A
kk
w h e r e Ck is t h e A;th coefficient i n t h e i n t e r p o l a n t Vf based o n t h e full d a t a set, a n d
is t h e kth d i a g o n a l element o f t h e inverse o f t h e c o r r e s p o n d i n g i n t e r p o l a t i o n
 17. Numerical Evidence for Approximation Order Results 147

m a t r i x . Since b o t h Cfc a n d A need t o be c o m p u t e d o n l y once for each value o f s

3
t h i s results i n 0(N ) c o m p u t a t i o n a l c o m p l e x i t y . N o t e t h a t a l l entries i n t h e e r r o r
vector E c a n be c o m p u t e d i n a single s t a t e m e n t i n M A T L A B i f we v e c t o r i z e t h e
c o m p o n e n t f o r m u l a (17.1) (see l i n e 13 i n P r o g r a m 17.2). T h e sine f u n c t i o n used
on line 5 is n o t a s t a n d a r d M A T L A B f u n c t i o n ( i t is p a r t o f t h e S i g n a l Processing
T o o l b o x ) . T h e r e f o r e we p r o v i d e i t i n P r o g r a m C.2 i n A p p e n d i x C.

P r o g r a m 1 7 . 2 . L00CV2D.m

% L00CV2D
% S c r i p t t h a t performs leave-one-out c r o s s - v a l i d a t i o n
/ (Rippa's method) t o f i n d a good e p s i l o n f o r 2D RBF i n t e r p o l a t i o n
0

% C a l l s on: D i s t a n c e M a t r i x
1 r b f = @(e,r) e x p ( - ( e * r ) . " 2 ) ; % Gaussian RBF
% Parameters f o r shape parameter loop below
2 mine = 0; maxe = 20; ne = 500;
3 ep = linspace(mine,maxe,ne);
% Number and type of d a t a p o i n t s
4 N = 81; gridtype = 'u';
7, Define t e s t f u n c t i o n
5 t e s t f u n c t i o n = @(x,y) s i n e ( x ) . * s i n c ( y ) ;
% Load data p o i n t s
,
6 name = s p r i n t f ( Data2D_/ d%s',N,gridtype) ; load(name)
0

7 ctrs = dsites; '/ c e n t e r s c o i n c i d e w i t h data s i t e s

7, Create r i g h t - h a n d s i d e v e c t o r , i . e . ,
7o evaluate t h e t e s t f u n c t i o n a t t h e d a t a p o i n t s .
8 rhs = testfunction(dsites(:,1),dsites(:,2));
'/, Compute d i s t a n c e m a t r i x between t h e data s i t e s and c e n t e r s
9 DM_data = D i s t a n c e M a t r i x ( d s i t e s , c t r s ) ;
10 f o r i = l : l e n g t h ( e p )
7o Compute i n t e r p o l a t i o n m a t r i x
11 IM = r b f ( e p ( i ) , D M _ d a t a ) ;
% Compute e r r o r f u n c t i o n ( i . e . , " c o s t " of e p s i l o n )
12 invIM = p i n v ( I M ) ;
13 EF = ( i n v I M * r h s ) . / d i a g ( i n v I M ) ;
7 Compute maximum norm of E F
14 maxEF(i) = n o r m ( E F ( : ) , i n f ) ;
15 end
16 f p r i n t f ('Smallest maximum norm: 7 e\n' , min(maxEF)) 0

17 f p r i n t f ( a t e p s i l o n = 7f \n',ep(maxEF==min(maxEF)) )
J

/ P l o t cost f u n c t i o n norm
0

18 f i g u r e ; semilogy(ep,maxEF,'b');

I n F i g u r e 17.3 w e show p l o t s o f t h e p r e d i c t e d m a x i m u m e r r o r s vs. e for a one-

148 Meshfree Approximation Methods with M A T L A B

d i m e n s i o n a l e x p e r i m e n t (left) a n d a 2 D e x p e r i m e n t ( r i g h t ) based o n d a t a s a m p l e d
f r o m t h e sine f u n c t i o n F\. E a c h p l o t shows several curves c o r r e s p o n d i n g t o different
choices o f N (set o n l i n e 4 o f L00CV2D.m). T h e o p t i m a l e values are l i s t e d i n
T a b l e 17.3.

0 5 10 15 20 0 5 10 15 20
e e

Fig. 17.3 Optimal e curves based on leave-one-out cross validation for interpolation to the sine
function with Gaussians in I D (left) and 2D (right) for various choices of N uniform points.

Table 17.3 Optimal e values based

on leave-one-out cross validation for
interpolation to the sine function
with Gaussians in I D and 2D for
various choices of N uniform points.

ID 2D
N optimal e N optimal

3 0.96 9 0.96
5 1.00 25 1.00
9 0.80 81 1.48
17 0.92 289 1.60
33 1.92
65 1.76

T h e graphs i n F i g u r e 17.4 s h o w side-by-side t h e o p t i m a l s curves for t h e t r i a l a n d

e r r o r a p p r o a c h a n d for t h e leave-one-out cross v a l i d a t i o n a p p r o a c h i n t h e case o f I D
Gaussian i n t e r p o l a t i o n t o d a t a sampled f r o m the test function F - 2 The similarity
o f t h e curves is c l e a r l y a p p a r e n t . T h u s , t h e leave-one-out cross v a l i d a t i o n a p p r o a c h
c a n be r e c o m m e n d e d as a g o o d m e t h o d for s e l e c t i n g a n " o p t i m a l " shape p a r a m e t e r
e since for t h i s m e t h o d n o k n o w l e d g e o f t h e e x a c t e r r o r is needed. A n o t h e r p a i r o f
c o m p a r i s o n p l o t s is g i v e n b y t h e G a u s s i a n i n t e r p o l a n t s t o t h e sine f u n c t i o n F\ i n
F i g u r e 17.1 (left) a n d F i g u r e 17.3 ( l e f t ) .
S i m i l a r conclusions h o l d for o t h e r basic f u n c t i o n s , o t h e r test f u n c t i o n s , o t h e r
d a t a d i s t r i b u t i o n s , a n d o t h e r space d i m e n s i o n s . F o r e x a m p l e , F i g u r e 17.5 shows t h e
 17. Numerical Evidence for Approximation Order Results 149

10 10"

W -5 Of "
10
10 5

10" 10"
10 15 20 10 15 20
e

Fig. 17.4 Optimal e curves based on interpolation to the test function with Gaussians for
various choices of N uniform points. Trial and error approach (left), leave-one-out cross validation
(right).

o p t i m a l e curves for i n t e r p o l a t i o n t o t h e I D F r a n k e f u n c t i o n F 2 with Wendland's

2
C f u n c t i o n (ps i o n u n i f o r m l y spaced p o i n t s , a n d o n C h e b y s h e v p o i n t s . N o t e t h a t
;

for t h i s c o n f i g u r a t i o n a l l c o m p u t a t i o n s are stable, a n d t h e o p t i m a l scale p a r a m e t e r

is q u i t e s m a l l , i.e., t h e s u p p o r t r a d i u s o f t h e c o m p a c t l y s u p p o r t e d basic f u n c t i o n is
chosen t o be v e r y large. I n o t h e r w o r d s , t h e best results for c o m p a c t l y s u p p o r t e d
f u n c t i o n s are o b t a i n e d w i t h dense m a t r i c e s .

10

10

10 y /

f 4
/ 4*
10" _ // * N=3
* N=5
N=9
10 t N=17
N=33
N=65
10
10 15 20

Fig. 17.5 Optimal e curves based on leave-one-out cross validation for interpolation to I D Franke's
function with Wendland's function 953,1 for various choices of N uniform points (left) and Cheby-
shev points (right).

I f we are n o t i n t e r e s t e d i n t h e e-curves d i s p l a y e d above, b u t o n l y w a n t t o find

a g o o d value o f t h e shape p a r a m e t e r as q u i c k l y as possible, t h e n we c a n use the
M A T L A B f u n c t i o n f m i n b n d t o find t h e m i n i m u m o f t h e cost f u n c t i o n for e. First,
we i m p l e m e n t t h e cost f u n c t i o n i n t h e s u b r o u t i n e C o s t E p s i l o n . m d i s p l a y e d i n P r o -
g r a m 17.3. T h e c o m m a n d s are t h e same as those o n lines 1 1 - 1 4 i n P r o g r a m 17.2.
150 Meshfree Approximation Methods with M A T L A B

P r o g r a m 17.3. CostEpsilon.m

7. ceps = C o s t E p s i l o n ( e p , r , r b f , r h s )
% Implements cost f u n c t i o n f o r o p t i m i z a t i o n of shape parameter
7, v i a Rippa's L00CV algorithm
% Example of usage i n L00CV2Dmin.m
1 f u n c t i o n ceps = C o s t E p s i l o n ( e p , r , r b f , r h s )
2 A = rbf(ep,r);
3 invA = p i n v ( A ) ;
4 EF = ( i n v A * r h s ) . / d i a g ( i n v A ) ;
5 ceps = n o r m ( E F ( : ) , i n f ) ;

In order to demonstrate the use of the C o s t E p s i l o n function we use a modifi-

cation of Program 17.2 which we list as Program 17.4.

P r o g r a m 17.4. L00CV2Dmin.m

'/. L00CV2Dmin
% S c r i p t t h a t performs leave-one-out c r o s s - v a l i d a t i o n
7 5
( R i p p a s method) to f i n d a good e p s i l o n f o r 2D RBF i n t e r p o l a t i o n
[Link] the help of MATLAB' s fminbnd
% C a l l s on: D i s t a n c e M a t r i x
7,Requires: CostEpsilon
1 r b f = @(e,r) e x p ( - ( e * r ) . ~ 2 ) ; 7o Gaussian RBF
7o Parameters f o r shape parameter o p t i m i z a t i o n below
2 mine = 0; maxe = 20;
7o Number and type of data p o i n t s
3 N = 81; g r i d t y p e = 'u';
7o Define t e s t f u n c t i o n
4 t e s t f u n c t i o n = @(x,y) s i n e ( x ) . * s i n c ( y ) ;
7o Load data p o i n t s
5 name = s p r i n t f ( 'Data2D_7od7 s ' ,N,gridtype) ; load(name)
0

6 ctrs = dsites; % centers coincide with data s i t e s

7 Create right-hand s i d e v e c t o r , i . e . ,
7o e v a l u a t e the t e s t f u n c t i o n a t the d a t a p o i n t s .
7 rhs = t e s t f u n c t i o n ( d s i t e s ( : , 1 ) , d s i t e s ( : , 2 ) ) ;
7o Compute d i s t a n c e matrix between the d a t a s i t e s and c e n t e r s
8 DM_data = D i s t a n c e M a t r i x ( d s i t e s , c t r s ) ;
9a [ e p , f v a l ] = fminbnd(@(ep) CostEpsilon(ep,DM_data,rbf,rhs),...
9b mine,maxe);
10 f p r i n t f ('Smallest maximum norm: 7oe\n', f v a l )
11 f p r i n t f ('at e p s i l o n = 7of\n' , ep)
 17. Numerical Evidence for Approximation Order Results 151

17.1.4 The Contour-Pade Algorithm

The Contour-Pade a l g o r i t h m was t h e s u b j e c t o f G r a d y W r i g h t ' s P h . D . thesis

[ W r i g h t (2003)] a n d was r e p o r t e d i n [ F o r n b e r g a n d W r i g h t ( 2 0 0 4 ) ] . T h e a i m o f
t h e C o n t o u r - P a d e a l g o r i t h m is t o come u p w i t h a m e t h o d t h a t allows t h e c o m p u t a -
t i o n a n d e v a l u a t i o n o f R B F i n t e r p o l a n t s for i n f i n i t e l y s m o o t h basic f u n c t i o n s w h e n
t h e shape p a r a m e t e r e tends t o zero ( i n c l u d i n g t h e l i m i t i n g case).
T h e s t a r t i n g p o i n t is t o consider e v a l u a t i o n o f t h e R B F i n t e r p o l a n t
N
'Pf{x,e) = ^2c (p (\\x-x \\)
j e j

j=i
for a fixed e v a l u a t i o n p o i n t x as a n a n a l y t i c f u n c t i o n o f e.
T h e key idea is t o represent Vf(x, e) b y a Laurent series i n e, a n d a p p r o x i m a t e
t h e "negative p a r t " o f t h e series b y a P a d e a p p r o x i m a n t , i.e.,
oo
d ek
V (x,e)
f ^r(e) + J2 k >
k=0
where r(e) is t h e r a t i o n a l P a d e a p p r o x i m a n t .
We t h e n r e w r i t e t h e i n t e r p o l a n t i n c a r d i n a l f o r m , i.e., as

Vf{x,e) = ^C V {\\X
3 - Xj\\)

j=i
T
= b (x, e)c
T
= b {x,e)A-\e)f
= (u*(x,e)ff
T
where b(x,e) - 3 = ip (\\x
- xj\\), A(e)ij = <p (\\xi
- x - \ \ ) , c = [a,...,
3 c],
N f =
T
[fi,-- -,fN] , and
_1
u*(x,e) = A ()6(a;,)

denotes t h e vector o f values o f t h e c a r d i n a l f u n c t i o n s at x (c.f. C h a p t e r 14).

I t is n o w t h e g o a l t o s t a b l y c o m p u t e t h e v e c t o r u* (e) for a l l values o f e > 0 w i t h -
1
o u t e x p l i c i t l y f o r m i n g t h e inverse A(e)~ and w i t h o u t c o m p u t i n g the m a t r i x vector
1
p r o d u c t A(e)~ b(e). Here t h e v e c t o r s u*(e) a n d b(s) are o b t a i n e d b y e v a l u a t i n g
t h e vector f u n c t i o n s u*(-,e) a n d b(-,e) on an appropriate evaluation grid.
T h e s o l u t i o n p r o p o s e d b y W r i g h t a n d F o r n b e r g is t o use C a u c h y ' s i n t e g r a l t h e -
o r e m t o i n t e g r a t e a r o u n d a c i r c l e i n t h e c o m p l e x e-plane. T h e residuals (i.e., co-
efficients i n t h e L a u r e n t e x p a n s i o n ) are o b t a i n e d u s i n g t h e (inverse) fast F o u r i e r
transform. T h e t e r m s w i t h n e g a t i v e powers o f e are t h e n a p p r o x i m a t e d u s i n g a
r a t i o n a l Pade a p p r o x i m a n t . T h e i n t e g r a t i o n c o n t o u r ( u s u a l l y a circle) has t o l i e
between t h e r e g i o n o f i n s t a b i l i t y near e = 0 a n d possible b r a n c h p o i n t s i n g u l a r i t i e s
t h a t lie somewhere i n t h e c o m p l e x p l a n e d e p e n d i n g o n t h e choice o f ip. D e t a i l s o f
t h e m e t h o d can be f o u n d i n [ F o r n b e r g a n d W r i g h t (2004)].
152 Meshfree Approximation Methods with M A T L A B

I n F i g u r e 17.6 w e show o p t i m a l e curves for i n t e r p o l a t i o n t o t h e I D a n d 2 D

sine f u n c t i o n F 2 u s i n g Gaussians a t e q u a l l y spaced p o i n t s . T h e s e curves s h o u l d be
c o m p a r e d w i t h t h e o p t i m a l e curves o b t a i n e d for t h e same p r o b l e m v i a t r i a l a n d
e r r o r (see F i g u r e 17.1 a n d T a b l e 17.1) a n d v i a leave-one-out cross v a l i d a t i o n (see
F i g u r e 17.3 a n d T a b l e 17.3).
T h e m a i n d r a w b a c k o f t h e C o n t o u r - P a d e a l g o r i t h m is t h e fact t h a t i f N becomes
t o o large t h e n t h e r e g i o n o f i l l - c o n d i t i o n i n g a r o u n d t h e o r i g i n i n t h e c o m p l e x e-
p l a n e a n d t h e b r a n c h p o i n t s i n g u l a r i t i e s w i l l o v e r l a p . T h i s , however, i m p l i e s t h a t
t h e m e t h o d c a n o n l y be used w i t h l i m i t e d success. M o r e o v e r , as t h e g r a p h s i n
F i g u r e 17.6 a n d t h e entries i n T a b l e 17.4 show, t h e value o f N t h a t has t o be
considered "large" is u n f o r t u n a t e l y r a t h e r s m a l l . F o r t h e o n e - d i m e n s i o n a l case t h e
results for N = 17 a l r e a d y are affected b y i n s t a b i l i t i e s , a n d i n t h e t w o - d i m e n s i o n a l
e x p e r i m e n t N = 8 1 causes p r o b l e m s .

0 5 10 15 20 0 5 10 15 20
E E

Fig. 17.6 Optimal e curves based on Contour-Pade for interpolation to the sine function with
Gaussians in I D (left) and 2D (right) for various choices of N uniform points.

Table 17.4 Optimal e values based on Contour-Pade for interpolation to the sine
function with Gaussians in I D and 2 D for various choices of N uniform points.

ID 2D

N max-error cond(A) N max-error cond(A)

3 1.7605e-003 1.10 3.3386e+001 9 3.3875e-003 1.10 1.1146e+003

5 4.0380e-005 0.70 1.3852e+006 25 5.5542e-005 0.70 1.9187e+012
9 3.9703e-009 0.45 7.7731e+016 81 3.6528e-004 0.00 oo
17 1.2726e-009 0.45 1.7327e+018

17.1.5 Summary

A l l strategies p u r s u e d i n t h i s c h a p t e r have s h o w n t h a t even t h o u g h t h e bound

(15.11) b y M a d y c h seems t o i n d i c a t e t h a t t h e i n t e r p o l a t i o n e r r o r for f u n c t i o n s i n
 17. Numerical Evidence for Approximation Order Results 153

t h e n a t i v e space o f t h e basic f u n c t i o n goes t o zero e x p o n e n t i a l l y as e 0, t h i s

does n o t seem t o be t r u e i n p r a c t i c e . E s p e c i a l l y those o p t i m a l e c u r v e s t h a t w e r e
c o m p u t e d r e l i a b l y w i t h t h e C o n t o u r - P a d e a l g o r i t h m a l l have a g l o b a l m i n i m u m for
some p o s i t i v e value o f e. I n m a n y cases t h i s o p t i m a l s v a l u e ( o r a n e close t o t h e
o p t i m a l v a l u e ) c a n be f o u n d u s i n g t h e leave-one-out cross v a l i d a t i o n a l g o r i t h m o f
P r o g r a m 17.2. F r o m n o w o n we w i l l f r e q u e n t l y use leave-one-out cross v a l i d a t i o n
t o find a n o p t i m a l shape p a r a m e t e r f o r o u r n u m e r i c a l e x p e r i m e n t s .

17.2 Non-stationary Interpolation

I n o r d e r t o i l l u s t r a t e t h e s p e c t r a l convergence p r e d i c t e d for i n f i n i t e l y s m o o t h basic

f u n c t i o n s such as Gaussians a n d g e n e r a l i z e d (inverse) m u l t i q u a d r i c s w e need t o w o r k
i n a s e t t i n g for w h i c h n e i t h e r t h e i n s t a b i l i t y due t o large p r o b l e m size o r s m a l l shape
p a r a m e t e r have a s i g n i f i c a n t effect o n o u r e x p e r i m e n t s . O t h e r w i s e , i f w e s i m p l y t a k e
a n " o p t i m a l " value o f e ( d e t e r m i n e d v i a t r i a l a n d e r r o r for a l a r g e N = 4225 p r o b l e m
i n t h e " g r a y zone", c.f. C h a p t e r 2) t h e n t h e s p e c t r a l convergence w i l l o n l y be v i s i b l e
for a l i m i t e d n u m b e r o f e x p e r i m e n t s (see T a b l e 17.5).

Table 17.5 2D non-stationary interpolation (e = 6.3) to

Franke's function with Gaussians on uniformly spaced and
Halton points.

uniform Halton

N RMS-error rate RMS-error rate

9 3.195983e-001 2.734756e-001
25 5.008591e-002 2.6738 8.831682e-002 2.3004
81 9.029664e-003 2.4717 2.401868e-002 1.7582
289 2.263880e-004 5.3178 1.589117e-003 5.0969
1089 3.323287e-008 12.7339 1.595051e-006 10.8015
4225 1.868286e-008 0.8309 9.510404e-008 4.8203

E v e n for a b a n d - l i m i t e d f u n c t i o n (see T a b l e 17.6) t h e s i t u a t i o n is n o t b e t t e r ; i n

fact worse, for t h e v a l u e o f e used.
I n F i g u r e s 17.7-17.8 we are able t o v e r i f y ( a t least t o some e x t e n t ) t h e c o n v e r -
gence estimates for n o n - s t a t i o n a r y R B F i n t e r p o l a n t s . W e o b t a i n t h e d a t a for a l l
e x p e r i m e n t s b y s a m p l i n g t h e sine f u n c t i o n f(x) = sin(7rx)/(7rx) at N uniformly
spaced p o i n t s i n t h e i n t e r v a l [ 0 , 1 ] w h e r e N r u n s f r o m 1 t o 100. E a c h p l o t shows s i x
m a x i m u m e r r o r curves ( c o r r e s p o n d i n g t o shape p a r a m e t e r s e 1 , 6 , 1 1 , 1 6 , 2 1 , 2 6 )
versus t h e n u m b e r N o f d a t a p o i n t s o n a l o g l o g scale. T h e e r r o r s are e v a l u a t e d o n a
g r i d o f 250 e q u a l l y spaced p o i n t s . I n o r d e r t o c o m p a r e these curves w i t h t h e t h e o -
r e t i c a l b o u n d s f r o m C h a p t e r 15 w e have p l o t t e d c o m p a r i s o n curves c o r r e s p o n d i n g t o
t h e t h e o r e t i c a l b o u n d s . F o r Gaussians t h e c o m p a r i s o n c u r v e is g i v e n b y t h e g r a p h
l h h
o f h i c ^ z \/ c o r r e s p o n d i n g t o s u p e r - s p e c t r a l convergence w i t h h = l/(N 1),
154 Meshfree Approximation Methods with M A T L A B

Table 17.6 2D non-stationary interpolation (e = 6.3) to the sine

function with Gaussians on uniformly spaced and Halton points.

uniform Halton

N RMS-error rate RMS-error rate

9 3.302644e-001 2.823150e-001
25 3.271035e-002 3.3358 1.282572e-001 1.6058
81 1.293184e-002 1.3388 3.407580e-002 1.7898
289 3.786113e-004 5.0941 1.990217e-003 5.3309
1089 3.476835e-008 13.4107 2.286014e-006 10.5905
4225 3.775365e-008 -0.1188 9.868530e-008 5.3724

Fig. 17.7 Maximum errors for non-stationary interpolation to the sine function with Gaussians
(left) and inverse multiquadrics (right) based on N uniformly spaced points in [0,1] and e
1,6,11,16,21,26.

1 / / / l
a n d for inverse m u l t i q u a d r i c s we have s p e c t r a l convergence w i t h h i e ~ . We
c a