Exercise 11.1

Suppose that the knots are located at \(t_1\) and \(t_2\). Then, the spline is linear on each of the three intervals: \(f(t)=\alpha_0+b_0t\) (for \(t<t_1\)), \(f(t)=\alpha_1+b_1t\), (for \(t_1\le t<t_2\)), and \(f(t)=\alpha_2+b_2t\), (for \(t\ge t_2\)). The six model parameters are: intercepts, \(\alpha_0, \alpha_1, \alpha_2\) and slopes, \(b_0, b_1, b_2\).

If we have data points at the knots, with function values \(f_1\) at \(t_1\) and \(f_2\) at \(t_2\), then the splines must pass through these coordinates: \(\alpha_0+b_0 t_1 = f_1\), \(\alpha_1+b_1 t_1 = f_1\), and \(\alpha_1+b_1 t_2 = f_2\), \(\alpha_2+b_2 t_2 = f_2\).

For continuity of function at the two knots: \(\alpha_0+b_0 t_1 = \alpha_1+b_1 t_1\), and \(\alpha_1 + b_1 t_2 = \alpha_2+b_2 t_2\). Note, however, that these two constraints will automatically be satisfied if the previous set of four, to force the spline through the data points, are satisfied – they are redundant. But note, that the opposite is not true, that is continuity does not imply that they pass through the data points.

In the outer two intervals we require the spline to be constant, hence: \(b_0=0\) and \(b_2=0\).


Exercise 11.2

If the constraints in Exercise 11.1 are written as a system of linear equations we obtain: \[\alpha_0+b_0 t_1 = f_1\] \[\alpha_1+b_1 t_1 = f_1\] \[\alpha_1+b_1 t_2 = f_2\] \[\alpha_2+b_2 t_2 = f_2\] \[b_0=0\] \[b_2=0\]

or in matrix form: \[ \begin{bmatrix} f1 \\ f1 \\ f2\\ f2\\ 0\\ 0 \end{bmatrix} = \begin{bmatrix} 1 & 0 & 0 & t_1 & 0 & 0 \\ 0 & 1 & 0 & 0 & t_1 & 0 \\ 0 & 1 & 0 & 0 & t_2 & 0 \\ 0 & 0 & 1 & 0 & 0 & t_2 \\ 0 & 0 & 0 & 1 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 & 1 \end{bmatrix} \begin{bmatrix} \alpha_0 \\ \alpha_1 \\ \alpha_2\\ b_0\\ b_1 \\ b_2 \end{bmatrix} \] Note that other orderings of rows and column are fine.

This could be solved by direct inversion. Let the system be written \({\mathbf y} = X \boldsymbol{\beta}\) then \({\hat{\boldsymbol{\beta}}} = X^{-1}{\mathbf y}\).


Exercise 11.3

Suppose we have data points at \((1,3)\) and \((2,5)\), then R the problem can be solved as follows, but note hat at most steps there are other ways to organise the calculations.

locations = c(1, 2)
mydata = data.frame(x=c(1,2), y=c(3,5))

plot(mydata, xlim=c(0,3), ylim=c(0,6), pch=16)

y = c(3, 3, 5, 5, 0, 0)

# Design matrix
X= matrix(c(1,0,0,1,0,0,
            0,1,0,0,1,0,
            0,1,0,0,2,0,
            0,0,1,0,0,2,
            0,0,0,1,0,0,
            0,0,0,0,0,1), ncol=6, byrow=T)

hatbeta = solve(X) %*% y

curve(hatbeta[1]+hatbeta[4]*x, 0, 1, add=T)
curve(hatbeta[2]+hatbeta[5]*x, 1, 2, add=T)
curve(hatbeta[3]+hatbeta[6]*x, 2, 3, add=T)


Exercise 11.4

Linear natural splines can be written as: \[ f(t) = a_0 + \sum_{i=1}^m b_i \left| t - t_i \right|; \quad \sum_{i=1}^m b_i = 0 \] and with \(m=2\) these become \[ f(t) = a_0 + b_1 \left| t - t_1 \right|+b_2 \left| t - t_2 \right|; \quad b_1+b_2 = 0. \] In matrix form, for the two data points, this can be written as follows: \[ \begin{bmatrix} f1 \\ f2\\ 0 \end{bmatrix} = \begin{bmatrix} 1 & |t_1-t_1| & |t_1-t_2| \\ 1 & |t_2-t_1| & |t_2-t_2| \\ 0 & 1 & 1 \\ \end{bmatrix} \begin{bmatrix} a_0 \\ b_1 \\ b_2 \end{bmatrix} \]


Exercise 11.5

With the earlier chosen data pints the system becomes: \[ \begin{bmatrix} 3 \\ 5\\ 0 \end{bmatrix} = \begin{bmatrix} 1 & 0 & 1 \\ 1 & 1 & 0 \\ 0 & 1 & 1 \\ \end{bmatrix} \begin{bmatrix} a_0 \\ b_1 \\ b_2 \end{bmatrix} \] Then, solving the system in R:

y2 = c(3,5,0)
X2 = matrix(c(1,0,1,
           1,1,0,
           0,1,1), ncol=3, byrow=T)

hatbeta2 = solve(X2) %*% y2

plot(mydata, xlim=c(0,3), ylim=c(0,6), pch=16)

curve(hatbeta2[1]+hatbeta2[2]*abs(x-1)+hatbeta2[3]*abs(x-2), 0,3, add=T, lwd=2)

# Add the earlier calculated spline
curve(hatbeta[1]+hatbeta[4]*x, 0, 1, add=T, col="red", lty=2)
curve(hatbeta[2]+hatbeta[5]*x, 1, 2, add=T, col="red", lty=2)
curve(hatbeta[3]+hatbeta[6]*x, 2, 3, add=T, col="red", lty=2)

The two splines appear identical, but you could also do the algebra to check.

Perhaps the second approach is preferable as the code is shorter, but the first approach may be more intuitive – the choice is yours!


Exercise 11.6

Make a copy of the R code from earlier and remove the set.seed line. Here, \(\texttt{par(mfrow=c(1,2))}\) has been added to get the graphs side-by-side.

par(mfrow=c(1,2))
par(mar=c(3,3,1,1), mgp=c(2,1,0))

x = 1:9; y = rnorm(9)

# spline deals with points
plot(x, y, xlim=c(0,10), ylim=c(-3,3), pch=4)

mysplinefit1 = spline(x, y, method="natural")
lines(mysplinefit1)


# splinefun produces a function
plot(x, y, xlim=c(0,10), ylim=c(-3,3), pch=4)

mysplinefit2 = splinefun(x, y, method="natural")
curve(mysplinefit2, 0, 10, add=T)

Running this repeatedly with different random data each time, the two approaches always agree: at any shared \(x\) value, \(\texttt{spline()}\) and \(\texttt{splinefun()}\) give identical fitted values, since both use exactly the same underlying natural cubic spline algorithm. The difference is purely one of interface. \(\texttt{spline()}\) returns a fixed list of \((x,y)\) pairs – by default \(3n\) points spanning the data range – ready to be plotted directly with \(\texttt{lines()}\). \(\texttt{splinefun()}\) instead returns a function that can be evaluated at any new \(x\) value on demand, which is why it must be paired with \(\texttt{curve()}\) rather than \(\texttt{lines()}\). In practice, \(\texttt{spline()}\) is convenient for a one-off plot, while \(\texttt{splinefun()}\) is more useful when the fitted spline needs to be evaluated repeatedly at arbitrary points, for example for prediction at new data. Of course, since the underlying \(y\) data are freshly randomised each run, the shape of the fitted curve itself is different every time – only the agreement between the two methods is a genuine similarity across runs.


Exercise 11.7

When considering the function in each interval, the important issue is to correctly remove the absolution value. In particular, \(|t|\) for values of \(t\in (-\infty,0)\) can be replaced by \(-t\), and \(|t-1|\) for \(t\in (1,\infty)\) can be replaced by \(t-1\). The full list of replacements are:

Term \((-\infty, 0)\) \((0,1)\) \((1, \infty\))
\(|t|\) \(-t\) \(t\) \(t\)
\(|t-1|\) \(-(t-1)\) \(-(t-1)\) \((t-1)\)

Within the intervals the spline \(f(t) = 3 + 2t + 4|t|^3 + |t-1|^3\) becomes, after expanding and collecting the terms:

Interval Spline function First derivative Second derivative
\((-\infty,0)\) \(f_0(t)=4-t+3t^2-5t^3\) \(f_0'(t)=-1+6t-15t^2\) \(f_0''(t)=6-30t\)
\((0,1)\) \(f_1(t)=4-t+3t^2+3t^3\) \(f_1'(t)=-1+6t+9t^2\) \(f_1''(t)=6+18t\)
\((1,\infty)\) \(f_2(t)=2+5t-3t^2+5t^3\) \(f_2'(t)=5-6t+15t^2\) \(f_2''(t)=-6+30t\)

It is easy to show continuity of function at the knots (\(t_1=0\) and \(t_2=1\)): \(f_0(t_1=0)=f_1(t_1=0)=4\) and \(f_1(t_2=1)=f_2(t_2=1)=9\). Then, continuity of first derivative: \(f_0'(t_1=0)=f_1'(t_1=0)=-1\) and \(f_1'(t_2=1)=f_2'(t_2=1)=14\). Finally, of second derivative: \(f_0''(t_1=0)=f_1''(t_1=0)=6\) and \(f_1''(t_2=1)=f_2''(t_2=1)=24\).

The function \(f\) is a cubic spline, but it is not a natural spline as it is not linear outside the range of the knots, that is for \(t<0\) and \(t>1\).

Another way to prove this result is to write \(f\) in the form \(f(t) = a_0 + a_1 t + \sum_{j=1}^n b_j |t-t_j|^3\), where \(n=2\) and \(b_1=4, b_2 = 1\). Since the function \(|t-t_j|^3\) is twice continuously differentiable at \(t=t_j\) (check) and is cubic for \(t<t_j\) and for \(t>t_j\), it follows immediately that \(f\) is a cubic spline. But since it is not the case that \(\sum b_j = 0, \ \sum b_j t_j = 0\), it follows that the spline is not natural.


Exercise 11.8

The function \(f(t) = 3+|t|-|t-2|\) takes the more explicit form \[\begin{align*} f(t) &= 3-t+(t-2) = 1, \quad t<0,\\ f(t) &= 3+t+(t-2) = 2t+1, \quad 0<t<2,\\ f(t) &= 3+t-(t-2) = 5, \quad t>2. \end{align*}\] Note that \(f\) is a natural linear spline (natural since \(f\) is constant for \(t<0\) and \(t>2\)). Its first derivative takes the form \(f'(t) = 0, \ t<0, f'(t) = 2, \ 0<t<2,\) and \(f'(t) = 0, \ t>2\). Hence \[ \int_{-\infty}^{\infty} \{f'(t)\}^2 \,dt = \int_{0}^{2} 2^2\,dt = 8. \] In terms of the \(2 \times 2\) matrix \(K = (k_{ij}) = (|t_i - t_j|)\), where \(t_1=0, \ t_2=2\) and the coefficient vector \(\mathbf b\) with \(b_1=1, b_2=-1\), we find that the quadratic form \[ -2 \mathbf b^T K \mathbf b = \begin{bmatrix} 1 & -1 \end{bmatrix} \begin{bmatrix} 0 & 2 \\ 2 & 0 \end{bmatrix} \begin{bmatrix} 1 \\ -1 \end{bmatrix} = 8 \] takes the same value as the integral.


End of Solutions to Exercises 11