# Convex Regression Model

Share Tweet

This morning during the lecture on nonlinear regression, I mentioned (very) briefly the case of convex regression. Since I forgot to mention the codes in R, I will publish them here. Assume that

where
is some convex function.

Then

is convex if and only if
,
,
Hidreth (1954) proved that if
then
is unique.

Let

, then
where
. I.e.
is the projection of
onto the (closed) convex cone
. The projection theorem gives existence and unicity.

For convenience, in the application, we will consider the real-valued case,

, i.e.
. Assume that observations are ordered
. Here

linear constraints.

is a piecewise linear function (interpolation of consecutive pairs
).

If

is differentiable,
is convex if

More generally, if

is convex, then there exists
such that

at
. And then

Hence,

is solution of
and
. Now, to do it for real, use cobs package for constrained (b)splines regression,

 1  library(cobs)

To get a convex regression, use

 1 2 3 4 5  plot(cars) x = cars$speed y = cars$dist rc = conreg(x,y,convex=TRUE) lines(rc, col = 2)

Here we can get the values of the knots

 1 2 3 4 5 6 7  rc   Call: conreg(x = x, y = y, convex = TRUE) Convex regression: From 19 separated x-values, using 5 inner knots, 7, 8, 9, 20, 23. RSS = 1356; R^2 = 0.8766; needed (5,) iterations

and actually, if we use them in a linear-spline regression, we get the same output here

 1 2 3 4  reg = lm(dist~bs(speed,degree=1,knots=c(4,7,8,9,,20,23,25)),data=cars) u = seq(4,25,by=.1) v = predict(reg,newdata=data.frame(speed=u)) lines(u,v,col="green")

Let us add vertical lines for the knots

 1  abline(v=c(4,7,8,9,20,23,25),col="grey",lty=2)

Share Tweet
__data