r - customized ridge regression function

Question

I am working on Ridge regression, I want to make my own function. It tried the following. It work for individual value of k but not for array for sequence of values.

dt<-longley
attach(dt)
library(MASS) 
X<-cbind(X1,X2,X3,X4,X5,X6)
X<-as.matrix(X)
Y<-as.matrix(Y)

sx<-scale(X)/sqrt(nrow(X)-1)
sy<-scale(Y)/sqrt(nrow(Y)-1)
rxx<-cor(sx)
rxy<-cor(sx,sy)

for (k in 0:1){
res<-solve(rxx+k*diag(rxx))%*%rxy
k=k+0.01
}

Need help for optimized code too.

Answer 1

poly.kernel <- function(v1, v2=v1, p=1) {   
    ((as.matrix(v1) %*% t(v2))+1)^p
}   

KernelRidgeReg <- function(TrainObjects,TrainLabels,TestObjects,lambda){

  X <- TrainObjects
  y <- TrainLabels                      
  kernel <- poly.kernel(X)

  design.mat <- cbind(1, kernel)

  I <- rbind(0, cbind(0, kernel))

  M <- crossprod(design.mat) + lambda*I
  #crossprod is just x times  traspose of x, just looks neater in my openion

  M.inv <- solve(M)
  #inverse of M

  k <- as.matrix(diag(poly.kernel(cbind(TrainObjects,TrainLabels))))
  #Removing diag still gives the same MSE, but will output a vector of prediction.

  Labels <- rbind(0,as.matrix(TrainLabels))

  y.hat <- t(Labels) %*% M.inv %*% rbind(0,k)

  y.true <- Y.test

  MSE <-mean((y.hat - y.true)^2) 

  return(list(MSE=MSE,y.hat=y.hat))

}

p = 1的内核将为您提供岭回归。

解决内置 R 函数有时会返回奇异矩阵。您可能想编写自己的函数来避免这种情况。