Multivariate Linear Regression

Multivariate Linear Regression:-

When more than 1 features are given in linear regression then problem comes under multivariate linear regression.All the techniques are same as used in univariate linear regression except vectorized form is used for better optimization .

Dataset:

so to find the best fit equation of line for linearly correlated dataset .first we assume the hypothesis 
                      
 $ H \left(  x  \right) =  \theta^{T} \times x   $

 WHERE
 $ x=  \left[\begin{matrix}  x_{0}  \\ x_{1} \\ \vdots \\ x_{n+1} \end{matrix}\right] ,   \theta = \left[\begin{matrix}  \theta_{0}  \\ \theta_{1} \\ \vdots \\ \theta_{n+1} \end{matrix}\right]   $
And n is the number of features. and here $ x_{0}=1 $


Now we suppose the cost function for the multivariate linear regression which is same as to univariate linear regression.

$  J =  \frac{1}{ 2n}\sum_{i=1}^{n} \left( \theta^T\times x-y^i  \right)^2 $

For the optimal values of  $ \theta $ found equation best fit the given dataset.
we can find optimal values of $ \theta $ by using gradient descent algorithm.

Repeat until convergence
{
  $  \theta_{i}=\theta_{i}- \alpha \times  \frac{\partial  J }{\partial\theta_{i}}  $
}

Matlab / Octave approach to Linear Regression :-

Dataset format is given as :

$ X= \left[\begin{matrix} 1 & a_{1,2} & \cdots & a_{1,n+1} \\ 1 & a_{2,2} & \cdots & a_{2,n+1} \\ \vdots & \vdots & \ddots & \vdots \\ 1 & a_{m,2} & \cdots & a_{m,n+1} \end{matrix}\right] , \theta = \left[\begin{matrix} \theta_{0} \\ \theta_{1} \\ \vdots \\ \theta_{n+1} \end{matrix}\right] ,  y=  \left[\begin{matrix}  y_{0}  \\ y_{1} \\ \vdots \\ y_{n+1} \end{matrix}\right]  $ 

Where a is the feature

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25

% X and y and theta's format is given above the codes in mathematical form m = length(y); X=[ones(m,1),x]; theta = zeros(m,1); % plotting the value of x and y plot(x,y); %gradient descent function %computing gradient descent function theta=grad_descent(X, y, theta, alpha,iterations) m=length(y); for iter = 1:iterations S=zeros(1,length(y)); <% vectorized implementation S=sum(((theta'.*X')'.-y).*X); theta=theta - alpha/m*S; end theta = grad_descent(X,y,theta,0.04,2500); % plotting the hypothesis hold on; % keep previous plot visible plot(X(:,2), X*theta, '-'); hold off; % Now we can predict the value of y for any value of x by predict_y = theta'*x_val;