Linear regression as a basic model
Case 1. Apartment valuation based on parameters
Implementation in pure PHP
(iterative optimization – gradient descent)
We will start without any libraries. This is useful not for production, but for building intuition. We will use gradient descent, a feature matrix $X$ of size $N$ x $4$, and a weight vector $w$ of length $4$.
We will add the bias as an extra feature with value 1.
Example of code
<?php
// Compute the dot product of two vectors: sum(a_i * b_i)
function dotProduct(array $a, array $b): float {
$sum = 0.0;
foreach ($a as $i => $v) {
$sum += $v * $b[$i];
}
return $sum;
}
// Training data: each row is an apartment described by features
// X: [area, floor, distance to city center, building age, bias]
$X = [
[50, 3, 5, 10, 1],
[70, 10, 3, 5, 1],
[40, 2, 8, 30, 1],
];
// Target values: apartment prices in dollars
$y = [
66_000,
95_000,
45_000,
];
// Initialize model parameters (weights) and training hyperparameters
$weights = array_fill(0, 5, 0.0);
$learningRate = 0.000001;
$epochs = 5_000;
// n – number of training examples, m – number of features (including bias)
$n = count($X);
$m = count($weights);
// Gradient descent loop: repeat several passes over the dataset
for ($epoch = 0; $epoch < $epochs; $epoch++) {
// Accumulate gradients for each weight over the whole dataset
$gradients = array_fill(0, $m, 0.0);
for ($i = 0; $i < $n; $i++) {
// Model prediction: y_hat = w · x
$prediction = dotProduct($weights, $X[$i]);
// Error for this example: true - predicted
$error = $y[$i] - $prediction;
// Accumulate gradient for each weight (derivative of MSE w.r.t. w_j)
for ($j = 0; $j < $m; $j++) {
$gradients[$j] += -2 * $X[$i][$j] * $error;
}
}
// Gradient descent update: move weights against the average gradient
for ($j = 0; $j < $m; $j++) {
$weights[$j] -= $learningRate * ($gradients[$j] / $n);
}
}
Implementation in pure PHP
(analytical solution – via normal equations)
First, we need a Matrix class that can perform the minimum required matrix operations.
Example of code
<?php
namespace app\classes;
use Exception;
class Matrix {
public array $data;
public int $rows;
public int $cols;
public function __construct(array $data) {
$this->data = $data;
$this->rows = count($data);
$this->cols = count($data[0]);
}
public function transpose(): Matrix {
$result = [];
for ($i = 0; $i < $this->cols; $i++) {
$row = [];
for ($j = 0; $j < $this->rows; $j++) {
$row[] = $this->data[$j][$i];
}
$result[] = $row;
}
return new Matrix($result);
}
public function multiply(Matrix $other): Matrix {
if ($this->cols !== $other->rows) {
throw new Exception('Matrix dimensions do not match for multiplication');
}
$result = [];
for ($i = 0; $i < $this->rows; $i++) {
$row = [];
for ($j = 0; $j < $other->cols; $j++) {
$sum = 0.0;
for ($k = 0; $k < $this->cols; $k++) {
$sum += $this->data[$i][$k] * $other->data[$k][$j];
}
$row[] = $sum;
}
$result[] = $row;
}
return new Matrix($result);
}
public function determinant(): float {
if ($this->rows !== $this->cols) {
throw new Exception('Determinant requires square matrix');
}
if ($this->rows === 1) {
return $this->data[0][0];
}
if ($this->rows === 2) {
return $this->data[0][0] * $this->data[1][1]
- $this->data[0][1] * $this->data[1][0];
}
$det = 0.0;
for ($c = 0; $c < $this->cols; $c++) {
$det += pow(-1, $c) * $this->data[0][$c] * $this->minor(0, $c)->determinant();
}
return $det;
}
public function inverse(): Matrix {
$det = $this->determinant();
if (abs($det) < 1e-10) {
throw new Exception('Matrix is singular (det = 0)');
}
if ($this->rows === 2) {
$a = $this->data;
return new Matrix([
[$a[1][1] / $det, -$a[0][1] / $det],
[-$a[1][0] / $det, $a[0][0] / $det],
]);
}
$cofactors = [];
for ($i = 0; $i < $this->rows; $i++) {
$row = [];
for ($j = 0; $j < $this->cols; $j++) {
$minor = $this->minor($i, $j);
$row[] = pow(-1, $i + $j) * $minor->determinant();
}
$cofactors[] = $row;
}
$cofactorMatrix = new Matrix($cofactors);
$adjugate = $cofactorMatrix->transpose();
$result = [];
for ($i = 0; $i < $adjugate->rows; $i++) {
$row = [];
for ($j = 0; $j < $adjugate->cols; $j++) {
$row[] = $adjugate->data[$i][$j] / $det;
}
$result[] = $row;
}
return new Matrix($result);
}
public function regularizedInverse(float $lambda = 0.0, ?int $excludeIndex = null): Matrix {
if ($lambda <= 0.0) {
return $this->inverse();
}
if ($this->rows !== $this->cols) {
throw new Exception('Regularized inverse requires square matrix');
}
$regularized = [];
for ($i = 0; $i < $this->rows; $i++) {
$row = [];
for ($j = 0; $j < $this->cols; $j++) {
$value = $this->data[$i][$j];
if ($i === $j && ($excludeIndex === null || $i !== $excludeIndex)) {
$value += $lambda;
}
$row[] = $value;
}
$regularized[] = $row;
}
return (new Matrix($regularized))->inverse();
}
private function minor(int $row, int $col): Matrix {
$minor = [];
foreach ($this->data as $i => $r) {
if ($i === $row) {
continue;
}
$newRow = [];
foreach ($r as $j => $value) {
if ($j === $col) {
continue;
}
$newRow[] = $value;
}
$minor[] = $newRow;
}
return new Matrix($minor);
}
}
Implementation with RubixML
Now we will do the same thing, but the way it is usually done in real projects. We will use linear regression trained with the least squares method. The library itself will solve the optimization problem and find the weights analytically.
Example of code
<?php
use Rubix\ML\Datasets\Labeled;
use Rubix\ML\Regressors\Ridge;
// Data: [area, floor, distance to city center, building age]
$samples = [
[50, 3, 5, 10],
[70, 10, 3, 5],
[40, 2, 8, 30],
];
$targets = [
66_000,
95_000,
45_000,
];
// Create dataset
$dataset = new Labeled($samples, $targets);
// Create linear regression model (Ridge)
// With alpha = 1e-6, Ridge regression is almost equivalent to linear regression
$regression = new Ridge(1e-6);
// Train the model
$regression->train($dataset);