Case 1. One term, different meanings (contextual sentence embeddings)

Comparing sentence embeddings where the same term appears in different contexts

In this case we use a transformer embedding pipeline to show that the same term can carry different meanings depending on context. We compute sentence embeddings with mean pooling and compare them with cosine similarity.

Example of code:


<?php

use function Codewithkyrian\Transformers\Pipelines\pipeline;

// Sentences where the word "key" appears in different contexts.
$sentences = [
    'He lost the key to his apartment',
    'She forgot her house keys',
    'The key to understanding the topic was unexpected',
    'The API uses a secret key',
    'An access key is used for API authentication',
    'The key idea helped explain the topic',
];

$embedder = pipeline(
    task: 'embeddings',
    modelName: 'Xenova/paraphrase-multilingual-MiniLM-L12-v2'
);

// Generate sentence embeddings with normalization and mean pooling.
$embeddings ??= $embedder($sentences, normalize: true, pooling: 'mean');

// Cosine similarity measures semantic closeness between two vectors.
function cosineSimilarity(array $a, array $b): float {
    $dot = 0.0;
    $normA = 0.0;
    $normB = 0.0;

    foreach ($a as $i => $v) {
        $dot += $v * $b[$i];
        $normA += $v ** 2;
        $normB += $b[$i] ** 2;
    }

    if ($normA == 0.0 || $normB == 0.0) {
        return 0.0;
    }

    return $dot / (sqrt($normA) * sqrt($normB));
}