Case 1. One term, different meanings (contextual sentence embeddings)

Comparing sentence embeddings where the same term appears in different contexts

In this case we use a transformer embedding pipeline to show that the same term can carry different meanings depending on context. We compute sentence embeddings with mean pooling and compare them with cosine similarity.

Example of use


                
<?php

include 'code-en.php';

$pairs = [
    [
        'index' => 0,
        'label' => 'apartment key ↔ apartment key',
        'sense' => 'completely identical phrases',
    ],
    [
        'index' => 1,
        'label' => 'apartment key ↔ house keys',
        'sense' => 'closely related everyday phrases',
    ],
    [
        'index' => 2,
        'label' => 'apartment key ↔ key to understanding',
        'sense' => 'same word, but different meaning',
    ],
    [
        'index' => 3,
        'label' => 'apartment key ↔ secret API key',
        'sense' => 'same word, but technical domain',
    ],
    [
        'index' => 4,
        'label' => 'apartment key ↔ API access key',
        'sense' => 'same word, but technical domain',
    ],
    [
        'index' => 5,
        'label' => 'apartment key ↔ main idea',
        'sense' => 'figurative meaning, semantically far',
    ],
];

foreach ($pairs as $pair) {
    $score = cosineSimilarity($embeddings[0], $embeddings[$pair['index']]);

    echo '0 ↔ ' . $pair['index'] . ' = ' . number_format($score, 4, '.', '') . "\t" . $pair['label'] . ($pair['index'] == 2 ? "\t" : "\t\t") . ' &raquo; ' . $pair['sense'] . PHP_EOL;
}

Documents:


            He lost the key to his apartment
She forgot her house keys
The key to understanding the topic was unexpected
The API uses a secret key
An access key is used for API authentication
The key idea helped explain the topic

Result: Memory: 0.001 Mb Time running: < 0.001 sec.


                0 ↔ 0 = 1.0000	apartment key ↔ apartment key		 » completely identical phrases
0 ↔ 1 = 0.6445	apartment key ↔ house keys		 » closely related everyday phrases
0 ↔ 2 = 0.2713	apartment key ↔ key to understanding	 » same word, but different meaning
0 ↔ 3 = 0.2772	apartment key ↔ secret API key		 » same word, but technical domain
0 ↔ 4 = 0.2772	apartment key ↔ API access key		 » same word, but technical domain
0 ↔ 5 = 0.1572	apartment key ↔ main idea		 » figurative meaning, semantically far