The correct answer is “An embedding.”

In the context of large language models (LLMs) such as GPT-3, GPT-3.5, or GPT-4, an embedding refers to a multi-dimensional numeric vector representation assigned to each word, token, or phrase. According to the Microsoft Azure AI Fundamentals (AI-900) study guide and Microsoft Learn documentation for Azure OpenAI embeddings, embeddings are used to represent textual or semantic meaning in a numerical form that a machine learning model can process mathematically.

Each embedding captures the semantic relationships between words. Words or tokens with similar meanings (for example, “car” and “automobile”) are represented by vectors that are close together in the multi-dimensional space, while unrelated words (like “tree” and “laptop”) are farther apart. This vector representation enables the model to understand context, similarity, and relationships between different pieces of text.

Embeddings are fundamental in tasks such as:

Semantic search: Finding documents or sentences with similar meaning.

Clustering: Grouping related concepts together.

Recommendation systems: Suggesting similar content based on text meaning.

Contextual understanding: Helping generative models produce coherent and context-aware text.

Option review:

Attention: A mechanism used within transformers to focus on relevant parts of input sequences but not a representation of words.

A completion: Refers to the generated text output from a model, not the internal representation.

A transformer: The architecture that powers models like GPT, not the vector representation of tokens.

Therefore, the correct term for a multi-dimensional vector assigned to each word or token in a large language model (LLM) is An embedding, which represents how meaning is numerically encoded and compared within language models.