Skip to content

Embedding

An embedding is a numerical representation of information that allows semantic similarity and relatedness to be compared mathematically.

An embedding is a numerical representation of information designed so items with related meaning can be located or compared mathematically.

For text, embeddings can represent semantic relationships rather than only exact keyword matches.

OpenAI describes embeddings as vectors of numbers whose relative distance can be used to measure relatedness between text inputs. OpenAI embeddings guide

What It Means in Practice

Consider:

dog-friendly accommodation

and:

hotels that allow pets

The phrases contain different words but express related meaning.

Embeddings can represent those phrases in a way that allows a semantic-search system to recognize the relationship.

Common Uses

Embeddings can support:

Embeddings and Vector Databases

An embedding might look conceptually like:

[0.024, -0.812, 0.341, ...]

Real embeddings may contain hundreds or thousands of numerical dimensions.

A Vector Database or vector store can index these representations and efficiently locate similar vectors.

Embedding Is Not the Original Content

The embedding represents properties of the content.

It is not a readable replacement for the original text.

A RAG system usually retrieves the original content associated with relevant embeddings and supplies that content to the model.

Why It Matters

Embeddings make it possible to search information based on meaning rather than exact wording.

That is particularly useful for:

Common Problems

Common problems include:

  • Assuming semantic similarity guarantees factual relevance
  • Poor document chunking
  • Embeddings generated from outdated content
  • No metadata for filtering
  • Sensitive content indexed without considering access controls
  • Using vector search where simple keyword search would work better

How to Improve

Treat embeddings as part of a retrieval system, not as intelligence by themselves.

Pay attention to:

  • Source quality
  • Chunking
  • Metadata
  • Filters
  • Re-indexing
  • Access permissions
  • Retrieval evaluation