Embedding
A numerical vector representation of text, images, or other data that captures semantic meaning in a high-dimensional space.
An embedding is a dense numerical vector — typically hundreds or thousands of numbers — that represents a piece of data (a word, sentence, image, or audio clip) in a high-dimensional space. The key property: semantically similar items have vectors that are close together. "Dog" and "puppy" will have nearly identical embeddings. "Dog" and "cryptocurrency" will be far apart.
Embeddings are generated by passing data through a neural network trained specifically to produce these representations. Text embedding models like OpenAI's text-embedding-3-large or Cohere's embed models output vectors of 768–3072 dimensions. These vectors are then stored in a vector database for efficient similarity search.
Common Uses
- Semantic search — find documents by meaning, not keywords
- RAG — retrieve relevant chunks to inject into LLM context
- Recommendations — find items similar to what a user liked
- Clustering — group similar documents automatically
- Classification — train lightweight classifiers on top of embeddings
Multimodal embeddings like CLIP put images and text into the same vector space — enabling cross-modal search where you describe an image in text and retrieve visually matching images. This is the foundation of many modern image search and generation systems.