Building intelligent search with Sanity embeddings and OpenAIBuilding intelligent search with Sanity embeddings and OpenAIBuilding intelligent search with Sanity embeddings and OpenAI

Large Language Models (LLMs) have ushered in a new era of interaction — transforming how we pose questions and expect answers. While we’re surrounded by an overwhelming abundance of information, our questions are typically precise, contextual, and human in nature. Yet, traditional keyword-based search interfaces fall short. They can narrow a scope but cannot truly distill, offering only the best-match outputs authored by someone else, often detached from our intent.

The real challenge isn’t access to information — it’s delivering precision, relevance, and synthesis in a form the user actually needs.

In our recent work on reimagining search and chat experiences for nearform.com, we partnered with Sanity to explore how LLM-powered interfaces could reshape this interaction. Through Sanity Embeddings, we’ve been prototyping enhancements that align structured content with semantic understanding, blending the flexibility of structured content modeling with the nuance of natural language processing.

Sanity has long been a trusted partner to Nearform. Not only in our client work but also as the foundation of our digital presence. That deep familiarity made it the ideal platform for innovation as we explore what’s next in intelligent content interaction.

This is not just about improving search UX; it’s about moving toward intent-aware content retrieval that’s useful.

Clarity from nothing is still nothing, search requires substance. The quality of any AI-enhanced search experience is only as good as the content it's built upon. That’s why Sanity is Nearform’s preferred partner and platform when evaluating content management systems.

Its foundational model — treating content as relational, composable modules, which enables a uniquely flexible architecture. Engineers gain precise control over querying and rendering, while business users benefit from intuitive, extendable interfaces to create and manage content. In effect, it operates as a content OS. Because of this balance of flexibility and structure, we not only recommend Sanity to our clients, we rely on it ourselves. It powers nearform.com as both site builder and internal content management system. As a database for user-facing content, Sanity has proven to be an exceptional context layer, making it the ideal foundation for intelligent, semantic search experiences.

The means for accessing that context is different for machine learning tools than for a human reader. Embeddings act as the representation of values in a data context where information is stored as “vectors” (arrays of numbers) that represent data in a way that machine learning models can understand and effectively evaluate for similarity against a “human like” query. Translating your content into a vector database is a largely solved problem, but it does require engineering a platform and logic for updating your store and like everything in software these things are neither free in terms of cost or complexity. Here is where Sanity’s Embedding Index feature elevates our already exceptional content management and enables advanced feature sets for both our internal and external users.

The Embeddings process with Sanity is a borderline push button system where a content manager may configure the content from your “data lake” they wish to be queried in this way, embed it leveraging openAI, and store it in a hosted Pinecone database in AWS where it can be accessed via Sanity’s API. The content you index is kept up to date and new entries that qualify are indexed as well with no further operation on your part. This feature therefore reduces your workload tremendously  in terms of engineering data ingestion and updates and it just makes sense as your content is already in a database managed by Sanity and any equivalent operation would  just be an extension of that: It makes little sense to pull it down, transform it, and store it elsewhere.

So what does this give us? At this point we have our information stored in a way where a machine learning model can access it and an API so we can take a user’s question and query all the content we’ve indexed in Sanity to see what articles are relevant. Sanity as service is handling all the automation behind the indexing which takes many items off our plate.

1. Hosting a server
2. Hosting a Database
3. Embedding logic
4. Exposing a search API
5. Event listening logic with Sanity for content updates
6. Ongoing Maintenance / Engineering Bandwidth

Perhaps it is better to describe a use case: Nearform is a proficient consultancy with a decade of experience and we have a catalogue of blogs and case studies detailing our expertise and approach. A potential client visiting our site is more than likely unaware of any of that, and while there is certainly no impediment to filter and read through our materials, what if you could just ask us your question in a chat format and be presented with a brief summary of our experience and relevant articles?

1. You need the relevant information itself stored in an accessible format

2. You need a process for synthesizing large quantities of content.

There are options for approach but we have been highly satisfied with a composition of Sanity Embeddings retrieving the most relevant articles per user query and feeding them as context to OpenAI.

"Im the head of engineering for a fast food startup, we want to prioritize delivery over ordering from the restaurant and I need help designing and creating a mobile application”

We can send this and are rewarded with our list of relevant case studies and work on Mobile Applications:

matches: [
  { score: 0.787796557, value: "https://nearform.com/digital-community/design-to-dev" },
  { score: 0.7862885, value: "https://nearform.com/digital-community/doomscroll-design" },
  { score: 0.785304, value:  "https://nearform.com/digital-community/timing-and-talking-lessons-from-working-with-screen-readers"}
]

We could stop here! As a feature this is a great enhancement to our Sanity backed frontend and the work put in so far has been…

1. Enabling Embeddings with Sanity on our Articles in our studio

2. A search input backed by an API call to the embeddings index API

From this we’re getting a very low effort, high rewards, intelligent search. Top articles are returned ordered by relevance score and the user can get a greatly condensed collection of the type of work we do that applies to their question. This type of informational interfacing becoming standard and easy to use powerful extensions such as this only further cement Sanity as a gold standard choice.

You have great smart search against your content but how do we analyze a body of content and summarize in a digestible format?

The LLM tooling landscape is lush, the most prominent provider OpenAI is quite accessible and for our needs is exceptional for this task. At this point we really have everything we need to tie this up, we have great articles and a flow that is pulling down the most relevant content and with AI we need to provide that information as context and a prompt.

For our use we’ll set up an orchestration API endpoint to retrieve relevant articles, concatenate their content into a single string, and ship that to OpenAI’s chat completions API to do the heavy lifting.

We have orchestrated smart search and AI backed chat with just 3 API calls and I don’t know about you, but I write 3 API calls all the time. The efficacy of Sanity’s content management and indexing has opened the door to a rather sophisticated feature with very little effort.

Sanity, as a content operating system, delivers exceptional flexibility for both engineers and content authors. It’s a first-class choice for building advanced platforms with dynamic content strategies. Even used conventionally, it exceeds expectations but, its true strength lies in how well it adapts to the evolving ways users access and consume information.

As AI platforms like LLMs begin to offload complexity from developers, Sanity complements this shift by further abstracting operational overhead. Its architecture extends well beyond the scope of a traditional headless CMS, enabling powerful, user-facing features that align naturally with intelligent, AI-driven experiences.

When implemented with Nearform’s team, Sanity becomes more than a CMS, it becomes a strategic enabler, accelerating time to value for intelligent content delivery at scale.