Beyond prompt engineering: the shift to context engineering

“If prompt engineering is the scrappy startup’s idea of tweaking words in a prompt to produce outstanding results then context engineering is a more mature version of the idea” (source)

Prompt engineering, the art of massaging words to guide an LLM, is giving way to something bigger: context engineering. Where prompt engineering focuses on surface input, context engineering is about designing the entire environment in which the LLM operates: data, memory, tools, and retrieval systems. Recent commentary by Tobi Lütke and Andrej Karpathy on Twitter highlights this shift.

So what’s the difference? And why does it matter for developers building production-grade AI systems?

For those not in the AI loop, prompt engineering is the method of tweaking the input prompt of an LLM in order to produce desirable output/behavior. Context engineering is a superset of prompt engineering that expands its focus by ensuring the LLM has both sufficient and custom-tailored data to fulfill its task.

Although prompt engineering has engineering in its name it is more of an art or craft. Developers tweak prompts and experiment without any formal methods. A better name would be to call it “prompt crafting”.

Context engineering is the maturation of prompt engineering to actual engineering, focused on the various factors that AI developers need to write high-quality reliable LLM apps.

Let’s compare the two concepts with the typical prompts that you would write:

• Prompt engineering: “You are a music-savvy assistant. Recommend me a song.”
• Context engineering: “You are a music-savvy assistant. Here are instructions how to query Spotify API for song data (omitted). Based on memory I prefer the artist Pink Floyd and I do not like genre: XYZ. Recommend me a song.”

Data Sources

The former prompt is simple, uses role-playing (i.e. music-savvy assistant) and describes what is to be expected (i.e. a recommended song). Some questions remain:

• How does the LLM know what you are in the mood for? The output is too variable to be useful.
• How does the LLM know what genres of music that you like?

The latter prompt builds on the former and adds more to the context: tool calling and long-term memory.

• Tool Calling: The latter prompt calls Spotify API to retrieve up-to-date information on songs. Why call APIs? A major gotcha is that there is a lag of knowledge between when your chosen LLM was last trained and today.
• Long-term memory: Stores persistent user preferences across sessions. The information stored across conversations. The latter prompt states my preferences and dislikes. This information helps the LLM to curate its recommendations and thus improve the quality of its outputs.
• RAG & Vector DBs: Supplies high-relevance knowledge retrieval via semantic search.

This added context closes the knowledge gap, personalizes responses, and enhances accuracy.

The Goldilocks' Zone

How much context should you give LLMs?

• Too little and they have to guess what you want.
• Too much and they can get overwhelmed and hit token limits. This limit is analogous to the human short-term memory limit of 7 objects. For more in-depth, check out this IBM paper on LLM context windows.

Modern AI devs must tune not just what context to provide, but how much. Techniques like context pruning, salience ranking, and caching come into play here.

Prompt engineering got us started. It showed how small changes in wording could influence large language models in surprising ways. But as we push beyond demos and into production-grade applications, the limitations of prompt-only thinking become clear. Context engineering is the natural evolution, a more structured, deliberate approach that equips LLMs with the right information, tools, and memory to perform reliably. It brings real engineering principles into play: system design, data flow, and user modeling.

Context engineering asks developers to strike the right balance across the entire interaction system. It’s not just about what the model is told but what it knows, what it can access, and how it remembers.

In the same way that prompt crafting was an art, context engineering is a discipline. And it’s the foundation for building LLM-powered systems that are not only impressive, but repeatable, scalable, and truly intelligent.

The future of AI isn't just about what you ask, it’s about what the system knows whats just right when you ask it.

Connect with our team to learn more.