Strategic insights on large language models: An interview

LLMs are advanced AI models that use huge data sets and deep learning to understand, summarise and generate content. LLMs can be trained to interpret proprietary corporate data or to scrape information from the internet to produce human-like content. LLMs are used in a variety of applications, such as chatbots, virtual assistants and content (text and visual) generation tools — e.g., ChatGPT is powered by an LLM.

The value that they can add is the ability to bootstrap. 

If in my day-to-day I'm building a transformation model, I first have to create a project structure, then some base models and then get some data for testing purposes. This takes time and takes me away from the problem I actually want to solve. 

An LLM, with the right prompt, can create the entire project, with the right structure, and some base models within a couple of minutes. I can have the entire thing, end-to-end, done in 5-10 minutes and then start on the problem I need to solve.

Then there’s the document summarisation capabilities. It’s having something that can take a long document and help you work out the gist of what’s being said. It’s either an end-point or it’s a great way to set up a bunch of biases before you read the document to confirm or dismiss them. 

Using LLMs frees us up to think more, exercise our imaginations, be more creative and do exciting things. We get time back to solve the problems we want to solve.  

Let's assume that if you're building new features with AI you’ll also be able to build them faster. Ultimately, you’re going to be throwing users a stream of new features and functionality. It’s going to become a bit of a burden on them, in that they’re going to need to learn these features to do their jobs efficiently. How much time do new analysts need to become productive? 3 months, 6 months, 12 months? That would be suboptimal, right? 

LLMs are a great fit here. Take your documentation and feed it to the LLMs and then search it easily using natural language. If you can integrate LLMs into your product and provide context about the user, alongside the search you can help steer users of your new features in the right direction.

If you have a very complicated set of tools that you typically expect someone to take 6 to 12 months to get up to speed with, an intelligent Microsoft Clippy clone saying, “I think you're trying to do a report on this. Have you seen the documentation on this particular set of functions that I currently have?” can help accelerate the learning process.

Every conversation I have at the moment seems to touch on LLMs and it’s always under the guise of AI. What's interesting is there's a real split between the people that we're talking to.

Some are asking broad questions like, ‘I've got LLMs and I have a product and I HAVE to leverage them. How do they actually fit with our product? Is this a good use case?’

Then we have people who are aware of the difference between generative AI and AI. They are trying to work out how they can combine approaches to deliver products — utilising the bootstrapping capabilities of LLMs to enhance user experiences and reduce the amount of time it takes their users to become proficient with their products. These same folks are obsessed with aspects of repeatability and confidence, the things that you get from statistical machine-learning approaches. They care deeply about using the right tools in the right places.

We are looking across a number of different industries at the moment. The general take is they're all leveraging hosted options at the moment. So nobody, that I'm aware of at least, is building their own model. It's all about prompt engineering and exposing the right data to models in a safe way.

What they're trying to do is effectively take specific, complex domain knowledge and feed that into a system. Essentially, these products observe what the user does, in a non-creepy way, such as the context they’re operating in and the actions they take. It uses this information to learn about you and the tasks you perform. 

This kind of information can then be used to make suggestions as to how users can improve workflows and use the functionality they have missed. At the same time, if they get stuck they can simply ask the system a question, be that about software or the domains they’re in e.g. niche legal fields or logistics forecasting software. Each question and response is stored, helping to create a user-specific assistant. There’s potential to get a specific answer or get pointed in the right direction. 

It’s a case of cutting a very fine line between providing these kinds of services and mitigating the risk that surrounds them.

I’m not going to lie, I’m loving this space.

Leveraging LLMs can automate tasks for organisations and help employees to complete their work faster. This frees up resources to work on more complex and innovative projects that drive revenue. It can give organisations a competitive edge by enabling them to release new features, products and services into the market more quickly.