Large language models: From abstraction to reality

Originally Posted, 8 October 2024 – Large language models: From abstraction to reality

Key Takeaways

• Large Language Models (LLMs) offer real-world impact, saving companies like Amazon many developer-years.
• Foundation models serve as a critical infrastructure layer for future AI innovations.
• LLMs improve productivity and efficiency in software development and business operations.
• The economic potential of LLMs will expand as major companies continue to invest heavily in infrastructure and talent.
• The evolution of AI models may lead to more specialised interfaces tailored to specific tasks.

• Related Products WisdomTree Artificial Intelligence UCITS ETF – USD Acc

 

The problem with Large Language Models (LLMs) is that it is difficult for a non-software engineer to visualise what it even is. This is made even more challenging when we say that the development and training of the biggest LLMs costs in the range of hundreds of millions of dollars.

Yet, we have seen the CEOs of some of the largest companies in the world indicate that they will be spending, as a group, more than $1 trillion in the coming years to build more computing infrastructure to run more of these models more feasibly^1&2.

We are always on the lookout for use cases or stories that allow us to translate the abstraction of something like an LLM into a true business impact. If we find enough of these cases, we might start seeing these impacts flow through into the broader statistics on economic productivity.

We were therefore excited to see the following come from Andy Jassy, CEO of Amazon, referring to Amazon’s Q system, which is essentially an LLM that is able to generate software code³:

The average time to upgrade an application to Java 17 plummeted from what’s typically 50 developer-days to just a few hours. We estimate that this has saved us the equivalent of 4,500 developer-years of work (yes, that number is crazy but, real).

In under six months, we’ve been able to upgrade more than 50% of our production Java systems to modernised Java versions at a fraction of the usual time and effort. And, our developers shipped 79% of the auto-generated code reviews without any additional changes.

The benefits go beyond how much effort we’ve saved developers. The upgrades have enhanced security and reduced infrastructure costs, providing an estimated $260 million in annualised efficiency gains.

4,500 developer-years??? $260 million in annualised efficiency gains??? These are big numbers. We recognise that we are still early in the journey of the AI revolution, but maybe Jassy’s conceptualisation of AI’s impact on his team’s developers inspires others to detail and publicise similar stories.

 

A conceptual roadmap for AI developments⁴

Figure 1, in our opinion, is instrumental in helping people quickly see another way to visualise the point of all of these LLMs. All knowledge workers can understand that their work consists of different buckets of tasks and each bucket can involve very different amounts of time.

The current versions of LLMS can response to simple questions or simple emails, but it is far less clear how these systems can build, from scratch, completely new reports or original ideas. That is not to say that they cannot – it’s simply to say that it is at a point where the level of review required on simple answers to simple questions is far different from the level of review required for a brand-new slide deck that could contain 60 original slides all developed by AI.

Figure 1: A conceptual roadmap for AI developments

Source: Stanley, Edward et al. “Mapping AI’s Rate of Change.” Morgan Stanley Research. 4 June 2024.

Are LLMs software, infrastructure, or both?

One of the things that some people call LLMs is ‘foundation models’. The word ‘foundation’ is thought-provoking in that, in many contexts, a foundation is something that you can build upon. If we think about value creation in a few ecosystems:

• Why are computers valuable? One could think about all the different hardware components and add up the value of each of these pieces to determine a measure of a computer’s value. However, if we think about the history of how computers proliferated, there was a time when it was thought that people wanted to build their own computers. The ease of use that Microsoft brought to the experience of doing specific things with the computer was immensely valuable, as well as the fact that many users adopted these software packages and learned to use them all simultaneously.
• Why is the internet valuable? One could say that the internet allows one to send information around the world far more easily. However, how do we think of Google, an example of a company that sought to organise the information available on the internet so that you could get what you needed far more quickly? How do you think we’d fare if we had to use the internet without search engines?
• When you think about the iPhone – is the value in the components of the phone and what the technology can specifically do, or is the value in all the different applications that are built on the iOS software?

In each of these cases, the real answer is likely that everything has some value, but the reason we often refer to products and services offered by companies above $1 trillion in market capitalisation is that different effects are falling back and multiplying exponentially. There are also gigantic network effects – nothing begets more users and more growth like a huge initial base of users.

Figures 2a and 2b give us a sense that we have seen things like this before:

• Figure 2a relates the revenues of IBM and Microsoft with the number of PC units sold globally. It makes sense that, first, IBM is selling PCs or enabling other companies to build and sell the so-called IBM clones. The number of PCs grows, and after a time, once the PC user base hits a certain level, Microsoft’s revenues take off because all of these PC users hear about and seek the benefits of using Microsoft’s operating systems. It wouldn’t make sense for Microsoft’s system to be valuable before PCs sit on almost every desk.
• Figure 2b relates US internet penetration to the revenues of Cisco and Amazon.com. Cisco’s devices allow more businesses and people to use the internet. As more and more people take advantage of the internet, suddenly, the value of what Amazon.com is providing – e-commerce – becomes more and more clear. It wouldn’t make sense for Amazon.com’s e-commerce effort to be valuable before a critical mass of users adopted the internet.

Figure 2a: Relationship of PC units sold globally to IBM and Microsoft’s revenues over time

Source: WisdomTree, Bloomberg. PC units sold data sourced from “Total Share: Personal Computer Market Share 1975-2010, Jeremy Reimer” and Gartner. Historical performance is not an indication of future results and any investments may go down in value.

Figure 2b: Relationship of US internet penetration to the revenues of Cisco and Amazon.com over time

Source: WisdomTree, Bloomberg, World bank. Historical performance is not an indication of future results and any investments may go down in value.

 

Conclusion: What will we build on top of foundation models?

The interesting thing about foundation models, at least in the second half of 2024, is that only the world’s largest and most profitable companies have the resources to continue developing and advancing them. Even if it looks like some of these are part of independent companies, the world’s largest firms tend to take major financial stakes that enable the appropriate and ever-increasing investments in talent and compute infrastructure needed.

We don’t know exactly what will come next – and we recognise that this can be a trillion-dollar question. However, we know that people tend to think about discrete, individual tasks and may not always need to access a model that can pass every major exam we have developed. We do, however, like the idea of more specialised interfaces based on more specific tasks that may then utilise parts of the broader models to get the job done.

However the picture evolves, we believe that the world’s largest companies connected to these foundation models will have an important role to play for years to come.

¹ Goldman Sachs, https://www.datacenterdynamics.com/en/news/goldman-sachs-1tn-to-be-spent-on-ai-data-centers-chips-and-utility-upgrades-with-little-to-show-for-it-so-far/
² Nvidia, https://www.linkedin.com/posts/leeps_nvidia-ceo-predicts-1-trillion-will-be-spent-activity-7101349410281836544-hB72/
³ Source: Excerpt from a Linked in Post from Amazon CEO Andy Jassy, as referenced from https://nextbigteng.substack.com/p/hello-ai-world-evolution-of-developer-economy-in-the-age-of-ai.
⁴ Source: Stanley, Edward et al. “Mapping AI’s Rate of Change.” Morgan Stanley Research. 4 June 2024.