As many readers already know, the stock market is near an all-time high, but the market “breadth” is very narrow (i.e. it is not that the entire market is up on average; rather the value of a handful of big companies has risen enormously). These few companies (which include Google, Meta, Microsoft, Nvidia, and a few others) have attracted investors because they stand to make huge profits on AI. If you subtract the contributions of those companies, the markets are in decline.

Thus the question: what happens if the current boom in AI is a bubble?

In this post, I’ll make the case that the current version of AI has reached the end of its potential, and it still has serious problems. If that is true, the anticipated profits won’t materialize and the value of the companies involved will contract dramatically. Furthermore, there will be consequences beyond the financial markets.

My AI background. First, let me emphasize that I am by no means an expert on AI (or financial markets), so please take my ideas with a grain of salt. I do, however, have a long-standing interest in AI. I first became interested in the 1970s and 1980s, when I read extensively about the various branches of AI that were blossoming then. I was attracted to the work of Roger Shank (who showed that computers would need real-world knowledge in the form of story templates or “frames” in order to understand text) and Douglas Lenat (who understood that computers would need to have a base of “common sense” in order to be useful to us). I was also interested in the ways robots were being taught to interact with the unexpected obstacles of the real world.

Both Shank and Lenat died in 2023, by which time they must have seen that their work had very limited impact on the current commercial AI products (like ChatGPT). Nor has the robotics work spread significantly into the commercial AI world, as far as I can see.

A different area of early AI, called “neural networks”, was less interesting to me. Given training, it could solve some kinds of problems. But because of its self-modifying structure, it was impossible to say precisely how it worked (when it was successful) or why it didn’t work (when it failed). For those reasons, I found it less interesting and doubted it had much of a future. Little did I know.

Neural networks, deep learning, and large language models (LLMs). Though I wouldn’t have predicted it, the neural network idea has grown and morphed into what is now called the “deep learning” approach which underlies today’s familiar AI packages. One specific branch, called “large language models” (LLMs) is the basis of the popular text-oriented services, such as ChatGPT and the search summaries you get at the top of Google searches. While the deep-learning framework used for these LLMs is much more sophisticated, and uses vastly more training data (and more processing power), than the early neural networks, the problem remains: it is impossible for humans to know exactly how it works, or why it comes up with bad results (which, invariably, it sometimes does).

Apart from the lack of “transparency” about how they work, LLMs have another fundamental limit—and they are currently reaching it. To improve the performance of an LLM, you must feed it additional training data. That has worked for several generations of LLMs, but now it is becoming a problem. OpenAI (the maker of ChatGPT) has fed its LLM all the text it can find on the internet, including copyrighted books and articles that it obtained without permission and text that it generated by extracting audio from all of YouTube and converting it to text (again, without permission). Basically, OpenAI has trained its LLM on all the world’s text, and there are no more resources it could use to improve its performance further. (This problem is thoughtfully described in Karen Hao’s recent book Empire of AI).

These two failings—lack of transparency and lack of additional training materials—suggest to me that LLMs have reached the point where further progress in AI will depend on a different way forward.

While LLM-based products are becoming quite profitable when they are implemented into products such as automated customer service and tech support systems, the problems of LLMs will become increasingly obvious and will limit their growth. For example, would you trust an LLM-based automated financial advisor that occasionally made bad mistakes and that couldn’t explain the reasons for its recommendations?

I showed some specific examples of an LLM’s bad answers to questions in a previous blog post.

What about LLM alternatives? As I mentioned, the deep-learning/LLM approach is only one of several techniques used for AI. If LLMs reach their limits, couldn’t another approach work? The answer is probably “yes”, but for the moment LLMs are the only game in town. OpenAI has pushed to expand the LLM approach (and the corresponding approaches for images and speech) as far as possible, and it has had great success, as evidenced by the popularity of ChapGPT. That success, in turn, has pushed the other companies involved in AI to also focus on LLMs almost exclusively. Very little work has been done by these commercial firms on other approaches (although some academic work has continued).

Because of this single-minded focus on LLMs, I believe it will take a while—probably years— for any other approach to reach commercial success. In the meantime, the LLM-based products will no longer be able to make significant advancements. If that is true, we are about to enter a period of little or no progress in AI, and that slowdown will cause investors to have second thoughts about AI-related investing. If the investors pull back, the value of the AI companies will drop—and with them, the markets as a whole.

What about those data centers? LLMs need vast amount of processing power, and that’s the fundamental reason for the projections of huge numbers of data centers consuming vast amounts of electricity for the coming years. Other forms of AI don’t require the enormous amount of  training data and related processing that LLMs do. So if the use of LLMs tapers off, so will the need for data centers and the electricity they demand.

I’m getting into the realm of speculation here, and I could have a flawed understanding of LLMs and the state of the AI industry, but it seems to me that if LLMs are reaching the end of their potential, in a few years we might find that there is a surplus of electricity, not a shortage. The predicted demand for new natural-gas generating plants might disappear, with a profound impact on the natural-gas industry as well as the suppliers of generating equipment.

I’d be interested in hearing from others with a view about whether that is a real possibility.