AI - Preparing Intelligence Explosion ?

by Pininvest Analysis
AI - Preparing Intelligence Explosion ?
Leo Visions - Jurasic Park redux / Unsplash

AI is supportive of targeted applications relying on narrowly defined data sets - such as in legal, accounting, financial or medical specialities 

Such 'intelligent' features are recognized and appreciated in the business community

Artifical General Intelligence (AGI) is not framed in terms of specific functionalities but by a super-human ability to analyze and to act conclusively - a promise underwritten by the huge capital expenditures in data centers around the world, mainly in the U.S.

As I argue in this note, the promise may - or may not - be honored. The model architects might know about deeply transformative change but we, the public and even the regualtors, only have those investments and floods of red ink to consider

 

--------

 

AI leaders are outspending every major research endeavor in history 

Orchestrated in 1940 by the U.S. government, with a clear focus on nuclear weapons, the Manhattan project, coordinated by the National Defense Research Committee, used to be the ultimate strategic investment

Not any more...

Preparing for intelligence explosion ? -Source Bloomberg 

 

By its magnitude, investments of the hyperscalers dwarf the Manhattan project to build the atom bomb during World War II by a factor of 100

 

Do the strategists of hyperscale know something we do not ?

The author of Jurassic Park, Michael Crichton, has some questions...

"Most kinds of power require a substantial sacrifice by whoever wants the power. There is an apprenticeship, a discipline lasting many years. Whatever kind of power you want. President of the company. Black belt in karate. Spiritual guru. Whatever it is you seek, you have to put in the time, the practice, the effort. You must give up a lot to get it.

Now what is interesting about this process is that, by the time someone has acquired the ability to kill with his bare hands, he has also matured to the point where he won't use it unwisely. So that kind of power has a built-in control. The discipline of getting the power changes you so that you won't abuse it.


But scientific power is like inherited wealth: attained without discipline. You read what others have done, and you take the next step. You can do it very young. You can make progress very fast. There is no discipline lasting many decades. There is no mastery: old scientists are ignored. There is no humility before nature. There is only a get-rich-quick, make-a-name-for-yourself-fast philosophy. Cheat, lie, falsify--it doesn't matter. Not to you, or to your colleagues. No one will criticize you. No one has any standards. They all trying to do the same thing: to do something big, and do it fast.


And because you can stand on the shoulders of giants, you can accomplish something quickly. You don't even-know exactly what you have done, but already you have reported it; patented it, and sold it. And the buyer will have even less discipline than you. The buyer simply purchases the power, like any commodity. The buyer doesn’t even conceive that any discipline might be necessary."

These thoughts belong to the dying mathematician Ian Malcolm, a character in Michael Crichton's Jurassic Park, published in 1990

 

"Science has always said it may not know everything now but it will know, eventually.  But now we see that isn’t true. It is an idle boast. As foolish, and as misguided, as the child who jumps off a building because he believes he can fly.

We are witnessing the end of the scientific era. Science, like other outmoded systems, is destroying itself. As it gains in power, it proves itself incapable of handling the power. Because things are going very fast now... it will be in everyone's hands. It will be in kits for backyard gardeners. Experiments for schoolchildren. Cheap labs for terrorists and dictators. And that will force everyone to ask the same question - What should I do with my power? - which is the very question science says it cannot answer."

[Michael Crichton, Jurassic Park - Oct. 1990]

 

Crichton was refering to the Manhattan project and to the atomic bomb, and to genetic engineering which 'created' Jurassic Park 

Crichton's ruminations about science as unfettered, brilliant and out-of-control stack of unstable inventions in search of a purpose, bear eery ressemblance to the debate around Artificial Intelligence

Things keep going "very fast", the questions remain the same and the lack of clear answers is haunting

What - indeed - should AI do with its power?


AI research stands out in every dimension

  • Research, experimentation and release of AI models are in private hands, at least in the U.S.
  • Investments, on their huge scale, appear to be private 
  • Governmental oversight (or more precisely 'interference' ?) is clouded by multiple interpretations about what to expect for public safety (fretting of potential misuse) and what to achieve in the national interest (speed)

 

What about AI models

The framework to build scalable applications separates the 'decision logic' and the 'orchestration' of AI systems

The "AI-decision logic" repeatedly executes a block of code and this 'while code' keeps running iterations in loops as long as a specified condition remains true

The “orchestration layer” is the interface between the AI-decision logic and the non-AI operational tasks

The operational infrastructure lives around the 'decision logic' loop and it is the big deal

In Claude source code, discussed by Jiacheng Liu and others on ArXiv (April 2026)

  • the core agent 'reasoning' layer is actually quite thin (1.6% of Claude's codebase)
  • the other 98.4% is responsible for executing actions and is described by the authors as "a permission system with seven modes and an ML-based classifier, a five-layer compaction pipeline for context management, four extensibility mechanisms (MCP, plugins, skills, and hooks), a subagent delegation and orchestration mechanism, and append-oriented session storage"

The main differentiating factor between models is not the core agent but the way the operational infrastructure is layered to serve the client in the real world 

In fact, over time, the AI models themselves will become less important than all the infrastructure being built around them

 

As Ramsha Suhail tells it on Medium,

"Orchestration is the logic that holds an entire LLM application together.

In practice, an LLM application is rarely a simple prompt → response interaction.

Instead, it is a structured sequence of actions involving databases and vector stores, APIs and internal tools, memory and state management, conditional decision-making, and sometimes human oversight"

Source - Ramsha on Medium - Dec. 2025

 

By fast-tracking the release of their models to stay ahead of competition, the entire AI ecosphere echoes the concern voiced by Michael Crichton 

 "Because you can stand on the shoulders of giants, you can accomplish something quickly. You don't even-know exactly what you have done, but already you have reported it; patented it, and sold it"

 

Conceptually, from the standpoint of regulators having the public interest at heart, confusion is great

  • The running battle between fearmongers with their dire warnings and prophets of a glowing tomorrow leave everyone with a sense of urgency, but no road map 
  • Most perplexing, the defining features of AI's progress seem - suspicioulsy and conveniently - inventions on the fly

 

Clarity about best use of AI power would streamline regulatory guardrails on a sound basis 

A sense of purpose would also prove wrong Crichton assertion

"What should I do with my power? - which is the very question science says it cannot answer."

 

By showing what AI can do and what it cannot, model creators would steer clear of the mystification of ill-defined projections

Targeted applications supported by AI will demonstrate extraordinary new functionalities, supporting solid business models 

General Intelligence  is an argument in reverse: pharaonic investments are proof of extraordinary expectations (as yet unspecified)

The conundrum is not about to be resolved because giant capital expenditures become the sole defining feature of company valuation (in private market transactions)

 

Targeted applications - an example in Finance

Paul Uren (JP Morgan) remains fairly non-committal

"Our AI tools enable ​us to access more information and quickly ​synthesize it with our internal ​systems," without specifying which AI tools bankers ‌were using.

"We're finding that AI streamlines ​the preparation of content ​and materials, as well as helping bankers engage with more clients more efficiently."

 

In the opinion of Michael Nathanson, chairman and former CEO of Focus Financial Partners

Advice and services provided by wealth managers in a client's interest do not require AI and investment returns have always been measurable

AI will be an inflection point for wealth managers by leveling the playing field between walled intermediaries such as banks, investment advisors (RIAs) and brokers

AI adds value by making the quality of financial advice of the intermediaries quantifiable on a relative basis

 

Database analysis and transparency in client-first investment advice reflect the strength of AI in pattern identification

M. Nathanson provides a test case evaluating financial advice

AI features identify and measure crucial properties, such as

  • detecting recurring anomalies or data trends to anticipate and mitigate potential threats
  • analyzing a client's trading profile and financial goals against market volatility

 

AI makes financial advice more measurable 

  • judging performance relative to the level of risk taken
  • aligning a portfolio with a client's risk profile
  • benchmarking fees against industry norms
  • analyzing tax efficiency, diversification, and long-term probability of success
  • indentifying unexploited opportunities for advanced estate or tax planning
  • flagging recommendations that deviate from a client’s stated objectives

 

The roll-out of the full potential of AI in finance will be compelling

The value proposition of targeted AI agents is coming into sight, regulated, transparent and much "flatter" than the familiar financial architecture

 

Artificial General Intelligence (AGI)

AGI lives on a  promise - 'super-intelligence' - which is the pole-opposite of the targeted AI applications

  • this 'intelligence' may be impossible to regulate
  • there can be no transparency to what is held up as a 'black box'
  • this 'intelligence' will not flatten organizations - but raise one or a few master organizations to the pinnacle

 

The hyperscalers have laid out a radical growth plan with their huge investments and they cannot backdown 

Either AI models are close to achieve something we do not know 

Or...as Michael Crichton concluded, "As [science] gains in power, it proves itself incapable of handling the power. Because things are going very fast now..."

 

AGI success - in terms we cannot even fathom - would benefit the few businesses coming out on top

With this outcome, regulatory power, instead of prioritizing protection of the citizen, would project international dominance of just one State, the U.S.  or China

AGI failure would not leave the AI field in disarray, but compel model competition to go all in with tightly targeted applications (in fact they already are...) and hope for a profitable outcome ...

 

Artificial General Intelligence is a mantra for the competing American firms, justifying immense investments and consequenty equally immense valuation of the firms themselves

At risk of appearing deeply ignorant, one wonders what is meant by "Intelligence" ?

Under the assumption that 'intelligence' is of the human kind, AGI might promise 'super-intelligence' as a change of scale in the number of steps defining what 'reasoning' means

 

Linked to AGI, a closed-loop recursive self-improvement (RSI) is expected, within a very short timeframe, to allow AI to rewrite its own code, without human intervention

  • The implication is awesome - AGI resets the goal and RSI sets about to execute the intent - if the improvements sought by AGI can be defined in the first place
  • Even clearing the hurdle by defining 'improvements', will this 'super-intelligence' just go about launching recursive iterations ?

 

So many questions of principle make it easy for AI researchers to disagree fundamentally on the 'high probability' of disaster

Probabilities are based on one of many model-outcomes of 'disaster' which could be any sort of catastrophic event 

As I hope to discuss in the next note - AI - a Cretaceaous extinction ? - pragmatism will win the day

 

Whatever high-flowing concepts are introduced, the issues any regulator might raise are straightforward

Are the AI firms outcompeting one another keeping control over the models they launch ?

What are the implications of recursive self-improvement and how are human decision points designed for anything "involving irreversible actions, significant cost, or reputational risk"

The restrictions placed on Anthropic's model Mythos without justifications displayed U.S. global power, leaving the regulatory playbook closed for now...

 

Grappling with the interaction between AI  models and human intent leaves a lot of leeway, even for a non-specialist like myself 

With remarkable contributions of targeted AI agents, operating within solid frameworks, the mist of overblown 'super-intelligence' might dissipate 

AGI may, after all, not be the massive asteroid which destroyed all life on earth at the Cretaceous extinction - as I wil argue in my next note