Catastrophic forgetting. Don’t worry. I am not talking about politics. Catastrophic forgetting is a “lovely” feature of deep neural networks. They overwrite old knowledge when learning new things. There are different ways to mitigate this, but the problem persists.
If this problem were solved, we wouldn’t have new papers with new mitigation methods. The most recent one is from July 11, titled “Catastrophic Forgetting Mitigation Through Plateau Phase Activity Profiling.”
Science still doesn’t know how human memory works. I don’t even need to discuss different theories about how memories are “stored” in the brain to prove my point.
Related: Microsoft wants to help you live longer
Quote from a study, Beyond the Pump: A Narrative Study Exploring Heart Memory, should be enough: “Studies indicate that heart transplant recipients may exhibit preferences, emotions, and memories resembling those of the donors, suggesting a form of memory storage within the transplanted organ.”
What is the next big advancement for AI, artificial hearts?
I’ve already written that there is no intelligence without consciousness, and you can’t get consciousness by “building” intelligence. It does not work that way.
Alas, I have to admit, forgetting is something that Homo sapiens does have in common with neural networks. Humans forget things too quickly. This is why our civilization works the way it does.
Don’t get me wrong: I don’t hate “AI”. Google’s DeepMind is doing wonderful research on deciphering the way Dolphins communicate, which is a perfect example of how large language models are put to good use.
Unfortunately, most companies are super focused on using LLMs to build “real” AI chatbots, which will lead nowhere.
One company that has become hellbent on building the best AI in the universe is Meta Platforms.
Meta’s investing into AI has no limits.
TheStreet/Shutterstock
Is Meta panic investing in AI?
Don’t you hate those horrible buzzwords like quiet quitting, career catfishing, and other cringeworthy horrors of LinkedIn? Well, I have one buzzword for Meta free of charge: panic investing.
Meta (META) thinks the clock is ticking, and some other company might get the holy grail of all tech, such as AGI, superintelligence, or, to be comical, Skynet. Whoever makes that AI will be spared by the Terminators. That is how Meta’s investing strategy makes it look.
I covered Meta’s string of recent investments, which ended with what to me looks like a desperate move. But let’s go over that list again.
Related: How Apple may solve its Google Search problem
The investment speed shifted gear after Meta invested $14.8 billion in Scale AI and acquired a 49% nonvoting stake.
The company poached big AI names like Alexandr Wang, Koray Kavukcuoglu, Dr. Jack Rae, voice AI expert Johan Schalkwyk, OpenAI researchers Lucas Beyer, Alexander Kolesnikov, and Xiaohua Zhai. The desperate move was hiring Apple’s executive in charge of AI models, Ruoming Pang.
Meta isn’t stopping; if anything, it is shifting gears again and investing even more.
Zuckerberg can’t stand Musk having a bigger supercomputer
Mark Zuckerberg posted his plan for superintelligence on Threads and Facebook on Monday.
“We’re also going to invest hundreds of billions of dollars into compute to build superintelligence. We have the capital from our business to do this,” said Zuckerberg in his message.
Zuckerberg also said that Meta is building several multi-GW clusters. The first one, Prometheus, is supposed to go online next year. That is an interesting name choice, but it is a bit ominous. At least it isn’t Skynet.
More AI Stocks:
- Veteran fund manager raises eyebrows with latest Meta Platforms move
- Google plans major AI shift after Meta’s surprising $14 billion move
- Analysts revamp forecast for Nvidia-backed AI stock
According to Tom’s Hardware, Elon Musk’s xAI is buying an overseas power plant and shipping it to the U.S. xAI’s new data center is supposed to be powered by one million GPUs and up to 2 Gigawatts of power under one roof, which could power almost 2 million homes.
It looks like Zuckerberg needed to top this, so he also announced that Meta is building Hyperion, which will be able to scale up to 5GW over several years.
So, I guess that will consume power for about 4.75 million homes. Nice, but why stop there?
“We’re building multiple more titan clusters as well. Just one of these covers a significant part of the footprint of Manhattan,” said Zuckerberg.
If this pace continues, Meta should announce a supercomputer covering a Rhode Island-sized area before the end of the year.
Related: Oracle CEO sends blunt 2-word message on its business