## Tags - Part of: [[Artificial Intelligence]] - Related: - Includes: - Additional: ## Significance - If the scaling hypothesis believers are right, as they have been to a certain degree so far, then [[superintelligence]] is coming soon. However, if they're wrong, all the hundreds of billions and potentially trillions of dollars invested could be viewed as one of the biggest bets that became one of the biggest wastes of resources in human history. [Can AI Scaling Continue Through 2030?](https://epochai.org/blog/can-ai-scaling-continue-through-2030) [X](https://x.com/EpochAIResearch/status/1826038729263219193), [$125B for Superintelligence? 3 Models Coming, Sutskever's Secret SSI, & Data Centers (in space)... - YouTube](https://youtu.be/QCcJtTBvSKk](https://youtu.be/QCcJtTBvSKk) Microsoft etc. wants to build 100 billion $ supercomputer for example. OpenAI [[o1]] showed new inference time scaling laws, so, we will see, how far will this go. ## Contents - The scaling hypothesis in artificial intelligence proposes that a model's cognitive abilities and performance scale with increased computational resources and data[1][2]. It suggests that as neural networks grow larger and are trained on more data, they not only become more accurate but can also develop new emergent capabilities that were not explicitly programmed[7]. The hypothesis predicts that continued scaling of model size and training data will lead to further improvements in AI performance and potentially human-level abilities across a wide range of tasks, though there is debate about its limitations and falsifiability[5][7]. Citations: [1] https://pubmed.ncbi.nlm.nih.gov/37823747/ [2] https://onlinelibrary.wiley.com/doi/full/10.1111/cogs.13341 [3] https://en.wikipedia.org/wiki/Widom_scaling [4] https://www.youtube.com/watch?v=NLKJdcb1E5I [5] https://philsci-archive.pitt.edu/23622/1/psa_scaling_hypothesis_manuscript.pdf [6] https://www.tcm.phy.cam.ac.uk/~bds10/phase/scaling.pdf [7] https://gwern.net/scaling-hypothesis [8] https://ocw.mit.edu/courses/8-334-statistical-mechanics-ii-statistical-physics-of-fields-spring-2014/31f360cf7db5b66068eacc5240c17aeb_MIT8_334S14_Lec6.pdf - [The Scaling Hypothesis · Gwern.net](https://gwern.net/scaling-hypothesis) - [The Bitter Lesson](http://www.incompleteideas.net/IncIdeas/BitterLesson.html?ref=blog.heim.xyz) - [[Images/63b01fe98bcc5a9069f976cc01038222_MD5.jpeg|Open: Pasted image 20240917183901.png]] ![[Images/63b01fe98bcc5a9069f976cc01038222_MD5.jpeg]]