Landscape of experiements of quantum gravity https://twitter.com/Kaju_Nut/status/1769065754769473859?t=rizywD7Zvg0P-ia0VTHCLw&s=19 https://twitter.com/ecardenas300/status/1769396057002082410 Mindblowing music https://twitter.com/burny_tech/status/1769154854818037977 https://twitter.com/MLStreetTalk/status/1769434043232243851 https://twitter.com/burny_tech/status/1769427883607417117 https://twitter.com/burny_tech/status/1769521823555739970 Quality dataset is all you need https://twitter.com/teortaxesTex/status/1769469624108695660 https://twitter.com/burny_tech/status/1769763717372096762 And yet we're exploring the features learned by both artificial and biological neural networks without needing to go into the quantumness https://twitter.com/burny_tech/status/1769804367983427665 How do we approach the complexity of life as a nonequilibrium system? Using physics and, in particular, landscape and flux theory, many systems, from cells to ecology and cancer, can be adequately represented and modelled. https://twitter.com/ricard_sole/status/1769836061679611986 https://twitter.com/burny_tech/status/1769963434798465518 Will AI training in the future be simulating whole universes with societies (Matrix) with embodied AI agents getting trained across parallel timelines at the same time? Is there a cost function for us humans under which we help to train in this universe that might be simulated? https://twitter.com/burny_tech/status/1770207631581364624 https://twitter.com/burny_tech/status/1770209519509135542 https://alpera.xyz/blog/1/ metalearning https://twitter.com/GPTJustin/status/1770233883629654503?t=SyEgKWD6140gLen0sQy8wQ&s=19 [GitHub - tensorush/Awesome-Physics-Learning: 😎 🌌 Collection of the most awesome Physics learning resources in the form of notes, videos and cheatsheets.](https://github.com/tensorush/Awesome-Physics-Learning) https://twitter.com/burny_tech/status/1770497046421651520 https://twitter.com/burny_tech/status/1770496133791465874 57. Global challenges: Science diplomacy, evidence-based policymaking, transdisciplinary research Reasoning: [Human-like systematic generalization through a meta-learning neural network | Nature](<https://www.nature.com/articles/s41586-023-06668-3>), [[2305.20050] Let's Verify Step by Step](<https://arxiv.org/abs/2305.20050>), [[2302.00923] Multimodal Chain-of-Thought Reasoning in Language Models](<https://arxiv.org/abs/2302.00923>), [[2310.09158] Learning To Teach Large Language Models Logical Reasoning](<https://arxiv.org/abs/2310.09158>), [[2303.09014] ART: Automatic multi-step reasoning and tool-use for large language models](<https://arxiv.org/abs/2303.09014>), [AlphaGeometry: An Olympiad-level AI system for geometry - Google DeepMind](<https://deepmind.google/discover/blog/alphageometry-an-olympiad-level-ai-system-for-geometry/>) (Devin AI programmer [Introducing Devin, the first AI software engineer](https://www.cognition-labs.com/introducing-devin) ) (Automated Unit Test Improvement using Large Language Models at Meta [[2402.09171] Automated Unit Test Improvement using Large Language Models at Meta](https://arxiv.org/abs/2402.09171) ) (GPT-5: Everything You Need to Know So Far [GPT-5: Everything You Need to Know So Far - YouTube](https://www.youtube.com/watch?v=Zc03IYnnuIA) ), (Self-Discover: Large Language Models Self-Compose Reasoning Structures [[2402.03620] Self-Discover: Large Language Models Self-Compose Reasoning Structures](https://arxiv.org/abs/2402.03620) https://twitter.com/ecardenas300/status/1769396057002082410 ) , (How to think step-by-step: A mechanistic understanding of chain-of-thought reasoning https://twitter.com/fly51fly/status/1764279536794169768?t=up6d06PPGeCE5fvIlE418Q&s=19 [[2402.18312] How to think step-by-step: A mechanistic understanding of chain-of-thought reasoning](https://arxiv.org/abs/2402.18312) ), [Magic](http://magic.dev) , (The power of prompting [Your request has been blocked. This could be due to several reasons.](https://www.microsoft.com/en-us/research/blog/the-power-of-prompting/) ), Flow engineering ( [State-of-the-art Code Generation with AlphaCodium - From Prompt Engineering to Flow Engineering | CodiumAI](https://www.codium.ai/blog/alphacodium-state-of-the-art-code-generation-for-code-contests/) ), Stable Cascade ( [Introducing Stable Cascade — Stability AI](https://stability.ai/news/introducing-stable-cascade) ), ( RankPrompt: Step-by-Step Comparisons Make Language Models Better Reasoners [[2403.12373] RankPrompt: Step-by-Step Comparisons Make Language Models Better Reasoners](https://arxiv.org/abs/2403.12373) ) Robotics: [Mobile ALOHA - A Smart Home Robot - Compilation of Autonomous Skills - YouTube](<[Mobile ALOHA - A Smart Home Robot - Compilation of Autonomous Skills - YouTube](https://www.youtube.com/watch?v=zMNumQ45pJ8>),) [Eureka! Extreme Robot Dexterity with LLMs | NVIDIA Research Paper - YouTube](<[Eureka! Extreme Robot Dexterity with LLMs | NVIDIA Research Paper - YouTube](https://youtu.be/sDFAWnrCqKc?si=LEhIqEIeHCuQ0W2p>),) [Shaping the future of advanced robotics - Google DeepMind](<https://deepmind.google/discover/blog/shaping-the-future-of-advanced-robotics/>), [Optimus - Gen 2 - YouTube](<[Optimus - Gen 2 - YouTube](https://www.youtube.com/watch?v=cpraXaw7dyc>),) [Atlas Struts - YouTube](<https://www.youtube.com/shorts/SFKM-Rxiqzg>), [Figure Status Update - AI Trained Coffee Demo - YouTube](<[Figure Status Update - AI Trained Coffee Demo - YouTube](https://www.youtube.com/watch?v=Q5MKo7Idsok>),) [Curiosity-Driven Learning of Joint Locomotion and Manipulation Tasks - YouTube](<[Curiosity-Driven Learning of Joint Locomotion and Manipulation Tasks - YouTube](https://www.youtube.com/watch?v=Qob2k_ldLuw>)) Memory: larger context window [Gemini 10 million token context window](<https://twitter.com/mattshumer_/status/1759804492919275555>), or [vector databases](<https://en.wikipedia.org/wiki/Vector_database>) (Larimar: Large Language Models with Episodic Memory Control [[2403.11901] Larimar: Large Language Models with Episodic Memory Control](https://arxiv.org/abs/2403.11901) ) Hardware efficiency: extropic [Ushering in the Thermodynamic Future - Litepaper](https://www.extropic.ai/future) , tinygrad, groq https://twitter.com/__tinygrad__/status/1769388346948853839 , ['A single chip to outperform a small GPU data center': Yet another AI chip firm wants to challenge Nvidia's GPU-centric world — Taalas wants to have super specialized AI chips | TechRadar](https://www.techradar.com/pro/a-single-chip-to-outperform-a-small-gpu-data-center-yet-another-ai-chip-firm-wants-to-challenge-nvidias-gpu-centric-world-taalas-wants-to-have-super-specialized-ai-chips) , new Nvidia GPUs [NVIDIA Just Started A New Era of Supercomputing... GTC2024 Highlight - YouTube](https://www.youtube.com/watch?v=GkBX9bTlNQA) , etched [Etched | The World's First Transformer Supercomputer](https://www.etched.com/) , https://techxplore.com/news/2023-12-ultra-high-processor-advance-ai-driverless.html , Thermodynamic AI and the fluctuation frontier [[2302.06584] Thermodynamic AI and the fluctuation frontier](https://arxiv.org/abs/2302.06584) , analog computing https://twitter.com/dmvaldman/status/1767745899407753718?t=Xe5sDPbrBVayUaAGX4ikmw&s=19 neuromorphics [Neuromorphic engineering - Wikipedia](https://en.wikipedia.org/wiki/Neuromorphic_engineering) , [Homepage | Cerebras](https://www.cerebras.net/) Meta learning: [Meta-learning (computer science) - Wikipedia](https://en.wikipedia.org/wiki/Meta-learning_(computer_science)) (Paired open-ended trailblazer (POET) https://alpera.xyz/blog/1/ ) Generalizing: [[2402.10891] Instruction Diversity Drives Generalization To Unseen Tasks](<https://arxiv.org/abs/2402.10891>), [Automated discovery of algorithms from data | Nature Computational Science](<https://www.nature.com/articles/s43588-024-00593-9>), [[2402.09371] Transformers Can Achieve Length Generalization But Not Robustly](<https://arxiv.org/abs/2402.09371>), [[2310.16028] What Algorithms can Transformers Learn? A Study in Length Generalization](<https://arxiv.org/abs/2310.16028>), [[2307.04721] Large Language Models as General Pattern Machines](<https://arxiv.org/abs/2307.04721>), [A Tutorial on Domain Generalization | Proceedings of the Sixteenth ACM International Conference on Web Search and Data Mining](<https://dl.acm.org/doi/10.1145/3539597.3572722>), [[2311.06545] Understanding Generalization via Set Theory](<https://arxiv.org/abs/2311.06545>), [[2310.08661] Counting and Algorithmic Generalization with Transformers](<https://arxiv.org/abs/2310.08661>), [Neural Networks on the Brink of Universal Prediction with DeepMind's Cutting-Edge Approach | Synced](<https://syncedreview.com/2024/01/31/neural-networks-on-the-brink-of-universal-prediction-with-deepminds-cutting-edge-approach/>), [[2401.14953] Learning Universal Predictors](<https://arxiv.org/abs/2401.14953>), [Spectral bias and task-model alignment explain generalization in kernel regression and infinitely wide neural networks | Nature Communications](<[Spectral bias and task-model alignment explain generalization in kernel regression and infinitely wide neural networks | Nature Communications](https://www.nature.com/articles/s41467-021-23103-1>)) (Natural language instructions induce compositional generalization in networks of neurons [Natural language instructions induce compositional generalization in networks of neurons | Nature Neuroscience](https://www.nature.com/articles/s41593-024-01607-5) ) (FRANCOIS CHOLLET - measuring intelligence and generalisation [[1911.01547] On the Measure of Intelligence](https://arxiv.org/abs/1911.01547) https://twitter.com/fchollet/status/1763692655408779455 [#51 FRANCOIS CHOLLET - Intelligence and Generalisation - YouTube](https://youtu.be/J0p_thJJnoo) ) (Quiet-STaR: Language Models Can Teach Themselves to Think Before Speaking [[2403.09629] Quiet-STaR: Language Models Can Teach Themselves to Think Before Speaking](https://arxiv.org/abs/2403.09629) ) Search: AlphaGo ( https://twitter.com/polynoamial/status/1766616044838236507 ), AlphaCode 2 Technical Report ( https://storage.googleapis.com/deepmind-media/AlphaCode2/AlphaCode2_Tech_Report.pdf ) Then we have no idea how to turn off behaviors with existing methods [Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training \ Anthropic](<https://www.anthropic.com/news/sleeper-agents-training-deceptive-llms-that-persist-through-safety-training>), which could be seen lately the last few days with how GPT4 started outputting total chaos after an update [OpenAI's ChatGPT Went Completely Off the Rails for Hours](<https://www.thedailybeast.com/openais-chatgpt-went-completely-off-the-rails-for-hours>), Gemini was more woke than intended ( [Google Has a New 'Woke' AI Problem With Gemini](https://www.businessinsider.com/google-gemini-woke-images-ai-chatbot-criticism-controversy-2024-2) [The self-unalignment problem — AI Alignment Forum](https://www.alignmentforum.org/posts/9GyniEBaN3YYTqZXn/the-self-unalignment-problem) ), or every moment I see a new jailbreak that bypasses the barriers [[2307.15043] Universal and Transferable Adversarial Attacks on Aligned Language Models](<https://arxiv.org/abs/2307.15043>). Regarding definitions of AGI, this is good from DeepMind [Levels of AGI: Operationalizing Progress on the Path to AGI](https://arxiv.org/abs/2311.02462), or I also like, although quite vague, a pretty good definition from OpenAI: Highly autonomous systems that outperform humans at most economically valuable work, or this is a nice thread of various definitions and their pros and cons [9 definitions of Artificial General Intelligence (AGI) and why they are flawed](<https://twitter.com/IntuitMachine/status/1721845203030470956>), or also [Universal Intelligence: A Definition of Machine Intelligence](<https://arxiv.org/abs/0712.3329>), or Karl Friston has good definitions [KARL FRISTON - INTELLIGENCE 3.0](<[KARL FRISTON - INTELLIGENCE 3.0 - YouTube](https://youtu.be/V_VXOdf1NMw?si=8sOkRmbgzjrkvkif&t=1898>))) [Quanta Magazine](https://www.quantamagazine.org/michel-talagrand-wins-abel-prize-for-work-wrangling-randomness-20240320/) https://twitter.com/jackm2003/status/1770221903661437086?t=qLZMcVHcXsEuePja5wP6vw&s=19 Tool Use in LLMs survey https://twitter.com/omarsar0/status/1770497515898433896?t=cZLFtMkVTSK9iwRrE4PEmw&s=19 [[2403.09613] Reawakening knowledge: Anticipatory recovery from catastrophic interference via structured training](https://arxiv.org/abs/2403.09613) I have chronic hypercuriousitia https://twitter.com/burny_tech/status/1770574104870937030 As the metalanguage continues to develop, it will be important to refine and expand its vocabulary and structures based on feedback from domain experts and the identification of new patterns and principles that emerge from interdisciplinary research. The ultimate goal is to create a powerful tool for facilitating communication, collaboration, and discovery across the sciences, enabling researchers to build upon each other's work and develop a more integrated understanding of the natural and social world. Crafting SciLang in reality would be a monumental task, likely the work of many generations of collaboration among scientists, mathematicians, philosophers, and technologists worldwide. The aim of this conceptual outline is to highlight the potential structures and considerations such an endeavor would entail, rather than to provide a definitive blueprint. https://twitter.com/burny_tech/status/1770596021845864606 Again, this is just a rough example, and a real metalanguage would need to be much more sophisticated and flexible. However, I hope it gives you a better idea of how such a metalanguage might be structured to describe the integration of different scientific domains. From the various leaks I feel like GPT5 will have this on a spectrum: quick inference mode like most of current LLMs/LMMs (large multimodal models) and "let it iteratively reason using chain of thought, selfverification, searching, planning etc. hardcoded in the architecture" mode for more complex tasks, like how humans don't come up with a complex solution on one go https://twitter.com/burny_tech/status/1770612931652100337 https://twitter.com/burny_tech/status/1770617540664041579?t=T46QYtq1DGsvq3k8GPMeiQ&s=19 [[2403.12417] On Predictive planning and counterfactual learning in active inference](https://arxiv.org/abs/2403.12417) https://twitter.com/burny_tech/status/1770624255895425032 https://alpera.xyz/blog/1/ https://twitter.com/johnsonmxe/status/1770690203620913183 [[2311.12871] An Embodied Generalist Agent in 3D World](https://arxiv.org/abs/2311.12871?fbclid=IwAR01rbnXNt2cvoAkz5xJ0twYSU6ZB1kwf0s5ITuHw2DbyDXN1yH3bzOh1vo) [Out-of-Distribution Detection by Leveraging Between-Layer Transformation Smoothness](https://arxiv.org/html/2310.02832v2) [[2403.13248] Mora: Enabling Generalist Video Generation via A Multi-Agent Framework](https://arxiv.org/abs/2403.13248) Theory of everything by Claude https://twitter.com/irl_danB/status/1770609096179216878?t=OZXu4uFJW8C-URgOo3ckjQ&s=19 (Also check my previous comments regarding Sora: https://twitter.com/tydsh/status/1759292596206309559?t=_WQkXTUbFgv9mOlyUia9RA&s=19 https://twitter.com/tydsh/status/1770614875708166557?t=Wj2i26uf7mohg3yuxwowTg&s=19 https://twitter.com/MengdiWang10/status/1770509917168058569?t=t_GfYLaWtqjVcrFoKAeQtw&s=19 [[2403.12482] Embodied LLM Agents Learn to Cooperate in Organized Teams](https://arxiv.org/abs/2403.12482) [[2403.13793] Evaluating Frontier Models for Dangerous Capabilities](https://arxiv.org/abs/2403.13793) https://twitter.com/AlkahestMu/status/1770605602256921082?t=_q1YSTJp-dNV3q-QazFoaA&s=19 Claude ToE https://twitter.com/irl_danB/status/1770638251759428090?t=-iTGEbXynOQRmEMgE0xzkg&s=19 https://twitter.com/burny_tech/status/1770916607029637435 [[2403.09629] Quiet-STaR: Language Models Can Teach Themselves to Think Before Speaking](https://arxiv.org/abs/2403.09629) https://twitter.com/MiltosKofinas/status/1770881928540963177 [[2403.12143] Graph Neural Networks for Learning Equivariant Representations of Neural Networks](https://arxiv.org/abs/2403.12143) https://twitter.com/AllanZhou17/status/1758609463945388423 [[2402.05232] Universal Neural Functionals](https://arxiv.org/abs/2402.05232) [[2207.02098] Neural Networks and the Chomsky Hierarchy](https://arxiv.org/abs/2207.02098) Break the cycle. Become who you really want to become, no bullshit mode, no cultural cognitive dissonance mode, pure selfactualization aligned with your deepest desires. Fuck the status quo forcing your to submit, be yours. The only limits are the laws of physics, but maybe even those might be eventually rewrittable with good enough technology. Mindblowing music https://twitter.com/burny_tech/status/1769154854818037977 [Imgur: The magic of the Internet](https://imgur.com/SaTs5MP) Neuralink details https://twitter.com/neurosock/status/1771129732652040411 [4] [Concrete open problems in mechanistic interpretability: a technical overview — EA Forum Bots](https://forum.effectivealtruism.org/posts/EMfLZXvwiEioPWPga/concrete-open-problems-in-mechanistic-interpretability-a) [[1503.01036] Amorphic complexity](https://arxiv.org/abs/1503.01036) [[2403.13187] Evolutionary Optimization of Model Merging Recipes](https://arxiv.org/abs/2403.13187) These topics are at the cutting edge of modern mathematical research and are typically only accessible to specialists with a deep understanding of the relevant background material. [Quanta Magazine](https://www.quantamagazine.org/how-chain-of-thought-reasoning-helps-neural-networks-compute-20240321/) 2. Meta’s works on latent space planning/search https://x.com/tydsh/status/1770614875708166557 3. Reverse Training to Nurse the Reversal Curse [[2403.13799] Reverse Training to Nurse the Reversal Curse](https://arxiv.org/abs/2403.13799) 4. “Can we adaptively generate training environments with LLMs to help small embodied RL game agents learn useful skills that they are weak at? EnvGen, an effective+efficient framework in which an LLM progressively generates and adapts training environments based on feedback from the RL agent's intermediate successes/failures.” [[2403.12014] EnvGen: Generating and Adapting Environments via LLMs for Training Embodied Agents](https://arxiv.org/abs/2403.12014) 5. Monitoring AI-Modified Content at Scale: A Case Study on the Impact of ChatGPT on AI Conference Peer Reviews — “Our estimates suggest that 10.6% of ICLR 2024 review sentences and 16.9% for EMNLP have been substantially modified by ChatGPT” [[2403.07183] Monitoring AI-Modified Content at Scale: A Case Study on the Impact of ChatGPT on AI Conference Peer Reviews](https://arxiv.org/abs/2403.07183) 2. Physicists Finally Find a Problem That Only Quantum Computers Can Do [Quanta Magazine](https://www.quantamagazine.org/physicists-finally-find-a.../) 1. Surgeons in Boston transplanted a kidney from a genetically engineered pig into an ailing 62-year-old man, the first procedure of its kind. Organs from genetically engineered pigs one day may make dialysis obsolete. [First-ever transplant of pig kidney to patient a success — Harvard Gazette](https://news.harvard.edu/.../first-ever-transplant-of.../) 3. “I’m very skeptical about the idea that accumulation of somatic mutations could be the cause of aging. However, in this study authors demonstrated how strikingly accurate rate of mutations accumulation between species correlate (inversely) with species lifespan.” https://x.com/SsJankauskas/status/1769090729475772550 [Imgur: The magic of the Internet](https://imgur.com/bQkF7Rt) [Imgur: The magic of the Internet](https://imgur.com/ynV4g8C) [[2403.05286v1] LLM4Decompile: Decompiling Binary Code with Large Language Models](https://arxiv.org/abs/2403.05286v1)