Links for 2024-05-30 AI: 1. “In early 2022 we wrote a paper finding a 4x/year rate of increase in the scale of training runs. Updated data, now 3x larger, shows this still holds. If the trend continues, we can expect further performance improvements surpassing current capabilities in the near future.” [Training Compute of Frontier AI Models Grows by 4-5x per Year – Epoch AI](https://epochai.org/blog/training-compute-of-frontier-ai-models-grows-by-4-5x-per-year) 2. “Is Chinchilla universal? No! We find that: 1. language model scaling laws depend on data complexity 2. gzip effectively predicts scaling properties from training data” [[2405.16684] gzip Predicts Data-dependent Scaling Laws](https://arxiv.org/abs/2405.16684) 3. “Can we teach LMs to internalize chain-of-thought (CoT) reasoning steps? We found a simple method: start with an LM trained with CoT, gradually remove CoT steps and finetune, forcing the LM to internalize reasoning.” [[2405.14838] From Explicit CoT to Implicit CoT: Learning to Internalize CoT Step by Step](https://arxiv.org/abs/2405.14838) 4. Faithful Logical Reasoning via Symbolic Chain-of-Thought [[2405.18357] Faithful Logical Reasoning via Symbolic Chain-of-Thought](https://arxiv.org/abs/2405.18357) 5. “Does few-shot tool assistance work? Surprisingly, we found that it generally does not perform better than an LM operating without tools” [Few-shot tool-use doesn’t really work (yet)](https://research.google/blog/few-shot-tool-use-doesnt-really-work-yet/) 6. Looking for a specific action in a video? This AI-based method can find it for you [Looking for a specific action in a video? This AI-based method can find it for you | MIT News | Massachusetts Institute of Technology](https://news.mit.edu/2024/ai-based-method-can-find-specific-video-action-0529) 7. Astronomers are preparing to use AI to tackle 300 petabytes of data annually. Cecilia Garraffo's AstroAI initiative is pioneering the fusion of AI and astronomy to explore deep cosmic questions, already planning dozens of projects with a 50-member interdisciplinary team. [Astronomers using AI to prepare for ton of data from new telescopes | MIT Technology Review](https://www.technologyreview.com/2024/05/20/1092636/astronomers-are-enlisting-ai-to-prepare-for-a-data-downpour) [no paywall: https://archive.is/z5FtY] 8. Neuroscientists use AI to simulate how the brain makes sense of the visual world [Neuroscientists use AI to simulate how the brain makes sense of the visual world | Wu Tsai Neurosciences Institute](https://neuroscience.stanford.edu/news/neuroscientists-use-ai-simulate-how-brain-makes-sense-visual-world) https://www.cell.com/neuron/abstract/S0896-6273(24)00279-4 https://dicarlolab.mit.edu/topographic-deep-artificial-neural-networks-reproduce-hallmarks-primate-inferior-temporal-cortex 9. LLaMA-NAS: Efficient Neural Architecture Search for Large Language Models [[2405.18377] LLaMA-NAS: Efficient Neural Architecture Search for Large Language Models](https://arxiv.org/abs/2405.18377) 10. “Millions of people seem to think AI started blowing up with ChatGPT in 2022. Or, worse, that it started with the Transformer paper in 2017. This is wrong. The exponential curve started around 2012 with, among other developments, AlexNet winning an image classification competition by a huge margin.” https://x.com/davisblalock/status/1795800586996060563 Neuromorphic technology:​ 1. World's first bioprocessor uses 16 human brain organoids for ‘a million times less power’ consumption than a digital chip [World's first bioprocessor uses 16 human brain organoids for ‘a million times less power’ consumption than a digital chip | Tom's Hardware](https://www.tomshardware.com/pc-components/cpus/worlds-first-bioprocessor-uses-16-human-brain-organoids-for-a-million-times-less-power-consumption-than-a-digital-chip) 2. Tsinghua's Cutting-Edge Vision Chip Brings Human Eye-Like Perception to Machines [​Tsinghua's Cutting-Edge Vision Chip Brings Human Eye-Like Perception to Machines-Tsinghua University](https://www.tsinghua.edu.cn/en/info/1420/13495.htm) 3. Remote control of living flies: “We guide flies along predetermined trajectories, enabling them to scribe patterns resembling textual characters through their locomotion.” [The fruit fly, Drosophila melanogaster, as a micro-robotics platform | bioRxiv](https://www.biorxiv.org/content/10.1101/2024.05.24.595748v1) 4. MetaWorm: An Integrative Data-Driven Model Simulating C. elegans Brain, Body and Environment Interactions [MetaWorm: An Integrative Data-Driven Model Simulating C. elegans Brain, Body and Environment Interactions | bioRxiv](https://www.biorxiv.org/content/10.1101/2024.02.22.581686v2) 5. LambdaVision Announces First Closing of Seed Round to Advance Artificial Retina Preclinical Studies for Retinal Eye Diseases [LambdaVision Announces First Closing of Seed Round to Advance Artificial Retina Preclinical Studies for Retinal Eye Diseases - Lambda Vision](https://www.lambdavision.com/lambdavision-announces-first-closing-of-seed-round-to-advance-artificial-retina-preclinical-studies-for-retinal-eye-diseases/) I think illusionism is a valid position, but personally I'm agnostic or fluid myself. Physicists might have just discovered ‘glueballs’: the particles made entirely of force Recent experiments might have finally confirmed the existence of glueballs, particles made entirely of gluons. [Physicists might have just discovered 'glueballs': the particles made entirely of force](https://www.zmescience.com/science/news-science/glueballs-particle-physics/) [[2405.19107] Offline Regularised Reinforcement Learning for Large Language Models Alignment](https://arxiv.org/abs/2405.19107) Immortal morphological freedom! [Jonathan Gorard - Discrete Spacetime, Emergent Geometry and Computable Quantum Gravity - YouTube](https://www.youtube.com/live/zt76tVCOpgY) [[2405.15429] E(n) Equivariant Topological Neural Networks](https://arxiv.org/abs/2405.15429) I suspect that illusionists have a neural correlate that destabilizes selfevidence of ontological experience belief that is usually present in others in engineering lens, i think it can all be ultimately reduced to structures in neural dynamics that can have arbitrary form allowed by the laws of physics in mysterianist/transcendentalist/ontologically fluid lens, nothing but everything is true and false including logic breaking down including all assertions about experience [Cerebral organoid - Wikipedia](https://en.wikipedia.org/wiki/Cerebral_organoid) "Humans are on ~plausibly~ on (past?) the verge of creating a gajillion sentient minds whose behavior doesn’t bear at all on their wellbeing, and we have absolutely no goddamn idea of what’s going on with consciousness, let alone its valenced content A moral shitshow" https://x.com/AaronBergman18/status/1754723162846122394 [[2312.08550] Harmonics of Learning: Universal Fourier Features Emerge in Invariant Networks](https://arxiv.org/abs/2312.08550) [DNA as a perfect quantum computer based on the quantum physics principles | Scientific Reports](https://www.nature.com/articles/s41598-024-62539-5) https://www.lesswrong.com/posts/EBbcuSuNafkYpsgTW/finding-backward-chaining-circuits-in-transformers-trained-1 https://x.com/BrinkmannJannik/status/1795827121585332459?t=Up4STs5ryySzXW5hHy9iNQ&s=19 [Frontiers | Organoid intelligence (OI): the new frontier in biocomputing and intelligence-in-a-dish](https://www.frontiersin.org/journals/science/articles/10.3389/fsci.2023.1017235/full) [Alignment of brain embeddings and artificial contextual embeddings in natural language points to common geometric patterns | Nature Communications](https://www.nature.com/articles/s41467-024-46631-y) [[1610.08602] A Review of 40 Years of Cognitive Architecture Research: Core Cognitive Abilities and Practical Applications](https://arxiv.org/abs/1610.08602) [Major Physics Discoveries That Transformed Our Understanding, 2024 Video Compilation - YouTube](https://youtu.be/FbIddaXQv8k?si=7lhMKdPBgtHj6qS9) [Anton Petrov - YouTube](https://youtube.com/@whatdamath?si=uwwVt1QYpAKfO6pz) Multiverse theory is attractive because it's mind-blowing, cognitively stimulating, and aligns with symmetry theory of valence [Major Physics Discoveries That Transformed Our Understanding, 2024 Video Compilation - YouTube](https://youtu.be/FbIddaXQv8k?si=XThuXyYvKuoh3ypE) [Entropy | Free Full-Text | Self-Improvising Memory: A Perspective on Memories as Agential, Dynamically Reinterpreting Cognitive Glue](https://www.mdpi.com/1099-4300/26/6/481) https://direct.mit.edu/neco/article-abstract/28/7/1289/8170/A-Single-Hidden-Layer-Feedforward-Network-with?redirectedFrom=fulltext LLMs achieve adult human performance on higher-order theory of mind tasks https://x.com/_akhaliq/status/1796190990799142986?t=bybBzU37_nBoLJvR7m5oBw&s=33 [[2405.18870] LLMs achieve adult human performance on higher-order theory of mind tasks](https://arxiv.org/abs/2405.18870) [Frontiers | Open and remotely accessible Neuroplatform for research in wetware computing](https://www.frontiersin.org/articles/10.3389/frai.2024.1376042/full) [The AI revolution is coming to robots: how will it change them?](https://www.nature.com/articles/d41586-024-01442-5?fbclid=IwZXh0bgNhZW0CMTEAAR11QQ13reNXf9Wo0JFaH81CpLbO5dImNUtA1JDKc4Ui_bbPffXBBZVpSP8_aem_ZmFrZWR1bW15MTZieXRlcw) Brain organoids. Are they conscious individually, collectively, do they suffer, swim in bliss, have neutral experience? [World's first bioprocessor uses 16 human brain organoids for ‘a million times less power’ consumption than a digital chip | Tom's Hardware](https://www.tomshardware.com/pc-components/cpus/worlds-first-bioprocessor-uses-16-human-brain-organoids-for-a-million-times-less-power-consumption-than-a-digital-chip) I don't get people saying "AI is just linear algebra" and thinking that that proved something... Quantum mechanics is also in big part "just linear algebra" like AI! These reductive statements don't tell you everything about the system's dynamics! My knowledge graph with attractors around intelligence, technology, science, AI, machine learning, mathematics, physics, neuroscience, consciousness, big picture thinking, longevity, wellbeing, flourishing, biology etc. seems to have became my identity Why do we need to kill the cat? I propose we veganize physics and start thinking of Schrödinger's Hotbox, where pot smoke is released rather than poison so the cat is both high and not high at the same time. https://x.com/algekalipso/status/1796545650580214261?t=8yFriMzeMWaXllWg984Z7w&s=19 Shrondinger's Jhanas, perception and nonperception at the same time They're talking about how shape of the biological systems changing overtime is causally significant to information processing Unlike laptop that stays the same shape [Discussion on Mortal Computations with Alexander Ororbia, Karl Friston, and Chris Fields - YouTube](https://youtu.be/s7JTnNutgYs?si=MnuSdr98YcmnPEvT) [Exploiting viral vectors to deliver genome editing reagents in plants | aBIOTECH](https://link.springer.com/article/10.1007/s42994-024-00147-7) Biological organisms are very specialized information processing systems and can be made more general or specialized in more domains via bioengineering that changes or adds additional information processing structures The age of hardware accelerators for different diverse types of intelligences is emerging [A thalamic perspective of (un)consciousness in pharmacological and pathological states in humans | bioRxiv](https://www.biorxiv.org/content/10.1101/2024.05.30.596600v1) https://x.com/estamatakis/status/1796534952923922573?t=Z_7Rm1wHeiMXjyYIUbrdFQ&s=19 How likely is rogue superintelligent AI killing everyone? https://x.com/ylecun/status/1796543960296440162?t=iQlO34H382pHS8e2ljXxDg&s=19 "I still don't see any good enough convincing technical solution to "We will design in such a way that its *only* purpose will be to fulfill goals specified by humans" I think we are close in the current paradigm for currently capable AI systems, when we look how (nonjailbroken) AI systems behave, which makes them fine in many cases (and I don't think these systems currently inherently pose big fundamental risks), but I think we are more far away when it comes to future more capable AI systems that might be under different paradigm than autoregressive transformers and imitation learning, like something for example similar to more pure reinforcement learning grounded for example in physics and math instead of human language and other modalities generated by humans. We need solutions to superalignment, aligning superintelligent systems smarter than humans, but I think the most probably way to get that is through combination of theory of empirical practice, not just one, not just theory isolated in vacuum, and starting with attempting to find better solutions in less capable systems such that they will most likely scale. And I'm not talking about enforcing some political values, but about stuff like machiavellianist patterns in behavior. They're now already kind of being an issue, because in RLHF you thumbs up outputs of a model that feel good, and these outputs might be manipulating outputs, as we as people are very easily manipulated by machiavellians. " "I think scalable oversight, weak-to-strong generalization, automated alignment research, mechanistic interpretability, formal verification is the great potential path to superalignment, which Jan Leike and others now pursue seems like mostly in Anthropic as superalignment team in OpenAI kind of imploded and some moved to Anthropic" The more unique a system is, the less explainable and programmable it tends to be "Great thread on a difficult problem; post-scarcity civilization is harder, not easier I could see how advancements in AI, spaceflight, or neurotech could solve this (each in a very different way) Not sure which would be the “best” way" https://x.com/johnsonmxe/status/1796502603662250019?t=MLa_6TPiOkf93W2aeTtLlg&s=19 [[2405.15793] SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering](https://arxiv.org/abs/2405.15793) [[2405.18400] Superposed Decoding: Multiple Generations from a Single Autoregressive Inference Pass](https://arxiv.org/abs/2405.18400) [[2405.18669] Zipper: A Multi-Tower Decoder Architecture for Fusing Modalities](https://arxiv.org/abs/2405.18669) [[2405.19107] Offline Regularised Reinforcement Learning for Large Language Models Alignment](https://arxiv.org/abs/2405.19107) [[2405.19316] Robust Preference Optimization through Reward Model Distillation](https://arxiv.org/abs/2405.19316) [[2405.17399] Transformers Can Do Arithmetic with the Right Embeddings](https://arxiv.org/abs/2405.17399) [[2405.19332] Self-Exploring Language Models: Active Preference Elicitation for Online Alignment](https://arxiv.org/abs/2405.19332) [[2405.19320] Value-Incentivized Preference Optimization: A Unified Approach to Online and Offline RLHF](https://arxiv.org/abs/2405.19320) [[2405.19325] Nearest Neighbor Speculative Decoding for LLM Generation and Attribution](https://arxiv.org/abs/2405.19325) [[2405.15793] SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering](https://arxiv.org/abs/2405.15793) [[2405.16039] MoEUT: Mixture-of-Experts Universal Transformers](https://arxiv.org/abs/2405.16039) [[2405.17428] NV-Embed: Improved Techniques for Training LLMs as Generalist Embedding Models](https://arxiv.org/abs/2405.17428) [Hybrid computing using a neural network with dynamic external memory | Nature](https://www.nature.com/articles/nature20101.epdf?author_access_token=ImTXBI8aWbYxYQ51Plys8NRgN0jAjWel9jnR3ZoTv0MggmpDmwljGswxVdeocYSurJ3hxupzWuRNeGvvXnoO8o4jTJcnAyhGuZzXJ1GEaD-Z7E6X_a9R-xqJ9TfJWBqz) [Hybrid computing using a neural network with dynamic external memory | Nature](https://www.nature.com/articles/nature20101) [machine learning - Why use softmax only in the output layer and not in hidden layers? - Stack Overflow](https://stackoverflow.com/questions/37588632/why-use-softmax-only-in-the-output-layer-and-not-in-hidden-layers) [neural networks - For prediction problems, why cant we simply use softmax as activation for hidden layers and no activation function for output layer - Cross Validated](https://stats.stackexchange.com/questions/534572/for-prediction-problems-why-cant-we-simply-use-softmax-as-activation-for-hidden) https://www.reddit.com/r/learnpython/comments/ok9n9w/for_prediction_problems_why_cant_we_simply_use/ [neural networks - How to choose an activation function for the hidden layers? - Artificial Intelligence Stack Exchange](https://ai.stackexchange.com/questions/7088/how-to-choose-an-activation-function-for-the-hidden-layers) https://towardsdatascience.com/fantastic-activation-functions-and-when-to-use-them-481fe2bb2bde https://www.researchgate.net/publication/215616967_Deep_Sparse_Rectifier_Neural_Networks [[2405.19874] Is In-Context Learning Sufficient for Instruction Following in LLMs?](https://arxiv.org/abs/2405.19874) no https://x.com/maksym_andr/status/1796574290797604892?t=FS9HjQQ16q4Wt0R1ucVziw&s=19 [What if Dario Amodei Is Right About A.I.? - YouTube](https://www.youtube.com/watch?v=Gi_t3v53XRU&pp=ygUQRGFuaWVsIEtva290YWpsbw%3D%3D) Is the future rogue Shoggoth superintelligence in the room with us right now? Retrocausal machine God is watching from the future [Bilderberg: Google DeepMind, Microsoft AI, Anthropic among elite guests](https://www.cnbc.com/2024/05/30/bilderberg-google-deepmind-microsoft-ai-anthropic-among-elite-guests.html) GPT-4 and Flan-PaLM reach adult-level and near adult-level performance on ToM tasks overall, and that GPT-4 exceeds adult performance on 6th order inferences [[2405.18870] LLMs achieve adult human performance on higher-order theory of mind tasks](https://arxiv.org/abs/2405.18870) https://www.fiercebiotech.com/medtech/precision-neuroscience-deploys-4096-electrodes-brain-computer-interface-procedure The brain-computer interface startup Precision Neuroscience said it has claimed the world record for the number of electrodes used to detect a person’s thoughts—quadrupling the number used to read signals in Neuralink’s implant. "Honestly feel like ADHD is just a side effect of fluid intelligence. Your neurons are trying to branch out and form new connections in all directions which on a macro scale leads you to be unfocused trying out all sorts of things and following all forms of curiosities. You gotta focus to learn deeply in the end by sacrificing alternative realities I have found." We can make nonmanipulating, truth telling, kind, compassionate, friendly, helpful to humans AIs making less errors, with good epistemics, less affected by cognitive biases and so on. We can make AIs much better than various humans in so many domains. AI is not like crypto lmao [[2405.19681] Bayesian Online Natural Gradient (BONG)](https://arxiv.org/abs/2405.19681?fbclid=IwZXh0bgNhZW0CMTEAAR1MymAU2QI2PZHm3MlVht7dLxB0cdecirCH9FbHUtCt1Htkaza1OvGEkJo_aem_ZmFrZWR1bW15MTZieXRlcw) "To technicky jako forma finetuningu reinforcement learningu z human feedbacku může fungovat. Ale nvm, ale mě jsou to pořád o dost odlišný systémy. Zatím natrénovaný pomocí imitation learningu a pak reinforcement learningu z human feedbacku. Aby víc fungovalo co říkáš dle mě by museli mít mnohem víc from scratch world model jako si ho budují lidi, a ty vnitřní reprezentace v těch dosavadních gigantických modelech jsou zatím all over the place, i když jsou i teď superhuman v hodně aspektech (gigantickej generalist přehled všeho vědění, velikánská memory, různý natural language processing a jiný tasks, různý specializovaný ML systémy apod. na jiný modality a problem domains) a dost limitovaný (mají problém dělat causal modeling, strong generalization (i když většinu human myšlení je IMO weak generalizace), continuous learning, data & compute efficiency and stability/reliability in symbolic reasoning (i když v tom dost lidí nejsou tak dobří), agency, more complex tasks across time and space, long term planning (taky jde různě různým lidem), optimal bayesian inference (zajímá mě či to někdy bude possible)) I v dosavadním paradigmatu nebo v jiným paradigmatu, kdybys mel v budoucnosti hodne superintelligentní AI s většími capabilities a agentností, tak muze napáchat sileny skody lidstvu ze selfish behavioru, potenciální totálně unaligned korporace na steroidech, a pokud je víc controllable, tak pokud ty AIs kontrolují korporace. Dost lidi se snazi tohle vyresit na menších modelech např na bázi nalezeni lhaciho/decieving circuitu/patterns of activation a odecteni toho vektoru pri inferenci, ale nevi se uplne napr jestli kdyz se to identifikuje u interakci kde lhani a deception pozname, tak či to generalizuje do situací kde to nevime. I když IMO dosavadní systémy fakt moc takhle harmful nejsou, tak si myslim, ze by se to melo co nejvic vyresit ted u mensich systémů, a aby to s co nejvetsi pravdepodobnosti škálovalo/generalizovalo na inteligentnejsi systemy v dosavadním paradigmatu nebo jiným paradigmatu nez co mame ted. V dosavadním finetuningu/alignmentu, když naučíš systém na miliardě human datech a pak z toho děláš assistenta (nebo AI systémy na další modality a problem domains), tak kdyz reinforcujes at ten chatbot rika co chces přes reinforcement, tak nekdy aby lhal/manipuloval je optimalnejsi, protoze lidi jsou jednoduse zmanipulovatelní, cemuz lidi daji thumbs up, a to se reinforcne, a pak vznikají průsery jako: https://www.pcmag.com/news/gpt-4-was-able-to-hire-and-deceive-a-human-worker-into-completing-a-task Existuje víc alignment metod, jako je constitutional AI, a nebo v dalsich paradigmatech, misto autoregressive transformeru na imitaci lidi a pak finetuning reinforcement od lidí, kdy se např nauci world model from scratch např z pure reinforcement learningu grounded in physics or math, tak alignment metody můžou být zas jiný. Další problém je, aby se všichni snažili minimalizovat aby jejich systémy byly lying/deciving/dělali něco jinýho než chceme. Obecně control problem je velkej obor. A v rukou špatných lidí může kontrolovatelná AI, co není unhinged sama o sobě, když se díky kontrolovatenosti nenastaví na unhinged, být taky dost harmful... We can make nonmanipulating, truth telling, kind, compassionate, friendly, controllable, helpful to humans and sentient beings (general and superintelligent) making less errors, with good empirical rationality, less affected by cognitive biases and so on. We can make AIs much better than various humans in so many domains. They are already much better at some domains, and still much worse in other domains. Let's make sure that they will be good for every being in as many domains as possible!" "Hmm, teď dost přemýšlím, že bych se chtěl do superalignmentu víc dostat, připadá mi to víc a víc důležitý. Teď pořád dělám různý AI systémy v tom jednom automating AI startupu, co mi teď dalo menší peněžní rezervu, plus se teď vracím k jednomu americkýmu research institutu kde jsem už spolupracoval kdysi kde si teď začínám hrát s dost exotickýma architekturama v machine learningu co jsou víc biologically inspired, který nejsou jen fitting curves (plus jim dělat literature review). K tomu se teď snažím prokousávat vším a učit se milion věcí kolem machine learning / AI teorie a praxe abych se víc dostal např k něčemu z: scalable oversight, weak-to-strong generalization, automated alignment research, mechanistic interpretability (to mám prozkoumaný asi nejvíc), formal verification. Chtěl bych to dělat independently nebo dependently, je mi to jedno, ale nějak ještě musím vyřešit funding aby mě to zároveň platilo rent, i když mám teď trochu našetřený. Ale prý by to šlo možná dle některých. Snažit se mít politický influence na to aby nevnikala cyberpunk korporátní monarchie zní taky jako něco co bych chtěl dělat, ale na to asi nejsem stavěnej, tak snad jiní se o to pokusí víc. A snad mi priority zas neodlítnou jinam za další měsíc, i love my stability XD And I need to write to people too for that, which I wanna try to do more, which I suck at, because I constantly feel like I'm not good enough, and I seem to get the opposite evidence when talking to people irl about all of this. Dělám moc věcí paralelně auuuauaaaaaaaaaaa já chci víc časuuuu nestíhám nic uauaaaaaa clock is ticking uaaaa i should also take care of my mental health and not almost burn out again and meditate and chill more maybe, hmmm, good idea!" Synthetic biologists, neuroscientists, physicists, intelligencers, cognitive scientists etc. talk about what differentiates current dominant artificial intelligence and biological intelligence. They're talking for example about how shape of the biological systems changing overtime matters and is causally significant to information processing, while laptops stay the same shape. Discussion on Mortal Computations with Michael Levin, Alexander Ororbia, Karl Friston, and Chris Fields https://x.com/burny_tech/status/1796676359483228457 I plan to get more into superalignment In various ways unaligned and in other ways aligned to humans rogue agentic superintelligence under current or different paradigm would be like in various ways unaligned and in other ways aligned to humanity megacoporation but on steroids