Let's make a benchmark testing for AI systems that can nicely do causal modeling, strong generalization, continuous learning, adaptibility, abduction, data & compute efficiency and stability/reliability in symbolic reasoning, agency, more complex tasks across time and space, long term planning, optimal bayesian inference etc. The ultimate benchmark would be giving Ai systems all the information that Newton, Maxwell, Boltzman, Einstein, Feynman, Edward Witten, Von Neumann etc. had before their discoveries in physics or other fields and then seeing if the system could come up with the same or isomorphic discoveries. You can just do stuff, you don't need to wait for perfect time or anyone's permission, start now, screw any artificially created constrains Post deep learning? https://www.reddit.com/r/deeplearning/s/ZuuorOfKd1 You don't have to buy into society's pessimism We can engineer amazing fulfilling lifes on individual and systemic level [[2405.13956] Attention as an RNN](https://arxiv.org/abs/2405.13956) [An Introduction to Universal Artificial Intelligence - 1st Edition - M](https://www.routledge.com/An-Introduction-to-Universal-Artificial-Intelligence/Hutter-Catt-Quarel/p/book/9781032607153) [[2401.05343v1] Spectral Topological Data Analysis of Brain Signals](https://arxiv.org/abs/2401.05343v1) [[2405.12971] BiomedParse: a biomedical foundation model for image parsing of everything everywhere all at once](https://arxiv.org/abs/2405.12971?fbclid=IwZXh0bgNhZW0CMTEAAR3Qm621G9qx5VEly5UHwdelfHD3L_cceO_TJDpa3MaTwagE8pZo0LTF-LA_aem_ZmFrZWR1bW15MTZieXRlcw) [Dynamic spread of happiness in a large social network: longitudinal analysis over 20 years in the Framingham Heart Study | The BMJ](https://www.bmj.com/content/337/bmj.a2338) https://x.com/NTFabiano/status/1793976123451462039?t=5l26JRdyMXDbDWSmKMa9cA&s=19 Illusionism and realism is an illusion Accelerate status quo disruption toward more optimal local minima of civilizational cybernetic thermodynamic system cerebrolysin cognition https://x.com/bryan_johnson/status/1793693564964323489?t=UaV-3QhlZplvP6UJ_Opxnw&s=19 Language controllable diffusion world model https://x.com/ZhitingHu/status/1793829896978067955?t=cYC_y8J1lKs3CGqu93ERtQ&s=19 [Human connectome topology directs cortical traveling waves and shapes frequency gradients | Nature Communications](https://www.nature.com/articles/s41467-024-47860-x) [[2405.11273] Uni-MoE: Scaling Unified Multimodal LLMs with Mixture of Experts](https://arxiv.org/abs/2405.11273) https://x.com/_philschmid/status/1793994211823346116?t=1qeRNA-WoCoLE9_kXMg5iQ&s=19 https://www.cell.com/current-biology/abstract/S0960-9822(24)00471-8 [[2402.10877] Robust agents learn causal world models](https://arxiv.org/abs/2402.10877) [[2405.13640] Knowledge Graph Reasoning with Self-supervised Reinforcement Learning](https://arxiv.org/abs/2405.13640) [[2405.13817] Thermodynamic Natural Gradient Descent](https://arxiv.org/abs/2405.13817) [[2405.14860] Not All Language Model Features Are Linear](https://arxiv.org/abs/2405.14860) https://x.com/JoshAEngels/status/1793990584719548493?t=BdlYuw_C1hHwoS_GLG2QIg&s=19 [Mixed Evidence for Gestalt Grouping in Deep Neural Networks | Computational Brain & Behavior](https://link.springer.com/article/10.1007/s42113-023-00169-2) [Is the Universe Entirely Mathematical? Feat. Max Tegmark - YouTube](https://youtu.be/HGG4HmlotJE?si=5x5TvgJ4BiyibnlB) [Stanford Seminar - Topological Data Analysis: How Ayasdi used TDA to Solve Complex Problems - YouTube](https://youtu.be/x3Hl85OBuc0?si=uS43UiJgSt-u18tn) [The flattening of spacetime hierarchy of the N,N-dimethyltryptamine brain state is characterized by harmonic decomposition of spacetime (HADES) framework - PubMed](https://pubmed.ncbi.nlm.nih.gov/38778818/) My gender is superposition of all possible mathematics [[2405.14205] Agent Planning with World Knowledge Model](https://arxiv.org/abs/2405.14205) How's your topological neural network attention doing today? [[2405.13817] Thermodynamic Natural Gradient Descent](https://arxiv.org/abs/2405.13817) Hey superintelligence, teach me as much advanced mathematics in parallel as possible from first principles [[2405.14094] Attending to Topological Spaces: The Cellular Transformer](https://arxiv.org/abs/2405.14094) [How are memories stored in neural networks? | The Hopfield Network #SoME2 - YouTube](https://www.youtube.com/watch?v=piF6D6CQxUw) "Exact generative diffusion models are mathematically equivalent to modern Hopfield networks with annealing temperature. This can be used to model "episodic memories" and "semantic memories" in a single unified neural energy landscape." [Entropy | Free Full-Text | In Search of Dispersed Memories: Generative Diffusion Models Are Associative Memory Networks](https://www.mdpi.com/1099-4300/26/5/381) [Revealing non-trivial information structures in aneural biological tissues via functional connectivity | bioRxiv](https://www.biorxiv.org/content/10.1101/2024.05.09.593467v1.abstract) [[1606.07937] Universal Darwinism as a process of Bayesian inference](https://arxiv.org/abs/1606.07937) *Hits bong* What if telepathy is qualia bridge using wormholes "How well can we understand an LLM by interpreting its representations? What can we learn by comparing brain and model representations? Our new paper highlights intriguing biases in learned feature representations that make interpreting them more challenging!" [[2405.05847] Learned feature representations are biased by complexity, learning order, position, and more](https://arxiv.org/abs/2405.05847) https://x.com/AndrewLampinen/status/1793717890988085738?t=zqwmOCRE3dTLzsiDBNCruA&s=19 https://www.lesswrong.com/posts/js4LxFXDQaDdWNvEr/humans-aren-t-fleeb [Deciphering Glyph :: A Grand Unified Theory of the AI Hype Cycle](https://blog.glyph.im/2024/05/grand-unified-ai-hype.html) Ještě zapomněl reinforcement learning vlnu Která nám dala superhuman systémy v šachách, Go, Dotě,... [AlphaZero is a computer program developed by artificial intelligence research company DeepMind to master the games of chess, shogi and go. This algorithm uses an approach similar to AlphaGo Zero. On December 5, 2017, the DeepMind team released a preprint paper introducing AlphaZero, which within 24 hours of training achieved a superhuman level](<[AlphaZero - Wikipedia](https://en.wikipedia.org/wiki/AlphaZero?wprov=sfla1>)) Ale nvm, věci jako např tohle mi moc platau zatím nepřipadají [Finetuned Gemini 1.5 math benchmarks +20](<https://fxtwitter.com/OriolVinyalsML/status/1791521517211107515?t=BV3X1UIQxYIT-L1lR9muAg&s=19>) https://fxtwitter.com/LiamFedus/status/1790064966000848911?t=yXI3OKp2mRE1_oC-PWhQ8Q&s=19 We'll see I guess Ty nejnovější systémy přestávají být pure LLMs trained in traditional way A breakthroughs co se různé teď dejou nejsou jen LLMs, jako je alphageometry co je neurosymbolic systém [AlphaGeometry: An Olympiad-level AI system for geometry - Google DeepMind](https://deepmind.google/discover/blog/alphageometry-an-olympiad-level-ai-system-for-geometry/) Nebo graph neural networks na fotbal [TacticAI: an AI assistant for football tactics - Google DeepMind](https://deepmind.google/discover/blog/tacticai-ai-assistant-for-football-tactics/) Nebo alphafold And how about text to video Sora, how about direct more humanlike voice to voice in GPT-4o atd. [OpenAI Sora Just Supercharged Filmmaking! - YouTube](https://youtu.be/IS0xphCc5rI?si=2HdvfERYAsBq9ipH) [Dog meets GPT-4o - YouTube](https://youtu.be/HU_4vMu9xFI?si=-JG7x5UnkHVB6fh1) Ale vnějšek teď vidí jen industry LLM hype A i když často tihle lidi koukají jenom na industry LLM hype tak často zmíní (IMO insignificant) katastrofy v industry a zignoruji úspěchy I feel like nobody actually cares about Rabbit R1 and Humane AI Pin How about different AI systems more and more automating so many aspects if used in nonclunky way Boston dynamics is very recent too. And how about (humanoid) robotics becoming better and better and automating stuff in practice in for example warehouses [Amazon Begins Using Humanoid Robot in Warehouse - YouTube](https://youtu.be/Z3J250fr_V4?si=JrCZVNQGiF6nsiDK) Or this whole rabbithole [MIT gives AI the power to 'reason like humans' by creating new hybrid AI architecture | Live Science](https://www.livescience.com/technology/artificial-intelligence/mit-gives-ai-the-power-to-reason-like-humans-by-creating-hybrid-architecture) [[2006.08381] DreamCoder: Growing generalizable, interpretable knowledge with wake-sleep Bayesian program learning](https://arxiv.org/abs/2006.08381) [[2405.04484] OptPDE: Discovering Novel Integrable Systems via AI-Human Collaboration](https://arxiv.org/abs/2405.04484) We should do neurosymbolic reasoning on top of this whole mapped out latent space full of features [Mapping the Mind of a Large Language Model \ Anthropic](https://www.anthropic.com/research/mapping-mind-language-model) "According to David Pearce, the correlation between liking the idea of reprogramming the biosphere so that sentient beings are animated by information-sensitive gradients of bliss and believing that it is possible is very high. In other words, in most circumstances, having a strong negative immediate reaction against the idea of, say, helping elephants with chronic pain or gene-editing factory-farmed animals so they suffer less, or designing ecosystems so that there is no predation, strongly depends on whether you can imagine these interventions being done responsibly and successfully. Ithink we haven't been very good at emphasizing that a project like this would actually be done in multiple steps with many filters to ensure the safety of the enterprise. Enclosed and self-sustaining minibiospheres would be a very straightforward way of starting the project, where small containers with a lot of study and research tools to quantify the amount of suffering in the ecosystem and ensure its sustainability would be set in place, and it's outcomes evaluated over years. Nobody is talking about immediately getting rid of all predators or simply modifying species at a massive scale. We're talking about a slow and deliberate, scientifically grounded, and consensus-driven approach over most likely several generations, where we explore the metastability of a number of possible ecosystem modifications. Painting realistic pictures of ecosystems that are low in extreme suffering is part of the project. In fact, one of the inspiring pieces that I recommend checking out is "Welfare for Elephants" by David Pierce, who attempts to make a realistic budget for what it would actually cost to ensure that elephants have access to healthcare. It wouldn't bankrupt us as a society. More so, the costs will dramatically drop down with the advancement of AI and robotics, to the point that from the civilization's perspective, this is going to be relatively trivial in terms of resource allocation. We also tend to underestimate the amount of cultural change that happens between generations, whether it is gay marriage or attitudes towards slavery. A few generations might be enough for people to wake up to the realities of ecosystems and the suffering they entail. I don't think it's a non-realistic prospect, and the more you imagine it, the more likely it seems. It's just a matter of being open to the possibility without prejudice. To me, it seems very likely." [A Welfare State For Elephants? Costs and practicalities of comprehensive healthcare for free-living African Elephants](https://www.hedweb.com/abolitionist-project/elephantcare.html) if i had a penny where someone in some blogpost/tweet etc. only focused on LLMs and said that's basically the only place where currently AI advancements are happening and ignored all other AI research, and then only highlighted terrible stuff happening in industry and ignored all the positive usecases and advancements happening in industry or research, i would have 7 trillion dollars Math is everywhere Math is comfy Math is stabilizing Math is euphoric Math is grounding Math is explanatory Math is predictive Math is love Math is divine Math is everything Math is stimulating Math is existence Math is being Math is Metta to the process of metta itself: Meta metta "We introduce DIAMOND (DIffusion As a Model Of eNvironment Dreams), a reinforcement learning agent trained entirely in a diffusion world model." [[2405.12399] Diffusion for World Modeling: Visual Details Matter in Atari](https://arxiv.org/abs/2405.12399) https://x.com/EloiAlonso1/status/1793916382779982120 Links for 2024-05-19 AI: 1. Are model converging towards the same representation of the world? Neural networks, even if they are trained with different objectives on different data and modalities, seem to be converging to a shared statistical model of reality. The Platonic Representation Hypothesis [[2405.07987] The Platonic Representation Hypothesis](https://arxiv.org/abs/2405.07987) 2. Language Models Model Us: “On a dataset of human-written essays, we find that gpt-3.5-turbo can accurately infer demographic information about the authors from just the essay text, and suspect it's inferring much more.” https://www.lesswrong.com/posts/dLg7CyeTE4pqbbcnp/language-models-model-us 3. “So the race is really heating up to build a truly universal force field. This is one of those powerful ideas that people in the field of molecular simulation have been dreaming about for decades. What exactly is it and how far away are we?” https://x.com/TimothyDuignan/status/1791101632731873398 4. Stanford robot learns from humans [Stanford robot learns from humans - YouTube](https://www.youtube.com/watch?v=P7FRO_6L-iA) 5. It’s Time to Believe the AI Hype — “There’s universal agreement in the tech world that AI is the biggest thing since the internet, and maybe bigger.” [It’s Time to Believe the AI Hype | WIRED](https://www.wired.com/story/its-time-to-believe-the-ai-hype/) [no paywall: https://archive.is/11fWX] 6. Song: "As an AI Language Model (I Can't Do That)" https://x.com/venturetwins/status/1791493318578511925 AI safety: 1. “I am deeply troubled and concerned that there is still so much unknown capability in these models among even OpenAI staff.” https://x.com/kenshin9000_/status/1791785678935642246 2. "Is it enough that 4 of TOP LEADERS hand-selected by Sam Altman to build OpenAI have pulled the alarm?" https://x.com/liron/status/1791516294833123536 3. “I lost trust”: Why the OpenAI team in charge of safeguarding humanity imploded [Why the OpenAI superalignment team in charge of AI safety imploded | Vox](https://www.vox.com/future-perfect/2024/5/17/24158403/openai-resignations-ai-safety-ilya-sutskever-jan-leike-artificial-intelligence) 4. Response from Altman and Brockman to Jan Leike's departure https://x.com/gdb/status/1791869138132218351 5. “in regards to recent stuff about how openai handles equity” https://x.com/sama/status/1791936857594581428 Technology: 1. “After almost 10 years of hard work, tireless research, and a dive deep into the kernels of computer science, I finally realized a dream: running a high-level language on GPUs. And I'm giving it to the world!” [Higher Order Company](https://higherorderco.com/) 2. Stable, chronic in-vivo recordings from a fully wireless subdural-contained 65,536-electrode brain-computer interface device [Stable, chronic in-vivo recordings from a fully wireless subdural-contained 65,536-electrode brain-computer interface device | bioRxiv](https://www.biorxiv.org/content/10.1101/2024.05.17.594333v1) 3. IndieBio’s SF incubator lineup is making some wild biotech promises [IndieBio's SF incubator lineup is making some wild biotech promises | TechCrunch](https://techcrunch.com/2024/05/17/indiebios-sf-incubator-lineup-is-making-some-wild-biotech-promises/) Miscellaneous: 1. New study in Nature (N = 180,000) links a "vegetarian-type diet" with "anxiety symptoms, depressive symptoms, mental distress, psychotic experience, self-harm and trauma and a relatively lower well-being score." [Associations of dietary patterns with brain health from behavioral, neuroimaging, biochemical and genetic analyses | Nature Mental Health](https://www.nature.com/articles/s44220-024-00226-0) 2. "In the terms of game theory, liberalism is a losing strategy, one that is perpetually vulnerable to cheaters and defectors." [Should liberalism die? - Aporia](https://www.aporiamagazine.com/p/should-liberalism-die) 3. Women in academia (psychology) are more censorious than men. https://x.com/jonatanpallesen/status/1791382029898035223 - Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet - Thermodynamic Natural Gradient Descent - Attention as an RNN - DeepSeek-Prover - Wav-KAN - YOLOv10 - SimPO - Financial Statement Analysis with Large Language Models - Agent Planning with World Knowledge Model - (Perhaps) Beyond Human Translation: Harnessing Multi-Agent Collaboration for Translating Ultra-Long Literary Texts - MoRA - Your Transformer is Secretly Linear - Not All Language Model Features Are Linear - Reducing Transformer Key-Value Cache Size with Cross-Layer Attention - Layer-Condensed KV Cache for Efficient Inference of Large Language Models https://x.com/TheAITimeline/status/1794442990892532145?t=ZRy8_wBb78h926Tsm_BunA&s=19 We can engineer amazing fulfilling lifes on individual and systemic level Links for 2024-05-23 AI: 1. “The next industrial revolution has begun — companies and countries are partnering with NVIDIA to shift the trillion-dollar traditional data centers to accelerated computing and build a new type of data center — AI factories — to produce a new commodity: artificial intelligence,” said Jensen Huang, founder and CEO of NVIDIA. [NVIDIA Announces Financial Results for First Quarter Fiscal 2025 | NVIDIA Newsroom](https://nvidianews.nvidia.com/news/nvidia-announces-financial-results-for-first-quarter-fiscal-2025) 2. BootsTAP: Bootstrapped Training for Tracking-Any-Point [BootsTAP: Bootstrapped Training for Tracking-Any-Point](https://bootstap.github.io/) 3. LANISTR: Multimodal learning from structured and unstructured data [LANISTR: Multimodal learning from structured and unstructured data](https://research.google/blog/lanistr-multimodal-learning-from-structured-and-unstructured-data/) 4. “What if everyone, everywhere could have their own personal AI tutor, on any topic? We’re making learning more engaging and personal with our new family of models, LearnLM.” [How Google’s LearnLM generative AI models support teachers and learners](https://blog.google/outreach-initiatives/education/google-learnlm-gemini-generative-ai/) 5. “American AI harmlessness vs Chinese AI harmlessness” https://x.com/teortaxesTex/status/1793404511601512892 Miscellaneous: 1. Non-neural tissues also do sophisticated information processing. [Revealing non-trivial information structures in aneural biological tissues via functional connectivity | bioRxiv](https://www.biorxiv.org/content/10.1101/2024.05.09.593467v1.abstract) 2. “Results strongly suggest Neanderthal-derived DNA is playing a significant role in autism susceptibility across major populations in the United States.” [Enrichment of a subset of Neanderthal polymorphisms in autistic probands and siblings | Molecular Psychiatry](https://www.nature.com/articles/s41380-024-02593-7) 3. Compiler Explorer: “The basic idea behind CE is very simple. Just enter the source code in the pane on the left and the site will immediately show you the compiled output - typically assembly language - in a panel on the right.” [Compiler Explorer - by Babbage - The Chip Letter](https://thechipletter.substack.com/p/compiler-explorer) 4. Dyson Spheres: Astronomers Report Potential Candidates for Alien Megastructures—Here’s What to Make of It [Dyson Spheres: Astronomers Report Potential Candidates for Alien Megastructures—Here’s What to Make of It](https://singularityhub.com/2024/05/22/dyson-spheres-astronomers-report-potential-candidates-for-alien-megastructures-heres-what-to-make-of-it/) Politics: 1. “Low birthrates will devastate the global economy if not reversed. A look at what is happening with fertility numbers, why this is such a big economic problem, and what some of the solutions look like. A great primer for the uninitiated.” https://x.com/MoreBirths/status/1793394520727429326 2. How China selects its governing elite. — "When it comes to society’s most critical need and difficult task – running the country – our smartest five percent seem poorly represented. Meanwhile, the opposite is true in China." [Rule of the Five Percent - Aporia](https://www.aporiamagazine.com/p/rule-of-the-five-percent) Politics is blue tribe vs red tribe vs grey tribe clusters of statistical memetic coherence in dissonance with eachother Optimal representatational intelligence (Ability to model, compressed knowledge, predictive world models ) is at a sweetspot between order and chaos at optimal amount of complexity modelling patterns from sensory data in as optimally compressed was as possible Are corporations becoming the new monarchs battling for power? Learn all the patterns of the whole universe across all interacting scales [[2211.09119] Token Turing Machines](https://arxiv.org/abs/2211.09119) "I" love structureless groundlessness full of infinite freedom and potentiality with superposition of all possibilities, but in order to satisfy various objective functions from the hierarchy of basic needs and what various memeplexes from the environment install into my biological neural computer that generate dissonant error signals or consonant approval signals, I need some structure for implementing behaviors of one's organism and mental moves that satisfy these, so there is an optimal sweet spot in a good environment to be found https://www.techrxiv.org/users/684323/articles/680481-computational-dualism-and-objective-superintelligence [Thermodynamic Gradient Descent - YouTube](https://www.youtube.com/live/Hxv_y_bI3zA?si=QepW9PNTWBgm-Wi9) [nopilot.dev | nopilot.dev](https://nopilot.dev/) Factory farming [Dominion (2018) - full documentary [Official] - YouTube](https://youtu.be/LQRAfJyEsko?si=GK05N3qQB0vqVq0G) Machine intelligence is the new currency of power - Guide for Evaluating LLMs - Efficient Multimodal LLMs - Scientific Applications of LLMs - Enhancing Answer Selection in LLMs - Claude 3 Sonnet Interpretable Features - Agent Planning with World Knowledge Model https://x.com/dair_ai/status/1794744500558217537?t=yfYeM26xOHx3wS6NfqHkdg&s=19 Groundless ground of superposition of all philosophical positions in metaphysics, ontology, philosophy of mind etc. feels natural to "me". Philosophical superpositionism. Predictivist realism = models are differently ontologically real depending on how better they predict data from various lenses, which is on a spectrum https://www.sciencedirect.com/science/article/abs/pii/S0149763421005807