I really wonder if AI would "scheme" if you didn't train them on scheming data and is it even possible to reliably classify scheming in training data to exclude it if you train on the whole internet or is it possible to remove it by ablation like many try or is the concept of scheming so interconnected in our structure of language that it's not possible if things are defined more relationally but at least the degree of scheming should be on a spectrum Claude is probably the only model that ive used a lot so far that feels like it has its own "soul", because of how well it was trained but that's just feeling when I use it i think this is very real [Imgur: The magic of the Internet](https://imgur.com/uFs7QK3) no other model makes me "feel understood" as much not sure if this is a good or bad thing its a great stack of transformers generating patterns in language that play with my neural dynamics very nicely i wonder to what degree people anthropomorphising AI systems is because of how well they generate language (or other modality) that play with their deeper parts of the brain because the more AI systems do this to me, the more I have these anthropomorphising urges i mean there are at least some similarities between AI systems and human brain when it comes to the algorithms and how they build represenations, when looking at NeuroAI, computational neuroscience and mechanistic interpretability results but there are countless differences too they're different entities, but some people equate them too much, and other people IMO make them too different but i still often wonder to what extend am I sometimes doing this 😄 : https://fxtwitter.com/erikphoel/status/1770421854731157556?t=MHLUc1PUmyFZfXJepNnHdQ&s=19 alphafold minimálně zabil některý části tradiční strukturalní biologie, ale to může být podobný tomu jak dneska už lidi dělají míň v assembleru díky high level jazykům s kompilátorama 😄 hmmm, LLMs zas zabily nějaký části tradiční NLP Ty skoky těch reasoning modelů např na nových matematických úlohách AIME, co vyšly po tom co už byly ty modely natrenovaný, mě přijdou ale celkem big Ale stejně si myslím že neurosymbolika bude na matiku lepší, tím jak je velká část matiky inherentně symbolic Hmm, plus už teďka AI pomohlo najít nějaký nový výsledky v matice apod., což byly většinou právě jiný neurosymbolický systémy jako FunSearch. Nebo možná bude větší breakthrough pro AI for STEM když se nám nějak podaří víc replikovat jak biologická kolektivní inteligence je schopná šíleně flexible problem solving schopnosti v hodně novel problem spaces, jako to např popisuje Michael Levin: Collective intelligence: A unifying concept for integrating biology across scales and substrates [Collective intelligence: A unifying concept for integrating biology across scales and substrates | Communications Biology](https://www.nature.com/articles/s42003-024-06037-4) Btw Veritasium udělal video na to jak AI vyhrálo Nobelovu za skládání proteinů (AlphaFold) a jaký velký to má implikace! 😄 [https://youtu.be/P_fHJIYENdI?si=9mk0pD8hB14KDN4c](https://youtu.be/P_fHJIYENdI?si=9mk0pD8hB14KDN4c) Nebo ještě vlastně před pár dnama vyšel neurosymbolický AlphaGeometry2 s gold-medalist performance in solving olympiad geometry na který ty dosavadní RL CoT reasoning modely nemaj v tomhle kontextu [[2502.03544] Gold-medalist Performance in Solving Olympiad Geometry with AlphaGeometry2](https://arxiv.org/abs/2502.03544) i think the future of general AI systems is neurosymbolic multimodal hybrids of all sorts of AI architectures, where deep learning and its ancestors will be part of it but not the whole story at all pokud chceme fakt co největší obecnost do jistý míry tím směrem jdeme modernější obecnější systémy mají menší a menší procento celkový architektury deep learning, plus je tam víc a víc reinforcement learning Chollet tímhle směrem taky dost argumentuje [https://www.youtube.com/watch?v=w9WE1aOPjHc](https://www.youtube.com/watch?v=w9WE1aOPjHc) ale k tomu moje predikce je že něco na způsob neuromorphic computing nebo liquid neural networks (víc biology+physics based), víc kombinovaný se symbolikou, bude do 5 let mnohem větší součást AI LiquidAI dělá na architektuře založený na liquid neural networks [Liquid AI: Build capable and efficient general-purpose AI systems at every scale.](https://www.liquid.ai/) deepseek-r1 boom: Btw myslím že je to trochu overblown, ten čínskej model podle dost nekontaminovaných benchmarků pořád horší než o1 od OpenAI A nějakou distillaci určitě dělali, ale to je minimální část toho trénování A to teď dělají všechny AI companies 😄 Ale většina trénování je v tom reinforcement learningu, který se netrenuje na těch lidských nebo syntetických datech Už vznikají replikace A ta majoritní část trénování s reinforcement learningem bez těch lidských a AI dat je ten důvod proč je to tak big Tipuju že mainstream se tohodle tak chytl z velký části protože DeepSeek dal o něco horší model než o1 ale zadarmo, takže normíci o1 level modely do teď neznali. Jako reakci do 24 hodin OpenAI spustí free model na úrovni o1 btw (o3 mini) zároveň furt vidím lidi psát rip nvidia kvůli tomuhle protože už nejsou potřeba čipy to mi ale nedává smysl už vůbec 1) Inference stojí šílený prachy, víc čipů se kupuje kvůli inferenci než kvůli trénování 2) U trénování je large scale scaling of reinforcement learning (což se dělá u tohoto novýho paradigmatu) overall ještě víc expensive než klasický škálování (human generates data and machine learns -> machine generates data and machine learns), a scaling laws tu zatím vůbec nehitly limit (see benchmark o3 results and associated costs) 3) The more compute efficient and accessible AI gets, the more use cases start becoming economically viable, the more we'll deploy AI, and the more compute we'll need, since people will want it more and more (což se už dejě s Anthropicem, co mají přístup k big tech GPUs, a servují taky částečně reasoning model co byl cheap na training) Dost researchers si myslí že ten "stolen data" claim cope od OpenAI co teď všichni berou at face value je dost nepradvěpodobnej. Většina trénování je v tom reinforcement learningu, který se netrenuje na těch lidských nebo syntetických datech. Ta majoritní část trénování pomocí reinforcement learningu bez těch lidských a AI dat je ten důvod proč je to tak big. Deepseek R1's original paper shows how they're using pure reinforcement learning via GPRO. This is different from previous approaches which either require a human to rate the outputs, or example outputs. The most important part of the training pipeline doesn't need external training data. There are already replications that are slowly confirming that this method works. Je možný že distilaci použili na část toho trénovaní nebo na celý trénování, ale vzhledem k tomu, že vznikají replikace, kde tu emergenci díky reinforcement learningu lidi taky pozorují, tak to co v jejich paperu zmínili vypadá že reálně funguje. OpenAI employees taky přiznali že DeepSeek přišel na velkou část základu o1. DeepSeek na konci dělal ještě trochu supervised finetuningu na alignment, což je dost malá část tý training pipeliny, co tam mohl nacpat to, že si ten model někdy myslí, že je ChatGPT, protože to bylo v těch datech, který mohli dostat jak z tý distilace, tak z huggingface datasetů nebo internet, kterej je teď plnej ChatGPT outputů. Navíc distilaci z frontier modelů teď v nějaký míře pravděpodobně dělají v podstatě všechny AI companies, protože to je nejjednoduší na lepší performance, proto si tolik LLMs myslí že jsou ChatGPT, ale to může být zároveň tím, že public datasety na huggingface a internet je teď plnej ChatGPT. The joy of open research, where you can't fake stuff, because others will easily prove you wrong through replications. Ale je pravda že ten hype dává alespoň smysl v tom, že rozbili velkou část moatu OpenAI s trouchu horším modelem než OpenAI's o1 (dle milion nekontaminovaných private benchmarků) za zlomek ceny (i když Gemini thinking od Googlu je taky o něco horší než o1, a je ještě levnější než DeepSeek API, ale je closed source), a co je open source a jde pustit locally. V dost aspektech true But my prediction is that the "OpenAI lost to China thanks to DeepSeek's many times cheaper better model" narrative (which is partly wrong because o1 is still a bit better than DeepSeek-R1 on many private uncontaminated benchmarks) will shift again when OpenAI releases o3 (or some other US company releases something similar), which is much further in terms of benchmarks, which is scaled even further and is even more expensive, sometimes thousands of $ per prompt for more complex tasks (see ARC-AGI), where a lot of this DeepSeek algorithmic efficiency hasn't been applied yet, and even when it will be applied, it won't drop the overall price so radically, so we will still have very costly frontier models Ale musí na to udělat dostatečně velkej media boom I really wonder if AI would "scheme", "lie", etc. if you didn't train them on data with these concepts. I wonder to what degree misalignment is consequence of one of these: 1) pattern matching personas with misaligned AIs in scifi stories in training data (or LessWrong posts) 2) emergent instrumental convergence from RL 3) system prompt 4) combination [[2412.14093] Alignment faking in large language models](https://arxiv.org/abs/2412.14093) Neural networks are shapeshiftable origami If you have a problem with gigantic combinatorial space of possibilities and know how to specify a reward function, then reinforcement learning will eventually conquer it When OpenAI did reinforcement learning for tool use for o3, did they call the tools in the training process or use some shortcuts? Since it would be really expensive https://openai.com/index/introducing-o3-and-o4-mini/ https://www.sciencedirect.com/science/article/pii/S0004370221000862 honestly im getting more and more reinforcement learning pilled recently, but i def think it's not the whole picture i think more ideal learning algorithm will be more of a hybrid (maybe add in some divergent novelty search or something, and evolution) and more biologically inspired (hebbian learning?) and more ideal architecture would be more biologically inspired (liquid foundational models?) and more neurosymbolic (dreamcoder?) but maybe biology just isnt optimal and we will find some general algorithm that will be much more effective than biology that differs from biology even more than the current mainstream meta https://fxtwitter.com/deedydas/status/1913588236959859095/ [https://storage.googleapis.com/deepmind-media/Era-of-Experience%20/The%20Era%20of%20Experience%20Paper.pdf](https://storage.googleapis.com/deepmind-media/Era-of-Experience%20/The%20Era%20of%20Experience%20Paper.pdf) Dreamcoder for physics AI architectures based on fluid dynamics DreamCoder + Reinforcement learning (neuralyl guided program synthesis trained by reinforcement learning) Combine theoretical mathematical AI frameworks How much of o3 behavior is emergent from pure RL? https://x.com/mckbrando/status/1912704921016869146?t=BhA5q_M_eIf_2t7Vw20ulg&s=19 I expected doom, I got really nice 1 hour discussion on why evolution isn't objective based, and doom later! I love how Liron constantly tries to contain Ken into various convergent categories, but Ken never conforms! 😄 My nuanced, divergently searching, interesting novelty seeking, multiperspectival, overall multidimensional, fluidly shapeshifting, man! ❤️ [https://youtu.be/GdthPZwU1Co?si=StimNVe3V6Ar6mJu](https://youtu.be/GdthPZwU1Co?si=StimNVe3V6Ar6mJu) " ChatGPT diagnosing rare disease https://x.com/zivotsnemoci/status/1915723349612790190 Tohle je dost cool, takových případů jsem už viděl několik. Sedí to i na nějaký benchmarky těch modelů co tohle ve slabší míře testují. Často se můžou trefit. Akorát to je jenom ChatGPT co teď všichni znají. Zajímalo by mě či jsi použila GPT-4o nebo o3 co už funguje dost jinak a může být víc grounded. Zajímalo by mě co by řekl Gemini 2.5 a Claude 3.5/3.7. Plus už jsou tvořený komplexní specializovaný systémy na diagnozu co mají LLMs jako součástky. Plus už jsou mimo OpenAI specializovanější modely na diagnózu, ať jazykový modely samotný, nebo napojený na různý specializovaný podmodely a data analysis metody co často nejsou vůbec jazykový a neuronkový, protože LLMs samy o sobě např nejsou optimální na numerický data. :D Ale rozhodně vyšší míra false positives je celkem univerzální, ale někdy to může zachránit tím jak to může vidět něco co lidi můžou přehlédnout, ale ty modely zase můžou přehlédnout něco co víc vidí lidi. Kombinace je optimální. :D Víc deterministický diagnostický metody jsou zase jinej kalibr. Je super když se všechny tyhle metody kombinují a využívají se co nejvíc jednotlivý výhody těch různých metod a supercharguje to celý ten diagnostický proces. Healthcare je obecně strašně pozadu co se týče technologický adopce, a v Česku ještě víc. Když jsem byl na sraze lidí co dělají AI a jiný technologie do zdravotnictví tak téma číslo 1 bylo že by se měly zmenšit šílený regulace co vůbec nestíhají technologický pokrok. :D " Can't wait for models trained to have identical mouse movements and clicks and keyboard copying to humans https://x.com/hibakod/status/1915914518901014779?t=xF9F4B0kcAgJSfLyYd51tw&s=19 What is the mathematical definition of what it means for AGI to love humanity? What is your favorite AI architecture and why? " Is AI self-improving? I think there are different types of self-improvement that have weaker and stronger versions AI (agent) running on GPUs putting training data to train it's own weights running on different GPUs is technically a form of self-improvement AI (agent) used to optimize nvidia kernels is technically a form of self-improvement AI (agent) used to optimize RL reward function is technically a form of self-improvement AI (agent) used to optimize hardware configuration is technically a form of self-improvement AI (agent) used to optimize some parts in its architecture (or as a whole possibly) is technically a form of self-improvement AI (agent) doing AI research from brainstorming to testing is technically a form of self-improvement Neural architecture search is technically a form of self-improvement Metalearning subfield of AI is technically a form of self-improvement But all of these forms of self-improvement are currently differently capable and differently strong right now, where some forms are used in practice a lot, and some forms don't work yet almost at all Maybe you can see all these forms of self-improvement as a continuous spectrum that evolves overtime with some semidiscrete phase shifts in capabilities " " What's next big thing in AI? I think the next big thing in AI will be either neurosymbolic breakthroughs combining matrix multiplications with symbolic programs, or physics based AI that uses differential equations. Or combination all of these. Nature and the universe has differential equations everywhere, in both physics and in computational neuroscience. Maybe it can be a relatively more adaptive type of math as the results in AI start to imply, and that's why it's everywhere in nature and the universe! For example, liquid neural networks (LNNs) have differential equations in them as part of the architecture where differential equation solvers are used, not just matrix multiplications. "The primary benefit LNNs offer is that they continue adapting to new stimuli after training. Additionally, LNNs are robust in noisy conditions and are smaller and more interpretable than their conventional counterparts." Liquid AI ( @LiquidAI_ ) with Joscha Bach ( @Plinz ) is building liquid foundational models AI based on these liquid neural networks as is destroying some benchmarks! God's programming language is differential equations. Maybe it will be the programming language of artificial general superintelligence too! [Liquid Neural Nets (LNNs). A deep dive into Liquid Neural… | by Jake Hession | Medium](https://medium.com/@hession520/liquid-neural-nets-lnns-32ce1bfb045a) [[2006.04439] Liquid Time-constant Networks](https://arxiv.org/abs/2006.04439) [From Liquid Neural Networks to Liquid Foundation Models | Liquid AI](https://www.liquid.ai/research/liquid-neural-networks-research) " " Do you think consciousness has any special computational properties? Depends on the definition and model of consciousness, but I like QRI's holistic field computation ideas IIT argues with integrated information maybe you truly need consciousness for information binding problem [[2012.05208] On the Binding Problem in Artificial Neural Networks](https://arxiv.org/abs/2012.05208) Global workspace theory argues with some form of global integration of information into some workspace Selfawareness isnt good in LLMs as emergent circuits are different than what the LLMs actually say (from last Anthropic paper on the biology of LLMs), so some recursive connections might be needed (strange loop model of conscousness?) Joscha Bach argues with conscousness being coherence inducing operator, maybe thats needed for reliability Neurosymbolic people need added symbolic components for strong generalization, like in DreamCoder program synthesis, and Chollet argues that's part of definition of consciousness Evolutionaries need evolution like evolutionary algorithms, maybe you could argue you can get consciousness only this way Physicists/computational neuroscientists need differential equations, like liquid neural networks, and some might argue consciousness only arises from this Some people need divergent novelty search without objective, like Kenneth Stanley, and you could also connect this with conscousness " " On the other hand one good thing about automating certain aspects of healthcare is that it might also mean more cheaper healthcare for everyone, as there's not enough doctors having not enough time for everyone (at least here in Europe i imagine that). So you get bigger combined total power and capabilities of human-machine collaboration in healthcare Radiology is probably the most developed then it comes to AI Recently I've seen this paper: [Should artificial intelligence have lower acceptable error rates than humans? - PMC](https://pmc.ncbi.nlm.nih.gov/articles/PMC10301708/) "A survey among employees at the Department of Radiology showed significantly lower acceptable error rates for AI (6.8 %) than humans (11.3 %)." I'm trying to watch the AI x healthcare space a lot. If i didnt want to get into AI x physics intersection more, I would want to go into this intersection. it seems like more generally when it comes to technology adoption at least here in Czechia, healthcare is terribly behind in terms of technology adoption When I was at a meetup of people doing AI and other technologies in healthcare, the number 1 topic was that we should reduce some of the regulations that don't keep up with technological advances. 😄 Like some hospitals here still use CDs to transfer data between doctors....... that was an experience " I find it fascinating that diffusion models use methods from nonequilibrium thermodynamics And now at the same time scientists are trying to use this subfield of physics to explain life Hmm, diffusion modely jsou dost podobný tomu že člověk má nejdřív nějakou vague noisy myšlenku a pak ji vzorama co zná postupně zkonkrétní 😄 Když to zjednoduším, tak začínají s textem/obrazem/videem/latentním prostorem/libovolnými daty plnýho náhodnýho šumu a postupně ho podle promptu vzorama "čistí" krok za krokem, dokud se z něj nestane text/obraz/latentní prostor/atd. se vzory co dávájí smysl. Učí se to tak, že text/obraz/latentním prostorem/atd. zašumují, a pak se učí tenhle proces obracet. A přes tohle se ta neuronka je schopná naučit nějaký limitovaně zobecňující emergentní koncepty a obvody aby to byla schopná dělat konzistentně. Pak ještě např různý hacky s generovanýma datama (což je v podstatě podobný snění), dělání Monte Carlo Tree Search (což je podobný přemýšlení nad možnostmi a vybíraní těch pravděpodobně lepších), apod. tomu může ještě ty reprezenace zrobustnit/zobecnit a výsledky zlepšit apod.