Platonic mental physical dimensions [Imgur: The magic of the Internet](https://imgur.com/90Ob9Js) [Formal system - Wikipedia]([Formal system - Wikipedia](https://en.wikipedia.org/wiki/Formal_system)) [Evolutionary algorithm - Wikipedia]([Evolution - Wikipedia]([Evolution - Wikipedia]([Evolution - Wikipedia](https://en.wikipedia.org/wiki/Evolution)))ary_algorithm) [OpenAI Insights and Training Data Shenanigans - 7 'Complicated' Developments + Guest Star - YouTube]([OpenAI Insights and Training Data Shenanigans - 7 'Complicated' Developments + Guest Star - YouTube]([OpenAI Insights and Training Data Shenanigans - 7 'Complicated' Developments + Guest Star - YouTube](https://www.youtube.com/watch?v=NfbTqo3GOtk))) ai explained [Algorithmic information theory - Wikipedia]([Algorithmic information theory - Wikipedia](https://en.wikipedia.org/wiki/Algorithmic_information_theory)) [List of emerging technologies - Wikipedia]([List of emerging technologies - Wikipedia](https://en.wikipedia.org/wiki/List_of_emerging_technologies)) [OpenAI Insights and Training Data Shenanigans - 7 'Complicated' Developments + Guest Star - YouTube]([OpenAI Insights and Training Data Shenanigans - 7 'Complicated' Developments + Guest Star - YouTube]([OpenAI Insights and Training Data Shenanigans - 7 'Complicated' Developments + Guest Star - YouTube](https://www.youtube.com/watch?v=NfbTqo3GOtk))) [OpenAI’s Custom Chatbots Are Leaking Their Secrets | WIRED UK](https://www.wired.co.uk/article/openai-custom-chatbots-gpts-prompt-injection-attacks) [Shane Legg (DeepMind Founder) - 2028 AGI, Superhuman Alignment, New Architectures - YouTube]([Shane Legg (DeepMind Founder) - 2028 AGI, Superhuman Alignment, New Architectures - YouTube](https://www.youtube.com/watch?v=Kc1atfJkiJU)) [KARL FRISTON - INTELLIGENCE 3.0 - YouTube]([KARL FRISTON - INTELLIGENCE 3.0 - YouTube](https://www.youtube.com/watch?v=V_VXOdf1NMw)) Effective altruism is just doing the most good using rational empirical evidence. How you define good, what you focus on, what are your beliefs, what political/ideological tribe you join, if you focus on stem (tech) or humanities, what methodologies you choose,... is up to everyone, and doesnt fundamentally define effective altruism itself, which is that initial general principle. [In Continued Defense Of Effective Altruism]([In Continued Defense Of Effective Altruism](https://www.astralcodexten.com/p/in-continued-defense-of-effective)) [World's Largest FREE WILL Debate w/ Top Physicists & Philosophers - YouTube](https://youtu.be/SSbUCEleJhg?si=BwsooIjPdk7ehykF) free will compilation Quantum probabilities are objective parts of the world to annobjevtivist, and degrees of beliefs for subjectivists (quantum bayesianism) (Hoffman) [World's Largest FREE WILL Debate w/ Top Physicists & Philosophers - YouTube](https://youtu.be/SSbUCEleJhg?si=GPq8ftgdsSn4QYgc) (Chris Fields) interesting: [[2207.00729] The Parallelism Tradeoff: Limitations of Log-Precision Transformers]([[2207.00729] The Parallelism Tradeoff: Limitations of Log-Precision Transformers](https://arxiv.org/abs/2207.00729)) [Ilya Sutskever (OpenAI Chief Scientist) - Building AGI, Alignment, Spies, Microsoft, & Enlightenment - YouTube]([Ilya Sutskever (OpenAI Chief Scientist) - Building AGI, Alignment, Spies, Microsoft, & Enlightenment - YouTube](https://youtu.be/Yf1o0TQzry8)?si=LmKkj-43cTDQbRod) illya interview [Mental Health Toolkit: Tools to Bolster Your Mood & Mental Health - YouTube]([Mental Health Toolkit: Tools to Bolster Your Mood & Mental Health - YouTube]([Mental Health Toolkit: Tools to Bolster Your Mood & Mental Health - YouTube](https://www.youtube.com/watch?v=CJIXbibQ0jI))) Je to z velký části odpověď na to jak poslední leaky v OpenAI byly pravděpodobně naškálování této metody [Improving mathematical reasoning with process supervision]([Improving mathematical reasoning with process supervision](https://openai.com/research/improving-mathematical-reasoning-with-process-supervision)) což vedlo k acing gradeschool level matiky, a limity kam až se to může dostat se zatím neví, i se všema ostatníma trikama (chain of thought, sebeverifikace, search,...) [Q* - Clues to the Puzzle? - YouTube]([Q* - Clues to the Puzzle? - YouTube]([Q* - Clues to the Puzzle? - YouTube](https://www.youtube.com/watch?v=ARf0WyFau0A))) [Microsoft Corporation]([Microsoft Corporation]([Microsoft Corporation](https://www.microsoft.com/en-us/research/blog/the-power-of-prompting/))) a o to stejný se snaží všechny ostatní AGI laby s jejich metodama 😃 https://twitter.com/ylecun/status/1727736289103880522 A často bývám dost skeptický u tvrzení, že používá jen známé techniky, a nekombinuje je nově nebo netvoří nový, když (na menších modelech) vidíme jak se Transformery učí zobecňující obvody [A Walkthrough of Progress Measures for Grokking via Mechanistic Interpretability: What? (Part 1/3) - YouTube]([A Walkthrough of Progress Measures for Grokking via Mechanistic Interpretability: What? (Part 1/3) - YouTube](https://www.youtube.com/watch?v=IHikLL8ULa4)) nebo AlphaGo v podstatě znovu našlo šachové zahajovací pohyby, nebo co dokážou generátory obrázků 😃 ten potenciál na kreativní kombinace a nový objevy v tom vidím hodně, a souhlasím s DeepMindem, že k LLMs to chce víc inkorporovat explicitní search jakožto algoritmický zlepšení, na čemž pravděpodobně taky dělají [Shane Legg (DeepMind Founder) - 2028 AGI, Superhuman Alignment, New Architectures - YouTube](https://youtu.be/Kc1atfJkiJU?si=o0xMdHPZ2L8nNl81&t=986) na kterým právě stojí ty revoluční AIs typu řešení protein foldingu nebo objevování nových materiálů mnohonásobně víc než jen lidi či jiný existující výpočetní metody [Millions of new materials discovered with deep learning - Google DeepMind]([Millions of new materials discovered with deep learning - Google DeepMind]([Millions of new materials discovered with deep learning - Google DeepMind]([Millions of new materials discovered with deep learning - Google DeepMind](https://deepmind.google/discover/blog/millions-of-new-materials-discovered-with-deep-learning/)))) ale zrovna vyšel zajímavý teoretický paper postulující limity o funkcích, který se transformery dokázou naučit (jsou logaritmický, kvůli velké paralelizaci, což nestačí na všechny polynomiální problémy), ale ještě to chce otestovat, a neřeší to chain of thought a jiný metody/hacky, na úrovni architektury nebo v inference time, který se používají [[2207.00729] The Parallelism Tradeoff: Limitations of Log-Precision Transformers]([[2207.00729] The Parallelism Tradeoff: Limitations of Log-Precision Transformers](https://arxiv.org/abs/2207.00729)) Ještě existuje existuje https://venturebeat.com/ai/meet-llemma-the-math-focused-open-source-ai-that-outperforms-rivals/ Líbila se mi odpověď od hlavního vědce v OpenAI Ilya Sutskevera než málem explodovali na otázku "can it propose new mathematical theorems" na kterou řekl "are you sure that the current models can't already do that?" 😃 [Q* - Clues to the Puzzle? - YouTube](https://youtu.be/ARf0WyFau0A?si=d95Lqlojx704pWSF&t=646) tak uvidíme kam až to půjde. Autoři ví o overfittingu, tady o tomto fenoménu mluví: [Ilya Sutskever: Deep Learning | Lex Fridman Podcast #94 - YouTube](https://youtu.be/13CZPWmke6A?si=DXmuFpozjCL5RDki&t=2164) předpokládám že se pořád snaží za každou cenu najít ten sweet spot v double descentu mezi overfittingem (když dimenze daz se rovná dimenzi modelu) a až moc velkým parametrickým prostorem relativně k množstvím dat rozbíjející přenost, což právě vede k dalšímu perfect fitu (po perfect fitu před iniciálním overfitingem v double descentu) a zároveň generalizabilitě díky menší sensitivitě na malý změny na kterým stojí většina LLMs. Zároveň tu využívá scaling laws. Je pravda že overfitting se děje i lokálně, něco preventnout jde líp, něco hůř. Ale ta empirická alchymie kolem toho jak to minimalizovat je cool: sposta metod trénování a benchmarků se právě snaží dělat takový tasky, aby se overfitting co nejvíc eliminoval, aby nutil flexibilitu, ale zároveň ne ztrátu konkrétních vědomostí nedávno vyšla i přímo architektura co nutí co nejvíc minimalizaci overfittingu a underfittingu vedoucí k humanlike systematický generalizaci, kombinující narrow a wide inteligenci [Human-like systematic generalization through a meta-learning neural network | Nature]([Human-like systematic generalization through a meta-learning neural network | Nature]([Human-like systematic generalization through a meta-learning neural network | Nature]([Human-like systematic generalization through a meta-learning neural network | Nature]([Human-like systematic generalization through a meta-learning neural network | Nature](https://www.nature.com/articles/s41586-023-06668-3))))) Větší inovace vidím právě k většímu přiblížení se k architektuře AlphaFoldu a GNoME, využívající jejich konkrétní problem solving co vede ke specializovaným inovacím a flexibilitu LLMs, což teď AGI labs s ostatníma jinýma trikama dělají typu sebekorekce a chain of thought [Millions of new materials discovered with deep learning - Google DeepMind]([Millions of new materials discovered with deep learning - Google DeepMind]([Millions of new materials discovered with deep learning - Google DeepMind]([Millions of new materials discovered with deep learning - Google DeepMind](https://deepmind.google/discover/blog/millions-of-new-materials-discovered-with-deep-learning/)))) tak uvidíme či se to povede It's gauge theory (symmetry breaking and their restoration via gauge forces) all the way down Life programming language [How does life come up with its programming language? - YouTube](https://www.youtube.com/watch?v=h5EtHLXSbHI) It's interesting that research still has no idea about quantum phenomena in the brain. And if they exist there, and if they're needed to get the humanlevel computational complexity at such energy cost and algorithmic efficiency, that would be interesting! [Experimental indications of non-classical brain functions - IOPscience]([ShieldSquare Captcha]([ShieldSquare Captcha]([ShieldSquare Captcha](https://iopscience.iop.org/article/10.1088/2399-6528/ac94be)))) [ShieldSquare Captcha](https://iopscience.iop.org/article/10.1088/2634-4386/ad0fec) A general-purpose organic gel computer that learns by itself [GitHub - OthersideAI/self-operating-computer: A framework to enable multimodal models to operate a computer.]([GitHub - OthersideAI/self-operating-computer: A framework to enable multimodal models to operate a computer.](https://github.com/othersideAI/self-operating-computer)) [Mother Buddha Love - Deconstructing Yourself]([Mother Buddha Love - Deconstructing Yourself]([Mother Buddha Love - Deconstructing Yourself]([Mother Buddha Love - Deconstructing Yourself]([Mother Buddha Love - Deconstructing Yourself]([Mother Buddha Love - Deconstructing Yourself]([Mother Buddha Love - Deconstructing Yourself]([Mother Buddha Love - Deconstructing Yourself]([Mother Buddha Love - Deconstructing Yourself]([Mother Buddha Love - Deconstructing Yourself]([Mother Buddha Love - Deconstructing Yourself]([Mother Buddha Love - Deconstructing Yourself]([Mother Buddha Love - Deconstructing Yourself](https://deconstructingyourself.com/mother-buddha-love.html))))))))))))) [#67 Prof. KARL FRISTON 2.0 [Unplugged] - YouTube]([#67 Prof. KARL FRISTON 2.0 [Unplugged] - YouTube](https://youtu.be/xKQ-F2-o8uM)?si=8zlsVrgjEBu0_MG4) [OPTICAL COMPUTING with PLASMA: Stanford PhD Defense - YouTube]([OPTICAL COMPUTING with PLASMA: Stanford PhD Defense - YouTube]([OPTICAL COMPUTING with PLASMA: Stanford PhD Defense - YouTube](https://www.youtube.com/watch?v=Mdh2pLwsK8Y))) P=NP [P vs. NP: The Biggest Puzzle in Computer Science - YouTube]([P vs. NP: The Biggest Puzzle in Computer Science - YouTube]([P vs. NP: The Biggest Puzzle in Computer Science - YouTube](https://www.youtube.com/watch?v=pQsdygaYcE4))) Reality is one giant infinitely complex inscrutable blob of almost ineffability aka just raw data aka matrices https://www.lesswrong.com/posts/GkC6YTu4DWp2zwf9k/giant-in-scrutable-matrices-maybe-the-best-of-all-possible when you consider the universe as one giant quantum turing machine [Quantum Turing machine - Wikipedia]([Quantum Turing machine - Wikipedia]([Quantum - Wikipedia](https://en.wikipedia.org/wiki/Quantum)_Turing_machine)) , but there are various compressing (mathematical) patterns, reducible pockets, that we can mechanistically interpret and verify by using it to better than noise predict, control, build patterns that serve us and explain various other patterns mechanistically or using less accurate less technical analogies. [Arbital]([Arbital](https://arbital.com/p/bayes_science_virtues/)) Different compressible patterns live in different subsets of reality, across different scales with different degrees of weakly emergent complexity, across different levels of abstraction, reverse engineering different problem subdomains using different tools with different equations governing the behavior of different variables. [Branches of science - Wikipedia]([Branches of science - Wikipedia]([Branches of science - Wikipedia](https://en.wikipedia.org/wiki/Branches_of_science))) This learning physical reality process is now already being automated using (interpretable) deep learning, such as learning on bayesian structured probabilistic causal graphical model [Graphical model - Wikipedia](https://en.wikipedia.org/wiki/Graphical_model) , DeepMind's finding of milions of new materials [Millions of new materials discovered with deep learning - Google DeepMind]([Millions of new materials discovered with deep learning - Google DeepMind]([Millions of new materials discovered with deep learning - Google DeepMind]([Millions of new materials discovered with deep learning - Google DeepMind](https://deepmind.google/discover/blog/millions-of-new-materials-discovered-with-deep-learning/)))) , many times the amount of that humans have previously found and automatically synthesizing them, or other automated AI theoretical and practical scientists projects [Future House]([Future House]([Future House]([Future House](https://www.futurehouse.org/articles/announcing-future-house)))) https://zenodo.org/records/8164667, or cracking protein folding with AlphaFold [AlphaFold Protein Structure Database](https://alphafold.ebi.ac.uk/) and creating a gigantic protein database out of it, which then interpretability research is reverse engineering on OpenFold (open source version of AlphaFold) [Mechanistic Interpretability - Stella Biderman | Stanford MLSys #70 - YouTube](https://www.youtube.com/live/P7sjVMtb5Sg) and finding that the internal Transformers go through phase changes: first they learn to predict a 2D representation, then to inflate it to a 3D representation, and then it fills in the details. It's slowly compressing as much features as possible in the least amount of as dimensions [Scaling Laws from the Data Manifold Dimension]([Scaling Laws from the Data Manifold Dimension]([Scaling Laws from the Data Manifold Dimension]([Scaling Laws from the Data Manifold Dimension](https://jmlr.org/papers/v23/20-1111.html)))) possible by most likely learning various (still undiscovered) circuits helping generalization [A Walkthrough of Progress Measures for Grokking via Mechanistic Interpretability: What? (Part 1/3) - YouTube]([A Walkthrough of Progress Measures for Grokking via Mechanistic Interpretability: What? (Part 1/3) - YouTube](https://www.youtube.com/watch?v=IHikLL8ULa4)) , in some way distinctly, in some ways similarly to humans, or discovering differential equations from physical dynamics by learning autoencoder [Can AI disover new physics? - YouTube]([Can AI disover new physics? - YouTube]([Can AI disover new physics? - YouTube]([Can AI disover new physics? - YouTube](https://www.youtube.com/watch?v=XRL56YCfKtA)))) to predict dynamics of a system 0.5 seconds in advance and finding out the number of variables needed by sparsely squeezing the compressed latent space as much as possible while preserving accurate predictions and extracting latent dimensionality to get number of latent variables needed to model that dynamical system, and we can use mechanistic interpretability using sparse autoencoders to extract monosemantic features [Towards Monosemanticity: Decomposing Language Models With Dictionary Learning]([Towards Monosemanticity: Decomposing Language Models With Dictionary Learning]([Towards Monosemanticity: Decomposing Language Models With Dictionary Learning]([Towards Monosemanticity: Decomposing Language Models With Dictionary Learning]([Towards Monosemanticity: Decomposing Language Models With Dictionary Learning]([Towards Monosemanticity: Decomposing Language Models With Dictionary Learning]([Towards Monosemanticity: Decomposing Language Models With Dictionary Learning]([Towards Monosemanticity: Decomposing Language Models With Dictionary Learning]([Towards Monosemanticity: Decomposing Language Models With Dictionary Learning]([Towards Monosemanticity: Decomposing Language Models With Dictionary Learning](https://transformer-circuits.pub/2023/monosemantic-features/index.html)))))))))) aka variables with that dimensionality corresponding to the variables we are looking for, and even automate this process of discovering the circuits itself. [A Walkthrough of Automated Circuit Discovery w/ Arthur Conmy Part 1/3 - YouTube]([A Walkthrough of Automated Circuit Discovery w/ Arthur Conmy Part 1/3 - YouTube]([A Walkthrough of Automated Circuit Discovery w/ Arthur Conmy Part 1/3 - YouTube](https://www.youtube.com/watch?v=dn4GqR0DCx8))) [Autism as a disorder of dimensionality – Opentheory.net]([Autism as a disorder of dimensionality – Opentheory.net]([Autism as a disorder of dimensionality – Opentheory.net]([Autism as a disorder of dimensionality – Opentheory.net]([Autism as a disorder of dimensionality – Opentheory.net]([Autism as a disorder of dimensionality – Opentheory.net]([Autism as a disorder of dimensionality – Opentheory.net]([Autism as a disorder of dimensionality – Opentheory.net]([Autism as a disorder of dimensionality – Opentheory.net]([Autism as a disorder of dimensionality – Opentheory.net]([Autism as a disorder of dimensionality – Opentheory.net]([Autism as a disorder of dimensionality – Opentheory.net](https://opentheory.net/2023/05/autism-as-a-disorder-of-dimensionality/)))))))))))) optics 3b1b [Why light can “slow down”, and why it depends on color | Optics puzzles 3 - YouTube](https://www.youtube.com/watch?v=KTzGBJPuJwM) [[2311.16081] ViT-Lens-2: Gateway to Omni-modal Intelligence](https://arxiv.org/abs/2311.16081) [[2308.10185] ViT-Lens: Towards Omni-modal Representations](https://arxiv.org/abs/2308.10185) [GitHub - TencentARC/ViT-Lens: [Preprint] ViT-Lens: Towards Omni-modal Representations](https://github.com/TencentARC/ViT-Lens) ViT-Lens: Towards Omni-modal Representations [Quanta Magazine]([Quanta Magazine](https://www.quantamagazine.org/how-space-and-time-could-be-a-quantum-error-correcting-code-20190103/)) AI learning feynman diagrams by polynomials? [François Charton | Transformers for maths, and maths for transformers - YouTube]([François Charton | Transformers for maths, and maths for transformers - YouTube](https://youtu.be/Sc6k06wVX3s?si=Oz545XT5qX_rq5aM)) [Anil Seth: Neuroscience of Consciousness & The Self - YouTube]([Anil Seth: Neuroscience of Consciousness & The Self - YouTube]([Anil Seth: Neuroscience of Consciousness & The Self - YouTube](https://youtu.be/_hUEqXhDbVs))?si=6Qal8QeCFksXDsJB) [Millions of new materials discovered with deep learning - Google DeepMind]([Millions of new materials discovered with deep learning - Google DeepMind]([Millions of new materials discovered with deep learning - Google DeepMind]([Millions of new materials discovered with deep learning - Google DeepMind](https://deepmind.google/discover/blog/millions-of-new-materials-discovered-with-deep-learning/))))