Can we use the method of reverse engineering dynamical systems by using autoencoders to learn their dynamics and squeezing them and extracting interacting variables from them to reverse engineer and make interpretable the dynamics of CNNs, LLMs and other AI architectures? [Can AI disover new physics? - YouTube]([Can AI disover new physics? - YouTube]([Can AI disover new physics? - YouTube]([Can AI disover new physics? - YouTube](https://www.youtube.com/watch?v=XRL56YCfKtA)))) [Concrete open problems in mechanistic interpretability | Neel Nanda | EAG London 23 - YouTube]([Concrete open problems in mechanistic interpretability | Neel Nanda | EAG London 23 - YouTube]([Concrete open problems in mechanistic interpretability | Neel Nanda | EAG London 23 - YouTube]([Concrete open problems in mechanistic interpretability | Neel Nanda | EAG London 23 - YouTube](https://www.youtube.com/watch?v=7t9umZ1tFso)))) for example one network learned composing trigonometry [[2301.05217] Progress measures for grokking via mechanistic interpretability]([[2301.05217] Progress measures for grokking via mechanistic interpretability]([[2301.05217] Progress measures for grokking via mechanistic interpretability](https://arxiv.org/abs/2301.05217))) another network learned group theory operations via representation theory [[2302.03025] A Toy Model of Universality: Reverse Engineering How Networks Learn Group Operations]([[2302.03025] A Toy Model of Universality: Reverse Engineering How Networks Learn Group Operations](https://arxiv.org/abs/2302.03025)) another one learned explicit looking at history https://www.lesswrong.com/posts/TvrfY4c9eaGLeyDkE/induction-heads-illustrated another one finite state automatalike circuits for compsing html [Towards Monosemanticity: Decomposing Language Models With Dictionary Learning]([Towards Monosemanticity: Decomposing Language Models With Dictionary Learning]([Towards Monosemanticity: Decomposing Language Models With Dictionary Learning]([Towards Monosemanticity: Decomposing Language Models With Dictionary Learning]([Towards Monosemanticity: Decomposing Language Models With Dictionary Learning]([Towards Monosemanticity: Decomposing Language Models With Dictionary Learning]([Towards Monosemanticity: Decomposing Language Models With Dictionary Learning]([Towards Monosemanticity: Decomposing Language Models With Dictionary Learning]([Towards Monosemanticity: Decomposing Language Models With Dictionary Learning]([Towards Monosemanticity: Decomposing Language Models With Dictionary Learning](https://transformer-circuits.pub/2023/monosemantic-features/index.html)))))))))) etc., and many more are being found each month, or from top down perspective you can play with latent vectors [My techno-optimism]([My techno-optimism]([My techno-optimism]([My techno-optimism]([My techno-optimism](https://vitalik.eth.limo/general/2023/11/27/techno_optimism.html))))) [ORIGINAL FATHER OF AI ON DANGERS! (Prof. Jürgen Schmidhuber) - YouTube]([ORIGINAL FATHER OF AI ON DANGERS! (Prof. Jürgen Schmidhuber) - YouTube]([ORIGINAL FATHER OF AI ON DANGERS! (Prof. Jürgen Schmidhuber) - YouTube](https://www.youtube.com/watch?v=q27XMPm5wg8))) [ORIGINAL FATHER OF AI ON DANGERS! (Prof. Jürgen Schmidhuber) - YouTube]([ORIGINAL FATHER OF AI ON DANGERS! (Prof. Jürgen Schmidhuber) - YouTube]([ORIGINAL FATHER OF AI ON DANGERS! (Prof. Jürgen Schmidhuber) - YouTube](https://www.youtube.com/watch?v=q27XMPm5wg8))) Ještě by to chtělo přidat ty nový najitý nelinearity pokud se chceme přiblížit mozku ještě víc 😄 [Dendrites: Why Biological Neurons Are Deep Neural Networks - YouTube]([Dendrites: Why Biological Neurons Are Deep Neural Networks - YouTube](https://youtu.be/hmtQPrH-gC4?si=UzYUxL4W86WSkFV3)) A použít forward forward algorithm místo backpropagation [Forward-Forward Algorithm. Geoffrey Hinton, a renowned researcher… | by Barhoumi Mosbeh | Medium]([Forward-Forward Algorithm. Geoffrey Hinton, a renowned researcher… | by Barhoumi Mosbeh | Medium](https://medium.com/@Mosbeh_Barhoumi/forward-forward-algorithm-ac24d0d9ffd)) Záleží co je pro tebe inherentní rozdíl. :D Je tam strašně moc podobností ale zároveň strašně moc odlišností. Pravda že mozek se jenom na pár killowattech výpočtů dokáže naučit celkem dobrou aproximaci světa za jeho život, která je zároveň univerzální napříč lidmi, a zároveň jsou lidi různě specializovaní a s trochu jiným genetickým a naučeným hardwarem a softwarem (colorblidedness, jiný kultury,...), mezitím co LLMs se trénují v gigawattech, protože nejsou limitovaný evolučním šetřením energie a limitovanými materiály XD. Kromě těch podobností co jsme řešili s méďou, tak co se týče inteligence a vzorů, co ty neuronky chytají, tak tam jsou taky podobnosti. [The brain may learn about the world the same way some computational models do | MIT News | Massachusetts Institute of Technology]([The brain may learn about the world the same way some computational models do | MIT News | Massachusetts Institute of Technology]([The brain may learn about the world the same way some computational models do | MIT News | Massachusetts Institute of Technology](https://news.mit.edu/2023/brain-self-supervised-computational-models-1030))) Mozek se pořád učí v celým jeho životě, mezitím co jazykový modely sežerou celej internet a pak už málo, nebo reinfocement learning má architekturu kde se učíš podle tvýho úspěchu v prostředí přes akce (akorát vždycky se pak jede odznova, chtělo by to učit se jako lidi bez úplnýho odznova xD) místo predikování dalšího tokenu, ale učící algorithmy se používají stený nebo podobný. Argumentoval bych že klasický neuronky, konvolučky, jazykový modely, zvířecí a lidský mozek jsou všechno jinak specializovaný a v jiných aspektech zase obecnější inteligence, v jistých aspektech (benchmarcích) jsou neuronky oproti lidem na úrovni batolat, ale v jiných jsou zase AIs, ať jazykový modely nebo jiný architektury, lepší než nejlepší člověk na planetě, a tenhle gap se zmenšuje víc a víc - AGI by mělo být ve všem lepší než člověk, dle mě ne jen kognitivně, ale taky v robotice (Google na tom dost pracuje). Lidi jsou specializovaní na přežití v našem světě a sebereprodukci a všechny ty další evoluční tlaky co nám tvoří základní pyramidu potřeb včetně specializované seberealizace, což neuronky zatím až tak nemají. Jako výsledek lidi mají dost explicitně specializovaných brain regions, ale pomalu zjišťujeme jak moc ten mozek je reálně dost fluidní a adaptibilní a ty regions nejsou až tak hardcoded, mezitím co neuronky se učí jiný typy specializovaných regions. Máme celkem systematickou generalizaci, která se zároveň umí učit konkrétně, na to taky vychází víc a víc architektur, co to napodobňuje víc a víc. [Human-like systematic generalization through a meta-learning neural network | Nature]([Human-like systematic generalization through a meta-learning neural network | Nature]([Human-like systematic generalization through a meta-learning neural network | Nature]([Human-like systematic generalization through a meta-learning neural network | Nature]([Human-like systematic generalization through a meta-learning neural network | Nature](https://www.nature.com/articles/s41586-023-06668-3))))) Neuronky mají šílený potencál být v tomto nesouměřitelně efektivnější než lidi, když se vytvoří pod podobnými tlaky, klidně jen v billionech simulacích. Možná potřebují realtime interakci, explicitnější symbolický model světa s fuzzy foundations a logikou, plánování, možná ne, možná jsou schopny se to všechno emergentně naučit. Jde tam nacpat i genetický algorimy na meta úrovni na simulaci evoluce. Neuronky jsou turingovsky kompletní a univerzální approximators, což znamená, že jsou technicky schopný se v teorii naučit libovolný algorithmus, ale otázka je, za jak dlouho, a jak moc se zasekávají v jiných řešeních, což řídí (fluidní nebo striktní) architektura, trénovací data a množství compute. Is perfect system ease of volunatary trade and difficulty of involuntary invasion? Differential technology [My techno-optimism]([My techno-optimism]([My techno-optimism]([My techno-optimism]([My techno-optimism](https://vitalik.eth.limo/general/2023/11/27/techno_optimism.html))))) Joscha x Friston [Joscha Bach Λ Karl Friston: Ai, Death, Self, God, Consciousness - YouTube]([Joscha Bach Λ Karl Friston: Ai, Death, Self, God, Consciousness - YouTube](https://youtu.be/CcQMYNi9a2w?si=3e6zk-daAD48jI3e)) text to audio sota [Audiobox](https://audiobox.metademolab.com/) small llm sota? [Microsoft Corporation](https://www.microsoft.com/en-us/research/blog/phi-2-the-surprising-power-of-small-language-models/) [Chris Olah - Looking Inside Neural Networks with Mechanistic Interpretability - YouTube]([Chris Olah - Looking Inside Neural Networks with Mechanistic Interpretability - YouTube]([Chris Olah - Looking Inside Neural Networks with Mechanistic Interpretability - YouTube]([Chris Olah - Looking Inside Neural Networks with Mechanistic Interpretability - YouTube]([Chris Olah - Looking Inside Neural Networks with Mechanistic Interpretability - YouTube](https://www.youtube.com/watch?v=2Rdp9GvcYOE))))) [[2311.05112] A Survey of Large Language Models in Medicine: Principles, Applications, and Challenges](https://arxiv.org/abs/2311.05112) llms in medicine Can we use the method of reverse engineering dynamical systems by using autoencoders to learn their dynamics and squeezing them and extracting interacting variables from them to reverse engineer and make interpretable the dynamics of CNNs, LLMs and other AI architectures? [Can AI disover new physics? - YouTube]([Can AI disover new physics? - YouTube]([Can AI disover new physics? - YouTube]([Can AI disover new physics? - YouTube](https://www.youtube.com/watch?v=XRL56YCfKtA)))) [Concrete open problems in mechanistic interpretability | Neel Nanda | EAG London 23 - YouTube]([Concrete open problems in mechanistic interpretability | Neel Nanda | EAG London 23 - YouTube]([Concrete open problems in mechanistic interpretability | Neel Nanda | EAG London 23 - YouTube]([Concrete open problems in mechanistic interpretability | Neel Nanda | EAG London 23 - YouTube](https://www.youtube.com/watch?v=7t9umZ1tFso)))) Is physics or computation fundamental? Or both, none, something third? So computation is emergent abstraction on top of those physical dynamics according to QRI's levels of analysis. Certain computationalists would argue that even all of that is special case of computation, or emergent from deeper computational principles in computational universe ontology. I suspect that's the first thing Joscha Bach would say, but I might be wrong. 😄 Or in universe being nonclassical turing machine, the evolution of fields can be seen as computation. Its defining information processing as being a lens on top of deeper physics versus physics being lens on deeper information processing. https://twitter.com/aniketvartak/status/1735137526770209023/photo/1 [Discord](https://discord.com/channels/991774141025886218/992229023667204137/1184669600386928751) Quora Answers 2015 - 2022 by David Pearce: towards a "triple S" civilisation of superintelligence, superlongevity and superhappiness [Quora Answers 2015 - 2024 by David Pearce: towards a "triple S" civilisation of superintelligence, superlongevity and superhappiness]([The Hedonistic Imperative]([The Hedonistic Imperative]([The Hedonistic Imperative](https://www.hedweb.com/)))quora/index.html) [Reddit - Dive into anything](https://www.reddit.com/r/MachineLearning/)comments/18hnh8p/d_what_are_2023s_top_innovations_in_mlai_outside/ more fun/learning/writing/projects/socializing/relaxed weekends sometimes more work or less work depending on mental state and how burning it is and priorities and socializing so on wake up before noon meditate (short or long) excercise (short or long) hygiene (short or long) eat with short audio/video check mail small thing from todo list (or big if burning) deep work 3 hours food smal lor big deep work 3 hours food small or big optionally more deep work social media max 1h other fun/learning/writing/projects etc. sleep before midnight [OPTICAL COMPUTING with PLASMA: Stanford PhD Defense - YouTube]([OPTICAL COMPUTING with PLASMA: Stanford PhD Defense - YouTube]([OPTICAL COMPUTING with PLASMA: Stanford PhD Defense - YouTube](https://www.youtube.com/watch?v=Mdh2pLwsK8Y))) [GitHub - danny-avila/LibreChat: Enhanced ChatGPT Clone: Features OpenAI, GPT-4 Vision, Bing, Anthropic, OpenRouter, Google Gemini, AI model switching, message search, langchain, DALL-E-3, ChatGPT Plugins, OpenAI Functions, Secure Multi-User System, Presets, completely open-source for self-hosting. More features in development](https://github.com/danny-avila/LibreChat) Buddhism map [Qualia Computing Networking](https://www.facebook.com/groups/1735260376685389/permalink/3655014251376649/) [Umělá inteligence už umí předpovídat počasí. Dělá to skvěle, ale dopouští se i hrubých chyb — ČT24 — Česká televize]([Umělá inteligence už umí předpovídat počasí. Dělá to skvěle, ale dopouští se i hrubých chyb — ČT24 — Česká televize](https://ct24admin.ceskatelevize.cz/veda/3631915-umela-inteligence-uz-umi-predpovidat-pocasi-dela-skvele-ale-dopousti-se-i-hrubych-chyb)) [ECMWF | Charts]([ECMWF | Charts]([ECMWF | Charts](https://charts.ecmwf.int/?facets=%7B%22Product%20type%22%3A%5B%22Experimental%3A%20Machine%20learning%20models%22%5D%7D))) [AR6 Synthesis Report: Climate Change 2023]([AR6 Synthesis Report: Climate Change 2023]([AR6 Synthesis Report: Climate Change 2023]([AR6 Synthesis Report: Climate Change 2023]([AR6 Synthesis Report: Climate Change 2023](https://www.ipcc.ch/report/ar6/syr/))))) [[2302.03025] A Toy Model of Universality: Reverse Engineering How Networks Learn Group Operations]([[2302.03025] A Toy Model of Universality: Reverse Engineering How Networks Learn Group Operations](https://arxiv.org/abs/2302.03025)) A Toy Model of Universality: Reverse Engineering How Networks Learn Group Operations [A new quantum algorithm for classical mechanics with an exponential speedup – Google Research Blog](https://blog.research.google/2023/12/a-new-quantum-algorithm-for-classical.html) A new quantum algorithm for classical mechanics with an exponential speedup Patrick Coles thermodynamic AI [Thermodynamic AI and the Fluctuation Frontier | Qiskit Seminar Series with Patrick Coles - YouTube]([Thermodynamic AI and the Fluctuation Frontier | Qiskit Seminar Series with Patrick Coles - YouTube]([Thermodynamic AI and the Fluctuation Frontier | Qiskit Seminar Series with Patrick Coles - YouTube](https://www.youtube.com/watch?v=dd1jURhLR8Y))) P=NP [P vs. NP: The Biggest Puzzle in Computer Science - YouTube]([P vs. NP: The Biggest Puzzle in Computer Science - YouTube]([P vs. NP: The Biggest Puzzle in Computer Science - YouTube](https://www.youtube.com/watch?v=pQsdygaYcE4))) [Quanta Magazine](https://www.quantamagazine.org/complexity-theorys-50-year-journey-to-the-limits-of-knowledge-20230817/) [Phenomenology (physics) - Wikipedia]([Phenomenology (physics) - Wikipedia](https://en.wikipedia.org/wiki/Phenomenology_(physics))) 1,200 tokens per second for Llama 2 7B on H100 [Optimum-NVIDIA Unlocking blazingly fast LLM inference in just 1 line of code]([Hugging Face – The AI community building the future.](https://huggingface.co/)blog/optimum-nvidia) introduction to metaphysics [How Long is Now? | Introduction to Metaphysics 🪐 - YouTube](https://www.youtube.com/watch?v=8HzIlKe--NU) automated web dev [Builder.io: Design to Shipped. Faster.](https://www.builder.io/) automated mobile app dev flutterflow [You probably won’t survive 2024... Top 10 Tech Trends - YouTube](https://www.youtube.com/watch?v=vyQv563Y-fk) lab grown babies [Lab-grown babies could become a reality within five years • Earth.com](https://www.earth.com/news/lab-grown-babies-revolutionary-science-or-ethical-disaster/) [GitHub - stas00/ml-engineering: Machine Learning Engineering Online Book]([GitHub - stas00/ml-engineering: Machine Learning Engineering Online Book](https://github.com/stas00/ml-engineering)) [No Turning Back: The Nonequilibrium Statistical Thermodynamics of becoming (and remaining) Life-Like - YouTube]([No Turning Back: The Nonequilibrium Statistical Thermodynamics of becoming (and remaining) Life-Like - YouTube]([No Turning Back: The Nonequilibrium Statistical Thermodynamics of becoming (and remaining) Life-Like - YouTube]([No Turning Back: The Nonequilibrium Statistical Thermodynamics of becoming (and remaining) Life-Like - YouTube]([No Turning Back: The Nonequilibrium Statistical Thermodynamics of becoming (and remaining) Life-Like - YouTube]([No Turning Back: The Nonequilibrium Statistical Thermodynamics of becoming (and remaining) Life-Like - YouTube](https://www.youtube.com/watch?v=10cVVHKCRWw)))))) No Turning Back: The Nonequilibrium Statistical Thermodynamics of becoming (and remaining) Life-Like [Autism as a disorder of dimensionality – Opentheory.net]([Autism as a disorder of dimensionality – Opentheory.net]([Autism as a disorder of dimensionality – Opentheory.net]([Autism as a disorder of dimensionality – Opentheory.net]([Autism as a disorder of dimensionality – Opentheory.net]([Autism as a disorder of dimensionality – Opentheory.net]([Autism as a disorder of dimensionality – Opentheory.net]([Autism as a disorder of dimensionality – Opentheory.net]([Autism as a disorder of dimensionality – Opentheory.net]([Autism as a disorder of dimensionality – Opentheory.net]([Autism as a disorder of dimensionality – Opentheory.net]([Autism as a disorder of dimensionality – Opentheory.net](https://opentheory.net/2023/05/autism-as-a-disorder-of-dimensionality/)))))))))))) [Testing predictive coding theories of autism spectrum disorder using models of active inference | PLOS Computational Biology]([Testing predictive coding theories of autism spectrum disorder using models of active inference | PLOS Computational Biology](https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1011473)) testing predicve coding theories of autism Adhd x autism x giftedness [Facebook](https://www.facebook.com/katyhigginsleemft/photos/a.8448708408476470/8628671087146867/?type=3) [Quanta Magazine](https://www.quantamagazine.org/biophysicists-uncover-powerful-symmetries-in-living-tissue-20231025/?fbclid=IwAR3ynF7k7LkZ2uQ9wMYrkwhfKOaaDtwDOG9lii5q3f4FbCqgndDqsrlqV4I) Mamba statespaces [Mamba - a replacement for Transformers? - YouTube]([Mamba - a replacement for Transformers? - YouTube](https://youtu.be/ouF-H35atOY?si=oixYX6Bl-pXYTKsH)) [[2302.13939v1] SpikeGPT: Generative Pre-trained Language Model with Spiking Neural Networks]([[2302.13939v1] SpikeGPT: Generative Pre-trained Language Model with Spiking Neural Networks]([[2302.13939v1] SpikeGPT: Generative Pre-trained Language Model with Spiking Neural Networks](https://arxiv.org/abs/2302.13939v1))) [Dendrites: Why Biological Neurons Are Deep Neural Networks - YouTube]([Dendrites: Why Biological Neurons Are Deep Neural Networks - YouTube](https://youtu.be/hmtQPrH-gC4?si=UzYUxL4W86WSkFV3)) dendrites are deep neural networks [Why there is anything at all - Wikipedia](https://en.wikipedia.org/wiki/Why_there_is_anything_at_all) Or let's shapeshift into Riemann sphere where division by zero is infinity! [What if we define 1/0 = ∞? | Möbius transformations visualized - YouTube](https://www.youtube.com/watch?v=hhI8fVxvmaw) I like the theory of it being chain of thought plus other hacks plus selfcorrection or finetuning, as papers on this note are coming from OpenAI and Microsoft recently (recent Microsoft prompt engineering is neat [Microsoft Corporation]([Microsoft Corporation]([Microsoft Corporation](https://www.microsoft.com/en-us/research/blog/the-power-of-prompting/))) ) and all other AGI labs are racing with very similar things which all i increases the probability of legitness to me. https://twitter.com/ylecun/status/1727736289103880522