OpenFold (like AlphaFold) from a mechanistic intepretability lens [Mechanistic Interpretability - Stella Biderman | Stanford MLSys #70 - YouTube](https://www.youtube.com/live/P7sjVMtb5Sg)?si=PxTnM1QWyMRZ57vp&t=625 when training, it goes through phase changes, first learns to predict a 2D representation, then it learns to inflate it to a 3D representation, and then it fills in the details, slowly compressing as much features as possible in the least amount of as dimensions possible by most likely learning various (still undiscovered) circuits helping generalization weather ML [ECMWF | Charts]([ECMWF | Charts]([ECMWF | Charts](https://charts.ecmwf.int/?facets=%7B%22Product%20type%22%3A%5B%22Experimental%3A%20Machine%20learning%20models%22%5D%7D))) [Microsoft Corporation]([Microsoft Corporation]([Microsoft Corporation](https://www.microsoft.com/en-us/research/blog/the-power-of-prompting/))) [God Help Us, Let's Try To Understand The Paper On AI Monosemanticity]([God Help Us, Let's Try To Understand The Paper On AI Monosemanticity](https://www.astralcodexten.com/p/god-help-us-lets-try-to-understand)) God Help Us, Let's Try To Understand AI Monosemanticity LLM deception [[2307.16513] Deception Abilities Emerged in Large Language Models](https://arxiv.org/abs/2307.16513) [[2311.07590] Technical Report: Large Language Models can Strategically Deceive their Users when Put Under Pressure]([[2311.07590] Technical Report: Large Language Models can Strategically Deceive their Users when Put Under Pressure](https://arxiv.org/abs/2311.07590)) [Stampy](https://stampy.ai/) [AI Risks that Could Lead to Catastrophe | CAIS]([AI Risks that Could Lead to Catastrophe | CAIS]([Center for AI Safety (CAIS)]([Center for AI Safety (CAIS)](https://www.safe.ai/))ai-risk)) [[2309.11495] Chain-of-Verification Reduces Hallucination in Large Language Models]([[2309.11495] Chain-of-Verification Reduces Hallucination in Large Language Models](https://arxiv.org/abs/2309.11495)) Chain-of-Verification Reduces Hallucination in Large Language Models AGI researchers first tried to make very sophisticated symbolic architectures, but that failed and resulted in AI winter. Then connectionist deep learning with neural networks and transformers that turned into ChatGPT came. [Imgur: The magic of the Internet](https://imgur.com/sFiUfuh) Now we're slowly seeing the merging of these paradigms into hybrid architectures, like in OpenAI, where we give the gigantic stacks of transformers tons of explicit symbolic heuristics such as chain of thought, selfverification, selfcorrection, search and so [Q* - Clues to the Puzzle? - YouTube]([Q* - Clues to the Puzzle? - YouTube]([Q* - Clues to the Puzzle? - YouTube](https://www.youtube.com/watch?v=ARf0WyFau0A))) on while creating mathematical theories of how transformers operate to make it more effective and modify or augment its architecture and steer the learned giant blob of almost inscrutable matrices by making them slightly more scrutable using bottom up [Towards Monosemanticity: Decomposing Language Models With Dictionary Learning]([Towards Monosemanticity: Decomposing Language Models With Dictionary Learning]([Towards Monosemanticity: Decomposing Language Models With Dictionary Learning]([Towards Monosemanticity: Decomposing Language Models With Dictionary Learning]([Towards Monosemanticity: Decomposing Language Models With Dictionary Learning]([Towards Monosemanticity: Decomposing Language Models With Dictionary Learning]([Towards Monosemanticity: Decomposing Language Models With Dictionary Learning]([Towards Monosemanticity: Decomposing Language Models With Dictionary Learning]([Towards Monosemanticity: Decomposing Language Models With Dictionary Learning]([Towards Monosemanticity: Decomposing Language Models With Dictionary Learning](https://transformer-circuits.pub/2023/monosemantic-features/index.html)))))))))) or top down mechanistic interpretability. [Representation Engineering: A Top-Down Approach to AI Transparency]([Representation Engineering: A Top-Down Approach to AI Transparency]([Representation Engineering: A Top-Down Approach to AI Transparency]([Representation Engineering: A Top-Down Approach to AI Transparency]([Representation Engineering: A Top-Down Approach to AI Transparency]([Representation Engineering: A Top-Down Approach to AI Transparency]([Representation Engineering: A Top-Down Approach to AI Transparency]([Representation Engineering: A Top-Down Approach to AI Transparency](https://www.ai-transparency.org/)))))))) [How plants can perform feats of quantum mechanics - Big Think]([How plants can perform feats of quantum mechanics - Big Think](https://bigthink.com/hard-science/plants-quantum-mechanics/)) Plants perform quantum mechanics feats that scientists can only do at ultra-cold temperatures Bose-Einstein condensates u mozku se u tohodle taky pořád hádají (Experimental indications of non-classical brain functions) [Experimental indications of non-classical brain functions - IOPscience]([ShieldSquare Captcha]([ShieldSquare Captcha]([ShieldSquare Captcha](https://iopscience.iop.org/article/10.1088/2399-6528/ac94be)))) body map of emotions [Imgur: The magic of the Internet](https://imgur.com/gol3k2e) https://www.pnas.org/doi/full/10.1073/pnas.1321664111 [I Won A Dog Competition With A Robot Dog - YouTube](https://www.youtube.com/watch?v=JP5FJ7fEyyc) I Entered A Robot Dog Into A Dog Competition [Mind uploading - Wikipedia]([Mind uploading - Wikipedia](https://en.m.wikipedia.org/wiki/Mind_uploading)) [[2309.11495] Chain-of-Verification Reduces Hallucination in Large Language Models]([[2309.11495] Chain-of-Verification Reduces Hallucination in Large Language Models](https://arxiv.org/abs/2309.11495)) Checkout AutoGen, OpenAgents, AgentVerse, XAgent, MetaGPT for multiagent frameworks with MemGPT for memory or sparse priming representations for compression, RAG for retrieval, now with OpenAI's Q* search plus chain of thought plus selfcorrection [Q* - Clues to the Puzzle? - YouTube]([Q* - Clues to the Puzzle? - YouTube]([Q* - Clues to the Puzzle? - YouTube](https://www.youtube.com/watch?v=ARf0WyFau0A))), and let's also add higher order attention from Meta... [[2311.11829] System 2 Attention (is something you might need too)]([[2311.11829] System 2 Attention (is something you might need too)](https://arxiv.org/abs/2311.11829)) and more general multimodality and embodiment from Palm [What's Left Before AGI? PaLM-E, 'GPT 4' and Multi-Modality - YouTube]([What's Left Before AGI? PaLM-E, 'GPT 4' and Multi-Modality - YouTube](https://www.youtube.com/watch?v=EzEuylNSn-Q)) and robotics, access to all of internet, all software, and embodiment with humanlike capabilities, and selfrewriting to the singularity and beyond, woo weee [World's Largest FREE WILL Debate w/ Top Physicists & Philosophers - YouTube](https://www.youtube.com/watch?v=SSbUCEleJhg) [Computational creativity - Wikipedia]([Computational creativity - Wikipedia](https://en.wikipedia.org/wiki/Computational_creativity)) lucid dream neurotech [Approach – PROPHETIC](https://propheticai.co/pages/approach) SotA commercial neurotech: [Cody Rall MD with Techforpsych - YouTube]([Cody Rall MD with Techforpsych - YouTube](https://www.youtube.com/@CodyRallMD/videos)) [[2311.10215] Predictive Minds: LLMs As Atypical Active Inference Agents]([[2311.10215] Predictive Minds: LLMs As Atypical Active Inference Agents](https://arxiv.org/abs/2311.10215)) LLM active inference [[2310.12036] A General Theoretical Paradigm to Understand Learning from Human Preferences]([[2310.12036] A General Theoretical Paradigm to Understand Learning from Human Preferences](https://arxiv.org/abs/2310.12036)) Optical computing with plasma [OPTICAL COMPUTING with PLASMA: Stanford PhD Defense - YouTube]([OPTICAL COMPUTING with PLASMA: Stanford PhD Defense - YouTube]([OPTICAL COMPUTING with PLASMA: Stanford PhD Defense - YouTube](https://www.youtube.com/watch?v=Mdh2pLwsK8Y))) Fyzika je láska :3 Teorie relativity ještě přidává guláš do času: Čím rychleji objekt cestuje, nebo pod čím větší gravitací je, tím pomaleji mu ubýha čas. GPS družice toho musí brát v potaz. Interstellar je na to cool film. :D [Interstellar (2014) - Scene “Messages span: 23 years" - YouTube](https://youtu.be/s_M1t0HE-Kk?si=wqPLM2QQuoLaqRbq) Interpretability by removing parts of the network and seeing what changes [Mechanistic Interpretability - Stella Biderman | Stanford MLSys #70 - YouTube](https://www.youtube.com/live/P7sjVMtb5Sg)?si=PxTnM1QWyMRZ57vp&t=625 [In Continued Defense Of Effective Altruism]([In Continued Defense Of Effective Altruism](https://www.astralcodexten.com/p/in-continued-defense-of-effective)) [The Internet is Worse Than Ever – Now What? - YouTube](https://www.youtube.com/watch?v=fuFlMtZmvY0) social media polarization Hmm, and could you recursively rationalize if rationales of rationales etc. were correct etc., create metarationales, metareflection [[2310.11511] Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection]([[2310.11511] Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection]([[2310.11511] Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection]([[2310.11511] Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection](https://arxiv.org/abs/2310.11511)))) tokens metaverifiers, or metaattention https://twitter.com/jaseweston/status/1726784511357157618, metatransformer architecture with metareinforcement learning with metarationales with mutually and selfimproving ecosystem of agents with trillions of compute? Train, finetune, vector steer [Steering GPT-2-XL by adding an activation vector — AI Alignment Forum]([Steering GPT-2-XL by adding an activation vector — AI Alignment Forum]([Steering GPT-2-XL by adding an activation vector — AI Alignment Forum](https://www.alignmentforum.org/posts/5spBue2z2tw4JuDCx/steering-gpt-2-xl-by-adding-an-activation-vector))) [Representation Engineering: A Top-Down Approach to AI Transparency]([Representation Engineering: A Top-Down Approach to AI Transparency]([Representation Engineering: A Top-Down Approach to AI Transparency]([Representation Engineering: A Top-Down Approach to AI Transparency]([Representation Engineering: A Top-Down Approach to AI Transparency]([Representation Engineering: A Top-Down Approach to AI Transparency]([Representation Engineering: A Top-Down Approach to AI Transparency]([Representation Engineering: A Top-Down Approach to AI Transparency](https://www.ai-transparency.org/)))))))), circuit steer [A Walkthrough of Progress Measures for Grokking via Mechanistic Interpretability: What? (Part 1/3) - YouTube]([A Walkthrough of Progress Measures for Grokking via Mechanistic Interpretability: What? (Part 1/3) - YouTube](https://www.youtube.com/watch?v=IHikLL8ULa4)), prompt engineer,... Let's also add in episodic memory, multimodality, virtual and physical tools to act with,... Feel the A G S I! Transformers with chain of thought is becoming a Turing machine: Here are some ways to program it [Lukasz Kaiser (OpenAI): GPT-4 and Beyond (Edge Summit) - YouTube](https://youtu.be/4vvzgBwH9OE&t=967) [[2303.14310] GPT is becoming a Turing machine: Here are some ways to program it](https://arxiv.org/abs/2303.14310) but they are internally already learning finite state automatas [Towards Monosemanticity: Decomposing Language Models With Dictionary Learning]([Towards Monosemanticity: Decomposing Language Models With Dictionary Learning]([Towards Monosemanticity: Decomposing Language Models With Dictionary Learning]([Towards Monosemanticity: Decomposing Language Models With Dictionary Learning]([Towards Monosemanticity: Decomposing Language Models With Dictionary Learning]([Towards Monosemanticity: Decomposing Language Models With Dictionary Learning]([Towards Monosemanticity: Decomposing Language Models With Dictionary Learning]([Towards Monosemanticity: Decomposing Language Models With Dictionary Learning]([Towards Monosemanticity: Decomposing Language Models With Dictionary Learning]([Towards Monosemanticity: Decomposing Language Models With Dictionary Learning](https://transformer-circuits.pub/2023/monosemantic-features/index.html)))))))))) Hinton: AI and Neurosci is starting to diverge [CBMM10 Panel: Research on Intelligence in the Age of AI - YouTube]([CBMM10 Panel: Research on Intelligence in the Age of AI - YouTube]([CBMM10 Panel: Research on Intelligence in the Age of AI - YouTube]([CBMM10 Panel: Research on Intelligence in the Age of AI - YouTube]([CBMM10 Panel: Research on Intelligence in the Age of AI - YouTube](https://www.youtube.com/watch?v=Gg-w_n9NJIE))))) [Major milestone achieved in new quantum computing architecture | Argonne National Laboratory](https://www.anl.gov/article/major-milestone-achieved-in-new-quantum-computing-architecture) https://github.com/1mrat/gpt-stats/blob/main/2023-11-22.csv [Kahneman resolves conflict on income-wellbeing study, finds point at which unhappiness stops decreasing for unhappy people | Kahneman-Treisman Center for Behavioral Science & Public Policy](https://behavioralpolicy.princeton.edu/news/DK_wellbeing0323) [What The Longest-Running Study on Happiness Reveals - YouTube](https://youtu.be/vSQjk9jKarg) "Money can keep buying happiness for already happy people, but among the most unhappy, the money helps stave off unhappiness only to a point." [Relphormer: Relational Graph Transformer for Knowledge Graph Representations | Papers With Code]([Relphormer: Relational Graph Transformer for Knowledge Graph Representations | Papers With Code]([Relphormer: Relational Graph Transformer for Knowledge Graph Representations | Papers With Code](https://paperswithcode.com/paper/relphormer-relational-graph-transformer-for))) Nvidia datacenter singularity [Imgur: The magic of the Internet](https://imgur.com/GG3ZmiB) [Absolute Infinite - Wikipedia](https://en.wikipedia.org/wiki/Absolute_Infinite) [Mental Health Toolkit: Tools to Bolster Your Mood & Mental Health - YouTube]([Mental Health Toolkit: Tools to Bolster Your Mood & Mental Health - YouTube]([Mental Health Toolkit: Tools to Bolster Your Mood & Mental Health - YouTube](https://www.youtube.com/watch?v=CJIXbibQ0jI))) Hiearchies of abstraction: [Predictive coding - Wikipedia]([Predictive coding - Wikipedia]([Predictive coding - Wikipedia]([Predictive coding - Wikipedia](https://en.wikipedia.org/wiki/Predictive_coding)))) Brainlike hardware: [Neuromorphic engineering - Wikipedia]([Neuromorphic engineering - Wikipedia]([Neuromorphic engineering - Wikipedia](https://en.wikipedia.org/wiki/Neuromorphic_engineering))) World modelling: [Yann LeCun: A Path Towards Autonomous Machine Intelligence | Shaped Blog](https://www.shaped.ai/blog/yann-lecun-a-path-towards-autonomous-machine-intelligence) Grokking circuits: [A Walkthrough of Reverse-Engineering Modular Addition: Model Training (Part 1/3) - YouTube]([A Walkthrough of Reverse-Engineering Modular Addition: Model Training (Part 1/3) - YouTube]([A Walkthrough of Reverse-Engineering Modular Addition: Model Training (Part 1/3) - YouTube]([A Walkthrough of Reverse-Engineering Modular Addition: Model Training (Part 1/3) - YouTube](https://www.youtube.com/watch?v=ob4vuiqG2Go)))) Scaling laws and Data fractal manifold dimensions: [Scaling Laws from the Data Manifold Dimension]([Scaling Laws from the Data Manifold Dimension]([Scaling Laws from the Data Manifold Dimension]([Scaling Laws from the Data Manifold Dimension](https://jmlr.org/papers/v23/20-1111.html)))) and [Towards Monosemanticity: Decomposing Language Models With Dictionary Learning]([Towards Monosemanticity: Decomposing Language Models With Dictionary Learning]([Towards Monosemanticity: Decomposing Language Models With Dictionary Learning]([Towards Monosemanticity: Decomposing Language Models With Dictionary Learning]([Towards Monosemanticity: Decomposing Language Models With Dictionary Learning]([Towards Monosemanticity: Decomposing Language Models With Dictionary Learning]([Towards Monosemanticity: Decomposing Language Models With Dictionary Learning]([Towards Monosemanticity: Decomposing Language Models With Dictionary Learning]([Towards Monosemanticity: Decomposing Language Models With Dictionary Learning]([Towards Monosemanticity: Decomposing Language Models With Dictionary Learning](https://transformer-circuits.pub/2023/monosemantic-features/index.html)))))))))) Q*, MCTS with Transformers: [Q* - Clues to the Puzzle? - YouTube]([Q* - Clues to the Puzzle? - YouTube]([Q* - Clues to the Puzzle? - YouTube](https://www.youtube.com/watch?v=ARf0WyFau0A))) Narrow and general intelligence taxonomy: [[2311.02462] Levels of AGI: Operationalizing Progress on the Path to AGI]([[2311.02462] Levels of AGI: Operationalizing Progress on the Path to AGI](https://arxiv.org/abs/2311.02462)) [[2310.20216] Does GPT-4 Pass the Turing Test?](https://arxiv.org/abs/2310.20216) GPT4 not yet passing turing test, but if it was finetuned for not being a an AI assistant with overly hardcoded certain priors via RLHF i i think it might palme multimodal embodied model [What's Left Before AGI? PaLM-E, 'GPT 4' and Multi-Modality - YouTube]([What's Left Before AGI? PaLM-E, 'GPT 4' and Multi-Modality - YouTube](https://www.youtube.com/watch?v=EzEuylNSn-Q)) Summary of interpretability anthropic [Chris Olah - Looking Inside Neural Networks with Mechanistic Interpretability - YouTube]([Chris Olah - Looking Inside Neural Networks with Mechanistic Interpretability - YouTube]([Chris Olah - Looking Inside Neural Networks with Mechanistic Interpretability - YouTube]([Chris Olah - Looking Inside Neural Networks with Mechanistic Interpretability - YouTube]([Chris Olah - Looking Inside Neural Networks with Mechanistic Interpretability - YouTube](https://www.youtube.com/watch?v=2Rdp9GvcYOE))))) iceberg model of self huberman lab [Imgur: The magic of the Internet](https://imgur.com/Bc0Q3rx) [Guest Series | Dr. Paul Conti: How to Understand & Assess Your Mental Health](https://www.hubermanlab.com/episode/guest-series-dr-paul-conti-how-to-understand-and-assess-your-mental-health) [Mental Health Toolkit: Tools to Bolster Your Mood & Mental Health - YouTube](https://youtu.be/CJIXbibQ0jI?si=nF-KCp8hs2Qz_A_i&t=5090) generative drive is the core of mental health [Mental Health Toolkit: Tools to Bolster Your Mood & Mental Health - YouTube]([Mental Health Toolkit: Tools to Bolster Your Mood & Mental Health - YouTube]([Mental Health Toolkit: Tools to Bolster Your Mood & Mental Health - YouTube](https://www.youtube.com/watch?v=CJIXbibQ0jI))) [Foundational Fitness Protocol - Huberman Lab](https://www.hubermanlab.com/newsletter/foundational-fitness-protocol) [Quanta Magazine](https://www.quantamagazine.org/a-new-approach-to-computation-reimagines-artificial-intelligence-20230413/) [Bayesian epistemology - Wikipedia]([Bayesian epistemology - Wikipedia](https://en.wikipedia.org/wiki/Bayesian_epistemology)) [[2310.12036] A General Theoretical Paradigm to Understand Learning from Human Preferences]([[2310.12036] A General Theoretical Paradigm to Understand Learning from Human Preferences](https://arxiv.org/abs/2310.12036)) Generalized framework unifying rlhf and dpo, with a new simpler algorithm [Experimental indications of non-classical brain functions - IOPscience]([ShieldSquare Captcha]([ShieldSquare Captcha]([ShieldSquare Captcha](https://iopscience.iop.org/article/10.1088/2399-6528/ac94be)))) Experimental indications of non-classical brain functions