[Q* - Clues to the Puzzle? - YouTube](https://youtu.be/ARf0WyFau0A?si=-qxo-RWoUZK7nAPd) Best breakdown on Q* so far! Step by step reasoning by more test time computation for math and science and using reward model verifiers on the individual steps (try lots of them) in inference time and choosing the top result [[2305.20050] Let's Verify Step by Step](https://arxiv.org/abs/2305.20050) instead of finetuning model directly, that scales much better... And further potentially finetuning a model on the correct outputs aka answers not the old way but by: keep going unt il you generate step by step rationales that get you to the correct answer and finetune on all of those rationales. [[2203.14465] STaR: Bootstrapping Reasoning With Reasoning](https://arxiv.org/abs/2203.14465) Selfimprovement generalizing beyond math across sciences and potentially further if we can beyond distribution few shot learning or/and get a good general cost function! https://twitter.com/peterjliu/status/1727564418802962823 Hmm, and could you recursively rationalize if rationales of rationales etc. were correct etc., create metarationales, metareflection [[2310.11511] Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection](https://arxiv.org/abs/2310.11511) tokens metaverifiers, or metaattention https://twitter.com/jaseweston/status/1726784511357157618, metatransformer architecture with metareinforcement learning with metarationales with mutually and selfimproving ecosystem of agents with trillions of compute? Train, finetune, vector steer [Steering GPT-2-XL by adding an activation vector — AI Alignment Forum](https://www.alignmentforum.org/posts/5spBue2z2tw4JuDCx/steering-gpt-2-xl-by-adding-an-activation-vector) [Representation Engineering: A Top-Down Approach to AI Transparency](https://www.ai-transparency.org/,) circuit steer [A Walkthrough of Progress Measures for Grokking via Mechanistic Interpretability: What? (Part 1/3) - YouTube](https://www.youtube.com/watch?v=IHikLL8ULa4,) prompt engineer,... Let's also add in episodic memory, multimodality, virtual and physical tools to act with,... Feel the A G S I! Feel the A G S I! UBI UBS Transformers with chain of thought is becoming a Turing machine: Here are some ways to program it [Lukasz Kaiser (OpenAI): GPT-4 and Beyond (Edge Summit) - YouTube](https://youtu.be/4vvzgBwH9OE&t=967) [[2303.14310] GPT is becoming a Turing machine: Here are some ways to program it](https://arxiv.org/abs/2303.14310) but they are internally already learning finite state automatas [Towards Monosemanticity: Decomposing Language Models With Dictionary Learning](https://transformer-circuits.pub/2023/monosemantic-features/index.html) Hinton: AI and Neurosci is starting to diverge [CBMM10 Panel: Research on Intelligence in the Age of AI - YouTube](https://www.youtube.com/watch?v=Gg-w_n9NJIE) https://nitter.net/arankomatsuzaki/status/1723882043514470629?fbclid=IwAR3cTQtF3959boQed8ypskGvIw-Qhth3x-HGl-2oM27yE7T3XcRbcud99RA JARVIS-1: Open-Ended Multi-task Agents with Memory-Augmented Multimodal Language Models - Exhibits nearly perfect performances across >200 tasks in Minecraft The universe is a quantum turing machine computing standard model quantum field theory Most paradoxes stop existing in finitism and computational constructive mathematics AI - OpenAI, Anthropic, Inflection, DeepMind Longevity - Altos, Retro, NewLimit Space - SpaceX, Blue Origin, Relativity, Rocket lab Fusion - Helion, TAE, Commonwealth BCIs - Neuralink, Synchron, Science Corp Robotics - Figure, 1x, Agility, Boston dynamics, Nvidia the manhattan projects never stopped, they’re just privatized https://twitter.com/Brad08414464/status/1728555170760695881 Level two systems thinking needed for AGI [[2311.11045] Orca 2: Teaching Small Language Models How to Reason](https://arxiv.org/abs/2311.11045) Orca 2: Teaching Small Language Models How to Reason [This Week's AI News Changes The World Forever! - YouTube](https://youtu.be/w6iThlhUbIA?si=SzcsXq2NaKTWKsUB&t=864) the best smaller locally runnable opensource LLM matching llama 2 with smaller size is dominance not intrinsic to intelligence? https://twitter.com/JosephJacks_/status/1728510229133119644?s=20 AI is becoming less delusional, more factual, more multimodal, more capable, more embodied, more creative Creativity can be formalized mechanistically many ways: Search processes finding the hidden gems in the statespace. Adding noise to existing model guided under certain goal. The first one is probably already engineered in existing AI systems. I swear to god that lots of AGI researchers must be fueled by trying to prove people saying "AI will never be able to do x y z" wrong by successfully building it the next day and the person just moves his goalpost. In a few years definition of AGI will be like ASI and it will be that it has to solve quantum gravity and Reimann hypothesis in one equation in one picosecond on one parameter. For AGI we need planning and modelling physical world with world models and counterfactual reasoning of consequences of their actions in embodied context in human context https://twitter.com/ylecun/status/1728496457601183865 https://www.anl.gov/article/major-milestone-achieved-in-new-quantum-computing-architecture Major milestone achieved in new quantum computing architecture Breakthrough realized for retaining quantum information in a single-electron quantum bit AIs are interns in some areas, but savants in others [gpt-stats/2023-11-22.csv at main · 1mrat/gpt-stats · GitHub](https://github.com/1mrat/gpt-stats/blob/main/2023-11-22.csv) [Kahneman resolves conflict on income-wellbeing study, finds point at which unhappiness stops decreasing for unhappy people | Kahneman-Treisman Center for Behavioral Science & Public Policy](https://behavioralpolicy.princeton.edu/news/DK_wellbeing0323) [What The Ultimate Study On Happiness Reveals - YouTube](https://youtu.be/vSQjk9jKarg) "Money can keep buying happiness for already happy people, but among the most unhappy, the money helps stave off unhappiness only to a point." [Relphormer: Relational Graph Transformer for Knowledge Graph Representations | Papers With Code](https://paperswithcode.com/paper/relphormer-relational-graph-transformer-for) Nvidia datacenter singularity https://imgur.com/GG3ZmiB [[1711.10455] Backprop as Functor: A compositional perspective on supervised learning](https://arxiv.org/abs/1711.10455) Backprop as Functor: A compositional perspective on supervised learning [Absolute infinite - Wikipedia](https://en.wikipedia.org/wiki/Absolute_Infinite) omni/acc steelman and synthetize all acels and decels join the openmindedness acceleration the models just want to learn Model theory [Calabi–Yau manifold - Wikipedia](https://en.wikipedia.org/wiki/Calabi–Yau_manifold) Multimodal rag LLM code https://twitter.com/clusteredbytes/status/1727741878970314964?s=20 Blockchain Universal primer for everything, initial and mathematical, pastebin Geometry, topology, homology, differential geometry, algebraic topology, algebraic geometry... Classifications Manifolds https://twitter.com/burny_tech/status/1728963232457789886?t=V2_RfbCQBrg6Bw6tFyiiTA&s=19 There is no center, the big bang happened everywhere Chain of thought math https://twitter.com/hughbzhang/status/1728834567241617861?t=uKs03UjPNSfYEcvo7MBiDw&s=19 Everything is caused by genetic and environmental factors. It's just about to which degree which ones are stronger. IQ, gender, sexuality, shape of head, beliefs, number of arms,... [Mental Health Toolkit: Tools to Bolster Your Mood & Mental Health - YouTube](https://www.youtube.com/watch?v=CJIXbibQ0jI) chceš větší vůli? můžeš si nějak zkusit zařídit elektrickou stimulaci anterior midcingulate cortexu 😄 https://www.cell.com/neuron/fulltext/S0896-6273(13)01030-1 je to v podstatě takový řídící centrum co rozdistribuovává glukózu (palivo) různě po mozku nebo jde boostnout tím že si člověk dá něco s gloukozou, ale ne tak že to s cukrem přehání pořád, potom je výsledek opak nebo se to trénuje tím že člověk dělá něco těžkýho proti čemu má odpor (např cvičení) nebo i to že věří že má hodně willpower má určitý nenulový důsledek na ty obvody, placebo je taky stronk dopaminogenní a noradrenalinový látky (prekurzory dopaminu v jídle, kofein, jiný stimulanty,...) s tím taky interagují (taky je tam dynamika typu fungující krátkodobý nebo dlouhodobý boost vs přesycování/tolerance) nebo antidepresiva a psychedelika a potenciálně dissciativa, všechno přes vyšší schopnost rewiringu a neurogeneze mě asi nejvíc motivace s tímhle vším teď generuje že mám danej cíl ke kterýmu jdu 😄 nebo to ještě hodně ovlivňuje sunlight, celkově světlo, tma v noci, celkově pohyb, prokrvování, nutrienty, social connection, stress control,... hmm, nebo do těchle ovládajících obvodů injektnout extra biologický nebo brzo umělý neurony na opravu nebo zesilnění Soon, probably [SINGULARITY AI: Ray Kurzweil Reveals Future Tech Timeline To 2100 - YouTube](https://www.youtube.com/watch?v=wYVktxbf6t0) Stochastic parrot refers to how machine learning is in big part statistics Hiearchies of abstraction: [Predictive coding - Wikipedia](https://en.wikipedia.org/wiki/Predictive_coding) Brainlike hardware: [Neuromorphic engineering - Wikipedia](https://en.wikipedia.org/wiki/Neuromorphic_engineering) World modelling: [Yann LeCun: A Path Towards Autonomous Machine Intelligence | Shaped Blog](https://www.shaped.ai/blog/yann-lecun-a-path-towards-autonomous-machine-intelligence) Symbolic AI: [Symbolic artificial intelligence - Wikipedia](https://en.wikipedia.org/wiki/Symbolic_artificial_intelligence) Grokking circuits: [A Walkthrough of Reverse-Engineering Modular Addition: Model Training (Part 1/3) - YouTube](https://www.youtube.com/watch?v=ob4vuiqG2Go) Scaling laws and Data fractal manifold dimensions: [Scaling Laws from the Data Manifold Dimension](https://jmlr.org/papers/v23/20-1111.html) and [Towards Monosemanticity: Decomposing Language Models With Dictionary Learning](https://transformer-circuits.pub/2023/monosemantic-features/index.html) Q*, MCTS with Transformers: [Q* - Clues to the Puzzle? - YouTube](https://www.youtube.com/watch?v=ARf0WyFau0A) Algorithmic improvements: https://twitter.com/burny_tech/status/1725578088392573038 Narrow and general intelligence taxonomy: [[2311.02462] Levels of AGI for Operationalizing Progress on the Path to AGI](https://arxiv.org/abs/2311.02462) [[2310.20216] Does GPT-4 pass the Turing test?](https://arxiv.org/abs/2310.20216) GPT4 not yet passing turing test, but if it was finetuned for not being a an AI assistant with overly hardcoded certain priors via RLHF i i think it might [Effective Environmentalism — Sollee](https://mirror.xyz/sollee.eth/DK0rkphAIsh9T5STMY3rd7PwYX--pTRzPhnRpbtMpeA) capitalist effective environmentalism Living systems theory https://twitter.com/bhohner/status/1729002021431267708?t=iu7v6CidEL9zc5EptEG4iw&s=19 Capitalism or alternatives to capitalism (Google) [ChatGPT](https://chat.openai.com/share/0b703cc9-cbcd-4c73-aa9a-3df21dd65a39) Math is like a modelling clay that you can compose like lego and in particular shape and configuration corresponds to real world physics I love to read all of Wikipedia or let GPT4 teach me everything from math to logic to physics to chemistry to neuroscience to biology to engineering to artificial intelligence to dynamical systems to sociology to philosophy by asking it "teach me the biggest concepts in field x" and if i dont know a word ask it to explain it recursively but i'm often rate limited but maybe solution would be like 5 accounts to escape the rate limit and you could do this while being on psychedelics too Great GPT optimized for learning, you can also give it wiki articles or books or papers or lecture transcripts for more accuracy [ChatGPT - Universal Primer](https://chat.openai.com/g/g-GbLbctpPz-universal-primer) Also this is well optimized for learning https://learngpt.art accelerating minds want accelerated world disaccelerating minds want disaccelerated world full turing completeness has never been tried, but q* asked sama to raise enough money to buy an infinite tape [Technological singularity - Wikipedia](https://en.wikipedia.org/wiki/Technological_singularity#Predictions) Downstream region in the brain is all you need https://twitter.com/dileeplearning/status/1728910903637402080 [Law of thought - Wikipedia](https://en.wikipedia.org/wiki/Law_of_thought) palme multimodal embodied model [- YouTube](https://www.youtube.com/watch?v=EzEuylNSn-Q) AI landscape feb 18 2023 https://www.lesswrong.com/posts/PE22QJSww8mpwh7bt/agi-in-sight-our-look-at-the-game-board Physics of LLMs meta [Physics of Language Models: Part 3.1 + 3.2, Knowledge Storage, Extraction and Manipulation - YouTube](https://youtu.be/YSHzKmEianc?si=oNmYmh0lK7dbbrBY) Summary of interpretability anthropic [Chris Olah - Looking Inside Neural Networks with Mechanistic Interpretability - YouTube](https://www.youtube.com/watch?v=2Rdp9GvcYOE) the more gradually you can explain your emotions, the more it correlates with wellbeing, but focusing too much on bad ones can also overly reinforce them [Mental Health Toolkit: Tools to Bolster Your Mood & Mental Health - YouTube](https://youtu.be/CJIXbibQ0jI?si=D-PzRAt4dhYxO3cW&t=4307) big sighs daily lead to mental health [Mental Health Toolkit: Tools to Bolster Your Mood & Mental Health - YouTube](https://youtu.be/CJIXbibQ0jI?si=Fc-3129iN8MxUpk_&t=4857) iceberg model of self huberman lab https://imgur.com/Bc0Q3rx [Guest Series | Dr. Paul Conti: How to Understand & Assess Your Mental Health - Huberman Lab](https://www.hubermanlab.com/episode/guest-series-dr-paul-conti-how-to-understand-and-assess-your-mental-health) [Mental Health Toolkit: Tools to Bolster Your Mood & Mental Health - YouTube](https://youtu.be/CJIXbibQ0jI?si=nF-KCp8hs2Qz_A_i&t=5090) sublimation prjection in predictive processing, seeing ourselves out there write down milestones, transformative moments, of your lifetime narrative to build meaningful context for selfactualization of your objective functions [Mental Health Toolkit: Tools to Bolster Your Mood & Mental Health - YouTube](https://youtu.be/CJIXbibQ0jI?si=X1E3WeuQ4Q-t-eh7&t=5380) Knowledge will grow infinitely Understanding is the ability to predict generative drive is the core of mental health [Mental Health Toolkit: Tools to Bolster Your Mood & Mental Health - YouTube](https://www.youtube.com/watch?v=CJIXbibQ0jI) fitness protocol [Fitness Toolkit: Protocol & Tools to Optimize Physical Health | Huberman Lab Podcast #94 - YouTube](https://www.youtube.com/watch?v=q1Ss8sTbFBY) Strength training naked and admiring your body is practicing selflove on steroids [Foundational Fitness Protocol - Huberman Lab - Huberman Lab](https://www.hubermanlab.com/newsletter/foundational-fitness-protocol) https://assets-global.website-files.com/64416928859cbdd1716d79ce/650e450994ef9ab775e16acf_Neural_Network_Newsletter_Foundational_Fitness_Protocol.pdf [I Did Andrew Huberman's Insane Fitness Routine Everyday for 6 Months | Results - YouTube](https://www.youtube.com/watch?app=desktop&v=JDRTZus9Mz4) I Did Andrew Huberman's Insane Fitness Routine Everyday for 6 Months | Results Recursive existence, atom is a universe [A New Approach to Computation Reimagines Artificial Intelligence | Quanta Magazine](https://www.quantamagazine.org/a-new-approach-to-computation-reimagines-artificial-intelligence-20230413/) [Bayesian epistemology - Wikipedia](https://en.wikipedia.org/wiki/Bayesian_epistemology) [[2310.12036] A General Theoretical Paradigm to Understand Learning from Human Preferences](https://arxiv.org/abs/2310.12036) Generalized framework unifying rlhf and dpo, with a new simpler algorithm [Radware Bot Manager Captcha](https://iopscience.iop.org/article/10.1088/2399-6528/ac94be) Experimental indications of non-classical brain functions Homology and cohomology in algebraic topology https://twitter.com/burny_tech/status/1729350450363654425 Anil Seth https://onlinelibrary.wiley.com/doi/book/10.1002/9781119767398 Mindfulness‐based Strategic Awareness Training Comprehensive Workbook: New approach based on free energy and active inference for skillful decision‐making https://twitter.com/burny_tech/status/1729355217299198147 Draw artificial general superintelligence! More general and superintelligent! More general and superintelligent! More general and superintelligent! Simplifying Transformer Blocks https://twitter.com/rasbt/status/1729123424990068743 https://openreview.net/pdf?id=KeWNER68iP An Information-Theoretic Understanding of Maximum Manifold Capacity Representations [Putin: West dominating AI industry, Russia must step up](https://interestingengineering.com/culture/putin-west-dominating-ai-industry-russia-must-step-up) [Starling-7B: Increasing LLM Helpfulness & Harmlessness with RLAIF](https://starling.cs.berkeley.edu/) best second model? https://chat.lmsys.org/ https://www.datacamp.com/tutorial/mistral-7b-tutorial https://www.lesswrong.com/posts/zaaGsFBeDTpCsYHef/shallow-review-of-live-agendas-in-alignment-and-safety leading ML researchers of the biggest labs, for example Shane Legg from DeepMind, or Dario Amodei from Anthropic, have their timelines of AGI arriving around 2026-2028. [Shane Legg (DeepMind Founder) - 2028 AGI, Superhuman Alignment, New Architectures - YouTube](https://www.youtube.com/watch?v=Kc1atfJkiJU) [Dario Amodei (Anthropic CEO) - $10 Billion Models, OpenAI, Scaling, & Alignment - YouTube](https://www.youtube.com/watch?v=Nlkk3glap_U) And Ilya Sutskever, Chief scientist in OpenAI, when asked if models are creative, if they can produce nontrivial mathematical conjectures, is like "Are you sure they cannot already do that" [Q* - Clues to the Puzzle? - YouTube](https://youtu.be/ARf0WyFau0A?si=FHp30hqXQKQeiG6T&t=651) Its interesting how our culture chose alcohol neuron damage over psychedelic neurogenesis https://twitter.com/burny_tech/status/1729388625756873123?t=RbRi8pvNPPKGP3jS0bfixg&s=19 Free will as ontological vs as feeling [[2311.10215] Predictive Minds: LLMs As Atypical Active Inference Agents](https://arxiv.org/abs/2311.10215) Predictive Minds: LLMs As Atypical Active Inference Agents The big bang didn't have a center. As an analogy, imagine an expanding sphere with our world being the surface of the sphere. The big bang corresponds to extrapolating to the past where the sphere was an infinitessimal point. This point does not exist as a specific "center" on the surface of the sphere. LLM benchmark [MT Bench - a Hugging Face Space by lmsys](https://huggingface.co/spaces/lmsys/mt-bench) Collective AGI for the benefit of all sentient beings guided circling nondual love and kindness meditation https://twitter.com/Plinz/status/1729056369222001018 When failure to repair Christian doctrine led to God's demise, Western moral philosophy lost its foundation in divine will. We got utilitarianism (optimizing quantified measurable properties), humanism (species level self idolization), and fascism (group level self idolization). Artificial intelligence researchers first tried to make very sophisticated symbolic architectures. And then deep learning and transformers that turned into ChatGPT came. https://imgur.com/j1lmlvZ nooooo you can't just scale up pure connectionist models on Internet data without inductive biases and modularization and expect them to learn real-world knowledge and grammar from form, or arithmetic and logical reasoning and causal inference-that's just memorization and superficial pattern- matching like Eliza, you need grounding in real-world communication with intent and social dynamics and multimodal robotic embodiment which can foster disentangled learning from guided exploration and self-directed goals expressed in Bayesian programs and probabilistic graphical models which are interpretable and pin down a unique semantics which can be debiased and expressed with uncertainty, and learned efficiently on tiny academic budgets, the cost only shows how this is a dead-end, we need to stop chasing SOTAS and model the complexity of the brain and consider the social context to decolonize AI's structural biases for Third World researchers... haha gpus go bitterrr I'm a lot for mechanistic interpretability from the very start in the architecture or analyzing the trained black box giant blob of inscrutable matrices, for theoretical and practical reasons. With that you can design, train, predict, steer it much better to make it do what you want more effectively, ideally more good and less harm. There are pretty relatively small models with performance better compared to much bigger ones [LMSys Chatbot Arena Leaderboard - a Hugging Face Space by lmsys](https://huggingface.co/spaces/lmsys/chatbot-arena-leaderboard) visualized dynamical systems [Dynamical Motifs in Neural Circuits](https://mitchellslapik.github.io/dynamical_motifs/) to jsou všechny empirický vědy, vědecká metoda = fuck around and find out, and maybe make at least slightly predictive theories about that chaos xD lastest LSD prodrug https://twitter.com/alieninsect/status/1729369035945693269 Any sufficiently advanced empirical science is indistinguishable from alchemist magic SSRIs increase neuroplasticity: Effects of escitalopram on synaptic density in the healthy human brain: a randomized controlled trial [Effects of escitalopram on synaptic density in the healthy human brain: a randomized controlled trial | Molecular Psychiatry](https://www.nature.com/articles/s41380-023-02285-8) i dont like ssris because they're numbing downers, but they work for lots of people, since they still work better than placebo at helping depression [How plants can perform feats of quantum mechanics - Big Think](https://bigthink.com/hard-science/plants-quantum-mechanics/) Plants perform quantum mechanics feats that scientists can only do at ultra-cold temperatures Bose-Einstein condensates můj oblíbený model deprese teď je že je to na úrovni jak chemickýho koktejlu, tak struktury a aktivity - jak moc je velká propojenost, flexibilita, coherence,... tak fungujících různých center (když nefungují hlavní řídící obvody tak je problém), tak softwaru (vzpomínky, předpoklady o světě apod.,), což je ovlivněný jak genetickýma tak environmentálníma faktorama a substance, neurotechnologie a terapie modifikují jiný části tohoto systému empirická vědecká metoda = fuck around and find out, and maybe make at least slightly predictive theories about that chaos xD je až fascinující jak se nám podařilo přírodu zotročit aby dělala v počítači statisticky dostatčně lineární deterministický kauzální operace čím víc prošťourám libovolný obor empirických věd tím víc zjišťuju jak je všechno v přírodě nelineární chaotický somehow functioning ducktaped systém s šílenou nepochopitelnou komplexitou na spoustě úrovních abstrakce A když už něco někde přeskočí nebo se protuneluje skrz tranzistor napříč nepravděpodobnosti, tak to většinou umíme detekovat a někdy aj opravit s tím jaký málo víme o vyšších škálách mě až fascinuje jak moc toho víme v standard model quantum field theory, tam dole je to o dost jednodušší mi příjde (tím že to dokážeme somewhat předpovídat), a pak se ta šílená komplexita enormního množství interagujících částic (excitací v poli přes feynmanovy diagramy) ukáže jako člověk ale stejně zajímavý jak univerzálně newtonovy zákony platí, pokud nejsme v těch extrémech typu rychlost světla, subatomární úroven, úroveň galaxií,. černý díry.. 😄 i když se nám pořád hodí i ty advanced teorie na dost praktický věci, jako je GPSka nebo zmenšování tranzistorů nebo kvantový počítače xD a pak v tomhle chaosu máme hledat nějaký univerrzální zákony mozku a ještě k tomu univerzální zákony prožitků jako je deprese s tím že se empiricky spolíháme jen na subjektivní questionares všechny metody kombatující s depresema líp než placebo jsou z velký části vědecký empirický alchemický zázrak mi příjde