https://twitter.com/SpencrGreenberg/status/1782042963205575147?t=FVYMPet_oQzLjD9f_DY01g&s=19 https://twitter.com/LordDreadwar/status/1781539910953971854?t=TCp43od-lGB6AjjZYj1u7g&s=19 https://twitter.com/OGdukeneurosurg/status/1723687287177830648?t=mVPb2cqt-FEVPb5RCb-INg&s=19 [[2404.02258] Mixture-of-Depths: Dynamically allocating compute in transformer-based language models](https://arxiv.org/abs/2404.02258) [[2008.00221] Commutators of spectral projections of spin operators](https://arxiv.org/abs/2008.0221) https://twitter.com/Plinz/status/1782248069360451751?t=r8H0o2ePrJDzlz4nOxiD4A&s=19 llm automated unit testing [[2402.09171] Automated Unit Test Improvement using Large Language Models at Meta](https://arxiv.org/abs/2402.09171) https://twitter.com/davidshor/status/1782503475705774495 [[2402.10210] Self-Play Fine-Tuning of Diffusion Models for Text-to-Image Generation](https://arxiv.org/abs/2402.10210) https://twitter.com/Andercot/status/1782482856343896073 https://twitter.com/thisismadani/status/1782510590839406904?t=XrOjfjS-mWQE78bR-sRTwg&s=19 5) https://twitter.com/jeremykauffman/status/1782440457433162197?t=-BetOnP-RfTb_Yd7v76aGQ&s=19 https://twitter.com/martin_rohbeck/status/1782386290245148944 How dollar works https://twitter.com/lopp/status/1782007744825802878?t=IF7ykjF4PR-wRd_pRI-0uw&s=19 [Quanta Magazine](https://www.quantamagazine.org/mathematicians-marvel-at-crazy-cuts-through-four-dimensions-20240422/) This expanded map includes more details on neural network architectures, optimization techniques, regularization methods, loss functions, evaluation metrics, tokenization approaches, embedding methods, attention variants, and efficient implementations. It covers a wide range of equations and extensions used in state-of-the-art transformer-based language models. However, please note that this is still not an exhaustive list, as the field of natural language processing and deep learning is vast and rapidly evolving. [[2312.10794] A mathematical perspective on Transformers](https://arxiv.org/abs/2312.10794) [[1804.04272] Deep Neural Networks Motivated by Partial Differential Equations](https://arxiv.org/abs/1804.04272) [[2302.04107] Can Physics-Informed Neural Networks beat the Finite Element Method?](https://arxiv.org/abs/2302.04107) [[2103.09177] Deep learning: a statistical viewpoint](https://arxiv.org/abs/2103.09177) [[2404.09516] State Space Model for New-Generation Network Alternative to Transformers: A Survey](https://arxiv.org/abs/2404.09516) [[2404.10981] A Survey on Retrieval-Augmented Text Generation for Large Language Models](https://arxiv.org/abs/2404.10981) [[2404.11584] The Landscape of Emerging AI Agent Architectures for Reasoning, Planning, and Tool Calling: A Survey](https://arxiv.org/abs/2404.11584) https://twitter.com/tsarnick/status/1781847492038181038?t=H4aTf1PCST5vdCvqEvt11g&s=19 [[2006.15136] Homotopy Theoretic and Categorical Models of Neural Information Networks](https://arxiv.org/abs/2006.15136) [[2403.10895] A Search for Classical Subsystems in Quantum Worlds](https://arxiv.org/abs/2403.10895) https://twitter.com/SpencrGreenberg/status/1781702814500016486?t=I6JyfTRzrJBAAc-MIbCzJQ&s=19 https://twitter.com/burny_tech/status/1781891648059855359?t=wQM7YnztLN_kUnf-kXdhFg&s=19 Robotics landscape https://twitter.com/burny_tech/status/1781824401517760646?t=S5ydOhZLH5hFvgFzDoxKEA&s=19 https://twitter.com/burny_tech/status/1782215842236694945?t=jBOjfPy03WJjtJ6W3eEeew&s=19 [Memory: How the brain constructs dreams | eLife](https://elifesciences.org/articles/58874) "Here is a gigantic, detailed map of software engineering: # Software Engineering Map ## Software Development Lifecycle 1. Requirements Gathering and Analysis - Stakeholder Interviews - Use Case Analysis - Functional Requirements - Non-functional Requirements 2. Design - Architecture Design - Monolithic Architecture - Microservices Architecture - Event-Driven Architecture - System Design - Database Design - Relational Databases (SQL) - NoSQL Databases - API Design - REST - GraphQL - gRPC - UI/UX Design - Wireframing - Prototyping - User Interface Design - User Experience Design 3. Implementation - Programming Languages - Object-Oriented Programming (OOP) - Java - C++ - C# - Python - Functional Programming - Haskell - Scala - F# - Scripting Languages - JavaScript - Python - Ruby - PHP - Web Development - Front-end - HTML - CSS - JavaScript - Frameworks - React - Angular - Vue.js - Back-end - Node.js - Express.js - Django - Ruby on Rails - ASP.NET - Full-stack - MEAN Stack (MongoDB, Express.js, Angular, Node.js) - MERN Stack (MongoDB, Express.js, React, Node.js) - Mobile App Development - iOS - Swift - Objective-C - Android - Java - Kotlin - Cross-platform - React Native - Flutter - Xamarin - Desktop Application Development - Java - C++ - C# - Python - Database Development - SQL (MySQL, PostgreSQL, Oracle) - NoSQL (MongoDB, Cassandra, Redis) - API Development - REST APIs - GraphQL APIs - gRPC APIs - DevOps - Continuous Integration and Continuous Deployment (CI/CD) - Jenkins - GitLab CI/CD - Travis CI - CircleCI - Infrastructure as Code (IaC) - Terraform - CloudFormation - Ansible - Containerization - Docker - Kubernetes - Cloud Platforms - Amazon Web Services (AWS) - Microsoft Azure - Google Cloud Platform (GCP) 4. Testing - Unit Testing - JUnit (Java) - NUnit (C#) - pytest (Python) - Jest (JavaScript) - Integration Testing - System Testing - Acceptance Testing - Performance Testing - Apache JMeter - Gatling - Security Testing - Penetration Testing - Vulnerability Scanning - Test Automation - Selenium - Appium - Cucumber 5. Deployment - On-premises Deployment - Cloud Deployment - Infrastructure as a Service (IaaS) - Platform as a Service (PaaS) - Software as a Service (SaaS) - Continuous Deployment 6. Maintenance and Support - Bug Fixes - Feature Enhancements - Performance Optimization - Security Patches - User Support ## Software Engineering Practices 1. Agile Methodologies - Scrum - Kanban - Extreme Programming (XP) - Lean Development 2. Software Design Patterns - Creational Patterns - Singleton - Factory - Builder - Prototype - Structural Patterns - Adapter - Decorator - Facade - Proxy - Behavioral Patterns - Observer - Strategy - Command - Iterator 3. Software Architecture Patterns - Model-View-Controller (MVC) - Model-View-ViewModel (MVVM) - Model-View-Presenter (MVP) - Clean Architecture - Hexagonal Architecture 4. Version Control Systems - Git - Subversion (SVN) - Mercurial 5. Code Review - Peer Code Review - Pull Requests - Static Code Analysis 6. Documentation - Requirements Documentation - Design Documentation - API Documentation - User Documentation - Technical Documentation 7. Project Management - Task Management - Backlog Management - Sprint Planning - Burndown Charts - Agile Boards (e.g., Jira, Trello) 8. Collaboration and Communication - Issue Tracking - Team Collaboration Tools (e.g., Slack, Microsoft Teams) - Documentation Collaboration (e.g., Confluence, Google Docs) 9. Software Quality Assurance - Code Quality - Test Coverage - Performance Monitoring - Error Logging and Tracking - Security Audits 10. Continuous Learning and Improvement - Technical Blogs and Newsletters - Online Courses and Tutorials - Conferences and Meetups - Open Source Contributions - Certifications This map provides a comprehensive overview of the various aspects of software engineering, including the software development lifecycle, programming languages, frameworks, tools, practices, and methodologies. It covers a wide range of topics, from requirements gathering and analysis to deployment and maintenance. The map also highlights important software engineering practices such as agile methodologies, design patterns, version control, code review, documentation, project management, collaboration, and continuous learning. Please note that this map is not exhaustive and there may be additional topics, technologies, and practices within each category. Software engineering is a vast and continuously evolving field, and new technologies and approaches emerge regularly." Here is a gigantic detailed map of hardware engineering: A[Hardware Engineering] --> B(Electrical Engineering) A --> C(Computer Engineering) A --> D(Mechanical Engineering) B --> E(Circuit Design) B --> F(Signal Processing) B --> G(Embedded Systems) B --> H(Power Electronics) B --> I(Control Systems) B --> J(Telecommunications) C --> K(Computer Architecture) C --> L(Digital Logic Design) C --> M(Microprocessor Design) C --> N(FPGA Design) C --> O(ASIC Design) C --> P(SoC Design) C --> Q(Firmware Development) D --> R(Mechanical Design) D --> S(Thermal Management) D --> T(Packaging and Enclosures) D --> U(Interconnects and Connectors) D --> V(Robotics) D --> W(MEMS) E --> AA(Analog Circuit Design) E --> AB(Digital Circuit Design) E --> AC(Mixed-Signal Circuit Design) E --> AD(RF Circuit Design) E --> AE(PCB Design) F --> AF(Digital Signal Processing) F --> AG(Analog Signal Processing) F --> AH(Image Processing) F --> AI(Audio Processing) F --> AJ(Speech Processing) G --> AK(Microcontroller Programming) G --> AL(RTOS) G --> AM(Device Drivers) G --> AN(Sensor Integration) G --> AO(Wireless Communication) H --> AP(Switch-Mode Power Supplies) H --> AQ(Power Factor Correction) H --> AR(Motor Drives) H --> AS(Battery Management Systems) H --> AT(Renewable Energy Systems) I --> AU(PID Control) I --> AV(Fuzzy Logic Control) I --> AW(Adaptive Control) I --> AX(Optimal Control) I --> AY(Robust Control) J --> AZ(Wireless Communication) J --> BA(Optical Communication) J --> BB(Satellite Communication) J --> BC(Fiber-Optic Communication) J --> BD(Network Protocols) K --> BE(Instruction Set Architecture) K --> BF(Pipelining) K --> BG(Cache Design) K --> BH(Memory Hierarchy) K --> BI(Parallel Processing) L --> BJ(Combinational Logic) L --> BK(Sequential Logic) L --> BL(Finite State Machines) L --> BM(HDL) L --> BN(Synthesis) M --> BO(CPU Design) M --> BP(GPU Design) M --> BQ(DSP Design) M --> BR(Memory Controller Design) M --> BS(Bus Architecture) N --> BT(HDL) N --> BU(Synthesis) N --> BV(Place and Route) N --> BW(Timing Analysis) N --> BX(Verification) O --> BY(HDL) O --> BZ(Synthesis) O --> CA(Physical Design) O --> CB(Timing Analysis) O --> CC(Verification) P --> CD(ARM-based SoC) P --> CE(RISC-V SoC) P --> CF(x86 SoC) P --> CG(GPU SoC) P --> CH(AI Accelerator SoC) Q --> CI(Bootloader Development) Q --> CJ(Device Driver Development) Q --> CK(Embedded Linux) Q --> CL(RTOS Integration) Q --> CM(Firmware Testing) R --> CN(CAD) R --> CO(FEA) R --> CP(CFD) R --> CQ(Tolerance Analysis) R --> CR(DFM) S --> CS(Heat Sink Design) S --> CT(Thermal Interface Materials) S --> CU(Fans and Blowers) S --> CV(Heat Pipes) S --> CW(Liquid Cooling) T --> CX(Plastic Enclosures) T --> CY(Metal Enclosures) T --> CZ(Ruggedized Enclosures) T --> DA(Ingress Protection) T --> DB(EMI Shielding) U --> DC(Board-to-Board Connectors) U --> DD(Wire-to-Board Connectors) U --> DE(RF Connectors) U --> DF(Fiber-Optic Connectors) U --> DG(High-Speed Interconnects) V --> DH(Kinematics) V --> DI(Dynamics) V --> DJ(Control Systems) V --> DK(Sensor Fusion) V --> DL(Machine Vision) W --> DM(Accelerometers) W --> DN(Gyroscopes) W --> DO(Pressure Sensors) W --> DP(Microfluidics) W --> DQ(Micromirrors) This map covers the major disciplines within hardware engineering (electrical, computer, and mechanical engineering) and breaks them down into various subdisciplines and areas of study. Each of these areas could be further expanded to include more specific topics, techniques, and technologies. Some key areas include: - Electrical Engineering: Focuses on circuit design, signal processing, embedded systems, power electronics, control systems, and telecommunications. - Computer Engineering: Deals with computer architecture, digital logic design, microprocessor design, FPGA and ASIC design, SoC design, and firmware development. - Mechanical Engineering: Covers mechanical design, thermal management, packaging and enclosures, interconnects and connectors, robotics, and MEMS (microelectromechanical systems). This map provides a comprehensive overview of the field of hardware engineering, showcasing its breadth and complexity. Hardware engineers need to have a strong foundation in multiple disciplines and be able to integrate knowledge from various areas to design and develop complex hardware systems. Machine Learning (ML), Artificial Intelligence (AI), Data Science, Big Data, and Data Analytics are interrelated fields that deal with data and its analysis to solve complex problems and make informed decisions. Here's a comparison of these fields: 1. Artificial Intelligence (AI): - AI is the broadest term, referring to the development of intelligent machines that can perform tasks that typically require human intelligence. - It encompasses various subfields, including machine learning, natural language processing, computer vision, and robotics. - AI aims to create systems that can perceive, reason, learn, and take actions based on data and knowledge. 2. Machine Learning (ML): - ML is a subset of AI that focuses on building algorithms and models that enable computers to learn and improve from experience without being explicitly programmed. - It involves training models on large datasets to identify patterns, make predictions, or take actions based on the learned patterns. - Key techniques in ML include supervised learning, unsupervised learning, and reinforcement learning. 3. Data Science: - Data Science is an interdisciplinary field that combines methods and techniques from statistics, mathematics, computer science, and domain expertise to extract insights and knowledge from data. - It involves the entire data lifecycle, from data acquisition and preprocessing to analysis, modeling, and communication of results. - Data scientists use various tools and techniques, including ML, statistical analysis, data visualization, and big data technologies, to solve complex problems and make data-driven decisions. 4. Big Data: - Big Data refers to the massive volumes of structured and unstructured data generated from various sources, such as social media, sensors, and transactions. - It is characterized by the "3Vs": Volume (large amounts of data), Velocity (high speed of data generation and processing), and Variety (diverse types and formats of data). - Big Data technologies, such as Hadoop and Spark, enable the storage, processing, and analysis of these large datasets to extract valuable insights. 5. Data Analytics: - Data Analytics involves the systematic examination of data to discover patterns, trends, and insights that can inform decision-making. - It encompasses various techniques, including statistical analysis, data mining, and data visualization. - Data analysts collect, clean, analyze, and interpret data to provide meaningful insights and support business decisions. These fields are closely related and often overlap in their applications and techniques. For example: - ML is a key component of AI and is widely used in Data Science and Data Analytics. - Data Science heavily relies on ML algorithms and big data technologies to analyze and extract insights from large datasets. - Big Data provides the infrastructure and technologies to store and process the data required for AI, ML, and Data Science applications. In summary, AI is the overarching field that encompasses ML, while Data Science and Data Analytics are broader disciplines that leverage AI, ML, and Big Data technologies to solve complex problems and make data-driven decisions. 7. Sakana https://www.maginative.com/article/new-startup-sakana-ai-wants-to-build-nature-inspired-artificial-intelligence/ 10. X AI" https://twitter.com/architectonyx/status/1749171280505643253 [Lie algebras: the math of rotations - YouTube](https://www.youtube.com/watch?v=gj4kvpy1eCE) [[2404.10952] Can Language Models Solve Olympiad Programming?](https://arxiv.org/abs/2404.10952) [NeurIPS 2023](https://nips.cc/virtual/2023/workshop/66530#wse-detail-83033) tentacle robot https://twitter.com/MachinePix/status/1778467765638345154 be not afraid robot https://twitter.com/khademinori/status/1761844043548352550 humanoid robot with more flexible limbs https://twitter.com/BostonDynamics/status/1780603212359205323 [Quanta Magazine](https://www.quantamagazine.org/physicists-use-quantum-mechanics-to-pull-energy-out-of-nothing-20230222/) [Imgur: The magic of the Internet](https://imgur.com/yODn5R7) complexity https://twitter.com/QuantumMemeing/status/1778132246261616846 [This Fusion Startup Is a Strong Newcomer - YouTube](https://www.youtube.com/watch?v=aP0H2-Dnsbk) [[2404.08801] Megalodon: Efficient LLM Pretraining and Inference with Unlimited Context Length](https://arxiv.org/abs/2404.08801) Cybergnosticism https://twitter.com/bhohner/status/1780277032816939181 [The Cosmic Battue](https://websim.ai/c/LHt0e2GtyqAkEEU48) https://twitter.com/burny_tech/status/1780378074569118153 https://twitter.com/fly51fly/status/1780193867264041232 [[2404.09173] TransformerFAM: Feedback attention is working memory](https://arxiv.org/abs/2404.09173) https://twitter.com/dw2/status/1733649018520064136 h/acc, u/pol, d/age Zamba, mamba plus Transformer https://twitter.com/QuentinAnthon15/status/1780280071304937978?t=wDEz651hD-HCXS2TN-UPXg&s=19 [Toward a unified theory of aging and regeneration | Regenerative Medicine](https://www.futuremedicine.com/doi/10.2217/rme-2019-0062) https://twitter.com/labenz/status/1780232966628675878?t=C5tmNkEyw532a9p1I8ZllA&s=19 [The Effective Altruist Case for Using Genetic Enhancement to End Poverty — EA Forum Bots](https://forum.effectivealtruism.org/posts/gaSHkEf3SnKhcSPt2/the-effective-altruist-case-for-using-genetic-enhancement-to) [Germany arrests suspected Russian spies over bombing plot](https://www.ft.com/content/9ee73d65-9575-410c-acba-3bc8b0bc08ae) Or do you want to pursue the scientific method to know the truth and get haunted by the replication crisis across all fields constantly destabilizing everything you know and the space of all possible interpretations being enormous and the space of all possible interconnected unanswered questions growing infinitely the more you know? [Imgur: The magic of the Internet](https://imgur.com/a4kKF9X) [3] [The Feynman Lectures on Physics Vol. II Ch. 19: The Principle of Least Action](https://www.feynmanlectures.caltech.edu/II_19.html) [4] Understanding Direct Preference Optimization | by Matthew Gunton [Understanding Direct Preference Optimization | by Matthew Gunton | Towards Data Science](https://towardsdatascience.com/understanding-the-implications-of-direct-preference-optimization-a4bbd2d85841) [5] Direct Preference Optimization Dpo - Lark https://www.larksuite.com/en_us/topics/ai-glossary/direct-preference-optimization-dpo [3] [[1805.08592v3] Computable Variants of AIXI which are More Powerful than AIXItl](https://arxiv.org/abs/1805.08592v3) [5] [[cs/0701125] Universal Algorithmic Intelligence: A mathematical top->down approach](https://arxiv.org/abs/cs/0701125) "