AI Mastery
I want to learn AI theory and applications as quickly and as deeply as possible. I have no technical expertise in the field, but as a reseacher in Philosophy of Mathematics, I am at ease with complex
Here's our living briefing document, capturing the essence and strategic reasoning behind our "Antifragile AI Skills" curriculum journey, designed to seamlessly onboard any new team member.
We embarked on this journey seven months, one week ago, initiated by Romain Peter, with the ambitious goal of building a robust AI knowledge base. Our core objectives were clear: acquire technical definitions, precise conceptual understanding, and intuitive grasp of AI concepts; connect ideas to real-world use cases; and implement relevant concepts in our AI practice. Romain created an initial "Plan d'apprentissage de l'IA" (AI Learning Plan) structured into three phases: consolidating foundations, specialization, and continuous project work/monitoring. This plan meticulously outlined resources for core concepts (Types of AI, Machine Learning, NLP, Bias/Ethics) and practical tools (Python, essential libraries, development environments).
Romain also created a parallel Journey, "Prompt engineering tinkering," as a dedicated space for free experimentation with AI knowledge, reflecting our commitment to practical, hands-on learning. Romain began exploring prompt engineering strategies, including speculative decoding, which we proposed for multi-step instructions, similar to a "tree of thoughts" approach. Romain defined speculative decoding as a technique where a smaller, faster model generates candidate token sequences, verified by a larger, more accurate model. He provided an intuitive summary: "Imagine speculative decoding as a collaboration between a speedy junior writer and a meticulous senior editor." We developed a "Final Prompt Template" guiding the AI through steps: "Analyze the Query," "Generate Multiple Drafts," "Evaluate and Cross-Verify Drafts," "Synthesize the Definitive Answer," and "Present the Final Answer." We also undertook a detailed conceptual analysis of Socra's architecture as a SaaS platform, defining its "Knowledge Management," "Technical Foundation," and "User Experience Design." We explored iterative logics in prompt engineering, focusing on guiding an LLM to elevate output quality from level 'n' to 'n+1'. Our core insight, attributed to Romain, was to frame this as a mathematical analysis, treating excellence as a "limit function," aiming for "|L - f(n)| < ε as n → ∞." To measure the gap, we proposed a "Multi-dimensional Distance Metric."
Concurrently, seven months, one week ago, Romain Peter created a Socra on "Webhooks," emphasizing their role in AI automation. We defined webhooks as event-triggered HTTP callbacks for real-time application communication, highlighting their utility in streamlining workflows with platforms like Make.com and Taskade. This early focus demonstrated our commitment to practical AI automation tools, aiming to "streamline processes and reduce manual work."
Just seven months ago, Romain Peter cemented our understanding of "Actionable insights." We defined these as data-driven conclusions that directly inform decision-making, emphasizing their relevance, clarity, timeliness, specificity, and impact. We explored examples like personalized discounts, predictive maintenance, and fraud detection. For our journey, actionable insights are crucial for leveraging AI tools effectively, enhancing our learning, streamlining research, and managing experiences, aligning with Romain's goal of using AI to manage his autism.
Six months, four weeks ago, Eduarda Ferreira engaged deeply with our approach, creating a "Socra's Architecture" Socra. She recognized our grasp of Socra's underlying design, acknowledging Mike as the "brain behind it" for technical details. Eduarda provided invaluable suggestions for making Socra's logic more intuitive. Her core insights included prioritizing "Mental Models First, Features Second," specifically emphasizing Architectonics (goal hierarchy) and Inheritance (contextual flow). She proposed a "Visual Learning Path" via a "Journey of Journeys" tutorial and "Interactive Patterns" using template Journeys. Eduarda highlighted Romain's "mental gestures" concept as key for developing user intuition, expressing keen interest in collaborating on learning materials due to Romain's "unique perspective on mathematical philosophy and cognitive patterns." She outlined "Key Starting Points for New Users," focusing on reflective questions and visual thinking, and proposed a "Mental Gestures Framework" encompassing vertical navigation, context transformation, pattern recognition, and modular assembly, suggesting this approach aligns with natural human cognition. Eduarda also proposed future explorations, such as designing specific exercises around modular construction and creating videos to parallel everyday cognition with Socra's capabilities, aiming to show how "Socra empowers me to do things I can't do in everyday life."
Most recently, six months, three weeks ago, Romain Peter developed a comprehensive guide on "The Building Blocks of Generative AI." This structured overview covers seven key areas: Core Understanding, Technical Foundation, Practical Applications, Conceptual Framework, Technical Vocabulary, Mathematical Connections, and Learning Strategy. We delved into foundational models and LLMs, distinguishing between open-source (e.g., Meta AI's LLaMA) and closed-source (e.g., OpenAI's GPT-4) models, alongside their selection criteria. We explored the critical role of semiconductors, chips, cloud hosting, inference, and deployment, identifying key players like Nvidia, AWS, Google Cloud, and specialized providers such as d-Matrix and CoreWeave. We then examined vector databases (Pinecone, Chroma), orchestration frameworks (LangChain, Fixie AI), fine-tuning (Weights and Biases), labeling (Snorkel AI, Labelbox), synthetic data (Gretel.ai, Tonic.ai), and model supervision/AI observability (Fiddler.ai, Arize, WhyLabs), integrating discussions on model safety and ethical considerations (CredoAI). We anchored these technical concepts with real-world case studies from Science.io and Innerplay, showcasing how companies utilize this Generative AI infrastructure stack. This detailed exploration is a cornerstone of our technical understanding and practical application of Generative AI.
Following this insightful collaboration, six months, three weeks ago, Romain Peter initiated a new sub-Journey, "12-Weeks antifragile skills AI Learning," aimed at building resilient AI skills. Our design principle is clear: daily 30-minute reading and 30-minute practice sessions, complemented by detailed 2-hour lectures for each day. We believe this blend fosters both theoretical depth and practical application, crucial for genuine skill mastery.
**Week 1: From Philosophy of Mind to Computational Thinking**
We began our exploration by grounding AI in its philosophical roots, understanding that a strong theoretical foundation is key to navigating the rapidly evolving AI landscape.
* **Day 1: Foundations of Machine Intelligence.** Our focus was the Turing Test and its profound philosophical implications. We aimed to dissect intelligence criteria through analytical dialogue, comparing diverse AI responses. Our decision to dedicate a full 2-hour lecture to this topic stemmed from its foundational importance. We segmented the lecture to facilitate focused learning, integrate practical exercises exploring the Turing Test's mechanics, and establish clear recap questions. Crucially, we committed to explicitly linking Turing's original ideas to contemporary AI, especially Large Language Models (LLMs) and current AI ethics. This connection is vital for new team members to grasp the historical lineage of modern AI. Given Romain's deep background in Philosophy of Mathematics, we identified Gödel's Incompleteness Theorems and their relation to modern AI capabilities, particularly LLMs, as a critical area for exploration. Our deep study documents for this day provided a comprehensive overview of Turing's 1950 paper, emphasizing his "imitation game" and "learning machines" as revolutionary concepts that foreshadowed modern machine learning. We also meticulously detailed Turing's nuanced stance on Gödel's incompleteness theorem, where he acknowledged machine limitations but argued they don't negate the possibility of artificial intelligence or invalidate the imitation game. To broaden our philosophical lens, we included a summary of Douglas Hofstadter's "Gödel, Escher, Bach," noting its early influence on AI by critiquing symbolic AI and championing emergent intelligence.
* **Day 2: Computational Theory of Mind.** Our attention shifted to functionalism and the computer model of consciousness, with the practical goal of mapping our own decision-making processes in computational terms. The 2-hour lecture provided a robust framework for understanding the mind as a computational system. We structured it to cover historical roots (formal logic, the Turing Machine, cognitive psychology), core tenets (mental representations, computational processes, the Language of Thought hypothesis, Modularity of Mind), and supporting arguments (productivity, systematicity, cognitive science's success, intentionality). We proactively addressed criticisms, including the symbol grounding problem, the frame problem, the problem of consciousness, and embodied/situated cognition, recognizing that understanding limitations is as crucial as understanding capabilities. We integrated reflection questions throughout the lecture to foster deeper engagement and provided curated resources for further study, ensuring comprehensive learning.
* **Day 3: Representation & Symbol Grounding.** This day tackled the profound symbol grounding problem in AI, with the practice involving prompt engineering to test meaning representation. Our 2-hour lecture, "Bridging the Gap Between Symbols and Meaning," was meticulously designed. The morning session laid the groundwork for "The Symbol Grounding Problem," defining symbols and meaning, introducing Stevan Harnad's pivotal challenge, and differentiating between bottom-up and top-down processing. We then explored diverse "Representation in AI Systems," including vector representations, embedding spaces (like Word2Vec and GloVe), semantic networks, and the architecture of meaning (compositionality, distributed representations, context sensitivity). The interactive discussion section included crucial case studies on GPT models, visual-language models, and the challenging representation of concrete versus abstract concepts, coupled with probing questions about AI understanding versus mere processing. The afternoon session, "Practical Applications," delved into the specifics of implementation, covering detailed representation techniques (Word Embeddings, Contextual Representations, Multi-modal Grounding, Knowledge Graphs) and common challenges (ambiguity, context-dependency, abstract concepts, cross-modal alignment). We then moved into "Experimental Design," outlining how we test AI symbol grounding, including symbol manipulation tests, meaning preservation experiments, context switching challenges, and abstraction level analysis, complete with rigorous evaluation metrics. We also created a dedicated document, "The Grounding Problem in AI and its Relevance to Recent Advancements (Especially in 2024)," which elaborated on Harnad's symbol grounding problem and Mollo and Millière's vector grounding problem. We identified five types of grounding: sensorimotor, communicative, epistemic, relational, and referential. We specifically highlighted the increased focus on "grounding" in 2024, particularly in mitigating AI hallucinations through Retrieval-Augmented Generation (RAG) methods being developed by Google and Microsoft, underscoring the real-world impact of this theoretical challenge. The document also emphasized the critical importance of ground-truth data, the application of grounded AI in diverse fields, and the evolving public perception. We outlined key research directions like neuro-symbolic AI, embodied AI, cognitive architectures, and Explainable AI (XAI), while acknowledging persistent challenges such as data bias, contextual understanding, and dynamic environments.
**New Subjourney: Databases**
In parallel with our core curriculum, we recognized a fundamental gap: the necessity of database knowledge for a comprehensive understanding of AI. Consequently, we initiated a new subjourney specifically for "Databases," starting from scratch. Our clear goal is to master database theory fundamentals and gain a practical understanding sufficient for tackling small projects and, crucially, for grasping the technological and AI advancements that inherently require database knowledge. This strategic decision ensures our team members are not only proficient in AI concepts but also equipped with the underlying data management skills that power modern AI systems. For Day 1 of this subjourney, Romain detailed a 30-minute theoretical module on fundamental data types (Integer, Floating-Point, Text, Boolean, Date/Time, NULL) and structures (arrays, lists, and especially tables as the cornerstone of relational databases). The 30-minute practical module involved setting up SQLite and learning essential dot-commands. We practiced creating and populating a `users` table using `CREATE TABLE` and `INSERT INTO` statements, followed by simple data retrieval with `SELECT` commands. This day concluded with an assessment and considerations for visual learning and common pitfalls. Romain then provided a detailed explanation of the `SELECT` command, covering its basic syntax, filtering with `WHERE` and logical operators, sorting with `ORDER BY`, and limiting results with `LIMIT`. We also explored aggregate functions (`COUNT`, `SUM`, `AVG`, `MIN`, `MAX`), grouping with `GROUP BY`, and filtering groups with `HAVING`, as well as handling `NULL` values. This explanation also briefly touched upon modern SQL features like Window Functions and Common Table Expressions (CTEs), including practical exercises and a visual flowchart of the `SELECT` query's order of operations. Finally, Romain presented SQL's syntax essentials, emphasizing its logical, language-like, and declarative nature, which is crucial for AI applications in data preprocessing, querying training datasets, and efficient data management. We covered core CRUD operations and typical `SELECT` clause structure, logical operators, and common aggregate functions. Romain highlighted common SQL patterns relevant to AI/ML, such as data sampling, feature extraction, and data cleaning queries. He also shared "gotchas" and best practices, including caution with SQL injection and using aliases. Looking beyond SQL, Romain noted the future benefits of exploring Object-Relational Mapping (ORM) tools, like Django's ORM, a favorite of Mike's, as a bridge between SQL databases and object-oriented code, underscoring that mastering SQL fundamentals will greatly enhance our ability to leverage ORMs and Python's data libraries effectively, as SQL's logical thinking patterns translate well. Mike is also building a Python library for rapid code mutation that Romain will be able to use.By Romain Peter