Prompt engineering tinkering
A place to tinker freely about how to use the knowledge I gather about AI.
We initiated this journey to create a dedicated space for exploring applications of AI knowledge, a place to "tinker freely about how to use the knowledge I gather about AI."
Romain Peter began exploring prompt engineering strategies:
* **Speculative Decoding (7 months ago):** We explored speculative decoding as an LLM engineering strategy to increase generative efficiency. This led us to propose its use as a multi-step instruction within a prompt, akin to the logic of a "tree of thoughts" prompt. Romain defined speculative decoding as a technique where a smaller, faster model generates candidate token sequences, which are then verified by a larger, more accurate model. He offered an intuitive summary: "Imagine speculative decoding as a collaboration between a speedy junior writer and a meticulous senior editor." We developed an optimized prompt template guiding the AI through steps: "Analyze the Query Thoroughly," "Generate Diverse Drafts," "Evaluate Each Draft Critically," "Synthesize the Final Answer," and "Present Your Definitive Answer." We also identified alternative prompting strategies following the same "broad to narrow" pattern, such as Brainstorm and Prioritize, Progressive Elaboration, and Socratic Questioning.
* **Conceptual Analysis (7 months ago):** We undertook a detailed analysis of Socra's architecture as a SaaS platform to understand its underlying structure. This involved defining its "Knowledge Management" (Conceptual Framework, Data Organization), "Technical Foundation" (Core Infrastructure covering Data Architecture and Processing Engine, and Platform Services covering Security Framework and Integration Layer), and "User Experience Design" (Interface Architecture covering Information Hierarchy and Interaction Design, and Performance Optimization covering Response Time and Scalability Design). This analysis provided a structured understanding of how such a platform functions.
* **Iterative Improvement (6 months, 4 weeks ago):** We explored iterative logics in prompt engineering, focusing on guiding an LLM to elevate output quality from level 'n' to 'n+1'. Our core insight, attributed to Romain, was to frame this iterative improvement using a mathematical analysis perspective, treating excellence as a "limit function." We defined the goal as achieving "|L - f(n)| < ε as n → ∞," where 'L' is the excellence limit, 'f(n)' is output quality at iteration 'n', and 'ε' is an acceptable margin of error. To measure the gap, we proposed a "Multi-dimensional Distance Metric": Gap(n) = Σ wi|Li - fi(n)|, where 'wi' is the weight and 'Li' is the excellence criterion for dimension 'i'. We aim to refine these dimensions, weights, and scoring methods to systematically measure the difference between the current output and our defined standard of excellence.By Romain Peter