/posts/good_prompt/Tue, 01 Jul 2025 12:00:00 +0000/posts/good_prompt/<p>Here are some prompts saved from twitter.</p> <p>The consensus opinion is that prompt engineering is a useless endeavour, because any gains must soon be eaten up by more fundamental improvements to the model. This may be true, but I would still be pleasantly surprised if Socratic questioning became the default model response to a request for tutoring in the presence of anything more than the mildest profit incentive. It&rsquo;s often not pleasant to realize how little you truly understand. And regardless, I think this page will make for a cool time capsule.</p>/posts/apple_bug/Tue, 01 Jul 2025 12:00:00 +0000/posts/apple_bug/<p><strong>Books</strong></p> <p>Make learning while reading delightful:</p> <ul> <li>The dictionaries could be expanded to include more loanwords, obscure words, phrases, and slang. Presumably dictionary size is less of an issue now that they&rsquo;re not printed on paper. If the Merriam-Webster word set must be used for prestige or reliability guarantees, use it but fall back to a larger dictionary when a query has no matches.</li> <li>When the user queries words or phrases not in any dictionary, use an LLM to try to explain it. A prompt specifying the book, release date, and the plot so far can produce a much more informative explanation than a dictionary.</li> </ul> <p>Avoid hawking and touting:</p>/extended_quotes/personal_history/Sun, 01 Jun 2025 12:00:00 +0000/extended_quotes/personal_history/<p>Born into a traditional wealthy East coast family, Graham&rsquo;s father initially bought the Washington Post in a bankruptcy auction, and when he moved on it seemed natural to Graham that the editorship would pass to her husband Philip rather than herself. But in 1963 Philip died by suicide and Graham became the leader of the Post. Despite her lack of experience, she sourced a string of strong collaborators and advisors and expanded her circle of competence over time in a recognisably feminine way, eventually becoming a little less traditional:</p>/posts/diplomacy/Thu, 17 Apr 2025 12:00:00 +0000/posts/diplomacy/<p>My favourite introduction to Diplomacy is the <a href="https://www.thisamericanlife.org/531/transcript">This American Life episode</a> where they send an elite American diplomat to help out an amateur at the National Championships. The most important takeaway is that the rules are very simple, such that performance is a fairly pure measure of the ability to negotiate and plan.</p> <p>Diplomacy is one of the preferred games for cutting edge RL research, alongside the co-operative card game Hanabi, poker, and a slew of video games including Pokemon and Among Us. I enjoy its vaguely threatening aura - players fight for control of Europe, and the threat of backstabbing is ever-present.</p>/drafts/stream/Thu, 17 Apr 2025 12:00:00 +0000/drafts/stream/<p><a href="https://x.com/MechanizeWork/status/1912904151874625928">https://x.com/MechanizeWork/status/1912904151874625928</a></p> <p>Core people at Epoch AI have founded Mechanize to develop RL environments, benchmarks, and datasets to automate white collar work.</p> <p><a href="https://x.com/nickcammarata/status/1912934075628949982">https://x.com/nickcammarata/status/1912934075628949982</a></p> <p>Interpretability lab Goodfire has raised $50M to do interpretability research. They have great taste and great people, very bullish.</p> <p><a href="https://152334h.github.io/blog/knowing-enough-about-moe/">Knowing Enough About MoE to Explain Dropped Tokens in GPT-4 </a></p> <p>Great piece on a surprising design choice for the MOEs in GPT-4 and DeepSeek that results in token dropping. I don&rsquo;t believe this design will be used for much longer.</p>/about/Sun, 13 Apr 2025 12:00:00 +0000/about/<p>I&rsquo;m Lucia, a neural network interpretability researcher interested in unsupervised methods and the aspects of human intelligence that evade description. I became interested in machine learning research after reading a <a href="https://waitbutwhy.com/2015/01/artificial-intelligence-revolution-1.html">blog post</a> about superintelligence during highschool.</p> <p>I like to train strange sparse autoencoders, most recently binary TopK autoencoders (BAEs) and TopK SAEs trained on backward pass gradients. I&rsquo;m interested in using the board game Diplomacy as a testbed to study the extent to which interpretability tools provide strategic value in multi-agent environments. I also have active projects in the fields of <a href="https://github.com/EleutherAI/bergson">data attribution</a> and tamper-resistant unlearning.</p>/posts/library/Sun, 13 Apr 2025 12:00:00 +0000/posts/library/<p>&ldquo;I’ve developed a great reputation for wisdom by ordering more books than I ever had time to read, and reading more books, by far, than I learned anything useful from, except, of course, that some very tedious gentlemen have written books. This is not a new insight, but the truth of it is something you have to experience to fully grasp.&rdquo;</p> <h4 id="meditation-and-the-social-mind">Meditation and the social mind</h4> <ul> <li>Zen Mind, Beginner&rsquo;s Mind, Shunryū Suzuki</li> <li>The Science of Enlightenment: How Meditation Works, Shinzen Young</li> <li>Right Concentration: A Practical Guide to the Jhanas, Leigh Brasington</li> <li>In the Buddha&rsquo;s Words: An Anthology of Discourses from the Pali Canon, Bhikkhu Bodhi</li> <li>Eastern Body, Western Mind, Anodea Judith</li> <li>Fidelity: How to Create a Loving Relationship That Lasts, Thich Nhat Hanh</li> <li>Already Free: Buddhism Meets Psychotherapy on the Path of Liberation, Bruce Tift</li> <li>Seeing That Frees: Meditations on Emptiness and Dependent Arising, Rob Burbea</li> <li>Be Here Now, Ram Dass</li> <li>The Elephant in the Brain, Robin Hanson</li> <li>Pyschoanalytic Diagnosis: Understanding Personality Structure in the Clinical Process, Nancy McWilliams</li> <li>Inner Gold: Understanding Psychological Projection, Robert A. Johnson</li> <li>The Dance of Anger, Harriet Lerner <ul> <li>In addition to being practical and insightful, The Dance of Anger makes an appearance in her son&rsquo;s autofictional <em>The Topeka School</em>.</li> </ul> </li> <li>Impro, Keith Johnstone</li> </ul> <h4 id="biographies-and-auto-biographies">Biographies and auto-biographies</h4> <ul> <li>The Singapore Story and From Third World to First, by Lee Kuan Yew <ul> <li>The &ldquo;Zero to One&rdquo; of building a first world nation.</li> </ul> </li> <li><a href="/extended_quotes/personal_history">Personal History</a> by <a href="https://en.wikipedia.org/wiki/Katharine_Graham">Katharine Graham</a> (1917-2001), former leader of the Washington Post.</li> <li>Steve Jobs and Einstein, both by Walter Isaacson</li> <li>Mao: The Unknown Story, by Jung Chang and Jon Halliday</li> <li>Just Kids, by Patti Smith</li> <li>Inside the Aquarium, by Viktor Suvorov (Vladimir Rezun) <ul> <li>A tank company commander, then a GRU intelligence officer. 1970s.</li> </ul> </li> <li>Korzybski, by Bruce Kodish. From 1879.</li> <li>A Higher Loyalty, by James Comey</li> <li>Hillybilly Elegy, by JD Vance</li> <li>Diary of Anaïs Nin, Volume VI. 1955-1966.</li> </ul> <h4 id="non-fiction">Non-fiction</h4> <ul> <li><a href="https://en.wikipedia.org/wiki/Man%27s_Search_for_Meaning">Man’s Search for Meaning</a></li> <li><a href="https://en.wikipedia.org/wiki/Hiroshima_(book)">Hiroshima</a></li> <li><a href="http://www.daviddfriedman.com/Legal%20Systems/LegalSystemsContents.htm">Legal Systems Very Different From Ours</a></li> <li><a href="https://michaelpollan.com/books/how-to-change-your-mind/">How to Change Your Mind</a></li> <li><a href="https://factfulnessquiz.com/">Factfulness</a></li> <li><a href="https://www.textpublishing.com.au/books/the-hatred-of-poetry">The Hatred of Poetry</a></li> <li>The Death and Life of Great American Cities, Jane Jacobs</li> <li><a href="https://en.wikipedia.org/wiki/Albion%27s_Seed">Albion&rsquo;s Seed: Four British Folkways in America</a></li> </ul> <h4 id="business-and-popular-science">Business and popular science</h4> <ul> <li>The Hard Thing about Hard Things, by Ben Horowitz </li> <li>Zero to One, by Peter Thiel</li> <li>The Singularity is Near, by Ray Kurzweil</li> <li>The Quantum Frontier, by Don Lincoln</li> <li>Sapiens, by Yuval Noah Harari (although in my opinion <a href="https://www.theintrinsicperspective.com/p/the-gossip-trap">this essay</a> by Erik Hoel is better)</li> <li>Surely You’re Joking, Mr Feynman!</li> <li>The Structure of Scientific Revolutions, Thomas Kuhn</li> <li><a href="https://www.goodreads.com/book/show/24040194-life-on-the-edge">Life on the Edge: The Coming of Age of Quantum Biology</a></li> <li><a href="https://atulgawande.com/book/the-checklist-manifesto/">The Checklist Manifesto</a></li> <li>Poor Economics, Abhijit Banerjee and Esther Duflo</li> <li><a href="https://www.earlymoderntexts.com/assets/pdfs/bacon1597.pdf">Essays by Francis Bacon</a>, Francis Bacon (translated by Jonathan Bennett)</li> </ul> <h4 id="autofiction">Autofiction</h4> <ul> <li>Trip, Taipei, and Leave Society, all by Tao Lin</li> <li>My Struggle, Karl Ove Knausgård</li> <li>Early Work, Andrew Martin</li> <li>Real Life, Brandon Taylor</li> <li>10:04, The Topeka School, and other works by Ben Lerner</li> <li>Let Me Tell You What I Mean by Joan Didion</li> </ul> <h4 id="fiction">Fiction</h4> <ul> <li>The Unbearable Lightness of Being, Milan Kundera</li> <li>Dream of the Red Chamber, Cao Xueqin</li> <li>Love in the Time of Cholera, Gabriel García Márquez</li> <li>Infinite Jest, DFW</li> <li>East of Eden, Steinbeck</li> <li>The Iliad</li> <li>The Odyssey</li> <li>Atlas Shrugged, Ayn Rand</li> <li>Rosa Alchemica, William Butler Yeats</li> <li>The Nameless City, H.P. Lovecraft</li> <li>Gilead, Marilynne Robinson</li> <li>The Culture novels by Ian Banks</li> <li>The Machine Stops, E. M. Forster</li> <li>The Gentle Seduction, Marc Stiegler <ul> <li>Inspired <a href="https://www.asimov.press/p/gentle-romance">The Gentle Romance</a></li> </ul> </li> <li>Remembrance of Earth&rsquo;s Past by Liu Cixin (i.e. The Three Body Problem)</li> <li>Cryptonomicon, Neal Stephenson</li> <li>The Nexus Trilogy by Ramez Naam</li> <li>Wolf Hall, Hilary Mantel</li> <li>The Curse of Chalion and Paladin of Souls, both by Lois McMaster Bujold</li> </ul> <p>&ldquo;I feel sometimes as if I were a child who opens its eyes on the world once and sees amazing things it will never know any names for and then has to close its eyes again.&rdquo;</p>