From Curiosity to Research Pt. 4 - Understanding Transformers and Transformer Circuits, with a surprise curveball!
Covering the papers and my thoughts as I'm studying the foundations of transformers and their place in mechanistic interpretability.
This is my personal space on the web.
Hi, I'm Wellington. Currently, I'm a Senior Software Engineer at OneRail and co-founder of Noble Base.
This is my personal blog where I mostly write about programming and generative AI, but also sometimes general career advice and thoughts on the future of work.
Feel free to explore my blog posts, check out my projects, or learn more about me.
Covering the papers and my thoughts as I'm studying the foundations of transformers and their place in mechanistic interpretability.
I started with a top down approach to studying, but I realized how terrible that was for my learning style, so I switched it up.
Covering the curriculum I've built for myself to explore mechanistic interpretability in neural networks.
Covering the journey from initial curiosity about neural networks to a deeper understanding of how they make decisions.
First blog post of the year