My Development Stack

My Development Stack

I’ve been programming since for 30+ years now. There have been a few times in my life when I’ve felt like a kid in a candy shop who couldn’t wait to get to their computer and start coding.

First - the very early days with Basic and a junky ZX Spectrum clone.

Everything was new and amazing and 🤯s were common. This was pre-internet, so we had to get programs on cassettes, load them up with, and then start to mess with them. The internet changed everything of course - programming took a bit of a hit as surfing (the web kind, non-athletic nerd here, thanks) took over.

Second - joining Google in 2008.

It’s hard to overstate how ahead of the rest of the industry Google was back then. Engineering practices that are considered obvious were anything but, except at Google. Code reviews were mandated, you could only choose amongst a few blessed languages, documentation was excellent, new libraries, tools, and infrastructure were being written every month that gave everyone super powers (protobufs, gfs, colossus, borg).

Third - now.

LLMs are crazy and every week, I’m breaking a new frontier in coding. The development world has evolved greatly over the last 30 years and gotten significantly more complex. This complexity is needed - as a species, we compute more than we’ve ever computed, and optimization matter. In the past, we’ve tried to wrap and hide this complexity by trying to create libraries that simplify things, only to inevitably run into the one time when we need to break the abstraction, and then everything goes to hell. However, a new fighter has entered the areana; two new fighters actually. One: LLMs in the co-pilot flavor, as an in-context assistant, and two: LLMs in the ChatGPT flavor, as an open-ended chatbot. Between these two, dealing with esoteric libraries, or arcane syntax, is easy again. I can write code in mostly-python and get correct python, or write code in english, and get mostly-python. Either way, coding is FUN again.

The Stack

If you’re doing any AI LLM stuff, your stack almost certainly involves either python or Typescript or both. I welcome this - I love python’s clean and efficient syntax and the batteries included philosophy of the Python community. With the development of newer languages like Mojo🔥, I expect Python becomes even more powerful (by potentially morphing into Mojo.)

As a result, I’m all in on Python and believe that developing one’s Python familiarity and using it as the primary stack language is a good choice. Here is a list of libraries I’m using regularly. I intend to keep this list current as I continue to build things:

Python basics

1. Pydantic - provides nice class helpers, similar to dataclass. By providing types, we get free validation while constructing objects, and serialization to dicts and JSON objects, both of which matter when dealing with OpenAi APIs (and other LLMs). pydantic[dotenv] helps in dealing with setings and .env files.
2. FastAPI - My servers are pure JSON API servers, written in FastAPI. It makes writing servers a breeze, has async request handling (needed for dealing with LLMs because everything takes so long), generates OpenAPI specs automatically, along with a neat little frontend where you can play with the API directly.
3. jupyter - I find it easy to tinker with things using a notebook, and then convert stuff to a library or app once I have a handle on things.
4. pytest and pytest-watch - Pytest has made unit testing a breeze. pytest-watch just monitors files for change and runs your test in a loop. Open a shell, start ptw and code away. Plus coverage is easy with pytest-cov.
5. click - Makes dealing with command line arguments a breeze eg. when writing a batch script.
6. poetry - Great way to handle python package management. python -m venv venv or virtualenv or other options work great as well. Poetry is just easier.

Not critical but very useful:

1. rich - Provide prettier output for most objects. from rich import print as pprint and then use pprint(obj) to get beautiful output.

AI stuff

1. openai - OG AI lib for LLM interactions. Great API out of the box and easy to use.

Interesting but not absolutely required:

1. langchain - Langchain is almost its own little world and you have to understand it to use it. Since its often a wrapper around a bunch of careful prompts, one doesn’t absolutely need langchain and for a lot of applications, it’s overkill.
2. chromadb or pinecone - Similarly, most people don’t need vectorstores. But if you do, they’re great options.

Early but interesting:

1. marvin - Build a chatbot quickly. The idea of an AI Function - essentially an ai_fn decorator that you can stick on any function, give it a docstring, and let the LLM do the rest, is incredible.

Frontend stuff

My frontends are written in simple React. I compile them into a set of static files and serve them up via the same FastAPI server above. This gets rid of all the CORS crap (its coming from the same server, so its all fine) but adds an additional build step. Oh, and its not optimized in the way some of the crazier Vercel/Next stuff is.

To be honest, I find frontend work quite intimidating and challenging. There are thousands of JS packages and frameworks, and all seem to have slightly different ways of expressing the same underlying complexity - how to manage application state and get the UI to work with said state. I eventually broke through with an understanding of React, thanks to Josh Comeau’s excellent Joy of React course. Still, I consider myself a 101 level frontend developer, so take everything with a grain of salt.

One breakthrough tool for frontend coding has been Vite. It takes all the mess of packaging and bundling and minifying and whatnot, and just puts it all together in one thing. Just use Vite and simplify life.

Test coverage

Unit testing is (was) painful. Not only would you have to think of how to test your code, you’d have to then construct all the mock objects and inject dependencies and what not. Net result - I almost never unit tested non-production code.

This has changed completely. I simply plug in my python code into ChatGPT and ask it to generate unit tests. It does a reasonably good job, I fix it up, have pytest running continuously in the background, et voila! 100% code coverage for everything.

It’s amazing to be coding toys with 100% coverage. Making changes and doing drastic surgery is just fine - unit tests are catching things left and right and if coverage drops, GPT fixes things up again.

The Joy of Hacking

If you’re a recovering engineer like me, there has never been a better time to dive into coding again. The tools are amazing, there are new developments every day, and you can build things that were impossibly hard just a few years ago. I’d love to see your hacks and projects - please reach out on Twitter and drop me a line.