The abstraction trade-off

Software tooling is rapidly improving, new languages and frameworks releasing every day. In a nutshell what they do is reduce complexity we face while programming, in other words they provide abstractions for communicating with our computer components.

I guess the same way that it’s turtles all the way down, in software it’s abstractions all the way down. Using an example of C:

Following that example we always reach machine code, each step unwrapping abstractions to provide less intuitive but objectively simpler operations. Of course, this was the only way to make sophisticated software we have today.

Never in history have we had more software developers and more code being written than today, meaning that abstracting away the complexity made coding more approachable to the wider audience and in essence, made the activity less complex. But is it possible we traded away the complexity behind low level operations and lost nothing essential in the process?

# Kamal experience

Recently I’ve had an opportunity to get familiar with Kamal - a new deployment tool by 37signals, if you’re in the Ruby world you probably heard of it. DHH showed an impressive demo of it at Rails World 2024 deploying 2 applications in a manner of minutes. 🙀 🙀 I’ve seen a lot of these “happy path” showcases that promise all sorts of wonders and had a feeling in advance that this is yet another example of something being too good to be true.

Kamal is the perfect example of a complexity wrapper. There are a bunch of tools that the most recent devops wave inspired. It starts with Docker and containerization which leads to container orchestration tools. It all gets complicated really quickly, and Kamal sets out to hide away all of that complexity and allow developers to bootstrap their applications in the shortest amount of time possible.

So the story goes, the client wanted Kamal - something that developers after me could intuitively use without needing expert devops knowledge that is required with say, Kubernetes. So I started getting into it. One of the first things that you do is specify the server you want Kamal to deploy on. You should just pop in the IP and the private key you use to authenticate via SSH and should be good to go, something like this:

servers:
  web:
    - 127.0.0.1 # IP of the provisioned EC2 instance, localhost for placeholder

ssh:
  user: admin
  keys_only: false
  keys: [ './private_key' ]

Kamal has a command kamal setup that promises to “Setup all accessories, push the env, and deploy app to servers”. So I spun up a fresh EC2 instance, pointed to the IP I got from the terraform output (which launched an EC2 instance for me) and ran the setup command only to discover that running the app on the servers requires the user to be in the docker group. Meaning the infamous command sudo usermod -aG docker $USER never gets executed after docker gets installed. EC2s don’t allow SSHing in as a root user so trying that workaround is a no-go.

Okay, thankfully Kamal offers hooks, pieces of code that can be run after each step performed. So I added a hook kamal server exec 'sudo usermod -aG docker $USER' that gets run after docker gets setup, and it should add the user to the correct group on all my launched instances. But another issue arises! The SSH session never gets refreshed after docker is set up, so my user still does not belong in the docker group…

After much digging I noticed two separate Kamal commands, kamal server bootstrap and kamal deploy. This forces the SHH session to restart and my config finally works on a fresh instance. I swear this was more complicated to reverse engineer than just setting up my K3s cluster and adding a new Raspberry PI into the cluster. Mind you, this was only the first step of the deployment process.

One thing after another and I feel like I’ve spent more time reading the source code of Kamal to reverse engineer it to get what I want rather than writing all of it myself. I understand that this is an early stage open source project and that if I have issues with it I should benefit to it and make it better (will look into helping out once I get a bit more familiar with it). I’m trying not to be ungrateful and not say that Kamal is bad when comparing it to making your own K8s setup from scratch. I guess that the point I’m trying to get at is that abstractions will always be missing something, most often a way to make something in a much more simple manner by having the required low level knowledge. Missing that lower level knowledge we might never realize that we can create something better.

# Deeper understanding

My hypothesis got confirmed watching Casey Muratori perform what I can only call C magic. Having an in depth understanding of the underlying assembly code he basically made the program run faster from thin air.

I’ve encountered the exact same problem in ORMs as well. They are a perfect example of being a complexity wrapper since all they do is provide a wrapper for SQL. In the early stages of using ORMs in a project, it is readable and makes life way easier than writing SQL. But almost always I’ve gotten to a point where some specific queries get so complicated that using ORMs no longer makes sense and someone sooner or later starts writing raw SQL queries. Oh, the hours I’ve wasted on optimizing N+1 ORM queries that I knew how to write in raw SQL already.

This also ties in well with my thoughts about using LLMs for programming. I think of it like a great abstraction when used for coding. For the most part I can get answers to my simple questions, but getting the correct answers about specific intricacies seems almost impossible. Having that missing, expert knowledge proves essential in those cases.

In conclusion, both my Kamal and ORM experience made me think that if in those cases complexity got traded away for simplicity and limitations, have I been limiting myself all this time by not knowing more low level details? Would I be able to perform all sorts of wonders if I got familiar with it? Or is it better to live in delusion and consider the high level programming languages the lowest I can go?

I get a feeling that I should always be aware that I’m standing on the shoulders of giants and while I’m personally unable to see all the way to the bottom under me, that knowledge is ironically what makes my knowledge complete and exceptional and making progress in it is time never wasted, especially nowadays. That same niche knowledge is what separates the great people from average and is something that should be encouraged more.