Taming the Supernode: My IPFS Maintenance Adventure

As part of this self-imposed mission I’ve given myself, I’ve had to really dive deep into the pond of IPFS—Interplanetary File System. Some of you may know it as “how the crypto folks store files,” but what’s not so well known is that it really has nothing to do with crypto, at least not directly.

When I began this little journey of discovery, I had plenty of emotional ups and downs. Some days I thought IPFS was the most amazing thing in the world, and other days I’d get frustrated with its quirks and shortcomings. These days, however, I think I’ve grown to genuinely appreciate the tech and understand its incredible potential. Most of my frustration came from my own lack of knowledge—something I painfully admit.

Among the important lessons I’ve been able to tattoo into my brain is the need for proper storage management. Unlike modern filesystems, IPFS—especially if you're running an older version—can get fragmented. I’m old enough to remember how crucial it was to defragment your drives. A problem of the past, for sure, but one that was part of my teenage routine.

The biggest challenge we’ve faced with 3Speak’s supernode is that we have to perform these tasks without stopping the show. As I type that, I can clearly imagine Freddie Mercury belting out “The show must go on!” And he’s right. We can’t—or rather, we shouldn’t—tell everyone to go away for a bit while we do maintenance unless it’s a very special case. After all, taking care of the user base is job number one.

As part of this maintenance work, I started making little scripts to run various tasks without having to type until one of my fingers ends up inside a Ziplock bag. These scripts started as just a handful, but they somehow evolved into a collection of over 21 scripts—some of which I no longer remember the purpose of. Funny how that works.

On my to-do list, a glaring item near the top was to condense these scripts and create a graphical interface that would allow any of us to run maintenance on the node. I keep telling @eddiespino that we need to put in the work upfront so that in the future we can cruise. That advice is rock solid for most things in life.

What’s hilarious to me is that as I began working on this GUI for the scripts, I ended up going back to only six—back to the original handful. I think I was in some sort of manic state while coming up with “improvements,” fueled by caffeine overdosing and lack of sleep. With that said, I think I’m finally done with it, and I’ll be deploying my shiny little tool later today. I’ve already spoken with Eddie about the logistics.

Just as I was about to call it a day, I began thinking of something else. A conversation with my new Copilot friend cemented an idea for an experiment.

You see, among the tasks one must do, a crucial one is garbage collection. And it is exactly what it sounds like.

In IPFS, when you unpin a file—basically deleting it—it’s not truly gone. You’ve only removed the reference to it, but the space on the drive is still being used. This is why, despite “deleting” thousands of failed or stuck videos, the drive usage has barely gone down. Yes, performance improves because the table of contents gets smaller, but the precious space doesn’t return home.

The bright idea is to run a smart script as a service. Something that activates at 2 a.m., maybe until 4 a.m.—adjustable, of course—and performs small spurts of garbage collection. It checks CPU load, number of clients connected, and the supernode’s demand, and suspends itself if needed. This means it’s slow, painfully slow, but honestly that’s the only way to keep the show alive.

I’m feeling pretty confident that with this little tool and this tiny service of lazy garbage collecting, we’re going to get a lot more out of 3Speak’s infrastructure.

Anyway, enough nerdy ranting for one night.

—MenO