When Modularity Trumps Efficiency

Not quite sure what to write about today.

But I'm still going hard on this SE KR Factorio game so there's always an infinite amount of content to talk about there, even if no one here really plays the game some of it is still relevant to the "real world" as it were.

I'm currently 334 hours into this game alone and still not even close to beating it, which is obviously batshit insane. I've never beaten the game using the Space Exploration mod, which is unsurprising seeing as how long it takes.

This is my second attempt after abandoning my first SE game. Why did I abandon my first game? Well it had been over a year since I had played it, my infrastructure was falling apart and failing to scale, and I really wanted to try both the Space Exploration mod and the Krastorio 2 mod at the same time. Both of these mods are complete overhauls of the game and they should be completely incompatible, but it is legit someone's "job" to make sure they stay compatible. Not sure if they get paid or not.

In any case the combination of both mods makes the game surprisingly more complex as there are even more resources that need to be managed. Luckily I have enough experience with the game to now understand what works and what doesn't: and these rules also apply to programming and logistics in general.

Be willing to waste resources for convenience.

This is a huge lesson to be learned in Factorio and in real life as well. In fact just the other day I was presented with a decision that prompted me to perform the exact opposite action than I normally would because of my Factorio experience. It's going to sound stupid but hear me out.

Alright so I had just gotten back from the grocery store and noticed there was some garbage in the car that needed to be thrown away. I live on the third floor in an apartment complex. The communal dumpster is at least 50 yards away from where I park. Should I walk over to the dumpster and throw away this small amount of trash, or just bring it upstairs with me and throw it in the bin there?

Normally I would just walk over to the dumpster because whatever. The idea of bringing it upstairs only to throw it away and then have to throw it away again irks me. It seems inefficient. I have to backtrack and throw the same piece of garbage away twice. Surly it must be smarter to just walk over and throw it away directly, and the only reason not to do it that way is if I was being lazy or procrastinating the inevitable.

However, after thinking about it in terms of gaming, a thing I do often when I'm playing a lot, it actually makes more mathematical sense to take the garbage out twice, as inefficient as it may seem at first glance. It's all about the overhead and modularity of the situation.

If I walk to the dumpster and throw away the garbage directly then I have to walk at least 100 yards. If I choose not to I have to go upstairs, but I have to go upstairs anyway, so that ends up being net-zero extra work being done. Say the amount of garbage I need to throw away is one tenth the size of my bin upstairs. That means on average every 10 times this situation happens I'm going to immediately need to take the garbage out, but 9/10 times I have the luxury of doing nothing.

If I take the trash to the dumpster all ten times that's at least 1000 yards of walking, whereas if I empty the bin one out of ten times I have to go up and down the stairs one extra time, but only have to walk 100 yards. So the question becomes: which situation is more ideal for the least amount of work? 900 yards of walking (1000-100) or going up and down 2 flights of stairs one time? Within this context it becomes painfully obvious that walking directly to the dumpster to throw away a small amount of garbage is a huge waste of time when the variables are isolated like this. 900 yards is more than half a mile, and the dumpsters are AT LEAST that far away (50y); probably farther. Obviously going up and down the stairs one time is way easier than walking half a mile.

The logistics of it all.

Again this might seem like a silly example, but so are things like the Traveling Salesman Problem and the Byzantine General's problem. When the problems are taken literally they seem ridiculous and even pointless, but when applied elsewhere can create a certain clarity for solutions that are unmatched by other examples.

The reason I was thinking about taking out the trash in this odd way is that I have a similar problem in Factorio. I have a research facility in orbit above my main base. This research facility has access to a lot of recycled cargo rocket parts that get reclaimed from the massive amount of resources I send into orbit.

The problem is that cargo rocket parts are needed everywhere else except my orbital platform. Also launching rockets is expensive with high overhead, but can accommodate a large amount of volume.

Much like taking out the garbage directly to the dumpster, there is a distinct urge in this situation to set up multiple rocket pads that point to multiple bases that need cargo parts so I can send outlying planets what they need directly from the source. However, as discussed earlier, this is not the correct answer, even though it seems like it might be efficient and might be able to cut down on total rocket launches, there are simply too many reasons to not build the infrastructure in this way.

Scalable and modular solutions are the answer.

The trick here is that scalable and modular solutions are almost always less efficient than a custom optimized design. There's a reason why Google uses scripting languages like Python all the time, and it has nothing to do with efficiency, as Python is an extremely inefficient programming language.

The way to navigate these situations is often to prototype the most modular and scalable solution first and see if that works, and then optimize the bottlenecks if any exist. You'd be surprised how tempted developers can be to optimize a solution that never needed fixing in the first place. This can actually make it worse by making it less modular and less agile than before. It can be very hard to upgrade a streamlined integrated circuit once it's been built, just like it's impossible to use Bitcoin ASIC machines to mine Litecoin or any other non-SHA256 POW token.

Technology creates abundance.

Efficiency used to matter a lot more than it does now. Back in the day a programmer might only have a few megabytes or even a few kilobytes of RAM to work with. That's a huge limitation for any app. Now we have Google Chrome browser hogging gigabytes or RAM for all kinds of reasons. Of course the programming for a phone is different than the programming of a PC because they have vastly dissimilar resources available. It all depends on context.

---

So what's the solution?

Well the solution in my Factorio game is to have one rocket on my orbital platform that acts as a recycling tool. It sends everything the base doesn't need back down to the surface where the assets become centralized and comingled with all my other products. From here I already have the infrastructure in place to keep all my outlying bases stocked up so while it might be slightly inefficient to do it this way on paper the reality of the situation, volume, is the real determining factor.

Centralization is often the answer

This is certainly something we crypto people don't like to hear, but there's a reason the world exists like it does. There's a reason why a corporation would sooner throw a product away or let someone have it for free rather than try to repair or refurbish it. They already have the systems in place to impose the current ideology, and creating a refurbishing center would be low volume and high overhead: aka not worth it in a lot of situations.

Conclusion

People often underestimate the complexity of logistics and the myriad of problems that go along with it. You buy something from Amazon: it shows up at your door a day later. Don't think about it too much. In fact, now this is the outcome we expect even though a couple decades ago it would have been unheard of.

There's still a lot of points and examples relevant to this conversation that I simply didn't even get around to. How do we ensure the solutions we are creating are scalable? When is decentralization (and the implied inefficiency of it) appropriate? When can we get the best of both worlds? I suppose I'll have to answer all these questions and more in another post.