V4V.app Backend Version 2 Is Getting Close
Vote for Brianoflondon's Witness KeyChain or HiveSigner
Support Proposal 342 on PeakD
This is a value for value post: see the explanation in the footer.
This post is very long overdue. I've been massively busy building a complete re-write of my v4v.app system backend to dovetail with what @vsc.network are building. My Hive Witness position has a dropped a bit so if you're looking to help me, you can consider giving me your witness vote.
Unless something dramatic happens, I won't be able to attend @HiveFest this year which I'm a bit sad about but I'll be there in spirit.
The Big Idea
My whole architecture and code for @v4vapp was pretty much the first major code I've ever written and put out on the Internet. I was a coder when I was a kid until I got my PhD and then largely stopped coding for decades.
The fact that the @v4vapp code has been running for three years now, largely without any updates or changes to the back end for months now, kind of shocks me!
But it is all being replaced and I've learned a huge amount about running a service like this since I wrote the original code. I've learned a lot about how to structure code, how to run services and a whole lot more. I also want to make this whole backend available for others to use and run, not just me.
The New Backend
The new backend is a complete re-write of the original code. It is in Python, using FastAPI and a whole lot of other modern libraries. It is containerised using Docker and orchestrated using Docker Compose.
Thanks to AI and in particular Grok, there is an Admin interface which I've never had before. Up to now I've been hacking around in the database, editing config files and sending obtuse JSON commands to the backend via my own API. Now I have a proper Admin interface which makes it much easier to manage the system.
Accounting
I can't begin to explain how different this is to my original code. It is a proper back end, with proper accounting, proper user management and a whole lot more. I've built an entire double entry book-keeping system because, having built the first one (which does work) I came to the conclusion that the system which was invented by a monk hundreds of years ago is the ONLY way to do accounting properly.
So with the help of Grok I built one.
Lightning
The system still connects to a dedicated Lightning node running LND. This is still not easy to set up and keep working: this will be something others will need coaching with. But I'm now connecting to the node in a much more robust way and the whole system is much more stable.
I have much better protection for the way in which Lightning can sometimes take a long time to send a transaction and I deal better with various forms of "HODL invoices" which caused my previous system issues and led to me losing a fair bit of BTC in one particular malicious attack.
VSC Network
Right now and with V2 when it comes out, @v4vapp is a centralised service. I run the backend and the database and the API and the front end. This is not ideal. The conversions of Hive and HBD to BTC are require me to have an involvement which I don't want going forward.
What @vsc.network are building is a decentralised network which will allow me to programatically exchange Hive and HBD for BTC without my direct involvement. That is the dream.
The way in which this will interface is that every movement within my system which involves Sats, (or Keepsats as I call it internally) will also be recorded on the VSC Network. A smart contract on the VSC Network will record movements of Keepsats, to start with I will still do the conversions, but going forward this will be done by the VSC Network.
I am looking for help: I want to start work on the GoLang smart contract code which will run on the VSC Network. If you are a GoLang coder or you want to learn about VSC Smart Contracts, please get in touch.
Open Source
Everything I've done is open source. The code is on GitHub. Most of my code has been open source but the original back end which has been running for years is not. I was mostly worried that opening the code would make it vulnerable to attack. But having the new code open source from the start is a new way of thinking and obviously I have to be more careful about what I put into the repo.
Architecture
A short word on the architecture. The whole system is containerised using Docker. Each component runs in its own container. The containers are orchestrated using Docker Compose.
Previously I had a FastAPI server as the main back end talking to a MongoDB database and to the LND node.
I now have a new set of docker containers:
- Hive monitor: watches the Hive blockchain for transactions and updates the database
- Node monitor: watches the LND node for transactions and updates the database
- Internal FastAPI server: this is much reduced and just handles calls from the external API
- Admin FastAPI server: this is a new component which handles the admin interface
- DBMonitor: this is a new component which monitors the MongoDB database.
- MongoDB: unlike last time, I'm working with a multi node replica set which is a great deal more robust but a nightmare to set up.
It is the DBMonitor which is the heart of the system. It watches the database for changes and triggers actions such as exchanges or balance updates based on those changes. This is a much more robust way of handling things than my previous approach. This means that Hive transactions or Lightning payments and invoices are picked up, put in the database and then processed by the DBMonitor.
This makes use of the MongoDB Change Streams feature which is a powerful way of reacting to changes in the database. It also means that (along with some careful locking) I can in theory run multiple instances of the DBMonitor to handle a large load. This is something I will be testing in the future.
The old system which remains unchanged for the Frontend looks like this:
- Frontend: Vue app running in a Docker container served via Nginx
- External API: FastAPI server running in a Docker container
Wrap Up
I've been working on the backend v2 for a year (I can barely believe it). I can't even count the number of commits but I'm getting close to it being ready to go live. I'll need to take down the existing system for probably a day or two to carefully migrate but then it should be ready to go.
Thanks for sticking around, and thanks to VSC Network and their proposal which is funding some of my work on this too.
Value for Value
For the last few months while building @v4vapp I was generously supported by the DHF. Going forward I have a much more modest support which covers direct server costs and a little of my time.
If you appreciate the work I do on and around Hive, you can express this directly: upvoting posts on Hive is great. Also consider a direct donation (there's a Tip button on Hive or a Lightning Address) on all my posts.
Support Proposal 342 on PeakD
Support Proposal 342 with Hivesigner
Support Proposal 342 on Ecency
Vote for Brianoflondon's Witness KeyChain or HiveSigner
You have a positive goal to achieve with the amount of time spent on your project which will definitely be worth it when your done with it.
Awesome work, Brian. Admire your dedication and endurance. @v4vapp is a really usefull bridge.
Keep it rolling and all the best for the Migration to the new „System“.
The only bit I might not be able to carry forward is the streaming sats part: it's a lot of work to recreate that and the system is pretty much dying. Very little use. Bit sad about that, I put in a huge amount of work in it but it simply isn't worth doing that again in v2.
Ah okay. Does that mean the people that used it (like me) will have to change something in their setup in Order to receive the Sats now in the normal Lightning Wallet, like it Fountain App?
we always support you because you are a strong witness here because you are very valuable to all of us here
That's is high level of consistency and commitment from you. Wow you have been working on the backend for more than 2 years
Thanks for v4v it's great to have around.