Privex Hive Node-in-a-Box Experience
Hextech witness update
Alright so it's been a week since I rented this Hive virtual server for the @hextech node and I thought I should give an update. First and foremost it was not the best idea to get started on something like this on the first day of head-cold symptoms. I was thinking, "Meh what's the worst that could happen?" But then I proceeded to completely mess up the instance and nothing worked and I basically made zero progress over the course of multiple days and scatterbrained effort. Classic Linux and/or coding experience to be honest. I did learn a couple things along the way so I thought I'd share them here.
The first problem with Hive development is always the documentation.
If I'm being honest I've always struggled with reading documentation even when that documentation is arguably done well. In the case of Hive the documentation is never well done. At least not in my experience. Information is scattered everywhere. Troubleshooting techniques are non-existent. If something doesn't work you basically have to ask someone who knows what they are doing or be competent enough to figure it out yourself (which is almost always going to be a suboptimal path no matter how good you are).
Luckily Hive seems to have a pretty helpful community when it comes to the technical side. Rishi has been helping me, which is super weird because his tutorial on how do this stuff literally cites the @hextech witness as the default example. I also have direct access to top 20 witnesses who are willing to answer questions. GTG has been pretty helpful, and themarkymark adds a little insight as well from time to time. I haven't asked yabapmatt any questions but I bet he could help me with something important if I pressed him on it, although he's a pretty busy dude. I also have a potential in with deathwing and ausbitbank. All in all pretty good resources to have.
So what happened?
Ah well so I sent Privex $50 worth of Hive after placing my order.
Then I was sent an email on how to access the node remotely (SSH).
Pretty painless experience.
Then I log in.
Simple.
This is what I assume I would have seen on my first login if I had bothered to read such things. However before even purchasing the node I was directed to a few different tutorials that all pretty much have the same themes but are also notably different in certain aspects.
https://peakd.com/hive-139531/@rishi556/how-to-set-up-a-hive-witness-using-hiab-at-version-1-24-4
Rishi's tutorial
https://peakd.com/hive-121566/@stayoutoftherz/wie-man-einen-hive-witness-node-aufsetzt-ein-tutorial-fur-computerlaien
Stayoutoftherz tutorial
https://peakd.com/hive-139531/@someguy123/how-to-set-up-a-hive-witness-or-seed-node-non-mira-upgrade-from-steem-docker-to-hive-docker
Someguy123's tutorial
This is the main theme of all 3 tutorials:
- Basically get the OS ready by updating everything and installing GIT.
- Install someguy123's docker for modularity/dependencies.
- Use ./run.sh commands to communicate with the node.
Ultimately my problem was that my ADHD brain combined with being sick at the time prevented me from doing simple things like reading what the console was telling me. Aint nobody got time for that, amirite? Yeah so I just started typing in commands from the tutorial and hoped for the best.
Didn't turn out so great for me in the end.
Essentially I think the first six commands didn't really do anything. They took no time and the console messages I was getting were basically something along the lines of "nothing has changed". The current Hive-in-a-box (HIAB) installations seem to take care of a lot of this stuff for you, as I found out much later in this journey.
We're going to need a montage.
Even Rocky had a montage.
Yep so basically after days of getting nowhere with scattered questions across Privex discord, @l337m45732 DMs, and this openhive chatroom I didn't even know existed until now... I finally just decided I should simply nuke the node and start over.
Luckily Privex gives you access to an account on their website now.
I'm getting vibes that this might be a newish feature.
But as you can see Reinstall is right there, and it worked great.
Kaboom.
In retrospect I'm fairly certain I made two really big mistakes.
The first one was issuing a ./run.sh install command even though all the things come pre-installed on the instance. I'm fairly certain I was issued some kind of warning in the console saying it would give me 30 seconds to change my mind, but I was distracted and by the time I actually read that warning it had already executed. Whoops!
The second mistake I made was skipping past this little gem on login.
I have very limited experience with Linux and am pretty accustomed to using sudo commands which is short for "superuser do". However when you become root by just typing su in the console it teleports you into a seemingly completely different environment.
So I actually started the entire tutorial process over again from here. Ran all the commands all over again. Did another ./run.sh install >> got another potentially fatal error >> ignored it. Basically nothing worked.
I could not open the CLI wallet to create a witness signing key.
Turns out I'm pretty sure I messed around with this part of the journey way way way longer than I should have. I wasn't actually sure if the witness signing key needed to be generated in a different way than normal keys. After all there are a handful of different frontends that can all generate a private key for you. Technically it is smarter to create this key within the node itself so that it is far less exposed to the outside world, but considering how many problems I was having just getting basic stuff done it really wants a priority.
There's basically a very important config file that you open and plug in the name of your witness account and the associated private key. Later when you go to actually produce and sign blocks this needs to be setup properly, but for people like me who have absolutely no idea wtf is going on I'd have to say it's a much higher priority to simply get the node syncing/replaying because that part takes potentially days to get done.
Force open at your own risk.
Probably database created by a different compiler
Eek
After troubleshooting various other problems (like putting a private key in quotations making it a string) This was essentially the error I got that convinced me to just nuke the entire thing. Not that it mattered at all as I had literally made zero real progress at this point.
After nuking the node and starting over.
I was feeling better, a lot less ill, and I was actually willing to read the login screen.
I decided that the obvious priority was to get the node syncing.
So I typed:
su
./run.sh start
And that actually worked.
I was honestly kind of shocked and annoyed; not only at myself but also the tutorials. If following a tutorial line by line leads to me irrevocably corrupting the install... is that really my fault? I mean I feel like not but what do I know? I was a bit delusional when this all started.
And that's where I am right now.
Just sitting here waiting for the node to sync.
At this rate the node is syncing up 10k blocks every 25 minutes or something, and I calculated with some napkin math that it would take something like 11 days to catch up to the head block. Which is discouraging to say the least especially because this node was pre-seeded and already contained something like 73M blocks. So with the resources I've been allocated it takes the better part of 2 weeks to simply process 7M blocks which is less than 10% of the entire chain.
The reason for this is because the Hive-in-a-box setups share a bunch of resources with each other. Definitely not an ideal scenario, and I've already been told by both Rishi and GTG personally that the option I've chosen is wholly inappropriate considering my rank on the witness list. Essentially my node isn't going to be reliable enough and will probably miss too many blocks so I'll likely have to start all over again on a better machine. Still I am somewhat interested in getting this setup actually working and sign a few blocks before I lay it back to rest.
Other tidbits of note.
As my node syncs I'm seeing that gtg and ausbitbank are running code that allows users to post data to chain even if the account in question doesn't necessarily have enough resource credits. I find this to be of particular interest because this one time I was claiming multiple account tokens using peakd's frontend feature I was allowed to create one more account than I should have and my RC balance went negative. I always wondered if that had been some kind of bug, but it seems as though this is completely intentional behavior. It's an interesting display of how RCs actually work under less rigid architecture (softfork adjacent). As long as other witnesses do not rollback the chain these operations are totally legal.
I'm also very annoyed at myself because I'm paying $50 a month for this virtual Hive-in-a-box setup when this dedicated server from Privex is only $70 per month. Something like this would be a huge upgrade from where I am now, and after being told that the HIAB setup is more for newbies to learn the ropes and not appropriate for the level I find myself... ah well that's kind of annoying.
I've also been hearing on the grapevine that tech is going to be extremely cheap during Cyber Monday into Christmas due to overproduction and overstock that stemmed from the economy looking like it was going to bounce back a while ago. I checked myself the other day and saw 1 TB NVMe drives for insane prices like $50, which is pretty crazy.
At this point I'm pretty tempted to build my own rig and tinker around with it. I have a keen interest these days in Hive nodes, BTC nodes, and potentially even CPU mining Monero. However again I'm told that running a Hive node from home is inappropriate for potential security and internet connectivity issue. In fact just the other day my Internet crapped out and I was like, "Oh yeah this is why you can't run an important server from home."
Next steps:
From what I can tell after the node syncs up to the current block it's ready to create blocks and sign them to the chain. In order for that to happen the config file needs to know which account it is running and a private key for signing blocks. The first time I opened the config file I had a hard time seeing them because they are commented out with a '#'.
Of course this is clearly documented in all the tutorials so chalk it up to brain fog.
Then in order for the witness node to actually turn on the public key to this private key has to be broadcasted to Hive. This transaction requires active key authority of the account.
https://hive.ausbit.dev/witness
It appears this can be done on multiple frontends, including peakd inside the witness tab.
Essentially the 'STM111111111111...' signing key signals to the network that the node is disabled and should not be considered for block production. I basically need to broadcast the pubkey there to signal I'm ready to turn it back on. Although I have already attempted this a couple times and Hive Keychain throws an assertion error and refuses to cooperate.
Guess I'll have to figure that out along with what the account_subsidy_budget and account_subsidy_decay are. The only subsidy I can think of for Hive accounts are resource credits for accounts that have very low or zero HP, so maybe that's what it is.
The price feed is very important as well and is a completely separate topic.
https://github.com/someguy123/hivefeed-js
I was told this one works pretty well but I'll have to look into it.
There are also other issues to consider like backup nodes,
but for now I'm just trying to get my witness back online and signing blocks.
Conclusion
Well this has been quite the adventure. I've learned a bit about running a witness node on Hive, and it doesn't seem too difficult. The problem is that hitting little snags along the way can end up being a huge setback, especially in the beginning when everything looks like a foreign language.
Still I plan on grinding on. I definitely like the idea of owning my own hardware and not centralizing my presence to Privex (even though that would be easier). However I also need to think about just biting the bullet and booting up one of these dedicated servers until I truly know what I'm doing. No matter what happens, it's been nice to jump back into the technical side of things and it's exciting to know I have enough momentum to keep it going. Cheers!
I always have had the idea to once try some more CLI based stuff on hive. Maybe even attempt setting up a node. This post give some good insight in the hoops one has to jump. Always a fun challenge :) Thanks for sharing, I'm tempted
why privex? just to be able to pay in crypto?
have you checked contabo? I'm running my witness with a vps there.
Thanks for the tip.
Why did you choose Contabo as a provider?
I already knew they have a good service, and for the vps I was looking for, they had the best price. If you are looking for dedicated, I think you will get a better price at https://www.leaseweb.com/.
I would look at reviews for Contabo, they are well considered one of the worst providers.
tbh I haven't looked at their reviews, but I have used them a few times and never had any problems, worked well for me.
I looked at them recently for a project, and there are a lot of bad reviews for them on Reddit. Likely due to the fact they are a super cheap provider. I've never used them myself because of this.
I think that vps is more luck, no matter the provider. All providers try to split servers into a number of accounts based on an average. You just got to hope that your account falls into a server that has a usage bellow average. Got to have good neighbors....
They do, and cheap providers oversell, meaning more customers per physical hardware. They have to, to make any money.
True!
But they are not that cheap too.... it's hard to compare providers because they are all offering different configurations. All of them have a cheap package.
In my case, I did look for the cheapest that had what I needed. But if I had a different need, maybe Contabo could not be the best choice (price).
I've had my dedicated servers for many years with iweb (now recently bought by leaseweb), I did choose them by the price (cheap), and for many years I've had no problems at all!! I'm super happy!
If I hire a service and I don't like it, I just cancel... they are cheap and I don't loose much by giving it a try. If it works out, great! I got a good deal.
I had also no problems so far.
I haven't used them personally, but when I did a search recently on Reddit and read the "Is contabo any good" and "should I use contabo?" type posts and it was mostly negative. A few "I've been using them and never had a problem" but the majority were negative due to them being one of the cheapest solutions out there.
That being said, most providers suck unless you are paying $1000/m for a $50/m server, like Rackspace where they bend over backwards to keep you.
Do you trust those Reddit reviews? Maybe the competition has something against them, as they might ruin the price level?
Not sure, but there was enough it wasn't a fluke. Generally Reddit is spot on as a collection.
I use Contabo too, good price/value ratio.
quite the journey I see, ser
I have been considering spinning up a Witness node as well. This post was rather insightful. Thanks!
Bookmarked this for later. Sounds like something I would do, often I’ve seen a wall of text and glossed over skipping an important part and jacked up something I was trying to run or query lol. Fun times.
What’s the witness account? Is it under Edicted or another name? I’m gonna go look to see but in case I forget or fall asleep.
@hextech
Oh cool I already support that one or at least I did. I changed my votes when the questionability of 20% HBD interest was the rage lol but I knew that. I thought you were setting up a second one besides that witness you already had.
That's not your actual private key in the image under 'Next Steps' is it?
Looks like good fun, especially while hallucinating from high fever. There's nothing quite like doing stuff on computers while hallucinating. I learned that from being dosed with PCP while repairing a clients computer back in the day. After the sky turned into Pixar plasticene I told my client it would be unwise for me to continue and declined another hit on the air bong.
Good times.
Thanks!
🤣 HAHA!
That is nightmarishly low number. On my development machine (while I was also working on it), the last full sync I've run (middle of February 2023, so up to block 72'230'000) was taking less than 1.9 ms per block. Newer blocks are even faster (they are either smaller than average or more packed with light operations like custom jsons).
There were (way) more of such messages, but I've silenced them all up to HF26 (and I might move that even further once we are close to HF28), since they were just spam at this point. Those are results of turbulent times when RC was just introduced, old bugs, disagreements between witnesses running different minor versions of the code, or sometimes deliberate breaches of "soft consensus". The ones you've quoted are from the fact that your node is older than what @gtg and others are running. Last change removed the ability of account to go negative as a preparation for RC to become consensus. Since the change was retroactive (as all RC changes - no point in doing it differently yet), node of gtg thinks the account in question has more RC, because it never run into negative, while the node you are running still allowed it to dip into up to two hours worth of regen below line. Once your node is updated and replayed, the last "Accepting" message you will see will be from block 72'779'954. All those messages were diagnosed by the way.
Of all five RC pools witnesses are in control of subsidized account token pool - the one that contains "free account tokens" to be claimed for mountain of excess RC. The values you have are default ones and they were never changed. It is best to leave them as is :o)
You are thinking about
share_type rc_adjustment; ///< compensation for account creation fee in form of extra RC. All accounts have that, not just those with low HP (although it is only relatively significant for them).Do you happen to know of an efficient way to run a node solely for transaction checking, without the need for signing or generating new blocks?
I vaguely recall hearing about a "light node."
Currently I use for testing purposes public nodes. (https://beacon.peakd.com/)
However, I was thinking it might be beneficial to run my own node to avoid consuming unnecessary resources from public nodes. This way, I could directly make API calls to my own node and experiment a little.
As for my current approach, it would be like to what edicted described in the post, but with a slight difference. I wouldn't configure the config file. I'd leave the witness and the public/private key lines commented out.
Running empty consensus node is very easy - once you have the binaries just
./hived -d .and it will start syncing, even withoutconfig.ini. You could actually start the node like that for a second to build default config, stop the node, add or remove plugins from the configuration, change paths to more convenient blockchain storage location, redirect logging and start it again from zero. The node will work by itself. This is enough for checking consensus and for using the node for communication withcli_wallet. If you have trusted source of snapshot or block log, you can greatly speed up process of node reaching live mode.The problem is in the details. You've mentioned that you want to use APIs. Depending on which APIs you need, you might actually need not just the node but also Hivemind, and since it is now a HAF application, you might need HAF as well. Some APIs are still available from the node itself (at least partially) if you enable proper plugins. I have personally never set up and run full HAF node locally, so I can't help you with that.
wow! thanks alot, this information is so valuable for me.
I find this information to be incredibly valuable. I agree that locating information for HIVE can be quite a challenge. More often than not, GitHub is the primary resource I turn to, and it's followed by what can feel like endless scrolling and searching through various Hive Discords (and there are many of them, which can be a bit overwhelming, to say the least!).
I've always wondered about ways to imporve the information for Hive. But, as you mentioned, who has the time for proper documentation? Right? :)
Additionally, I'd like to point out that it can be more challenging for people who can't access witnesses directly or are not well connected. However, in my humble experience, the witnesses are sometimes approachable and genuinely helpful.
https://peakd.com/hive-167922/@mangowambo/re-themarkymark-s3iqz8 (Here are some insights from themarkymark regarding my question about how witnesses establish consensus)
I just recently tried to edit and upload a smart contract for NFTs; first time I have been on the dev side and I only managed because someone was kind enough to help me. Really steep learning cruve, but I assume it gets better? Seems like on Hive we really are fortunate to have many that are willing to help.
@tipu curate
Upvoted 👌 (Mana: 56/66) Liquid rewards.
I'm sure you know so many thing when it comes to running a witness node on Hive
Also, it is my first time hearing about privex
Any update? Your node is now already quite a while offline.