RE: Scaling 3Speak: Introducing IPFS Hot Nodes & the Traffic Director - Time to Upgrade your Encoder Nodes
You are viewing a single comment's thread:
I have put only 50GB for now... of NVMe storage. But eventually will separate to a few TBs storage node.
0
0
0.000
here in a few days, i'll reach out. Maybe we can setup a little hot node with your server too.
It would be a cool experiment if anything.
Happy for it on the IPFS side of things... it should be something (in my side that is always connected and should have a couple TBs available from SSDs... I also have quite a good capacity on my link, so should be very quick serving things (even if almost high latency for everyone because I am on far side of the world).
Let me know when you can have the time. Currently I have IPFS running on the same VM as the encoder, but the aim is to separate it to the ARM node and then run one or more encoders whenever I can :D
Hey bro, I'm doing a critical update on the Encoder tonight... the legacy Encoder Gateway is really failing us. I'll send you a dm, so you can git pull.
ok, heading home soon anyhow.
we are good to go... when you get a chance, do the git pull and update your env
Done.
Giving it a try... does it need to serve publicly too, and you add my hot node somewhere, or can it be done via IPFS, and I don't need to expose a public domain?
I now have a 1TB configured IPFS and was trying to set up this, but if this needs public HTTPS exposure, I need to think about it.
OK, not public, for now... let's see if this thing will just do the migrations.
Where you able to make it all work? I'm yet to clean up the install scripts - need to revise the readme/guide
Setting up Kubo and the public gateway took me days. To get it fine tuned.
Not really, having a couple of problems to troubleshoot:
Will have to delve through the install scripts to see what I missed :D I have the bad habit of never reading the manual :D
Ok, I think I found why...
I had ENABLE_LOCAL_FALLBACK to true, and it looks like the code expects the hot node on the same machine behind localhost.
Will be looking for one more error (after reverting to no local fallback) - for now...
Which is the video processor... trying to look locally too for the video.
Oh... that can't be it... I'll make a video explainer, hopefully it will make sense then. I know manuals are no fun.
The SuperNode has been having some big issues, but Im going to deploy the nuclear fix. Which is to make the staging server also have a https ipfs gateway and set that to be the main source to download the job, and we fallback to the local ipfs node.
I'll make a video.. I promise this will make sense.
Note (regarding the VideoProcessing one, not the fallback), I have nothing on 127.0.0.1:5001 at the encoder node... because my IPFS part is (now) on a remote IPFS server... on another node (and I haven't set any nginx redirect here).
The video processor tries to download from localhost... here:
https://github.com/Mantequilla-Soft/3speakencoder/blob/8cbb76659b95a2d3a66f002a0e73dc17e0e3e4fd/src/services/VideoProcessor.ts#L964
I have changed this to catch the API node value that people specify on the .env config. And that should revert the search to my remote IPFS node.
But thanks, will be waiting for the video... talking with people is much easier to speed up the understanding of these things, without having to go through the code. I have read the .mds, and that helped a bit, but I haven't yet got through all the things.
Ok, found another thing that might have been in the way...
The password thingy... was failing for me because I had a toooo strong password LOL
I either had special characters or too many characters... but switched to a normal words password and now it's working fine... even could do checking with the super node.
Now just need a successful encoding and see if it pins automatically.
Exciting...
you are ahead of the game hahaha! we have to setup your hotnode in the round robin bro... hahahaha anyways. I'll make videos for this too. The Hotnode is still alpha software, but its safe. You will end up helping me fine tune both installation and bugs.