The RNDR Beta Experience — Parley Labs
Building software is hard, building distributed systems is extremely hard. Having been involved in the RNDR beta since its launch in 2018 has given me the privilege to see how challenging it can be and how amazing the RNDR team has been. Now I’m finally able to share my experience with the rest of you, as there has been some impressive progress:
- A network of distributed untrusted GPU’s securely rendering artists scenes through OTOY’s ORC cloud rendering service
- Watermarking of sample renders
- A Simple client
- Better understanding of network performance
- Fully integrated rendering pipeline with ORC
- Real render work being distributed on the network
- bug fixes throughout the year
- Functioning RNDR escrow and payment
- Improved job management and allocation
I’ll be sharing with you my experience as a beta “miner” and what I’ve learned throughout this beta period. I use the word miner here tongue in cheek, as this is nothing like crypto currency mining. The only similarities are the simple client that you run and the Ethereum wallet where you are paid for rendering work. The similarities end there. There are no mining pools, no random number generator and your GPU’s are only being utilized when rendering work that was requested by an artist.*
* In the future this may be when someone is requesting any 3d asset stored on the RNDR network, not just artists requesting work to be rendered.
You know what’s also really cool? Seeing some of the work that was rendered partially on yours and hundreds of other machines distributed all over the world.
Simplicity
RNDR was built to be simple and hands off; no configuration necessary. The client software is a container that handles encryption and connection to the network, which then downloads the proper rendering software to do its job. When there are no jobs, it sits idling, connected to the network, waiting. When there are jobs, the client will download the scenes, load them into memory and start the rendering process. It’s dead simple and anyone can do it.
The RNDR Client Process
- Download executable from OTOY’s website
- input your wallet ID and run client application
Client Application:
- Connects to the RNDR network
- Idling and waits for jobs
- Downloads scene to system memory
- Renders & periodically send status back to central server
- Reports job status
- Uploads job to storage server
- Checks node availability
- Repeats cycle
System Specifications
You need RAM, bare minimum 16GB, but 32GB is the recommended minimum. You can get away with a lower end CPU if your GPU’s are the minimum required NVIDIA 1070’s, but the larger and more complex the scenes, the better you’ll want your CPU to be.
I built two systems at the start, one of which I’ve upgraded to handle these larger more complex scenes. We’ll call the upgraded system #3 for the sake of comparison.
System Performance
System 1 has been performing well throughout the beta, aside from the normal growing pains of beta releases. My most consistent crashes were due to bad riser cables. Not seen when mining crypto currencies, but a big issue when rendering. Any drop of GPU will cause system reboots or client crashes. It doesn’t matter during crypto mining because the miner software will reset the GPU or ignore it. You can’t do that mid render, the render software will hang or your system will crash due to hardware failure. Failures weren’t always caught when running OctaneBench either, so this is something to watch out for.
Bad USB riser cables will be the number one cause of failures for miners upgrading their systems for the RNDR network.
System 2 performed great during test net (see below), but as more complex jobs were introduced to the network, I experienced more crashes and longer scene loading times (on the order of an hour). This is BAD because you’re not getting paid to download or load the scene, you’re getting paid to RENDER it.
System 3 upgrade was necessary because it was being allocated larger jobs. Scene loading times on the order of seconds and no more crashes. Why the difference? The Celeron is OLD and slow and the single 1080ti on a USB 1x riser was a huge bottle neck. Upgrading the CPU and directly connecting all 3 1080Ti’s on the motherboard removed all bottlenecks. You can read more about the impacts of CPU cores and speed over at Puget Systems as they’ve done lots of testing.
Job Allocation
The RNDR network allocates jobs according to the performance and capacity of your system.
Miner nodes are tiered based on factors like Total System OB, Node History (Successful % of jobs processed), Minimums for VRAM and RAM which construct a Composite node Score. Users selecting a specific tier of work are matched to nodes from the selected tier on RNDR.
So System 1 would always get smaller render jobs while System 2 and 3 would get larger more complex jobs. This is why issues with System 2/3 were never seen with System 1.
Testing
First thing first is to run OctaneBench. This is a great way to do a quick sanity check to make sure your GPU’s are stable and haven’t been overclocked too much. This will also give you a general idea of how your system scores compared to other systems. Once everything checks out, the systems would be tested against the RNDR network.
Test-Net (Phase III A-C)
Phases A-C lasted almost a year with continuous sample jobs being sent out. Systems would crash, tweaks were made, lots of questions were asked and bug reports sent in. The team crushed through the bug reports by the end of test-net, closing 162 of 174 bugs submitted.
Working with the developers and all of the other beta testers has been a really great learning experience. I’ve learned so much in this space from everyone with their breath of experience as render artists, network experts and large scale crypto miners.
Main-Net (Phase III D and Beyond)
Main-net launch was very promising, but us beta testers may have been expecting too much too soon. We expected lots of artists to be on-boarded and system utilization to be high. But there was neither, and for good reason too.
As all things beta goes, new edge cases and system improvements were identified. Going from test-net to main-net meant adding in real world renders, which introduced a lot of unknowns and helped the team discover issues outside of their sandbox. With each update, we would see a more stable network and a growing number of jobs.
We’re in Phase III-D right now and it seems like we’re going to be on-boarding more people soon. When? I have no idea, they never tell us these things, but I’m seeing a lot more from their social media accounts on requesting beta access.
What’s Next?
My thoughts, assumptions and predictions…
Because of their Tiered system, they won’t get rid of their AWS solution, but supplementing it. You can now use the existing ORC system at the existing price ($1=100 OB4/Hr), or scale up. They’ve made it appealing for artists on a budget or those with pressing deadlines. Need lots of GPU’s but no real time crunch? Get 8–16x the compute for the price of their AWS solution. Need something with higher security and high priority? Pay the existing AWS price at $1 Per 100 OctaneBench4/Hr.
What they’ve essentially done is scale out their network beyond what was available, allowing them to service their customers at all different price points. They’re tapping a market that didn’t have the budget, but still the need, and connecting them to GPU’s that were doing nothing.
What remains to be seen is the method in which artists will be able to purchase their RNDR tokens. Will they have to learn how to use crypto currency exchanges to participate or will there be a more user friendly method?
Originally published at https://parleylabs.com on February 5, 2020.
