[R] Best practices for implementing and benchmarking a custom PyTorch RL algorithm?

Hey, I'm working on a reinforcement learning algorithm. The theory is complete, and now I want to test it on some Gym benchmarks and compare it against a few other known algorithms. To that end, I have a few questions: Is there a good resource for learning how to build custom PyTorch algorithms? How optimized or clean does my code need to be? Should I spend time cleaning things up, creating proper directory structures, etc? Is there a known target environment or standard? Do I need to dockerize my code? I'll likely be writing it on a Mac system.