I’ve managed to work on my robots quite a bit without developing or using a simulator (besides Jared’s Mini Cheetah simulator for some of the quadrupeds). However, as I start to see what the robotics world is achieving with reinforcement learning and imitation learning, I recognize the importance of having an accurate simulator.

I opted to use Mujoco because I had some experience trying to use it in the past and because Deep Mind has a few Colab notebooks with examples of them training policies to control legged robots.

My first step was to make the robot model.

CAD-wise, I’m pretty particular with details: modeling every bolt, setting material properties, overriding mass for prints and COTs assemblies, etc, so I started by chunking up my model into the bodies I expected. The only noteworthy change was I oriented the bodies to be aligned such that my CAD’s coordinate frames were already aligned with Mujoco’s, and I placed the origin of each subassembly at the joint location.

From these detailed models, I measured the mass, COM location, and inertia diagonal.

I then created simplified models to use as bodies/meshes. This model was also kept with individual parts having coordinate frames and joint locations taken into consideration. In this case, I did mirror the leg parts, and exported left and right versions of the stls to avoid any mirroring confusion.

I then plugged away at writing the XML, using meshes (exported as coarse STLs) as bodies. When I start trying learning, if I have issues with the mesh collisions being too computationally expensive, I might consider a simplified model. But for now it suffices. I used simulate.exe to validate a lot of the XML as I wrote it.

For actuators I used position servos for the hip and knee joints, and velocity servos for the wheels, which seem to be fairly representative of the real actuators (20khz PD servos).

My first instinct was to try and run the simulation with a 20khz timestep to match the motor controller PD loop frequency but the simulation was quite slow (unsurprising but worth trying). I settled on 2khz for now. Because the onboard motor control loops do track quite well, it seems their dynamics are captured decently with this larger timestep.

The control loop for the simulated robot runs at 500hz just like the real robot.

For visualization and user input I used pygame because I am reasonably familiar with it from high school. I included code for driving the robot with a Logitech F310.

With that, I ported the existing C++ robot controller that was running on hardware to python, and the simulated robot worked! I included stand-up, balance control, gimbal control, height control, and roll control so far.

My goal with this model is more to start working towards a general purpose RL balancing policy so I don’t necessarily plan on implementing my jumping and stepping controllers necessarily yet. Although simulation is quite exciting and fun so maybe…