Most of the optimization work focused on batching. Run nvidia-smi during your job and see it it’s actually using 100% of the GPU