It's about recoil. For every action, there is an equal and opposite reaction.
The more recoil you have, whether from a heavier bullet or a hotter load will cause more rearward force and therefore more muzzle rise while the bullet is exiting the barrel.
Plow handles tend to roll more, a reason some prefer the Bisley style grip. But ... a lot of this has to do with the size of your hand vs. the size of your grip, one handed vs. two handed, how firmly you grip, etc.
Although I consistently read that bisley grips are preferred for heavy recoil, I do not concur. I find the plow handle much more comfortable. But I do have fairly large hands; perhaps that is the difference. And I like to shoot my single actions one handed.
Try varying your loads by 5 grain increments and see where your group prints. I think you will find it interesting.
For me, this lesson was learned early with single shot muzzle loading handguns. There is no way to adjust the site, so changes in load becomes a necessity to change vertical grouping at a given distance.