I hate those fiber optic sights too. My older 101 357 Mag I've had for about 20years has standard sights and there just right. Those fiber optic ones Suck IMHO.
Mine with a 6" barrel shoots great 4th and everyone else I know that has one shoots well with them too. Were you having the twitch's when you were shooting it or what???
Maybe a bit of group tightener would help,,,I prescribe 3 fingers of Jack ,,,that should tighten up those groups some mister !!!
I must admit mine did take a bit of breaking in before I got really good accuracy out of it, but once I did and found the load the gun really liked,,wow, it's a keeper.
The 357fed mag is a new one on me too,I hadn't happened to see that caliber before either. Ya learn something new every day!