I work with those steels all the time. Here's my suggestion.
Pre-heat both the bushing and plate to no more than 400 F, then make a small stringer pass, let the piece cool back down to 400 F before making your next pass. If you're making a fillet weld larger than 1/4" then the distortion will be even more noticeable.
This will help minimize the distortion, but will not eliminate it.
The reason the plate bows upward is because the weld metal shrinks as it cools and pulls the plate toward the side that the weld is on.