Chasetown led 2-0 after a kick off delay due to an issue with one floodlight.
Jazz Luckie powered home a header and then Tom Hill added a second with a neat finish over the advancing keeper.
Rushall fought back in the second half though and netted twice in a short period, despite Chasetown claims that Curtis Pond had kept the ball from crossing the line for the Pics equaliser.
The game went to a shoot out which Rushall won 5-4 after 6 penalties aside.
Curtis Pond scored one of Chasetown's shoot out kicks, the first time a Chase keeper has taken a penalty in a shoot out.
Only authorised club members can view team selection.