A brilliant brace from James Ashmore saw Frickley turn the tables at North Ferriby to give them a deserved first away success of the season.
Manager Peter Rinkcavage rang the changes, with Michael Simpkins suspended, Kelvin Bleau and David Cyrus partnering Lincoln Adams and Stuart Ludlam in the back four, while new signing Peter Knox, who has been at Bridgewater Town, led the attack.
United threatened in the eighth minute when Wayne Brooksby picked up a stay pass from Ashmore and hammered a 30 yard drive just over the bar, but the Blues responded, Ryan Mallon getting downn the left and finding Knox at the far post, only for his shot on the turn to flash the wrong side of the upright.
Another accurate cross from Mallon in the 20th minute was controlled well by Knox on the edge of the penalty area and his shot was held by home keeper Steve Wilsdon diving to his right.
Adam Billard did well to palm away a stinging 20 yard shot from Russell Fry in the 26th minute, and showed anticipation and skill in tipping a 30 yard effort by Paul Harsley over the top moments later.
With the wind in their favour after the interval, the Blues thought they had taken the lead in the 53rd minute when Lincoln Adams challenged Wilson and the ball went into the net off Harsley, buit it was disallowed for a foul.
On the hour, Bleau put a ball over the top of the United defence for Knox to collect, holding off his marker before drilling a shot a whisker wide of the left hand post.
Billard saved a close range header from Phil Jackman three minutes later, and recovered quickly to smother the rebound from Gary Bradshaw.
Ferriby grabbed the lead in the 70th minute when Kevin Larvin raced away down the left and whipped in a cross for Sam Mulready to head home just inside the far post.
Frickley responded, and were rewarded eight minutes later when a corner by Leon Henry was only half cleared and Ashmore lashed a 25 yard rocket into the top left hand corner.
Philip Jackman fired into the side netting for United in the 80th minute, before play switched to the other end, Henry crossing from the left for Ashmore to collect, only for his shot to be blocked on the line.
Ashmore made no mistake in the 86th minute, curling a 25 yard free-kick in off the left hand post to put the Blues in front.
A 25 yard free-kick by Bradshaw shaved the bar two minutes later, but Frickley were determined not to surrender any spoils and kept the home attack in check for an agonising six minutes of stoppage time.
North Ferriby: Steve Wilson, James Williams, Kevin Larvin, Mark Greaves (Sam Belcher 61min), Nathan Peat, Paul Harsley, Russell Fry, Sam Mulready, Philip Jackman, Gary Bradshaw, Wayne Brooksby; unused subs, John Leonard. Mathew Stirland, Luke Thompson, Ben White.
Frickley: Adam Billard, Kelvin Bleau James Kay 85min), David Cyrus, Danny Walsh, Lincoln Adams, Stuart Ludlam, Alex Callery (Ashley Longstaff 75min), James Ashmore, Peter Knox (Ben Leonard 90min), Ryan Mallon, Leon Henry; unused subs, Bradley Grayson, Karl Jones.
Referee: Mr Steve Meredith