The visitors netted twice in two first half minutes to hold a 2-0 lead at the break. Rory Hill thumped home on the rebound on 16 minutes, after a great block by Nic D'Arenzio, to give Cray Valley the lead.
And the Millers netted their second soon after. Denzel Gayle's cross found Marcel Barrington who turned his man and fired home off the post.
Hill pulled a goal back on the hour. Lewis Finney stroked the ball to Lewis Taylor who fired into the bottom corner.
Four minutes later the visitors were reduced to ten men. Hassan Ibrahiym was shown a second yellow for a foul on Hillians keeper Taylor Seymour.
Hill were level in second half stoppage time. Finney found Max Miller on the edge of the box and the striker powered the ball home.
And the Hillians completed a remarkable turnaround moments later. Taylor's dinked cross was met by Finney who looped a header into the back of the net to spark pandemonium at the More Than Tyres Stadium.
Photographer Chris Neal was there to capture a thrilling afternoon in Mid Sussex.