Australia quickly got into their stride, and Kofi Danning was just inches away from giving them the lead on three minutes. The No7 received possession to the right of the Costa Rica penalty box, and with disregard for the tight angle at which he was positioned, released a fearsome strike that almost made the top corner.
On 11 minutes, Dimitrios Petratos executed a perfectly-weighted through-ball round the ball of the Ticos defence. Kerem Bulut raced on to it and tried to put the ball across Mauricio Vargas and into the bottom corner, but the goalkeeper outstretched his leg to make a brilliant stop.
Bulut went close again on 18 minutes, charging beyond the Costa Rica rearguard and firing a powerful shot from just inside the box narrowly over.
However, against the run of play, Costa Rica went in front. Joel Campbell received possession inside the area, faked a shot and cut inside his marker, before slotting the ball past Mark Birighitti.
Australia equalised on 26 minutes. A long throw near the corner flag was flicked on by Bulut, and Tommy Oar's outstretched leg got to the ball first to flick it past Vargas.
Within seconds, Costa Rica had regained the lead. Australia failed to deal with a cross from the right and Campbell masterfully half-volleyed the ball home.
A superb 39th-minute Kantarovski long ball picked out Bulut, but the No9 was just offside when he would have been clean through.
The second half started at a slow pace, but Australia almost grabbed a leveller on 53 minutes. Terry Antonis crossed from the right and, again, Oar stole in front of his marker and flicked the ball goalwards, only to see it hit the post.
Joel Campbell then thrilled the crowd with one exquisite drag-back, which conned two Australians and prompted one of them to foul him.
But it was the Aussies who were raising the decibel level three minutes later. The board had just gone up. The No10 was on it. It was time for cult hero Mustafa Amini's entrance. The Central Coast Mariners man made an immediate impact, taking opponents on and playing some neat passes, with his every touch cheered loudly.
The Young Socceroos drew level on 64 minutes. Francisco Calvo tried to clear a right-wing cross, but instead put the ball into his own net.
Australia were hungry. They were in control. If anyone was going to score the game's fifth goal, it was Jan Versleijen's team.
However, on 72 minutes, Costa Rica won a corner, and from Joel Campbell's centre John Ruiz glided home his second goal on this FIFA U-20 World Cup.
As the match entered its last ten minutes, Campbell sprinted forward, took on his man and was brought down. The referee played the advantage, with Ruiz through on goal, and the 17-year-old's firm, low strike forced Birighitti into a superb diving save.
On 89 minutes Campbell was substituted to a round of applause from the Costa Ricans and the neutrals. A few minutes later his team-mates enjoyed the sound of the full time whistle, having moved on to four points to leave themselves in a strong position to advance.
Australia will stay in Manizales to face Spain on Saturday, while Costa Rica meet Ecuador in Pereira.