The visitors took the points with a breathtaking burst at the end after Adam Szalai's second-half opener had been cancelled out by Finland's Mikael Forssell. The win was Hungary's second UEFA EURO 2012 qualifying Group E victory in the space of four days.
Both sides started brightly but it was the home team who threatened first when Alexsei Eremenko shot wide in the 14th minute. A minute later it was Hungarian left-winger Dzsudzsak who had a chance from a free-kick but he could only shoot at the wall of defenders.
Krisztian Vadocz shot narrowly wide of Jussi Jaaskelainen's post from a free-kick. Zoltan Gera was then denied by Jaaskelainen shortly afterwards as the visitors looked to capitalise on their pressure but they were unable to force their way through a stout Finland defence.
Indeed it was the hosts who had the last word before half-time when Mika Vayrynen's 24-yard shot bounced off the top of the crossbar and over with Hungary goalkeeper Gabor Kiraly beaten.
The visitors put that close shave to the back of their minds though and took the lead four minutes into the second half, substitute Vladimir Koman sending Szalai clear for him to slot home from 15 yards out.
Forssell nearly forced an equaliser and the goal did come minutes later when Forssell equalised with five minutes to go, a loose ball coming off a defender and falling kindly for him with only goalkeeper Kiraly to beat. The striker produced a cool low finish to level and seemingly earn a point.
However, PSV Eindhoven midfielder Dzsudzsak had other ideas and from 17 yards he planted a superb shot into the bottom left corner to secure the win for Hungary.