Iceland shocked Les Bleus by taking a two-goal lead through Birkir Bjarnason and Kolbeinn Sigthorsson. However, Laurent Blanc's side reduced the arrears through Mathieu Debuchy seven minutes after the break, but they had to wait until the 85th minute to equalise through Franck Ribery before Rami's late strike won it.
France had the first shot on goal, Yohan Cabaye's curling attempt from the left blocked by midfielder Kari Arnason and from the resulting corner the Newcastle United midfielder drilled the ball wide.
Sigthorsson's attempt from the right was blocked by Philippe Mexes, while Aron Gunnarsson repelled Karim Benzema's effort for France with neither side creating clear-cut chances. However, it was Iceland who took a surprise lead in the 27th minute when Bjarnason volleyed home a right-footed shot into the left-hand corner of the goal from Sigthorsson's header.
Samir Nasri shot wide soon after from distance as France looked to respond. Following another attack moments later, Nasri sent in an inswinging corner which found Yoann Gourcuff, but his left-footed shot from the centre of the area was blocked by Arnason.
Rami then drilled in a 35-yard shot from the left which was parried by Iceland goalkeeper Hannes Thor Halldorsson. France were shocked further by Iceland who got a second goal in the 34th minute when Sigthorsson converted Rurik Gislason's wicked cross from the right.
Benzema volleyed over the bar and then shortly before the break was thwarted by Halldorsson, the keeper blocking the Real Madrid forward's drilled shot from the right of the area.
France attacked from the off at the start of the second half with Benzema's shot deflecting off a defender, but Gourcuff just failed to capitalise. The hosts were rewarded for their persistence when Benzema's shot was deflected into the path of Debuchy and the defender slotted past the keeper.
Iceland almost restored their two-goal lead with a flowing move which ended with a stretching Bjarnason hitting the ball straight at keeper Steve Mandanda after he lost his marker. At the other, Gourcuff curled in a free-kick from the left which almost went over the keeper, but he managed to keep hold of the ball at full stretch.
Mathieu Valbuena, Rami and Mexes all had efforts as France pressed and their persistence was rewarded five minutes from time when substitute Ribery played a slick one-two in the box with Olivier Giroud before lobbing the keeper. Just two minutes later and France scored what proved to be the winner, Rami volleying home from Giroud's knockdown.