The visitors fell behind when Aleksandrs Cauna converted a 43rd-minute penalty, but bounced back after the break with strikes from Nikos Spyropoulos in the 57th minute and Theofanis Gekas in the 70th minute.
With the first attempt on goal in the 16th minute, Greece midfielder Kostas Fortounis headed into the arms of goalkeeper Andris Vanins. Kostas Katsouranis tried his luck moments later, but shot wide of Vanins in goal. Gekas, meanwhile, was just inches away from scoring in 35th minute as he fired against the post.
Latvia had hardly threatened, but with the half drawing to an end they snatched the lead, Kyriakos Papadopoulos fouling Aleksejs Visnjakovs in the area and Cauna scoring the resulting spot-kick.
Within seconds Aleksandrs Fertovs had an effort blocked by Spyropoulos before Gekas brought a save out of Vanins.
Greece looked dangerous after the restart, with Giannis Maniatis firing wide before substitute Georgios Samaras shot over the bar. Fortounis then drilled a great effort too high and three minutes later the away side were level thanks Spyropoulos' right-footed finish.
Samaras had another attempt blocked before having to be substituted due to injury and a minute later, Gekas struck past Vanins to make it 2-1.
The visitors pressed for a third, but could not add to their score, with Gekas heading off target, Vanins catching a shot from Katsouranis, Sotiris Ninis drilling too high and Vassilis Torosidis seeing an effort diverted by Oskars Klava.