Gorkss met a Aleksandrs Cauna cross after 59 minutes to decide the game and seriously dent Israel's hopes of reaching next summer's finals in South Africa. Latvia by contrast moved level with Greece on 13 points after the Mediterranean nation lost 2-0 at leaders Switzerland.
Latvia host the Swiss in Riga on Wednesday and with just three points separating them know that victory could send them to the top of the group. Israel had started the stronger side with Elyaniv Barda bursting into the area and crossing to the back post where Latvia goalkeeper Andris Vanins did well to snuff out the danger.
A minute later Vanias denied Omar Golan, before a dangerous Cauna free-kick at the other end forced Dudu Aouate to leave his line and clear. Israel continued to push for an opener with defender Yuval Shpungin heading over from a corner in the 26th minute.
Veteran Latvia striker Maris Verpakovskis then should have opened the scoring when he was one-on-one with Aouate, but sent his effort wide. Golan came the closest in the opening period when he struck the post with an athletic overhead kick after Yaniv Katan's clever pass.
Israel continued to probe away after the half, but went behind just before the hour mark. Cauna's corner from the left found the Gorkss and the Queens Park Rangers defender headed the ball into the net.
Five minutes later Latvia captain Vitalijs Astafjevs, who was making his 155th international appearance, hit the crossbar with powerful shot from distance. Andrejs Rubins could have double Latvia lead in 72nd minute, but missed the target after a pass from Verpakovskis.