Elyaniv Barda gave Israel the lead in a first half they dominated, and Latvia were fortunate not to be out of sight by the time the half-time whistle went. But they hung on and Kaspars Gorkss looked to have given them a point with an equaliser just after the hour until Kayal struck with nine minutes left.
Israel continued to press for an opening and went close again seven minutes later, with Bebars Natcho having a shot tipped around the post by Vanins. That was not the end of the defending for Latvia either, with Yoav Ziv whipping his corner just wide of the post.
It was all one-way traffic and it was of little surprise when Israel made their pressure tell after 16 minutes. Barda was the scorer, pouncing on a loose ball inside the six-yard box to give Vanins no chance.
Buoyed by his goal, Barda looked for a second and dragged a shot wide from just inside the area, before Kayal had a close-range effort blocked by Gorkss. Latvia were fortunate not to be further behind, and Rafaelov gave them another lifeline when he blazed over from six yards out, before Ivans Lukjanovs put Latvia's first chance wide.
The visitors tried to force the pace more after the interval and Maris Verpakovskis drilled wide from 25-yards, but it was Israel who continued to fashion the better chances, with Rafaelov firing just over the bar from inside the box.
There had been little to suggest Latvia were capable of getting back on level terms, but then, in the 62nd minute, a Barda foul on Andrejs Pereplotkins gave them a free-kick and Gorkss was on hand to turn Jurijs Zigajevs' free-kick home.
The goal seemed to give Latvia some belief and Artjoms Rudnevs nearly scored following a goalmouth scramble, while at the other end, Ben Sahar drilled wide. Lukjanovs then took aim from distance and Kayal squandered a golden chance as the game opened up, with Ritus Krjauklis then planting a free header wide for Latvia.
Maor Bar Buzaglo went close as Israel hit back and with nine minutes left, his cross was powered in by Kayal to restore their lead. Sahar had a chance to make it three late on, but there would be no more scoring.