The Hoops had scored seven on their last visit to Tynecastle in December but the runaway league leaders failed to register a shot on target in a tight first half.
Bottom-of-the-table Hearts were much improved this time round and were hopeful of halting Celtic's run of 15 straight victories before Griffiths pounced in the 58th minute to open his scoring account for the Hoops.
Substitute Derk Boerrigter hit the bar in the 85th minute before there was a scare for Forster in the second minute of stoppage time when he failed to deal with Callum Paterson's long throw-in and Efe Ambrose had to head clear off the line.
However, the Hoops sealed the win a minute later when substitute Teemu Pukki scored a rare goal as the Glasgow giants remain unbeaten after 26 matches and move 24 points clear at the top of the table before nearest challengers Aberdeen's match with Partick Thistle.
Forster's 13th consecutive Scottish Premiership match in a row without conceding a goal saw him surpass Bobby Clark's Scottish record set with Aberdeen in 1971.
The English keeper has now gone 1,215 minutes without being beaten as he closes in on former Manchester United goalkeeper Edwin van der Sar's British record of 1,311 minutes.
Celtic manager Neil Lennon hailed his keeper's achievement. "I am very proud as we have broken a 43-year-old record today. It is a fantastic achievement and it was 43 years for a reason because it is a very difficult thing to do," Lennon said.
"I felt we were in control of the game but never really worked the goalkeeper enough in the first half and it took a special goal from Leigh to get us off the mark."
Hearts manager Gary Locke looked for the positives in defeat. "The last time Celtic played us here it was a no contest and I think you can see how far our young players have come," Locke said.
"For long spells of the game we matched the best team in the country so there is a lot to be positive about."
Forster's touch of nerves
Hearts were undefeated in four games coming into the match and made a decent start at Tynecastle.
Forster knew he only had to last 31 minutes without conceding a goal but he showed a touch of nerves in the 20th minute when he dropped a cross in the box, only to recover quickly to pounce on the loose ball.
Hearts captain Danny Wilson had to make an important challenge to block a shot from Griffiths on the edge of the box before Scott Brown fired the rebound well over the bar.
Celtic came close to opening the scoring in the 54th minute when Commons whipped in a free-kick to the back post where Stokes bulleted a header towards goal but 'keeper Jamie MacDonald showed great reactions to parry it away.
Former Hibernian striker Griffiths had been taking abuse from the Hearts fans all afternoon but he silenced the home support when he grabbed the opener in the 59th minute.
Stokes launched a ball forward for Griffiths who turned Hearts defender Wilson one way then the other before unleashing a fierce left-foot strike past MacDonald from just inside the box.
Substiute Boerrigter fired an Adam Matthews cross off the crossbar before Hearts came close to ending Forster's long run of shut-outs when Paterson's throw-in wasn't dealt with by the 'keeper, forcing Ambrose to nod the ball off the line in stoppage time.
Hearts were left to rue that missed opportunity when substitute Pukki sealed the win in the third minute of stoppage time. Fellow sub Amido Balde used his strength to outmuscle Jamie Hamill on the sideline before squaring the ball to Finnish striker Pukki who applied the finish.