Stewart spot-kick stuns Celtic
Tony Mowbray's team fell behind midway through the second half when Stewart slotted home from the spot after Danny Fox's foul on former Sheffield United forward Christian Nade.
The hosts piled on the pressure in search of an equaliser but Georgios Samaras wasted several chances before Celtic defender Stephen McManus was sent off in stoppage time to complete their misery. Hearts join Rangers, St. Mirren and St. Johnstone in the last four.
Mowbray made four changes to the team that defeated Hamilton at the weekend, including an overhaul of his forward line as Australia striker Scott McDonald and Samaras were replaced by China's Zheng Zhi and New Zealand's Chris Killen.
The Hoops almost went a goal behind within a minute when Nade lobbed over keeper Lukasz Zaluska, but right-back Andreas Hinkel was able to get back and clear before the ball crossed the line.
Mowbray's men were back on the attack when Shaun Maloney played a one-two with Zheng and brought Hearts keeper Janos Balogh into action from the edge of the penalty area.
First Zheng fired wide from the edge of the area, then Balogh denied Killen with a fine stop from point-blank range.
Paddy McCourt, on as a half-time substitute, gave a glimpse of his ability to conjure up a moment of magic out of nowhere. The Celtic midfielder surged away from Craig Thomson and lashed a ferocious strike against the crossbar.
But Hearts took a shock lead in the 58th minute. Fox tripped Nade inside the box as the Frenchman drove in at goal and referee Craig Thomson pointed to the spot. Zaluska got his hand to Stewart's penalty but couldn't keep it out.
Aiden McGeady set up Samaras for a close-range shot blocked by Balogh and seconds later Samaras chipped the Hearts goalkeeper from 14 yards, only to see the ball strike the base of the post and bounce out.
Samaras wasted Celtic's final chance to force extra-time when he headed wide from McGeady's cross.
Celtic's frustration boiled over in the second minute of stoppage time when McManus was sent off for a wild lunge at Driver after Thomson, who had initially shown a yellow card, upgraded the offence to a red card on the advice of his linesman.