The win sent Joachim Loew's side eight points clear of the chasing pack in Group A, with Belgium, Austria and Turkey all seeking to close the gap on Tuesday. Kazahkstan remain pointless and goalless at the bottom of the group.
"I think the first half was good," said Loew. "We scored three goals and had plenty of chances against a team that played very defensively. But in the second half, we didn't keep up the tempo from the first half and we played a bit too slowly.
"We've still got some tricky games to come, but I'm fairly sure we will qualify."
Loew had called on his men to entertain against their opponents and his players did not take long to warm the crowd up, taking the lead within three minutes. Kazakhstan defending allowed Bayern Munich striker Klose to sneak in completely unmarked at the back post from a free-kick on the left and slot the ball home from less than a yard out.
In the first half, the visitors' goal was under constant siege and Germany missed a couple of chances before finally doubling their advantage in the 25th minute. Real Madrid playmaker Mesut Ozil whipped in another set-piece, from the right this time, into a crowded penalty box and Mueller beat his marker to the header, nodding it smartly into the corner.
The 22-year-old made it three just before the break, chesting down another cross from Ozil, finding himself totally unmarked in front of goal and firing it past the David Loria between the Kazahkstan posts.
With Germany enjoying almost unlimited possession at the Fritz-Walter Stadium, home to Bundesliga club Kaiserslautern. The visitors only had a few breakaway chances of their own, forcing goalkeeper Manuel Neuer into two sharp saves.
The introduction of the Bundesliga's top scorer, Bayern Munich's Mario Gomez, did not give Germany the required cutting edge as they too often tried to pass the ball into the net.
Another pin-point pass from the impressive Ozil found his club colleague Sami Khedira in the 81st minute, but the midfielder could only head wide, capping a frustrating second half for the home side.
However, the dependable Klose gave the fans something to cheer about on their way home, tapping home a fourth for his 61st international goal, bringing him within seven of Gerd Mueller's all-time German record.
"We are pleased we won the game and I think it looks good for us in terms of qualification now," Mueller said.