The win at Munich's Allianz Arena, in front of 68,000 fans, leaves Germany five points clear at the top of Group C with three games left, including Tuesday's away trip to bottom side Faroe Islands.
Veteran striker Klose, 35, equalled Gerd Muller's 39-year-old record of 68 goals for West Germany when he gave his side a first-half lead on his 129th international appearance. Bayern Munich pair Toni Kroos and Thomas Muller added second-half goals.
Muller, 67, netted the last of his 68 goals with the winner in the 1974 FIFA World Cup Final, when West Germany beat the Netherlands 2-1 in Munich on his 62nd international. Klose has needed more than double the amount of matches to equal the German record.
Another milestone was honoured before kick-off as captain Philipp Lahm was presented with his 100th cap, becoming the eighth German to reach the milestone. Michael Ballack, whom Lahm succeeded as German skipper at South Africa 2010, was also honoured having retired last year and made the last of his 98 appearances in March 2010.
The best chance of the opening half hour came when Marco Reus's shot was parried by Austria goalkeeper Robert Almer, who managed to get back on his feet fast enough to save Klose's header.
But five minutes later the veteran was not to be denied his landmark goal when Muller fired in a cross and Klose fired home off the shin of Austria defender Aleksandar Dragovic on 33 minutes.
Kroos doubled the hosts lead on 51 minutes when he slotted home a Reus pass from well outside the penalty area which gave Almer no chance.
Having leaked nine goals in their last three previous friendly games, Germany's defence was noticeably more robust, even with Joachim Low's team missing vice-captain Bastian Schweinsteiger and attacking midfielder Mario Gotze.
Low brought on Borussia Dortmund's defensive midfielder Sven Bender for Klose ten minutes from time to safeguard the lead, and Muller netted the third on 88 minutes when he converted replacement Benedikt Hoewedes's pass in front of goal.
Victory gave Loew's side their sixth win from their seven qualifiers with only last October's dramatic 4-4 draw with Sweden in Berlin - when Germany threw away a 4-0 lead - the only blot. The defeat leaves Austria third in the group, three points behind Sweden, who moved up to second after beating Republic of Ireland 2-1 in Dublin on Friday.