Germany led 2-0 early in the second period when Mesut Ozil converted a penalty, but Austria replied and had their visitors on the ropes for the final half-hour, missing numerous chances to make the breakthrough and grab at least a point.
Indeed, Germany struggled from start to finish against a hungry Austria, but Low said all his team had failed to do was land a decisive blow. "We had the chances to make it 3-0, which we should have done," he said.
"They showed a lot of energy but they tired after an hour or 70 minutes and we should have got the third goal and that would have been that. Instead, we let them come at us and committed too many mistakes."
Low admitted he was not surprised by Austria's strength, or their doggedness, in what he described as "the biggest game they can have".
He told Germany's ARD television: "I don't think there is one game in which Austria want to win more than against Germany. This makes them so motivated. We lacked self-assurance and got nervous and we did not deal with that very well."
Real Madrid midfielder Sami Khedira agreed: "We can be happy with our three points, but not with the way we won them. We were up 2-0 and then we let Austria back into the game with tactical errors. That should not happen."
Bayern Munich's Thomas Mueller added: "We made mistakes and literally handed our opponents two or three invitations (to score). We did not deliver a perfect game. We were just lucky in the end."
Austria were left frustrated at the final whistle given the number of opportunities they wasted. Marko Arnautovic missed the best chance in the closing moments of the game, but he received support from his team-mate Martin Harnik.
"I reminded him that he set up our goal well and had played a good game and it could just happen to anybody" said the Stuttgart attacker. "It's deja-vu from last year - we really had to win. Germany had maybe three chances and we had at least double so we are disappointed. Now we have just got to pick up the points against other nations and not Germany."