After Portugal's 2-1 win over Holland in Kharkiv, another three points at Arena Lviv confirmed Germany as Group B winners with Portugal as runners-up from the so-called "Group of Death" while the Dutch and Danes go out.
Despite getting off to a flying start at Euro 2012 with a shock 1-0 win over the Netherlands in their opening match, Danish hopes of progressing to the knock-out phase foundered on defeats to Portugal last Wednesday and now Germany.
Germany took the lead in Lviv when Arsenal-bound forward Lukas Podolski celebrated his 100th international with the opening goal on 19 minutes.
But Denmark hit back just five minutes later thanks to a header from Michael Krohn-Dehli, who scored the winner against Holland, before Lars Bender marked his first start for Germany with his first goal.
"We have to be disappointed, but I'm not disappointed with the team," said the 62-year-old Olsen, who suffered his first defeat to Germany in four games during his 12-year reign as Denmark coach.
"It was a fantastic performance and we played against one of the world's elite - unfortunately we have been punished for our carelessness.
"We were playing against a great team, of course they were better than us and to beat them we would have needed a bit of luck."
Goalscorer Krohn-Dehli could not hide his disappointment as the Danes exit Euro 2012 at the first hurdle, but said they leave with heads held high having beaten FIFA World Cup™ finalists Holland.
"Right now I am very disappointed just like the rest of the team," said the 29-year-old Brondby star. "Of course I am very proud of our efforts and I thought we added an extra dimension to the tournament."