Arsenal saw goalkeeper Wojciech Szczesny sent off after record-signing Mesut Ozil had missed an early penalty, and eventually went down 2-0 against Bayern Munich in the UEFA Champions League last 16 first leg clash at the Emirates Stadium.
There was a blistering start to the European tie - a year on from their last encounter in north London - and the holders could have snatched an early lead when Toni Kroos fired in a 25-yard effort towards the top corner, which Szczesny tipped over.
Yaya Sanogo, who was selected in attack ahead of France striker Olivier Giroud, saw a close-range shot palmed away by Manuel Neuer, before the German keeper saved a tame spot-kick from Ozil on eight minutes.
There was more drama just before the break as Arsenal goalkeeper Szczesny was shown a red card for bringing down Arjen Robben in the penalty area - only for full-back David Alaba to side-foot his spot-kick against the base of the post.
Bayern, though, went ahead on 54 minutes with a superb 20-yard strike from Kroos, which was whipped into the top corner past substitute keeper Lukasz Fabianski.
Arsenal tired heading into the final 20 minutes as Robben exploited the space in front of a jaded backline.
Eventually, the pressure told as substitute Thomas Muller headed in a late second goal to all but kill off the tie ahead of the return leg at the Allianz Arena next month - where Arsenal won 2-0 last season, but ultimately lost on away goals.
Milan bossed the first leg at the San Siro for long spells but ultimately the class of the La Liga title challengers came to the fore as Costa capitalised on Ignazio Abate's poorly-attempted clearance from a corner to nod home a priceless away goal in the 83rd minute.
For much of the contest it seemed Clarence Seedorf's unrivalled Champions League pedigree might well guide Milan to a valuable win ahead of the return leg at the Vicente Calderon on 11 March, but the Dutchman - four times a winner of this competition as a player - will have come away hugely frustrated after seeing his charges slip to a damaging 1-0 defeat.
The outcome might have been different had Milan not come up against the formidable presence of Thibaut Courtois in the Atletico goal, the giant Belgium international producing two brilliant saves in the first half to keep out first Kaka and then Andrea Poli.
Although much of the first half would belong to the hosts, it was Simeone's men who threatened first, launching an attack straight from the kick-off and forcing the home defence into a hurried clearance from inside their own box with barely 20 seconds on the clock.
Milan quickly settled to their task though and with a quarter of an hour played forced Courtois into action for the first time, the highly-rated 21-year-old keeping out Kaka with a brilliant save, the slightest touch with the tips of his fingers redirecting the ball away from the top-left corner and out of play via a glance off the crossbar.
Yet the Chelsea loanee managed an even better stop two minutes later, somehow tipping the ball onto the post from Poli's free header just six yards out.
The home side rode their luck in defence when Adel Taarabt tried to play the ball out from the back instead of kicking long, but Atletico were unable to capitalise.
And the action was soon down the other end of the pitch as a slick Milan attack ended with the ball at Kaka's feet ten yards out but his powerful side-footed effort curled just over the bar.
Three minutes from the break it was Mario Balotelli's turn to step up as Milan continued to boss proceedings, the Italy international curling the ball just the wrong side of the upright with a first-time effort from 20 yards.
The game tightened up in the second period, although Kaka should perhaps have done better with a shot which was dragged wide of the upright just before the hour.
Balotelli then fired a free-kick into the wall with the continuation of the one-way traffic suggesting Atletico would be more than happy to escape the first leg with a goalless draw ahead of the return in Madrid.
Taraabt fluffed his lines when he was presented with a chance inside the area as the game ticked over into the final 20 minutes, although the momentum soon began to turn.
Raul Garcia stung the palms of Milan goalkeeper Christian Abbiati in the 75th minute as Atletico began to look a little more menacing going forward, and there was further concern for Milan in the 78th minute when Balotelli came off clutching his shoulder.
The game's decisive moment arrived with seven minutes left and is one that Abate will want to quickly forget as it was his attempted header from Gabi's corner that span high into the air and back towards his own goal. Costa still had plenty to do as he back-pedalled to get under the ball before leaping to nod in at the far stick - where a defender on the line should surely have been stationed.
The goal clearly deflated Milan, who had been good value for at least a draw, and when Adil Rami flashed a ferocious free-kick wide in the closing moments their last chance of salvaging a result had gone.