Stoke and Fulham were the biggest casualties in the League Cup second round as the Premier League clubs slumped to shock defeats against Swindon and Sheffield Wednesday respectively.
Tony Pulis's Stoke had held Arsenal to a 0-0 draw on Sunday, but League One side Swindon, managed by former West Ham striker Paolo Di Canio, upset the odds with a dramatic 4-3 victory after extra-time at the Britannia Stadium.
Fulham were the other top-flight side to bow out as Martin Jol's team were beaten 1-0 by Championship club Sheffield Wednesday.
Pulis made six changes from the Arsenal match and one of those switches quickly backfired as a mistake from reserve goalkeeper Thomas Sorensen allowed James Collins to score in the 27th minute. Collins struck again four minutes before half-time with a header from Raffaele De Vita's cross.
Kenwyne Jones launched Stoke's fightback when he scored in the 63rd minute and Jon Walters came off the bench to force extra-time with an 86th minute equaliser. Aden Flint put Swindon back in front in the 105th minute and, after Peter Crouch equalised in the 111th minute, Collins capped a rollercoaster clash when he completed his hat-trick with the winner in the last minute of extra-time.
At Hillsborough, Fulham became the latest victims of Wednesday's revival under boss Dave Jones, who has inspired promotion from League One and an unbeaten start to life in the Championship since taking over in March. Gary Madine's 50th minute penalty, awarded when Chris Maguire was hacked down by Stephen Kelly, was enough to see off the Cottagers.
Aston Villa, beaten in their opening two Premier League matches, won for the first time under new boss Paul Lambert as goals from Fabian Delph, Chris Herd and Darren Bent secured a 3-0 victory over League One side Tranmere at Villa Park.
Japan striker Tadanari Lee scored his first goal of the season for Southampton as Nigel Adkins' team won 4-1 at Stevenage. Billy Sharp, Jason Puncheon and Ben Reeves also got on the scoresheet as Saints won for the first time this season.
Swansea maintained their 100 per cent start to the season as Michael Laudrup's side defeated Championship team Barnsley 3-1 at the Liberty Stadium. Korea Republic midfielder Ki Sung-Yueng, the club's record signing from Celtic, made his Swansea debut, but it was Danny Graham, after 24 minutes, and Luke Moore, in the 59th and 88th minutes, who got the goals.
Mali striker Modibo Maiga bagged his first West Ham goal to seal a 2-0 win over League One club Crewe at Upton Park. Nicky Maynard put Sam Allardyce's side ahead before Maiga, a pre-season signing from French club Sochaux, struck in the 55th minute.
Former Chelsea defender Jose Bosingwa hit his first goal for QPR, while Shaun Wright-Phillips and Bobby Zamora also netted in a 3-0 win over League One team Walsall at Loftus Road.
Sunderland gave debuts to new signings Adam Johnson and Steven Fletcher as Martin O'Neill's side strolled to a 2-0 win over League Two outfit Morecambe thanks to two goals from James McClean at the Stadium of Light.
Wigan cruised through with a 4-1 win at Championship club Nottingham Forest, with Mauro Boselli, Maynor Figueroa, Jordi Gomez and Callum McManaman scoring for Roberto Martinez's side, who also had defender Antolin Alcaraz sent off for a second booking after the interval.
Shane Long scored twice as West Bromwich Albion came from behind to beat League One side Yeovil 4-2 at Huish Park, while Norwich beat League One club Scunthorpe 2-1 at Carrow Road.
Reading fell behind twice against Championship team Peterborough, but eventually went through 3-2 at the Madejski Stadium.