He has also included uncapped Doncaster midfielder Brian Stock for the first time in almost a year.
Skipper Bellamy, Koumas and Myhill were all out of the squad that lost 2-1 in Montenegro last month.
Bellamy's non-appearance in Podgorica came after his club, Manchester City, admitted they forgot to tell the Football Association of Wales that the striker was injured.
But Bellamy insisted there were no doubts about his commitment to Wales, and Toshack has drawn a line under the incident and will welcome back the 56-cap front man.
Koumas has not been involved with Wales since the home defeat by Finland in March, with injury then ruling him out of games with Germany, Estonia, Azerbaijan and Montenegro.
The unlucky Stock has travelled frequently with the squad without actually getting on the pitch for his debut, but has been included because Toshack believes he now lacks experience in his midfield following the retirements of Carl Fletcher and Carl Robinson.
West Ham's Jack Collison is also included despite having to attend his father's funeral on Monday. He is also struggling with a calf injury.