Sserumaga from leading Ethiopian club Saint Georges crossed for Dan Wagaluka to head past Ali Mwadini three minutes before half-time at the National Stadium in capital city Dar es Salaam.
A low pass from the left split open the Cranes defence at the start of the second half and Ali Badru evaded Isaac Isinde before firing past Iceland-based goalkeeper Abel Dhaira to equalise.
Substitute Sula Matovu and Emmanuel Okwi were involved in the build-up to the decisive goal 12 minutes from full-time with Sserumaga unleashing an unstoppable shot past Mwadini.
Burundi, who came closest to winning the annual championship when finishing second behind hosts Ethiopia seven years ago, cruised to a 4-1 triumph over Somalia in the other Group B game.
Zimbabwe were included in the tournament as last-minute replacements for Namibia and open their Group A campaign against Djibouti Sunday, a day after title holders Tanzania tackle Rwanda in the same pool.
Kenya, Ethiopia, Malawi and Sudan -- all former winners of a competition whose origins date back to 1926 -- comprise Group C, which appears the most difficult to qualify from.
Teams finishing first and second in each mini-league advance to the quarter-finals of the Challenge Cup along with the best two third-place sides.