Milito put I Nerazzurri ahead in the first half and they resisted a spell of pressure from the home side before Cassano added the second in the last ten minutes. The win lifts Inter into the top five, leaving Torino beaten for the first time this season.
Possibly showing some nerves on their first big home appearance of the season, the Torino defence invited Milito to give Inter the lead in the 14th minute.
A mix-up involving three players led to the ball breaking loose for Milito, who did not even need a touch to steady himself as he swept the ball in low from 20 yards.
Torino's response was a similar effort from centre-back Angelo Ogbonna which Samir Handanovic saved. Wesley Sneijder tested Jean-Francois Gillet with a dangerous direct free-kick in the 26th minute as Inter continued to capitalise on Torino's insecurities in defence.
As the game progressed, though, Torino shook off their nerves and emerged, with Alessandro Sgrigna hitting the post, although the linesman's flag had already been raised.
Torino captain Rolando Bianchi missed the home side's biggest chance of the game in the 65th minute when he volleyed over the crossbar with only Handanovic to beat.
A minute later, Riccardo Meggiorini broke free, but shot wide of the upright. Those two missed opportunities dented Torino's belief and the chances of them finding an equaliser reduced significantly.
Inter sensed the chance to finish Torino off and they did just that, with Cassano scoring his second goal in as many games in the 83rd minute to end the home team's hopes.