Though only with one available substitute on the bench the Africans were brave throughout and briefly were in front in extra time only to concede a late penalty.
The match started with a bang as the two No9s - CIV: Daniel Kouassitchi and Shusei Yamauchi – swapped goals.
But the contest was a disjointed affair at times as both teams battled hard in the late afternoon Papeete heat.
Japan did not reach the levels they did two days ago in defeating Paraguay but eventually their pressure told.
Shinji Makino was dismissed for a second caution late in the game leaving Japan a player short for two minutes at the end of the game.
Star Ivorian striker equalised with a great snap shot only for Ozu Moreira to equalise straight from the kick-off, and Naoya Matsuo then hit the winner from the penalty spot with 41 seconds left on the clock.
Paraguay's result later in the evening - a 4-3 loss against Russia - ensured Japan booked second spot behind Russia.