The scolding came in the course of his address on Tuesday in the Ethiopian capital Addis Ababa to the 54-member continental bloc.
"Africa's democratic progress is also at risk when leaders refuse to step aside when their terms end," Obama said. "No one should be president for life."
Obama said he himself was looking forward to handing over to his successor.
"I have to be honest with you: I just don't understand this," he said.
"Under our constitution, I cannot run again. There's still so much I want to get done to keep America moving forward. But the law is the law and no one is above it, not even presidents.
"And, frankly, I'm looking forward to life after being president. It will mean more time with my family, new ways to serve, and more visits to Africa."
Obama singled out Burundi's President Pierre Nkurunziza - whose re-election to a third term provoked weeks of unrest in the small central African nation - as an example of the dangers of trying to stay put.
"When a leader tries to change the rules in the middle of the game just to stay in office, it risks instability and strife, as we've seen in Burundi."
He said clinging to power was "often just a first step down a perilous path".
"If a leader thinks they're the only person who can hold their nation together, then that leader has failed to truly build their country," Obama said.
He cited Nelson Mandela, the late South African statesman, as an example for the continent.
Obama also called on Africa to end "the cancer of corruption" and embrace democracy to ensure continued progress.
He also warned African governments against cracking down on opposition groups and dissent.
He said Ethiopia "cannot unleash the full potential of its people" if it jails journalists and restricts legitimate opposition groups. Ethiopia claims that journalists have been detained because of crimes.
