The main incentive wad never true colonization – that was a by product. Europe countries were focused on the vast raw resources of Africa and exploited the people and the land for their own benefit.
One word. Greed.
From a certain point of view, it took them centuries to colonize Africa. Portuguese navigators started feeling their way around Africa’s west coast in the later Middle Ages, but aside from Britain’s Cape Colony and a few outposts here and there, Africa remained not a place to go, but a place to go around until the late 19th century.
But, yes, once they actually started in on it, it went quickly. In the not-quite-40-years between the Berlin conference and WWI, Africa went from lightly colonized to almost completely divided between foreign powers. How? I’m chalking it up to two major categories of causes.
First, there’s technology. Some of it is primary transportation technologies starting with steam power and going from there. Steamboats provided relatively high volumes of transport up and down rivers. The construction of railroads like the Uganda railway likewise opened up the interior of the continent. Telegraphs provided rapid communication. And then there’s military technology. It wasn’t impossible to beat Europeans and European-backed native forces on the battlefield, as Khartoum and Isandlwana proved, but it was very, very difficult; those victories tend to stand out, after all, because they were so rare.
Then there are secondary effects of Europe’s massively more sophisticated technological infrastructure. By the late 19th century, medicine provided meaningful treatments for a number of tropical diseases which had up to that point stymied European exploration and occupation of parts of the continent. Better agriculture meant Europeans were better fed and has a relatively enormous supply of people they could send out into the world. And European industry could crank out manufactured goods with which to buy allies.
Then there’s the political will. When the starting gun was fired at Berlin in 1884, the governments of Europe made it a point to make serious moves on African territory, substantiating abstract claims with a physical presence on the ground. They didn’t just have the technological capacity to do it, but they also made it a national priority, bringing those capabilities together in an unprecedented way. The old saw about Britain in particular acquiring a empire in a fit of absent-mindedness doesn’t apply here. There was a definite positive effort to take over.
First: northern Africa was a former Christian region, occupied by the Arabs. Therefore, there was a religious imperative to ‘free’ those zones.
Second: most of maritime commerce in Southern Europe and Mediterranean was under the attack of pirates from northern Africa. Therefore, there was a coimmercial imperative to ‘free’ those zones.
Third: most of the northwestern African cities were important commercial hubs, and made lots of money selling products that, by them only the Genovese and the Venetians would bring from the East. Therefore, it would be cheaper for western European nations to occupy some such cities and acquire those products at a cheaper rate.
Fourth: Meccah had to be invaded and destroyed for religious reasons. In Iberia it was common for a kingdom to ally with other kingdom to control some kingdom in between the two. If they could find some Christian kingdom the other side of the muslim nations, they could ‘sandwich’ the muslims and attack on two fronts. Besides, there was talks about a very rich and powerful Christian king somewhere the other side of Africa. (Such King would contact the Portuguese and the Pope in middle 1450’s.) To surround Islam and find such nations, they would have to sail down the African coast and into the Indian Ocean: the Portuguese had conquered the coast all the way to Equador from 1415 to 1460, and then from 1470 to 1490 they conquered the rest of Africa all the way to Cape of Good Hope.
Fifth: During their progression south, the Portuguese lacked enough men for battle, to protect their conquered cities, to man the ships, to work the fields. But, ‘luckily’, in 1444 they found a new market in Africa: slave trading! Now, with the slaves working the fields and manning the ships, they had enough soldiers for battle. (That’s when Alphonse V, ‘The African’, conquered everything in modern day Maroc and Mauritania.)
Sixth: Gold! A huge gold market in Africa! So huge the Portuguese built a fort and called it “A Mina” (The Mine).
Seventh: After the American continent and a passage to India was found, the Portuguese needed to protect their trading routes. To accomplish that, they needed forts and military and commercial outposts. They would establish embassies and diplomatic relations with local kings.
Eigth: when the entire world jumped on the maritime bandwagon (pun intended), other nations, carrying their own religious and political prejudice, decided they would take a peace of the African pie, by force if necessary. And colonialism began.