The once chain-smoking, haggard and dishevelled man with his trademark scarf around his neck, had spent more than 16 years at the centre of Kosovan politics, pushing to establish the province as a democratic, sovereign state independent of Serbia.
The United Nations - still administering Kosovo - has launched talks on the final status of the province.
Mr Rugova's long-held vision of a new Balkan future will face a crucial test after his death - largely because of the iconic - if slightly mysterious - aura surrounding his life.
Hailed as the "comeback kid" of Balkan politics when he won Kosovo's presidency in 2002, Mr Rugova's Democratic League of Kosovo (LDK) party was forced to share power after parliamentary elections in 2001.
An atmosphere of mutual distrust has soured relations between the LDK leader and his main political opponent, ex-guerrilla leader Hashim Thaci who now leads the second-largest party, the Democratic Party of Kosovo (PDK).
And the intra-Albanian splits have failed to redress poor relations with the province's Serb minority which is growing increasingly apprehensive as talks beckon.
Many were forced out of Kosovo in the immediate aftermath of the war - others have been on tenterhooks since the March 2004 bout of inter-ethnic violence that culminated in Albanians going on a violent rampage through Serb enclaves.
With Serbia still vowing to oppose any move towards full independence for Kosovo, Mr Rugova's political legacy remains uncertain.
Pushing for change
Mr Rugova was born in western Kosovo in 1944, the son of a shopkeeper who was executed after World War II by the advancing Yugoslav Communists.
Nevertheless the son prospered, going on to study linguistics at the Sorbonne in Paris, before becoming a writer and professor of Albanian literature.
He boasted a passion for poetry, mineral rock samples and Sar mountain dogs from the southern Kosovo border area.
Rarely seen without a trademark silk scarf, he cut a distinctive figure.
He was drawn into politics in 1989 after being elected as head of the Kosovo Writers' Union, which became a breeding ground for opposition to the Serbian authorities.
This activism hardened after Belgrade stripped Kosovo of its autonomy later that year, and led to the establishment of Mr Rugova's LDK.
Throughout the 1990s Mr Rugova was seen as the moderate, intellectual face of Albanian opposition to Slobodan Milosevic's Belgrade regime.
His ambivalent attitude and eventual political support for the Albanian guerrillas of the Kosovo Liberation Army (KLA) went largely unquestioned as support grew in the West for military action against Serbia's brutal rule in Kosovo.
But his involuntary appearance alongside Mr Milosevic at the height of the conflict undermined his reputation - especially among the KLA rebels.
Many felt the man who for years had called for Western intervention was now urging Nato to stop the bombing.
Many Albanians were furious, with some accusing him of treason.
When the Serb authorities allowed him out of house arrest during the conflict Mr Rugova left the Balkans for Italy, his political career apparently over.
Back in charge
But the man sometimes known as "the Gandhi of the Balkans" returned home and used his experience and pedigree as a proponent of Kosovan nationalism to win his coveted presidency.
Long before the KLA arrived on the scene in the mid 1990s, Mr Rugova led the parallel government which the Albanians declared at the start of Mr Milosevic's brutal crackdown.
The LDK was as much a party as a popular social movement. He built the loyalty and trust of the people, which lasted the course.
Ibrahim Rugova campaigned on a pledge to push ahead with demands for full independence from Serbia. In fact, he never changed his tune with pronouncements that at times sounded nothing short of prophetic - if not outrageously unrealistic.
But the repetition and mysterious lack of detail appeared to work wonders with the electorate that trusted him.
And he recovered a bit of his lost ground when he took the stand against his old enemy, Slobodan Milosevic, during the former Yugoslav president's war crimes trial in The Hague.
His home and car have been attacked by bombers, although he has escaped unharmed from each assault.
Despite all his efforts, though, the future of Kosovo is not yet clear.
Talks are due to start shortly, and many Kosovo Albanians will feel sad that their leader will no longer chair their delegation. BBC