Kosova Protection Corps carry Rugova's body to its final resting place.
Rugova, a familiar figure on the world stage, died on Saturday from lung cancer at the age of 61.
Weeping onlookers threw flowers at his flag-draped coffin as it was moved in procession from parliament.
Serbia's president has criticised the authorities in Kosovo for refusing him permission to attend the funeral.
President Boris Tadic, whose country once governed Kosovo, said the decision missed a chance to improve relations between the two sides.
"My wish was to pay respect to the man who thought differently from me, who devoted himself to the peaceful realisation of his ideas and who was a representative of Kosovo Albanians, with whom we share a living space," he said in a written statement.
Rugova spearheaded a non-violent campaign by Kosovo's ethnic Albanian majority to end Serbian rule. Despite being eclipsed by Kosovan guerrillas in the late 1990s, he remained a figurehead for ethnic Albanians.
Kosovo's parliament held a special session to pay its respects on Sunday and 15 days of mourning were declared.
Regional leaders, including the Albanian president and prime minister, Alfred Moisiu and Sali Berisha, and the European Union's foreign policy chief, Javier Solana, joined thousands of ethnic Albanians who had been paying tributes at the parliament building since Sunday.
On Thursday, the coffin was carried to a ceremonial hall where the dignitaries are expected to attend a funeral at midday, before his burial in the Tomb of the Martyrs memorial complex in the Velanija district of Pristina, where Rugova had lived.
Talks on the future status of Kosovo were scheduled to begin this week but have been postponed.
Ethnic Albanians, who make up 90% of the population of Kosovo, hope the talks will lead to full independence from Serbia - an outcome opposed by Belgrade.
Kosovo has been a United Nations protectorate since a Nato bombing campaign forced Serb forces to halt operations against ethnic Albanian separatists in 1999.
Serbia insists Kosovo should remain a part of its territory but the ethnic Albanian majority in Kosovo is demanding full independence.
Soren Jessen-Petersen, head of the UN mission administering Kosovo, described Rugova as "the defining element of politics in Kosovo".
International leaders have paid tribute to him as a moderating force and vowed to continue his work for a stable future for the province.
Parliament has three months to vote in a new president.The head of parliament, Nexhat Daci, has been named acting president.