The by-election was held following the death of Datuk Noor Zahidi Omar (BN) on Aug 20.
Muhyiddin said Barisan had already communicated the win to Prime Minister Datuk Seri Najib Razak, who is now in New York.
"The Prime Minister congratulated the BN election machinery and the candidate, Ustaz Mat Razi on the win."
He said Barisan would like to thank the voters in Pengkalan Kubor as the party only lost in three of the 12 polling district centres.
The win was also because by-election director Datuk Seri Mustapa Mohamed succeeded in repelling PAS attacks on issues including the petroleum royalty.
Muhyiddin said Barisan would deliver all the promises made such as upgrading of infrastructure adding the win was a catalyst to capture Kelantan from PAS.
He urged Mat Razi to serve the electorate regardless of race, religion and political affiliation.
Mat Razi said he was grateful for the win and promised to continue the good work of former state assemblyman, the late Noor Zahidi.
"I would like to thank all those who helped me including the family of the deceased."
He said the win was beyond expectation adding that he would like to see Pengkalan Kubor properly developed. - Bernama