Libertarianism as a movement seems to be full of difficult and unlikeable people, with a mindset of extreme individualism and independence. They struggle to form and maintain large enough social groups to make meaningful change.
Conservative and progressive parties attract compatible, socially cohesive people working on (in essence) the reduction of societal friction through standardisation of interaction.
Libertarianism conversely aims to remove any standardisation and leaves maximum societal friction. You are free to conduct your business how you individually please, but so is your customer, competitors etc.
Libertarianism has meritocratic success outcomes, and abandons the unproductive. This is detrimental to at least the lower 50% of society.
So a libertarian party relies on hyper-individualistic, high-merit people, forming a group and coming to a consensus which is all the exact opposite of their nature. It may happen occasionally but it seems unsustainable.
Anthropologists say the single biggest success contributor for Homo sapiens was their ability to form larger communities over Neanderthals who could only form small communities.