There is currently no UI to manage your interests / followed hashtags other than the feed configuration setting, which really isn't the best management for these, but at least you could see all of the ones you're following and then browse the to the hashtag to unfollow it. Perhaps this should be under list management?