Using your example, let's call your follower a spammer. If nobody has ever liked, followed, replied or interacted in any (nostr event) way with the spammer then you will not see it. Your network starts from you and who you follow and goes forward from there. Who follows you is not automatically part of your network unless any interaction happens.