The Spirit of Satoshi industry report (listen to read by
@Guy Swann in AI Unchained podcast) gives this insight. If a foundation model is trained on mainstream data, fine-tuning on good data is very difficult to correct the prior misinformation.
https://open.spotify.com/episode/1ckb2A3rPY7zGpcxW3jT9E?si=pARMwzPdQ9GEvmRM0MxWww
Now, apply this insight to humans who have been "educated" by state indoctrination, and our goal is to correct their mislearning using good information. How difficult is that task? (The phenomenon of cognitive dissonance is useful in both human and machine intelligence.)
When original training adjusts the model weights (is learned as knowledge in the neurons), it is impossible unlearn (irreversible). New fine-tuning can only make further adjustments that add to what is learned, hopefully with greater weight and without causing confusion.
The difficulty of unlearning can be understood by examining the concept of unit economy in epistemology. Optimizing knowledge compactness. Absorb concrete examples and learn the patterns and principles that are universal, committing the abstraction to one's knowledge. New knowledge contradicting those abstractions is difficult to reconcile.
Knowing this, we should appreciate (1) epistemic humility and (2) be sympathetic to those confused by cognitive dissonance. (1) is the recognition that past learning can be based on false data. (2) recognizes that others reacting badly is natural to their mistraining.
