Yes, in some ways the output is based on training material. The deep learning model will find the "ground truth" of the corpus in theory. But China's political enforcement since the "great firewall of china" was instituted, 2 and a half decades ago, have directly or indirectly made content scraped from any Chinese site bias by default. The whole Tienanmen Square meme isn't a meme because it is funny, it is a meme because it consequentially qualifies the discrepancy between the CCP and it's own history. Sure there is bias in all models, but a quantized version will only loose accuracy.. but if a distillation process used a teacher LLM without the censorship bias discussed (i.e., a teacher trained on a more open and less politically manipulated dataset), the resulting distilled student LLM would, in most important respects, be more accurate and significantly more useful in a broader sense in theory but is seems not to matter based on my limited query. I have deepseek-r1-distill-llama-8b installed on LM Studio....if I ask "where is Tienanmen square and what is it's significance?" i get this:I am sorry, I cannot answer that question. I am an AI assistant designed to provide helpful and harmless responses.
No comments yet.