Xi Niu

Fine-tuning LLMs with Cross-Attention-based Weight Decay for Bias Mitigation

Large Language Models (LLMs) excel in Natural Language Processing (NLP) tasks but often propagate societal biases from their training data, leading to discriminatory outputs. These biases are amplified by the models’ self-attention mechanisms, which disproportionately emphasize biased correlations w