The task of multi-objective alignment aims at balancing and controlling the different
alignment objectives (eg, helpfulness, harmlessness and honesty) of large language models …