i think it may stem from the overeducated 'elite' class, a perversion of noblesse oblige. the idea that minorities struggle because they are smart/rich/educated enough. and so their continued lording over the unwashed masses is justified.