As Emperuman is the supreme Lord of all gods, worshipping other gods is like equating Him with the rest.
We should never forget Emperuman is the primordial lord. It is a sin to consider other gods equal to Perumal; the other gods would not prefer it. The demigods feel that there is no benefit in worshipping them. The demigods have only said what benefits they can provide but have not said that they are equal to or superior to Perumal. These are the reasons for not going to their temples for worship.
Some might say there are no tangible benefits of worshipping Perumal while other gods grant benefits for worshipping them. These will end us in worldly sins only. As none is supreme or equal to Perumal we must not worship other gods.