This question appears to be worldly; as one is devoted to Perumal and one who is aware of this tradition’s greatness, it is better to wear attire as per this custom.
Suppose if the family members object to this type of attire there is nothing wrong in wearing it as per their custom.