Vinayaka has gained popularity only recently. We can make them understand this by pointing out:
Vinayaka has not been mentioned in the Vedas. Out of the many gods that have been mentioned by Vedas the name Vinayaka is not there. There is no doubt that Vinayaka is a devata, but he is not Vedic God.
During ancient times Vaidikas including Smarthas did not worship gods like Iyappa, Vinayaka, etc as they are not mentioned in the Vedas.
Vedas have indicated Lord Vishnu alone is the Supreme Lord. We shall not draw the wrath of other demi-gods when we do not worship them as we are the devotees of Lord Vishnu.
The principle of Veda is to worship Lord Vishnu which in turn pleases Vinayaka.