Tomatoes took awhile to take hold, with it not really taking off until the 19th century. I've heard it explained as due to a general distrust of 'exotic' foods and vegetables generally at that time and a belief that it was poisonous and would sap your vitality.
There were lots of similar folk beliefs, such as diseases being related to the resemblance to the food you ate. E.g. potatoes looking like tumours therefore giving you tumours. In Germany they were popularised by Frederick the great putting a cursory guard around a field of them and then letting peasants steal them.
From memory tomatoes were for some time grown as ornament in Europe before they were commonly part of cuisine.
