The basis for this belief was that tomatoes belong to the nightshade family of fruits and several nightshade plants can cause death if eaten in large quantities.
It is unknown if the plant is poisonous to humans, but given that most members of the nightshade family bear poisonous fruits (including potatoes), it is not recommended for consumption.