the isolated world of scholars who cannot relate to the uneducated
—The university scientists tried to tell the farmers how to grow their crops, but the farmers told the scientists to come down from their ivory tower and see how actual farming is done.

Solomon wrote the Song of Solomon about his love for a woman who becomes his wife. In it, he describes his beloved as a beautiful lady, though he sometimes uses comparisons that sound rather strange to our ears today:

Your neck is like a tower made of ivory [tower of ivory—KJV].
Your eyes are the pools in Heshbon
by the gate of Bath-Rabbim.
Your nose is like the tower of Lebanon
overlooking Damascus. (Song of Solomon 7:4)

In 1837, Charles-Augustin Sainte-Beuve wrote a poem, “Thoughts of August,” in which he says that another poet lives as if he is in an “ivory tower,” going to bed before noon. Since then, “living in an ivory tower” has taken on the meaning of being so caught up with scholarly activities that one is out of touch with the common man.