The Arch of Triumph was "pulverised" by the militants who control the city, a Palmyra activist told AFP news agency.
It is thought to have been built about 2,000 years ago.
IS fighters have already destroyed two ancient temples at the site, described by Unesco as one of the most important cultural centres of the ancient world.
"The Arch of Triumph was pulverised. IS has destroyed it," Mohammad Hassan al-Homsi, an activist from Palmyra told AFP on Monday.