Militants seized part of the town of Tadmur located on a strategic east-west route next to Palmyra on Saturday but had been pushed back from the ruins.
According to UNESCO, Palmyra houses the ruins of one of the most important cultural centres of the ancient world and there has been fear around the city that IS fighters may now damage or destroy the site.
Syria's antiquities chief said the insurgents would destroy the ancient ruins if they took control.
The group has demolished similar antiquities in Iraq, seeing them as symbols of idolatry.
Video of ISIS destroying artefacts with sledgehammers at Mosul museum
The Syrian Observatory for Human Rights, a UK-based monitoring group has said a third of Tadmur had been taken by IS after battles with government soldiers and allied militiamen.
"People are very afraid of what will happen, because IS has the capability to get to the heart of Palmyra," an activist in the town told the AFP news agency.
Rising out of the desert and flanked by an oasis, Palmyra contains the monumental ruins of a great city that was one of the most important cultural centres of the ancient world, according to Unesco, the UN's cultural agency.
Irina Bokova, the Unesco Director-General said she was "deeply concerned" by the situation.
"The fighting is putting at risk one of the most significant sites in the Middle East, and its civilian population," she said in a statement.
Palmyra and Tadmur are situated in a strategically important area on the road between the capital, Damascus and the contested eastern city of Deir al-Zour, and close to gas fields.
A US-led coalition has carried out air strikes on the jihadist group's positions since September 2014. However, it says it does not co-ordinate its actions with the Syrian government.