Government troops captured the mountainous al-Mahfad area in Abyan province last week, leaving Azzan in Shabwa as the militants' main remaining stronghold.
"An official military source in the third military region said that units of the armed forces and the security have entered the town of Azzan in Shabwa Province," it said.
"The source said security and stability were gradually returning to the regions which had been cleansed of terrorists"
Major powers are keen on Yemen curbing the Islamist insurgents and restoring order in the south to prevent threats to top oil exporter Saudi Arabia next door.
Yemen is home to al Qaeda in the Arabian Peninsula (AQAP), one of the movement's most active branches, which has planned attacks against international airliners.
Security forces on Wednesday killed a prominent Islamist militant suspected of masterminding attacks on Westerners, including a French security agent gunned down two days ago, the country's supreme security committee said. REUTERS