A security official in the south of the country, where Islamist fighters have seized parts of an entire province, said detainees fled by digging a tunnel leading beyond the prison's walls.
Sixteen were at large, another local official said.
It was the second major jailbreak involving Al-Qaeda members since June, when dozens of Al-Qaeda militants escaped from a jail in another city, Mukalla.
Islamists have been gaining power in the south as Yemen was witnessing a massive uprising against President Ali Abdullah Saleh, who last month finally agreed to step down after 32 years in power.
compiled from agency reports