The sources said that the explosions happened when the Houthis were gathering at the training institution near al-Hodayda airport in the port city, which was taken over by the Shiite group in September.
At least 18 Houthi militants were killed, the sources said, adding that the death toll could reach dozens.
Houthi militants have cordoned off the area, while ambulances were seen crowded at the scene.
A medic told Xinhua by phone that up to 15 wounded were sent to hospital, most of them in critical condition.
Meanwhile, al-Qaida in the Arabian Peninsula (AQAP), known locally as Ansar al-Sharia, claimed responsibility for the attack in its twitter account.
It said a car carrying half a ton of explosives run into the building controlled by the Houthi, and then attackers wearing explosive belts broke into the building, killing dozens of the Houthi militants.
Pictures distributed by local media show that the building was completely destroyed in the blast. And local residents said huge explosion rocked the whole city.
Houthi sources said they repelled the attack and killed two al-Qaida attackers.
This is the second bomb attack against the Houthi group in a week.
On Tuesday, two car bomb explosions targeting Shiite Houthi gunmen killed 40 people, including about 20 children, and left dozens wounded in the country's central province of al-Bayda.
Security situation in Yemen rapidly worsened after the Shiite Houthi group took over the capital Sanaa by force in September. More than 400 people were killed in the week-long fighting.
The Houthi group clashed with Sunni tribes and the AQAP in the past two months in the country's southern regions where the group was trying to expand its influence.