In 1210 King John seized the castle, and later granted it to William Marshall, Earl of Pembroke. Llewelyn the Great tried to take Haverfordwest on two occasions, but though he succeeded in burning the town, the castle itself held out. In 1257 Llewelyn ap Gruffudd tried to take the castle, but he, too, failed.
Edward I and Queen Eleanor stayed here on a pilgrimage to St Davids in 1283. The Queen was so struck by the castle that she purchased it and paid some £400 for improvements to the fortifications. Welsh insurgents tried once more to take Haverfordwest in Owain Glyndwr's ill-fated 1405 rising, and once again the castle defenders held out.
The castle remained a crown possession until the Civil War. It was taken and retaken several times before eventually falling into Parliamentary control. Cromwell ordered the castle slighted, and it never again served a useful military purpose. That was not the end of its life, however.
The castle was converted to a prison in the 19th century. Several of the prison buildings were later converted to serve as the county archives office and the town museum. Other ruins are two round towers, a chapel, a hall, and a dungeon. The castle is now little more than a scant shell, though it seems very impressive when viewed from below.
It is a short stroll from the castle to the wonderful medieval parish church of St Mary's.