The builder was Sir John de Clevedon (d. 1336), who incorporated an existing 13th century tower in his new house. The tower forms the eastern end of the current manor house.
The medieval house was extended in 1570 when John Wake added a west wing. The Wake family remained owners until 1630 when they sold the estate to the Earl of Bristol.
In 1709 the house was purchased by Abraham Elton, a wealthy merchant from Bristol. Elton was made a baronet in 1717, and his son Sir Abraham Elton III, laid out the terraced gardens, using part of the medieval earthwork defences. The house was rebuilt several times, most recently after a fire in 1882.
Clevedon has associations with the poet Alfred, Lord Tennyson. Tennyson wrote In Memoriam for Arthur Hallam, nephew of Sir Charles Abraham Elton, owner of Clevedon. Hallam is buried in the Elton vault in the nearby church of St Andrews, and the poet stayed at Clevedon in 1850 when he came to visit the tomb. Author William Makepeace Thackeray also visited on several occasions.
The Elton family still live at Clevedon, though the property is now owned and preserved by the National Trust. One of those descendants pioneered the form of pottery we know as Eltonware, and among the interior highlights at Clevedon is a large collection of Eltonware pottery. In addition to the pottery there is a large collection of Nailsea glass on show.
The core of the house is the original medieval hall, with part of the screens passage still intact, leading to a buttery and kitchen. There is a peculiar two storey 'hanging' chapel projecting out over the lower floor. The official English Heritage listing for Clevedon Court describes it as 'one of the most valuable relics of early domestic architecture in England', a rare, largely unaltered example of a medieval house built not for defence, but for comfort.