The hillfort was built in two phases; a rampart wall enclosing an area of 4.8 hectares, followed by a much larger enclosure of 10.7 hectares that completely enclosed the first area.
There are structural remains of Iron Age buildings within the outer walls, both round and rectangular in layout. Further Iron Age foundations are visible outside the walls.
As for the medieval castle, it stands on the highest point of the hill, defended by a sheer cliff to the west. The castle is composed of a drystone wall enclosure without any visible internal buildings. Castell Carn Fadrun was one of several late 12th century Welsh castles in Gwynedd, others including Dinas Emrys, Castell Pen-y-garn, and Castell Deudraeth.
Unlike later Welsh castles thar were built as defence against the English, these late 12th century fortifications were built as part of the power struggle between Owain Gwynedd's sons, each eager to impose his authority over their father's kingdom.