The church itself dates to the period 1180-1220 and has a low bell turret in place of a tower. The interior is beautifully painted in a pastel two-tone design that enhances the simple Georgian fittings. There is a west gallery, and box pews leading to a wooden chancel entry and a three-decker pulpit. The size of the pulpit indicates how important preaching was in Georgian times when sermons could go on for hours.
The box pews were given different sizes compared to the importance of the people they seated. At the front was the tenant of Chalbury Farm, behind was the tenant of Didlington Farm, and behind that the tenant of Uppington Farm. On the south side of the chancel was a long seat for the rectory servants, and a raised seat opposite was for the use of the Earl of Pembroke and his family.
The arrangement shows how pervasive class consciousness was, and how even a simple thing like church seating arrangements and pew size reinforced those social classes.
The church consists of a nave, chancel, and south porch. The east window is a three-light design and set into the south chancel wall is a trefoil-headed piscina. An ancient window is set into the north wall of the nave.
Though the hill on which Chalbury stands is only some 330 feet above sea level, the views are quite wonderful from the churchyard, looking south towards the coast and as far as the Isle of Wight on a clear day. The view is so good that many years ago a tall elm tree in the churchyard was used as a navigation aid by sailors in the Channel. The elm tree was damaged in a great storm in 1703.
The summit of the hill was almost certainly used as a sacred site before the coming of Christianity. In 1878 the rector discovered an ancient urn in the rectory garden, suggesting the presence of a burial mound on the site. The name Chalbury comes from the Old English Cheoles Byrig, meaning Ceol's Fort.