The church dates to the Norman period, with a lovely round tower at the west end. The tower is Norman, though it was heavily restored in the Victorian period. There is a south aisle with a clerestory rising above it and a very nice south porch.
In the interior, the best feature is the huge 13th-century font. This is a massive square bowl, with panelled sides, set on four slender pillars. Beside the font is a bench with fascinating carved figures on the ends, including a dog with an oversized head, and what looks like a giraffe.
There are some very nice remnants of medieval stained glass in the north wall windows and some suitably worn medieval benches. On the south wall of the chancel are several panels of a medieval rood screen, with traces of the original paint still obvious. Set into the north wall near the altar is an ornate Easter Sepulchre, flanked by crocketed pinnacles and decorative finials.
There is a 17th-century table which probably served as the high altar, and in the vestry is a 17th-century parish chest, secured by three locks. The door giving access to the belfry stairs is medieval, and the stonework around the door is carved with interesting graffiti, including the name of John Sellynge and geometric patterns.
The churchyard is managed by the Suffolk Wildlife Trust as a habitat for local animals and insects. All Saints is no longer used for regular worship - not too surprising when you consider its isolated position - and is now cared for by the Churches Conservation Trust. South Elmham is a huge parish - at last count, there were fully nine churches serving the inhabitants.
Do take the time to visit South Elmham Minster, just over a mile away beside the ruins of a medieval bishop's palace.