At the centre of Eye is a Victorian town hall built of flint and brick in 1857. The design by architect EB Lamb is not to everyone's taste, and the hall caused a bit of controversy when it was first built. The best place to get a good overview - literally - of Eye is from the walls of Eye Castle.
The castle mound was built in the 11th century, and a stone keep was added later. Much of the keep is gone now, but enough has been restored to give some idea of the original structure.
You can climb up a modern stair to the level of the ramparts and look out over the town, and across to the parish church of St Mary. The church boasts a medieval painted screen, a pair of 15th-century tombs, and a superb modern rood by Sir Ninian Comper.
Next to the church is the wonderful timber-framed medieval Guildhall, dating to the 15th century. Near the centre of town is a crinkle-crankle wall, a serpentine barrier of red brick enclosing the grounds of Chandos Lodge.
The local council puts out an extremely useful 'Eye Town Trail' leaflet with details of a circular walk that takes in most of the historic buildings and gives a fascinating overview into the history of the town. It is well worth taking the time to follow the trail, which should take only 30 minutes or so, depending on how long you stop at the church and castle.
If you gather from the above that I really enjoying visiting Eye, you're right. It is a delightful town, more village than town-sized, chock full of interesting historic buildings.