A red house and an old church sit next to each other in a small town in south-west Wales.