With intriguing history, melt-in-your-mouth grub and Palaeolithic caves, there are many reasons to visit this relatively undiscovered hidden gem.