How do I go anywhere other than the town

Go back to the lookout point by walking left along the dock.

Go under the large arch.

But you'll eventually have to sail off the island.