You can't ride past it.
Go to the Sultan(ess)'s Palace. Try some of the black circles.
Where does the one in the well take you?