You can only stop the genie after you've followed the vizier. Go upstairs to follow him and wait for Shamir to show up again.