When I gave a skull to the Trader the Master of Ordeals makes me leave

Hint 1 of 4:

You have given the Trader all 5 skulls.

Hint 2 of 4:

You have finished the Ordeals, now you need to go to the temple.