That's right.
But there is something else you could climb.
Check your inventory.
After you followed Hen Wen into the house, what did Dallben give you?
Use the rope.
Then DO to climb the rope.