You did get the ghost potion from the natives, didn't you?
Use the ghost potion on the ghost pirates.