That's probably because all you did was scare him. You didn't bother doing anything to him -- something to get rid of him permanently.
When Schwarzy is scared, he won't notice if you touch him.
Or attach him to anything.