Just regular handsoap of dish washing does the job, together with some hot(ish) water and a good rubbing in with bare hands. For maximum results remove the handle from the stick. Also making a paste from baking soda (adding a bit water) and letting that dry in should do something, although I have jet to try this method myself.