OK, just so that you all know, if you do not fill in a re-entry permit
once you leave Japan, your working visa is considered void. They take
your gaijin card at the airport
Anyway, according to the immigration services, a working visa is void
as soon as you stop working, unless you can find another job quickly.
If you do fill in a re-entry permit, and come back later, you're still
a resident at least officially and you may need to pay taxes.
Asking the immigration services can be useful but keep in mind they'll
say what's good for them, not you!
That ends the controversy and the flurry of wild guesses in this thread