Oddly, I've been stopped by police five times since I've lived here
(three times in Tokyo, twice in rural areas), but only once did they
ask to see the card. Depending on the person and the circumstances, I
think in a way they're actually reluctant to demand it, just because it
feels rude somehow. Maybe I'm being overly sympathetic. But even the
one time I was asked (Tokyo), I really can't blame the guy. It was dark
and at first he didn't realize I wasn't Japanese, but then there he is,
baton in hand, and what does he say to this guy? The card kind of broke
the ice and we ended up having a nice conversation about WWII history.
These days I feel it's like giving them a meishi. Why not?

I do try to draw the line at official people, though. One time in
Fukuoka a hotel demanded not only to see it but to make a photocopy, so
we went to a different hotel.

Maybe your sign was suspicious ;-)