Matt,

Try Tsuruya Golf. They are probably Japan's largest golf retailer and
have a number of stores in the Tokyo area (Yaesu near Tokyo station,
Kanda, Nishi-Shinjuku etc). Here's a full list of their stores:

http://www.tsuruyagolf.co.jp/map/map1.jsp

Another option if you are looking for good deals is, believe it or not,
Yamada Denki:

http://www.yamada-denki.jp/store/index.html

You can get great deals on direct-import clubs through them. They don't
have any stores in central Tokyo but a trip out to one of their stores
in Chiba or Saitama may be worth it. You need to go to a store that
stocks sports equipment though - not all their stores do.

One other option is Victoria Golf. They have stores at Kanda/Ochanomizu
and Shinjuku. I find them over-priced though compared to Tsuruya Golf.

http://www.victoria.co.jp/shop/shop_type.asp?catcode01=0200

For 2nd hand clubs, try:
http://www.golfpartner.co.jp/

Anyway, that's a start. Let me know if you want a game sometime!

Cheers,

Scott