I really can't see a problem with importing an "ex-Japanese import" back into Japan, just so long as it's unmodified and none of the original things that made it roadworthy in Japan (eg: the emergency flare is there, the catalytic converter ok, lights/horn etc all working) are still intact. For example, I took some pics of some LHD Celsiors (Lexus SC400s) at the Kobe SuperAutobacs show and (although they were modified) seemed to be something that a few enthusiasts were willing to go through to be unique (having a LHD Celsior I guess is not so unique now though?).
Maybe the question isn't "can it be done?" but "if it's possible?" under small volume import laws...
It's all just a guess though, I don't know anything really. What car is it? Can I ask why you want to do it?