Mate you basically need to play the victim (which you are) tell him "after being burned on this deal, you no longer have any confidence in him and his ethics and feel its best to walk away" and ask for your deposit back.
Don't get abusive or anything stupid like that, but make him realize he has lost your trust and nothing he can say will change your opinion