Will I be liable if I refuse to work with an employer who harasses me?

I have agreed to be a company director to someone who's living in UK. I have submitted documents and details of me to the company.

The guy I am working with has mental issue and I wasn't able to provide him help. He swears and speaks ill of me. I told him I am no longer willing to work with him after the way he behaved.

He threatened me and said he will file a case against me. Will it be possible as he is in the UK and I am in the UAE?

He has my passport details. But I merely did not do anything offensive against him. I have all his voice recordings as proof.