Joining another company during probation

I have signed a contract with an employer and also received the visa, but I have changed my mind and decided not to join the company due to the indecent behaviour of the employer.

I did not join the company and I want to have my visa cancelled. I am ready to pay the visa expenditures incurred by the employer.

What are your advice and suggestion? I may leave the country to take up a job in another country.