Refund of the broker's fee in case of no agreement with the landlord

On 20th April, we paid 5% agency fee and 5% landlord deposit for a new property rental contract, ready to move in on the 1st May.

A week after the 20th April, we contacted the agent and landlord to request to urgently pull out of the agreement due to husband losing his job. We lost the 5% deposit to the landlord as he insisted on cashing our cheque.

However, are we entitled to ask for the 5% agency fee back as we didn't even move into the new property?

We have tried to contact the agent but he is not communicating with us.