This is a serious problem. Achieving a durable solution through a negotiated settlement is based on the good faith of the parties. If you have concerns that the other side is not acting in good faith then you should raise this with the mediator. Give them a chance to address this problem with the other side. It could be a perception issue or it could be a genuine lack of commitment to the process.
As a voluntary process, you are free to withdraw from the mediation so you may need to do so. If you are mediating in Hong Kong and have filed a Notice to Mediate under Practice Direction 31 then you will need to consider the minimum participation specified in your agreement.
If the other side is not acting in good faith and the mediator establishes this then they may terminate the mediation on the basis of their ethical concerns.