English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

With the boom of Large Language Models (LLMs), the research of solving MathWord Problem (MWP) has recently made great progress. However, there are fewstudies to examine the security of LLMs in math solving ability. Instead ofattacking prompts in the use of LLMs, we propose a MathAttack model to attackMWP samples which are closer to the essence of security in solving mathproblems. Compared to traditional text adversarial attack, it is essential topreserve the mathematical logic of original MWPs during the attacking. To thisend, we propose logical entity recognition to identify logical entries whichare then frozen. Subsequently, the remaining text are attacked by adopting aword-level attacker. Furthermore, we propose a new dataset RobustMath toevaluate the robustness of LLMs in math solving ability. Extensive experimentson our RobustMath and two another math benchmark datasets GSM8K and MultiAirthshow that MathAttack could effectively attack the math solving ability of LLMs.In the experiments, we observe that (1) Our adversarial samples fromhigher-accuracy LLMs are also effective for attacking LLMs with lower accuracy(e.g., transfer from larger to smaller-size LLMs, or from few-shot to zero-shotprompts); (2) Complex MWPs (such as more solving steps, longer text, morenumbers) are more vulnerable to attack; (3) We can improve the robustness ofLLMs by using our adversarial samples in few-shot prompts. Finally, we hope ourpractice and observation can serve as an important attempt towards enhancingthe robustness of LLMs in math solving ability. We will release our code anddataset.

MathAttack: Attacking Large Language Models Towards Math Solving Ability