English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Zendy Plus

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Zendy Tools

Zendy Open

One of the new developments in chit-chat bots is a long-term memory mechanismthat remembers information from past conversations for increasing engagementand consistency of responses. The bot is designed to extract knowledge ofpersonal nature from their conversation partner, e.g., stating preference for aparticular color. In this paper, we show that this memory mechanism can resultin unintended behavior. In particular, we found that one can combine a personalstatement with an informative statement that would lead the bot to remember theinformative statement alongside personal knowledge in its long term memory.This means that the bot can be tricked into remembering misinformation which itwould regurgitate as statements of fact when recalling information relevant tothe topic of conversation. We demonstrate this vulnerability on the BlenderBot2 framework implemented on the ParlAI platform and provide examples on the morerecent and significantly larger BlenderBot 3 model. We generate 150 examples ofmisinformation, of which 114 (76%) were remembered by BlenderBot 2 whencombined with a personal statement. We further assessed the risk of thismisinformation being recalled after intervening innocuous conversation and inresponse to multiple questions relevant to the injected memory. Our evaluationwas performed on both the memory-only and the combination of memory andinternet search modes of BlenderBot 2. From the combinations of thesevariables, we generated 12,890 conversations and analyzed recalledmisinformation in the responses. We found that when the chat bot is questionedon the misinformation topic, it was 328% more likely to respond with themisinformation as fact when the misinformation was in the long-term memory.

Those Aren't Your Memories, They're Somebody Else's: Seeding Misinformation in Chat Bot Memories