If you are active on Discord, you may have seen some claims and outrage going around in the past week or so about Discord "sneaking" AI image editing bots into servers. After some explanations in my own Discord servers, I collected information and links to help correct those claims and address the relevant concerns about these applications, and I am sharing that information here for anyone else who might want it (or want something easy to link to).

The Claims

I cannot directly link to the posts I saw, as I only saw them in forwarded screenshots. The "chain" part of them was a screenshot of a Discord message with the following text:

Discord recently partnered with a Generative AI called DomoAI and apparently they’re in every server but sneakily don't show up in the member list.

However they still use the art/images sent in the server to train their AI.

To bypass this, people have found the AI’s ID so we can ban them. The AI has already been banned in this server, but for anyone here who has a server or is in another art/creative space, please use this advice and ban the following ID using the /ban command.

1153984868804468756

It may also be smart to start watermarking your pieces.

They linked to this article from the Domo AI app's website. Some other threads also suggested other bots were "snuck in" to servers and listed the user IDs for those bots: Viggle.AI, Glif.app AI, and Picsi.Ai.

Other Corrections

While I'm certainly not the first person to correct this misinformation, I made my own write-up because none of the posts that did so had quite all the information I wanted or the sources/format I prefer. Here are some good threads on BlueSky that similarly debunked these claims.

This thread by catboy.ceo is a good explanation about the difference between server and external apps, and why the suggested "solution" in those claims wouldn't work. This thread by essem.space focuses on how external apps work, what they can do, and why they can't scrape servers.

The Facts

Server vs. User Applications

There are two ways to use applications on Discord: server applications and user applications. https://support-apps.discord.com/hc/en-us/articles/26593412574359-how-to-use-apps#h_01j8jg3phmqmkwfs18e0g5213g

Where You Can Use Apps on Discord
Apps on Discord can be added to servers by server admins or installed by users to their own accounts.

1. Apps added to servers: Users need to be the server owner or have the Manage Server permission enabled in order to add an app to that server. It is then available by default to all users in that server. It can also ask for permissions to do things in a server like a user, such as making channels, deleting messages, or editing roles. An App added to a server will appear as a member in that server.

2. Apps added to your account: When you add an app to your account, that app is then available for you to use in all your servers, DMs, and GDMs. When you use it, other users in the space may also be able to interact with it.

Server applications may include popular bots like Carl-Bot, Sesh, Friend Time, etc.
To see which user applications you have added, go to User Settings > Authorized Apps.

The image editing tools being discussed here are user applications. These particular applications are more visible due to being promoted by Discord, but they operate the same as other third-party user applications. The individual user has to choose to enable these applications, and choose which images they use them on, entirely at their discretion. It is functionally equivalent to saving the image to one's computer and uploading it on another website.

A user application cannot read content in a server of its own accord, and thus cannot "scrape" a server for AI training data. https://support-apps.discord.com/hc/en-us/articles/26501864012951-moderating-apps-on-discord#h_01hzqqqead1ncvp0mb1vqxv5c0

Because these apps are not installed directly to your server, they don’t have permissions to do anything other than send messages, including attachments and embeds.

Server settings cannot stop users from using external user applications in the server. https://support-apps.discord.com/hc/en-us/articles/26593412574359-how-to-use-apps#h_01j8jffxhv1qf8hrctbh40h5c8

Q: Is it possible to disable apps that users in my server have installed to their account?

A: Although it’s not possible to fully disable these apps in your server, you can edit permissions so only a user that uses a command sees the results.

To edit permissions, head to Server Settings > Roles and with the correct role selected, tap on Permissions. Scroll down until you see Apps Permissions and make sure that Use External Apps is disabled.

More information on moderating apps can be found in the Discord support article Moderating Apps on Discord.

Banning Bots

The only thing banning the accounts would do is prevent moderators from adding the app as a server application. This is a redundant action, as bot accounts cannot join servers without a moderator adding them. From what I can tell, banning the bot does not even prevent moderators from adding it, as the act of adding the bot seems to automatically unban the account first.

(Even if a bot was somehow be added without moderator action, which there is no evidence of, many servers are set up in such a way that bot users need to be given a specific role or permissions to see most channels. As such, this hypothetical "hidden" bot would still have no way to access most server content. This may be a reassurance to members of servers with this sort of gated set-up.)

Servers demonstrating a successful ban of these bot accounts is not "proof" these bots were ever in those servers to begin with. When using the /ban command and user IDs, it is possible to ban users from servers regardless of whether those users were ever present in the server.

So What Are These Apps Doing?

There is no evidence, as far as I know, of the applications using any content from Discord to train any AI technologies. (Even if there were, the content would be limited to that which users choose to give them.) These types of services make money as "freemium" services which people pay to get full use of, as do many other applications on Discord. These apps being so easily accessible is a method of enticing users into trying their application. That promotion is likely the primary benefit and goal of their partnership with Discord.

You may still dislike the partnership from that angle, or you may find it intrusive that there is a built-in menu for the inclusion of promoted apps. You are free to give Discord feedback about that! I'm not telling anybody that they need to like these applications or their promotion. But I would imagine your feedback has a better chance of being taken seriously if it complains about things that are actually happening, and is based on your own user experience rather than social media outrage.

Discord's AI Policies

Other information about Discord's relevant policies are below. (Some may not trust or believe these policies are being followed. I am not putting these forward as an argument one way or another, but as links to relevant information.)

Discord prohibits vendors they work with from using anything on Discord to train algorithms or models. https://support.discord.com/hc/en-us/articles/24774824186263-master-services-agreement

(g) Restriction on Use of Discord Data. Provider will not use any of Discord Data to which Provider has access to train any algorithm or model, including any large language models, generative AI, or other artificial intelligence technology.

Discord prohibits app developers from using content to train AI models. https://support-dev.discord.com/hc/en-us/articles/8563934450327-discord-developer-policy

21. Do not use message content obtained through the APIs to train machine learning or AI models (including large language models) unless express permission is granted by Discord.

Discord itself uses certain content to train algorithms for the sake of moderation, which is very common. It is the only current way I know that Discord uses user-provided content to train algorithms (which does not necessarily mean they use it to train LLMs or generative AI). https://discord.com/privacy#4

We also use certain information, which may include content reported to us, content that violates our Terms of Service, and certain other content widely available on the service (such as public posts, usernames, avatars, banners, user profiles, server names, server icons, and server banners), to create systems and models that can be automated to more swiftly detect, categorize, and take action against prohibited content or conduct.

Discord's own summary of how they use AI (admittedly this is from a blog post which is a bit outdated, but I don't think the policy has changed): https://discord.com/blog/ai-on-discord-your-place-for-ai-with-friends

We work hard to ensure that everyone on Discord is able to do all of this in a safe, positive environment and are committed to protecting the privacy and data of our users. Our AI features use OpenAI technology, but OpenAI cannot use Discord user data to train its general models. Like other Discord products, these features can only store and use information as described in our Privacy Policy, and they do not record, store, or use any voice or video call content from users. Rest assured, if our policy ever changes, we will disclose that to our users in advance of any implementation.

One way Discord does use AI is conversation summaries: https://support.discord.com/hc/en-us/articles/12926016807575-in-channel-conversation-summaries

Conclusion

I have plenty of my own, more subjective thoughts on this issue, but I didn't really want to get into those too heavily here. They mainly boil down to the sentiment that generative AI companies have no need to sneakily obtain training data from a relatively small number of hostile users, when they've got plenty of easier training options, and a wide pool of eager or at least apathetic users to work with. I'm all for mistrust and skepticism, but that's different from inventing things to worry about. We have plenty of existing things to worry about!

I hope this write-up can be useful to someone else. Thank you for reading!