Now that devices are starting to have built in features with AI automatically combing through all information on them, the idea of this sort of stuff being logged in the first place is concerning.

For instance, should someone prompting an AI to describe them beating up and torturing their boss be flagged for "potentially violent tendencies"? Who decides the "limit" where "privacy" no longer applies and stuff should be flagged, logged and sent off to authorities?

As I see it, the real issue is people being hurt, not text or fictive materials, however sickening they might be.

If the resources invested in spying on people and making databases were instead directed towards funding robust and publicly available psychiatric care I expect that'd be more efficient.