There's certainly a vicious cycle where the more guardrails they put on their tool, the more liable we hold them for the things that it produces. But I think part of the secret is that the version with no guardrails at all is far worse than we have been imagining. Like if I was trying to ask a troll on 4chan for help with my programming assignment -- who knows whether they even want to help me. Maybe they'll just berate me for being an idiot who doesn't already know the answer to my question and tell me to kill myself. Or give me something plausible but that will ruin my computer just for the simulated lulz.
In some sense the completely no-guardrails version is literally impossible to use functionally! Even setting an LLM up as a chatbot with "no guardrails" requires a bunch of prompting
There's certainly a vicious cycle where the more guardrails they put on their tool, the more liable we hold them for the things that it produces. But I think part of the secret is that the version with no guardrails at all is far worse than we have been imagining. Like if I was trying to ask a troll on 4chan for help with my programming assignment -- who knows whether they even want to help me. Maybe they'll just berate me for being an idiot who doesn't already know the answer to my question and tell me to kill myself. Or give me something plausible but that will ruin my computer just for the simulated lulz.
In some sense the completely no-guardrails version is literally impossible to use functionally! Even setting an LLM up as a chatbot with "no guardrails" requires a bunch of prompting