Your main worry is attacks on the model
If you need to catch prompt injection, jailbreaks, or data leaking out through a reply, use a runtime security product such as Lakera or Zenity. They're built to detect attacks. VeriProof checks business rules and doesn't look for attacks at all.