.png){kind=logo}
Is AI a Tool or a Potential Vulnerability?
- Chris Perumal
- Apr 23
- 2 min read
Has your team adopted the use of AI as a part of its business development efforts? Many have. While it may offer some benefits, giving AI tools access to your valuable business data can pose real risks. We recommended treating AI as a system whose risks are managed, just like any other enterprise risk.
So how can we be safe while using these tools? NIST and OWASP recommend the following to protect your valuable data while using AI tools.
Classify your data.Users are the first line of defense for the enterprise. Determine what types of data/documents are acceptable to send to AI. As an organization, assign classification to your documents, such as “public,” “internal,” or “confidential,” and then decide which levels of classification can be used with AI. Simple guidelines such as “do not paste sensitive information into AI tools” can go a long way to guard against leaks.
Restrict what the models can do and see. When working with large volumes of structured data, have AI generate scripts or an algorithm that can perform the desired action rather than uploading the file to the AI tool. For example: If you’re working with an excel sheet that contains user information, instead of uploading the document to AI and exposing valuable user information, ask AI to write a script that can pull out the first name and last name out of the excel sheet. This way, valuable data is never exposed outside of the company.
Treat AI outputs as untrusted until validated. AI is not perfect and can make mistakes. Before using or trusting any of the data that AI gives you, (including scripts, SQL statements or information that you’re trying to lookup/reference), make sure that you verify the information manually. Sometimes the output from unknown tools can be hazardous. OWASP recommends that output from AI tools must be sanitized to remove cross-site scripting, SQL injection or remote-code execution attacks. This helps us minimize the risk.
Look out for data and model poisoning. Currently, a very popular type of cyberattack is the insertion of “backdoors” into an organization’s software repository. A backdoor attack is when a malicious user creates or makes use of an existing vulnerability to access a system, application, or dev/ops framework; while bypassing normal security protocols that are in place. Think of it as secretly installing a hidden entrance that no one knows about. Attackers may try to use this to steal data, or poison existing data or models that affect reliability and accuracy. It may also be used to insert biases into your data, changing its meaning. This risk can be controlled by monitoring the model’s behavior to ensure that it stays consistent, vetting data vendors, or executing untrusted data sources in an isolated sandbox environment and tracking where the data came from.
By treating AI as a managed risk - through careful data classification, controlled access, vigilant validation, and proactive threat monitoring - organizations can harness its benefits without compromising the security and integrity of their most valuable information.
Sources:
