In the advisory, CERT-In describes various uses of AI language-based applications, including understanding, interpreting, and enumerating cybersecurity contexts, reviewing security events and logs, and interpreting malicious code and malware samples. I added that I am using the application. .
“These applications can be used for vulnerability scanning, translating security code from one language to another, or transferring code into natural language, performing security audits of code, VAPT, or interfacing with SOCs and SIEMs for surveillance reviews. It may be used for application integration, to generate alerts,” the advisory states.
However, according to CERT-In, AI-based applications can also be used by threat actors to perform a variety of malicious activities, including:
-
Threat actors use this application to create malicious code, exploit vulnerabilities, perform scans, perform privilege escalation and lateral movement, and build malware or ransomware for targeted systems There is a possibility.
-
AI-based applications can generate output in the form of human-written text. It can be used to spread fake news and scams, generate misinformation, create phishing messages, or create deep fake texts.
-
Attackers requesting promotional emails, shopping notifications, or software updates in their native language can get well-crafted responses in English, which can be used in phishing campaigns.
-
It uses domains similar to AI-based applications to create fake websites and webpages that host and distribute malware to users through malicious links and attachments.
-
Creation of fake applications impersonating AI-based applications.
-
Cybercriminals use AI language models to gather information from the internet, such as articles, websites, news, and posts, to obtain personally identifiable information (PII) and extract text without explicit consent from the owner. It has the potential to build a corpus of data.
Here are some of the security measures that CERT-In states in its recommendations to minimize adversarial threats arising from AI-based applications.
-
Educate developers and users about the risks and threats associated with interacting with AI language models
-
Check domains and URLs spoofing AI language-based applications and avoid clicking on suspicious links.
-
Implement appropriate controls to protect data security and privacy. Do not submit sensitive information such as login credentials, financial information, copyright data, etc. to such applications.
-
Make sure the generated text is not used for illegal, unethical activities, or dissemination of misinformation.
-
We use content filtering and moderation techniques within our organization to prevent the spread of malicious links, inappropriate content, or harmful information through such applications.
-
We secure our systems and infrastructure and conduct regular security audits and assessments to identify potential vulnerabilities and information leaks.
-
Organizations may continuously monitor user interactions with AI language-based applications for suspicious or malicious activity within their infrastructure.
-
Organizations may develop an incident response plan, establishing a set of activities to follow in the event of an incident.
Read the full CERT-IN advisory here
