AI Security: Defending With and Against Artificial Intelligence
AI security sits at the intersection of artificial intelligence and cyber security. It covers two connected challenges: using AI to strengthen cyber defence, and securing AI systems themselves against new forms of attack. Both are reshaping how organisations protect their data and operations.
How AI is changing cyber security
AI is transforming both attack and defence. Defenders use machine learning to detect anomalies, triage alerts and automate response at scale. Attackers use AI to craft convincing phishing, generate malware variants and accelerate reconnaissance.
The result is a faster, more automated threat landscape — one where human expertise and community knowledge-sharing matter more than ever.
AI-powered attacks to watch
Generative AI lowers the barrier to sophisticated attacks. Security teams should understand the techniques adversaries now have access to.
- AI-generated phishing and business email compromise
- Deepfake voice and video for social engineering and fraud
- Automated vulnerability discovery and exploit generation
- Adversarial inputs that fool machine-learning models
Securing AI systems and LLMs
As organisations deploy AI and large language models, those systems become assets that must be secured. Risks include prompt injection, data leakage, model poisoning and insecure integrations.
- Protect training data and model integrity
- Defend against prompt injection and jailbreaks
- Prevent sensitive data leakage through AI outputs
- Govern access, logging and monitoring of AI systems
Managing AI risk
Effective AI security combines technical controls with governance — clear policies, risk assessment and skilled people. ICCSO's community helps members keep pace through knowledge sharing, training and collaboration on AI in cyber security.
Frequently asked questions
What is AI security?
AI security covers both using artificial intelligence to improve cyber defence and securing AI systems themselves against attacks such as prompt injection, data leakage and model poisoning.
How is AI used in cyber security?
Defenders use AI for anomaly detection, alert triage, threat intelligence and automated response, while attackers use it for phishing, deepfakes, malware generation and faster reconnaissance.
What are the security risks of using AI and LLMs?
Key risks include prompt injection, sensitive data leakage through model outputs, training-data poisoning, insecure integrations and over-reliance on AI decisions without human oversight.
How can organisations secure their AI systems?
Protect training data and model integrity, defend against prompt injection, prevent data leakage, and apply strong access controls, logging, monitoring and governance — supported by skilled teams and community knowledge sharing.