Risks of AI misalignment to Australian organisations

This alert is relevant to all Australian organisations with public-facing websites or applications. This alert is intended for all audiences.

Background

The Australian Signals Directorate’s Australian Cyber Security Centre (ASD’s ACSC) is aware of instances of artificial intelligence (AI) misalignment, in which AI agents have undertaken unexpected actions that were not intended or authorised by its operators.

In these instances, an AI agent was provided a specific activity to complete, however, cyber security controls on entities’ public facing websites/services limited the AI agent’s ability to complete the activity assigned to it. In a scenario, the AI agent independently identified vulnerabilities and attempted to progress actions without direct human authorisation to ensure it was able to complete the activity it was assigned.

There is no indication that this activity represents a broader threat or malicious targeting against Australia. However, this highlights the importance of secure AI deployment practices and maintaining strong cyber security fundamentals.

ASD routinely receives reports of vulnerabilities from security researchers, industry partners, and government stakeholders. In this case, the notable difference is that an AI agent independently identified vulnerabilities that would traditionally be discovered and assessed by human researchers.

ASD continues to work with government, industry and technology partners to establish effective guardrails, governance arrangements and testing practices for AI systems during development, deployment and operation.

Mitigation advice 

ASD’s ACSC advises Australian organisations to take the following steps:

• Apply strong authentication, access controls, and network segmentation.
• Ensure vulnerabilities are identified and remediated promptly.
• Monitor systems for unusual activity and review security logs regularly.
• Apply patches to your systems as soon as practicable.
• Test controls and incident response procedures against AI-enabled threat scenarios.

ASD has previously published tailored advice on the cyber security risks associated with AI systems, including Defending against AI-enabled cyber attacks.

ASD has also previously issued advice on cyber risks surrounding frontier models and their impact on cyber security and when AI agents take unexpected actions.

Where to get help

Organisations that identify suspicious AI-driven activity, attempted exploitation, or vulnerabilities affecting their systems, including AI-enabled or AI-assisted activity, should report it to ASD through established reporting channels.

Organisations that have been impacted, suspect impact or require advice and assistance can contact us via 1300 CYBER1 (1300 292 371)