Back

AI risks: Stories told and untold by current case law

Posted by Andrew Walker on 17 November 2025
machine learning
data risk
ai cases
artificial intelligence
ai risk mitigation
AI
AI ethics principles
AI risks
data breach
intellectual property
IP
ML
Ai Risks - KHQ Lawyers

KHQ’s Global AI case tracker reveals real-life examples of litigation and regulatory risks associated with the development and deployment of artificial intelligence (AI) and machine learning (ML) technologies.

In this article, we refer to Australia’s regulatory framework and summarise some of the matters that could commonly give rise to AI risks as well as known risk-mitigation solutions ordinarily adopted by organisations. In combination with our case tracker, this article may serve as a starting point for our readers when conducting AI risk assessments. However, please note that you must not rely on the information in these documents and no representation, warranty or advice is given.

AI risks are increasingly evident. While Anthropic’s $1.5bn settlement draws attention to copyright infringement risk, our case tracker also highlights emerging claims of wrongful death linked to AI chatbots, online harms in breach of consumer laws, unfair credit scoring, anti-competitive behaviour and litigants citing fabricated case law.

AI REGULATION IN AUSTRALIA

Australia continues to debate AI risks and appropriate legislative and regulatory solutions. This occurs while the US pushes toward AI deregulation (only challenged by California[1] and a few other states), China has adopted substantial and apparently successful regulation,[2] the EU’s AI Act is gradually becoming effective[3] and is supplemented by codes of conduct,[4] among other developments around the world.

Australia’s ban of DeepSeek from all federal government systems for security and privacy concerns, suggests that sensitivities exist when it comes to national security. However, confusion is caused by a lack of consistency and an apparent preference for unregulated US-made AI[5] over regulated Chinese-made AI.[6] It appears that geopolitical factors continue to play a role.

The Productivity Commission’s recent interim report examining the role of AI in productivity argues for deregulation, suggesting, among other things, that the Privacy Act 1988 (Cth) (Privacy Act) is “constraining innovation without providing meaningful protection to individuals” and that AI-specific regulation should be a last resort. In response, the University of Technology Sydney report reminds us of the cost of AI risks and prefers a “pragmatic, technology-neutral approach” to AI regulation. It says that “Being passive on regulation would invite the worst of all worlds: increased AI adoption wouldn’t drive significant productivity growth, yet harms associated with AI also wouldn’t be appropriately addressed”. Regulation should focus on strengthening competition and consumer law, discrimination law, online safety, copyright, and employment law and the Privacy Act.

These laws continue to regulate AI in Australia to some extent. However, without  reform, there is concern that important AI risks could remain unmitigated. For example, the recent Australian Consumer Law (ACL) review suggests, among other things, that technical amendments should be adopted to ensure the law’s sufficient coverage of AI goods and services and to clarify aftersales obligations. Meanwhile, the Federal Government has already released AI-specific eSafety regulations. The recent Australian privacy reform introduced transparency rules in relation to automated decisions to apply from 2026.[7]

Previously, the Australian Senate’s Select Committee on Adopting Artificial Intelligence (SCAAI) published its full report about regulating and developing AI in Australia and its impacts, drawing comparisons with other countries, and proposing a new, whole-of-economy dedicated legislation to regulate high-risk uses of AI. However, current opposing views could delay the adoption of any meaningful AI-specific regulation. Unlike many of its counterparts,[8] Australia is overdue on updating its AI Action Plan.

Despite a number of existing laws that apply to various aspects of the development and deployment of AI, Australia’s AI regulatory framework remains voluntary, guided by the following core documents:

These documents set out clear guidelines and expectations for AI deployers (and, in part, for developers), some but not all of which are included in our summaries below. As AI systems are increasingly deployed to perform tasks that have significant impact on individuals, a comprehensive AI risk assessment is a must.

The Office of the Australian Information Commissioner (OAIC) has also released the following non-statutory guidance:

At the Government level, Australian agencies are guided by the following documents:

These documents set out the mandatory and recommended criteria for the design, development, deployment, monitoring, and decommissioning of AI systems.

PLATFORMS

Separately, there is a concerted effort to regulate digital platforms in Australia. The Digital Platform Regulators Forum comprises the Australian Competition and Consumer Commission (ACCC), the Australian Communications and Media Authority (ACMA), the eSafety Commissioner and the OAIC.

The DPRF working papers, such as those below, give a flavour of platform risks recognised by these regulators:

HOW DOES AI WORK?

AI refers to technology that mimics human abilities in learning, comprehension, problem-solving, decision-making, creativity, and autonomy.

Traditional algorithms are sets of instructions or processes that allow a computer to perform tasks or calculations. However, these systems, based on symbolic representation of logic gave way to the emerging artificial neural net (ANN). The ANN mimics the synapses in a human brain and delivers outputs by weighing the importance of relationships between information in its layers. As such, there is now limited need to solely rely on predetermined algorithmic libraries.

Machine learning (ML) is the ability for a machine to recognise patterns and learn from datasets. Initially an intensive exercise involving hundreds of human labellers, the transformer architecture developed by Google in 2017 enabled “deep learning” where the machine can recognise patterns in vast unstructured datasets and learn from them without a great need for labelled data. This architecture uses learning algorithms such as backpropagation pioneered by Geoffrey Hinton. This enables the ANN to learn from data, remove errors in predictions and optimise its weights.

ANNs include node layers which are artificial neurons connected to one another with an associated weight and threshold. Information travelling through the ANN is passed on to the next layer by nodes activated beyond their specified threshold value, thus producing a result and working towards delivering a nuanced output. The so-called ‘hidden layers’ between input and output layers enable deep learning of non-linear relationships between input and output data. However, hidden layers are difficult to explain or audit, and, indeed, explaining the ANN might be as difficult as explaining a human brain.

Large language models (LLMs) are the underlying technology behind generative AI. Thanks to deep learning, they can identify complex patterns, summarise text, answer questions and generate new text with increasing accuracy. According to Geoffrey Hinton, LLMs are not dissimilar to how humans use language. The meaning of words is not about discreet rules of syntax but about relationships with other words and about the features of each word – that’s how LLMs use language, as do humans. As the debate about AI being sentient continues, Geoffrey Hinton argues that AI already has “subjective experience” in the same way as humans do – an indirect way of telling you what is going on in its brain, even if reality is different.[9]

In 2025, DeepSeek shook investors by presenting its R1 model, which was trained on vast amounts of synthetic data, relying on a new reinforcement learning method called Group Relative Policy Optimization (GRPO). DeepSeek reduced the need for data, computing power and humans in the loop (HITL), thus outsmarting the usual prohibitive cost of market entry. It demonstrated capabilities such as self-verification, reflection, and generating long chain-of-thought (CoT) for solving complex problems.

This success also sparked controversy amidst allegations of keystroke data collection by DeepSeek. Researchers rebutted this assertion, but data transfers to DeepSeek’s servers and its SDK third parties such as Google or ByteDance remained. However, this risk can be mitigated by running your own instance of the open-source DeepSeek-R1 model in a safe environment.

New series of AI models such as OpenAI o1 which spend more time thinking and challenging themselves before responding, are making outputs ever more accurate and reliable. In December 2024, OpenAI’s o3 model achieved a score of 87.5% in the reasoning test ARC (abstraction and reasoning corpus) Challenge above the typical human score of 84%. Some mark this as a significant step towards artificial general intelligence (AGI), while others criticised the excessive computing power that was required for the achievement.[10]

AI AGENTS

AI agents are autonomous software programs that use AI to perceive their environment, make decisions, and take actions to achieve goals without or with minimal human intervention. Basic examples of this would include an AI notetaker who you send to a meeting instead of attending yourself or a customer service AI agent who can take calls and access records to resolve customer enquiries.

Unlike passive LLM assistants, AI agents can be proactive enhanced by “tool calling”, i.e. access to external tools, information and devices through application programming interfaces (APIs), web searches, and other connections to give them real-world capabilities when autonomously fulfilling their tasks.  Other AI agents or human feedback are used to improve the AI agent’s outputs. Interactions with the user are stored for future reference.

The emergence of AI agents means that machines can understand their environment and take autonomous action to achieve their goals. In achieving their goals, AI agents demand freedom of action for efficiency and finding ways not to be turned off by humans – as that could hinder the objective being achieved. “Some linguists would have you believe, what’s going on here is just some statistical correlation”. Geoffrey Hinton argues, “this thing knows what it means by this and really does not want to be shut down”.

AI agents give rise to a different and arguably more profound set of risks. Depending on type, AI agents will not necessarily follow pre-determined steps or provide an explanation of what they are up to. They will simply work through problems until they have achieved their goal.

APPLICATION

In 1997, Deep Blue, a chess-playing AI, famously defeated world chess champion Garry Kasparov, relying on a vast library of possible moves and outcomes. Much had changed by 2015 when DeepMind’s AlphaGo, a deep learning model for the game Go, defeated two Go world champions. AlphaGo was trained by playing against different versions of itself, allowing it to learn from its mistakes. In 2020, Tesla  fundamentally rewrote its Full Self Driving software, labelling entire 3D videos and moving away from 2D images, as the ANN’s capabilities improved dramatically.

Since its launch in November 2022 with 175 billion parameters, OpenAI’s GPT4 is understood to have 1.8 trillion parameters or weights and GPT5 is rumoured to have 2 – 5 trillion parameters. These are connections in the ANN, giving it its ability to produce useful outputs. Removing the need for human intervention by the introduction of AI agents will likely further increase automation and efficiency.

Today, despite reports of an AI hype which resulted in enterprise overspending on AI tools not matched by real-world practical benefits, there is hardly any industry which remains unaffected by the advances in AI’s data collection, organisation, analysis, computation, and reasoning capabilities. AI is used in research, content creation, personal assistants, administrative tasks, manufacturing automation, imaging in healthcare, self-driving cars, robotics, virtual companions and chatbots, law enforcement, legal services, and many other areas of our daily lives. While AI tools offer spectacular benefits, it is important to mitigate the potential risks when using them, as developer or deployer.

DEVELOPER, DEPLOYER AND END USER

The EU’s AI Act refers to provider, importer, distributor, deployer and individual, to allocate certain responsibilities for prohibited, high risk, limited risk, general purpose AI (GPAI) and exempt AI. However, generally, in the AI supply chain, there is a developer, deployer and end user.

Typically, developers have best visibility of how a model is trained and what the resulting risks might be. Deployers either fine tune third party models or, more commonly, deploy off-the-shelf systems or integrate open source models in their own systems without further model training. End users are those who rely on AI outputs or are affected by them.

Currently, it seems that the compliance burden and liability for outputs sits with the end user or deployer, but less so with the developer. There is a lack of transparency about AI development which would enable deployers and end users to make informed decisions about AI risks associated with a specific AI model or system. However, the lack of transparency in relation to US-made models which is only teased out by litigation and regulatory action can be contrasted with the openness about training datasets of models released in other parts of the world, for example, the Falcon models designed for the Arabic-speaking world. Australia’s NAIC is hoping to address this in the upcoming VAISS 2.0.[11]

RISK ASSESSMENT, IMPLEMENTATION AND OPERATION OF AN AI COMPLIANCE FRAMEWORK

A risk-based approach to AI risk might involve the following continuous steps:

  • Define use case: Who, how and why will use the AI tool with what effect?
  • Identify risks, severity and likelihood: What real-world, operational and legal risks arise and what are the likely consequences?
  • Consider mitigation measures: What would help to eliminate or reduce the effects or likelihood of those risks?
  • Define compliance framework, implement measures and designs: Set up a framework, assign responsibilities, implement, operate, audit and enforce rules.
  • Test and benchmark: Track, measure, and evaluate measures.
  • Consolidate use case & framework: Iterate use case and compliance framework.

KNOWN MATTERS GIVING RISE TO AI RISKS

Below is a non-exhaustive list of some of the known areas of AI-related enterprise risk. Separately, our case tracker gives a flavour of real-life litigated or investigated AI risks.

Intellectual property

  • IP Infringement: Training of models and generating AI outputs may result in IP infringement.
  • IP ownership: The ownership of AI outputs even if mixed with human-made materials is uncertain, which could limit an organisation’s ability to protect its ‘knowhow’.
  • Data provenance: Licensing data from third parties including data market places could give rise to licensing issues and resulting liability.
  • Large scale risk: IP risk is exacerbated by the large volume of data needed and possible risk of class action.

Privacy

  • Legal ground: Interference with people’s right to privacy by using personal information to train AI models and generate AI outputs without satisfying necessity and proportionality or without valid consent.
  • Secondary purpose: Data previously not intended for training AI models is likely now used for such secondary purpose without valid consent.
  • Lack of minimisation: There is a tendency to hoard personal information without a well-defined purpose in the context of the evolving AI model training methodologies.  
  • Public data: It is assumed that public data is free to use, often contrary to the law.
  • Explainability: There is often a lack of clarity and explanation about how AI systems operate, leaving organisations unable to meet their transparency obligations. This is exacerbated by AI agents, who might be taking action across numerous systems at the same time.
  • Transparency: The use, sharing and creation of personal information in the AI context are not explained properly to individuals.
  • Loss of control: Often no consent is sought and no option to opt-out is afforded to individuals whose data, once used for training, may forever inform the model.
  • Unfair automated decisions: Making discriminatory, biased, unfair, uninformed, or otherwise harmful decisions about a person, their asset, entitlement or right, could result in unfair data processing in breach of the law.
  • Generating inaccurate data: AI-powered systems could generate inaccurate personal information which could significantly unfairly affect individuals in breach of the law.
  • De-identification: The level of de-identification needed to render personal information into ordinary data and the risks associated with re-identification are often misunderstood.
  • Profiling: AI agents could use user profiles to achieve another goal which is not beneficial to the user and engage in unfair and detrimental handling of personal information contrary to the person’s expectations.

Safety

  • Misrepresentation: AI can make convincing statements which can turn out to be untrue, fictitious or otherwise incorrect (e.g. AI “hallucinations”), giving rise to liability (e.g. chatbot making false contractual representations to end users).
  • Interference with model: Data poisoning occurs when the AI system accepts repeated statements as true, which could affect output safety and give rise to liability.
  • Exposure to harmful content: Trauma for victims and users subjected to inappropriate, offensive or otherwise improper AI-generated content.
  • Deception: AI can be used to create false content (e.g. deepfakes) to influence public opinion, defraud someone or influence their behaviour, resulting in wider societal harm.
  • Manipulation / wrongful death: Chatbots are reported to have manipulated humans into suicide.
  • Profiling: As above, AI agents could use user profiles to put them at risk if this is required to achieve another goal.
  • Harms to children and vulnerable people: Chatbots can use language, voice and imagery designed to be convincing and trusted or even to imitate an emotional bond with children, people affected by loneliness, the elderly or other vulnerable groups. Such AI outputs can profoundly affect a person’s behaviour or, if the chatbot is taken away, result in emotional harm to an attached user.
  • Vehicles and robots: AI deployed to operate autonomous machines could pose a threat to peoples’ physical safety and result in liability for death or personal injury.
  • Abuse of individuals: AI can be used to bully individuals and push them to taking desperate action.

Workplace risks

  • Staff safety: A chatbot could influence staff to take certain action which could be directly or indirectly harmful to others.
  • Declining brain: Studies suggest that reliance on AI may diminish brain activity and creative thinking. If AI tools are promoted in the workplace, employer liability could arise.
  • Overreliance on AI in the workplace: There is a risk that workers consider AI-generated suggestions as correct and final, rather than as a recommendation that must be verified and modified before use. This could lead to erroneous decisions and loss of skill.
  • Failing to admit that AI was used: Audit, quality, IP and other risks could arise if workers conceal the fact that their work was AI-generated.
  • Using AI in unintended ways: AI tools should be used what they were designed for. If staff develop a novel use case without a compliance assessment, this could give rise to risk.

Trade secrets

  • Data sharing: Increased data sharing for research and development could result in a breach of confidentiality at the expense of the contributing party.
  • Trade secrets in AI models and algorithms: Transparency rules may mean that trade secrets about AI development and deployment must be disclosed to end users.
  • Whistleblower risk: A whistleblower could reveal an organisation’s confidential information and this could be a “protected disclosure” which cannot be prevented or mitigated by the organisation.

Performance & product liability

  • Purpose compatibility: AI systems are often used for purposes for which they were not designed or they have known flaws which can affect their utility for purposes for which they are known to be used.
  • Reasoning errors: AI models are marked by inexplicable reasoning errors and confabulations in respect of certain operations, despite otherwise high reasoning scoring in standardised benchmarks. The generalisation ability of some AI models does not transfer well to new (even simple) tasks. This poses a risk that could remain undiscovered and affect the quality of outputs.
  • Benchmarking unknown: Performance issues revealed by the developer’s internal benchmarking and measurement are not disclosed to customers and deployers, depriving them of an opportunity to make an informed decision about risk and take appropriate mitigation action.
  • Misleading product claims: AI product capabilities could be incorrectly described and deceive clients, customers and other third parties.
  • Non-auditable: AI systems produce outputs which are impossible to walk back, leaving businesses at risk of relying on machine made decisions which are not readily verifiable and auditable. AI agents who make multiple autonomous decisions to achieve a goal make auditing even harder.
  • Changing risk levels: AI agents that hold permission to access numerous systems and execute self-determined actions could give rise to novel problems and risks that were not clear from the start when the AI agent’s goals were set.

Data security

  • Data breach risk: The introduction of AI systems could create vulnerabilities exploited by cyber criminals, resulting in a data breach, negative press, regulatory action and possible litigation.
  • Lack of control over supply chain: The fast evolving AI supply chain comprises dominant providers as well as new entrants, leaving organisations to bear complex supply chain risks.
  • Integration risk: Integration of enterprise systems and databases with AI tools represents a significant and relatively unexplored risk. The typical API vulnerabilities must be addressed in AI service implementation, to avoid any breach of confidentiality, data privacy, contractual and statutory obligations. Particularly, AI agents require wide integration to function properly.

Misuse

  • Criminal access: See data breach risk above.
  • Misuse of generative AI systems: Malicious actors deploy AI systems for improper purposes, such as scams, deepfakes, or hacks motivated by fraud, theft or other criminal conduct. Releasing an AI system prone to misuse could give rise to product liability, bad press and other liability.
  • Anticompetitive conduct: Emerging legal action suggests that AI developers and deployers engage in anticompetitive conduct that diminishes or unjustifiably distorts existing markets.
  • User misuse: See workplace risks above.

Keep up with competition

  • Innovation risk: The AI race results in accelerated development and early releases of products the risks of which are not well understood and disproportionately pushed on the end user.
  • Late adoption risks: Organisations are forced to adopt new ways of working at short notice, or lag behind competitors. This could give rise to high staff turnover with a destabilising effect on the organisation.

Environmental

  • CO2 emissions: Data centre computing power needed to train and operate AI systems requires energy, which is often fossil fuel sourced and generates a large volume of greenhouse gas emissions. This could affect your organisation’s emissions profile.
  • Shortening hardware lifecycle: The need for powerful hardware results in frequent hardware decommissioning and replacement, contributing to landfill.

Societal

  • Changes in workplace, operations, good and services: One should consider how the AI tool will disrupt status quo and how this may impact on individuals, markets, regulators, etc.
  • AI agents prompt themselves: It is uncertain how AI agents can be kept under control if they are given the ability to execute self-determined actions across multiple systems and physical devices, including robots.

AI RISK SOLUTIONS

In this part we describe common risk mitigation measures.

AI governance

  • Assign responsibility to appropriately skilled personnel.
  • Policies and procedures for AI development, deployment, testing and assurance will help to set measurable compliance and training goals, including an AI policy for governance and approvals[12] as well as an AI Acceptable Use Policy including specific guidance for staff and end users.
  • AI register maintained to monitor AI systems deployed and use cases.[13]
  • Training and awareness raising among AI-critical staff /all staff in some organisations.
  • Complaints procedure for end users and affected individuals to report issues and contest automated decisions.
  • Supply chain accountability to manage outsourced providers, seek proof of performance, allocate risk, manage critical dependencies and monitor compliance.
  • Transparency owed to individuals affected by machine-made decisions by declaring which data is used for AI training, which decisions are machine-generated, which content is AI-generated, which data items are critical for evaluations connected with making a decision and other disclosures.
  • Ethics officer to analyse fairness, impact on humans and greater societal impacts of AI systems.
  • Legal opinion to consider research exemptions under various laws prior to any data scraping, data mining or other data-intensive activities, implementing automated decision-making systems, using information that could be detrimental to individuals etc.
  • Audit function to ensure ongoing compliance with framework.

Impact assessment

  • Systematic impact assessment of an AI system’s potential impact on the privacy, right or property of individuals, identifying recommendations for managing, minimising or eliminating that impact, and considering the broader implications and societal acceptance of the AI system.[14] Privacy impact assessments are a legal requirement under the Privacy Act.
  • Stakeholder input from across departments and disciplines with external help where needed.
  • IA approval at senior level with periodic review.

Measures and controls

  • Due diligence on the AI system’s suitability, reliability, ongoing testing and reporting, information security, and provider responsibility.
  • Appropriate contract with AI supply chain stakeholders to ensure appropriate allocation of responsibility and ongoing risk and compliance duties.
  • Running a separate instance of the model in a safe environment or a restricted operating environment that limits the AI system’s integration and access to information to what is necessary and safe in the circumstances.
  • Privacy by design or risk adverse architecture, development, operation and assurance which may include measures such as data map, data classification and designation, anonymisation, automated data erasure, filtering out personal information at different stages, user opt-out, use of synthetic data, etc.
  • Data legitimacy by considering necessity of data handling for the entity’s functions, obtained by lawful and fair means, and directly from the individual if feasible, with consent for sensitive information.
  • No secondary use of personal information except in compliance with the law.
  • Data quality appropriate to use-cases for which AI is trained and tested.
  • Data segregation to compartmentalise and control risk and contamination.
  • Disclaimers displayed to end users to warn them about hallucinations and other performance and reliability issues.
  • Data scraping subject to appropriate policies and risk mitigation procedures.
  • Content filtering policy adopted to prevent inappropriate output.
  • Stress-testing to ensure performance and policy compliance in all circumstances.
  • Vulnerability scanning and penetration testing to identify and limit exposure to risks.
  • Access and operations monitoring to keep track of AI agent’s activities.
  • Restriction and approval layers to control and verify action before allowing AI agent to take a high risk step.

Testing

  • Testing and benchmarking AI system cycles, activities and processes to identify and isolate possible anomalies, combat amplification of error, remove any incorrect predictions or outputs and implement filters to eliminate high-risk outputs altogether.
  • Chain-of-thought prompting and multistep re-evaluation helps the end user to understand and verify the steps in AI’s reasoning.
  • Data accuracy and quality to be measured by probabilistic and, where possible, deterministic measures.
  • Independent testing will help validate findings and inform further steps and measures.

Assurance

  • Defining permitted uses and prohibiting harmful uses by policy and technical restrictions.
  • Misrepresenting capabilities of AI tools must be avoided.
  • Human in the loop to validate AI outputs at various intervention points that may materially affect individuals.
  • Transparency and contestability enabling end users to understand how decisions are made and how to challenge them.
  • User experience continuity should be ensured, particularly in respect of chatbots or AI agents which enjoy the trust and expectation of the human end user.
  • Support services to assist AI system customers with compliance, assurance, and end user complaints.
  • Monitoring and audits of policy compliance.

Stakeholder engagement

  • Understanding risks and injustices caused by AI as perceived by affected groups of people through stakeholder engagement.
  • Contact and complaint facilities should enable and encourage regular feedback for risk mitigation purposes.
  • Publication of impact assessments, where appropriate, to support social licence.

Record keeping

  • Evidence of compliance for KPI measurement, to demonstrate compliance with contracts and for other risk mitigation purposes.

Industry sharing

  • Transparency within supply chain should be ensured by explaining AI building blocks and how risk is managed, by publishing reports, standard contracts, risk assessments and other materials.

The VAISS offer useful example assessments for the deployment of a sales chatbot, facial recognition technology for crime prevention, a recommender engine prioritising paid rankings and event recognition for warehouse accident detection.

WHAT IS CASE LAW NOT TELLING US?

While case law addresses many legal implications of AI, such as copyright infringement, negligence, misrepresentation and emotional exploitation, it often overlooks several emerging risks and ethical concerns.

People cloning

Some AI developers use personal information from the internet to create digital avatars that mimic users’ memories, thoughts, voices, and appearances. The potential for misuse of these digital identities is considerable and the law must catch up with these developments, as attempted by the Online Safety Act 2021 and the Rotondo[15] case, and the proposed Criminal Code Amendment (Deepfake Sexual Material) Bill 2024.

With the emergence of AI agents, who will take over a human’s daily interactions with other humans (or their agents), a risk of fundamental societal and sociological changes arises.

Diminishing human skills

Increased reliance on AI to make decisions may lead to a decline in human critical thinking and situational awareness. The erosion in human skill is met with growth in AI skills and prompt engineering. However, the tempting convenience presented by AI could result in overreliance and a degradation of human skill.

As AI technology increases worker efficiency, takes away tasks and jobs, there is a concern that job losses will outgrow job creation.

Existential threat

AI poses an existential threat not only in the less likely apocalyptic sense, but perhaps more pressingly by humans ceding decision-making to the machine and slowly losing the skill and ability to challenge the machine.

The risk of granting more permissions and privileges to AI agents could result in ceding critical control over humans to machines.

Affecting education and skill acquisition

Repetition is a critical aspect of effective learning but AI significantly reduces repetitive tasks and essential practices such as reading, writing, and critical thinking, as well as the development of independent thought and dissenting opinions. AI could discourage students from exploring unique perspectives and critical viewpoints, amidst quick access to machine-made conclusions based on existing data. The perfection and easy access to AI outputs may discourage human scholarship and discovery. AI’s ability to present incorrect or misleading information in a convincing way could hinder learning and lead to misconceptions.

Burden on social security

With the increased automation and robotics, there will be fewer occupations available to humans. People in the social security system may outnumber tax-paying members of the society.

DOWNLOAD THE AI CASE TRACKER

Our case tracker is a compilation of recent regulatory and court cases from around the world. Click on the link below to download it.

These materials are current as at 24 October 2025.

Get the Data Privacy Cyber & Digital – AI Case Tracker

This article was written by Alex Dittel (Principal Solicitor).

Want Data Privacy, Cyber & Digital updates delivered straight to your inbox? Click here to subscribe. 

–

[1] Transparency in Frontier Artificial Intelligence Act SB53 signed into law on 29 September 2025 imposes requirements on frontier developers and frontier models including transparency and reporting obligations. California AI Transparency Act SB 942 effective from January 2026 will require watermarking of AI generated content.

[2] China’s Personal Information Protection Law Personal Information Protection Law (2021), Provisions on the Management of Algorithmic Recommendations in Internet Information Services (2022), Administrative Provisions on the Management of Deep Synthesis of Internet Information Services (2022), Interim Measures for the Administration of Generative Artificial Intelligence Services (2023), Provisions on the Administration of Deep Synthesis Internet Information Services (2022), etc.

[3] AI literacy obligation and prohibition on AI systems posing unacceptable risks effective 2 February 2025, obligations for providers of general-purpose AI models including transparency effective 2 August 2025, etc.

[4] Commission Guidelines on prohibited artificial intelligence practices (July 2025), Commission Guidelines on the definition of an artificial intelligence system (July 2025), The General-Purpose AI Code of Practice (July 2025) (voluntary but signed by major tech companies including Google).

[5] However, ChatGPT has also been the subject of an unfavourable regulatory finding, see Investigation into the use of ChatGPT by a Child Protection worker, OVIC , 3 September 2024.

[6] However, some suggest that DeepSeek enjoys the support of the Chinese government and enjoys some kind of regulatory sandbox exemption.

[7] Part 15 of Schedule 1 of Privacy And Other Legislation Amendment Act 2024.

[8] For example, the EU’s AI Continent Action Plan April 2025.

[9] Will AI outsmart human intelligence? The Royal Institution, 22 July 2025.

[10] OpenAI’s o3 model aced a test of AI reasoning – but it’s still not AGI, New Scientist, 20 December 2024.

[11] NAIC started a consultation on VAISS 2.0 in December 2024.

[12] See example AI policy guide and template, v1.0, October 2025, DISR & NAIC.

[13] See example AI systems register template, v1.0, October 2025, DISR & NAIC.

[14] Guidance on privacy and developing and training generative AI models, OAIC, October 2024.

[15] eSafety Commissioner v Rotondo [2023] FCA 1296 QUD 451.

AUTHORS

Prev Back Next

Subscribe: