Aardvark Daily

Please visit the sponsor!

AI, the new attack vector

We are all told that AI is going to change the world and I don't doubt that for one minute.

Increasingly, businesses, governments and individuals are placing their trust, their faith and a reliance on AI that must be tempered with caution -- if the events of this week are anything to go by.

What am I talking about?

Well just ask an unknown number of Instagram users who have discovered, to their cost, that AI has no conscience and often no ability to tell right from wrong.

According to media reports, hackers were able to hijack an unknown number of Instagram accounts by using the AI system run by parent company Meta.

Was this done using some super-sophisticated back door or crazy exploit involving intricate computer code and other advanced techniques?

No.

These hackers simply used a VPN to spoof their location and then asked Meta AI to change the email address associated with the target accounts to one that they had control of.

It beggars belief but the Meta AI did exactly as was requested, without any attempt to authenticate the credentials of the hackers. Those hackers were then free to request a password reset -- which was subsiquently sent to that new email address -- and thus the account was compromised.

Seriously... how simple was that?

Obviously there are some pretty important questions to be asked in the wake of this.

Firstly, why does *any* public-facing AI system have the power to change critical data associated with any user's account?

Secondly, why was their no attempt made to verify the credentials of the person interacting with the AI? Even a simple email to the old account requesting a confirmation before enabling the change to a new account would have been enough to invalidate this attack.

Thirdly, why weren't two-factor authentication logins mandatory on these accounts? It's worth noting that only accounts without TFA were compromised by this attack.

Herein lies the problem with AI... we can't always trust it to do what would appear obvious to us.

Its "learning" is done from such a massive trove of data that there are bound to be countless situations where bad actors are able to exploit weaknesses "learned" from that training data. We simply have no way of proving that it is invulnerable to such exploits so the risk will always remain, regardless of how many "guardrails" and "safety barriers" we try to build in.

This is why smart people don't ask AI to do stuff for them. Smart people still do the hard work themselves and simply request AI to evaluate/validate the conclusions that they have reached.

I'm using AI more than I thought I would but every single day I find myself encountering halucinations, bad reasoning and erroneous conclusions being offered up by LLMs that appear 100 percent confident in what they're saying. When challenged, these LLMs inevitably apologise and concede that they were wrong. Every time this happens it further reinforces that only a fool will take the output of an LLM as gospel -- but sadly, many people do because "AI is great", or so we're told.

I shudder to think how, in a few short years, we may be reading regular reports of the use of AI as a vector for launching ransomware or other attacks that will rake in rich rewards for bad actors who are prepared to find the flaws.

The utter irony is that these bad actors may actually be using their own AI systems to find those flaws in AI systems used by others so that they can launch those attacks.

AI is a great tool but it should not be used as a crutch or a drug that creates unsafe levels of dependency.

Carpe Diem folks!

Please visit the sponsor!

Here is a PERMANENT link to this column

Rank This Aardvark Page

Change Font