Aardvark Daily

Please visit the sponsor!

AI vs Open Source

Something bad happened this week, something very bad.

If you use open source software and figure that the "openness" of the product contributes significantly to its security, you might be wrong.

The theory is that if a thousands eyes can peruse the source code, there are a thousand chances for any bugs or unsound design which could lead to vulnerabilities, will be spotted and fixed.

Whilst this does sometimes happen, the arrival of powerful AI systems has turned this strength into a weakness, as was proven last week.

Crypto currency, or at least the keys to it, can be stored in a "wallet". This is supposed to ensure that unauthorized parties can't do things with your hard-earned crypto-money, only you can buy stuff or move it around.

One popular open source wallet is the Cold Card a bit of hardware that runs open source software supposedly ensuring that your crypto stash is safe from the hands of hackers and villains.

Sounds great and this is just one of many such devices that are designed to simplify the task of keeping all those bitcoin or whatever carefully and safely stored away.

One of the big boasts of the Cold Card is the fact that its firmware (software) is all readily available on this Github repository. If you want to you can peruse it or even build it yourself from the files provided.

Unfortunately, this has been part of the problem that caused one Canadian user, Jonathan Goodman, to lose CAD$1.6 million last week.

In this X post, Jonathan relates his experiences. In that post he says:

So what went wrong? How did he lose such a staggering amount?

Well apparently some evil sod(s) used AI to scan the source code for the Cold Card software and discover that there was a vulnerability in the way it was generating random numbers -- a key part of the encryption code.

The Cold Card device is built around an STM32 microcontroller. This chip has built-in random number generation hardware, very handy for encryption-based stuff.

However, it would appear that the developer of the software was having trouble getting the hardware-based random number generator to work. The code would not compile properly because of mismatched RNG (security) settings. Rather than come up with a solution to that problem, it seems that they opted instead to use the algorithmically derived random number generator built into Python.

This made the hacker's job so much simpler because now they could rely on a repeatable set of numbers being used by the software instead of a truly random set.

Owch!

This serves as a very sobering example of why every open source developer/team should be running their entire repository through as many AI systems as they can, in the hope that they'll catch the vulnerabilities that hackers will be looking to exploit -- before they do.

I suspect there's a lot of frantic AI-based code auditing going on right now on both sides of the battle.

Carpe Diem folks!

Please visit the sponsor!

Here is a PERMANENT link to this column

Rank This Aardvark Page

Change Font