Aardvark Daily

New Zealand's longest-running online daily news and commentary publication, now in its 25th year. The opinion pieces presented here are not purported to be fact but reasonable effort is made to ensure accuracy.

Content copyright © 1995 - 2019 to Bruce Simpson (aka Aardvark), the logo was kindly created for Aardvark Daily by the folks at

Please visit the sponsor!
Please visit the sponsor!

AI is now on steroids

12 Jun 2024

Yes, it's AI (artificial intelligence) time again.

Right now, large language models (LLMs) are forms of AI that require huge amounts of computing resources. In fact, the estimates are that if AI adoption continues at its present rate, a significant amount of the energy mankind presently uses will be going into the systems on which these LLMs run.

This tech also requires the use of expensive hardware such as AI processors or repurposed GPUs, due to the extraordinary number of calculations that must be performed at extremely high speed.

However, all of this may be about to change, at least a little bit.

On of the most common and numerically burdensome parts of an LLM's processing is matrix multiplication (MatMul).

A paper published by a group of researchers claims to have come up with a way to produce LLMs without the burden of MatMul operations and, if this is so, the cost, energy requirements and speed of LLMs could be about to undergo a significant change.

According to the paper, training speeds can be hiked by over 25 percent and memory requirements, after further optimisation, could be reduced by a factor of 10.

In systems that are as large as those usually required for commercial-grade LLMs, this can represent a huge saving on hardware costs to deliver the same level of performance as MatMul-based systems.

Also, because the very complex and processor-intensive MatMul processing is avoided, researchers have been able to build a custom FPGA-based accelerator that processes billion-parameter scale models using just 13W of power, thus demonstrating the potential for "brain-like efficiency" in future lightweight LLMs.

What are the implications of all this?

Well I'm no LLM/AI specialist and I have to concede that most of the contents of this paper soar miles over my head and level of understanding but it would appear that this breakthrough could lead to faster, cheaper and more energy-efficient AI processing and that will result in more AI everywhere.

Is that a good thing?

I don't know.

Because I'm now in the second year of my eighth decade on the planet, part of me is averse to the level of change that AI is bringing to our world. Fortunately however, I am aware of this age-related bias and thus do not discount the value of this new tech.

AI is just a tool (at this stage) so whether it's a good thing or a bad thing will be entirely determined by how we actually use it.

If we recklessly apply it to everything we do then some of the outcomes will be far less than optimal. However, if we're prudent and measured in our use of AI, the benefits could be huge and may produce see unprecedented progress in the areas of science and technology during the years ahead.

For those who might want to play around with AI at an experimenter's level then it seems that a new NPU (Neural Processing Unit) has been announced for the good old Raspberry Pi. The Hailo AI kit promises to bring AI capabilities to the humbe RPi for a suitably low price and is something I may pick up so that I can become a little more familiar with our soon-to-be computer-based overlords.

Carpe Diem folks!

Please visit the sponsor!
Please visit the sponsor!

PERMALINK to this column

Rank This Aardvark Page


Change Font

Sci-Tech headlines



The EZ Battery Reconditioning scam

Beware The Alternative Energy Scammers

The Great "Run Your Car On Water" Scam


Recent Columns

Is digital ID too risky?
Around the world, governments are rolling out digital ID systems for their citizens...

Speed limiters coming to NZ soon?
I've already written columns on the strange paradox that sees us driving cars which are capable of more than twice the legal speed limit...

Arming the Karens
There is a real discussion going on in the drone community right now...

How far has the media fallen?
Sadly, I believe the mainstream media as reached a new low...

Oh no, not again
Eggs used to be a staple of the Kiwi diet...

The Taiwan kill switch
There have been some interesting articles published recently with details of exactly how Taiwan's TSMC would respond to an attack by China...

Not a light, a train!
Yesterday I wrote about how the news industry seems to have totally lost its way and is struggling to survive in the age of the internet and social media...

Breaking news: Times are changing
I was having a think the other day, about what I'd be doing right now if I wasn't already crazy-busy...

Houston, we have a problem
We all know that the space immediately around Earth is becoming increasingly filled with space junk...

The world is getting smaller
The internet has made the world a much smaller place...