110
Apple wants AI to run directly on its hardware instead of in the cloud
(arstechnica.com)
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
this post was submitted on 22 Dec 2023
110 points (100.0% liked)
Technology
37554 readers
134 users here now
A nice place to discuss rumors, happenings, innovations, and challenges in the technology sphere. We also welcome discussions on the intersections of technology and society. If it’s technological news or discussion of technology, it probably belongs here.
Remember the overriding ethos on Beehaw: Be(e) Nice. Each user you encounter here is a person, and should be treated with kindness (even if they’re wrong, or use a Linux distro you don’t like). Personal attacks will not be tolerated.
Subcommunities on Beehaw:
This community's icon was made by Aaron Schneider, under the CC-BY-NC-SA 4.0 license.
founded 2 years ago
MODERATORS
CPUs can have special hardware accelerators for stuff like this, and you'd be surprised how powerful our little phone CPUs are and how optimized stuff like this can become.
Awesome, thanks for the insight.
I'm showing my age here, but much like we had math coprocessors running beside the 286 and 386 gen CPUs to take on floating point operations; then graphics cards offloaded geometry-based math operations to GPU's - are we looking at AI-style die or chips to specifically work on AI functions?
Excuse my oversimplification, this isn't my field of expertise!
Well, your not too off. Like ASICs are made for mining cryptocurrency. Specialized processing designed for specific computations. This indeed make it's efficiency greater than a general purpose CPU.
They already have dedicated hardware they call the neural engine, and use for coreML, ARKit, some of the magic they do to turn terrible sensors and lenses into passable images, etc. There's a lot of processing that already happens on your device. Being able to search your images by subject might be something Google does too, but Apple does it locally.
So my guess is they'll just adjust the architecture of the neural engine to accommodate any new requirements, rather than adding a "new core". But it's kind of all semantics. There will be new hardware components and intercommunication at a low level.
Apple added (a while back) what they call a “Neural Engine,” which is hardware dedicated to efficient execution of ML workloads.
https://en.m.wikipedia.org/wiki/Apple_A11
They have been refining it ever since. I would not be surprised if they made advancements in both the hardware and software used for local GAI.
And Google did the same with the Tensor Processor Unit in the Pixels.
Yes!
not a dedicated chip per se, the trend is to build it directly into the SoC (mobile devices) or the dedicated GPU
Yup, technology and especially phones have come a disgustingly long way in such a short amount of time. Running AI efficiently on them is the next step, one that we probably won't struggle with too much.