‘Skate to where the puck is going…’ How Apple plans a chip that stays relevant for years

Bob Borchers, Vice-President Worldwide Product Marketing at Apple, gets a bit philosophical, though with an ice hockey quote, to explain how the company manages to design a processor that remains relevant even after so many years. He tells indianexpress.com that they use this quote a lot, “because probably one of the most important parts of our job is to try and figure out where it’s going so we can anticipate and build the tools and capabilities to get there”.

Tim Millet, Vice-President Hardware Technologies, elaborates that at Apple they do their best to make sure that they are “not just provisioning the applications that are available today, but anticipating where applications are going to be in a couple of years”. As head of Apple’s Platform Architecture team, he would know this best.

“So our best-in-class CPU is one of the most impactful ways that we can future-proof the device because we know that when we are improving year over year, we are going to provide more headroom for applications to migrate forward,” Millet explains to indianexpress.com.

Apple Bionic 15 chipset and its key features. (Image credit: Apple)

He says the same goes for the GPU as well. “We want to make sure there’s enough GPU in there to satisfy our internal needs for the features that we build in and third-parties. There is no end to the amount of compute that video gamers or the professional media processing applications want, and so we do our best to pack in as much as we can at every generation.”

Apple has launched its latest iPhone 13 series powered by the new A15 Bionic processor, which powers a lot of the new features especially with the ProMotion adaptive screen refresh rate and intuitive Cinematic mode in the video camera even as it improves battery life over previous generations.

Millet says a lot of future-proofing is now also going into the neural engine, where “you kind of see this explosion of machine learning that’s happening. This renaissance is driving, lots and lots of new applications, new developers”. He adds: “We make sure when we deliver it we deliver it with our APIs and our Core ML so that it’s available for application developers and we improve that every year. But we also know that new algorithms pop up every day, and so we make sure that our CPU and GPU are there to provide flexibility, just in case the neural engine isn’t exactly the right engine.”

Borchers says the best way to be “good stewards of the Earth is to make sure that your products last as long as they possibly can whether it’s from a hardware perspective or a software perspective.”

On the A15 Bionic processor, Borchers underlines how people tend to get “lost in the specifications” and lose sight of the fact that it is this chip that is allowing Cinematic video, Centre Stage, and such other new features.

An Apple iPhone 13 Pro is displayed on its first day of sale, in New York, Friday, Sept. 24, 2021. (AP Photo/Richard Drew)

“Because we are so focused on Performance Per Watt and all these pieces working together in tandem, we can have these advanced capabilities that consumers can use and have great battery life and efficiency,” he says, adding that it all happens because they “work together as one team and design for a single product and we have a singular vision in mind”.

Millet takes this thinking to an extreme. “We are all about the product. I talked to my team about that, I said look we are not building chips to win benchmarks, we are building chips to enable capabilities,” he says.

For years, the conversations around chips have been about pushing the numbers and winning overclocking challenges, and pumping in more power. But at Apple user experience seems to be the benchmark they are gunning for.

Millet explains how much hard work goes into getting this experience right. “We work with our camera technology team years in advance, lining up the compute capabilities with the camera hardware and the software that we want to run on that new computational photography domain… when we deliver something in the phone our part of it on the chip side is really just to enable all the fundamental compute and the efficiency,” he says, adding that it is also about being able to do this in a way that it extends battery life or adds more capability to the GPU to ensure a feature such as ProMotion works.

With the iPhone 13 Pro series, Apple has enabled ProMotion adaptive refresh rates on the display with the phone deciding what refresh rate — between 10Hz and 120Hz — will be best for the content on the screen. Apple has made this fully automatic with no manual over-ride and this helps save on the battery life too on the new phones even with features that are technically going to use up more power.

Cinematic video mode is seen on the iPhone 13 Pro max’s camera. (Image credit: Nandagopal Rajan/Indian Express)

So does Apple plan features for the capability of the next processor or does the processor get developed to enable features planned for the next iPhone?

“For the most part, we are working in line with our system team and software teams and industrial design teams and human interfaces team to deliver an experience… it’s all about that so we are single-mindedly focused on that,” explains Miller. “But we do technology development on the side. We do keep our eyes on the lookout for long-term investments, and the neural engine’s a good example of that.”

Millet remembers that the “renaissance in machine learning” started around 2012. ”We felt like the right way to introduce neural compute was to do it in concert with the camera processor and so we built it right in. Once we saw the impact it was having on photography, and the broader impact it’s having on the industry, we realised it is the time to go big and pull it out and make it a standalone engine. That’s when the Apple neural engine was really introduced so that we could do it, we could use it for our internal applications for photography and for other things we do internally like FaceID.”

“We don’t wait,” says Millet, “for someone to come and give us a list of things to do. We are busy in the background, looking for obvious ways that we can enhance and enable the capabilities of the device, but it’s never done in isolation.” He adds: “We didn’t invent the neural engine and wonder how it was going to be used. The concept starts years in advance. We work closely with our software team to make sure it’s all going to be lined up, make sure the camera system, all the APIs are available so that when it comes out, it’s a complete story, it’s a complete solution.”

Asked about how Apple plans for the same processor running different devices with different capabilities, Millet explains that a part of this is about weighing the benefits of energy efficiency. “Putting a much, much bigger GPU in a device that doesn’t need it doesn’t help the battery life. So what we do with those devices is to make sure that we are putting in the right size GPU to enable the experiences that are appropriate for that device, that screen size and the software that runs on it, and the battery life. And then for the bigger devices, the Pro devices with the bigger displays and the more complex camera systems, we know we are going to need a little bit of additional performance, so that’s where we can extend the GPU,” says Millet, adding that it is his team’s job to build that scalability story so that the software team doesn’t see something different, but something very familiar.

Borchers elaborates on how this plays out in the different devices, like the new iPad mini and the iPhone 13 Pro series both of which have the five-crore GPU. “On the iPhone that enabled Cinematic mode, whereas, on the iPad Mini, it enables CentreStage which is its own kind of magical video experience using the same underlying characteristics.”

Apple iPhone 13 Pro Max is seen in this photo. (Image credit: Nandagopal Rajan/Indian Express)

On how smartphones have reached a point where they can now offer 4K video recording without really heating up like before, Millet explains: “It is hard. There is nothing easy about delivering a 4K Cinematic video for an extended period of time, and we absolutely are not interested in delivering gimmicks with demonstration modes that are only supportable for a couple of seconds. Our team agonises over micro Joules of energy that go into all these transactions. We are constantly modelling power, working with our technology teams to make sure we have the best transistors for those critical applications like the camera processor, because that’s one of our toughest internal targets, to be able to extend the feature set and the length of time that people can use these amazing features on the phone, and it comes with generations and generations of hard work.”

Millet underlines that it all comes down to efficiency, “because we know in a constrained enclosure like the phone, efficiency translates to performance. If it could be more efficient, you are going to get more performance out of the device, and that’s going to allow you to enable all these things and the efficiency does absolutely translate into battery life.”

In fact, the extra battery life, despite the new features, is being touted as one of the USPs of the new iPhone 13 series. Millet explains how the A15 does this with the example of the ProMotion display.

“We recognise the natural frame rate and the update rate of the application. When there’s an application that benefits from 120Hz, we enable it. When we have an application that is not, there’s no point. If someone’s staring at an email, refreshing the display of 120 Hz is a complete waste of energy,” explains Millet. This is when the processor brings the refresh rate way down to 10 Hertz and turns off the GPU so that it’s not leaking energy.

Millet is very proud of how his team “does nothing really well”, sitting there waiting. “And when I say nothing it is you are waiting for a hair trigger, waiting for the user to touch the display, scroll, do something that’s going to trigger a burst of activity, and we want to make sure we jump on it and that’s when we unleash the power of the A15 Bionic to really make it feel like the device never left 120Hz when in fact it is sitting at a much, much-reduced energy level.”

Millet thinks it is pointless to talk about the capacity of batteries. “We spend a lot of time looking at what we call all-day battery life, that’s what really matters to people it’s not the capacity of the batteries,” he says, underlining how it is really about how much useful work do we get out of the battery.

Borchers chips in: “The reason that we talk about all-day battery life as opposed to milliamp hours is that talking about milliamp hours is kind of like talking about the size of the tank of your car without talking about how many miles per gallon it gives. So I have got a 20-gallon tank, but depending on how I provide the efficiency everything else may not get me to where I need to go. We focus on making sure that the entire system, including the tank, works together to give our customers as best as we possibly can, that all-day battery life experience and it’s based on understanding what people do with their phones over the course of the day.

Borchers says they achieved better battery efficiency this year by increasing the size of the battery a bit and doing some smart things like the ProMotion display and efficiency of the A15 Bionic. “It’s a system-wide approach.”

Millet concludes: “Why would we make the gas tank bigger than it needs to be to deliver that all-day battery experience? Why do we want to make a heavier device? Why don’t we make a healthier device? We want to do it the right way, the Apple way, which is to really focus on delivering that range that people want, without compromise.”