Thoughts on Google I/O 2017

There were an enormous number of announcements at this year’s Google I/O. In particular there seemed to be the most advancements in Android development announced since 2014. Rather that attempt to comprehensively summarize the event, here are some scattered thoughts.

Android:

The biggest Android-related news was clearly the surprise adoption of Kotlin. The main reason it was a surprise was because of the tone the Android team conveyed in response to requests for Kotlin support at last year’s I/O. While many team members used Kotlin, they seemed to suggest that Java was going to remain the platform language for the foreseeable future.

I’m not a programmer so my opinions on languages are invalid, but everything I’ve ever read about Kotlin makes it seem like a pretty good language. I have no idea how well it performs, though. Because Kotlin was designed for seamless Android interoperability from its inception, it can pretty much immediately replace Java for any developer now that it is officially supported in the Android SDK. There will be some gaps in tooling, but it’s as easy a developer transition as they come. The next step will be transitioning APIs to Kotlin. I don’t think many members of the Android team will miss Java. 

App lifecycles have always been a nightmare on Android. The new architecture model proposed by the Android team surely has to be an enormous improvement, but it understandably necessitates new classes. The team is notably proposing more of a reactive-style model instead of a model-view-viewmodel (MVVM) paradigm, to quote Yigit Boyar. We’ll see how that turns out.

Android O is a major performance release, probably the most significant one since 5.0 Lollipop. One of the key efforts by the Android team for O was the elimination of binder lock contention. The result is a “massive improvement in jank immediately after boot as services all start up.” Running the O beta on my Nexus 6P, I was amazed how much of an obvious and appreciable difference this makes. For a thorough description of Binder by Dianne Hackborn, see here.

Significant improvements in OS boot time are also claimed, with the Pixel smartphone starting up in half the time on O. App startup times have also been sped up by different optimizations. I suspect all those app racing videos on the internet played a roll in spurring this effort, though I would strongly caution you not to assume those videos are really representative benchmarks.

Also of major note is that Android O features an optional new rendering pipeline, which you can enable from developer settings. Nothing has been said about it, but it is based on Skia. I don’t know anything about graphics libraries, but both Android and Fuchsia have used Skia from day one so I have no idea what the new renderer entails. Perhaps it’s a Vulkan rewrite or a more fully hardware-accelerated pipeline, but I’ve found no information yet online. If anyone knows more, please reach out.

Regarding runtime improvements, ART has switched from using a mark-and-sweep algorithm to a concurrent copying garbage collector. Claimed results are less time spent reclaiming and allocating memory and lower overall usage. I know very little about garbage collection, so I wonder what the tradeoffs are. You should watch the ART presentation if you want to learn more about the new collector and the many other improvements. I do know, however, that Josh's desired simple optimization was unsurprisingly not implemented.

You may be surprised to learn that ART has also at long last added support for automatic vectorization. I won’t explain SIMD utilization here, but I may write an entire article about this topic in the future.

One nice addition that will indirectly improve performance and battery life is that the Play Developer Console will now flag apps that perform poorly in regard to wake locks, wakeups, and jank. Google also said that wake locks will be restricted in future releases of the platform, so developers be warned. These restrictions should be deeply appreciated by users.

Because Android releases are not developed in public, we still know extremely little about Project Treble. Aside from some vague high-level comments, the most specific information given was that "device-specific vendor components" have been sandboxed in some manner. Reducing the bring-up costs of Android updates for higher-end vendors like Qualcomm should be a huge help for the overall ecosystem, but I am skeptical that it will have much impact on a company like MediaTek, which has little financial incentive to provide updates in general. Treble also does not chance the fact that Linux does not have a stable driver ABI. I should point out that it sounds like the transition was a miserable technical slog for many members of the Android team, so thanks to them for their efforts.

With the announcement of Android Go, I immediately wondered if the platform's memory profile had changed. The last time there was a significant increase in RAM requirements for Android was the 5.0 release. There is no Android Compatibility Definition Document for O yet, so it is unclear if the minimum memory requirements will be changing (Section 7.6.1). Based on the ART session, however, overall memory usage should be lower in O.

Much to my surprise, graphics drivers are now updatable through the Play Store. This is not a small detail, and I suspect it was a benefit of Project Treble. Google is also now offering Android developers the equivalent of Apple’s App Slicing. Within security, tamper-resistant secure elements (ala Apple’s “secure enclave”) are now supported in O.

As anyone could predict, deep learning was a huge focus at I/O. Google’s TensorFlow happens to be the most popular deep learning library. While it has been available on Android since launch (and was later made available on iOS), Apple managed to provide a GPU-accelerated mobile framework before Google, with convolutional neural network kernels available in Metal Performance Shaders on iOS 10. The lighter-weight TensorFlow Lite was thus a big (and much needed) announcement, although developers will also at least be able to leverage vendor-specific acceleration libraries through Qualcomm’s Snapdragon Neural Processing Engine SDK. In the near future, TensorFlow Lite will leverage the new Android Neural Network API to allow developers to accelerate their AI algorithms on GPUs or DSPs.

I won’t beat around the bush — the changes in Android 5.0-9.0 have made the platform much more similar to iOS overall. I think the Android team has prioritized the right areas of improvement, even if they’re shipping fewer obvious consumer-facing features.

Everything else:

Johnny Lee confirmed that Google’s VPS (Visual Positioning Service) is marketing's branding of the combination of area learning and point clouds stored in the cloud. Standalone Daydream VR headsets use stereo visual odometry and 3D SLAM to provide positional tracking, with drift correction based on area learning. The combination of hardware and algorithms is marketed as WorldSense, which is of course based on a specialized version of Tango.

Google also showed off some amazing VR technology called Seurat, which renders extremely high-fidelity graphics on mobile in real-time via unknown means. The technology could be anything, but it isn’t magic. For similarly impressive demos, check out OTOY’s “eight-dimensional” holographic light field rendering. (Update: Seurat is indeed supposedly some form of light field rendering. This Disney Research paper was released simultaneously.)

Within deep learning, Google announced that it was working on CTC-based sequence models for natural language processing, with “a whole bunch of new implementations" coming soon.

Lastly, I was wrong about the Flutter sessions. There were numerous memes, but none of the animal variety. I apologize for the error.

"Understanding Color"

Romain Guy did not disappoint with his presentation on color at I/O. Color is so complicated that there’s no way to give a succinct introduction to it on Android in 40 minutes, but Romain did about as well as you possibly can.

I would say this is an intermediate level presentation, but if you're interested in learning about color for the first time it will probably feel like an advanced one. Don't worry about all the concepts you don't understand yet. I knew basically nothing two years ago before learning all the material in this talk. Even though color is a massive topic, the individual concepts are relatively easy to grasp. Give it a shot, and you can definitely learn all of the major points over time.

Virtual teardown of a Samsung Galaxy S8

To learn more about the silicon content of a Galaxy S8, I decided to identify many of the components in a US AT&T model. The following list is not intended to be particularly comprehensive. If any of the ICs listed below are outdated, there are likely newer revisions of these components in the phone.

Sensors:
ams TMD4906 optical module — ambient light sensor (ALS) + proximity sensor + IR LED
STMicroelectronics LSM6DSL SiP with accelerometer + gyroscope
STMicroelectronics LPS22HB barometer
Asahi Kasei Microdevices AK09916C magnetometer
Maxim Integrated MAX86907E heart rate monitor
Sealed Sensor Connector (SSC) System - maybe from TE Connectivity?
Semtech SX9320 grip sensor (an “ultra-low power capacitive Specific Absorption Rate (SAR) controller” for detection of RF attenuation by the user’s hand)

MSM8998 SoC:
ARM CoreLink MMU-500 system memory management unit
Qualcomm Adreno Venus 3XX video decoder/encoder (annoyingly marketed as a “VPU”)
Qualcomm WCD9341/9340 Aqstic (Tavil?) audio codec

Miscellaneous:
Broadcom BCM4361 Wi-Fi combo chip in a Murata module (Yes, Broadcom is still making mobile Wi-Fi combos.)
Toshiba THGBF7G9L4LBATRA 64 GB UFS 2.0 NAND + controller
Sony IMX333 and IMX320 BSI CMOS image sensors
Synaptics Namsan fingerprint sensor
Silicon Mitus SM5720 PMIC?
Maxim Integrated MAX98506 digital audio codec (DAC) + headphone amplifier (Hilariously, there are people selling this online as a USB charging controller for some reason.)
NXP PN553 NFC controller
Texas Instruments DRV2624 haptic driver
RichWave RTC6213N single-chip broadcast FM radio tuner
Xilinx XC4000 FPGA? (not sure)
Xilinx XC5000 FPGA? (not sure)
NXP PCAL6524-GPIO GPIO expander
Trustonic Trusted Execution Environment (TEE)
Possibly a Microchip USB controller?
There might be an Xceive XC2028 TV tuner in Korean GS8 models.

Used in system bring-up:
ARM CoreSight STM-500 System Trace Macrocell

Though they’re not terribly useful to publish, here are also some web benchmark results for reference:
JetStream 1.1 (Samsung browser): 75.710 +/- 0.26588
JetStream 1.1 (Chrome): 67.077 +/- 0.56466
Kraken 1.1 (Samsung browser): 2,342.0ms +/- 0.9%
Kraken 1.1 (Chrome): 2,837.6ms +/- 0.5%
Octane 2.0 (Samsung browser): 12,541
Octane 2.0 (Chrome): 11,322
WebXPRT 2015 (Samsung browser): 166 +/- 4
WebXPRT 2015 (Chrome): 158 +/- 3

Note that the beta of the Samsung browser is testing faster at the moment.

If anyone who works with silicon has any corrections, please let me know. I will have more to say on the GS8 in the future. For now I recommend following AnandTech’s technical hardware coverage.

Expectations for Google I/O 2017

The following notes are not intended to be comprehensive or predictive, but are merely some stray thoughts:

I have far fewer expectations this year than in previous years, as most of what I was anticipating was already introduced in the Android O Preview. Namely, this included finally making JobScheduler mandatory, and no longer allowing apps to register broadcast receivers for implicit broadcasts in their manifests.

I’m expecting the following topics to be discussed during the opening keynote: deep learning, deep learning, and deep learning. The machine learning sessions and office hours should be comically over capacity.

Regarding Project Treble, we have basically no details right now. I’m hoping the Android team will elaborate on it, at least during the Fireside Chat.

I was expecting the usual ART optimization improvements, and this year seems fruitful. The new garbage collector should be exciting. Josh Ho also suggested an obvious, albeit minor win to me last year: performing full ahead-of-time (AOT) compilation instead of profile-guided optimization (itself a form of AOT compilation) while on power for apps that have not run yet. The purpose would be strictly to improve first run performance and energy usage. I'm not expecting this to be mentioned, if it was even added.

Yigit Boyar is going to introduce some improvements for Android app architecture. As part of this, the new approach to managing app lifecycles was previously teased and will now be announced in detail.

Romain Guy is going to present on color. You would not believe how complicated the topic is. Color management on Android is still a work in progress, so even though not everything will ship this year, I have faith that everything (such as HDR) will fundamentally be addressed eventually.

I have no idea if anything will be announced at I/O, but at some point UHD content will be launched on Google Play. I’ve seen that the Chromium team is working on HDR support, which will be ready before color management is added to the browser. We may thus see HDR content limited to sRGB before UHD content is launched. I expect Google to push Dolby Vision. Either way, I don’t expect any Android devices will fully support HDR this year.

For anyone interested in Fuchsia, there are two talks on Flutter. I expect at least 50% of the slides to feature animal memes of some kind. Wise developers should embrace Flutter as the future of mobile development on Google’s platforms.

Lastly, I’m personally curious to learn more Android Things.

Initial thoughts on the design of the Surface Laptop

Yesterday Microsoft held its Education event, where it unveiled Windows 10 S and the Surface Laptop. While I will have more to say another time, I first want to discuss the intriguing new hardware.

With the new Surface Laptop, Microsoft seems to have substantially improved the overall quality of its hardware. I would argue that its hardware to date hasn’t been truly competitive at the high end of the market, but I would still recommend Microsoft’s devices over those from every other Windows OEM due to their superior hardware-software integration. Based on my experience with the Surface Pro 4, you are going to get a much smoother, more polished experience for criteria like trackpad performance by running software tailored by Microsoft.

The company is comparing its new laptop directly to the 13" MacBook Pro, particularly emphasizing how the Laptop weighs 0.26 pounds less than the Pro. Part of the weight difference is due to the Laptop's Alcantara surface, which I find to be the most interesting engineering decision. This material choice trades off structural rigidity and thermal dissipation efficiency for lower weight and greater comfort.

It is critical to note, though, that the Laptop only offers 15W U-series Core CPUs from Intel, while the 13" MacBook Pro also offers 28W CPUs for its more expensive configurations. In other words, the Surface Laptop has been aimed at a lower TDP, and thus lower performance, target than the 13" Pro. An eventual 15” Surface Laptop with H-series CPUs now seems likely, and many would be excited by such a product. Microsoft’s concession to its OEM partners is that it is once again only competing at the very high end of the market.

First, the bad news. The Laptop features one “full-size” USB Type-A port and one Mini DisplayPort, but no Type-C ports. At this point, Microsoft’s affinity for legacy ports and eschewing of any and all progress in connector standards is comical. Enterprise usage isn’t even a real concern, so there’s really no excuse.

I also strongly recommend not buying the base configuration with only 4GB of RAM. That makes the real starting price $1,299, in my opinion.

To be frank, there have been plenty of issues with Microsoft's hardware in the past. While it hasn't skimped on battery capacity in its recent products, the company has never shipped particularly spatially efficient computers with its Surface line. For the Surface tablets this philosophy went so far as to deliberately utilize empty space internally in order to optimize weight distribution, a design decision I found bizarre.

Previously questionable design tenets seem to have been abandoned with the Surface Laptop, however, and I think Microsoft's willingness to somewhat normalize on design has resulted in its most compelling device to date, at least on paper. And as it acknowledged on stage, that's exactly what its fans have wanted it do - just make a normal laptop.

In the grand tradition of PC vendors harmlessly creating new marketing terms for laptop audio, Microsoft has branded its speaker implementation as Omnisonics, not to be confused with ambisonics. Since it could not cut holes into the Alcantara surface, it has instead placed its two speakers behind the keyboard, radiating sound out through the keycap edges. The result is going to be mediocre audio quality from the speakers. On the plus side, the Surface Laptop uses Dolby Audio Premium DSP algorithms and meets whatever hardware requirements the license entails.

Since the Laptop is not a tablet, I can’t think of a single good reason why the aspect ratio of the display should be 3:2 instead of 16:9 or 16:10. Panos Panay stated that Microsoft wanted to maximize display area (optimizing closer to 1:1), but this strikes me as absurd for a product that never changes orientation. I’m particularly not a fan of the aspect ratio because it makes the laptop lid more likely to wobble. (This is one of the main reasons the Surface Book form factor did not work.)

Microsoft ships the most accurate displays of any Windows OEM, but while its panels have been very bright, they haven’t been competitive on efficiency. (Seriously, please never source Panasonic panels.) Based on the rest of the system design, though, I’m hopeful that the Laptop features an efficient display. Although there are not yet many Win32 apps in the Windows Store, keep in mind that there is effectively no High DPI support for Win32 apps.

Now for the good news. One of the best things about the Surface Laptop is that Microsoft has learned from the Surface Studio color calibration debacle. Unlike with the Studio, the Laptop’s display is correctly calibrated to sRGB, because there is no system-wide color management in Windows. It’s great to see Microsoft improving like this. The company continues to individually calibrate its displays, a practice going back to the Surface 3 which is a big deal and should result in great color and greyscale accuracy.

Microsoft didn’t exactly ship Kaby Lake-U early, but it can tout it as a significant advantage over Apple, as Kaby Lake provides the rough equivalent of a 300MHz CPU frequency boost over Skylake. The one battery life claim made, up to 14.5 hours of video playback, also emphasizes the advantage of Kaby Lake's addition of HEVC hardware decode, which Skylake was sorely lacking.

The entire rest of the laptop basically looks great, as do the four available colors. One thing that many people have missed is that, though, is that the GPU and DRAM frequencies for the priciest SKUs are lower than those for the 13" MacBook Pro.

(Sidenote to Microsoft: please make the technical specifications listed in your Fact Sheets more directly accessible to prospective buyers. They are important. Compare to Apple. I would also criticize Apple’s minimal consumer disclosure, mind you.)

The combination of lower clock speeds, the Alcantara, and the size of its singular fan makes me somewhat skeptical about the energy and thermal efficiency of the case design. I would expect conservative DVFS tuning. Public testing will have to wait on a review by AnandTech. Panay did weirdly seem to suggest that the keyboard feels warm during normal use.

Even though much of this article has been criticism and concerns, overall I have a very positive impression of the product. Microsoft is clearly on a roll, and the Surface Laptop appears to be its best hardware to date. While the device is particularly aimed at college students, especially since MacBooks have traditionally done well at US universities, I think it will sell well to a broad audience.

LG Display rumored to be investing in mobile OLED production

The Electronic Times recently reported that Google wants to invest ~$880 million in LG Display for future production of OLED displays. If this rumor is true, I suspect the potential strategic investment would not just be for securing displays for future Pixel devices, but for helping LG Display to seriously re-enter the mobile OLED market. The company previously sold flexible OLEDs to LG Electronics for its G Flex and G Flex 2 smartphones in 2013 and 2015, respectively, but I am not aware of any other smartphones ever using LGD OLED.

To date Samsung Display has been far ahead of everyone else in mobile OLED due to its vastly greater investment in the segment. LG Display could probably make reasonable OLED displays, though, if it had the financial incentive to make major investments in smaller panels. It has already proven its OLED capabilities with its Apple Watch displays, however difficult they were to make, and of course its leading W-OLED panels for the TV market.

This rumored change in strategy would probably be more about industrial design demands than display quality considerations. Several vendors, including Google, want to be able to compete with Samsung and Apple’s (upcoming) bleeding-edge smartphones that strongly associate OLED displays with high-end industrial design. While OLED is not necessary at all for creating a design with minimal bezels, some or even most of these vendors likely require OLED because they want to bring curved displays to market, sacrificing some image quality in the process.

To be clear, there are many advantages (and some disadvantages) to working with OLED displays over traditional LCDs from an industrial design point of view, which I won't fully enumerate here. One of the major differences, while it sounds obvious, is that OLEDs do not have LCMs (liquid crystal modules).

No matter what, it's not at all clear that investing in OLED over LCD long term would be a smart move, and most display suppliers remain skeptical of the former. OLED is better overall now, but it has strong downsides in terms of lifetime, costs (due to lower yields), and various quality deficiencies such as severe off-angle color shifting and chromatic aliasing. LCD meanwhile constitutes the lion's share of the market. microLED won't come to market for years, but it has greater potential than OLED should its production become economically feasible.

If vendors bothered to pay for high quality displays, we would see smartphones other than those from Samsung with correctly calibrated OLEDs with leading-edge quality. Perhaps a vendor or two other than Apple may one day do that. For now, given Samsung Display’s massive lead I remain skeptical that anyone can compete with it on quality over the next few years.

"Samsung introduces HDR10+ format to combat Dolby Vision"

Samsung: "Would you like another hole in your head?"
Consumer: "Yes, Samsung. Yes, I would."

We weren't necessarily going to have an HDR format war. But Samsung pushing yet another HDR standard could possibly precipitate one.

The only party that benefits from HDR10+ is Samsung.

Google retires Octane

A couple weeks ago I had caught wind of some web benchmark being killed. The first thing I did was check Octane, but the test harness was still unchanged.

Yesterday Google announced that Octane has been retired. It claims the deprecation is due to Octane being over-optimized against for years, sometimes to the detriment of real world application performance. You can still run the benchmark, but the page notes that it is no longer being maintained.

This looks really bad. Octane was far from the worst benchmark, and it had been optimized against for years by pretty much everyone anyway. It did not suddenly become outdated overnight. (This does not in any way mean that benchmarks are somehow useless or unnecessary.)

If Google is working on a new browser benchmark, great. But it's hard to believe that Google didn't actually kill Octane because Edge now beats Chrome on it, which Microsoft immediately promotes upon launching Edge in the Creators Update for Windows 10.

Why the new iPads are delayed

After Apple's rather surprising admission of fault yesterday about not updating the Mac Pro, I would like to address another area where the company happens to be blame-free. By now I have read every possible conspiracy theory under the sun about why Apple hasn't shipped [insert_your_desired_device_here]. Some of the most recent speculation is that Apple didn't announce new high-end iPads because some upcoming iPad-specific software is not ready yet. This is probably nonsense.

Anyone who follows mobile silicon knows how simple the current situation is in all likelihood: the iPads are delayed because Apple can't ship the A11X (Fusion) in sufficient volumes yet to its desired quality metrics (final clocks, et al.). More succintly, Apple has not yet introduced an A11X iPad because 10nm is a bit of a disaster. And despite what you may read, a 10nm tablet SoC would be an A11X, not an A10X.

Everything I have heard points to both Samsung Foundry and TSMC suffering very poor yields currently, in the realm of 30-40%. 10nm is just a shrink node, but it turns out that shrinking transistors is excruciatingly challenging these days because of pesky physics. And if 10nm ends up being an outright bad node, we've seen this leaky transistor nightmare before.

If you're not familiar, 10nm from Samsung Foundry and TSMC are not at all the same as Intel's 10nm. Their 10nm is actually very comparable to Intel's 14nm, with nearly equivalent density. All node names are marketing nonsense these days anyway. 14nm was particularly egregious given the reuse of 20nm's BEOL, and TSMC didn't even call it 14nm simply because "four" sounds like "death" in Mandarin; "16nm" doesn't really exist.

Internal delays are still real delays. With yesterday as an extreme exception, Apple doesn't like to talk about products until just before they're ready to ship. When it does talk in advance, even off-the-record, things can go wrong, and forward-looking statements can go unfulfilled. It doesn't suffer a negative marketing impact by keeping internal delays internal, but much more importantly it also doesn't realize the greater profits it would have if it had been able to ship on time. The A11X delay hurts its bottom line.

The situation really is probably that simple. It's not that Apple suddenly feels like it can and should wait longer between iPad refreshes (19 months now for the 12.9" iPad Pro). And despite Intel's newly constant delays, it is often not actually to blame for Your Theoretical New Mac of Choice not being released. This is a broader topic I may address another time.

On Tizen and buffer overflows

"'It may be the worst code I've ever seen,' he told Motherboard in advance of a talk about his research that he is scheduled to deliver at Kaspersky Lab's Security Analyst Summit on the island of St. Maarten on Monday. 'Everything you can do wrong there, they do it. You can see that nobody with any understanding of security looked at this code or wrote it. It's like taking an undergraduate and letting him program your software.'"

Eh, it's Tizen. I already expected this.

"One example he cites is the use of strcpy() in Tizen. 'Strcpy()' is a function for replicating data in memory. But there's a basic flaw in it whereby it fails to check if there is enough space to write the data, which can create a buffer overrun condition that attackers can exploit. A buffer overrun occurs when the space to which data is being written is too small for the data, causing the data to write to adjacent areas of memory. Neiderman says no programmers use this function today because it's flawed, yet the Samsung coders 'are using it everywhere.'"

...

Sometimes reblogging takes the form of a desperate prayer that people will finally care about how unbelievably bad things are.

ARM's big announcements

While ARM's big.LITTLE has evolved from its initial simplistic CPU migration and cluster migration iterations to simultaneous multi-processing (global task scheduling) to energy aware scheduling, the strict segmentation between big and LITTLE clusters has remained non-ideal. For example, the efficiency of shared memory access among CPUs and the speed of task migration have been significant downsides. Yesterday ARM announced the future of multi-core CPU design for its IP ecosystem, a series of technologies collectively branded DynamIQ which has major implications for SoC design.

I believe the reason DynamIQ took so long to announce, and was probably a ton of work to bring about, was how many interconnected systems were required to be redesigned in concert. And it was probably harder still to get them all working together well and efficiently. New interconnect designs and features are necessary to make the newly possible cluster designs work, and there is a new memory subsystem design for which no details are yet being provided. IP blocks such as accelerators can also now be plugged into these fabrics through a new dedicated low-latency port.

One of the biggest takeaways is that it will finally be possible to combine different core designs together within a cluster. If all of the enabling cache design and scheduling are well-implemented, such CPU designs could theoretically realize significant performance and efficiency improvements. Hopefully a DynamIQ system will not be too much harder to implement for silicon vendors, but I wouldn't assume it will be easy. ARM will really have to make it workable with its stock IP at the very least.

It's hard to say much more about DynamIQ, as ARM is still holding back most of the important details, which I would not really be qualified talk about anyway. There are other announcements such as new CPU instructions for deep learning, which I personally care less about but are still very important for many designs, such as IoT systems without a DSP or GPU. Since Cortex-A53's successor is likely coming at some point, depending on ARM's design goals, I wonder if the first DynamIQ systems will be based entirely on that new core.

Fuchsia’s hypervisor

It's hard to imagine launching a new mobile OS without support for existing apps. In regards to Fuchsia, I initially thought the Magenta kernel might be compatible with the Android user space. While the potential implementation details for Android support have not been clear, another possibility has become more apparent. I'm not saying anything specific will or will not happen, so please take the following with a grain of salt.

Before starting this blog I spotted some initial comments in Fuschia source about a hypervisor. There was extremely little mentioned, it seemed almost tangential or experimental, and honestly I thought, "nah, that wouldn't make any sense."

Google has since been developing a hypervisor as part of Magenta though. (The first commits were seemingly on the same day as my first blog post.) I have been hesitant to write anything, because you can virtualize anything.

This one difference could imply a ton. If this is what Google is doing, then Fuchsia really is a fully new OS, and I can understand why some people would take offense to the idea of it being a mashup of Android and Chrome OS. Significant amounts of code (Mojo) are derived from Chromium, however, and the ability to run Android apps in VMs could look roughly the same as user space compatibility would to users, at least superficially. Fuchsia will still end up gradually replacing Android and Chrome OS for consumers, while Chrome OS will live on for education.

In this scenario, Google would not be swapping kernels outright, but instead technically would be running two different kernels on a Fuchsia device. (I will again stress that the consumer definition of an OS is not just the kernel either.) The kernel underlying both Fuchsia and Android/Linux would be Magenta. Perhaps using a microkernel would make this a far more manageable or performant approach.

It is entirely possible to run virtual machines like containers, and many companies offer such solutions. Given such an implementation, it is possible to virtualize an individual app without providing an entire second desktop environment for users to manage, despite the app being run on a guest VM.

Hypervisors and containers are not new ideas whatsoever. Fuchsia’s implementation is a Type-1 hypervisor (bare metal) and is currently being built for x86 and MSM8998, which offer hardware-assisted virtualization commonly featured by CPUs. I have basically zero knowledge about virtualization, so there's not much more I can say beyond that.

Running Android on top of a hypervisor would make Android more of a legacy environment than a legacy API for Fuchsia, per se. It would also make Google’s statements that Chrome OS is not going away and that it wouldn’t make any sense for Android and Chrome OS to merge strictly true on a technical level. Again, this still means Fuchsia would crucially provide a stable driver ABI that Linux does not offer.

The massive upside to this approach would be that Magenta would be a clean slate free of the constraints of Linux, Unix, and possibly POSIX (to some degree?). I’m not a kernel expert, but I understand why this would be a huge deal resulting in numerous important technical differences vs. a *nix OS. I’m sure the Fuchsia team would stress the advantages of its decisions. Performance, efficiency, and a million other things could potentially be improved over Linux.

As for the downsides, wouldn’t a hypervisor significantly hurt mobile battery life? Performance and other functional tradeoffs also seem inevitable; virtualization is certainly not costless. But if the downsides are moderate, Fuchsia’s native performance is hopefully unaffected, and the need for virtualization is eliminated long term by Fuchsia replacing Android entirely, these seem like potentially reasonable costs. Without benchmarking, though, there is no hard data upon which to base an opinion.

Relatively seamless Android compatibility is a must in my opinion, because I don’t see how Fuchsia could be so much radically better on its own that consumers and developers would pounce on it otherwise. I’m sure there will be all sorts of carrots and sticks to incentivize writing Flutter apps, but it’s hard to picture Android developers re-writing all of their apps anytime soon. This is the same developer base that Google has struggled to even get to adopt something as crucial as the JobScheduler API to not waste users' battery life, which should have been the OS’ responsibility in the first place.

Google will probably first market Flutter heavily at I/O 2017 as a better, reactive, and cross-platform way of making apps, and then at some point add on “and they’ll run on Fuchsia as well!” It’s hard to imagine wild developer and device vendor enthusiasm for such an approach without making it clear that Android will eventually become legacy. This is due to the network effects implicit with technological platforms. (Unfortunately “network effects” is used as a hand-wavey phrase with little substance behind it, and no one in the tech industry seems familiar with the technical academic details from economics, but that is a discussion for another day.)

There are other considerations at play such as the AOSP vendor ecosystem, and I’m sure Google performed all due diligence in assessing its strategic technical options. There’s also still a lot in flux technically. Mojo IPC was changed, for example, and Mojo itself was seemingly absorbed into Magenta — Fuchsia’s API could end up being called anything regardless. Previous tricky questions such as “how will Fuchsia maintain compatibility with libraries like bionic?” would become irrelevant, though, if Android is simply virtualized. One remaining key question is how Fuchsia and Android apps would interact. This challenge seems extremely important and far from trivial.

As with any of my articles, technical corrections are both encouraged and highly appreciated.

Intel to acquire Mobileye for $15 billion

I am very, very against instant takes on these sorts of things in general, but it's hard not to instinctively think this is a mistake. Mobileye's tech is going to be ancient history.

Geneva 2017

The Geneva International Motor Show is going on this week. It’s generally regarded as the greatest auto show in the world, and I would kill to attend it to see the new Italian exotics alone.

One of the biggest headlines this year is the 2017 Porsche GT3. The GT3 has regained a manual option, so all is right again in this world. The GT3 in my opinion sets the standard for all other drivers’ cars, as it vies for Car of the Year awards with alarming consistency.

The above link is a walkthrough of the car by Andreas Preuninger, the head of GT cars at Porsche and one of the world’s leading experts on engineering drivers’ cars. This is unfortunately a rather short interview with Andreas, but here are some gems from the past, describing two phenomenal recent Porsches.

The launch I was most looking forward to, though, was that of Ferrari’s successor to the F12berlinetta, the 812 Superfast. As with the rest of its current line, the 812 was penned in-house by Ferrari, unlike the F12 which was of course co-designed with Pininfarina. The 812’s styling is controversial, though I actually like it in the metal, much to my surprise. I didn’t care for the surface detailing of the F12, so Ferrari has “fixed” its front mid-engined GT in my mind, at least if you’re looking at the rear from above. Aerodynamics is not doing any favors for taillight design these days…

The Nintendo Switch's hardware

Nintendo's newest system launches tomorrow. By now, there is very little to discuss about the Switch's hardware that has not already been covered in detail online, though I would like to highlight the excellent technical overviews of both Digital Foundry and AnandTech. Ryan Smith also figured out immediately that the Switch actively converts DisplayPort to HDMI. (I don't like Nintendo's docking implementation, for what it's worth.)

If you're familiar with mobile silicon, it wasn't very hard to understand the Switch's hardware. The key points are that it uses a revised NVIDIA Tegra X1 SoC with a Maxwell GPU, still fabbed on TSMC's 20SoC process. 20nm was unequivocally a bad node. Transistor leakage was a massive challenge on the process, which came just before the transition to FinFETs. Essentially this means that the X1 in the Switch is far from competitive in terms of computational efficiency.

Furthermore, in order to fit its power and thermal budgets, Nintendo had to downclock the X1's CPU, GPU, and memory quite considerably to provide reasonable handheld battery life. Resulting performance is not very impressive to say the least. There wasn't much that Nintendo could do, however, since NVIDIA had nothing newer to sell it that could ship within Nintendo's target deadlines.

Some, namely Apple, were able to wrangle 20nm well enough to take advantage of its benefits, but many silicon vendors stumbled severely with it. To see a game console utilize 20SoC is frankly a bit depressing. A lesser problem is that ARM's Cortex-A57 is not exactly a terribly efficient CPU architecture by 2017 standards. The Maxwell GPU, however, did feature a new, more efficient tiling architecture that was perfectly competitive upon its initial release.

Less known to many is that the original X1 shipped in a rather sad state. NVIDIA failed to make a working interconnect, and ended up shipping broken silicon. The end result was that the four LITTLE CPUs had to be disabled, so only the four big CPUs were actually active in the original SHIELD TV and Google's Pixel C. This did not stop NVIDIA from advertising eight functional CPU cores, however.

New to the Switch is a revised X1 chipset, for which NVIDIA probably removed the four LITTLE cores entirely, and likely replaced the broken interconnect with something simpler and hopefully fully functional. This would be the minimal level of fixes that I hope Nintendo demanded. Beyond that, the revised X1 in the Switch and the 2017 SHIELD TV likely features fairly minor improvements. It's possible that there are differences between the two new chipset revisions, but there is no public information available either way.

Updates:

1) To be clear, even if NVIDIA is calling both chipsets (2015 and 2017) "X1," if it really has removed the LITTLE cores and replaced the interconnect of the latter, it would actually be a new chipset. It's also worth noting that both SHIELD TVs come with 3 GB of LPDDR4 RAM, while the Switch features 4 GB, an important difference.

2) I was wrong: the A53s are still there, and the logic is still broken. Unbelievable. There's no public evidence of what has changed then, though if I had to guess, there might be some semi-custom tweaks to the GPU and not much else. (Those could be important differences, mind you.) Not making any claims!

Why Fuchsia needs color management

There is no substitute for color management. Unfortunately, Android has a major color management problem.

Ultra HD (UHD) is a generational advance in consumer display standards. UHD marketing generally refers to the combination of high dynamic range (HDR) output, an expanded color space beyond sRGB, and a minimum resolution of 2160p (4K). While I intend to write an introductory HDR article in the future, today I want to focus on the many technologies required in order to correctly render UHD content.

There are only two color gamuts permitted for devices and content to be certified by the UHD Alliance: DCI-P3 and Rec. 2020. I am specifically referring to the gamuts, not the color spaces. In order to support an HDR-capable panel, a display vendor also has to utilize an HDR-capable DDIC (display driver). And in software, there will be a separate ICC profile specifically for HDR. (HDR for mobile may also require local tone mapping, but I’m not knowledgeable enough to know the details.)

Android gained "HDR" support in Nougat for Android TV. While strictly speaking this is true, I put HDR in quotes because there is no color management. What Google means is that Nougat supports the HDR10, Dolby Vision, and HLG electro-optical transfer functions (EOTFs), aka non-linear gamma. This almost works.

UHD content currently requires a display that targets the P3 color space. If you watch UHD content on a P3 display from, say, Google Play on Android TV, what actually happens is that desaturated frames are presented to the hardware because sRGB is the assumed color gamut, but the TV then oversaturates the image back to the correct P3 color space. The wrong color values get reversed, in other words. The net result will be a correct image, but only on P3 displays. All of the UI controls overlaid on top of the content will, however, be oversaturated.

While LG and Samsung — if LG even manages to correctly calibrate a panel to a target gamut for once — will claim that they are shipping HDR devices with the G6 and Galaxy S8, and the hardware will be technically capable, Android still has no color management. Want to watch a UHD movie in a separate window while multi-tasking with other apps at the same time? Too bad, all of your color and gamma will be wrong. Color (or HDR) modes are not a real solution, despite what Microsoft will tell you.

Extended sRGB, or e-sRGB, is a linear extension of the sRGB color space. This is an available option for some kinds of applications, such as OpenGL apps. Unfortunately HDR will probably break windowed e-sRGB apps on HDR displays due to the non-linear gamma. There is no substitute for color management, and all apps will require it.

It will be interesting to see whether the LG G6 is calibrated for sRGB or P3, because targeting one inherently compromises on the other. Neither strategy will work perfectly regardless, because Android is stuck on sRGB. sRGB remains the de-facto standard for most content, and this critically includes web content. Without color management, there is simply no way to simultaneously support multiple color gamuts. This is why Samsung calibrates to sRGB and includes a separate P3 color mode, even though almost no consumers will ever know or bother to change color modes. At best, this is a very annoying necessity.

The LG G6 will also suffer from color banding, due to insufficient color gradation steps. Even though its display hardware should be capable of 10-bit output, Snapdragon 821 only has an 8-bit display controller pipeline. Sadly, the iPhone 8 will probably be the first mobile device that can support UHD end-to-end, despite other hardware being HDR-capable dating back to 2016’s Note7.

It is not clear to me if adding color management to Android is even technically possible at this point. But for Fuchsia’s new rendering pipeline, there is probably no excuse not to design it with color management in mind from the outset. Mobile UHD devices require it, if they are ever going to actually work. Don’t let consumers down, Google.

 

Updates:

1) I'm extremely happy to say that Romain Guy himself says "there is no reason why color management cannot be added to Android." Awesome! Hopefully these problems will be addressed in Android O then :)

2) Color management is indeed a feature of Android O.