New TVs that launch with Android TV 14 or later on Linux kernel 5.15 or higher will be required to meet Google’s Generic Kernel Image (GKI) requirements in order to pass certification!
This means that GKI is now enforced on all major Android form factors with AArch64 chipsets: handhelds, watches, automotive, & televisions.
↫ Mishaal Rahman
What this means is that all the major Android form factors will be running kernels that adhere to the GKI requirements, which means SoC and board support is not part of the core kernel, but instead achieved through loadable modules. This should, in theory, make it easier to provide long-term support.
Oh, and a stable abi for modules, such that modules and kernels can be updated without requiring updates from the other. Basically what some have screamed for linux to have for years. I get it, it removes the incentive for open source modules, and upstreaming them. But that hasn’t worked, this does.
Bill Shooter of Bul,
I’ve been screaming for it too. We’ve been stuck for so long and something needed to change.
I sure hope it works as advertised with no gimmicks! I find it doubtful mainstream linux would merge this, but if 100% of android devices support it then this could really be a boon to android alternatives including lineageos. A stable ABI could drastically help increase the list of supported devices as well as long term support for those devices.
This is not what you think it is.
https://source.android.com/docs/core/architecture/kernel/android-common
Android 14 has two launch kernels that is 6.1 and 5.15
Yes a kernel module that loads with 5.15 is not required to be loadable with 6.1
Yes as what is noted above devices updating from prior version so android don’t have to be 5.15 or newer.
Yes the android13-5.15 kernel is not the same as android14-5.15.
Lot of prior releases of Android new device new soc got away with using a generation or two old android kernel instead of the one of the current version. So party releasing andorid 13 device with new soc chip in fact using android 11 kernel did happen. Also you have had totally custom vendor kernels as well. Then throw in a few doing custom kernels as well and you have quite a mess.
The kernel stable ABI of Android is exactly the same as the redhat one kABI
https://access.redhat.com/solutions/444773
Yes only stable for first 2 numbers of the Linux kernel version number. and the kernels are built with the same configuration. When I say exactly the same the reality Android developers use the same tools redhat uses and developed for their kABI with the same limitations. Just the kernel builds are built from different kernel configurations so they are incompatible with each other.. It been this way from the start of the Generic Kernel Image (GKI) project of Android.
The change is now standard signed google Linux kernels will be provided and expected to be used and check if being used for certification. This means up-streaming all kernel alterations that cannot be done as kernel modules. No more totally custom SoC kernels while using Android trademarks on device. This is more open sourcing required by soc vendors. More chance that a stock kernel will somewhat work in the case soc vendor kernel modules don’t.
Yes GKI-2.0 that is with Android 12 this is when google started providing signed kernels and was expecting vendors to use them by Android guidelines. Of course different vendors have been doing custom kernels from Android 12 to now even that been against google guidelines the complete time. The change with Android 14 you don’t use a google provide kernel your device now fails certification and failed certification no using Android trademarks. Use old version of Android kernel on a new device with Android 14 also fail certification. Yes the 5.15 kernel from Android 13 even that it a 5.15 kernel the Android 14 certification will fail because its not the Android 14 5.15 kernel. No more of make new device use generation or 2 old kernel version.
I would say these changes will be some advantage to Android users buying new devices as their kernels will not be open to as many known CVE issues due to newer kernels being forced used that google can push out updated versions for. But these changes don’t make Android have universal drivers. No more custom soc kernel under android will force more SoC vendor core kernel alterations into the mainline Linus kernel. Yes the google provided GKI kernels have a Linux kernel mainline first policy. Yes this equal more open source not less.
oiaohm,
Stability at the major version numbers is still much better than the linux mainline policy of having no stability at all. Many of us had asked for stability for one or two major versions, so I am very pleased this is happening at all and being enforced by google! We’ll have to see how it actually plays out of course, but it could be a huge gift to alt-OS by being able to release one kernel across potentially thousands of devices thanks to drivers using a “more stable ABI”. I don’t want to count chickens before they hatch, but I am optimistic this could bring progress and help to break the role that drivers played in locking us to manufacturer builds.
–Stability at the major version numbers is still much better than the linux mainline policy of having no stability at all. —
There is point here people miss. Redhat kabi collected data on Linux kernel symbol stability.
–Many of us had asked for stability for one or two major versions, so I am very pleased this is happening at all and being enforced by google–
RHEL/Redhat Linux as long as they have had kABI they have enforce the same thing google is.
You cannot use a kernel module from android13-5.15 in android14-5.15 or the reverse safely. These are the same major versions.
https://www.kernel.org/doc/Documentation/process/stable-api-nonsense.rst This here contains a list of what need to be true to allow Linux kernel ABI to be stable like redhat and google are doing.
[Depending on the version of the C compiler you use,]
So you must lock down the compiler used to build the kernels.
[Depending on what kernel build options you select,]
You must use the same kernel configuration(kconfig)
These two are why the android13-5.15 and android14-5.15 cannot use each other kernel modules safely those two things are in fact different..
[Now a number of these issues can be addressed by simply compiling your module for the exact specific kernel configuration, using the same exact C compiler that the kernel was built with. This is sufficient if you want to provide a module for a specific release version of a specific Linux distribution.]
This bit is what Redhat and Google are in fact doing. Google uses the Android[number] thing before their kernel to define the kernel config and the compiler used do note those being the same still does not allow you to safely use modules between Android14-6.1 and Android14-5.15.
Alfman the reality google is not doing anything Redhat has not been doing for decades. It good for less fragmentation in the android eco system. Also Redhat is not the only comerical Linux distribution to do a stable kabi
https://drivers.suse.com/doc/SolidDriver/SUSE_Kernel_ABI_Stability.html
[Keep the kernel ABI stable for the lifetime of each service pack. Kernel Module Packages built for a specific service pack will remain compatible with all update kernels to that service pack. The same policy holds for kernels updates between the initial release of the product and the first service pack.]
Some with different level of coverage.
Alfman the problem here to have Linux kernel modules compatible between Linux distributions this requires getting all those distributions to use the same kernel config and compilers and for the major versions of those kernels to the same.
Limited scope as in contained to a distribution like the android, redhat, suse…. with binary drivers working with a stable ABI have happened inside those limited scopes has happened over and over again this is nothing new..
The reality here what google is not doing anything that special from a driver ABI stability point of view with Linux. The big thing is the google demand that all android devices come down to a limited number of kernels instead of the current fragmented mess. One survey of Android 13 arm64 devices found over 120 different major kernels versions yes different configurations and different compilers so when I say fragmented mess I am not kidding particularly when google only supported 5 versions to exist yes over 115 too many. Yes google did need todo something here. Please note that survey was not every Android device.
Android device vendor tweaks kernel config this can improve power efficiency/performance and so on. So these changes result in them winning in benchmarks done by people telling people want device to buy. So it going to be interesting to see if Google brute force method with Android 14 is able to make Android device vendors stop altering core kernel settings to win sales.
Do remember the upstream kernel.org project if they started controlling what compiler/kernel config users had to use build the kernel they would have a backlash so fast it not funny.
Upstream Linux kernel is unable to provide a stable kernel ABI because the items you need to lock down is not under their control and if they attempt to make those things under their control the kernel will most likely get forked because too many parties want to tweak the kernel settings/use different compiler so their solution performs differently so they can win over competition. Android problem and main Linux distribution problem with stable kernel ABI is the same thing. This is why it will pay to watch if Google brute force works or if we end up with more forked of Android on devices that don’t have to use the google trademarks..
oiaohm,
That’s good for redhat I suppose, but they don’t have google’s footprint on android & SBC devices where the need for a stable ABI & drivers is greatest.
I think the statement is overly broad. While there are thousands of settings and countless permutations, it’s only ones that matter for a stable ABI that matter and if google are going to mandate compatibility with their kernels then this benefits others as well.
–think the statement is overly broad. While there are thousands of settings and countless permutations, it’s only ones that matter for a stable ABI that matter and if google are going to mandate compatibility with their kernels then this benefits others as well.–
Google mandate will effect others by the hopefully increased upstreaming not the stable ABI bit.
The statement appears overly broad I will give you that but its not as much over broad as you are guessing either. The problem is what the Linux kernel is yes the modular monolithic design comes into play. You do a lsmod and you see
items like
scsi_common [number] [number] scsi_mod,usb_storage,pktcdvd,uas,libata,sg,sr_mod
That right the scsi_common depends on 7 other modules. This is the effect of the modular design that individual driver under Linux can be using parts from other drivers. This means the Linux kernel in kernel ABI is huge. Every kernel configure option that enables another module add more in kernel ABI to the pile because now other drivers can use that as well.
The horrible reality Alfman is 90%+ of the options in linux kernel configure in fact matter for sure when you are talking about stable kernel ABI for third party drivers and the other ~10% we are not sure of but they could matter.
There is items that are unique about Android kernels.
https://docs.kernel.org/admin-guide/binderfs.html
Like you don’t find binderfs enabled in debian/ubuntu/redhat ….. kernels this is basically a Android only thing. Of course a driver written for android instead of mainline Linux can have a hard dependency on binderfs being there. There are more examples.
Your non android SBC devices don’t use Android kernel with Android kernel configuration. Remember unique items of Android kernel enabled increase kernel memory and cpu foot print over not having them enabled so they are performance liability if you don’t need them.
Yes like it or not SBC very quickly comes one size does not fit all due to limited resources. Yes linux kernel means to customize to remove everything you think you don’t need is a nice path to opps that third party driver does not work because I removed something it depended on.
Something we can be sure of with the Android generic kernels there are going to be socs that are running slower for users because Google has mandated a unified kernel so a few extra parts had to be included that a vendor kernel would have stripped out. Stable ABI is absolutely not a free lunch to have it there are prices to pay.
Microsoft provides a driver SDK that has it own compiler for the reason you cannot mix and match compiler when attempting to have a stable driver ABI. Even Microsoft kernel developers have admit at times some of MS windows lower performance in particular benchmarks is due to being stuck on a particular version of compiler.
Stable ABI has a price to pay. Lot of parties are not happy when they hear price is few percent of performance up to 10% as the cost of having stable ABI particularly on some SBC where you don’t have resources to spare. Yes its the Linus hurding cats problem attempting to get people to agree on syncing the parts to make the stable ABI. You are always going to have some of the cats prize winning benchmark competitions or having more processing time to perform their tasks over having stable ABI. Yes those attempting to compete against those cats end up being forced into the same setup without stable ABI.
oiaohm,
Time will tell if anything more gets upstreamed, I have doubts. But either way it’s not mutually exclusive, there are benefits to upstreaming AND the stable ABI bit.
“90%+ of the options in linux kernel configure” is clearly over the top. The majority of kernel options will not affect a given driver. In any case though, google will be enforcing stable ABIs for it’s purposes, and this should have carry over benefits for other forks as well.
You can though and I’ve personally used GCC to write windows drivers. Obviously you have to know what you’re doing and take care to use the right calling conventions and structure layouts.
https://en.wikipedia.org/wiki/X86_calling_conventions
Beyond that though the CPU doesn’t care what compiler or even optimization flags used to build a driver (you need to set a compatible CPU target, but that’s a given). Obviously I’m not claiming microsoft officially supports it, but one driver does not have to use the same compiler in order to be compatible with the kernel ABI.
— You can though and I’ve personally used GCC to write windows drivers. Obviously you have to know what you’re doing and take care to use the right calling conventions and structure layouts. Beyond that though the CPU doesn’t care what compiler or even optimization flags used to build a driver —
This is lack of knowledge on your part because it not the CPU.
https://www.kernel.org/doc/Documentation/process/stable-api-nonsense.rst
The note the compiler is here because of structure layouts. Different compiler optimization passes can mess with your structure layouts and alignments. so yes optimization flags matter This is fine if everything is messed up the same way because you used the same complier and optimization flags. Total disaster when it not. Think about a optimization that has moved a structure slightly so it better aligned to be read by CPU now it may not be doing this movement all the time. Basically you just made a works of me argument and did not know. Those with more experience know that due to the kind of corner cases you deal with by mixing compilers in kernel space you don’t do it. Its risky in user space in kernel space it system goes splat mistake when you hit it and can be very hard to defect to locate and work out why due to how horrible complex some optimization pass rules can be…
–this should have carry over benefits for other forks as well.–
Looking at coloros and many other android forks they are already not using google GKI. Some these are they are a china based company so cannot be directly dependent on an USA company for supply so they are always going to build their own kernels. Third party rom replacements also don’t follow GKI in most cases. Google power on this is a little more limited than what you are thinking.. The biggest gain is those using direct google android will upstream more. Those not using direct google android the stable ABI stuff is not getting to them and that the way it going to be.
–The majority of kernel options will not affect a given driver. —
Problem here user wanting a stable driver abi is normally not after a single given driver instead all third party drivers that you can be hit by. This makes the problem space way larger and way harder todo. This is just you incorrectly narrowing the problem space.
oiaohm,
I know kernel stability is taboo in the echelons of the linux kernel and I admire your experience and depth of knowledge on many topics. Really. However in this case not only does the logic dictate that ABI stability should work across compilers as long as they can support the same calling conventions and structure layouts at the ABI boundary. I am right about this, I’ve done exactly what you said does not work and it does in fact work.
This is just a minor caveat for someone experienced with kernel development. Microsoft’s structured exception handlers was a bigger challenge at the time because these were microsoft’s own extensions to C.
https://en.wikipedia.org/wiki/Microsoft-specific_exception_handling_mechanisms
Obviously in the past it wouldn’t work, but assuming google enforces it across 100% of the android echo system, then it starts to make more sense for forks to support these ABI drivers in the future.
Disagreed. Using the same compiler settings that google themselves use is a win/win.
–I know kernel stability is taboo in the echelons of the linux kernel and I admire your experience and depth of knowledge on many topics. Really. However in this case not only does the logic dictate that ABI stability should work across compilers as long as they can support the same calling conventions and structure layouts at the ABI boundary. I am right about this, I’ve done exactly what you said does not work and it does in fact work.–
This is a mistake on your part. There is another party with Linux made the same mistake you just did. Those attempting to bring rust coded drivers to Linux.
–This is just a minor caveat for someone experienced with kernel development. Microsoft’s structured exception handlers was a bigger challenge at the time because these were microsoft’s own extensions to C.–
No this is coming from someone who looked at the Linux kernel problem and the way it ABI is. The worst part is a combination of structure layout and RCU with Linux. Linux does have structured exception handlers as well.
Remember I wrote that Linux kernel is a modular monolithic this has a key property that makes defining down you abi harder than Windows Hybrid. The key property is direct interaction between kernel modules under Linux is allowed by items like RCU and others. Module dependencies of Linux kernel has very scary meaning for ABI construction.
Windows hybrid is based around microkernel not monolithic so interactions between drivers is between core kernel servers and drivers. The monolithic bit of Linux has direct interaction driver to driver with quite a few bipassing the Linux core kernel.
Yes those attempting to bring rust drivers to Linux attempted a real mix of llvm built code (for the rust kernel module) mixed with gcc built code(for all the other parts of code). Yes 1 version of llvm and 1 version of gcc. This showed up the RCU nightmare and quite a few other things Linux kernel does because it monolithic design that means drivers are not that isolated from each other. There is not a nice defined ABI splitting drivers from each other in the Linux kernel.
This side ways interaction between drivers is why so many Linux kernel config settings like it or not have a direct effect on the Linux in kernel ABI. ABI in kernel space of Linux is not kernel core part to drivers instead its kernel core parts to drivers and driver to driver.
–Obviously in the past it wouldn’t work, but assuming google enforces it across 100% of the android echo system, then it starts to make more sense for forks to support these ABI drivers in the future.–
We do have example in the past in the linked to the android eco system. https://en.wikipedia.org/wiki/Libhybris Yes libhybris to use userspace closed source android graphics drivers. Over time those get replaced by open source implementations.
The issue of coloros and other android forks will still apply. You have a closed source driver made by some other third party who happens to be from a different country to you that your country could go to war with now you cannot use that driver in your solution.
–Using the same compiler settings that google themselves use is a win/win.–
With the Linux kernel due to how it ABI is that not good enough.
What you got away with Windows drivers the rust driver developers for Linux documented with the Linux kernel this does not work and it due to what the Linux kernel is. The Linux kernel modular monolithic design for it in kernel ABI is quite a nasty thing that very touchy to compiler differences but modular monolithic does have high performance due to the direct interaction between kernel modules/drivers being allowed. Yes this single feature causes the real Linux in kernel ABI surface to be huge when dealing with windows direct driver to driver by design is technically forbid..
Yes your windows experience with using a different compiler is playing in a way simpler sandbox where is simpler to create a works for me example and not see the trap. Linux in kernel ABI in surface area the rust developer estimate it somewhere between 1000 to 1 million times bigger than the Windows driver ABI and the million might be low. Stable ABI in kernel with Linux is hard because the core design module monolithic make the in kernel ABI huge and connected almost everywhere with direct connections.
Limiting the kernel configuration of the Linux kernel limits how huge the in kernel ABI is in fact. There as a time in the Linux kernel past when kernel modules and kernel core shipped as two different parts back then there was way less direct driver to driver connections so the Linux kernel ABI back then was way simpler. The introduction of RCU and other parts happened after this increased the number of direct driver to driver connections inside the Linux kernel massively so making the Linux in kernel ABI one of the most complex things on earth. Design choices have prices.
oiaohm,
This is a mistake on your part.
I’m sorry but you are wrong. I’ve 100% done it and there is a zero percent chance of you convincing me it doesn’t work. Let’s move on.
You may feel that bringing rust to linux is a mistake, but that’s just your opinion.
Well, I for one am glad that google are doing this because from my perspective the lack of a stable kernel ABI has created a lot of trouble and we’ve been stuck with the shortcomings of vendor locked kernels on android/ARM for so many years with very little progress. Of course we have to see how things will pan out, but I’m more optimistic now that things finally might be changing for the better. Time will tell.
I’m all for open source implementations of everything, but too often that’s not what we actually get.
–You may feel that bringing rust to linux is a mistake, but that’s just your opinion.–
I am not anti rust being used the Linux kernel in fact I think long term is a good idea. Linux kernel C is really C+sparse. Lot of ways without sparse checking things the in kernel ABI for Linux would have collapsed into a heap of too much complexity before now. The rust developers doing this tried using a different compiler for the rust kernel modules to what built the kernel itself. So the rust developers for the Linux kernel have real world experience of what happens when you attempt to use a different compiler to what the linux kernel was built with.
What would be a minor gcc or llvm optimization alteration that has zero effect in user-space for you dynamic linked executable in Linux kernel space would alter something leading to major incompatibility is something the rust linux module developers found. The Linux kernel for kernel modules is closer to static linked than dynamic linked in it behaviors to what will cause failure but it even more touchy than that. Basically the Linux in kernel ABI is very sensitive to minor compiler differences.
–Well, I for one am glad that google are doing this because from my perspective the lack of a stable kernel ABI has created a lot of trouble and we’ve been stuck with the shortcomings of vendor locked kernels on android/ARM for so many years with very little progress. —
https://en.wikipedia.org/wiki/List_of_custom_Android_distributions
You go down this list pull out everyone that is made by a phone vendor you will find not one of them uses the google GKI provided kernels. Heck a few of them you cannot even get the kernel source that you should be able to do to be in alignment with the GPL license.
Many of these vendors cannot be pressured by google to use the GKI because the need independent to USA supply.
https://www.oppo.com/ae/newsroom/press/oppo-secure-fourth-place-in-global-smartphone-market-in-2023/
Yes Samsung Xiaomi Oppo Transsion listed here that oppo was happy they were in 4 places the bad news is that no one of them in fact using Google Android. Yes those 4 are 50% of smartphone market. Problem is what people would think of as Android smart phone really is not. Not one of those 4 vendors need the google trademarks to sell their devices they have their own OS builds and are not restricted by the rules enforcing GKI usage.
Less than 15 percent of what people think of as Android is effected by this google ruling of must use GKI kernel. The soc vendors in this 15% over 98% of them are up-streaming their drivers into mainline linux kernel so making the Stable ABI caused by GKI enforcement pointless for everything other than less than 1% of devices. When GKI started there was uptick in these lower end phone soc vendors up-streaming there kernel drivers into mainline linux kernel. GKI biggest effect is more open sourcing of Soc kernel drivers.
This GKI change is going to be better for buyers of these bottom end devices for getting kernel updates that in the past they would not been able to get. It also stops bottom end device from doing tweaked kernels to benchmark better. This is the major changes.
Effect on non android SBC usage absolutely nothing.
Yes surprise google power to control the Android like OS on devices is bugger all. Google can only control devices that are really using Google Android. Anything listed as an custom Android distribution google in reality has basically zero control over when it comes to the Linux kernel is in it. This is why I don’t see this change as a big thing because at best effecting less than 15% of devices is not dominate change. The big effect will be speed up mainlining efforts and we are already seeing this with Linux kernel merge request to back port drivers into 5.15 from some of these phone soc vendors in the effected area.
–I’m all for open source implementations of everything, but too often that’s not what we actually get.–
This is the thing in the area effected by the new google rule open source for everything is what we are getting majority of the time any how for the past 5 years. Of course that leaves the 85+ that is not effected by this rule that you would have wanted effected if this rule was going to change anything.
This new google rule change is basically pointless form the usable stable ABI point of view because the parties you would want effected are not going to be effected by it.
oiaohm,
As long as the calling conventions and structure definitions are stables, that’s what’s important It’s not just kernels, there would be hundreds of thousands of libraries that would break if compilers where regularly changing structures and calling conventions, which is why they rarely change.
If the article is accurate, the requirements should apply to all android branded phones in the future. Granted, if it’s not enforced it will not work. Presumably google would decertify devices that don’t comply and they would no longer be allowed to use android trademarks.
Sure, hardware manufacturers could sell their products under different branding to escape google’s branding requirements. But android forks don’t really have to worry about that because it really is not their responsibility to port android to non-android phones.
Except that many SBCs are not compatible with mainline because for better or worse broadcom/qualcom are selling the ARM cpus for use in android and that’s all they support.
With all due respect though, speak for yourself.
–As long as the calling conventions and structure definitions are stables, that’s what’s important It’s not just kernels, there would be hundreds of thousands of libraries that would break if compilers where regularly changing structures and calling conventions, which is why they rarely change.–
There is a problem what you wrote is not true for the Linux kernel. The major example RCU.
Is RCU a standard define calling convention. The answer is no. Is RCU in the Linux kernel used in places in a classic design OS used something covered by standard defined calling convention. Next is how RCU is passing structures. Create process of a structure to be transferred by RCU to the compiler looks like a function internal structure. Yes if something looks like internal structure like it or not compiler optimizer see no problem playing with the layouts.
RCU is particular evil because it MMU operation. So you tell RCU what pages you want the MMU to map into processes that ask for it with the update command. Yes the stucture appears in another process by read lock by it being mapped into the that area of memory. These operations are not crossing you tradition calling conventions.
–Except that many SBCs are not compatible with mainline because for better or worse broadcom/qualcom are selling the ARM cpus for use in android–
This has been changing in the past 5 years. This starts with project mainline in 2020 this leads GKI 1.0 in 2021.
https://wiki.postmarketos.org/wiki/Qualcomm_mainline_porting
Qualcom all Soc themselves since 2022 run the mainline Linux kernel from kernel.org without needing patches. Devices needing patches now .are like the samsung ones with custom chips. Qualcom moved up their work to mainline stuff into kernel.org after 2020 google project mainline.
The vendors that use Qualcom socs that use custom chips that would need a third party driver are not covered by this GKI requirements. Remember these custom chips are extra cost and all parties doing that are making their own custom android distribution so not covered by change.
Broadcom is in basically location where they are for their new Soc chips they are kernel.org Linux kernel.
This is the problem when you look at the SoC chips being released now that this new rule is going to apply to the kernel binary ABI bit pointless because these SoC chips are kernel.org mainline supported.
The big change will be that the people on the entry level devices under the google’s branding requirements will in fact get kernel updates. It will not be dependent on the vendor of the device to push out kernel updates.
The stable abi requirement in the devices covered is basically non existent because of how since 2020 the different soc vendors have been mainline into kernel.org their soc chip drivers.
This is my problem to have effect stable ABI the rule need to be broader applied. Like if you are not using GKI provide kernel you cannot use the google play store but the problem is doing something like this would run into an anti-trust law suit against google..
By the way with Qualcom and broadcom we are seeing the old chips get patches into kernel.org mainline as well from Qualcomm and Broadcom developers themselves with this lining up that Qualcomm and Broadcom has just got a new order for that chip by some vendor so they updated the drivers and merged all the required patches so mainline kernel.org does not need patches any more.
This took a major turn in 2020 that you have missed seeing Alfman. This turn makes the stable in kernel ABI for lot of these devices as pointless feature because it not going to be used..
oiaohm,
I am afraid you are mistaken, using different compilers works fine so long as the calling convention and structure layouts are respected. I’ve done this personally and the problem that comes up is that the ABI is not stable.
Still, to whatever extent you want to pretend the compiler is a problem (which you are wrong about), that’s still not a problem for an android fork because they can use the same compiler and settings that google used. This is something you keep ignoring…to the extent that the ABI is stable enough for google, it’s stable enough for forks too!
–I am afraid you are mistaken, using different compilers works fine so long as the calling convention and structure layouts are respected. I’ve done this personally and the problem that comes up is that the ABI is not stable.–
I am not mistaken. The rust developers doing mix of llvm and gcc documented how both them when they think a structure is internally used use one set of alignments and when structure is believed exported use a different set. Yes the alignments done to speed up cpu interactions.
— that’s still not a problem for an android fork because they can use the same compiler and settings that google used. This is something you keep ignoring…to the extent that the ABI is stable enough for google, it’s stable enough for forks too–
First is do they the answer is no they don’t.
https://reproducible-builds.org/docs/perimeter/
Compiler not to have strangeness end up being complete build environment.
Lot of the vendor ones are using soc chips that google made compiler is not built for.. Google Android GKI items are built with LLVM. Lets say you have a Linux kernel driver from third party for some chip your phone is using that used in non android SBCs this driver will be certified with gcc. Lot of the forks are using gcc built kernels and that is so they can take third party drivers that have already been vetted in non android SBC usage.
The issue rust developers run into with LLVM and GCC in Linux kernel space not playing ball happens to be major factor for why majority of android forks and google android GKI are not going to be compatible. Yes most people miss that like rockchip, broadcom and so on for non Android SBC usage build with gcc. Yes your Linux desktop/server distributions majority the kernel is built with gcc. Google Android kernel build compiler choice is a outlier.
There is more to being driver compatible than having a stable kernel ABI. Remember drivers can be using custom machine code for that platform with these soc chips. Welcome custom compiler required. This is what happens when Samsung and number of other vendors gives you the Linux kernel source code to some of their phones soc it is bugger I cannot rebuild that I need the samsung gcc and binutil patches found in the latest branch so the code can build because there is extra machine/asm code instructions here that are not android but instead exclusive to this soc chip.
Yes the first problem before in kernel ABI is does the compiler tool suite having everything required to build the drivers. Nest is how validated is this driver with this compiler choice. Gcc broad usage there are lot of cases the gcc build is more validated than the llvm build.
Alfman you are writing they can follow Google lead but in reality many cases they cannot because of Google choice that android GKI kernels will be llvm built.
Gcc and llvm 128 bit number processing is in fact completely different on how they store in memory. and there are other examples like this as well. Remember what you said about structure layouts. Gcc and LLVM are only memory layout compatible if you don’t use particular features across the boundary because gcc and llvm implemented different areas because of like 128 number processing storage structures being different without the optimization mess that you hit with RCU. There are lot traps for the unaware when you mix compilers.
Thank you for your opinion, I appreciate that linux base has a long standing opposition to stable ABIs and that’s largely driving the arguments against them, but I am going to go with my own experience on this one. Please understand that regardless of what you say, I am still glad that google are pushing this.
–I am going to go with my own experience on this one.–
Alfman you need to look at the rust developers who attempted the llvm/gcc mix with the Linux kernel. For the first year and half it was seaming like it was going to be successful.
They get to RCU things start going wrong with structures. Then they get to 128 bit and both compilers have completely different structures and CPU instructions used. There are 40 more items after this they found.
Under Windows you were only dealing with one form of complier incompatibility that shows it self in a big way being the exception handing.
Do note current Microsoft compliers don’t support 128 bit instructions in C yet llvm and gcc do because the x86 and arm instructions sets provide more than 1 way todo 128bit operations. Yes gcc and llvm have chosen different ways to do 128bit operations. Yes Linux kernel in the in kernel ABI do use 128 bit integer operations but the interface exposed to user-space for stability does not export 128 directly instead gives you 2 64 bit values to get a stable structure layout.
There are many layers of trouble here. There is a long list of items not allowed by the Linux kernel to be put on the syscall interface because different compilers implement them differently. Not having wrapper code to make them stable in kernel space avoids those operations causing overhead.
Alfman your experience with windows is a lot simple because Microsoft complier is not a feature rich as llvm and gcc as well. So lots of the different trouble seen when attempting a Linux kernel driver with wrong complier you don’t see with windows because those features are not in the Windows kernel interfaces because Microsoft compiler does not support them.
Yes RCU is not something Windows in kernel level uses as well.
oiaohm,
Keep in mind I didn’t bring up windows, you brought it up initially. I’ve done linux kernel development too and please understand that I will use my own experience over your claims. I know how adamantly some linux devs are about protesting these stable ABIs, and I get it. They want to encourag drivers to be up-steamed. Truthfully I am not against this. But the lack of progress in the past decades is just…ugh we’ll all be dead before things improve.
Manufactures don’t give a crap and can afford to ignore our pleas indefinitely. This has turned into an unwinnable turf war. I’m not against your goals, but I am against the permanent state of inaction that we’ve been in. I understand why you hate to concede anything, but we desperately need change and I am just glad that google are trying to do something about it.
Ok, so it’s just an LTS kernel with backports. I read through the info on it, and I thought Google wrote a shim to smooth out the transitions between kernels.
Google really should have written their own kernel with a stable driver interface.
–Google really should have written their own kernel with a stable driver interface.–
Google has tried this with Fuchsia and others. There is one major problem. Soc vendors they have Linux developers those developers don’t code for anything other than Linux for free in most cases.
Trying at this stage to convince soc vendors to employ a new team to developers to support a new OS is kind of a uphill battle. Remember Microsoft had to pay billions to Qualcomm to get them to do arm soc support for Microsoft Windows and they have to keep on paying. Remember that just to get 1 vendor to support a new platform there are
Soc vendors are doing Linux support for free because they are selling enough units to users who are using Linux to cover the cost. Even google making their own soc chip has bought IP(silicon designs) for that chip form others and that IP has Linux drivers already written. Again linux vast usage means the IP seller can sell for massive number of different chips having Linux support so recover cost quite effectively writing Linux drivers.
This is economy of scale problem. Linux has the economy of scale behind it to be cost effective to make drivers for. A new OS just does not have the economy of scale.
Apple doing high end products have been able to invest the money to write all the drivers themselves and get return on investment. Doing low end products this comes a problem very quickly. Low end devices you want economy of scale effects to reduce costs.
Going away from Linux will see vastly higher costs for most likely a decade and attempting it may let some other Linux using competitor to enter the market and beat their ass due to their lower cost. Going after the low cost market causes some serous limitations but to have large market share you have to go after the low cost market.
Google is really stuck here between rock and hard place in many different ways. Writing there own new OS is not a magic silver bullet they have tried this and found this out already..
Flatland_Spider,
oiaohm,
Google has tried this with Fuchsia and others.
I agree, google had the opportunity to do this with Fuchsia before they laid off thousands of employees. I get the impression that major management shifts at google refocused on existing markets and moved projects that weren’t already profitable to the chopping block. It would have been nice to see what Fuchsia could have brought to mobile, and google is one of the few companies that could make significant waves happen. Many google projects are cool until they end up in the google graveyard.
Alfman
–I agree, google had the opportunity to do this with Fuchsia before they laid off thousands of employees. —
While google had Fuchsia staffed they were unable to get a single SoC vendor to come on board to provide drivers.
This is a major problem. Google can employ all the staff they like but if you don’t have SoC vendors on board there is no way for your kernel support at SoC release to have drivers for that SoC.
Then google tried making their own chips. Then found out IP(silicon design) licensing is horrible for their own OS. Linux kernel driver provided. Lot of cases the technical information to write driver from scratch is not provided. If you want your OS supported and you want to use their silicon IP many cases you have to pay them to code the driver for you if your OS is not Linux. Yes sometimes the terms on the silicon IP forbid writing your own driver without paying the IP owner for consultancy.
There is a reason why apple silicon turns out to be so much apple own silicon design. The barrier to entry here is very high it going to involve spending lots of money your competition will not be spending..
oiaohm,
I find that hard to believe as google have loads of influence. If executives were actually committed to the Fuchsia phone project (which they weren’t), not only do I believe they would get partners, but they’d even be able to place orders for their own custom silicon, just as apple did. That said, if you can link to a source that supports your claim I’d certainly go read it.
Alfman it was 7 years ago Google put out formal requests for soc vendors to come on board and they did not with Fuchsia.
https://www.freertos.org/partners/semiconductor.html
This from FreeRTOS Fuchsia never got.
Like the second generation nest hub using Fuchsia with Amlogic cpu inside but you don’t have Amlogic as an Fuchsia developer. This is a trend every device that Fuchsia has been used in the Silicon vendor was not onboard with Fuchsia development.
Remember Microsoft with windows on arm64 gave quacomm exclusive contract to come on board at first. But this is going to push you price up because the silicon vendor will know they can charge you arm and leg because you are locked in and cannot go else where.
Google has not given up on Fuchsia yet but Google is simple failing to get silicon vendors on board. Using Fuchsia is price sensitive devices at first is kind of a mistake that does not give you the room you need to grease the hands of silicon parties like quacomm, broadcom and so on to get them on board. And you need them on board to have driver development starting before silicon is released.
“loads of influence.” Microsoft has load of influence as well and to get Windows support by arm64 silicon vendor Microsoft was forced to sign exclusive contract and pay billions.
Yes there is a saying in the silicon world. That it cheaper and simpler to using 1000USD dollar bills at toilet paper than develop silicon chips. So companies like broadcom/qualcomm and so on are very worried about profitability so avoid speculative OS development unless they are going to be compensated..
Yes there is such thing as a real 1000 dollar USD dollar it disconnected note type and collectable. Yes your toilet paper running at 100000USD a piece at the time that old saying in the silicon world is from today those notes go for over 1 million a piece. Yes there is another joke hidden from the discontinued date. The 1000 USD bill was discontinued in 1969.
This you understand why Apple went the custom silicon route. Also google is trying to go the custom silicon route and also finding it very expensive. Cost of custom silicon order align to volume bigger your order cheaper the fab will make your chips for. Lots of cost getting a new OS off the ground. One of your largest costs is going to be getting silicon vendor support or becoming your own silicon vendor to solid prove viability to the other silicon vendors.
oiaohm,
Did you share the wrong link? The one you provided doesn’t mention fuchsia or google.
Can you provide any sources that back your claims?
Of course it’s expensive. But we’re talking apple and google here, not some mobile startup.
–Of course it’s expensive. But we’re talking apple and google here, not some mobile startup.–
This is kind of shock do note something.
https://www.counterpointresearch.com/insights/global-smartphone-ap-market-share/
When it comes to silicon with Google with likes of their custom Tensor chip we can see Google direct market share in smartphones is less than 1%.
There are mobile startups with more smartphone market share using one of the major vendor soc chips than Google has on their own silicon.
This is the base problem if google said every phone they made would have X vendors soc chip if they provided Fuchsia this would be less sales than decently backed by carrier mobile startup will do.
Yes the means to see how small Google direct market share is in the smartphone market has came clear since they started using their own Tensor chip in the pixel phones so spitting their own numbers out into their own pile on the likes of above numbers.
Apple has direct phone sales and they had very hard time dealing with soc vendors. Google in hardware has very few sales in fact.
Alfman you are over presuming how much power google has. Hardware side google makes lots of noise but themselves have very little power to change anything.
oiaohm,
I’m well aware of google’s hardware marketshare, I’ve cited similar numbers myself. But suggesting that google don’t have influence over the mobile market because they don’t sell much hardware is the same as suggesting microsoft don’t have influence over the PC market because they don’t sell much hardware. In both cases their influence and resources are much greater than their own hardware sales.
You could argue they can’t pull it off because they are mismanaged, then I’d have more reason to agree with you. But I don’t agree with your arguments about google lacking money or influence, these are not google’s problem.
Have Googl e found a zeroday hack or something that they can commercially exploit? My trust level in the MegaTechs are slim to none. As for AndroidTV, That’s my next foray in the near future with the NVidia Shield as AppleTV is way too walled off.
Hey Thom Holwerda,
This update from Google with respect to the Generic Kernal Image (GKI) prerequisites for Android gadgets is huge. Here is a breakdown of what it means:
1. Requirement Expansion: New televisions sending off with Android television 14 or later, running on Linux portion 5.15 or higher, should meet Google’s GKI necessities for confirmation.
2. Enforcement on All Major Form Factors: GKI is currently required for all significant Android structure factors, including handhelds, watches, car gadgets, and TVs, that utilization AArch64 chipsets.
3. GKI Benefits: GKI guarantees that the piece sticks to explicit prerequisites, isolating SoC (Framework on-Chip) and board support into loadable modules instead of center bit parts. This approach is pointed toward working on long haul support endeavors for Android gadgets.
Fundamentally, this get by Google plans to normalize and smooth out part the board across different Android gadgets, upgrading similarity and backing for future updates and advancements.
msbi online course
Thanks
(Marcos)
What does the MSBI course have to do with the rest of the post? Spam?
Flatland_Spider,
For a first post, chances are this account was created for link spam.
The amount of work and detail that went into the post would have been pretty high for a human working for a spam company. I believe this could be a case of spammers turning to AI to generate spam comments. I gotta say, with the exception of the spam link, it’s done an exceptional job coming across as human.
I doubt there will be a responses this time, since it’s not a human, but in the future spammers could start automating deeper posts to steer discussions towards their spam.