Automating the little things

As the mainline porting workflow becomes more streamlined, we found ourselves able to automate and simplify a lot of things that previously required a lot of manual work. A great example of this is the mdss-panel-driver-generator which is able to convert vendor devicetree panel control sequences into a driver which meets the requirements for upstreaming into mainline.

More recently, tools like the new msm-firmware-loader for postmarketOS, and the potential for scripts which can extract and package firmware simply given a link to an OTA update, we might now have a generation of porters with little/no knowledge on how firmware works in the context of downstream / mainline. Qualcomm platforms differentiate between mdt firmware files where the firmware is split into many files, and mbn files where it is squashed into one larger one. Most downstream devices have the firmware split into the `mdt` file and a bunch of files with the data in, but for mainline we tend to combine them all into an mbn file. If there is a tool which takes care of this all, we may end up in a situation where only a few people in the community have active knowledge on how it works / how to maintain it, and why it works – particularly as in this case there is little documentation other than code, and what docs do exist are often littered around various parts of the internet. Clarification: whilst there are some issues with documentation being a bit tricky to find, in the case of these two examples documentation is available, I used these projects as an example of where a task was automated, and how that /can/ lead to knowledge becoming more scarce. I did not mean to suggest that these projects do suffer from these issues. As long as these tools continue to be maintained and supported, they can save a lot of time for new developers, and offer a fallback in the case of a device where the firmware is device-specific or packaging it would pose other issues. For those interested the msm-firmware-loader code is packed with comments which explain in detail how it works, this is a great example of a tool which is designed to last for a long time and be maintainable by anyone with some knowledge of bash.

The current generation of phones are proving to be a huge challenge for new porters, requiring that whole new Android tools be ported to Linux just to do something simple like mount the userdata partition without formatting it (new device porters like the idea of dual booting, as often the phone they wish to port to is expensive and their daily driver). Mounting the system or vendor partitions on any device which uses "dynamic partitions" is still not supported by postmarketOS and barely supported on a non-bionic (not Android) userspace.

Tools that work to simplify the process of porting new devices and make it easier for new developers to get involved are always a positive for the community. It is a really great sign to see people creating solutions for pain points such as extracting / packaging firmware for many devices, it shows that people in the community are passionate about the goal of supporting more devices, and helping new developers.

So, are we making the porting process "too easy"? Probably not, we're still at least 2 generations behind the latest Android devices, and we need all the help we can get, simplifying the stuff that we as a community now have a good understanding of lets us spend more time on new challenges.

Documentation is something that we should probably take a lot more seriously, especially around Qualcomm devices where a lot of discoveries and knowldge end up being lost to the matrix chat history. I would love to summarise this article with a list of ideas - or even just one - that could help make things better, but I simply don't know. Writing documentation takes up a lot of the usually quite limited time of mainline Linux mobile developers, despite this postmarketOS have an absolutely stellar wiki, containing pretty much everything you'd want to know about the OS. It's just a shame the same can't be said for SoCs.

Conclusion

Automation and good tooling can save developers a huge amount of time an effort, this is especially valuable when time is already at a premium. postmarketOS have proven that by creating good resources and making it easy for people to get Linux running on their phones even with little to no embedded programming experience, you can attract a lot of interested people. The over 300 booting devices should be evidence enough of this.

However, with new developers not having to learn how the tooling works, we can end up in a situation where when something breaks there are only a few or even just one person who can fix it.