blocks

Building blocks: what you should look for from open source code

Engineering

3/31/2021 11:00 AM

Open Source GitHub Web Development

Open source code makes up most of your software supply chain. But how do you choose one open source product over the other?

Everyone uses open source. That's what made a question we received from a customer so interesting. They were getting questions in turn from their own customers, asking why they used 51Degrees’s services rather than an alternative and our answer to that perhaps deserves a wider hearing. Our libraries and the alternative are both open source, as most code is; now that this is no longer a discriminator, what do you use to choose between products? Knowing how to make that choice is key to your tech strategy: here we’ll explain things beyond code openness that you should be considering. We’ll look at how open source makes up most of your software supply chain, even if invisibly; how it’s critical for you to understand who builds that software and why they do so; and how you can help shape that software so your company benefits.

You might make a SaaS product or a hardware platform; devops and sysadmins have scripts and management tools. All the software your company writes contributes to your value-add, but most of the software you write is not actually written by you! It's libraries and tools written by someone else and which you use, and there's a good chance that almost all of that is open source code. It’s like an iceberg: the small part of your software at the top, the bit above the water that people see, is the part that you make and where you differentiate. The much larger bottom part is existing open source code; it's the engine that does the bulk of the work at your direction.

iceberg

And it's important to understand where it comes from... and more importantly where it'll come from tomorrow. The importance of cloud deployments illustrates this; most cloud technologies are open source and they all make extensive use of open source, and CIOs and managers and developers all prefer them.

Your Supply Chain

That's your supply chain: all the software you write is critically reliant on the existing libraries that form your foundation, the building blocks that you build upon. When Isaac Newton made his point about standing on the shoulders of giants, he was actually talking about Github (probably). As the overseer of that supply chain, you need to understand where it comes from, and what ensures that it'll continue to be available to your company.

There are a number of myths that are told about open source software. One tells of a community of amateur developers working for free to build something robust and useful. Countless companies have fallen for this myth by publishing code for a project and assuming an army of developers will appear to fix bugs and maintain it without having to pay them... which will not happen.

A properly prepared company can say, for each of their software dependencies, what the developers' incentive is to keep providing it.Stuart Langridge

But there is a counter-myth, which is that an open source project with a company involved will be magically more reliable than an open source project without. This is equally untrue, and is normally implied (or outright stated) by someone trying to sell you something. You need to look at what motivates the project contributors in each case.

Understanding Contributor Incentives

The incentive might be the simplest one: you use their software and pay the company for a support contract. Their reason to continue maintaining the software and listening to your feedback is that you'll keep paying them to do so. The incentive may be collaborative rather than directly financial. Many open projects are maintained and developed by a community of companies and developers collectively, because they all need that project to go on existing and keep working. Companies contribute changes that are directly relevant to them, or (and better) staff developer time to the project even on work that doesn't directly benefit them because it ensures the overall health of this code that they depend on. Hadoop, for example, is one of these; multiple companies help maintain it because it plays into their commercial offering, or complements it. Helping your product's complements is a valuable use of time; there's no need to give away the razor and sell razor blades if you can help someone else give away the razor instead. Whole industry segments are based around a shared open source core with each contributor taking that core and selling products based on it.

community

For each contributor to the software, look at what their motivations are, and what might change those motivations and stop them from continuing to contribute.

Contributor Their Incentive Risks to you
Primary commercial sponsor This is a key product for them The company fails, pivots, or is purchased. The strategic direction changes away from what you need
Company using software in a product They have products built on this The company fails, pivots, or is purchased. The company replaces this dependency with a competitor
Developer at company on company time Their company pays them partially to work on this This work is deprioritised. The developer leaves without backfill
Developer in personal time They use the software, or enjoy working on it, and may get perks or visibility from the commercial sponsor The developer gets a more demanding job, or a change in personal circumstances means they have less free time

The key point here is that it doesn't necessarily matter what these incentives are, so much as that you understand them and think about them critically. For each of these building blocks, ask yourself: the people and companies and community who make this, what motivates them? What might change that motivation? What if the major funder cuts funding, or the project team are acqui-hired by your competitor, or the major contributor changes job?

This is the same due diligence you’re already doing for expensive proprietary software, of course: how do I know this partnership will be viable in a year, or five years? What are my break clauses in this contract; what happens if they go out of business; what happens if they're acquired, or pivot? You consider stability and business continuity; that doesn't change when the software is downloaded from Github rather than purchased with a signed contract. Open source projects may not necessarily come with contractual guarantees, but it may not need to because much more of the process is available for your perusal. You can tell whether this building block is actively maintained, is kept up to date with industry changes, is worked on by a diverse and wide-ranging community or has a bus problem by looking at the history of the project in public.

discover

This is exactly why we practice what we preach here; the 51Degrees libraries and developer resources are deliberately maintained in the open and well-documented exactly so that you can see why you would choose to use that code and our services. We dedicate time and resources to documentation and guidance and support because these are vital parts of a complete open source offering. Business continuity is critical; a good open source project can convincingly demonstrate to their users that the project is safe to depend on today because it will still be here tomorrow.

These are two sides of the same coin; a large open source project, especially one largely maintained by a single company, should often be treated as de facto proprietary software. If the open source software you're using is too large or too complex for anyone else to meaningfully contribute to, then go through the same procurement process as you would for a commercial and proprietary software product even though you're not actually paying or signing a contract.

Not somebody else's problem

The big difference now that the industry is primarily using open source is that you are not merely an observer of this process; you are part of it. The weakness with proprietary (or de-facto proprietary) software is that if it doesn't work how you need it to, there is little you can do about that. If your assessment is that this project needs more help, you can do that; financial contributions to allow the project to spend more money on maintenance, or by having your own dev team spend paid time taking a lead in that project’s maintenance and development. Make this part of your due diligence: if this project went away tomorrow, what would that cost you in time and effort? How would you replace it? How much would that set you back by? A little extra now can save you a lot of expenditure later; read Stephen Walli on software sustainability for a detailed study.

But with open source you can influence the strategic direction and the viability of the building blocks on which you depend.Stuart Langridge

Many, many open source maintainers are declaring that they're close to burnout on the critical projects they maintain without support. Álvaro Trigo wrote about this recently, and the Heartbleed debacle demonstrated that it is a risky strategy to depend on critical open source building blocks while assuming that maintenance is somebody else’s problem. You can help here by leaning in, with money or time from your dev team; you might also consider paying a professional services company such as Igalia for browser development or Tidelift for maintainer support who do that on your behalf.

There may be a space in the market here for professional advice to CIOs on how to best assess your open source dependencies, and where your support of them is best targeted. This could, and should, be a whole separate specialism; high-quality, sober consultancy on where your company may be exposed in your use of open source and how to mitigate that exposure. This is not something that's well-served by automated reports to be ignored in your inbox every Monday; it's a holistic look at the continuity of the hidden 90% of your company's software products. Someone should take that idea and run with it.