Over the past 2 decades I’ve had the chance to develop software for a wide range of businesses: robotics, insurance, telecom, online retail, fintech, medtech, media and my own startup. Regardless of the diverse range of products and technologies, in my observation there’s a symbiotic relationship:
- Products shape the people: The type of product has a strong influence on what type of professionals find meaning in contributing to it. Sure people join and leave all the time, but the type of person who stays longer or has an influence usually has an interest deeper than the paycheck. This is more true for sectors which are competing for the labor market (eg. software engineering) and even more true for places where the pay difference is not big enough to motivate a job switch (eg. Scandinavian countries).
- People shape the products: Those employees in turn build the company culture which influence the product success or failure. This effect is stronger with visionary leaders who are invested in the cause of the product (eg. Steve Jobs and Elon Musk) but even stronger when the product itself is something the employees use (eg. Netflix).
It is easy for this symbiosis to turn into a closed feedback loop and create echo chambers immune to new ideas and innovation:
We’ve always done it that way.
There are two ways to break that vicious cycle:
- Change the product: this particularly hard if the product is doing well or tied to the identity of the company (like Windows).
- Change the people: and by change I don’t mean firing people, but rather increasing diversity which widens the spectrum of accepted ideas and raises the threshold of innovation.
But this article is not about organizational culture or product management. I want to rather narrow down the topic to how the product and people affect the risk appetite.
First let’s build a common vocabulary without getting too deep. Risk is the likelihood of loss to some sort of asset: data, equipment, trust, money, lives, etc. Vulnerability is a weakness in the asset that may allow a threat to cause loss.
The three are sometimes combined into an intuitive (but not literal) equation that looks like this:
Risk = Threat x Vulnerability
It simply means that the risk is high if:
- The threat is very likely even when there’s not much vulnerability. For example Whatsapp comes with the promise of E2EE (end to end encryption) so it has a low vulnerability but due to its high popularity, it contains a lot of valuable data (assets), which makes it interesting to many threat actors from hackers to governmental 3 letter agencies, which have the resources and motivation. See Pegasus for example.
- The threat is unlikely but the vulnerability is high. For example individual IoT devices don’t enjoy a high level of security due to limited resources and fragmented architecture which leads to higher vulnerability (that “S” in “IoT” stands for security 😉). The individual IoT assets like light bulbs or routers may not be that interesting to hackers but thousands of them can be used to create a botnet posing a serious risk. See Mirai for example.
Risk assessment is the process of evaluating threats, assets and their vulnerability to identifying and prioritizing risk. There are many risk assessment processes depending on the type of product or who is assessing it. The assessment is usually followed by risk mitigation which is all about coming up with a strategy for how to reduce the risk.
If you’re curious to learn more, Eli the Computer Guy has an approachable video about this topic.
Risk is not inherently bad. Ever heard of the “high risk high reward” phrase? Taking risks can potentially yield higher rewards. The key here is to take calculated risks which is easier said than done because a common theme in risk assessment is ambiguity.
The amount of risk a business is willing to take depends on many factors:
- The type of product
- The people behind the product
- The likelihood of the risk
- The type of loss that the risk poses
- The cost of the mitigation strategy
All things considered, the amount of risk that a business is willing to take is called its risk appetite.
Let’s have a look at 4 short annecdotes:
A few years ago I joined a global financial company. Upon start, they put all employees through a series of mandatory security training. The training covered many common sense aspects like having long & complex passwords but also some best practices that was new to me at the time like the “clean desk policy”: not to leave any critical information like customer data, PII (personally identifiable information) and certainly not any credentials in plain text at your desk. Then there was a whole lot about PCI compliance due to the nature of the business.
Since the stakes were high (money and personal information), we were all required to go the extra mile to protect the company assets despite inconveniences. Our product too went through elaborate quality assurance procedures to ensure despite slowing down the pace of development. This is not for everyone. Most employees I came across were competitive and ambitious young men who cared about numbers.
Later that year, I joined a global medical technology company which was in the business of cancer treatment planning using advanced software and machine learning. There I met some of the most dedicated and humble people who genuinely cared about “fighting cancer with code”. Some were cancer survivers while some others lost dear friends or relatives to cancer. Almost half of my colleagues were doctors, quality assurance and compliance experts. At the end of every 2 week sprint, my team sat together with a compliance assessment officer to report what we did and evaluate how each individual code feature or bug fix maps to the two core risks:
- harming a person
- causing financial loss
I was very surprised when our product manager asked us for a list of our 3rd party vendors to log for compliance audit purpose. We were using NPM which is notorious for the sheer number of packages it installs for even the most mundane projects. We had 900 of them at the time but eventually settled for only logging the first level dependencies.
Because of the heavy regulatory and compliance processes, we had an extremely slow release cycle: we released once per year! Each copy of the software sold for over a million US dollars and customers had to go through training to use it. MedTech is certainly a conservatie industry often lagging behind in technology due to the low risk appetite and that is fine! I certainly don’t want my hospital to run an A/B test on me or worse kill me due to bad UI which makes human errors more likely. See Therac-25 for examples of such fatal incidents.
A few years later I joined the identity platform at an international group. The platform allowed users to log in to each company of the group using a shared login and user database. Some of the companies were online marketplaces where money was involved. Ever heard of “the strength of a chain is its weakest link”? When it comes to risk, the opposite is true:
A poison is as deadly as its most dangerous ingredient.
We had to be extremly careful not to put the group’s name in jeopardy. Due to GDPR, Apple Intelligent Tracking Prevention and other 3rd party cookie elimination initiatives we faced many interesting challenges. I learned a lot about privacy and computer security working at that team.
After a few years, when I switched teams at the group to work with the media sites, I realized that the same level of security doesn’t apply to other parts of the company. Ironically for a business that makes money from content, it wasn’t simply possible to convince people to comply by security best practices. “Talk is cheap, show me the code”, a quote originally by Linus Torvalds was the mantra. So I ended up doing many proof of concepts to prove the point.
One company may have different risk appetites for different products.
The media companies are extremely fast paced compared to FinTech or MedTech. The stakes are simply not as high. When I started we had 1 release every 2 weeks. By the time I ended (a bit over 3 years), we had a massive inner source product shared with a community of about 150 developers who deployed 2–10 times a day without even involving the core team. Of course we went the extra mile to ensure the quality and reliability of the stack as much as possible but it goes to prove the point that when the risk is relatively low, the risk appetite is higher. We didn’t work with money, certainly not human lives and consequently enjoyed a much higher development cadence.
We didn’t have the concept of sprints or release management and had the chance to work with the latest technologies. We occasionally had a few days of release freeze (where the release was not advised but totally possible) like the national election or extremely important news events.
We talked about how the product and people live in a symbiosis and some of the ways they can affect the risk appetite. Every person who is added to an organization adds to its DNA bringing their experiences, opinions, ambitions and ideas. Every person who existed prior to that also has their experiences, opinions, ambitions and ideas. The company as a whole has a written or unwritten risk appetite that depends on its ambitions, competition and type of product. Hopefully this article could elaborate on that point.
What is the risk appetite at your company and how do the product and people affect it? Let me know in the comments below.