For quite a while I hesitated to write this story, but I decided that now it is time. It is a story of misinterpretation and bias that come from people that we consider as leaders in the software industry, the things they proclaim and promote and how it is related to peer-reviewed research and cognitive biases.
One year ago, in October 2017, I got engaged in a Twitter discussion about a controversial topic of gender disbalance in the software industry. I can consider myself an industry veteran since I am involved in software development since the age of 14, so that would make it 30 years of experience or something. I definitely agree that for many reasons, some obvious and some not that obvious, it is mainly dominated by men, but until a few years back I was really paying attention to this. We all know as a matter of the fact that many industries have gender disbalance and it is not always easy to find the cause for that. During the last several decades, the gender inequality has been diminished significantly in many countries, where European countries were among the leaders in this movement. Still, I live in Norway for the last seven years, and in two companies where I worked during the last six years, almost all developers were and are men. In Statoil, where I worked during my first year in Norway, the gender disbalance was not that extreme. Some teams there had even more women than men. I find this situation fascinating and somewhat disturbing, so I am still curious about what I personally can do in the company I work for now, to improve the diversity. Note, that this situation is only applicable to the technical side of the organisation, while the rest (majority) of the organisation has a great level of gender diversity.
So, I engaged by replying in this Twiter thread:
Here is my actual reply:
My goal was to share my personal experience at the concrete moment and place, and maybe get some useful advice. But, as usual, Twitter conversations are subjected to assumptions, when people prefer telling others what to do and what to think, without knowing their context.
Shortly after that, Jez Humble himself decided to engage too. Here are his replies:
Here, I stated the fact that is based on my personal experience in life. I have not changed my mind since. It is impossible to achieve equal gender distribution across all profession and activities. For example, it really bothers no one that we see fewer women doing plumbing, masonry or working in coal mines and offshore drilling platforms, just because these are dirty jobs that often require applying substantial physical strength, working in harsh conditions and include a substantial change of things to go wrong with severe consequences for health, and sometimes causes death by accident.
Jez, however, used a straw man of the doctor example. Indeed, the industry used to have a very high status, and there were not that many doctors around before the beginning of the 20th century, and all doctors were men. However, here comes the first sign of limited cultural background of a privileged white male from a “first world”, who was born in the UK, went to a private school and got a degree in Oxford. Let’s look at this “status” statement and what really happened in medicine in the 20th century. As a person born in the 1970th in USSR, I perfectly know that medicine there was dominated by women. As a matter of fact, in the United States, the restrictions for women in the medical field were lifted in the early 1970th by Title IX of the Higher Education Act Amendments of 1972 and the Public Health Service Act of 1975, banning discrimination on the grounds of gender. On the other side of the globe, in USSR, the results of the 1970 population census showed that 70% of people in the medical industry were women. Why could this happen? Of course, the WWII played a significant role in that, decreasing the part of the male population significantly and putting more women in hospitals, who then were able to continue their education to become doctors. Apparently, there were no gender restrictions on medical education at that time. What does this all have to do with status? Well, the only “status” that one can get during the communist regime, was based on ideology and high position in the management chain. The proportion of women in management positions were very low. Being a doctor meant nothing regarding status. As soon as the restrictions that were imposed by men were lifted, this professional field quickly became dominant by women. The disruption of the class system that happened after 1917 in Russia led to significant changes in what was considered a higher-class profession, like a doctor or an engineer. However, we still observe industries were there were and are no restrictions based on gender, where the gender disproportion is significant. It appears to have little to do with class or origin. I will be getting back to classes and inequality later.
But now I want to get back in time, to 1989. I was 16 years old at the time. Two years before that my mother insisted that I do my secondary school in a better facility. So, I started in a math class in another school in the city centre. There, I got exposed to programming using a programmable calculator. In 1988 we were first exposed to BK-0010, the first consumer-grade PC in USSR. It had 32 Kb of RAM, from which 16 Kb were used by the screen, so programs were limited to operate in 16 Kb of RAM. I quickly got bored playing games and started learning Focal, the programming language that eas built-in to those computers. Then, I learned assembler for that processor, since there weren’t many things you can do in 16 Kb or RAM using a high-level programming language. But, the story derailed a bit. Let’s look at my class, here is the photo:
This is the math class, which was considered to be a prestige place to study. However, everyone could have joined. I didn’t need to pass any tests, they only looked at my grades, and I had an interview. I was raised by a single mother, living in a flat with a single room and a kitchen. We didn’t have many connections and no money. My mother worked as an engineer-programmer for a research facility that was designing parts for submarines. The facility wasn’t classified though, so I was able to visit my mother’s workplace and see those huge machines with huge disks, loads of perforated cards and so on. That’s where I probably got inspired and influenced towards and programming career.
Nevertheless, our class got exposed to computers as a part of the school-based professional education, where we spent one day per week learning something that can be helpful for the further education. We had a choice to learn about refrigerators, do something work computers, learn how to drive a truck, and how to operate a sewing machine. Most of our class chose computers. Later, we got computers at school too. Look at the class photo again. You see there 18 boys and 11 girls. Not that bad at all for the math class. But now guess how many of us were really into programming? Well, I tell you — five or six. How many girls? None. I have no explanation for this, but one thing I know. All of us who spent more time with computers than going out and stuff like that were seen as geeks and nerds. We weren’t cool, we weren’t popular really. Well, until the point when the programming exam became a requirement, then we got very popular. For a week. Could that be a reason why girls weren’t interested in that subject at all? Maybe. I have no data, no research, I only have my experience as the secondary school student.
Interesting enough, most of my mother colleagues were women. None of them had any sort of a computer-related education since it didn’t even exist at the time. My mother graduated as a math teacher, so as many of her colleagues, but some were engineers from different fields. The engineering jobs weren’t related to any status or class, and the income was average or below. Now I know that despite 90% of women in USSR worked, they earned significantly lower than their male colleagues. The concept of stay-at-home wife almost didn’t exist, unlike in the US, where in the 1950s only 26% of married women worked although that changed and by the mid-1980th the percentage grew to 67%. However, in the US the increasing number of working women was part of the housewife liberation movement. Communists started “liberating” women already in the 1920th. But this “liberation” turned to become a fake. In addition to all the housekeeping “traditional” duties at home, women started to work full-time. The patriarchal nature of the society didn’t change. Since in USSR nearly all people were equally poor, it was not possible to hire someone to do the housekeeping. The whole picture is quite clearly demonstrated in the Moscow Doesn’t Believe In Tears film that won the Academy Award in 1981 as the best foreign movie. In one scene, a successful woman who works as director-general of a large factory, is in the suburban train sits in front of a friendly guy and they start talking. He said — “You aren’t married.” “How do you know?” she asks. He replied — “You are looking at my shoes. There are two types of people that can look at the shoes like you do. Managers or not married women. You can’t be a manager, so you aren’t married”. They start dating, but at one point the guy finds out that she is indeed a high-profile manager and earns much more than he does, although he is a very high-rated mechanic. This brings him to depression, he breaks the relationship and disappears, later to be found drinking and explaining that the whole situation is unacceptable for him as a man. After his disappearance, the lady spends time crying and reflecting on what has she done wrong (well, nothing, despite being a successful woman) and makes an effort to bring the guy back. Surely everyone in the country could understand that just because that’s how the society worked
For me, those 70 years of a social experiment on hundreds of millions of people is something that cannot be easily ignored, like it didn’t happen. Opening doors for everyone to go and choose what they want to do is not enough. The economic and social pressure, of course, play a significant role in that too. Sometimes, it is even hard to understand the gender disbalance in some professions. For example, in USSR nearly 100% of bus drivers were men. But also, nearly 100% of tram drivers were women. For trolleybus, strangely enough, it was like 50/50. I have a very little clue about the reasons for this, but surely there are some. With that in mind, we come to the Gender Equality Paradox.
Here we come to the second part of that conversation. To remind you, my goal there was to share personal experience and maybe get some advice. Here is where I came by at last:
Here, I see the magic words peer-reviewed science coming from Humble. I wasn’t paying attention at the time, but it will become more important later, and I will have to get back to it. So, let’s see. It appears to be that I am “wrong”. I have to admin I sued this word first, and it was not smart. At the same time, what was I wrong about in this particular context? Norway is the number one in the UN human potential development index; it is a matter of fact. I also wrote in response that here, people get a free choice what to do with their lives. The contrast is visible when looking at this:
What is the difference between Norway and Malaysia, where, as we know, there are a lot of young women choosing their career in the software industry? The answer is obvious, and there are researches that confirm that. It is a matter of status. There is no other way for girls to get respect in the society and earn a substantial income. It is not the case for a country like Norway.
Let’s look at the paper that Humble pushes me to read. It is a research paper called Countries with Higher Levels of Gender Equality Show Larger National Sex Differences in Mathematics Anxiety and Relatively Lower Parental Mathematics Valuation for Girls by Gijsbert Stoet, Drew H. Bailey, Alex M. Moore and David C. Geary, published by PLOS in 2016. The study focuses specifically on the question of how sex differences in mathematics anxiety are related to societal and family variables, trying to establish a link between the anxiety of girls in math with the lower percentage of girls choosing STEM (Science, Technology, Engineering, and Mathematics) education and career in developed nations. They used the data from the Programme for International Student Assessment (PISA).
The study concludes that the hypothesis of parental influence on their children is not supported by the analysed data. Also,
We propose that while economic considerations may play a more prominent role in STEM-related interest for individuals living in less developed countries, intrinsic subject-specific interest will play a more important role in educational and occupational attitudes and choices for individuals living in countries with higher levels of economic well-being. When the relative role of interests become more important than the financial drivers, and when men and women have more freedom to pursue their intrinsic interests, the well established sex difference in occupational interests will become more strongly expressed.
Further, we propose that the influence of parental opinion on children’s mathematics anxiety is not well-established, and that correlations may reflect the influence of children’s interests on their parents’ opinions at least as much as parents influence their children’s interests. We hypothesize that when parents are asked by researchers how important they value mathematics for the future of their children, parents will likely take into account the levels of mathematics-related anxiety and interest in mathematics that children express at home and with respect to their mathematics experiences in school.
Here I have to specifically mention that the researchers have a remark that the parental opinion and expectation that girls are anxious in math is not being observed in Scandinavian countries.
In short, the conclusion is in countries where people can have a free choice when deciding what subject to pursue in their education and career, some more profound expectations from the society play a more significant role. Some of these expectations are coming from parents; however, in Scandinavian countries, this is not the case.
Interesting enough, although Humble chooses to use this particular study to illustrate me being wrong, by stating this issue as a confirmation of some well-established fact, that is not the case. That very fact that in countries with better gender equality the proportion of girls choosing STEM for their career is smaller is known as the Gender Equality Paradox. You can easily find a number (not just one) studies about it.
The recent study The Gender-Equality Paradox in Science, Technology, Engineering, and Mathematics Education published by SAGE in February 2018 comes to the following findings:
Girls, even when their abilities in science equaled or excelled that of boys, often were likely to be better overall in reading comprehension, which relates to higher ability in non-STEM subjects. As a result, these girls tended to seek out other professions unrelated to STEM fields.
Interesting enough, they also conclude the following:
Wealthier nations tend to have greater economic opportunities, allowing women to make choices based on factors beyond mere economics. In poorer and less gender-equal countries, however, women find that employment opportunities aren’t so easy to come by, and so the security and good pay that comes with a STEM career attracts more women.
The latter result perfectly correlates with my statement above that concerns inherent inequality and class issues in less developed countries, which gives an additional stimulus for girls to seek a career in technology, to improve their social status and gain more substantial income. We can also correlate these findings with the male doctor argument of Humble. This argument only works in societies with high inequality and an established class system. It’s worth noting that both the United States, where Jez lives and works now, and the UK, where he grew and got his degree, have a significant level of inequality and also have an established and rigid class system, where people are being discriminated by their origin probably to a similar or even higher scale than their race or education.
What is even more fascinating in the whole story, is that the authors of the study that I just cited. It is written by the same people, who wrote the paper that Humble sent to me to point me of being wrong.
The newer paper suggests that:
But when it comes to their relative strengths, in almost all the countries — all except Romania and Lebanon — boys’ best subject was science, and girls’ was reading.
The authors conclude:
What’s more, the countries that minted the most female college graduates in fields like science, engineering, or math were also some of the least gender-equal countries. They posit that this is because the countries that empower women also empower them, indirectly, to pick whatever career they’d enjoy most and be best at.
So, here comes the Choice thing with a capital C, which Humble was so sarcastic when replying to my tweet stating the same thing. It appears that my “political slogans” as he chose to call them, are in line with the newest finding by the same people who wrote the research that Humble advised me to “choose” for reading. It is quite fascinating.
Quite interesting is also to look at this graph:
My attention was drawn to the three outlier dots on this graph for Norway, Finland and Sweden a bit behind, which confirm quite well the conclusion that parental expectations of girls being successful in math, in Scandinavian countries influences the number of female STEM graduates in these countries.
There is, however, yet another side of the coin that is expressed by the publication on The Nordic Gender Equality Paradox website and book, suggesting that:
Nordic welfare states are — unintentionally — holding women back. Public sector monopolies and substantial tax wedges limit women’s progress in the labour market. Overly generous parental leave systems encourage women to stay home rather than work. Welfare state safety nets discourage women from self-employment. On the other hand, the much-avowed affirmative action laws in Norway have not helped further women’s career possibilities.
However, this is not peer-reviewed research, although it contains some sound thoughts.
Another paper called Not Lack of Ability but More Choice by Ming-Te Wang, Jacquelynne S. Eccles and Sarah Kenny, published in March 2018 by SAGE, suggests that people with higher abilities in math but lower ability in reading are likely to choose the STEM career. At the same time, people with high proficiency in both math and reading are less likely to choose STEM career and instead do something else in life. They conclude:
Our study provides evidence that it is not lack of ability that causes females to pursue non-STEM careers, but rather the greater likelihood that females with high math ability also have high verbal ability and thus can consider a wider range of occupations.
Fascinating again, the word Choice now appears in the article title.
The topic, however, remains highly controversial. If you read comments for the article by Stuart Reges, you’d find concerns about the nature of choice that might be oppressed by expectations laid down on an individual by parents, school teachers and people around. That discussion is very interesting and despite papers are being published, there is no consensus for the topic. Even more, societies around the globe aren’t the same. I could easily apply some comments about expectations and attitude of parents that influence the choice of their daughters not to pursue the STEM career. However, I am more interested in what happens in Scandinavia and Norway in particular, because this influences my job and my life. Studies show that the expectation of parents in math performance towards their daughters and sons are very close in Scandinavian countries. Also, I cannot confirm that women candidates are being discriminated, undervalued or excluded from the hiring process based on their gender, in any of the companies that I worked for in Norway. Quite the opposite, we tend to prefer female candidates to bring more diversity to our organisation. It is quite weird to talk about gender discrimination when it comes to a country that has a female prime minister, minister of finance, minister of local government and modernisation, minister of foreign affairs, former minister of defence and so on. I am still not sure about the exact reasons for not getting enough female candidates for the company I work for at the moment, but it might be something related to our location. In bigger cities like Oslo, Bergen, Trondheim and Stavanger, the situation is vastly different. The people we have often have their degree obtained abroad, I might only guess that for some reasons girls decide not to do it. Although I also know that on the career choice meeting at our school, the local representative of the Norweigan social service clearly pursued the agenda that girls need to consider joining the technical career since there is and there will be the industry where the demand is large.
So, what does this all have to do with biases? This year, my main talk for conferences where I speak is Simplicity versus Simplification. In this talk, I try to push people not to oversimplify problems that are complex by nature. One of the reasons that we tend to seek simple explanations for complex matters and why we prefer to choose the first solution that comes to our minds and ignoring all other possibilities is the confirmation bias.
This bias forces our fast-thinking system (System 1) to carefully consider the facts that confirm what we already believe in and ignore everything that doesn’t fit this picture of the world.
How would you see the situation, when a person refers to a research, written by a group of people, as a peer-reviewed science, and at the same time is not willing to look at the new paper, written by the same people, that contains more findings on the same topic, but those findings aren’t exactly what that person wants to hear? For me, that is a classic example of the confirmation bias. I never thought I’d need to think about it about Jez Humble. The guy is one of the industry leaders, who wrote a highly-influential, award-winning book Continuous Delivery, and recently published several peer-reviewed papers on that subject? What is more fascinating, Humble and some influential ex-ThoughtWorks consultants consequently refuse to take critique of these papers and the findings that are summarised in the new book Accelerate by Nicole Forsgren, Jez Humble and Gene Kim, by using the magic words peer-reviewed research?
Assuming that Jez would be interested to read some newer findings on the gender disbalance topic and the Gender Equality Paradox, I approached him by email a few weeks back and sent some links to the peer-reviewed science. To be honest, I wasn’t expecting him to even remember that discussion on Twitter and didn’t have much hope for getting a reply. In the end, he is a busy man and an expensive consultant. However, I got a reply the next day. Jez started by saying that he decided to break the debate because I was “wasting his time” by ignoring the links he was sending. Interesting enough, I have read them all and was writing about it the next day.
Then he stated that he has no time and no interest in reading anything that I sent him on the subject because he chooses to have the least possible distraction in life. Hmm, that sounds a bit weird already, it the peer-reviewed science at the end? The tone of the text was getting more expressive, and I almost felt like letters beginning to glow with impassion and anger. The final words were that he usually doesn’t engage in personal email communication like this and that it was all very frustrating for him.
Wow, that’s nice. First of all, after almost a year, he clearly remembers this thing. There comes something personal on the line and then comes frustration alongside. The research I sent was again written by the same people, that seem to continue their study about the fundamental reasons of the Gender Equality Paradox. But it looks like the confirmation bias is very strong here, one doesn’t want to see what’s not fitting to the already formed picture of the world. One gets frustrated when this picture become disturbed, and new facts and new research needs to be considered. That is work for the slow-thinking system. But when reading that answer, I clearly hadn’t seen any sign of the System 2 even coming online when the email was written. That fact saddens me the most.
I am not famous nor that influential. My work is about software architecture and sometimes that involves working with the organisation as a whole, trying to make the software delivery processes better. I work with people all the time, for quite a while I work with the same people and see them grow (or not), new people joining and some people leaving. I try to learn from all my experiences, but I am really not sure what I learned here, except that someone I respect can be an asshole. Of course, the whole thing brought me to deeper discoveries and interesting learning. Oh, yes, I know. I understood how I would never behave towards other people, learning by example.
Now comes a big question. I observe a clear case of WYSIATI (What You See Is All There Is, from the book Thinking Fast and Slow by Daniel Kahneman) applied to a person that is positioned to be one of the leaders in the software industry. He writes influential books, which people use to improve their organisations and probably their lives. The last book, Accelerate, is based on peer-reviewed research, but what that could mean to a person that is only willing to see and listen to something that confirms his beliefs and thoughts? Where is the critical thinking, where’s an ability and will to accept something that might be not very comfortable to admit? Where’s a realisation that some topics are complex by nature, especially when it comes to social and psychological sciences, where there are no simple answers, and there is no right or wrong? The final question is — how unbiased is the work of Humble when it comes to continuous delivery? What degree of certainty is there when the magic words peer-reviewed research is being used as a substitution of “the only valid answer?” In the end, the sociotechnical nature of the topics that the Accelerate book touches aren’t straightforward either, they go more towards cultural, psychological and anthropological aspects of organisations as complex adaptive systems.
I’d like to finish by citing Martin Fowler’s foreword for the Accelerate book:
So, as you may expect, I’m delighted that they’ve put this book into production, and I will be recommending it willy-nilly over the next few years. (I’ve already been using many bits from its drafts in my talks.) However, I do want to put in a few notes of caution. They do a good job of explaining why their approach to surveys makes them a good basis for their data. However, they are still surveys that capture subjective perceptions, and I wonder how their population sample reflects the general IT world. I’ll have more confidence in their results when other teams, using different approaches, are able to confirm their reasoning. … Such further work would also make me less concerned that their conclusions confirm much of my advocacy — confirmation bias is a strong force (although I mostly notice it in others ;-)). We should also remember that their book focuses on IT delivery, that is, the journey from commit to production, not the entire software development process.
What also bothers me is that we have a plethora of people that tell us what to do and what not to do. When it relates to the software industry leadership, we can clearly see a bunch of privileged white males with an Anglo-Saxon origin and Oxbridge or The Ivy League degree. They live and work in a highly competitive environment in places like London of the Valley. Very often, the atmosphere itself is poisoned by a power struggle, inequality, discrimination, harassment and mobbing. Then, these individuals find themselves in a position and will to teach the rest of the world how to live, work and behave, what to believe in, what is right and what is wrong. What they often fail to understand is the inherent diversity of the world. I lived a relatively simple life, but I grew up in communist Russia, have seen the country disappearing from existence, ruining lives of millions and opening doors for those without principles and morale. I saw tanks on the streets of Moscow in 1991, and I heard the sounds of bullets very close in 1993. I had a career is a bank and become the IT head there, then chose to move somewhere else and found myself in The Netherlands, where I worked as a developer and earned less than I spent during the first year or two. I was an entrepreneur and consultant, learned bookkeeping and accounting (the industry dominated by women) and many other domains, lived in The Netherlands, Switzerland, the UK, Belgium and now Norway. I learned how vastly different people could be even when we say Europe, among all those different countries. This brilliant diversity in culture, experiences, lifestyle, believes and goals make us who we are. I can’t accept that some single-cultured dudes that speak no other language than their own and have learned about the world from white papers and academic studies would influence the way we think, the way we do things, outside of their area of expertise, and question even that.