Why the Software Development Industry is Different - Part 3 of 3

by John Bridges - May 14, 2013

The final part of our insight into the software industry. In this post we look at the drivers and dynamics of the software industry


The software industry is a complex mechanism. Even then simplest features require a handful of code to make sure everything works well

© Neuron Global


Read part 2  here: "The Software Industry – Where Chaos Meets Technological Evolution"

Software is different than any other industry on earth. It is new.  It is abstract. It is complex. It has a vast, ever-evolving knowledge domain and it is rapidly changing. But what do each of these statements actually mean?Let’s explore a bit further…

 


Much like an iceberg, 90% of a software's processes are not shown in the final product.
 

1) Software is Abstract and “Open to Interpretation”

Software is the most abstract product on earth. The vast majority of the product, even when complete, lies under the surface.

Software is analogous to an iceberg; 90% of it’s mass lies unseen beneath the water. UI Components, UI Logic, persistance store access,  file system, look-up services, system configs, acess controls, third party API’s and data access managers and many others components create a  virtual ecosystem of unseen yet, interrelated parts that need to be conceptualized constantly, just to be understood and adjusted.

All these Ghosts in the Machine make the software development process fundamentally different than other types of building, such as a house or a car where you can use your five senses much more effectively; sight, sound, smell, touch, even taste. You can hear the engine when it’s not running right, you can see and feel a leaky roof that will lead to serious water damage if not fixed. You can smell a broken gas pipe. You can see that that tires on a car are getting low on air. But the five senses, which come in so handy during every day life, are virtually useless in this software world of abstraction –instead you have piles of programs and code, a monitor, and your mind.

 


The bigger it gets, the more chaotic it becomes. Software is this complex

2) Software is Complex

Even small pieces of software can accumulate into frightening complexity. A small program requiring the work of only one or two programmers can easily end up turning into tens of thousands of lines of code. Significant products, like the latest versions of Microsoft Windows, run into the tens of millions. And complicating matters further is the exponential nature of software’s complexity. You might expect that a 100,000-line program would be ten times more complex than a 10,000-line program. However, a program’s complexity depends not just on the instructions, but also on the interactions between the instructions. The 100,000-line program has ten times as many instructions interacting with ten times as many instructions, so we should actually expect it to be a hundred times as complex.


Take the following quote from renowned information science author James Gleik:

Computer programs are the most intricate, delicately balanced and

finely interwoven of all the products of human industry to date. They

are machines with far more moving parts than any engine: the parts

don’t wear out, but they interact and rub up against one another in

ways the programmers themselves cannot predict. [Gleik 1992]

The baseline takeaway is that software is complex - beyond reason and beyond what the typical human mind can comprehend - and this complexity must be accounted for when undertaking such a project.
 


The tools of the trade in software development isn't a one-size-fits-all scenario

3) Software is Built Upon a Vast Body of Knowledge (One That is Rapidly Evolving)

From Java to .NET, from PHP to Ruby on Rails, enterprise application frameworks contain a wealth of functionality, and beyond that there is plenty more functionality available from third-party software components. This means that even when working on a variety of applications and systems, a developer is incapable of gaining experience with more than a small portion of the frameworks and platforms that exist. This makes specialization an absolute imperative. Too much is changing within even just one software domain to for people to be able to stay up-to-date.

To state it another way, it is nearly impossible to have a back-end developer who specializes in a framework like .NET, AND is also capable of developing a front-end responsive design web platform in PHP,WHILE developing a MySQL database. Each of these skills requires massive amounts of focus and each of these skill sets can get outdated in as little as 6 months’ time. A new version can come out, new standards, new technologies to integrate, etc. and if the developer isn’t up-to-speed on all of these changes, one of two things will happen: 1) your project will get stuck, or 2) you’ll pay a multiple in time and money for the developer to re-educate themselves. Both of these options contribute to the aforementioned “68% delay or failure rate”.

 


You need a team of specialists to ensure you cover every part of the development correctly.


An analogy could be useful here, so imagine you are building a car. You have different prefabricated materials like rubber tires, steel frames, and now, even mini computers.  Asking a single developer to master all of the skills necessary to build and maintain a complex software program would be the equivalent of asking the engineer and builder of the car to master tire making, steel frame construction and computer hardware manufacturing. Maybe it can be done, but why would you want to bother? It’s a waste of time and resources to have one person try to learn all these different things if a combination of specialists could do it in a fraction of the time/cost. Software is no different.

If that didn’t hammer home the point, there is one more industry that an apt analogy can be made.  The United States legal code is a vast, and some would say, a perilous domain. It encompasses everything from getting married, buying a house, paying your taxes, having a baby, and even passing on your wealth once you depart this great earth. Lawyers can choose to specialize in 100’s of different domains within the vast field of law including;  property law, constitutional law, Intellectual property law, criminal law, environmental law, health care law, corporate law, first amendment law, contract law, civil law, international law, divorce law, equity law, labor law, family law, commercial law, immigration law, among hundreds of others. No lawyer specializes in all fields – heck, it’s hard enough to find an entire firm that can specialize in everything! But because the legal industry is very old and established no one really questions the wisdom of having all these different specialties or the need for legal specialists. Most reading this would agree that it would be quite absurd to ask an immigration lawyer to assist with divorce proceedings or to ask a real estate lawyer about how to protect your intellectual property. People just know that you “don’t do that”.

Due to its relative newness, and a lack of the public’s understanding of how software works, this is exactly what happens in the software industry every day. People and companies who are looking to develop software are constantly understaffing projects and asking developers to try their hand at certain programs that they have no business working in. Remember, the software industry is just as vast and complex as the legal code…but it changes about 100x as fast. Hmm, its sounds like there needs to be a recalibration in the way that the majority of us think about software and its development!

 

4) Finally, Software and Technology Have a Constantly Changing Value Chain

New technologies are coming to market on a daily basis; impacting the entire software value chain. For the purposes of this conversation, the software value chain is the connection and interconnection between all the various technologies that are required to create a useful piece of software that works on the internet. The value chain includes: the Domain Name Service (DNS) Server, web server, 3rd party API’s, Core Framework/Core Platform, email server, database server, server security, web browser and operating system. Any changes or updates to one of these can have a negative impact on the whole chain and lead to a downed website or bad user experience. This means that if you build a website or software platform butput no effort forth to maintain it, then any update or change to any link in the chain can lead your once perfectly-running web platform to bug out or even worse, crash.

Let’s take a simple example: the browser update. Say Internet Explorer (IE) comes out with a new version and the good people at Microsoft explicitly state that they’re no longer going to support the old version in six months’ time. This means that if your platform isn’t adjusted and made compatible to the newer version, in six months’ time when people visit your website, the UI features that once looked beautiful and elegant may go awry. This is only the simplest of examples, but you get the point. Unlike a house or car, software, at its very foundation, is in a constant state of flux and needs a level of dedicated attention unseen in other industries and products.

 


If the hardware industry changes fast, imagine how much faster software evolves to fit our everyday needs

Conclusion

If you’ve come this far, clearly, something that’s been said has struck a nerve. Maybe your company put you in charge of hiring a development company to create a piece of internal software….you got (what you think) is a great price, but very few things have gone right since the contract was signed. Maybe you are a freelance developer who’s frustrated by the fact that you’re giving a best effort on each and every one of your projects…only to find that your clients are not fully satisfied with the work you’ve done. Maybe you’re an IT services provider who’s “had it up to here” with a client who is constantly complaining about issues…only they refuse to create a budget to upgrade the software and systems that make their business go. Whatever your case may be, we empathize with you and we hope that this has helped to more clearly articulate the frustrations you have.