Prototype or System

It is more than just intent. It’s really the application of knowledge in order to achieve expected results. That is, it is one thing to just play around with building something in the hopes that it might work someday. It is another to have gained the knowledge and experience necessary to build it with the certainty that it will do exactly what it is supposed to do.

Wikipedia’s engineering page references the ECPD which states that engineering is:

[T]he creative application of scientific principles to design or develop structures, machines, apparatus, or manufacturing processes, or works utilizing them singly or in combination; or to construct or operate the same with full cognizance of their design; or to forecast their behavior under specific operating conditions; all as respects an intended function, economics of operation and safety to life and property.

Effectively, if you are just “guessing” when you build something, you’re not applying any engineering. If you can accurately predicate how it will operate because you understand it, then it is engineering. It’s also worth noting that in the Wikipedia page, if you are applying knowledge and science, you can be considered an ‘engineer’ even if that work isn’t certified (I always thought that the designation meant both).

Sometimes when developing software we read a bit of the documentation, then we just start hacking together a bunch of code. If the code is relatively simple and what we are doing is straightforward there is a very good chance the code will work as expected. We don’t need any depth -- understanding of what happens below each line of code or in other parts of the system -- we can just work at a higher level and hope that there are no unintentional side-effects, or that the lower parts won’t change significantly over time.

That works for smaller systems, but it tends to accumulate technical debt without us being aware. It’s not what we know; it’s what we don’t know that is slowly and dangerously building up. Thus the main behavior of the system works, but a great deal of what could happen remains unknown. Generally I like to call this type of code a prototype. It’s the basic functionality, but in quickly hacking it together we skipped over the depth required to insure that it will always function correctly.

When I work like this, I consider it to be hacking. It can be fast and fun, but there are too many uncertainties to feel comfortable having people dependent on this type of construction.

For larger systems that have complex architectures, relying on ‘guessing’ is usually a fatal error. A small problem in a little prototype can become a massive headache in a big system. It can require months or years of cleanup to rectify it. Thus, hacking is no longer a reasonable way to work. Instead we need to get more depth in our understanding. We don’t need to understand it all the way down to the chips, but we do need to understand it down to any levels that were not well encapsulated. That is, if someone else solved the problem correctly, we don’t need to rethink it, but as often is the case in software, if they only partially solved the problem we need to understand the consequences.

To me, when I sit down to build something with enough understanding of the technology to know precisely what it can and will do, that is ‘programming’. There is no guessing, and there is no ambiguity. A programmer knows the steps needed to be taken-- both for the results but also for handling any errors -- and applies that knowledge to produce something that they are certain will work.

The key difference here is the knowledge and certainty. If you know how to write the code, you are programming. If you are just taking wild guesses then you are hacking. You may get lucky hacking, and for prototypes you may not have an alternative, but you are essentially leaving the end result to chance.

In that sense, it is very clear that programming can and should become a form of engineering. There are many programmers out there that have a huge depth of knowledge that allows them to build exactly what they want, in a rather predictable time frame. It is also clear that someone that has only lightly-learned a programming language, and can do a few simple things with it, is mostly hacking. Hacking is no doubt a lot more fun, but it is akin to learning some magic tricks without any real understand of how they work. They may be successful most of the time, but Murphy’s Law insures that when you need them most, they will fail.

Generally we can see this differentiation in action. In the more modern languages that purport to take care of the memory management -- because people no longer understand what is happening underneath -- the run-times have become massively bloated. Moore’s law saves us from this being a disaster, but it goes to show how many people are really hacking at their work. Another good example is the proliferation of threading bugs. A language like Java makes it easy to create threads and protects against many types of crashes, but does nothing to stop spaghetti logic from disrupting the application’s behavior. Again people hack at threads without a deep enough understand of how they work. To work with them in predictable ways, you have to understand how they work and what you can do with them.

To sum it up, there are always times when we have to resort to hacking to figure out the underlying behavior or limits, but there is no substitute for knowing how it really works. Hacking together a prototype is often necessary, but when it comes to building a real system, a huge amount of knowledge is mandatory.