Software Engineering at Google Lessons Learned from Programming Over Time

Good day, it has been a while since my last post. Have been quite busy at work and home. The holidays are approaching so things might improve somewhat.

In this post I am not going to write code. I am going to let you know about things that called my attention in the past few weeks.

I am currently reading “Software Engineering at Google Lessons Learned from Programming Over Time”, curated by Titus Winters, Tom Monshreck & Hyrum Wright, published by O’Reilly. And yes, it is the book with the pink flamingo on the cover.

I have read about two thirds of the book so far. I tend to read about twenty minutes first thing in the morning and before I go to sleep. By doing so every single day, I have a chance to read, learn, and reflect on the topics at hand. In my opinion, you can get more from a book this way than by reading several straight hours a day. The idea is not to finish the book and go to the next, but to learn from it.

The book at hand is somewhat different. The book is curated by three people. Each chapter of about twenty pages or so is written by one or two people and edited by one of the curators. As you are reading the book, you can tell the different writing styles of the authors. I do not recall reading a book like this.

The book covers topics of software engineering as seen by the authors of the different chapters. So far I have enjoyed the book. It is always interesting to learn about how companies that developed and maintain such large code bases do it.

In particular what called my attention was chapter 11 Testing Overview. Among several testing topics one classification regarding the size and scope called my attention. The tests are classified as: small, medium, and large. The number of lines is not that important but the scope is. In the small size, one can only use a single process without threads in a single machine. You cannot make use of databases or connect to other computers. I guess one can relate them as a type of unit tests on steroids.

In the medium size, you can use all the resources of a single computer without network access. For example, in the small you cannot make use of a database, but in the medium size you can as long as it is running in the same computer.

Finally you have the large, which is like a medium test that you can run on multiple computers. By doing so you have network latency to deal with so operations are non-deterministic.

If you find this approach interesting, I suggest you get your hands on a copy of the book and get the information directly.

The reason that the classification called my attention is the fact that software engineers that believe in testing and in particular, the Test-Driven Development write their code and after it is finished they write unit tests and depending in the magnitude and features of the software, it might need system level testing.

It is my turn to describe how I have been developing software for a long time. First I spend time thinking about how to approach the problem or task at hand. I document the approach with a diagram or two and some words.

The next step is to experiment with the components that I will use in order to get the software together to address the requirements.

Now is time to write some code which will provide parameters and then pass them to the module under development. Depending on what one is working on, it could be an API using REST, RPC or a call to a method in a class.

The first time you pass the arguments, as expected nothing works. You then start implementing additional code in both your solution and your test as you discover potential issues or need modifications to make the interface as simple and robust as possible.

As you repeat the steps your software grows and gets closer to the solution. You may encounter situations that you dislike so you get back to the drawing board and come up with new ideas that require some modifications, or perhaps a completely new approach. Keep in mind that changing direction as early as possible to prevent issues down the line saves time and money, especially when your solutions hit customers.

In a previous company I developed a storage server. When a new feature was required, I would start editing a test program called testss.exe (test storage server) to create a new test that would provide input data to the new API. First the end point would not do a thing so the server would not know about the attempt. Later the request would arrive at the storage server but it would respond with a message indicating an unknown request. After several modifications we could get the parameters entered in tests.exe to the storage server but it would not know what to do. At some point in time, the parameters on the request would perform the required task and return results to the caller.

We could automate tests.exe to repeat specific tests in parallel or in series. In the evening before leaving for the day at the office, I would set a few machines in the lab and would let them run overnight.

The next day I would verify the logs and determine what happened. The test and the code would be updated as needed and the process would repeat. After being satisfied with the results, other tests would be enabled to add to the stress of the servers.

It seems that the mechanisms I used are obviously not at the same scale as Google, but I could see some similarities that caught my attention. My takeaway is that it is positive and acceptable to use a test driven approach in which the tests evolve with the code under development. After the code is done then unit,, module and system tests are needed in some automated and standardized form.

In my opinion, I am not sure yet if the engineer who developed the code should also write the final set of tests. The issue that I see is that one might test against the code and not the functionality.

That is all I have for now. If I run into something else while reading the book I will generate a new post on the subject.

Before I finish this post I will check my blog. I had to update some software. It seems that we have 14,782 subscribers. Thanks for being one. If you are not, subscribe. It is free.

Hope you enjoyed reading this post and decide to read the book in question. Last time I checked it was available on Amazon.


Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.