A step too far
The picture above was found among materials promoting benefits of DevOps to a large organization. I find it both wrong and dangerous, because it suggests that user testing may be replaced by test automation. Such belief can have disastrous consequences for any software project. One recent US presidential election was lost in no small part because of such mistake.
US presidential campaigns often rely on volunteers who go door-to-door, handling out leaflets and promoting the candidate in personal interactions. Those efforts intensify on the election day, some of volunteers would mark off the people who have voted. The others make phone calls to registered supporters of a party who did show up, encouraging them or even driving people to the polls. Such actions can have tremendous influence on the result, especially in swing states.
In 2012, Mitt Romney campaign was preparing for a difficult battle. They took to software: decided to build a website to coordinate such volunteer efforts. It would help to direct their active supporters to areas and people where their help could have the most impact. It would also give the campaign managers real time information about situation.
Project ended with a disaster. Website was served over HTTPS protocol. Most of the users typed in or had auto-completed HTTP. There was no redirect, just a generic error message. All phone lines were overwhelmed with confused volunteers and quickly become unreachable. As a result, the campaign staff was “flying blind” during the decisive hours. At the same time, more than thirty thousand volunteers were confused and frustrated. Mitt Romney lost by a small margin in four states.
This kind of error can not be found in automated tests but would be found immediately when a typical user would be asked to use the system. Belief that any user-facing system can be responsibly developed without testing how actual users work with it is foolish.
I feel compelled to explain that I’m not against DevOps or automated test. Eliminating repetitive deployment tasks has obvious benefits. TDD enables predictable software development and use of tools such as Selenium can save hundreds of hours of testes work. Those things have many real merits and there is no need to invent false and misleading ones.
Acknowledgement: a better (and more opinionated) article about where to look for cost savings in a software project can be found here.
State of Scala
Two years ago, I wrote about some things I didn’t like about a particular Scala conference. I thought there is something wrong with Scala but didn’t articulate it. Today I found a talk by John A. De Goes which is spot on. It vividly explains how Scala is focusing on the needs of people who want to write a PhD thesis, but those who want to use it to write useful software are left in the cold.
Designing in simple steps
I like reading books about military history. Recently, I have read “One bullet away” by Nathaniel Fick. The author describes the training he had to complete in order to become an officer in US Marine Corps. One of the things he described was how the young officers are made to write lots of written orders in five-paragraph format called SMEAC. It stands for situation, mission, execution, administration and logistics, command and signal. The point of all this writing was to make sure that all those issues are remembered extremely well and become part of the thought process of the trained person.
I think that my line of work, writing computer software, can learn a lot from the military. So today, for my own use, I created a similar format I will follow when given a task to design a software component. The elements are:
- Mission statement – what should the component do? It should contain any functional and non-functional requirements.
- Environment – where should it run?
- Restrictions – what are the known restrictions on the component? Those usually are existing legacy systems and interfaces.
- Assumptions – for me, this means stating clearly what are the things I don’t know when creating the design and how I deal with the resulting risks.
- Description of the component – most designs I see seem to only have this part and quite often it leads to lots of wasted effort.
Just as the officer on a battlefield, I often have no time to write formal documents. But I am posting this very simple process description on this blog so that I remember about it. I hope that even considering those things in my mind before starting work will be a good thing.
Scala performance analysis with perf and Flame Graphs
I’ve been toying with Scala profiling lately. I find CPU Flame Graphs to be cool and useful, so decided to see what I can do with that. After some hacking and automating I got this:
One thing I find strange here is that scala.collection methods are shown as yellow, which according to The Source means they are C++ code. Overall, it seems like in this very simple case, it’s possible to see where the problems are. If there were any kernel-level operations involved, like network communication, the chart would be even more useful.
Scalar 2016
I attended the scalar conference in Warsaw. This is a short write-up of some of my thoughts about it.
By far the most interesting talk was given by Mathias Doenitz, who described his project Swave, which is a Rective Streams protocol implementation intending to be usable both in place of standard Scala/Java collections and distributed processing systems like Akka Stream or Apache Storm. I was looking forward to such a “write once, run everywhere” framework for quite some time, I will definitely be watching it’s progress.
On last year’s Scalar, the most interesting talk was given by Heather Miller, a PHD student of professor Martin Odersky. It seems someone from “Scala headquarters” is sent to Warsaw every year. This year’s emissary was Dmytro Petrashko, who talked about some unexpected behaviours of Scala initialization order. It was definitely interesting, but did gave me a pause thinking about unexpected complexity of having multiple mixins in a class.
Presentation of Amira Lakhal was also quite interesting but not from technical point of view. Amira is “one of the Duchess France leaders where she helps women in IT to became role models”. She demonstrated a simple but complete machine learning system, gathering accelerometer data from an Android device to detect if the user was standing, walking or running. Talk was very well prepared and it seem Amira’s strategy is to show some real-world, tangible application. Frankly, it was a welcome break because many of the other presenters (not mentioned here) were so out of touch with reality that it felt like a symposium on string theory.
The conference itself was very well organised, in particular the venue was really great. However, all of the talks were really short and there was almost no time for questions. I must also say that the conference host seemed rude and sexist. But maybe I just have high expectations, having attended conferences hosted by Rafał Motriuk – this guy is just pure awesomeness on stage. Despite those minor glitches, the conference was great and I am looking forward to visiting it next year.
Deamon slayer
A nice gotcha – what happens when you forget about destructors when writing server-side code in C++ can be found on Rachel’s blog.
Programming like it’s 1975
Some tips about getting optimal performance from modern hardware and operating systems. Author argues that application programmer has no way of knowing if the memory in RAM or cached out to disc, so it’s best to not to interfere with kernel. The other smart point is using worker threads in “most recently busy” fashion – a bit counterintuitive but makes perfect sense.
Edit: A discussion on hackernews raises several points when it comes to memory management. It’s getting increasingly rare to find a technical discussion there, this one seems better.
Stop the train!
A nice example of branch prediction can be found here. Discussion under “Update” mentions Intel compiler and it’s cool ability to do loop interchange. Extra credit for well-fitting railway analogy.
Oracle – how add milisecond to a timestamp value?
Recently somebody needed to add a millisecond to a timestamp value in Oracle. You can’t do just
select systimestamp + 1 / 24 / 60 / 60 / 1000 from dual;
because the ‘+’ converts timestamp to date and fractional seconds are lost. The (supposedly) wise guy here seems to suggest that the way to go is to create a custom function in Java… Come on Oracle, there should be a better way to do this!
OpenJPA errors are sometimes less than informative
When you see OpenJPA error claiming that a column does not exist in database and you clearly see it, make sure the instance field in question is private. At least, packet visibility seems to cause problems in case of annotations at field level. Such an error just ate 4 hours of my time.
This seems to be at odds with JPA 3.0 standard, which explicitly allows protected and packet visibility for instance fields (chapter 2.1). The current OpenJPA documentation for 2.2 release doesn’t list this deviation from standard (http://openjpa.apache.org/builds/apache-openjpa-2.2.1-SNAPSHOT/docs/docbook/manual.html#jpa_2.0_incompatibilities).
kwoz 