No, YOU test it!
Testing! Or not. And looking at the news in the past few... years.. I think that the general consensus on the matter is that "testing" for bugs is just a waste of time and money. Is not the first time that I lament the fact that lots of the software that should be sent to "production" is to say the least "not well tested" and to say the truth, not tested ad all, so when it does go to production the results are what *I* expect.
But if we look outside the IT environment, we can clearly see that the idea is taking roots.
Some years ago there was a big scandal when a known german motorcar company (Cough! CoughVOLKSWAGEN! Cough...) "cheated" the economy and ecology tests by adjusting the electronics of their cars in such a way that the engine was working perfectly to pass the tests but when used "normally" the result were... a lot less eco-logical or eco-nomical. In short, they built special conditions into their system specifically to pass the tests. Lately, a manufcturer of civilian airplanes got itself with hundreds of airplanes grounded because a certain bit of the on-board electronic wasn't correctly tested.
If you read technical publication, you should be used to see security bugs reported in many products that should've been discovered by a casual observer, but remained unpatched for years. Some of them are so well-established, that fixing them breaks lots of other things.
From everywhere you can hear screams and yells, "why this thing hasn't been tested?"...
The answer is, in many cases, because testing stuff COSTS MONEY AND TIME. And, as discovered in many cases, testing most of the time doesn't fixes the problem.
What we see, more often than not, is that manufacturers (especially of software), decide to go with "pre-releases", call them "alpha" or "early access" or wathever. The point is that the product ain't ready, but if you really want, you can get it right-fucking-now and have a go at it.
This is also known as "using your customers as testers", that is named and shamed by many peoples around, but is more or less normality for many things... and from my point of view... is perfectly fine.
Ok, put away the pitchforks and the torches please, the insurance said that they don't cover lynch mobs damages.
What I mean is that, in many circumstances, the only functional way to discover some problems is to emply somebody that has NO FUCKING CLUE of what he is doing, try the product and then see what the heck does he do. Yes, I know, there are professionals for that, peoples that are very good at doing it, but usually they wants money, and the better they are, the more money they want. That is fine, but in some cases the company manufacturing ... wathever that thing is, has no money whatsoever to pay them, or no money whatsoever -POINT-. So the way of "early access" is the only one. No, most of the time is not the best one, and yes, sometimes the company is not really interested in fixing the bugs or improving the product, and in many cases the company COULD employ professional testers (Microsoft), and sometimes they even do that and then still go the "early access" way.
And in some cases, the "testers"... aren't told that they are actually "testing" the thing... Like some airlines discovered on their skin, or the skin of their customers. I mean, if $carmanufacturer decided to test a new type of "active braking system", I'd like to know about the possibility that pressing the brakes could have the car speed up instead of... the normal way reaction.
And now, let's talk about fact.
Once upon a time there was, and still there is, a company, let's call it $companyX, that provided services. One of these services was a "B2B messaging system", and if you don't know what that means, is basically a mail system that is used only by computers to exchange messages between each others.
Basically a computer send a message to another computer that after receiving send back a reply that says "got it".
If you have used once a mailing list, you know what is like. Yes, what is today "sold" as "AI" or "ChatBots"... that regularly doesn't work is sucks and that worked fine about 20 or 30 years ago.
To get a system that was very flexible, $companyX had picked as "backed" for the whole thing a product developed by a german software house, software that was very flexible but also very expensive.
And it came the day that $companyX decied the system required replacement because it was quite old. And it was also no longer maintained, especially in some areas, like antispam. To avoid paying a very high price for a simple upgrade, the company decided to retire the old system and use instead a new system mostly based around postfix.
Nice, but since it was used for B2B, replace the "middle" could very well mean to "break" everything that was connected to it.
In fact, during the update of another system (let's call it "brother") there had been a complete disaster, that provoked a couple of hundreds tickets for "communication problems" in a couple of hours and required a roll-back and full restore of several things in the wee hours of the night.
Obviously, the UL of $companyX wasn't in the mood for a repeat of that performance and so was pressing for a "better management" of the transition.
And here is the point of discontent, where I got against the "project manager" of the thing. Or better the project UNmanager.
Allow me to clarify.
There were 2 guys involved. One of the two had a good knowledge of the technical part of the thing, but he wasn't officially a "manager", the second one had NO FUCKING CLUE about the technical part, or the operative part, or the administrative part... in fact, he had no fucking clue at all, but he was the official Project Manager.
Now, if you ask me, the logical move would have been to send the second guy to fuck himself and promote the first one to 'project manager'. The problem was that the first one had ZERO INTEREST in becoming a project manager because otherwise he also had to take responsabilities. In fact, that first guy was the one that toke the decision to send the other product I said before to production, knowing that nobody had tested it and without informing anyone about the thing. Basically he had turned the switch and when the shit had hit the fun, he started backpedalling at warp speed.
So we were in discussion about putting the new "thing" into production. Me this UL that actually wasn't an UL and the other one (UL2).
Me - ...all the users have been created in the new environment and the messages have been reported, however, the messages that have been reported are all marked as "new" messages, there is no way to know which messages are not new.
UL - Why no?
Me - Because the format of the database of the other product is unknown, and reading one message at a time from IMAP takes way too long.
UL2 - And what does it means?
Me - That we need to tell the users. That they have to check their messages or the way they process the messages.
UL - But... then we have to tell the users!
Me - Obviously we have to. In fact, the sooner we tell the user, the better so they can test things before we go to production.
UL2 - Why?
Me - What do you mean "why"? Becaue we are not talking about you reading your mail, these are messages that are sent between computers, if something doesn't work is very possible that they have to do some changes in their processing flow and since we have no idea what the other product did or how, the only way to know is for the customers to do the testing.
UL - But we should do all the test and be sure that the new system does exactly the same thing as the old one.
Me - Yes but we can't. Because we have NO IDEA what the customer do or check or look at of the messages, we can only grab the messages and throw them into a mailbox, what happen next we don't know. And anyway... you wanted to go in production on saturday, so when did you wanted to do those tests?
UL - I wanted to go on production but then there was the problem with the billing...
Me - Yes, and that problem is still there, but the point stays. If you want to go on production on day X, when are you going to do the 'tests'? X-5 minutes?
UL - I'm kinda busy...
Me - That's your problem. You sent around that "plan" where you specified that before the "go" you want to perform tests. So?
UL - Do you have any suggestion about the tests?
Me - Sure. Let's contact a few of the customers and let THEM do the tests.
UL and UL2 together - WHAT???
Me - Is the best way. They know what should happens before, during and after. If they test and say everything works, it means it works. If not, then we have work to do.
UL - And what happens if they start using the system before we've adjusted for the billing?
Me - That's a completely different problem and has nothing to do with the testing. Anyway, you still wanted to go on production on saturday, so what was your idea about the billing?
UL2 - But what should we tell the customers? Who should sent the mesages? And how? And When?
Me - And why?
UL2 - What?
Me - Since you were asking all the questions...
UL2 - What?
Me - (massaging my nose) ... We need to communicate that we are doing a big upgrade to the system to a new platform, that is the time since it's fucking 20XX already, and there are going to be lots of changes, new interface, new administrative functions and so on, so would be nice if they could test the system. We give the the IPs and let them do their stuff. It shouldn't take too much.
UL - But... if they found problems?
Me - That's why we do the tests. Then we have to see what problems and what to do about them.
UL2 - What kind of problems could be?
Me - I don't know, that's why I say that the customers are the best peoples to do the tests.
Yes, because in many circumstances, you try yourself o you let whoever should use it to try it. And when you have no idea how is going to be used, there is only one way.
Davide
02/02/2020 11:52
