Re: Predictive failures

On 4/21/2024 12:37 PM, boB wrote:

On Fri, 19 Apr 2024 12:10:22 -0700, Don Y
<blockedofcourse@foo.invalid> wrote:
 

On 4/19/2024 11:16 AM, boB wrote:

Intermittent failures are the bane of all designers.  Until something
is reliably observable, trying to address the problem is largely
wack-a-mole.

>
The problem I have with troubleshooting intermittent failures is that
they are only intermittend sometimes.

>
My pet peeve is folks (developers) who OBSERVE FIRST HAND a particular
failure/fault but, because reproducing it is "hard", just pretend it
never happened!  Really?  Do you think the circuit/code is self-healing???
>
You're going to "bless" a product that you, personally, know has a fault...

 Yes, it may be hard to replicate but you just have to try and try
again sometimes.  Or create something that exercises the unit or
software to make it happen and automatically catch it in the act.

I think this was the perfect application for Google Glass!  It
seems a given that whenever you stumble on one of these "events",
you aren't concentrating on how you GOT there; you didn't expect
the failure to manifest so weren't keeping track of your actions.
If, instead, you could "rewind" a recording of everything that you
had done up to that point, it would likely go a long way towards
helping you recreate the problem!
When you get a "report" of someone encountering some anomalous
behavior, its easy to shrug it off because they are often very
imprecise in describing their actions; details (crucial) are
often missing or a subject of "fantasy".  Is the person sure
that the machine wasn't doing exactly what it SHOULD in that
SPECIFIC situation??
OTOH, when it happens to YOU, you know that the report isn't
a fluke.  But, are just as weak on the details as those third-party
reporters!

I don't care to have to do that very often.  When I do, I just try to
make it a challenge.

Being able to break a design into small pieces goes a long way to
improving its quality.  Taking "contractual design" to its extreme
lets you build small, validatable modules that stand a greater
chance of working in concert.
Unfortunately, few have the discipline for such detail, hoping,
instead, to test bigger units (if they do ANY formal testing at all!)
Think of how little formal testing goes into a hardware design.
Aside from imposing inputs and outputs at their extremes, what
*really* happens before a design is released to manufacturing?
(I haven't seen a firm that does a rigorous shake-n-bake in
more than 40 years!)
And, how much less goes into software -- where it is relatively easy to
build test scaffolding and implement regression tests to ensure new
releases don't reintroduce old bugs...
When the emphasis (Management) is getting product out the door,
it's easy to see engineering (and manufacturing) disciplines suffer.
:<