Re: Diagnostics

On 10/24/2024 10:52 AM, Waldek Hebisch wrote:

Don Y <blockedofcourse@foo.invalid> wrote:

On 10/18/2024 8:53 PM, Waldek Hebisch wrote:

One of the FETs that controls the shifting of the automatic
transmission as failed open.  How do you detect that /and recover
from it/?

>
Detecting such thing looks easy.  Recovery is tricky, because
if you have spare FET and activate it there is good chance that
it will fail due to the same reason that the first FET failed.
OTOH, if you have propely designed circuit around the FET,
disturbance strong enough to kill the FET is likely to kill
the controller too.

>
The immediate goal is to *detect* that a problem exists.
If you can't detect, then attempting to recover is a moot point.

 In a car you have signals from wheels and engine, you can use
those to compute transmission ratio and check is it is expected
one.  Or simply have extra inputs which mountor FET output.

But a *user* can't do that.  They can only claim "something doesn't
feel right about the drive"...
So, if the controller doesn't do it, what recourse?

The camera/LIDAR that the self-drive feature uses is providing
incorrect data...  etc.

>
Use 3 (or more) and voting.  Of course, this increases cost and one
have to judge if increase of cost is worth increase in safety

>
As well as the reliability of the additional "voting logic".
If not a set of binary signals, determining what the *correct*
signal may be can be problematic.

 Matching images is now a stanard technology.  And in this case
"voting logic" is likely to be software and main trouble are
possible bugs.

The data must be available concurrently in order to "vote" on
them.  And, must be "close enough" to not consider them to differ.
For high reliability applications, you often *compute* the results
in different ways / algorithms -- to highlight any issues in
one implementation over the other.  So, the temporal path to
"their solutions" isn't the same.

(in self-driving car using multiple sensors looks like no-brainer,
but if this is just an assist to increase driver comfort then
result may be different).

>
It is different only in the sense of liability and exposure to
loss.  I am not assigning values to those consequences but,
rather, looking to address the issue of run-time testing, in
general.

 I doubt in general solutions.  Various parts of your system
may have enough common features to allow single strategy
in your system.  But it is unlikely to generalize to other
systems.  To put it differently, there are probabilites
of various events and associated costs.  Even if you
refuse to quantify probabilites and costs your design
decisions (assuming they are rational) will give some
estimate of them.

I've asked for other peoples' experiences.  I've not expected
them to have solved *my* problem.  Nor do I expect my solution
to solve theirs.  Likewise, why something like Linux wouldn't
have "the" solution.

Again, I am not interested in "recovery" as that varies with
the application and risk assessment.  What I want to concentrate
on is reliably *detecting* faults before they lead to product
failures.
>
I contend that the hardware in many devices has that capability
(to some extent) but that it is underutilized; that the issue
of detecting faults *after* POST is one that doesn't see much
attention.  The likely thinking being that POST will flag it the
next time the device is restarted.
>
And, that's not acceptable in long-running devices.

 Well, you write that you do not try to build high reliablity
device.  However device which correctly operates for years
without interruption is considered "high availability" device
which is a king of high reliablity.  And techniques for high
reliablity seem appropiate here.

No.  Most devices can't afford the cost/complexity of a true
high reliability/redundant solution.
Your car has SOME redundancy in how it handles braking (two
chamber master cylinder plus "emergency/parking" brake PLUS
using the engine to slow the vehicle).  Yet, absolutely NO
protection against a catastrophic failure of the steering!
Redundancy in braking is relatively easy to provide -- esp
in the volumes produced.  So, adds little to the cost and
complexity of the vehicle.  Adding a redundant steering
mechanism... where would you even BEGIN to address that?
Cars are a great example of the tradeoffs involved.  You invest
a *little* to detect and report problems instead of a LOT to
continue operating in their presence.  Why not have duplicate
turn signal indicators (front, rear and side) to guard against
a bulb failure?  Much easier and cheaper to detect that a filament
has opened and report that to the driver (and HOPE he gets around
to fixing it).
If the driver can't be TOLD of faults and failures, then he
is in the dark as to how effectively his device is performing its
required actions.  "CHECK ENGINE" really does mean that the
engine *needs* attention.  What difference, "CHECK DRAM"?

It is VERY difficult to design reliable systems.  I am not
attempting that.  Rather, I am trying to address the fact that
the reassurances POST (and, at the user's perogative, BIST)
are not guaranteed when a device runs "for long periods of time".

>
You may have tests essentially as part of normal operation.

>
I suspect most folks have designed devices with UARTs.  And,
having written a driver for it, have noted that framing, parity
and overrun errors are possible.
>
Ask yourself how many of those systems ever *use* that information!
Is there even a means of propagating it up out of the driver?

 Well, I always use no parity transmission mode.  Standard way is
to use checksums and acknowledgments.  That way you know if
transmission is working correctly.  What extra info you expect
from looking at detailed error info from UART?

That assumes you can control the messages exchanged.  If I
attach a TTY to the console -- routed through a serial port -- on
my computer, what should the checksum be when I see the "login: "
message?  When I type my name, what checksum should I append
to the identifier?
I.e., serial port protocols don't *require* these things.
If the computer sees "do~n" -- where '~' indicates an overrun
error in the preceeding character's reception -- it KNOWS
that I haven't typed exactly three characters:  d, o, n.
So, it shouldn't even ASK for my password, choosing, instead,
to reissue the login: banner (because it wouldn't know which
password to validate).
Likewise, if I saw "log~in: " on my TTY, I *know* that it
isn't saying "login: " because AT LEAST one character has
been omitted in that '~'.
This is easy to fix -- in ALL interactions.  But, requires
the driver to propagate these errors up the stack and the
application layer to act on them.  I.e., if the application
layer encounters lots of overrun (or partity/framing) errors,
SOMETHING is wrong with the link and/or the driver.