On Thu, 5 Jun 2025 09:36:10 +0100, Martin Brown
<'''newspam'''@nonad.co.uk> wrote:
On 04/06/2025 19:54, Joe Gwinn wrote:
On Wed, 4 Jun 2025 17:53:19 +0200, David Brown
<david.brown@hesbynett.no> wrote:
On 04/06/2025 16:55, Joe Gwinn wrote:
>
Peer review of the code works better, because no pattern scanning tool
can tell spaghetti from inherent complexity.
>
>
That's certainly true in some cases. It surprises me a little that your
experience was so much like that, but of course experiences differ. My
experience (and I freely admit I haven't used complexity analysis tools
much) is that most functions can be relatively low complexity - the
inherently high complexity stuff is only a small proportion of the code.
In one situation where this was not the case, I asked the programmer
to re-structure the whole thing - the code was badly designed from the
start and had become an incomprehensible mess. Peer review did not
help, because the peer (me) couldn't figure out what was going on in the
code.
All true, but at the end of the day, complexity metrics and coverage
tools didn't come even close to paying for itself, and so they
gradually faded.
>
Complexity metrics and dataflow analysis tools worked OK for me when
working as a consultant digging companies out of deep holes they had got
themselves into. They invariably looked hurt at the long list of defects
that I would find almost immediately and I was almost always right about
code with an insane value for McCabe's CCI being full of latent bugs.
Yes. BTDT. The problem there is accidental complexity.
In my world, requirements that I wrote years before will pop up in
inappropriate places, but nobody understands them or how they got
there, so the people implementing those requirements are rightly
afraid to touch anything. So one of my unspoken tasks is to delete
stray requirements. The savings can be quite substantial.
Then there programmers in above their head.
Some code is irreversibly complex and necessarily so, but a lot of it is
just the software equivalent of a rats nest prototype in electronics but
treated as if it was production quality code. Management's ship it and
be damned policy since they always wanted their sales target bonus.
Yep. One thing I learned the hard way when still young was to always
prototype in a convenient language unrelated to the language of the
delivered project, precisely to prevent that abuse.
However, it is entirely true that some code will be marked as very high
complexity by tools and yet easily and simply understood by human
reviewers. If that is happening a lot in a code base, automatic tools
(at least the ones you are trying) are not going to be much use.
>
That goes with the territory but you only have to look at it once.
True, but it's still an investment decision - does this approach
result in worthwhile savings? Or should the effort be spent
elsewhere?
I would expect AI to be more useful in the context of static error
checkers, simulators, and fuzz testers rather than code coverage at
run-time.
Forgotten from earlier: "Just to be clear - are you using
non-intrusive statistical code coverage tools (i.e., a background
thread, timer, etc., that samples the program counter of running code?
Or are you using a tool that does instrumentation when compiling? I'm
trying to get an understanding of the kinds of "false hits" you are
seeing."
The focus here is non-intrusive code evaluation tools.
We also use intrusive tools and instrumentation in the integration
lab.
>
I find instrumentation often disturbs the problem that I am looking at. YMMV
This is very much a significant problem. My approach, unloved by
process and language bigots alike, is to manually write the wiretap
points in assembly code that rides the iron, invoking plain C (not
C++) only if the wiretap decides to collect something. This is done
because there are many more wiretaps in place than can be collected at
once, so the decision to collect must be very quick.
Which wiretaps are active is determined by a big bitmap array at a
static location in memory.
One form of wiretap checks for violation of a pre-specified assertion,
collecting only on violation. This part is usually coded in plain C.
Again, this must be quick, because assertions usually don't fail later
during integration.
Analysis of the collected data is wholly offline.
I'm a fan of Intel's vTune for finding hotspots in serious code.
This is after my time in the integration lab, but I'm curious.
What I used was a special assembly-coded ISR triggered by a CPU timer
register. The time period was chosen to be mutually prime with all
significant periods in the kernel, application, or library code. This
ISR took a snapshot of the memory addresses that were executing when
the ISR ran. This data was histogrammed, and the peaks chase back to
locations in the software.
This was very useful on Ada83 projects, where lots of problems were
because something that looks clean and simple actually mapped to a
clumsy bit of Ada runtime system code - no amount of expert code
review was going to figure this out. Once the problem was understood,
we were able to paraphrase our way out of trouble, usually only by
total avoidance of major parts of the language.
Joe