Re: Unit Testing: from theory to real case

Il 30/08/2024 10:18, pozz ha scritto:

Il 29/08/2024 22:56, Don Y ha scritto:

On 8/27/2024 3:52 AM, pozz wrote:

I read a lot about unit testing, but unfortunately I usually work on single-developer projects with stressing time constraints, so I never created full tests for an entire project in the past. This means I'm a newbie in this aspect of software development.

>
Fix it now or fix it later -- when you have even LESS time (because customers
are using your defective product)
>
Testing should start when you define the module, continue while you are
implementing it (you will likely notice "conditions" that could lead to
bogus behavior as you are writing them!), and when you consider it "done".
>
Thinking about testing when you draft the specification helps you
challenge your notion of the suitability of such a module for the task(s)
at hand as you imagine use cases (and MISuse cases).
>

I know the importance of testing, but we have to admit that it increases the cost of software development a lot, at least at the beginning. Not always we have the possibility to invest this price.

>
If you assume there are two types of "software" -- stuff that TRIES to work
and stuff that HOPES to work, then the cost of the latter can be a lot less...
because you really don't care *if* it works!  Apples; Oranges.
>

These days I'm working on a calendar scheduler module. The client of this module can configure up to N events that could be:
- single (one shot)
- weekly (for example, on Monday and Saturday of every weeks)
- monthly (for example, the days 3-5-15 of every months)
- yearly (for example, the day 7 of months Jan, Feb and Mar)
Weekly, monthly and yearly events have a starting time and *could* have a maximum number of repetitions (or they could be forever).

>
Testing aims to prove that:
- your specification for the module accurately reflects its need (suitability)
- the module actually implements the specification (compliance)
- the module is well-behaved in "all" possible scenarios, even when misused
- changes to the module haven't compromised past performance (regression)
>
It also gives you an idea of how your "process" is working; if you are
finding *lots* of bugs, perhaps you should be testing more aggressively
earlier in the process (there is a tendency to NOT want to make lots of
changes/fixes to code that you've convinced yourself is "almost done")
>
And, it provides exemplars that you can use to evaluate performance.
>

The calendar module depends on some other modules. First of all, it asks for the current time as time_t. It calls make_actions() function, with certain parameters, when an event occurrence expired.

>
Treat each as an independent, testable entity.  This makes it easier to
design test cases and easier to isolate anomalous behavior(s).
>

I'm confused. How to scientifically approach this testing problem? How to avoid the proliferation of tests? Which tests are really important and how to write them?

>
Make a concerted effort thinking of how to *break* it.  E.g., If you try to
schedule an event for some time in the past, how should it react? Should it
immediately "trigger" the event?  Should it silently dismiss the event?
Should it throw an error?
>
What if "the past" was just half a second ago and you've been unlucky
enough that your task was delayed a bit so that the clock ticked off
another second before you got a chance to schedule your event AHEAD of
time?
>
If there are multiple steps to scheduling an event (e.g., creating a structure
and then passing it on to a scheduler), consider if one of the steps might
(intentionally!) be bypassed and how that might inject faulty behavior into
your design.  E.g., if you do all of your sanity checks in the "create
structure" step, BYPASSING that step and passing a structure created
by some other means (e.g., const data) avoids that sanity checking; will
the scheduler gag on possibly "insane" data introduced in such a manner?
>
Can a client become confused as to which structures are "still active"
vs. "already consumed"?  If an active structure is altered, can that
lead to an inconsistent state (e.g., if the scheduler has acted on *part*
of the information but still relying on the balance to complete the action)?
>
Can a client safely repurpose an event specification?  Or, once created,
does the scheduler "own" it?  Is there some safety window in which such
alterations won't "confuse" the scheduler, outside of which the scheduler
may have already taken some actions on the assumption that the event IS
still scheduled?
>
What happens if someone changes the current *time*?  Do all events that are
now "past" instantly trigger?  Are they dismissed?  Do they move forward or
backwards in time based on the delta introduced to the current time?
>
[This is a common flaw in folks trying to optimize such subsystems. There is
usually a need for relative events AND absolute events as an acknowledgement
that "time" changes]
>
These interactions with the rest of the system (clients) can help you
think about the DESIRED functionality and the actual use patterns.  You
may discover your implementation strategy is inherently faulty rendering the
*specification* defective.
>

 Thank you for your reply, Don. They are valuable words that I read and hear many times. However I'm in trouble to translate them into real testing.
 When you write, test for this, test for that, what happens if the client uses the module in a wrong way, what happens when the system clock changes a little or a big, and when the task missed the exact timestamp of an event?
 I was trying to write tests for *all* of those situations, but it seemed to me a very, VERY, *VERY* big job. The implementation of the calendar module took me a couple of days, tests seem an infinite job.
 I have four types of events, for each test I should check the correct behaviour for each type.
 What happen if the timestamp of an event was already expired when it is added to the system? I should write 4 tests, one for each type.
 AddOneShotEventWithExpiredTimestamp_NoActions
AddWeeklyEventWithExpiredTimestamp_NoActions
AddMonthlyEventWithExpiredTimestamp_NoActions
AddYearlyEventWithExpiredTimestamp_NoActions

That is true if the repeated events (weekly, monthly, yearly) have a limited number of repetitions *and* the system time is over the last repetition when they are added to the system.
Otherwise, even if the system time is over the event timestamp (first occurrence), other repetitions could happen in the future, so they should be added to the list of active events.
So there isn't just one test for the WeeklyEvent, but there are more:
WeeklyEventWith10Repetions_SystemTimeIsOverAllOfThem_IgnoreEvent
WeeklyEventWith10Repetions_SystemTimeIsOverOnlyTwoOfThem_AddEvent
WeeklyEventWithInfiniteRepetions_SystemTimeIsOverSomeOfThem_AddEvent
This for weekly, but also for monthly and yearly.
And what if the addition of events (the call to calendar_init) is made with multiple events and not only one? Maybe the module behaves correctly when just one event is added (expired or not), but behaves bad when two or more events are added.
Maybe it behaves well when all the events are completely expired, but bad when one is partially expired and on the totally expired.
The combinations that we could imagine are infinite.