Cyclomatic complexity

[AndyBeez]

LSD: or write a single function using generics and recursion, in base 12.  With no explicit conditionals either.

Complexity is not just the number of lines of code in the state machine, but how well the programmer understands how to build the state machine. Sometimes you need functions and other times it is just not worth it.

CC: What is more 'complex', Shakespeare or an infinite number of monkeys with typewriters? Surley the monkeys are just a function that concatenates random letters in an infinite loop. What could have a lower CC than this function?

Code: [Select]

Monkeys(){while !sonnet{sonnet+=random(asciiCharacter)}return sonnet}


[golden_labels]

Yes, we very often write test cases for Shakespeare. After all who of us does not write Shakespeare every weekend?

[PlainName]

CC: What is more 'complex', Shakespeare or an infinite number of monkeys with typewriters? Surley the monkeys are just a function that concatenates random letters in an infinite loop. What could have a lower CC than this function?

There is a slight difference: Shakespeare did it in one, whereas it would take the monkeys until at least the sun goes out to get the desired result (unless they are very, very, very, very lucky). To all intents and purposes, except 'in theory', the monkeys fail.

[rhb]

A large (e.g. 64 test) can be dead trivial.   On one contract I was working on software to convert data from one package into other software input formats.  First test case was to convert all 8 formats into all 8 formats.  I then added tests if there were other things that the program did.

That was a 500 KLOC VAX FORTRAN port to  Unix on Intergraph, Sun, HP, IBM, SGI & DEC systems.  The first year of use there were fewer than a dozen user submitted bug reports.  The programs were instrumented to report usage to a company server in Dallas.  There were well over 250,000  program runs during that first year.  Usage went up as we added programs and features and user bug reports went to zero.  After a merger, the package continued in use for another 6+ years before it had all become obsolete without any support *or* problems.

The primary requirement is designing for test.  Metrics such as CC may be useful or not depending on the situation.  A large case switch is not complex no matter what some metric might say. 

For  an HPC code I once wrote a several hundred line computed GOTO because it was the simplest and fastest  solution.  This was on a DEC LX164 in 1998.  It was 3-5 times faster on floating point than any other processor on the market.

Truth is, I didn't hand code the thing.  I generated it using a short awk script.  Testing showed that if I unrolled the loop this was embedded in 2x I could get 10-15% faster throughput.  There were a number of parameters that changed depending upon how i configured the program, most of which was written in C using MPICH with only the floating point code in FORTRAN.  The loop was too complex to have the compiler unroll it which was the reason for writing a program to generate the code.

BTW this was a code with run times on clusters that ran for days.  As a practical matter common practice is to break such jobs into 7-10 day runs despite checkpointing to allow restarts if there was a system failure which is quite probable if you have several thousand machines  working on the processing job.

Reg

[Siwastaja]

That is true, but I'm not sure why it's so important.  Either you have test coverage or you don't.  If you have coverage you don't CC to tell you how many test cases you need.

I agree. And I wonder why somebody thinks that requiring 50 test cases for a function is "impossible", where is this coming from? For example, if the function is a novel implementation of 16x16->32 multiply, 2^32 test cases are required but are utterly trivial to generate. Now depending on the implementation, Cyclomatic Complexity of that function might be anything from 1 to 2^32 or even beyond, but that would not affect the testability. Much more important question for testability is, is there state stored within the function or is it pure? And are the inputs complex enough that they can't be all tested brute force, and if yes, are there then hard-to-see, hard-to-reach paths, where nesting depth is much more interesting than number of paths.

[tggzzz]

I can see a clever coder would have made it table-driven, with tables of offsets into the packets for different values (…)

Table-based state machines, some cases of LUTs and similar solutions may bring down the cyclomatic complexity value as calculated by an automated analyzer. But this is just “cheating” in the metrics. Not really improving them.

"Cheating the metrics" and "not really improving them" - perhaps. But anybody that has metrics as a goal is in a state of sin.

Table-based state machines can be the ideal implementation:

• if the states and transitions match those in the specification FSM. Many systems are specified in terms of FSM behaviour, and having the implementation directly and visible match the specification is A Good Thing. Of course that implies the spec must be what's needed, but if the spec is wrong the implementation's correctness is irrelevant
• you can see that all sequences of states and events have been considered and dealt with. For a non-trivial system, that's very difficult with bog-standard procedural implementation techniques

Given the choice between something that is visibly correct vs something that has good metrics, only a twat would argue for the metrics.

[tggzzz]

Either you have test coverage or you don't.

I've seen projects with an imposed/mandated requirement for 70% code coverage. That percentage was measured using commercial tools.

The tests didn't actually indicate whether the code did the right thing; the code was executed and that was what mattered. Isn't it?

A common procedure on that project was to find a NullPointerException thrown in foo(), and to solve[1] the problem by the infamous:
try {
  foo();
catch NullPointerException npe {
}

But the 70% coverage was achieved, so people were happy. Well, most people!

[1] I refer you to Humpty-Dumpty

[golden_labels]

But anybody that has metrics as a goal is in a state of sin.

Of course. Confusing benchmarks with what they measure is a common mistake. Ubiquitous far beyond tech world. Police focusing on improving stats, pupils made aiming for universally high grades, authorities trying to make bad-looking values zero. People are people and misuse of tools is widespread.

Again, neither tool’s fault nor evidence for them not being useful. I did not advocate for making cyclomatic complexity low for the sake of it being low. In fact my first post in this thread includes examples of how it may overestimate. But it is a tool, which tells something about the code. One should not aim to reduce CC for nicer reports, but should take a hint from high CC, understand the problem with code and eliminate the problem. Which in turn reduces CC. As an effect, not as a goal.

[SiliconWizard]

Either you have test coverage or you don't.

I've seen projects with an imposed/mandated requirement for 70% code coverage. That percentage was measured using commercial tools.

The tests didn't actually indicate whether the code did the right thing; the code was executed and that was what mattered. Isn't it?

Well, yes, code coverage is just a guarantee that a certain % of the code is going to be executed during testing. That's all there is to it.
It obviously doesn't guarantee that the covered code is correct.

But what it does say is that anything below 100% coverage means that at least some parts of the code are NEVER executed during testing, which means it's fully unknown territory.
This is how code coverage should be interpreted.

[tggzzz]

Either you have test coverage or you don't.

I've seen projects with an imposed/mandated requirement for 70% code coverage. That percentage was measured using commercial tools.

The tests didn't actually indicate whether the code did the right thing; the code was executed and that was what mattered. Isn't it?

Well, yes, code coverage is just a guarantee that a certain % of the code is going to be executed during testing. That's all there is to it.
It obviously doesn't guarantee that the covered code is correct.

But what it does say is that anything below 100% coverage means that at least some parts of the code are NEVER executed during testing, which means it's fully unknown territory.
This is how code coverage should be interpreted.

I wasn't sufficiently explicit, and you chose the honourable interpretation.

What I meant to imply was that whether or not the code worked correctly, the tests were so grossly inadequate that they didn't test that. Completely unacceptable - to me at least!

But the lack of adequate tests meant the traffic lights were green, so everybody else was happy.