Is there a software attribute (like reliability) that describes a bug free experience?

Question

TL;DR

Do non-crashing bugs always affect Functionality, despite possibly affecting other quality attributes as well? Or is there a more specific quality attribute that non-crashing bugs always affect, like Functional Suitability?

---

This might seem like an ambiguous or unclear question, but keep in mind that core software attributes are very well defined by standards like ISO/IEC 9126, which describes:

• Functionality
• Reliability
• Usability
• Efficiency
• Maintainability
• Portability

There is a really large list on Wikipedia as well, but many of those approach the fuzzy, flexible land of the blogosphere. I want to keep this question unambiguous and bound to accepted definitions.

I'm looking for a software attribute that any non-crashing application bug affects. For instance, I consider a crashing bug to be a Reliability issue, but I'm not sure how to generally classify a non-crashing bug.

Correct me if you think I'm wrong, but here's why crashing bugs are most generally Reliability issues. If an application crashes (closes unexpectedly, freezes and doesn't recover, etc), it could be a security issue if protected assets are exposed or destroyed somehow. It could also be a safety issue if the application is, for instance, an X-Ray machine control panel. But a crash is always a Reliability issue, regardless of where else it falls.

But a non-crashing bug? Examples:

• some valid input is treated as invalid

  EX: February 29, 2012 is incorrectly disallowed, but is a leap day.

• some valid input is represented incorrectly

  EX: 11:59 AM + 1 minute is shown as 11:60 AM rather than 12:00 PM.

• configuration or assets that are supposed to be persistent do not persist

  EX: User needs to log in every visit, despite selecting "Remember Me"

• input is ignored in some cases

  EX: File extension is ignored if dragged-and-dropped

These all affect Usability, but that might not always be the case, so Usability is not a completely general classification. I'm leaning toward Functionality, because Functionality pertains to user needs and requirements fulfillment, and I think a non-crashing bug will always deviate from expected behaviour and/or correct behaviour. But something more specific than Functionality is desirable.

ISO/IEC 25010, the 2011 revision gives these definitions for Functional Suitability:

4.2.1
functional suitability
degree to which a product or system provides functions that meet stated and implied needs when used under specified conditions

Note 1 to entry: Functional suitability is only concerned with whether the functions meet stated and implied needs, not the functional specification (see C.6).

• 4.2.1.1
  functional completeness
  degree to which the set of functions covers all the specified tasks and user objectives

• 4.2.1.2
  functional correctness
  degree to which a product or system provides the correct results with the needed degree of precision

• 4.2.1.3
  functional appropriateness
  degree to which the functions facilitate the accomplishment of specified tasks and objectives

  EXAMPLE:A user is only presented with the necessary steps to complete a task, excluding any unnecessary steps.

...which does seem to fit. I can't fathom a non-crashing bug that doesn't affect either Functional Completeness or Functional Correctness or both.

And finally, by bug, I'm referring to all three steps in the dependability chain: the fault in source code, the erroneous program states caused by the fault, and the interface failure caused by the error.

Whew, that was a lot of work for terminology's sake!

Answer 1

I would call non-crashing bugs a "Reliability" issue as well, because you don't just have to be able to rely on a program to stay up; you have to rely on it to do what it's supposed to do.

Imagine a program that never crashed but sometimes thought that 2+2=5 when certain edge cases were hit. I wouldn't consider such a program to be reliable. Would you?