Exceptions in Programming

Created  .:|:. Updated 

Introduction

On the use and value of exceptions in programming - a discussion of how and when to use exceptions and whether or not the exceptions thrown by a method should be declared and their handling enforced by the compiler.

Note: Specific examples in this article are drawn from the Java programming language, since that’s where I have extensive experience with exceptions.

Error Codes vs. Exceptions back to top

In the Beginning, Man Created the Status Code

The earliest programming languages, such as BASIC, COBOL and RPG, had no concept of a function call. If a subroutine needed to indicate that an error had occurred, the programmer needed to set aside a global variable in which to store the subroutine result. Since the result code was usually not unique to any specific subroutine each subroutine had to be careful to explicitly set it prior to completion, to either an error indication or some agreed upon “no error” value, typically zero.

The C language, which rose to mainstream popularity in the late 1970’s and still remains immensely popular today, had functions with arguments and a return value. In many APIs, such as those for system I/O, the functions returned a result code which was positive if the function was successful (including 0), or -1 if there was an error. In the event -1 was returned, a global status variable, errno, indicated the specific condition. Other APIs, such as those memory management, returned 0 or null on failure and non-zero on success.

Was That a Stop Sign Back There?

The essential problem with passive error signaling such as return codes and global status codes is that they require the caller of the API to actually check the result after each and every API call. A common source of very difficult to debug errors when programming in C, even today, is allocating memory and failing to verify that the pointer returned is not null. In this case the program blunders on and overwrites memory in the data segment starting at offset zero, corrupting the program’s memory space, or in older operating systems, the system’s memory space, and often not triggering a real fault until some time later - sometimes not until much, much later.

To use an analogy, the programmer would write a program which blissfully blows through stop signs. Most of the time there is no cross-traffic (no failure) and therefore no problem… but at some point there is the inevitable collision with a semi-truck coming through the intersection in another direction.

A big problem with unchecked result codes is that most of the time the program works as designed; it’s not until something fails that things come undone, and then they tend to really hit the fan in some spectacular manner… but often not until some point far removed from the actual cause of the failure, and always without any information of what actually when wrong.

Another problem is that all those if(ret!=0) { ... } statements after every function call which clutter up the code and get in the way of the program flow and the programmer’s thinking as the program is created.

Enter Stage Left… Exceptions

Exceptions are a mechanism in a programming language that signal an error by interrupting the program flow and unrolling the call stack until the exception is handled. In the vernacular, an exception is thrown by the code which triggers it, and is caught by the code which handles it. In conjunction with constructs such as try/catch/finally and using/dispose the code between throw point and the catch point can clean up such that the program state for that code remains valid.

Exceptions allow the programmer to use an API’s methods with the core assumption that they will succeed, which allows the logical flow of the code to remain uncluttered with error handling. Instead, the error handling is consolidated at the key points at which the programmer can take action. More importantly, the program execution does not blunder on, blissfully unaware that it just failed. Essentially, exceptions allow the program to fail-fast with no effort required on the part of the caller - while the program has still failed, it has done so in a clear and identifiable manner.

Exceptions in modern languages which record the call stack provide another crucial piece in the puzzle, that is, the exact location at which the failure occurred.

Exceptions have been conceptualized in two essential forms, checked and unchecked.

Checked Exceptions back to top

To date, the only language which supports checked exceptions is Java, in which it is commonly held that this was done as an academic experiment. Many in the industry consider checked exceptions to be a failed experiment. For myself, the jury is still out, although all my programming in Java has used unchecked exceptions.

A checked exception is one which must be handled in the method in which it is thrown or be declared as being thrown by that method. Furthermore, any method which uses another method which declares that it throws a checked exception must itself either handle the exception or declare it to be thrown. In this way the compiler, assuming rigorous and correct use of checked exceptions, can at least in theory enforce that all exceptional conditions are handled by the code.

Underlying this is the assumption that any condition which the program should at all be able to handle is thrown as a checked exception. The benefit is presumed to be that when the program is complete the programmer will have given adequate consideration to the ways in which it can fail and have already written the proper error handling so that it either recovers to the maximum extent possible, or it fails “gracefully”.

The idea and hope is that problems are then found during development and do not cause a program failure at runtime. This is in keeping with the generally accepted wisdom that it’s better to find as many problems as possible at compilation time.

The particular and primary benefit to using checked exceptions is that they are forcibly and rigorously documented. It’s not possible for the programmer to forget to document the ways in which the code can fail, because the compiler and the documentation system are explicitly informed of said ways. And it’s not possible for the programmer to mask exceptions which arise from called code because, again, the compiler requires that they become part of the contract of the calling method if they are not handled.

Critical to making checked exceptions work is to handle and transform exceptions which they cross API boundaries. This means that if some code uses another API package, any exceptions thrown by that other package must be caught and transformed into an exception belonging to the calling code. That is, if package X uses package Y it’s important that package X never throw a YException - instead it must transform a YException into an XException and throw that.

Checked exceptions have, however, exhibited several problems in practice which has undermined there apparent utility. In particular, they have been accused of working well in small examples but failing to scale well to larger programs (I must confess that I am not yet convinced yet that this is the case, although I do lean that way).

Considering Errors Too Soon

The primary complaint with checked exceptions is that they force a programmer to think about problems too soon. That is, when the programmer should be concerned with laying down the program logic, they are instead forced to think about what can go wrong and what to do about it.

This has led to a common mistake of swallowing exceptions, a problem made worse by many IDE’s which shell in a template such as:

InputStream                         inp=null;

try {
    inp=new FileInputStream("SomeFile.txt");
    (more code here)
    inp.close();
    }
catch(IOException thr) {
    // ignore exception for now
    }

Here is the worst of all possible ways to handle an exception - swallowing it. In this case the programmer has “temporarily” deferred the error-handling consideration with a stub block that does nothing. Except that it does something very subtle and very important - it silently ignores the exception. In doing so it re-introduces the blundering on problem which existed with error codes. Unless the programmer is diligent to return to the code and fix the error handling, it’s just silently broken.

For a long time, I used to handle Java’s InterruptedException this way. As a result I had a lot of code which did not correctly handle being interrupted, usually because it simply silently ignored the fact that it was interrupted.

Now the argument can surely be made, and might even be valid, that the code is not complete until the error handling has been done, and I’m not going to disagree with that. But the fact of the matter is that many programmers will simply push the error handling to one side while they “first get it working”. When I was first learning to program in Java I was guilty of this on multiple occasions.

Like it or not, one of the side-effects of forced error handling is harmful error handling, which is worse than no error handling.

Considering Errors Out of Context

Another related problem is lacking adequate context about how and why the exception occurs and how to best handle it when it does. The result of this is typically inadequate error handling.

It takes considerable effort to mentally step sideways from the task at hand, which is writing the code to accomplish the current goal, and think about what things can go wrong and what can be done when they do. This task much easier in the light of a concrete example of when it has gone wrong; however, that context often arrives very late in the process - possibly after shipping when the customer is trying to use the software.

Assumption of Guilt: Phantom Exceptions

The essential problem here is that many problems are technically possible, but in practice only theoretical. This occurs especially when preconditions have already been checked by the caller. This is the “guilty until proven innocent” effect. I call these phantom exceptions.

The end-result tends to be extraneous error handling which throws an unchecked exception with the comment that it can never happen. And that’s if the programmer is handling it well; the big temptation is to just swallow the exception, which is fine if it truly never happens… until it does under some weird circumstance.

Good examples are FileNotFoundException when the file is checked for existence first and UnsupportedEncodingException when the encoding is mandatory:

public void readConfigFiles() {
    parseFile(new File("Base.json"));

    File custom=new File("Custom.json");
    if(custom.exists()) { parseFile(custom); }
    }

private void parseFile(File src) {
    InputReader                     inp=null;

    try {
        inp=new BufferedReader(new InputStreamReader(new FileInputStream(src),"UTF-8"),1024);
        (input parsing here)
        inp.close();
        }
    catch(FileNotFoundException thr) {
        // can't happen - file must exist
        }
    catch(UnsupportedEncodingException thr) {
        // can't happen - UTF-8 is required by JVM spec.
        }
    catch(IOException thr) {
        (code here to show the user a window with the exception details)
        System.exit(0); // can't continue without config!
        }
    finally {
        // ignore IO problems with close and NullPointerException if open failed to avoid masking real exception
        try { inp.close(); } catch(Throwable thr) {;}
        }
    }

The problem here is that in all likelihood the programmer is quite correct, the two exceptions which are simply ignored will probably never happen, but they were forced to construct a lot of boiler plate code to “handle” them. And the handling is incorrect in the event that they actually do (either because the programmer was mistaken, or because something truly unusual happened, like a disk failure or an administrator deleting the directory as the program is started).

Another common example is with factory APIs such as Java’s security support:

try { MessageDigest md = MessageDigest.getInstance("MD5"); }
catch (NoSuchAlgorithmException thr) { throw new RuntimeException(thr.toString(),thr); }

As well, there’s the good ol’ URL exception with a constant (or other known good) URL:

try { URL u = new URL("http://www.yahoo.com/"); }
catch (MalformedURLException thr) { throw new RuntimeException(thr.toString(),thr); }

I have learned to never ever silently discard an exception like this. Nor do I log the stack-trace and carry on. Since the introduction of exception chaining in Java 4, the best way to handle these is as follows:

    try {
        inp=new BufferedReader(new InputStreamReader(new FileInputStream(src),"UTF-8"),1024);
        (input parsing here)
        inp.close();
        }
    catch(FileNotFoundException thr) {
        throw new RuntimeException(thr.toString(),thr); // can't happen - file must exist
        }
    catch(UnsupportedEncodingException thr) {
        throw new RuntimeException(thr.toString(),thr); // can't happen - UTF-8 is required by JVM spec.
        }
    catch(IOException thr) {
        (code here to show the user a window with the exception details)
        System.exit(0); // can't continue without config!
        }
    finally {
        // ignore IO problems with close and NullPointerException if open failed to avoid masking real exception
        try { inp.close(); } catch(Throwable thr) {;}
        }

and in Java 7 the boiler-plate can be reduced with multi-exception clauses:

    try {
        inp=new BufferedReader(new InputStreamReader(new FileInputStream(src),"UTF-8"),1024);
        (input parsing here)
        inp.close();
        }
    catch(FileNotFoundException, UnsupportedEncodingException thr) {
        throw new RuntimeException(thr.toString(),thr); // file must exist and UTF-8 is required by JVM spec
        }
    catch(IOException thr) {
        (code here to show the user a window with the exception details)
        System.exit(0); // can't continue without config!
        }
    finally {
        // ignore IO problems with close and NullPointerException if open failed to avoid masking real exception
        try { inp.close(); } catch(Throwable thr) {;}
        }

Lastly, the try/catch/ignore on close() is itself an interesting example. Close on a file input stream can never actually throw an exception, but close() is declared as throwing IOException just in case it matters for some input stream. More importantly, an exception on close will almost certainly be preceded by an exception while reading, and nothing can be done with the close exception in the finally block which would not obscure the real exception that preceded it. In the end, the finally clause is really to make a best effort to release the file handle resource.

Here, the real underlying flaw is that whether an exception should be checked is sometimes context dependent.

Unexceptional Exceptions

These occur usually as the misuse of checked exceptions. They are, in fact, checked exceptions which represent an error on the part of the caller which can be fixed by correcting the call. This would be things like using a checked exception to signal that a passed argument is invalid and other assertion-like checks.

The real flaw highlighted with these kinds of exceptions is the burden placed on the programmer to correctly determine when an exception should be checked and when it should be unchecked.

Throws Clause Proliferation

This occurs when the rule to transform exceptions across API boundaries is violated. The symptom is a method that declares itself as throwing any more than a single exception appropriate to it’s own abstraction layer. It looks like:

public void readConfig(...parms...) throws IOException, SocketException, SQLException, DOMException {

A bigger problem with this situation is that it causes abstraction leakage. Ideally the caller of readConfig show not be concerned with, nor even be aware of, the fact that someMethod uses I/O, sockets, SQL or XML API’s. This information represents leakage to the caller. The caller should be concerned with configuration errors which express an abstraction appropriate to a configuration abstraction. If an attempt is made to read an invalidly formed XML configuration file then that should be returned as, for example, a ConfigurationFormatException, not as a DOMException. Among other things, if the configuration format is JSON then an XML exception is utterly irrelevant, but a format exception remains just as applicable to JSON text as to XML text:

public void readConfig(...parms...) throws ConfigException {

Exception Coarsening

The result of another shortcut, this time an attempt to avoid lengthy throws clauses, this is when a declares that it throws some common superclass instead of specific detailed exceptions. The pathological cases are throws Exception and throws Throwable. Technically the programmer has met the demands of the compiler. However, the specific causes for failure have been lost and all the caller knows is that something can go wrong - well, gee wiz genius, I could have guessed that!

Declaring throws IOException is not much better, although many Java class-library do exactly that. The question is: “Well, what kind of I/O exception might occur”. Without that information, handling the error correctly is often impossible except in the vaguest of terms - often simply present the error text to the user and hoping that they can figure out what to do differently.

More importantly, because this kind of throws declaration does nothing to aid the caller in error handling it often does nothing more than make the programmer have to consider what to do if an I/O exception occurs, which, absent specifics, is only marginally better than using unchecked exceptions.

Interface Declares Exception Which the Implementation Does Not Throw

This occurs when a particular implementation of some pre-defined interface (small i) does not actually need to throw an exception declared by the defining interface. Consider the following class:

public class NullOutputStream
extends OutputStream
{
public NullOutputStream() {
    }

public void write(int ign) {
    }

public void write(byte[] ign) {
    }

public void write(byte[] ign, int ofs, int len) {
    }

public void flush() {
    }

public void close() {
    }
}

This class cannot throw exceptions, by its very definition, however a perfectly reasonable use-case illustrated next won’t compile:

class Test
extends Object
{
public void main(String... args) {
    Writer wtr=new OutputStreamWriter(new NullOutputStream());
    wtr.write(0);
    wtr.close();
    }
}

Why not? Because write() and close() of Writer are declared to throw IOException, and the compiler can’t know that in this case they cannot actually do so. This is a special case of phantom exceptions caused by the exceptions contracted in the interface not being applicable to the implementation.

Interface Does Not Permit an Exception Which the Implementation Needs To Throw

This occurs when a reasonable interface design declares some exception class and the implementation does not neatly (or even remotely) fit within the conceptual bounds of the declared interface. It usually occurs as the result of the interface not declaring an appropriate exception abstraction, often instead inheriting the basis exception from whatever made sense in the initial implementation (the one which demanded the creation of the interface in the first place).

This might happen, for example, when an interface is created in the context of an initial implementation using an SQL database. The interface might subsequently end up with a throws SQLException without thought or consideration given to a future implementation which sources data from text files, or a data-store, or some other database system.

This illustrates a particularly difficult burden placed on an interface designer due to checked exceptions. One has to carefully consider the broader context of the interface abstraction. But the pay-back is that the interface user gets to have a clearly defined base for the exceptions which can occur. However, since that exception base is necessarily inexplicit, the caller has no real idea of exactly what can go wrong absent documentation, which again, is little better that unchecked exceptions.

Middleware

A particular problem occurs when the calling code is separated from the implementation by some sort of generic middleware layer including that which occurs with callback style APIs. The problem is that the middleware cannot generally predict nor effectively limit the scope for exceptions which can occur. This is because it is utterly unaware of the context of the exception except in the broadest of terms.

Consider a simple generic callback interface:

public interface ObjectProcessor<T>
{
public void processObject(T obj);
}

a DirectoryTree class:

class DirectoryTree
extends Object
{
...
public void processTree(ObjectProcessor prc) {
    (walks the tree and invokes prc for each file system entry)
    }
}

and a DataStore class:

public class DataStore
extends Object
{
...
public void saveStore(ObjectProcessor prc) {
    (walks the b-tree index and invokes prc for each record)
    }
}

The ObjectProcessor for the DirectoryTree wants to throw a DirectoryTreeException while the data store ObjectProcessor wants to throw a DataStoreException. The interface’s processObjects method can’t declare itself as throwing anything useful to these two diverse implementations; at best it can declare throws Exception, which once again tells the caller nothing useful.

Unchecked Exceptions back to top

Unchecked exceptions do not require explicit handling nor a throws declaration. Consequently their biggest shortcoming is that they are easily left undocumented and easily bleed through abstraction layers. They require more discipline on the part of the caller to consider the exceptions declared in the package and appropriately transform them.

However, because their handling is not forced it is much less common for them to be mishandled. This is because their handling tends to be in the context of specific need, either because of forethought or because the exception actually occurs during testing.

I have noted, with several thousand classes written, that the theoretical problems with unchecked exceptions seem to have been exactly that, theoretical. In practice the nesting of code layers is quite shallow and many of the exceptions which could crop up in theory never do in practice.

Coded Exceptions - Taming Class Explosion back to top

One final thought pertains to the use of coded exceptions. In this style, instead of using a separate exception class for every error condition, I use one exception per package, and define specific error codes in that exception. This avoids one problem of Java exceptions, being the code bloat caused by the large number of specific error conditions that can accumulate in a large code-base.

An example might be ConfigException:

public class ConfigException
extends Exception
{
private final String                    code;

public ConfigException(String cod, String msg) {
    this(cod,msg,null);
    }

public ConfigException(String cod, String msg, Throwable cau) {
    super(msg,cau);
    code=cod;
    }

public String getMessage() { return (code+": "+super.getMessage()); }

public String getCode() { return code; }

public boolean isCode(String val) { return (code.equalsIgnoreCase(val); }

// *************************************************************************************************
// STATIC PROPERTIES - EXCEPTION CODES
// *************************************************************************************************

static public final String              SOURCE_NOT_FOUND    ="SourceNotFound";
static public final String              FORMAT_INVALID      ="FormatInvalid";
(etc)
}

One of the negative side effects of this style is that specific exception codes can be added at any time without breaking a throws ConfigException contract, which means you should always handle specific conditions with a final default other code clause. However, this is no different than an API adding a new subclass of a parent exception, for example, adding ConfigSourceNotFoundException as a subclass of ConfigException (except that it’s a lot more concise).

With this kind of exception, handling usually takes the form:

public void readConfigFiles() {
    try {
        parseFile(new File("Base.json"));

        File custom=new File("Custom.json");
        if(custom.exists()) { parseFile(custom); }
        }
    catch(ConfigException thr) {
        if(thr.isCode(ConfigException.SOURCE_NOT_FOUND)) {
            (handle missing config file)
            }
        else if(thr.isCode(ConfigException.FORMAT_INVALID)) {
            (handle invalid format)
            }
        else {
            throw new ApplicationException(ApplicationException.CONFIG_ERROR,("Unhandled error reading config: "+thr),thr);
        }
    }

Which ends up being quite clean in practice.

Conclusions back to top

Hopefully, if there is one thing that becomes clear here, is that there are no categorically correct answers. Handling errors is hard, and handling errors well is even harder. Like many things in computer programming, it is as much art as it is science.

My thinking, lately, is leaning towards the idea that checked exceptions might be most useful in library packages and less useful or undesirable in application software.

Further Reading back to top

Discussion

Guest comments are welcome - Use "Pick a name", then "I'd rather post as a guest", below.