Objectopia

Author Archive

The arrogance of Gantt charts

Posted by Chris Burnley on November 17, 2009

Gantt charts, popularized by Microsoft Project, are a mechanism for managing projects. The idea is that you list all the tasks required to complete a project. You then estimate how long each task will take. Once you figure out the dependencies between the tasks you the layout a plan from left to right connecting up all the dependent tasks. This allows for some parallelism between non dependent tasks. Given the project start date and all the durations you get the project end date.

As long as everyone sticks to the plan, then the project will be completed on time.

Simple, eh? Err – not so fast! The major flaw is that Gantt charts are task focused so when a new task is discovered the project is guaranteed to be late. The reason behind this is that developers are focused on delivery of the tasks they are assigned – not value to the client. Coupled with the truism that tasks will take as much time as they are given and you will be late.

The task-centric approach only hints at the real problem with Gantt charts: that better planning and task discovery up front would have resolved these issues. This is the most deplorable aspect: God-like wisdom can be achieved and would have fixed the problem.

Historically, software development has used the notion of “defined process control” where all aspects of the plan can be controlled, to a low-level and to a high-degree of accuracy. This does not work for most software projects because software development is not a simple undertaking- it is usually a complex process where small changes in the inputs can greatly affect the outcome.

Big, upfront planning that depends on infinite wisdom does not work and the only mechanism we have for complex problems is Empirical Process Control. This is the notion of frequent inspection and adaptation to a process.

Emperical control is _the_ key to Agile software. You’re not are Agile if you don’t have a regular process for improvement – you’re just hacking away.

Gantt charts have no place in an agile process; they are the tool of bluff used by the project manager that really doesn’t understand software development.

Posted in Opinion | Tagged: , | 6 Comments »

Thought for Friday: What are we going to do about the crap code ?

Posted by Chris Burnley on July 24, 2009

Yesterday, my manager asked me “What are we going to do about the crap code?”

It’s a hard question to answer because before we can answer what to do about it; we must understand what crap code is and what causes it.

What is crap code? This is still a hard question, but I think we can at least attempt to answer it by defining what it isn’t:

  • Crap code isn’t maintainable
  • Crap code isn’t easily testable
  • Crap code isn’t as efficient as it could be
  • Crap code isn’t understandable without lots of comments

What causes crap code? In my experience it is some combination of:

  • Ignorance
  • Fear
  • Lack of experience, pride, oversight, accountability, time and analysis/understanding of the domain
  • Resume-building perhaps?

So what is the solution? There is no silver bullet, it requires vigilance and potentially each of these causes needs to be tackled individually.

Ignorance: One way of tackling this is encouraging learning, peer review and discussion in the organization. Time needs to be allocated to ensure that project work doesn’t stop these activities from flourishing.

Fear: One of the major differences between a good developer and an average one is the fear factor; good developers don’t have the same fear of changing something, or fixing something since they are confident they are not going to break something without knowing about it. Because of this, fear has the effect of destroying reuse – copy and paste code thrives in this environment. Besides learning how to study code, unit testing and other forms of automated testing are good for addressing this.

Experience: You need to install a mentoring system to ensure that the less experienced team members have someone to go to for advice. Team members need to be exposed to different types of development work. E.g. database, business-logic, front-end / web etc. They’ll never get enough experience doing the same thing day in, day out. Ensure every commit to the code base is emailed to the rest of the team, experienced developers should be reviewing the code and suggesting changes if required.

Pride: This is a hard one, and probably comes from the side effects of the other causes. One way is to allow team members to look after a particular module, service / sub-system.

Oversight / Accountability: Oversight is simple – review code and don’t compromise on quality – you’ll end up regretting it at some point. Generally speaking accountability decreases the more segregated the teams are.  I.e. if you have a team of testers that are separate to your developers, the less responsibility the developers will feel to getting something right – you’ll get a “throw it over the fence” mentality. Create a team with all disciplines (developers, testers, dbas, infrastructure etc.,,).

Time: This is about planning properly and ensuring that developers are aware of all the steps to produce bit of functionality. It’s also about being realistic when it comes to providing your estimates. Make sure you include time for producing your unit and integration tests, and don’t underestimate just so you get the job. Stress to developers that the job isn’t done until it is tested and works.

Resume-building: This is subjective but I’ve been on projects where the tech lead has read that this new piece of technology is the ducks and they wants to get in on it just so they can put it down on their resume. They manage to get all the newest and greatest stuff in there but at what price? The code under the covers becomes a maintenance nightmare because it is far to complicated for what it needs to do. Ensure that the technology you use is decided up front. Don’t deviate unless you get further sign-off from the stakeholders. Create a list of “guiding principles” (N.B. not coding standards) to decide how you’re going to go about writing the code – and enforce code reviews on these.

Understanding the domain:  When you start a new project it is really important, that at every step in the process you constantly to try to refactor based on your current understanding of the code. Data models are really hard to change once a system goes into production, so you’ve got to make an effort to get it right. If some aspect of the design doesn’t feel right, then it probably isn’t it. Don’t be in denial – fix it.  I’ve seen so much bad code and bugs caused because the new requirements or understanding about a system are applied to a broken domain.

The bottom line is, does crap code really matter? Who cares if it works, right? I think it does. And crap code is directly proportional to defects in my experience.  Defects will cost you money fixing them and cleaning up the consequences of them.

Fixing the crap code is no doubt a hard problem. The solutions involve constant vigilance and culture change – but it is worth it in the end.

Posted in Thought For Friday | 4 Comments »

We only hire the best

Posted by Chris Burnley on July 23, 2009

I was recently having dinner with an acquaintance who works for a massive multinational IT company. He was telling me that at a recent company conference that, despite the economy, the CEO said the company was doing well because “we only hire the best”.

I almost choked on my steak, trying to contain myself. The sad thing was that he actually believed it. How come all companies that I’ve worked at only hires the best ? Surely all of them can’t have the best ?

Why do companies maintain this illusion ? Is it some sort of psychological trick so you don’t quit and join a company full of bozos ? Or do people at the top truly believe it, in some sort of paradoxical delusion ?

I don’t know, but it’s odd isn’t it ?

Posted in Opinion | 1 Comment »

Routing around damaged processes

Posted by Chris Burnley on July 16, 2009

It is often said that “The Internet interprets censorship as damage and routes around it”. I don’t know how true this statement is, but I have seen this affect applied to organizational processes.

When you work at a sufficiently large organization, you’ll start to employ people who’s sole responsibility is creating process. The problem with these roles is that, if no new process is created, then those people are not seen to be doing their job.

Once they get started, there’ll be a process for each aspect of developing software, production releases, production data changes, process for creating a process, etc. etc. ad nauseam. (I’m not saying that there is anything wrong with processes, but more that it unsustainable to continue to create them indefinitely).

This usually has either two outcomes:

  1. good people who just want to do their jobs get fed up with the red tape and leave or;
  2. people “route around the damage“d process.

I see #2 happen all the time. The problem is unnecessary risks are taken because it has just gotten too hard to do anything.

How do you solve this ? Ideally, the people that author the process should also have to follow it themselves. At the very least, you should involve the people that have to follow the process in its creation.

The most important thing is, you have to make following the process easy and fast. If you slow people down, they will just route around the damage (you!).

Posted in Opinion | Leave a Comment »

Please ! Stop using floating point numbers for financial data

Posted by Chris Burnley on July 3, 2009

Again and again I see developers using doubles for storing financial data (e.g. monetary figures, interest rates etc.). This always causes some sort of rounding issue down the line.

try this in your IDE:

double total = 0;
for (int i =0; i <  10; i++){
  total += 0.1;
}
System.out.println(total);

If you expect the answer to be 1 or 1.0, then you’re wrong !

Question: are you writing a game ? Answer: no; then use an effing BigDecimal!

Grrr…

Posted in Grrrr..., Java etc... | 2 Comments »

Non-anaemic models and service oriented architectures

Posted by Chris Burnley on May 20, 2009

The term “anaemic domain model” is used to describe an entity layer that contains very little business logic. Typically in this architecture, the business logic is contained in services and/or delegates. There are several problems with this approach:

  • Business logic quickly gets scattered and duplicated across the system
  • It is difficult to maintain because it is hard to determine which class needs to be changed
  • It is difficult to test because the service/delegate usually has many dependencies
  • It is not particularly object oriented (e.g. polymorphism, which can reduce a lot of complexity by reducing ‘type-logic’, is rare)

The trend these days is towards Domain Driven Design (DDD), and is a strategy against anaemic domain models. In this architecture, services are used as an ‘anti-corruption-layer’ to translate requests from the external domain to the internal domain of the system. The majority of the logic is contained within the domain model (this includes persistent and non-persistent objects).

When developing a non-anaemic model (e.g. DDD), it is tempting to call services directly to perform logic (to perhaps an external system, or send a message), to hide complexity from the client:

class Account {
    Balance getBalance() {
        for(Transaction t : getTransactionService().getTransactions(…)){
            //… sum them up
        }
    }

    TransactionService getTransactionService() {
        // lookup service
    }
}

I think this not good design for a number of reasons:

  • The risk of Leaky Abstraction (e.g. having to declare checked Exceptions to methods)
  • Generally speaking, the domain model doesn’t have enough information about the nature of the method call to perform well. The service would be better suited to know how the model will be used (e.g. in a loop).
  • It is harder to test the business logic because you’ve got to mock out too many dependencies.
  • I believe that entities should be deterministic, the behaviour must be determinable given the state of the system/database.
  • Potentially introduces circular dependencies between service->model->service->model etc. which causes problems for maintainability, testing and debugging.

So how do we have a nice, clean, rich domain model ? One way is to pass an implementation of an interface to the domain (e.g. a delegate to a service) so the domain can perform its logic:

class Account {
    Balance getBalance(TransactionRepository repos) {
        // …
    }
}

There is a subtle difference between the two, but the second IMO is a cleaner, more-flexible solution.

Posted in Java etc... | 1 Comment »

Re: A general approach to exceptions in Java

Posted by Chris Burnley on March 23, 2009

My previous post about exceptions in Java begs the question: Where do so called “business exceptions” fit in with all of this?

Business Exceptions are Checked Exceptions

Following on from my reasoning, the user is something that you have no control over. Therefore you expect these types of exceptions to occur. Making them checked exceptions forces you to handle them in some way.

A classic example of these errors are validation errors. These should be checked exceptions because usually you’d need to force the user to fix the entered data before the form can be submitted.

Dealing with Checked Exceptions

The problem of checked exceptions is: how do you handle them ? You’ve got to catch them somewhere (or rethrow), how do you know what to do with them ? This is usually a requirements question; this should be a hint to you to go back to the business and get some clarification on the requirements.

Posted in Java etc... | Leave a Comment »

Persistent Strategies

Posted by Chris Burnley on March 22, 2009

I’ve just finished working on a cash management system. The core of what the system does is listen to messages produced by various sources and record a transaction (i.e. a debit and a credit) between two accounts.

For example, the system will listen to trade messages produced by our trading system and charge a trade fee to the client. This involves debiting the client and crediting the Fee house account.

This seems innocent enough, but to create a transaction you must have a reference to each of these accounts.

The client’s account is easy enough to find as all the required information is supplied in the trade message. But what about the house account ? How does the system find an instance of the house Account ?

One could do something like this, store the account number in a constant somewhere:

Account account = accountService.getAccount(FEE_HOUSE_ACCOUNT_NUMBER);

// create Transaction with client and house account

This is pretty bad IMO because there is duplication and “hard coding” ( even if it is in a properties file) of what house account numbers there are.

If you didn’t want to store the account number in code (or in a properties file which is almost as bad) you could add it to the static data for that transaction type:

@Entity
class TransactionType {
    String name;
    Account creditAccount; // this could be null to indicate that the account is based on the message
    Account debitAccount; // this could be null to indicate that the account is based on the message
    //...
}

then you would just find the TransactionType and the get the credit or debit account.

This would work, but it is very specific, since sometimes we may want to find the debit / credit account in a different way (e.g. “find the loan account associated with the client’s cash account”).

Enter the Persistent Strategy.

Basically, the idea is that rather than store links directly to Accounts in your static data, store a Strategy to obtain the right Account.

@Entity
public class TransactionType {</code>

    private String type;

    @ManyToOne
    private AccountAllocationStrategy toAccountStrategy;

    @ManyToOne
    private AccountAllocationStrategy fromAccountStrategy;
}

@Entity
@Inheritance
public abstract class AccountAllocationStrategy {
    public abstract Account getAccount(Transaction transaction);
}

What we’ve done is added a strategy for find the from / to account and we can have specific implementations:

@Entity
public class FixedAccountAllocation extends AccountAllocationStrategy {

    @ManyToOne
    private Account account;

    public Account getAccount(Transaction transaction) {
        return account;
    }
}

@Entity
public class InvestorAccountAllocation extends AccountAllocationStrategy {

    @Override
    public Account getAccount(Transaction transaction) {
        return transaction.getInvestorAccount();
    }
}

(there are a few more implementations I’ve left out for brevity).

You could achieve a similar effect by using a code of some sort (e.g. an enum) to denote the strategy and look up the strategy instance via Spring, but I like this better since it is more transparent and doesn’t require involving other frameworks. The major problem with the Spring approach is you’ve still got to specify the account numbers somewhere in your Spring context file which is duplication (once in the Spring file, once in the database) which, we all know, is bad :)

It might seem like abusing entities or the domain model but it is a really powerful pattern. Think of these entities as your rule domain. JPA makes mapping entities really easy and the number of strategies are finite so you can be quite creative in your mapping and inheritance hierarchies. You can also fully cache the strategy entities so performance will not be an issue.

We use this pattern in a few places in the system and it has made the code much cleaner and maintainable.

Posted in Java etc... | Leave a Comment »

A general approach to exceptions in Java

Posted by Chris Burnley on March 22, 2009

Checked vs. Unchecked Exceptions in Java. The topic is pretty much as controversial as it comes. The debate over when to use a unchecked (those that are subclasses of java.lang.RuntimeException) vs. a checked (those that do not extend RuntimeException) is as old as Java itself.

I’ve witnessed many a debate over the topic over the years. When to use unchecked ? Are checked exceptions even a good thing ? Everyone has a different opinion on the subject so I’ll attempt to explain a general approach that hopefully will be helpful to decide when to use one type of exception over the other.

Both checked and unchecked exceptions are useful

Yes, both exception types are useful. You will sometimes find extremes of people who will only use one or the other, but by doing this you will not be able to benefit from the unique properties of each type: Unchecked exceptions don’t obviously have to be declared as part of the method signature, so the exception does not need to be handled or exposed in all the method signatures up the call stack. Checked exceptions are the opposite, they force the caller to either deal with the exception or inform its caller that it needs to deal with it and so on. These are both unique characteristics that are useful in different circumstances.

When to use each type

Exceptions are thrown because there is some sort of error that the system cannot deal with. An error can be categorised as either “there is an error in the code” or “there is an error with the system’s operating environment”. These two categories can can distilled into “an error that can be prevented”, vs. “an error that the system has no control over”.

So, if the system was perfectly designed, i.e. no bugs, then the first type of exception (an unchecked exception, an error in the code) should never occur. These are RuntimeExceptions, since there is no point bothering the caller of your object with errors that should not occur ! Examples of these are NullPointerException (check for null !), ArrayIndexOutOfBoundsException (check the length of the array !) etc.

The other type of exception, checked exceptions, are completely out of the control of the system and clients should be forced to deal with them. Examples of these are JDBCException (you can’t control whether the database will be available), FileNotFoundException (sure, you can check to see if a file exists, but you can’t guarantee it’ll be there when you read it) and RemoteException (you can’t ensure that the remote service will be available or bug free).

Perfect code vs. perfect environment

So, in summary I follow the following rules: use unchecked exceptions for errors that can be prevented and checked for those that cannot. It can also be thought of like this: If you have perfect code there should be no runtime exceptions. If you have a perfect environment there should be no checked exceptions. Obviously neither of these are possible, this is why we need both types of exceptions.

Posted in Java etc... | Tagged: | 1 Comment »