Chickens and Eggs - Methodology and Tools

Which came first?  Object Oriented Methodology or Object Oriented Languages?

From where I was standing at the time I'd have to say Object Oriented Methodology.  Our practical experience with software led us to value encapsulation, inheritance and polymorphism - but we didn't have to tools to enforce those concepts in our code.  End result: We knew how to write good code but it was awfully hard.  We had to hand-craft coding conventions that Object Oriented Languages gave us for free.

Object Oriented Languages led to a Renaissance in programming... The tools codified the methodology - leading to unquestioned validation of the methodology - leading to wide adoption of the methodology.  Said another way: Most programmers began to think in terms of Objects.

Methodology by itself probably would not have led most programmers to think in terms of Objects.  Languages by themselves probably would not have led most programmers to think in terms of Objects - If you don't believe me go back and look at most of the C++ code that was written in the early 90s (most programmers continued thinkin in C for several years).

Object Oriented Design and Object Oriented Languages have been a huge factor in my programming career - but they aren't enough.  Objects are just a subset of what I need to solve the problems that I have been asked to solve.  I deal with Processes and I deal with Services.

Processes and Services are hand in glove - Processes Orchestrate and Choreograph Services - so any good Process Tool has got to also be a good Service Tool.

There are some really good Process and Service Tools out there - but they're mostly proprietary (expensive).  Those of use who are fortunate enough to use them have really been able to immerse ourselves in the methodologies - We think in Processes and Services.

Most of the widely available Process and Service Tools feel a lot like C++ to me... They're  Object Oriented environments that have "just enough" Process and Service tacked on to be useful - Many folks who use these tools still mostly think in Objects.

It's hard for me to convey exactly what I am talking about - unless you yourself think in Processes and Services you probably won't grok the nuances that make them powerful - but here's a stab at something that you might be familiar with.

The Spring Framework has a very good component (Spring Web Flow) for controlling the flow between web pages that are presented to a user.  If you're a Process guy like me, this is a wonderful little subset of what's available via BPMN.  You are essentially modelling a Process with a single participant as they navigate through a series of screens.

Unfortunately, the IDEs that support Spring Web Flow pretty much shuffle it off to the side.  They encourage good old OOD with this funky little XML thingy that you use to govern page flow.

A Process Oriented IDE would turn this paradigm on its head - When you started a new project you would begin with the Web Flow.  Your Pages would begin life as stubs with just enough functionality to set the data the Flow Controller needs to get to the next Page.  Instead of a "folder tree view" of class hierarchies, you'd find the elements of your project by drilling down through the Flow diagram to the Pages and the underlying Services (objects) that support those Pages. 

What I've described is pretty much what you'll find in the top-tier BPM products... Use these tools and you'll quickly start thinking in Processes and Services.

Once again this reminds me of the early days of OO - Smalltalk was impractical for most of us, so we had to make due with C++.  The mainstream programming tools were still procedural (rather than object) in flavor... but in just a few years the critical mass of programmers started thinking in Objects.  The rest is history.

My guess is that Process Methodology and Tools are at about the same point where OO Methodology and Tools were in the early 90s... What do you think?

Just Enough to Not Be Dangerous

My blog entry on User Manual Driven Development prompted Chris Adamson to share some thoughts of his own as For Reasons Unknown over at java.net.  My blog entry relates my early experience of developing an application from a pre-written user's guide.   Since then I came across a very similar approach that's used by Daniel Brolund that he calls User Guide Driven Development... obviously great minds think alike ;-)

Chris's thoughts helped me boil down my premise to a single statement:

Programmers should understand the applications that they are writing.

You might think everyone would agree with that statement... but you'd be wrong.  Go look at the comments on Chris's blog.  My favorite is the following from James Browning:

"It's a lot easier for a typical developer to know if their IDE is useful than if they're building an accounting package"

Which is s good reason why the proposed methodology has limited use. I am currently on a contract in an insurance underwriters. I am working on their accounting package and for me to understand everything about how the software is to be used and why I would basically need a degree in accountancy.

That's why we have business analysts ;-)

James has a point, of course.  Most programmers will never understand everything about how a particular application will be used... But surely they should understand something about how the software is used.  If not, then programmers really are interchangeable cogs and we all deserve to have our jobs outsourced.

So what's the minimum?  When do you know "just enough (about the business) to not be dangerous"?

The same is true for Business people.  It's very important for the Business Analysts (that specify software) to know "just enough (about software) to not be dangerous".

Those of you who are sick of my blogs on Process Driven Design should stop reading now... I'm about to make my same old pitch...

Business analysts and programmers need a common language.  Programmers need "something to read" that tells them how the business works.  Business analysts (who specify software) need "something to read" that tells them how the software works.

Business Natural Languages are a big step in this direction.  Business Rules Engines are a big step in this direction.  BPMN Environments are a big step in this direction.

Business analysts don't need to know all of the aspects of the software, and programmers don't need to know all of the aspects of business... but both sides need to understand the common middle ground.

If you  follow my blogging it will come as no surprise that I think that BPMN is a great way of describing that common ground.  The business analysts draw the boxes and connect the lines - the programmers implement all the little details that make those boxes actually work.

BPMN is evolving, and there's a raging debate on just how expressive BPMN should become. Should BPMN just describe the highest level details of a process, or should you be able to precisely express a lot of details?  I tend to agree with Tom Baeyens's views... BPMN should stick to modelling and steer clear of execution sematics.

Regardless... I know from experience that (some) business analysts can learn to "think" in BPMN.  I know from experience that (some) programmers can learn to "think" in BPMN.  I know from experience that if you combine a BPMN literate business analyst with a BPMN literate programmer you will probably end up with a kick-ass managed business process.

Another look at Object Oriented versus Process Driven Design

My friend Bob was troubled when I blogged "Object Only Programming is Silly". Bob knows from experience that life as a programmer after OO is dramatically better than life was before OO... so my criticisms just don't sit well with him. Obviously I must have gone off the deep end :-)

Bob's not one to let a friend languish in ignorance, so he recently appealed to my literary interests by enlisting Ayn Rand to bring me back into the OOO (Object Oriented Only) fold:

"Ms. Rand was obviously a fan of OOP, and would have likely loved Java even more than C++… not just from the name she gave her philosophy, but from its precepts and what they lead to… The Axiom of Existence leads to the Law of Identity… The Law of Causality (things act in accordance with their natures) is recognized as “the law of identity applied to action”.

“Rand rejected the popular notion that the causal link relates action to action. According to Rand, an ‘action’ is not an entity, rather it is entities that _act_, and every action is the action of an entity.”

"Therefore, you _must_ have an Object before an Action can be performed."

Thanks Bob, but with respect to Ms. Rand...

You don't care who performs the actions, you care that the actions get done.

Programming is the craft of transforming requirements into something that a computer can execute... so programmers need to focus on the requirements.

The Primary Requirement is always that the action be performed correctly and on time by a qualified entity (or entities)... so your design should be focussed on satisfying the action.

(Actions are service. Entities are service providers... nothing more)

The entity (or entities) that will perform each action is secondary, and should not be your primary focus. It falls out of the design rather than drives the design. Said another way... figure out the actions first and then you'll be able to figure out who (what enties) are required to perform them.

In Process Driven Design there is a direct linkage (via the process model) between the requirement for an action and the code that satisfies that requirement. The sequence and flow requirement are all right there in the model.

Contract this with your typical Object Model in UML... it captures the Actors...maybe... but you are left to figure out what the application actually does, unless of course you create an Activity Diagram (which isn't really OO any more.. it's behaviour).

Most programming students are now being trained to (maybe) write a use case and then start identifying objects... Then they'll create Sequence Diagrams showing the order in which objects send messages back and forth to each other.

So you've got a bunch of entities that are passing messages back and forth in some sequence... you still don't know (from looking at the model) what it is that this bunch of entities is trying to accomplish.

Contrast this with a BPMN diagram... by comparison you know exactly what the process is supposed to accomplish.

Process and Data lead to Objects. Not the other way around.

I don't disagree with Bob's insight from Ms. Rand: "You _must_ have an Object before an Action can be performed"... but I will add the precursor: You don't need an Object unless you have an Action that needs to be performed.

The Era of Process-Centric Design

If you follow my postings, then you won't be surprised to learn that I think we are at the dawn of a new era in programming. As with all previous eras... nothing is really new, it's just a point in time where existing good ideas rapidly ascend to a pervasive level of acceptance.

We are at the dawn of the Process-Centric Era

During the Object-Oriented Era, encapsulation, polymorphism, inheritance and reuse were king.... all excellent ideas... But not enough to insure the creation of agile applications... applications that can easily be adapted to meet changing requirements.

Quite frankly, adapting to changing requirements is tough and unless you set some limits and constraints it's never going to get easier... and that's where Process-Centric design comes in.

On our desktops, most of the applications that we know and love are not process-centric. Our word processors, spreadsheets, paint programs and games are marvels of ad-hoc creation where our own whims and preferences guide composition rather than any pre-ordained process. The Object-Oriented paradigm was perfect for creating these sorts of applications, because the OO paradigm sets very few constraints on the developer. Imagination and creativity are the OO developers only limitations.

Imagination and creativity are wonderful things... but a huge percentage of the applications that the world needs are not concerned with creativity and imagination... they are concerned with implementing mundane processes.

Implementing processes within the guidelines OO paradigm is not by itself particularly difficult to do... but the flexibility of OO does not force the developer to implement the process in a manner that will make it relatively easy to modify the process in the future.

Processes constantly change. As businesses grow, as business conditions change, as the whims of the population at large ebb and flow... processes need to change, and the applications that implement the processes are often the limiting factor in how quickly the processes can change.

Get a group of Java EE architects in a room, not the R&D infrastructure and product guys, but the ones who build applications for business groups, and listen to their war stories. We all seem to have the same story about the project from hell which required each developer to know half a dozen diverse technologies to simply modify a screen... JSP or JSF (with embedded AJAX) on top of Struts (but we're migrating to Spring) on top of EJBs on top of a Relational DB (except for the part that uses Hibernate) and the use of JMS at one point and Web Services at another... And by all means don't forget the custom rules-engine and half-a-dozen XML-to-Java-to-SQL libraries.

We've all implemented scores of business processes, but we've implemented them as a loose confederation of somewhat related "point" applications that each implement some aspect of the process (with ad-hoc mechanisms for passing information and process flow), or we have implemented them as uber-monolithic behemoths that are a mess to maintain.

That's where the Process-Centric paradigm "comes to the rescue". It's not a general purpose paradigm, like OO, but it's just as significant. The Process-Centric paradigm imposes order. It is a domain-specific paradigm that first and foremost establishes the Process as the leader. A project may still incorporate diverse technologies, but those technologies have to comply to the dictates of a Process-Centric architecture. If you won't play by the rules, then you can't play.

In the Process-Centric paradigm the Process Definition is decoupled from the Activities that are brought together to implement the Process. Communication between the Process and the Activities, and between the Activities themselves, is through well defined message-oriented interfaces. In essence, the activities are implemented as Services (as in SOA)... but some of the Services are automated, and others are human-powered. Once in the realm of Service implementation, programmer imagination and creativity can once again assert themselves to perform the task in the optimum manner... as long as this creativity doesn't try to short-circuit the process framework.

The end-result should be truly process-centric implementations that are primed for the inevitable process level changes. The programmers do give up significant freedoms, but they end up with fewer hassles down the road, and probably a much improved relationship with their business colleagues.

Both Business and IT folks are tired of the status quo where each side feels besieged by the other... By our very natures there will always be a disconnect between the two sides (we're very different types of people) , but hopefully by introducing an intermediary that we both understand (the process definition) life in the Process-Centric era will be better for all of us.