Tips for the Business Process Developer - Getting Tasks to the Right Participants

Recently, I was working with a client to implement a managed Loan Origination Process using Lombardi's Teamworks BPM suite...

(Obligatory notice: I am an employee of Lombardi Software, but the opinions and advice presented here are my own.)

As with any real-world process, there were a few interesting "gotchas", but on the whole it was a pretty standard process that many of you would easily recognize.

Modelling the Loan Origination Process using BPMN diagrams was fairly straight-forward, but when we got to the point of implementing Task Assignment and Routing things got interesting. Let me very loosely paraphrase some roughly similar Task Routing requirements (that I made up for this blog entry):

"When a Loan Application comes in, make the application available to all of our Credit Officers who have sufficient Lending Authority to process the loan, and who haven't already met their quota of applications for the week. Of course if everybody has already met their quotas for the week, then let everyone have an opportunity to work on the new application.

One more thing, only a few of our Credit Officers can approve loans over $100,000, so don't route low value loans to them unless they are the only available credit officers.

And don't let anyone claim more than 3 applications at a time. Our Credit Officers are compensated for the number of loans that they process, and some of them will claim all the applications that they can in hopes of earning more money."

These Task Assignment/Routing requirements are kind of involved, but they make sense in the real-world. If you can't handle real-world problems in your managed business process then you aren't going to make many business people happy; So how do you satisfy complex Task Routing/Assignment requirements like this?

There are three key elements to any business process:

1. The steps and flow of the process
2. The data the process generates and consumes
3. The participants who perform the process

Task Routing deals with the Process Participants: How do you know who the right participants are and how do you assign tasks and route information to them?

All BPM suites have one form of task list or another that holds all of the tasks that are assigned to a user and all of the unassigned tasks that the user can claim. Sometimes the task list appears on some sort of a Portal. Sometimes the task list is integrated with a mail client like Microsoft Outlook. Often task notifications are emailed to users. Task Routing in a process is all about getting the right tasks to show up on the right user's task list at the right time.

A BPMN swim lane indicates the Role (in the process) of the Participants who execute any of the Tasks in the lane. These BPMN Roles can be very specific, both at the process definition level and at the process instance level. For example, an individual who is acting as a Reviewer in one instance of a process may be acting as a Requestor in another instance of the same process.

In many processes, a single task may be routed to a group of people rather than assigned to a single user. Let's refer to this group of users as the Task Group. All of the Task Group members are eligible to claim the task until one member of the group claims the task.

If you've dealt with Authorizing a User in a "normal" application, then Assigning a Task in a process will seem very familiar. When people log on to web-based applications there is almost always a two step process occurring under the covers: Authentication and Authorization.

Authentication is conceptually simple: Is the user who they say they are? This is usually handled by prompting for a user id and a password. If the two match, then the user is someone who can log on to the system. In Single Sign On environments, authentication is generally handled by a token (such as a SAML assertion ) that vouches for the user's identity.

Authorizing a User is often a bit more involved than Authenticating a User: Is the user allowed to perform a specific task or view some specific information?

Authorization is less straight-forward than Authentication because a single application may be written to service users with significantly different permissions to perform tasks and to view information. If the logged on user has specific permissions, then they can perform specific features and view specific information. Conversely, if they don't possess the proper permissions, then they cannot access some of the application's features or view some of the information.

The relationship between Task Assignment and User Authorization is pretty obvious:

Authorization logic is applied to determine whether or not the logged on user can perform a task or view some information.

Task Assignment logic is applied to determine the group of users who are authorized to perform a task or view some information.

Many organizations use an LDAP repository like Active Directory to hold the Authorization information associated with users. In this model, users are assigned to specific groups and Authorization is fairly straight-forward. Assigning a BPMN swim-lane to users who are members of a specific LDAP group is a no-brainer.

Unfortunately, in the real world the membership of a Task Group can be very dynamic... Here is an example slightly modified from the loan origination process that I worked on:

• The Task should be assigned to all Credit Officers with a Lending Authority Limit of more than $50,000 who have processed less than 5 applications this week.
  

At first glance, it seems reasonable to create an LDAP group for Credit Officers with Lending Authority over $50,000... but it's really not. The required Lending Authority Limit is actually based on the amount of the loan. You need a Credit Officer with authority greater than the amount of the loan, so this is really an attribute of the Credit Officer rather than a specific group that they belong to. The second half of this requirement, "who have processes less than 5 applications this week", is an extremely dynamic categorization... There's no way that you would store this sort of dynamically changing data in an LDAP repository.

To a Business Process Analyst, specifying a Task Routing rules like these are very straight-forward. To the Business Analyst, it's extremely clear who should be eligible to claim the task, and who shouldn't be able to claim the task.

To a Business Process Developer, trying to implement a Task Routing rule like this is a nightmare. It's usually very unclear how to access the data necessary to implement this rule.

Enter the Task Routing Service...

The "right" answer to solve a complex routing problem like this is to develop a custom Task Routing Service to determine the list of users who should be given the opportunity to complete the task. If the conditions for eligibility are very dynamic (if they could change in a few minutes) then it's also a good idea to develop a related service that will tell you if a specific user is eligible to claim a specific task.

When a task is ready to be performed, invoke the first service to get all of the eligible users.

When a user claims the task, call the second service to determine if they are (still) eligible, and return the task to the pool if they aren't.

Encapsulating the Task Routing Policy (the logic to determine the eligible users) in a separate service frees us from having to worry about the details of determining those users while working on the other aspects of the managed process. Our custom Task Routing services can implement any routing policy that is necessary.

Refining the Task Routing policies is often one of the areas with the most penitential for improving the overall performance of a process. If tasks are languishing on a queue, waiting for someone to claim them, then nobody is happy. If "simple" instances of tasks are routed to the "most skilled" participants, that's probably not the best use of resources (and once again nobody is happy).

The Task Routing Service that I've described for this loan origination process requires access to information from several sources:

• Information from the current process instance
  
• Information about all of the possible participants
  
• Cumulative information from "related" process instances
  

Getting information about the current process instance is never a problem. Since the Task Routing Service is called from within the process itself, any relevant information can be supplied.

In this specific process implementation, the possible participants (the Client Officers) are defined in a database table... There is not a group in LDAP that distinguished the possible participants. The same table holds attributes of the Client Officers that are needed, such as "Lending Authority".

Cumulative information from related process instances is often the hardest to come by. This is generally the information that is used to load-balance the workload of each participant, and the total workload of any participant may be dependent on many types of processes (not all of which may be managed). Frequently, when a project is begun to implement a managed business process, some of the information necessary to implement a complex Task Routing Policy will not be available (and may not become available before the first release of the project).

So what do you do if the information that you need for your Task Routing Policy isn't available?

Do the best that you can with what you have.

The first release of the Task Routing Service for the Loan Origination Process that I have described will be implemented as a Human Powered Service. Prior to this BPM project the task routing was performed by a person. When new Loan Applications were received a central "Traffic Cop" would review each application and assign it to a specific Credit Officer. This manual assignment process was actually working quite well since the number of Credit Officers was small and the "Traffic Cop" was really good at the job. Although the ultimate goal is to automate the task, we are "punting" and will create a UI for a human to make the routing decision in our Task Routing Service.

Here's another key reminder: Don't spend a lot of time and effort (and money) implementing a complex Task Routing Policy unless it really is needed. Back to my example, the human "Traffic Cop" routing service really may be the right solution for the present. Until the "Traffic Cop" is overwhelmed by volume, replacement of this routing service with an automated one might not provide a reasonable ROI.

Task Routing in a managed business process is often very simple and easy to implement with the out-of-the-box functionality of a BPM suite. For those times when it's a bit more complex (or maybe even insanely complex) don't despair. With a well-designed Task Routing Service you can handle almost anything.

Why do Java developers hate BPM?

Java developers hate BPM.

The preceding sentence is (of course) intentionally tailored to be controversial. People tend to read controversial blogs, and I'd like you to read this one. Now that I have hopefully grabbed your attention I'll tone it down a bit...

A lot of Java developers hate having to use BPM tools instead of the object oriented tools that they are comfortable with.
I work on a lot of BPM projects, and I work with a lot of other folks who work on a lot of BPM projects, and we have all encountered resistance from traditional Java developers. Java developers (in general) would rather use frameworks like Struts and Spring than be saddled with the constraints of a BPM suite.

Java frameworks like Struts and Spring are in the background... they provide just enough support to "set your creativity free" so that you can be a real programmer. You can build almost anything with Spring or Struts (if you've already mastered the intricacies of Java). They are light-weight, they're agile, and they look sexy on your resume.

BPM suites are in-your-face. They rob you of your creativity. They dictate to you how you will develop your application.

BPM suites make programming boring. They force you to use point-and-click and drag-and-drop tools to design your process diagrams, data models and forms. What's worse, they actually encourage Business People to model processes and design forms on their own... Fortunately most Business People are too intimidated to use these tools, but it does open the door for them to look over our shoulders and meddle in our affairs.

That certainly doesn't sound like something that real programmers would like, does it?

I'm not being subtle, am I?

BPM suites are a threat to traditional Java programmers. These suites are far from perfect, but even in their current state we can see where things are heading. The days of the Java Guru as indispensable are fading... We've used Java to build tools that make knowing Java itself less important, and that's opened up competition for us from folks who didn't spend years learning Java.

We're victims of our own success... programming isn't as hard as it used to be... and that's going to cost us.

That's why Java developers hate BPM.

Process Visibility

"Succeeding at process optimization requires a user to have unrestricted visibility to all process elements, their relationships and how those have changed over time."

The preceding is a quote from one of Lombardi Software's training manuals, and I think it highlights a key guideline for architecting successful applications in a Service Oriented environment (Obligatory disclaimer: I work for Lombardi, but the opinions and ideas expressed in this blog are my own).

If you can't see it, you can't fix it.

"It" in this case refers to any of the activities or services that must be performed in order to successfully complete a Business Process.

In a SOA environment Services are not generally created for a specific Business Process. Services somewhat mirror their real-world namesakes... Businesses seldom provide a Service that is only of use to a single customer (unless that customer is very, very important).

In a similar vein, Services are seldom unadvertised. Generally speaking, the ROI for developing a Service is closely related to the number of customers who consume the service (which is why advertising is crucial).

I recently worked on a project that required delving into employee details that were spread across LDAP and a relational data base. Much to our chagrin we were far into the project before learning that a WebService provided all the data that we needed in a one-stop shop (so to speak).

If you don't know that it exists, you can't use it.

The bottom line in Business Applications is to get the job done in as reliable and efficient a manner as possible. As experience is gained and external conditions change, the procedure for "getting the job done" may change.

The Holy Grail for Business Programmers is to never be the limiting factor for how quickly the business can adapt... Actually it's beyond that... The ideal Business Programmer should be able to help the Business Analysts know when and how to adapt.

Business Applications do not require fancy and finely tuned User Interfaces... They require effective User Interfaces that can be quickly adapted when the business process needs to change. Sadly, many well-meaning Business Analysts focus more on the aesthetics of the applications that they want rather than on the business tasks that the application supports.

If elements of the UI don't closely map to Process Steps, then it probably won't be easy to change the UI when the process has to be changed.

Back when OO first came into vogue many UI developers made the mistake of tying their UIs too closely to the underlying objects. While this is not always a bad thing, as is evidenced by the popularity of NakedObjects, it often led to quite unnatural interactions between users and applications (unnatural from the human's perspective). We could make a similar mistake by tying a UI to closely to a Process Step... but it's a risk worth taking.

Objects are Nouns... Services are Verbs... SOA hinges on designing the right Verbs

I still find postings that express confusion about the relationship between object-oriented architectures and service-oriented architectures... so I would again like to offer my own $0.02 worth of insight...

Objects are Nouns; Services are Verbs

That's it. That's all there is to it.

An Object is primarily a Thing that is manipulated (and is generally stateful), a Service is primarily an Action that is performed (and is generally stateless).

In an object-oriented architecture, the software components are primarily organized as entities (things) that are to be manipulated. These entities generally have relationships to other entities, and that leads to the emphasis on design artifacts such as UML class diagrams.

In a service-oriented architecture, the software components are primarily organized as tasks (actions) that are to be performed. These tasks are generally designed to function as steps in more complex processes, and that leads to the emphasis on design artifacts such as BPMN diagrams.

Businesses use software to automate business processes. Business processes do involve Nouns (Objects), but they are generally more concerned with Verbs (Services).

Let me explain: Most businesses process forms of one sort or another. In general, manipulation of the form (an object) is not the primary concern of the business; the primary concern of the business is to perform a set of tasks (services) based on the contents of the form.

If the software components are packaged as distinct services, the mapping between the business requirement to "process the form" and the software implementation of the process is relatively straight forward. This clean mapping between requirements and implementation results in solutions that can better respond to changing requirements over time (Trust me, all business requirements change over time).

Sun's Tim Bray has recently expressed his opinion that the term "SOA" is meaningless, and that he prefers the term "Web-style". With respect to Tim, I disagree with his opinion.

I do not agree with Tim that "Web-style" is a better term than "SOA".

I agree with Tim that the term SOA has:"become damaged, weakened by over hype, over use, over promise (and) under deliver", but unlike Tim, I think that I can "explain what the difference is between SOA and Web services". Web services are a protocol specific form of service. To implement an SOA you must eventually pick a protocol, but it needed be a Web oriented protocol.

Tim asks for "an explanation that's meaningful in terms a programmer can understand, so a programmer can go and build one of these things". Perhaps it is my own arrogance or naivete, but I think my explanation is a darn good start.

In my experiences with many developers, I have found that explaining how to design and build SOA solutions is not any more difficult or "verbose" than it was to explain how to design and build Object-Oriented solutions. I confess that I do resort to "paragraphs and paragraphs (of) prose", but everyone who knows me understands that I always resort to paragraphs and paragraphs of prose ;-)

But if Tim is right, and the term SOA has become a dead alabatross, then I'd like to offer an alternative to "Web-style": How about Process Oriented Architecture? Kind of has a nice ring to it; Don't you think?

Regardless of the acronym or term that we use to describe it, the point remains the same: If your primary task is to automate a business process, then you should package (most of) your software components as verbs... er, I mean services.

(cross-posted at my java.net blog)