I Love NoTechlogy!

No, that's not a mistype... It's not Technology's praises that I sing tonight, it's No-techlogy's praises.

Notechlogy is the power behind the scenes that makes all the wonders of modern life possible... Allow me to give you an example:

Weather is crap all over the US today. The Northeast is buried under record snowfall, the Midwest is shut down due to ice and sleet, and I am stuck an extra night in a hotel room in Austin.

This could be a miserable night for me, but in fact I am comfortable and content... all because of the wonders of NoTechlogy.

Let me explain... When I plan a trip, I just log on to a travel agency's web site, tell it where I need to go and when I need to get there, and it finds me a selection of flights that meet my requirements. All I have to do is make a selection and show up at the airport.

Through the magic of Notechlogy, I don't have to do a darn thing to notify the airline, which in this case is American Airlines. American knows me well, assigns me the kind of seat that I like, and usually gets me where I need to go at approximately the time I need to get there (nobody's perfect).

Because I am a pretty frequent customer - American, or more precisely an employee of American usually gives me a call if they have to cancel a flight... Today they cancelled a boatload of flights (poor metaphor, but you know what I mean). Through the power of Notechlogy, once a decision is made (by a person) to cancel the flight, another person books me some new flights and makes every effort to let me know what's up.

I'm often unable to answer my phone - so I set up a Google Voice account. Google Voice is supreme Notechlogy. When someone calls my Google number, a series of phone numbers that I've specified gets called... If I don't answer any of those numbers, Google takes a message.

Now the awesomeness of Notechlogy really shows up. When I check my messages at Google Voice - I see a pretty darn accurate printed transcript of my messages. I don't even have to play back the actual voice recording most times.

Today, I just checked my messages, saw my cancellation notice and new flight times, and called the hotel to extend my stay. Piece of cake.

But wait... you say... that's TECHNOLOGY that's making all this possible. Why the heck are you giving credit to something called NOTECHLOGY?

My dear readers... Technology is something that only nerds love. Technology has flashing lights and whirring motors and arcane acronyms to obscure it.

Technology is what we always envision the future to be full of.

Notechlogy on the other hand is that wonderful stuff that works so well that you take it for granted. I never think of my iPhone in terms of Technology. It's just a wonderfully simple to use communication and entertainment system that I can carry around in my pocket. Likewise the cars that I drive. I don't think of them as a collection of incredible technologies that can propel me around town in safety and comfort on a few ounces of gasoline... I think of them as transporters that get me to where I need to go.

If you've ever seen a re-run of the original Star Trek series, then you've seen a "future" that is full of technology. Lots and lots of flashing lights drawing attention to amazing gadgets. Think of the "Tricorders" for example... Beeping, Humming, incomprehensible Readouts... not just Technology, this was honest-to-god TechNERDlogy.

If Captain Kirk had been seen with an iPhone the viewers would have rebelled. It simply does too much - and in too natural a fashion - any simpleton could use it, so why would Kirk have one?

Fortunately for us, the "future" that is dominated by TechNERDlogy isn't going to happen. Technology is quickly evolving to Notechlogy - It's there, it's powerful, it's really really helpful, and the best part of all it that we don't even have to consciously learn how to use it.

I Love NoTechlogy!

BusinessScript - COBOL-izing JavaScript?

My friend Scott Francis recently posted some thoughts on Server-Side JavaScript over on his BP3 blog... We're both Lombardi Teamworks affcianados and well aware of the advantages of using the same language on both the Client and the Server.  It's a safe bet that JavaScript is here to stay.

JavaScript was created by Brendan Eich to give Web designers and part time programmers a "glue language" for adding dynamic behavior to HTML coded Web pages.  Brendan succeeded beyond his wildest dreams in making Web pages dynamic... but not so much on his goal of creating a language that was easy to use by amateurs and novices.

My work involves a lot of interaction with business people who aren't frequent programmers... and I often introduce them to JavaScript.  Mostly they "get it"... but usually they get a bit frustrated at the tedium required to do really common things.

Try explaining to a business person turned part-time programmer the JavaScript (and DOM knowledge) needed to add two numbers that were entered on a form and display the result.  It's painful... especially if the business guy already knows how to do the same thing with a spreadsheet.  It only gets worse if they need to deal with currency math and nicely formatted outputs.  Simple things are surprisingly hard to do.

A relatively easy answer to this problem is to guide business folks towards business-focused application development suites - Essentially use a tool to generate forms and spreadsheets and let the tool figure out how to transform the input into a Web-based application.  This is certainly a viable approach - but it's generally a proprietary approach.  Language-based approaches like JavaScript tend to be "open" approaches... the skills that you learn can be applied to products from a number of vendors.  Tool-based approaches tend to be "closed"... the skills that you learn to master one vendor's tool aren't easily transfered if you switch to another vendor's tools.

That's why I'd like to see an Open-Source-Business-Friendly dialect of JavaScript.  Despite the "no coding necessary" promise of many vendor's tools, you almost always have to write a bit of code somewhere... and I'd like to see that code written in a scripting language that is a lot easier for me to teach to business folks who are occasional programmers.

From my own experience, I can state that JavaScript becomes a lot "friendlier" to business folks when you augment it with the "right" function libraries... and there are some tremendously great JavaScript function libraries out there.  I also know that JavaScript becomes a lot "friendlier" to business folks if you don't mention some of its features - If you want to clear the business folks out of a room, just start discussing anonymous functions.  Many of the folks that I teach a bit of JavaScript get off to a pretty good start, but then they start googling for additional answers and get hopelessly lost.

Perhaps we need a brand-new language that is as explicitly focused on developing business applications as COBOL was... Probably something that generates JavaScript (so it will run in every browser) but something that isn't quite JavaScript to keep "occasional programmers" from hanging themselves with some feature or technique (found via Google) that they don't quite understand - let's call it Business Script.

I'm not smart enough to design Business Script - I see people struggle and I can make suggestions, but I don't really know the root causes of confusion.  Professional Programmers are pretty happy with the status quo...  so my bet is that the successful design of Business Script will have a lot more to do with the Psychology of Programming than with Software Engineering.  BusinessScript needs to be more, and less, than JavaScript - but it's not going to be easy to figure out the right mix.

Easy or Hard? - Business Programming and Software Engineering

Business Programming ought to be easier.

There's a perpetual argument in the programming community about whether or not programming has to be hard.  I seem to write a blog entry about this topic every year or so, and I'm a bit overdue... so here goes another stab at it...

Programming is the act of transforming requirements into something that a computer can execute... thus the first step to successful programming is to get your requirements right.  For the sake of this posting, let's call the person responsible for dictating the requirement "the client"...

There are various software development methodologies... but generally requirements are transformed into a functional specification - a very precise written description of what the software should actually do.  As far as "the programmer" is concerned, the functional spec is the bible - if the client doesn't put it in the spec, then it won't get implemented.

Once the functional spec has been agreed to by both the client and the programmer, the programmer usually takes over and starts coding.  From this point on, the client often has no clue what's going on "under the covers" - all they see is the results when they test to see if the software matches the functional spec.

What does this have to do with programming being hard?  Stay with me, I'm getting there...

Sometimes it's really, really hard to implement a requirement.  A simple requirement like "get the customer's credit score" might require a tremendous amount of infrastructure plumbing to actually pull off.  You probably need web service connectors, some sort of federated trust with the credit agency, etc.  You have to deal with service outages, latency due to transaction load at the service provider, etc.  It can make the programmer's hair turn gray.

Building software infrastructure and tuning solutions to provide optimal performance requires Software Engineering skills.  These aren't the sorts of skills that you pick up in your spare time.

When programmers are presented with hard problems they solve them.  That's what programmers do.

Unfortunately, programmers often solve problems in a manner that obscures the functional requirement that caused the problem in the first place.  The business requirement (get the customer's credit score) often gets lost in a forest of code.  The trail of bread crumbs that leads you from  the functional requirement to the code that implements the requirement ends up very hard to follow.  There are twists and turns and ups and downs and before you know it you're getting dizzy.

Some requirements are going to be hard to implement, so some aspects of programming are always going to be hard - but it's this absence of a clear map from requirement to implementation that ends up making most programming harder than in has to be.

If the "requirement to implementation map" was maintained by the programming environment environment itself, then most programming would be easier.

 I've seen the proof of this in my own work with BPM solutions -  In a BPM development suite the process requirements are documented by drawing a process diagram (usually using BPMN symbols).  The diagram is the functional spec for the process.

The task of transforming the functional process requirements to "code" is automated.  The mapping between the process requirement and the implementation is obvious.  What you see is what you get.

With a BPM suite, programming is easy - at least as far as implementing the process requirements goes.  When you get into the non-process related aspects it can still get messy... but I hope you see my point.  Those BPMN "maps" keep things really clear... and "all you have to learn" is how to make those maps.

Software engineering skills will still come into play in most programming projects - there will almost always be problematic aspects that require thoughtful engineering solutions - but to make things easier we have to constrain those solutions to fit within the boundaries of the "requirement to implementation" map.




That's the key to making "business" programming easier... Build more tools that help programmers and their clients "make their maps" and build more software engines that "follow those maps".  This isn't a new idea - software modeling tools have been around for decades, but they've often lost their way and focused on modeling the software engineering aspects rather than the business requirement aspects of the problem.

Focus your development tools on the business functionality of the problem and programming the business functionallity will become easier.  We've done that with the process aspects, so now all we have to do is deal with the data aspects and the business rules - Piece of cake :-)

Open Source + User Builders = Great Products

I read this story over at npr.org this morning:

"Before Jonathan Kuniholm had a tour of duty in Iraq, he worked for Tackle Design, an industrial design, research and development firm. After that tour, he was missing part of his right arm — which he lost when his Marine patrol was ambushed near Haditha.

When Kuniholm returned to his design shop, he brought along three prosthetic arms given to him at Walter Reed Medical Center — the same body-operated hook many veterans have used since World War I, a shorter utility prosthetic, and a new, state-of-the-art myoelectric arm. Each one had its drawbacks — and when Kuniholm and his Tackle Design colleagues disassembled them, they quickly concluded that they could improve on the designs. They founded the Open Prosthetics Project, an open-source collaboration that makes its innovations available to anyone. And Kuniholm signed on with Revolutionizing Prosthetics, an initiative of the Defense Advanced Research Projects Agency, or DARPA."

There's an important lesson here for every company that builds products:

1. Users are a better judge of what your product should do than your developers.
   
2. Users who have the skills to build your product themselves will improve your product if given the chance.
   
3. If you don't let your User/Builders improve your product, your product won't be as good as it should be.

Get the Words (and Notation) Right

Dan North (the chief evangelist for Behavior Driven Development) loves to say:

"Get the Words Right"

In the context of creating software if you don't know what somebody wants you to build - then you can't build it for them... That statement is true for lots of contexts but let's stick to software for the moment.

If the person who wants something built uses terms that you don't fully understand, then you'll have to translate those terms into something that you do fully understand. Translation is never perfect, so you're bound to screw up something. When you are ready to explain what you're building to the client (to confirm that you are in fact building the right thing) if you use terms that the client doesn't fully understand, then they'll have to translate your words - and they are likely to misunderstand what you're building.

Sometimes a 3rd party will be on hand who understands both the client's and the builder's terms (I do that a lot)... but then you are dependent on the translation skills of the 3rd party. The good news is that both you and the client can blame the 3rd party, but your software still stinks.

Dan's solution to that problem is to spend more time up front agreeing on a common vocabulary between builder and client... with a definite prejudice towards adopting the client's vocabulary whenever possible.

Seems pretty much like common sense, but it's very hard to actually do it.

Many programmers just don't really want to understand their client's businesses... they're excited about programming, not business. Business concerns are mundane and boring compared to the challenges that they face when creating software... in their opinion. Many business people are just as bored by technical details as programmers are by business details.

It's hard - but in truth it's the easiest way to succeed. When everyone is on the same page building projects are actually fun... and the "buildings" are really fun to "live in".

In the Business Process software domain, we have a common notation for expressing the "flow" aspects of processes - BPMN. BPMN is relatively simple to learn and it has proved to be a very effective means for Business and Programmer to communicate about the "flow" aspects of their business processes. Unfortunately, BPMN isn't good at all when communicating all of the other aspects of a business process (data requirements, routing rules, etc.) but it's a good start.

BPMN is Notation rather than words... but Dan's point is hugely relevant - you have to get the Notation right.

I've worked with BPMN for a few years, and I think the Notation has elements that just aren't right. They don't convey the meaning of the symbol to most Business people.

Here's my least favorite symbol - the Multiple Instance Activity:

When you see this symbol, you are supposed to know that many copies (instances) of this Activity will be performed at the same time. You "know" this because there are those two little parallel lines on the Activity - They're parallel to each other, so of course that means there are parallel Activities taking place.

I am not a Business person, but I doubt that many would describe Activities-That-Happen-At-The-Same-Time as parallel Activities. If someone out there knows of a study that proves me wrong, please tell me.

I think a better symbol would be something like the following:

With this symbol everyone can see at a glance that Multiple-Things-Are-Happening (hopefully at once). I'd also add some indicator of the number of "Things"... although often that's only determined when the process is running.

I suspect that the current Multi-Instance symbol was chosen because it is easy to draw by hand. Fair enough... but I find that most of my hand-drawing is done on white boards, and I almost always draw something that looks a lot more like my proposed symbol.

Fortunately, with the current state of BPMN editors it is usually pretty easy to swap out the "official" notation's symbols for your own... There are much bigger problems to solve in the realm of conveying Business Requirements to Programmers than this little icon - so please don't waste any time lobbying for it to change. I'm just using it to illustrate the point:

Words Matter - Notations Matter - Diagrams Matter

Pictures are worth a thousand words, so take the time to get the picture right before you start building your next project.

Programming Language Sociology

Bruce Eckel shares his reminiscences about the dawn of C++ and Java in his blog entry: Why? Language Archaeology.

This is great stuff... Insight into the "why" C++ and Java are the way they are (from Bruce's perspective). My take-aways from this essay are the following:

• "Things" often make no sense because the requirements that dictated those "things" are no longer requirements
• People skills can be just as important as Technical skills

I really appreciate Bruce admitting that his opinions about Java are colored by his dealings with Gosling... Social aspects impact Technical aspects. How you feel about someone impacts your evaluation of their work. That is a hugely important thing for us to remember.

I was an early adopter of C++ and loved it... but I loved Java more. Was that due to Java's technical superiority?

Nope.

I loved Java more because it coincided with the Web... and the Web allowed me to collaborate with other Java developers in a way that I had never been able to collaborate with other C++ developers. The social aspect of this collaboration is why Java trumps C++ on my "fondness scale"... Java had (and still has) a much stronger sense of community than C++.

The stereotype of the programmer nerd with poor social skills had a lot of basis in fact... but in my experience programmers really crave social interaction within a society where they feel comfortable.

My programming focus is now in the realm of Business Process Management, so the society in which I find myself is much broader than that of my C++ days. In my C++ days I wrote stuff for business people. Now I write stuff with business people. Huge difference.

Successful Business Programming requires a society where both Business and Technical people feel comfortable. That's tough to accomplish because the passions of those two camps rarely coincide...

Next time you're in a conference with both Technical and Business people, sit back and watch:

When the Business folks get into an animated discussion the Technical guys will be checking their Blackberries. When the Technical folks get into an animated discussion, the Business folks will start looking at their watches.

The challenge in business programming today is not about programming language features - it's about the social aspects of application development. We've got to solve the problem of engaging everybody in the development process. We need to fully engage the Business folks to make sure that we build the right things, and we need to fully engage the Technical folks to make sure that the things are built right.

Compared to the real challenges of business programming, agonizing over whether or not to include the "new" keyword in Java is seems positively "quaint". Bruce's recollections are a fond trip down memory lane for me... but lets not get stuck in the past lest our descendants look back on our concerns as "quaint" too.

Good Engineering != Good Design

This thought-provoking statement grabbed my attention...

"Most tech products—heck, most products in general—aren’t as good as they can be because they’re put together by the people with the technical knowledge required to build them."
Jason Snell, Macworld.com

Jason makes some really good points in this article... Go read it ;-)

Artists (and programmers) hate negative feedback

If you really want to piss off an artist, just tell them how their painting/sculpture/etc. could be improved.

The best possible reaction that you'll get is a politely disingenuous "Thank you"... More likely you'll get some form of "Mind your own business" delivered with various levels of hostility based on the strength of the artist's self-esteem.

The creation of art is intensely personal... Art is an extension of the artist - created primarily to satisfy the artist's need to create. Art is presented to the public - not shared with the public. The artwork may be sold - but it always belongs to the artist.

What does this have to do with Programming?

For many programmers - Programming is their Art... and that's the root of many of the conflicts that you will have with programmers.

There's an anecdote about Frank Lloyd Wright: The story is that he attended a dinner party in a house that he had designed, and before leaving he had rearranged the furniture back to his original plan. Wright couldn't let go... The house was held hostage to his vision rather than released and allowed to become a home. Wright liked to preach that "Form follows function" - but in this case he totally forgot the true function of a home.

When you have to provide negative feedback to a programmer - focus on conveying the big picture: What do you really need? Instead of focussing on how the current software does not meet your needs, challenge the programmer's inner artist to create something that will fulfill your needs... You'll probably be much happier with the results.

Applications as Process Activites

I return to this topic a lot... BPM solutions automate the transitions between Activities in a Process, but they don't necessarily automate the Activities themselves. Some Activities can be automated, but others require Human interaction or Human judgment to complete. If we are convinced that BPM is a good idea, and we want BPM to be as pervasive as spreadsheets, then we need to make it easier to implement the Human-centric Activities that our Processes need.

Many BPM suites support building Human-centric Activities within the suite itself, but sometimes you'd rather re-purpose an existing application to do your bidding.

Let me walk you through an example from my own wonderful life as a Travelling Guy...

When I get a new assignment there are a lot of things that I have to do. I've got a mental checklist that I follow, but I'd really rather have a Managed Process to keep me from screwing up.

Most of my gigs are somewhere else, and that makes it very important for me to make travel and lodging arrangements. I almost always need a flight and a hotel, and based on the location I might need a rental car.

To perform this Activity I use my company's preferred Travel site - it happens to be Orbitz, but it could just as easily be Expedia, Travelocity or any number of sites. Each of these sites supports the concept of a "Trip" and each lets me make flight, hotel, and car reservations. These travel sites also provide feedback - they'll send me email when a reservation is confirmed or "something" changes.

As I am going through my process of getting ready for a gig, I log on to Orbitz and make the necessary reservations. When I am "done" I will "check off" that task from my list of things to do.
This happens a lot in BPM solutions. An Activity (a task) is assigned to an individual Participant. The Participant gets directions on how to perform the Activity (what it is that they need to do) but they have to use some "external" application in order to actually perform the task (in this example, the Travel site). When they are "done" with the task they "check off" the task and the process continues.

Processes that are implemented like this can be really error prone. The Process Manager has to rely on the Participant's assertion that the task has been performed correctly. You just have to trust the Participant to do the right thing.

If you had the time and if you had the money, then you could build all of the tools that the Participant may need to use in order to complete their tasks. Quite frankly, that would be a huge waste of both your time and your money.

The better approach for all concerned is to encourage the makers of those "external" tools to become Process Aware. Get then to modify their existing applications to be aware of Process Flow.

Let's go back to my "Get Ready for a Gig" Process... Some time after the Process starts I will need to perform the "Make Reservations" Activity. The Process Instance knows the Location of the Assignment, and it knows the Dates of the Assignment (the Process was started by a message that contained this information).

When I run the "Make Reservations" Activity from my Task List I'd like Orbitz to open with the Dates and Location of my Trip pre-populated. After I make my reservations, I would like the site to send (structured) confirmation information directly to the Process Manager.

Based on the information that Orbitz returns, the Process Manager would either move on to the next task, or require me to try again. For example - if Orbitz says that my reserved flight is on the wrong day the Process Manager should tell me.

If you are a Programmer, then I think you can easily envision how you would make the Webapp that implements the Orbitz Travel Site "Process Aware". The trick is in recognizing that there is a beginning and an end to planning for each Trip. Modify the Webapp to support "Dates" and "Location" on the invoking URL - along with a "Return Address". When the Webapp determines that planning for the Trip is complete, use the "Return Address" to send structured information (most likely XML) back the the Process Manager.

For Web Services (automated Activities) we've done this for several years. Service providers (like Orbitz) publish WSDLs that define the Services that we can invoke. It's time to do the same for Activities that cannot be Automated. It's time to standardize interactions with what I've called Human Powered Web Services - Web Applications that can serve as Process Activities.

In my ideal world, all Travel Sites would support "the same" interface for starting a new "Trip" and for returning results. That might be a bit much to hope for... but it's a good target to shoot for. Even if every site had a unique interface, we could build adapters for our Processes - a much better situation than today (from a Process Manager's perspective).

If you are a Developer who builds sites where folks go to "do things", then I hope I've inspired you to think a bit about those "things" as part of a larger Process. It's not a huge leap to build into your site the hooks that a Process Guy like me needs to incorporate your site into my Managed Process. I'll be grateful, and I'll bet the users of your site will be too.

Clarifying your ideas

I love the Webcomic xkcd by Randall Munroe... Geek humor for sure, but often very insightful...

"You'll never find a programming language that frees you from the need to clarify your ideas"

Amen to that.

Programming is the act of transforming requirements into something that a computer can execute... but before those requirements can be transformed they'd better be clear. You might think that this would be obvious, but you'd be wrong... In way too many cases the "idea" that spawned the software either wasn't clarified or it wasn't communicated clearly.

Programming languages - both textual and graphical - won't substitute for an idea that hasn't been clarified. But "better" programming languages can help to insure that the idea has been implemented properly (or at least make it clear what was actually built).

All Big Brothers Aren't Bad

Joel Trammell beat me to the punch with his posting: You can't manage an economy you don't measure:

"this problem, in part, was caused by lack of oversight"

I'll try to keep my outrage in check because I'm not an economist or an accountant or a lawyer. I don't really know what happened any more than anyone else (probably less than many people) but Joel's point resonates with me: If somebody had been watching and willing, then these problems could have been detected and dealt with earlier. In retrospect, it's obvious that excessive risks were taken for short term gain, and some folks were just plain crooks.

It's ironic that folks in the Financial Services industry were blind-sided by this crisis. For years they have been at the forefront of analysis and pattern recognition technology, driving many of the advances in the field. Software that could have detected Madoff's ponzi schemes is trivial compared to the sophisticated financial gaming models developed by the Quants. There's really no excuse except for a lack of someone who was willing to look out for everyone else.

At times like these we realize that sometimes a Big Brother can be a wonderful thing to have.

I'll bet that most of you think of George Orwell's Big Brother when you hear the term. Orwell's Big Brother scares the hell out of us - a omnipresent leader spying on everyone - but he's just a one dimensional character from a horror story.

We should also think of the Big Brothers Big Sisters organization... proving the point in real terms that having someone to look out for us can be a very good thing indeed.

But what about privacy?

My personal belief is that real privacy is an illusion in the 21st century. Maybe not for those living on the margins of society, but for someone like me the walls are transparent and thin. If you are curious and persistent you can probably dig up every embarrassing and questionable act that I've ever done. You can figure out where I live, where I shop, where I eat, where I travel. You can dig up former girlfriends, past employers and folks who can't stand me. My life may not be an open book - but opening my book is way too simple.

I'm not all that happy with my lack of privacy, but I am more concerned by the thought that someone can use that publicly available information to pretend to be me (identity theft). As I wrote in my 2004 blog Privately Famous:

"I know that many people fear a National Identity Card, and I can sympathize; Governments are notorious for turning oppressive. I hold the other view; I want a National Identity Card, and I want my DNA and finger-prints on file. Without a trusted identification authority (and I know that nothing is foolproof), it's just too easy for someone else to pretend to be me, and too hard for me to prove that I am me."

The price of Privacy can be way too high if it's carried to ridiculous extremes. Our fear of government prying led us to condone the gutting of regulatory oversight that we just naturally assumed was protecting us.

Forget the financial mess for the moment: Did you ever think that American Peanut Butter would be dangerous in the 21st century? Didn't Teddy Roosevelt institute food safety oversight a hundred years ago?


Regulatory oversight is certainly a potential evil - but it's a necessary evil. With the tools that we have today Big Brother can watch us, but we can also watch him. We have no real privacy, but neither does he.

Technology can help. We can help reduce the "red tape". We can help spot the suspicious. We can help predict the long term. We can help the watchers watch, and we can help watch the watchers.

We can help avoid messes like this in the future.

The Science of Art

If you've looked at my LinkedIn profile you've probably noticed that I've changed jobs quite a bit over the years. There have been many reason for changing jobs... but usually I've moved on to grow. I've moved on because I found something that interested me more, or something that challenged me more. On a few lucky occasions I've found things that were both more interesting and more challenging.

On other occasions I have moved on because I just couldn't stand it any more. Frustration over some aspect of my working days would build up until it was obvious (to me) that I was wasting my time and my talents. Something about the company was perversely keeping me from doing my job in the way that I knew it really should be done (or so I felt at the time).

I think that all of us who work for a living have felt something like this from time to time, and that's why I found Douglas Bowman's post on his departure from Google to be so poinient. Bowman's artistic design philosophy crashed head first into Google's statistical mantra. Classic case of irreconcilable differences... Make a clean break and part as friends.

Changing jobs is a personal matter... but Bowman raises an issue of wider interest in the community of UI designers and programmers:

"Is UI Design Science or Is It Art?"

The answer is of course both... and everybody has always known that. What's changed is our radically increased ability to apply Science to our designs. Analysis of our designs used to be woefully limited to focus groups or beta testers, but as Stephan Shankland points out: Google's practice of "Choosing color shades and pixel widths on the basis of the behavior of millions of Web page users is a fascinating development to the form-follows-function school of design."

With SaaS (Software as a Service) solutions it's much simpler to roll out a minor design change and "see what happens". It's much easier to employ the Scientific method. It's much easier to test instead of speculate.

Take this new "try it and see" capability with a grain of salt - you'd need Google's millions of users to even notice the effect of subtle changes - but the promise is intriguing.

Bowman's case warns us that there's also a down side if Science is improperly applied. Demanding statistical justifications for artistic design decisions is fraught with peril... You'll drive off your best artists and you'll likely lose out on true innovation.

Inspiration seldom comes from statistical analysis... and inspiration is usually at the heart of all good designs. Statistical analysis can measure the success of a design, and it should be more widely applied. Statistical analysis helps us continually improve a design, but it's not the genesis of good design.

Art first - then Science.

Is that a bad thing?

Joe Trammell recently blogged an interesting anecdote that concludes with the following moral:

"The first step in trying to improve any process in IT is convincing the people with the power to make decisions that the current process is flawed."

I'd remove the words "in IT" from Joe's statement.  In any organization, unless "they" agree that "it" is broken, "it" is not going to get fixed. Joe goes on to say:

"enterprise computing isn't just technical knowledge, it's also social interaction"

Amen!

In my experience, most software development obstacles are cultural.  People are different. One person's crisis is another person's acceptable annoyance - I'm not going to help you fix something unless I agree with you that it's broken.

There's no quick fix for this problem, but I think that we're headed in the right direction - BPMN, Business  Rules, Business Natural Languages -all attempts to foster better communication between Programmers and Business.  All attempts to help each camp understand what the heck the other camp is talking about.  All attempts to help each other understand what the problems really are.

We're not there yet. Communication is still a bit awkward and stifled - but we're getting there.

== != =

This week I visited a client in the Detroit area, and on the whiteboard in their meeting room I saw the following:

== != =

For those of you who aren't programmers, this is a slam at perhaps one of the worst programming language syntax decisions that has ever been made.  You can read the above expression as "Equal equal is not equal to equals".   Still lost?  

In the C programming language (and derivatives such as C++, Java, JavaScript and C#) the way that you test to see if two variables are equivalent to each other is to use the "==" operator.  For example:

if ( a == b ) { doSomething(); }

If you want to assign a value to a a variable you use the assignment operator "=".  For example:

a = b;

Believe it or not, even folks who have been programming for years occasionally screw up and use the assignment operator "=" when they meant to use the test for equivalence operator ==.   Here's an all too common example:

if ( a = b ) { doSomething;}

The line above sets the value of variable "a" to the value of variable "b".  If "b" is not zero, then the function "doSomething()" will be invoked.  Perhaps that's the behavior that the programmer desired, but I really doubt it.

According to the wikipedia article comparing the Pascal programming language to C: 

"It is common for programmers new to C to accidentally put assignment expressions in conditional statements such as if (a = 10) { ... }, which will always execute because the assignment expression a = 10 has the value 10 ≠ 0 and is therefore considered "true" in C. Also note that a has now got the value 10, which may affect the following code. This kind of mistake cannot happen in Pascal, as assignments are not expressions; using the wrong operator will cause a compile error (and it's not likely that anyone would mistake the := symbol for equality test)." 

I have to take issue with this statement... It's not just programmers who are new to C.  I made this mistake last week :-)

This mistake is so common that some programming editors will actually flag statements like this to warn the programmer.  I've no idea how much time has been lost on this little syntactical monkey wrench, but I think it's probably quite a bit.

Long ago before I learned C I used Pascal.  In Pascal the "=" operator is used to test for equality.  If you want to set a value you used ":=".  When reading this symbol out loud I'd always say "set equal to".  For some reason, I don't ever remember confusing the two or knowing anyone else who ever confused the two.

The BASIC programming language is even simpler... the interpreter uses the context of where "=" appears in an expression to determine what it means.  IF $A = 0 THEN is interpreted as a comparison.  $A = 3 is interpreted as an assignment.

I don't know why the authors of C chose to use "==" and "=" in this way - but unfortunately that decision was inherited by most of the programming languages that are widely used today.  Every  programmer who includes some JavaScript on a web page might screw up and make this mistake... so just remember:

== != =

Programming or Software Engineering?

There's an interesting thread going on over at InfoQ regarding the relationship between BPM and Software Engineering.

One subtext of this discussion raises a wider question - When is it "Programming" and when is it "Software Engineering"?

I'm not sure that the distinction really matters... it's kind of like trying to nail down a definition for "Scripting Languages" rather than "Real" programming languages... but I'd like to share my thoughts.

I think that it's Software Engineering when your primary focus is on the relationships and interactions between software components. Programming is when your primary focus is on implementing specific requirements.

I know that's a pretty mushy distinction, but it's the best I can come up with for now...

To build on this distinction a bit more: It is my very strong belief that everyone should be taught how to Program. Reading, Writing, and Arithmetic just aren't enough for the 21st Century - Everybody should be able to write simple programs.

In contrast, Software Engineering is a very specialized field and way beyond the needs of most people. Like Electrical, Mechanical, Civil and the other Engineering disciplines, it takes years of preparation to master. In my experience all Engineering boils down to math and metrics (How else can you measure the effectiveness of an algorithm?).

Bringing ths back to the BPM/Software Engineering discussion going on at InfoQ... I agree that BPM is not Software Engineering (it's more akin to Process Engineering). There's a huge amount of Software Engineering that goes into building a BPM suite, but that doesn't generally concern those who are actually using those suites.

When implementing a Managed Business Process you must employ good Programming skills - but in most cases you are building within the dictates of your BPM suite's Software Architecture. To me that's not really Software Engineering, it's Programming.


What do you think?

cess Dia

Sometimes (usually about 2 o'clock in the morning when I am keyed up and can't sleep) I have amazing epiphanies that (if properly shared) may potentially alter the future course of All Human Endeavors from this point in time until The End Of All TIme... Here's my latest:

If you take the phrase "Process Diagram" and remove from it "cess Dia" you end up with the word "Program".

What an amazing correlation between Process Diagrams and Programs!

Okay, so maybe that's not such an amazing observation, but I'm going to pursue it a bit further because it fits very nicely with a pet theory of mine...

A Flowchart is worth a thousand Words.

Think back... in my case think way back...  When you were first introduced to Programming didn't somebody use a Flow Chart to illustrate what a Program is?  For some of you, especially those of you who are relatively young, perhaps not... but for old-timers I'll bet there's a scribbled Flow Chart somewhere back there in your past.

Why did we abandon Flow Charts and come to rely only on Words?

If you've used any of the Graphically Oriented Programming Tools out there (like Lego Mindstorm's Robolab) then I'm sure that you know using Icons to program can become a bit tedious.  It's just easier to use words because you can type really fast.  I'll give you that... but it's also a lot easier to make mistakes with words, and it's often really hard to discern the structure of your program from that mass of words.

Apparently I'm not the only one who thinks so... I you Google "Code to Flowchart" you'll find an amazing number of tools that will parse your Code and produce (you guessed it) Flow Charts.

So here's the point of all this rambling...  I think that it's pretty useful to create a flowchart from code, but I think it's even more useful to create code from a flowchart.

From this point forward I'm going to transition from the generic "Flowchart" to the more specialized "Process Diagram"...

Process Diagrams are a great way to visualize the structure of (a certain class of) programs, and they're a much more effective programming tool than words.  A few symbols replace thousands of words, and those few symbols express the intent of the program far better than words ever could.

User Manual Driven Development

One of the best jobs that I ever had was back in the mid-80's working for Scott Cutler at Tandy Electronics Research and Development.  I worked on the team that built the DeskMate II GUI shell and its associated applications.  This was back in the glory days when many vendors were still pushing their own graphical environments - Apple clearly in the lead - Windows not yet quite measuring up - Atari and Digital Research's GEM still viable contenders.

We wrote everything in C and 8086 assembler... Life before objects: Smalltalk was around but too much of a resource hog  for the Tandy 1000  (256K of RAM and a 5 1/4" floppy).  C++ was just a rumor as far as we were concerned... Structured programming was king.

Our development methodology was simple but effective... the marketing group wrote the deskmate user manual and we built the applications to match the manual.  To be more precise, the marketing group would write a proposed manual, we'd negotiate with them over what could and what couldn't be done, and then we'd build to the manual.

I guess you could call this User manual Driven Development, and I think that it still can be a very valid approach.  In most respects User Manual Driven Development is very close in spirit to Test Driven Development - The function of the software is clearly defined, so it's very clear if the software doesn't do what it is supposed to.

Some time between the 80's and now we mostly abandoned the idea of User Manuals.  Let's face it... few users ever bothered to read the manual (Leading to the ever present RTFM reply on many support forums).  Few companies even bother much with online help... Users are mostly encouraged to just play with the software and learn by doing.

I'm not advocating a return to User Manuals... but I do advocate tailoring our development methodologies to return the programmer's focus back to the users.  The prime objective is always to satisfy the needs of the users... The users really don't care about object hierarchies, they don't care about Java vs. Ruby.  They care about being able to do what they need to do.

I think most of you will agree that some of the best software that you can find is software that is primarily used by programmers.  Eclipse, NetBeans, Visual Studio...  This comes as no surprise:  Programmers understand the needs of other programmers.  Since I understand the purpose of the software, and I know how I would like it to work, then I am extremely qualified and motivated to develop the application.

It always stuns me when I encounter an engineering manager who believes that his staff doesn't need to know how to use the product that they are developing.  I've heard people say "Just give us the requirements an we'll build it".  I simply cannot agree with this philosophy... Requirements are never enough.  Ever.

To write great software, you have to put yourself in the shoes of the user and walk a few miles.  Perhaps it's a bit like method acting... Unless you "become" the user you're just going through the motions.

Another look at Object Oriented versus Process Driven Design

My friend Bob was troubled when I blogged "Object Only Programming is Silly". Bob knows from experience that life as a programmer after OO is dramatically better than life was before OO... so my criticisms just don't sit well with him. Obviously I must have gone off the deep end :-)

Bob's not one to let a friend languish in ignorance, so he recently appealed to my literary interests by enlisting Ayn Rand to bring me back into the OOO (Object Oriented Only) fold:

"Ms. Rand was obviously a fan of OOP, and would have likely loved Java even more than C++… not just from the name she gave her philosophy, but from its precepts and what they lead to… The Axiom of Existence leads to the Law of Identity… The Law of Causality (things act in accordance with their natures) is recognized as “the law of identity applied to action”.

“Rand rejected the popular notion that the causal link relates action to action. According to Rand, an ‘action’ is not an entity, rather it is entities that _act_, and every action is the action of an entity.”

"Therefore, you _must_ have an Object before an Action can be performed."

Thanks Bob, but with respect to Ms. Rand...

You don't care who performs the actions, you care that the actions get done.

Programming is the craft of transforming requirements into something that a computer can execute... so programmers need to focus on the requirements.

The Primary Requirement is always that the action be performed correctly and on time by a qualified entity (or entities)... so your design should be focussed on satisfying the action.

(Actions are service. Entities are service providers... nothing more)

The entity (or entities) that will perform each action is secondary, and should not be your primary focus. It falls out of the design rather than drives the design. Said another way... figure out the actions first and then you'll be able to figure out who (what enties) are required to perform them.

In Process Driven Design there is a direct linkage (via the process model) between the requirement for an action and the code that satisfies that requirement. The sequence and flow requirement are all right there in the model.

Contract this with your typical Object Model in UML... it captures the Actors...maybe... but you are left to figure out what the application actually does, unless of course you create an Activity Diagram (which isn't really OO any more.. it's behaviour).

Most programming students are now being trained to (maybe) write a use case and then start identifying objects... Then they'll create Sequence Diagrams showing the order in which objects send messages back and forth to each other.

So you've got a bunch of entities that are passing messages back and forth in some sequence... you still don't know (from looking at the model) what it is that this bunch of entities is trying to accomplish.

Contrast this with a BPMN diagram... by comparison you know exactly what the process is supposed to accomplish.

Process and Data lead to Objects. Not the other way around.

I don't disagree with Bob's insight from Ms. Rand: "You _must_ have an Object before an Action can be performed"... but I will add the precursor: You don't need an Object unless you have an Action that needs to be performed.

Is Programming Hard?

Is Programming hard... or are we making programming hard?

That thought came to mind while I was reading Johan Den Haan's well stated opinions on Reasons Why Model-Driven Approaches (will) Fail over at Info Q. He's not saying that MDA won't work, just that it's not the be-all and end-all to solve all of the headaches of programming.

I mostly agree with Johan's opinions about MDA... in my lexicon MDA is synonymous with UML (which Martin Fowler's christened as the Unwanted Modelling Language) and modelling objects just hasn't been the central problem on any of the programming projects that I've ever worked on.

My problems have always related to Requirements.

Programming is the craft of transforming requirements into something that a computer can execute.
Let me say that again:
Programming is the craft of transforming requirements into something that a computer can execute.

Over the years there's been exponential growth in the things "that a computer can execute". Back when I started programming OCR (recognizing text) was still a major technical hurdle... now you find it in hand-held devices.

What hasn't progressed as quickly is the "other" part of my programming equation: Transforming Requirements.

It's often extremely difficult to follow the trail from a Customer's Requirement to the Programmer's Code that implements that requirement.

MDA is an attempt to solve that problem... the idea being that a Model is a more recognizable expression of the Requirements than several hundred lines of code. The "map" from the Requirements to the Model ought to be pretty simple. My problem with MDA stems from a disagreement over what to model... In my opinion a Process Models is a clearer representation of Requirements than an Object Model is. Regardless, I think Models are a Good Thing.

Improving the mapping from Requirements to Code can certainly make programming easier, but the fundamental problem remains: Requirements.

If you don't have the Right Requirements (or they are ambiguous, vague, or self-contradictory) then programmng is probably going to be a nightmare.

This is where the "we" in my original question "are we making it hard?" comes from. John Wood wrote a great blog entry from a typical programmer's view of what makes programming hard... but I think he missed something important....

We seem to regularly block our customers (those who dictate Requirements) attempts to understand what the heck we are doing. We don't want them checking our work until we are done. Said another way, we often block cutomers attempts to validate the requirements that they gave us.

We do this in subtle and perhaps unintentional ways (and in not-so-subtle and blatantly in-your-face ways)... sometimes hinting that "what we do" is more important than "what they do", and mumbling that "they wouldn't it understand anyway".

So how does blocking customer inspection of our work make programming hard? It all goes back to those darn Requirements. I think that many of our problems are caused by focussing (obsessing) on the wrong things. We get sucked in by those nifty techno-glitches that pop up all over the place... and sometimes ignore the very features that the customers really need.

We all know that what customers really need isn't the same as what they want... but that's another blog.

I think the way to programming nirvana is to aggressively pursue techniques that more closely align Requirements to Implementations. If we have a better idea how (and if) we've implemented what they want, then I think we'll stay focused on the right things.

I don't think this is impossible... in fact I think we're already moving in the right directions:

1. Process Modelling and Execution Environments (BPM)
2. Test Driven Development (align the Tests with the Requirements)
3. Aspect Oriented Programming (there are different types of requirements, and aspects can help express that)
4. Domain Specific Languages (if the SMEs can read it, it's a good thing)

We need to stop focussing on tools that are purely developer focussed, and start focussing on tools that foster communication between the developers and the SMEs (Subject Matter Experts). We need to integrate tools that let the SMEs implement what they can with the tools that let prgrammers do what they must (let's face it, we know how to do things that they don't need to know how to do).

Yeah... Programming is hard... But it doesn't have to be as hard as it is.

Most Software Development Obstacles are ______

Fill in the blank in the following statement:

Most Software Development Obstacles are ______
(A) Technical
(B) Cultural

If you answered (A), then I am intensely jealous :-)

When I became a programmer, my answer definitely would have been (A) and I probably wouldn't have had a clue why (B) was even on the list. I doubt that I had any idea what "Cultural Obstacle" even meant in the context of Software Development.

I was drawn to programming because of the technical challenges (I love solving puzzles and building things). My University classes (except for electives) were all concerned with Science and Engineering. Now I realize that I should have taken some basic classes on Psychology.

Let me give you one example:

The implementation methodology that I practice and the one that I encourage others to practice can be expressed pretty simply:

Build "something that works" and start using it.
Identify the next most iimportant thing to extend or enhance and then build and use that thing.
Continue these cycles until everything is "good enough".

I now call this "iterative development"... but I use to call it "the shaft of light theory". Others might call it Agile.

I arrived at this approach pretty early in my career... as I think many of us do. I found myself working on overly ambitious projects that just seemed to drag on and on with no end in sight. Nobody knew what worked and what didn't. Everything on the project plan was 80 percent complete.

As deadlines approached we'd start dumping feature right and left until we got down to the stuff we could actually finish by the due date.

On top of that we'd often "finish" only to find that the output of our labors wasn't what the end user really wanted. Perhaps we had misunderstood or perhaps they had changed their minds. Regardless, nobody was happy despite our hard work.

I am a huge fan of "play backs"... Demonstrate the application to the users as you develop it to insure that what you are building is what they need. Often the requirements will change as the users become fully aware of what (you think) they asked for... and that's okay with me.
Admittedly, this approach won't work for everything... but for the jobs that I do (where there's a lot of human interactions) it works most of the time.

Most of the Quality Assurance people that I know hate my approach. How can they assure the Quality of the product if the requirements are in flux?

Look at QAs objections from a process perspective: one measure of a good process is the amount of necessary rework - less rework is good, more rework is bad.

From a QA perspective iterative development is bad because it requires them to do a lot of rework - testing "the same" functionality over and over. Prior to a production deployment everything must be tested, and in many cases even regression testing is foiled by new requirements.

From my perspective this is a small price to pay... It's more important to get "something" to the user as early as possible, and we're much more likely to build what they really need.
I have to admit that QA has a point though... Without a comprehensive plan from day one... How can you reliably estimate when you'll be done?

I've focused on QA and Development - but you know the same applies to Security, DB and Network Admins, etc. Each have their own jobs to do and their own ideas on how to do their jobs well.

There are no right answers here... It's a clash of cultures rather than a question of technical merit.

"Measure twice and cut once" versus "Shave off a little and see if it fits".

Both approaches "work"... But they don't work together.

What's crucial is for each organization to develop a culture that works for them. If you hire an Agile Development Manager then you'd better hire an Agile QA manager. Some might argue that it's good to have divergent viewpoints within an organization, but my experience has taught me that everyone is much happier (and everything works smoother) if the department heads agree on the fundamentals. I don't agree with executives who believe that underlings who fight with each other produce better results.

So what do you do when you find yourself working with others who disagree with your approach? What do you do when you disagree with their approach? My guess is that you've already dealt with these questions many times... sometimes with positive outcomes and sometimes with negative outcomes.

Hopefully your positive outcomes are on an upward trend... If not it may be a good time to reflect on "life, the universe and everything" and adopt a more effective approach for dealing with others.

They may be morons. They may be idiots. They may even be slackers and lazy. But I doubt it.
They probably just belong to a different culture than yours (they value things differently than you).

If you want to be an effective programmer, you are going to have to find a way of dealing with them - or your efforts will be wasted...

Welcome to the wonderfuil world of programming ;-)