Business Rules to the Rescue in the 21st Century!

Bonnie O’Neil, Westridge Consulting/Apex Solutions

The Problem:  The M Word

What is the biggest problem facing Information Systems Professionals today? The M-Word: Maintenance.  Nobody likes to admit they do maintenance, and in fact everyone hates it.   Maintenance is the largest cost of a software project: 60-80% of any software project is spent on maintenance (Marca, 1984).  In addition, more than 60% of Fortune 500 Companies spend more money and resources on maintenance than on new applications.  Why is maintenance such a problem?

The main reason is complexity.  Information systems are extremely complex.  Even so-called “modular” systems are still rampant with complexity.  When something changes in the business, it seems almost impossible to figure out where the code is that enforces the business policy; and even after you find it, it seems that side effects occur, and other stuff breaks when you make the needed change.

“Islands of Information”

In the past, as information systems usage grew within an organization, often applications were designed when a specific need arose.  The result was "islands of information", redundant data stored in different computers, used for different purposes, which cannot be linked together.  To complicate matters, the same fact may be represented in several different formats, for example a part number may be two characters and three digits in the Sales application, six characters in the Invoice system, and four characters and three digits in the Inventory system.  How do you track a piece of merchandise throughout the sales cycle electronically?  Not without great difficulty!  What if the CEO wants a report concerning how many widgets were sold in 1993?  This might be very difficult to achieve if each sales office represented widgets differently.

Legacy Applications

The term “Legacy Application” is very pejorative; it refers to an information system stuck in the past, which stores information as it was, like a snapshot in time, years ago.  Everyone knows that a legacy application is a burden that we are stuck with.  Since legacy systems are so hard to change, or take so long to change, they are often left alone.  The problem is, the business tries to store current information in an old, brittle system enforcing antiquated processes using incorrect data structures.

What about new applications?  Are we just creating more “legacy applications”?  Unfortunately, the answer is “yes”!  The moment you stop gathering requirements, the system goes legacy. Currently the requirements of a system are handed off to a group of developers at time A and at some time B they come back with a system to support the requirements. In the time between A and B, the business will inevitably have changed. In some environments a lag of 6 months is acceptable, in others a lag of 6 months the window of opportunity will be closed and bottled shut!

This means that the new systems we are creating today will be unresponsive to the business tomorrow.  How can we avoid this?  It seems that we are in a vicious cycle.  Wouldn’t it be nice to build applications in a new way, with non-stop requirements gathering?

Change Doesn’t Change: It’s Here to Stay

It has been said that the only certainties are death and taxes. One more item that really should be added to the list is ‘change’. Every day, change is being brought about more and more rapidly. Businesses must adapt to changing market forces and changing competition in order to survive. Businesses that fail to adapt quickly enough will wither and die.

More and more, a business’ ability to change is hampered by its information systems. Currently, producing software that supports a business is very hard to do. Those businesses that are able to quickly change the information systems that support them will have an evolutionary advantage over those businesses that cannot.  Software that is flexible and capable of enabling a business appears to be unattainable with the tools available today.


The Newest Challenge: The WEB

The Web has turbo-charged application development: Web-based applications are “life in the fast lane”.  Dot-coms have incredible pressure to prove themselves viable in an extremely short amount of time.  Conventional companies also feel incredible pressure to “web-ify” themselves, to provide e-commerce applications and customer service, and be the first in their market sector to do it.  And change also occurs faster and faster, especially in these new market sectors.  The applications must respond faster and faster to rapidly changing business requirements.  How can we ever keep up?

Given this intense environment of change, it is amazing that we in the application development field haven't figured out how to manage it better.  We may be able to produce applications faster than before, but we cannot change them quickly and respond to the rapidly shifting needs of the business.  Applications are not produced so fast that they can simply be thrown away and a new one built every time the business changes.  Just as the business that does not respond to change will be obsolete, so will our applications.

Focusing on the Wrong Problem

Current technology forces us to be thinking about the wrong problem entirely when building Information Systems to support a business. When automating a problem we are usually forced to be thinking about the technology and not the business.  This bias is lurking behind the scenes:  If you look at many popular techniques, you will see this bias.  Take for example, “Use Cases”.  Use cases can be helpful, but they are technology-driven; they focus on the interaction between the user and the system.  Too much emphasis on the system; what about the business?

Wouldn’t it be nice to give maintenance to the users? After all, it’s their system!

Reuse

The Object Oriented technologist mantra in recent years has been reuse.  The goal of rapid development and easy maintenance later on is reuse.  Reusable objects, reusable code whenever possible.  But reusability is very difficult to achieve.  Why? “I can’t reuse your object because mine is different; it’s not the same!”  Perhaps we have focused on something more abstract. Perhaps we should focus on something more concrete, like the underlying policies and guidelines that run a business.

Business Rules to the Rescue

Business Rules have always been with us. We’ve called them different names. We’ve used different methods to capture them. We’ve used different technologies to automate them. And, we’ve spent billions trying to maintain (or preserve!) those technologies. It is only just recently that Business Rules are getting the attention they deserve.

A formal definition of a Business Rule is “a statement [that] constrains the business, [that is,] is intended to control or influence the behavior of the business.”  Examples of Business Rules from a hypothetical University might include:

 “A Professor may teach no more than 15 hours per week unless otherwise authorized.”

“A Professor must hold the minimum degree required in a specific subject to teach a specific course.”

As you can see, Business Rules are stated in the language of the business, and contain specialized business terms.  Because the sentences are using English (“natural language”), they can often appear to be ambiguous.  The process of Business Rules Analysis is therefore to peel away the layers of ambiguity -- much like peeling an onion -- by gathering statements from business people in their own language and iteratively identifying and defining the specialized business terms or jargon. This process should continue until the analyst has captured and defined the lexicon of the business.  Interestingly, this process uncovers a complete definition of the business itself. This ‘Business Definition’ can then be analyzed and understood by the business people themselves, allowing them to communicate better and to operate more efficiently and effectively.

Business Rules vs. Models

As information systems professionals, every time we start to produce a system that automates some aspect of a business we normally perform some form of analysis. This analysis usually feels much like the method described above: questions are iteratively asked of the user community until the actions the system is to perform have been fully described. However traditional methods of analysis differ from Business Rule Analysis is one important respect: the result of Business Rule Analysis is a definition of the business; the result of traditional analysis methods is a model of the business. As we shall see, a definition is much more useful than a model.

Other problems exist with traditional analytical methods:

·       The analysis is usually tightly bound to the immediate automation effort and doesn’t go far enough

·       The analysis results aren’t fed back to the business people in a form they can use and understand

·       The intelligence gleaned from the analysis gets disconnected from the analysis and buried so deep in the system, that when the business  changes we cannot easily change the system

In short, the real fruits of any analysis -- the elucidated Business Rules -- are not normally recognized for what they are and the intelligence they contain.  They are usually discarded, only to be re-discovered on the next automation effort.

Declarative Business Rules

Programmers admit the importance of business rules, but they usually use the label “business logic”.  They think in terms of procedural code. But business people think in terms of declarative statements.  The new move toward business rules is toward declarative, natural-language like sentences to describe business constraints.

Current products have varying support for business rules; but there is a dearth of  products on the market which enable a separate construct of business rule to be stored as an English language statement which can be linked to other business rules as well as modeling constructs.  Business rules are important in their own right; users may want to analyze them in conjunction with other elements of the business.  Most of the time business rules are buried in CASE tools as different things: definitions, domains, relationships, etc. Wouldn’t it be nice to be able to view all the business rules about “customers” as English language statements?  Because business rules are not viewed as a common construct, it is impossible to get such a list in most tools.

Business Rule Semantics

Proper capturing and documentation of business rules in a web-based environment is essential, because of its decentralized nature.  Business people can easily relate to rules, and they can validate them quickly.  The trick is to state them in an unambiguous manner.  This is easier said than done, because the English language is very ambiguous.  Business rules should be stated clearly and distinctly, and validated with test cases (see Barbara von Halle's columns appearing in DM Review for some good guidelines on how to write and validate business rules).

A Business Rule Grammar

We need a way of overcoming the ambiguity in our language.  One possible approach is to express business rules in a formalized grammar.  This would allow for more rigorous expression of the rules, and easier translation into code.

A possible format for a definition rule might be:

 “A <business term> is defined as <definition statement>”

with a “meta-rule” imposed limiting terms used in the <definition statement> as known terms (already defined).   Grammatical elements can then be defined which express qualities of rules, such as mandatory/optional, cardinality, etc.

An example of the rule format above might be:

 “A Premier Customer is defined as a Customer who has a High Credit Rating.”

Customer, Credit Rating, and High Credit Rating are all business terms, which must also be defined for this rule to be valid.

One of the best grammatical approaches to modeling can be found in Designer 2000’s relationship sentences.  If used correctly, every relationship between two entities will generate two complete business rules, utilizing a formal grammatical structure.  For example,

Each <entity1> may be|must be <verb phrase> one and only one|one or more <entity2>

When this is applied, the relationship between “Customer” and “Product” looks like this:

            “Each Customer may be purchasing one or more Products.”

            “Each Product may be purchased by one or more Customers.”


Developing a Business Rules Approach

Capturing business rules in the analysis phase helps get to the important issues.  It helps users validate assumptions.  But we also must be concerned with how to enforce the business rules. 

It is not enough to capture business rules in an unambiguous manner and get them validated so we can use them in the system.  What happens next? 

One development project used business rules as a checklist for development.  As the developers would produce the applications, they would check off the rules.  But then the list of rules went obsolete once the system was delivered.  What incentive was there to maintaining a list of business rules?  No one really thought it would be helpful for maintenance, because it wasn’t really “hooked” to anything.  Business rules didn’t drive the application, the code did.  Therefore the business rules became just another infrastructure baggage that would be “nice” to maintain but was left to decay.  It became another maintenance burden!  Horrors!  This was certainly not the intention!

Business Rule Enforcement

Another project developed a simple classification for business rules that helped developers and DBAs determine where the business rules should be enforced: client, server, or both.  A record was kept of code modules in both client and server, which enforced the rules.  This technique is used in several products currently on the market, such as Designer 2000 and Versata.  The technique used in this project was manual, designed for impact analysis and to assist in targeting the modules to be edited if any given business rule changed.  These commercial products mentioned have gone a step further and automate the process.  If you specify that a rule should be stored in the server, trigger code will be generated by the tool; if client, it will go into the form.

This approach is problematic because it is essentially two-tiered: Rule enforcement is limited to either the client or the server, or sometimes both.  A “virtual” three tiered approach is desired:  Rule enforcement should not be tightly coupled to either client or server.  Many times, rules should be enforced on the client for performance reasons and for ease of use (to prevent the multiple commit “user hostile” application mentioned earlier), but should also be available in the server for batch applications or direct data access.  The rules should therefore be stored in a middle tier, between client and server, and should be able to “float” to whatever platform is appropriate, even a physical middleware server or TP-monitor like configuration.

Business Rules Controlling Applications

The goal is to store and maintain business rules centrally.  This means that when the rule changes, all you have to do is change it in the repository and all places where it is enforced will automatically be updated. 

Hidden Business Rules

Separation of business rules from the other “tiers” of an application can be hard to do.  For instance, domain checking is almost always done in the form, or front end.  Domains are business rules—they should not be put in a form, because all forms that reference that field will have to be changed when the business rule changes.  For instance, “Valid project codes are O for Open, C for Closed, X for Canceled, and P for Pending”.  If you put these codes in the front end, you’re in trouble if 5 forms reference project code.  You will have to change all manually.

What about putting them in the server as a check constraint?  This is actually worse, because you have to get a DBA to change it, which means modifying the table.  Yuuk!  Not a good idea.   Avoid check constraints like the plague!

What about putting them in Designer 2000 as a domain, or in a lookup table, and referenced by a pull-down list in the form?  This is better, because it can be controlled centrally from Designer 2000.  If it changes, you can propagate the change automatically to all forms that use it.  However, you still have the problem common to CASE tools if you have tweaked the underlying code generated by the tool (and who hasn’t?).

The major flaw with Designer 2000 enforced business rules is they really don’t allow users to modify the rules directly.  If you want users to be able to change project codes, you have to create a mini-application with a form for them to do so.  And they can’t change it in their own language—they have to change it in a data way, with a form.

This is not that big of a deal for domains, but it is when you get into more complex business rules.


Business Rules Can Be Useful to the Business and Programmers

Useful to the Business: Nuggets of Policy

As the business tracks its business rules in a formalized process, it can learn about itself, and it has a centralized mechanism of information about how the business is run.  Remember policy and procedure manuals? Ever notice that they have gone out of style? Remember how updates would come out periodically, with instructions like “replace page 3 with this new page 3, and put this page 20A between page 20 and 21”? Remember how the first few times you followed the instructions diligently, but after a while you gave up because you became too busy? But what happens when you have to refer to it for a very important business procedure, and you didn’t replace the all-important page? 

Business rules can be an automated policy and procedure manual, and because they are easy for business people to understand and are stored in small nuggets of discrete information, they can easily be modified by the business people who created them in the first place (with the proper security, of course; consult the Guru, Marlene Theriault).

Business Rules are essentially metadata.  Metadata is “data about data”.  

Validation Information

Metadata is often associated with data warehouses, because they must deal with the integration of disparate data sources.  But metadata is not just for data warehouses.  A business rules repository can also capture and link business rules with other facets of the business, such as validation information:  Who authorized and/or validated the rule, when it was last updated, etc.  The repository tracks all elements important to the application environment, such as entities, relations, and attributes.  So it would be very easy to get a report of all the business rules involving several specific entities and/or attributes.  How powerful this repository can be!

Useful to Programmers: Control and Standardize Error Handling

A business rule design structure coupled with metadata can also be used to facilitate centralized and customized error handling.  You can use metadata tables to store standard error numbers and messages for use throughout your application.  You can then associate an error number and text with a specific business rule.  This provides consistent error handling across applications, and is a nice feature to offer in a Client/Server environment.

Survey of What’s Happened in the Last Ten Years

How has the business rule approach permeated the programming landscape? Not much, and lots: Both at once.  There are a few business rules-oriented products on the market today, which is more than there were ten years ago.  They are by no means “mature”.  Here is a quick summary of the types of tools available:

1.       Tools that enable capture and classification of business rules, essentially a Business Rules Repository. An example of this type of tool is BRS RuleTrack.

2.       Tools that are “engine-oriented” and require that you write the application itself.  They provide a sophisticated rules engine with conflict resolution and other features.  These tools require a very highly educated programming staff to use.  They are getting more and more user-friendly, but for the most part they require a lot of work.  Examples of vendors offering this type of tool include Blaze Software and RuleMachines Corporation.

3.       Tools that enable you to create an application with less effort, using a visual development environment that uses rules as the centerpiece of the business logic.  An example of this kind of tool is offered by Versata.

The most interesting work in Business rules, however, is the “homegrown” stuff, the fact that the approach itself is favored by many application developers doing the work “from scratch”.  Many of my personal friends in the industry have jumped onto the business rules bandwagon, and are using the approach to design and build their own custom applications from the building blocks they have available, such as UML, Oracle Designer 2000, etc.  There is the emergence of a grass-roots movement here.  It is sad that the vendors are so far behind.  I firmly believe that this approach is the only hope we have in keeping up with application backlog and the growing need to satisfy constantly changing business requirements.  When will we see more vendors implementing the business rules paradigm?

Some other interesting developments:

·       IBM has a project employing business rules for e-commerce (BREC) and they have released a prototype of CommonRules called AlphaWorks, based on something called Business Rules Markup Language (BRML).  I do not know the current status of this; the stuff on the Web that I accessed was dated 1999.

·       Some of the tools offer a specialized grammar where rules can be expressed in pseudo-English and modified by non-programmers.  This is definitely going in the right direction.  Some of my friends think the grammar problem is too hard and they are not trying to implement this.  That is OK; if a rules-based approach is implemented using code, the code can always be replaced later if a better grammar comes along.

·       The consulting group within Oracle is doing some very nice work in business rules; catch their presentations at the ODTUG conference.

Summary

Due to the pressures of an ever-changing environment, we need to rethink the way that we design and build database applications.  We need to be more responsive to change, and be able to modify applications quickly and easily when these changes occur.

Business rules can be used to succinctly capture information about the enterprise and ensure their consistent enforcement in the brave new world of the Web.  Applications should be designed with a new infrastructure, maximizing user involvement directly with business rules and facilitating maintenance by decreasing the role of the programmer.

To be truly useful to a business, Information Systems should be based on the following principles:

·       Information Systems should be created from a business perspective, by business people, and not be biased by technologists.

·       The Business Rules should be collected in one location, and be maintainable easily by business people.

·       As the business changes, the business rules are changed by business people, and are reflected instantly in the business’s Information Systems, with no coding.

This paper has presented the business rules approach as an emerging paradigm. This approach indeed represents a new way to build applications.  It typically requires more rigorous analysis work up front, but will allow users a great deal of control over their applications, and a lot less headache later for programmers.  In the future, it will free up programmers to do the more fun work, like designing the look, feel and flow of applications and the discovery of business rules, without all the pain of locating and fixing distributed code (and somebody else’s code at that!).

Acknowledgement

Thanks to Kieron Hardy, who helped with the first edition of this paper when it appeared in several conference proceedings in 1997.

References

Bloesch, A.C. and Halpin, T.A. “ConQuer: A Conceptual Query Language”.  1996: E/R Conference, Germany.

Blaze Software, a division of Brokat Technologies website: http://brokat.com/en/products/advisor/builder.html

Dangov, John.  "Incorporating Business Rules Using Sybase in a Client/Server Environment".  International Sybase User Conference, May, 1993.

IBM. Business Rules for Electronic Commerce: Project at IBM T.J. Watson Research. www.research.ibm.com/rules/home.htm

Marca, David.  Applying Software Engineering Principles.  1984: Little Brown.

Moriarty, Terry.  "Architect for Change", Database Programming and Design, April 1992.

Kolber, Allen, et al. Business Rule Project Report, GUIDE International, 1995.

O'Neil, Bonnie K.  "Designing for Oracle7's Server-Based Integrity".  International Oracle User Week 1993.

O'Neil, Bonnie K.  "Rules About Ingres Rules", Ingres World '92.

O'Neil, Bonnie K. and von Halle, Barbara.  "A Common Vision", Database Programming and Design, February, 1993, pp. 11-13.

Ortner, Erich and Schienmann,  Bruno.  “Normative Language Approach, A Framework for Understanding”.  1996: E/R Conference, Germany.

Ross, Ronald G. The Business Rules Book.  1995.

Rule Machines website: http://rulemachines.com

Sandifer, Alice and von Halle, Barbara.  "Business Rules:  Capturing the Most Elusive Information Asset", "Business Rules:  A Case Study", Handbook of Data Management. Auerbach Publishing, 1993, pp. 453-486.

Sandifer, Alice and von Halle, Barbara.  "Designing by the Rules", Database Programming and Design, January, 1991, pp. 11-13.

Seybold, Patricia.  “Start Your Business Rules Engine”, Computerworld, 12/9/96.

Vaskevitch, David.  Client/Server Strategies.  1993: IDG Books.

Versata website: www.versata.com

Von Halle, Barbara. 5-Part Series on Business Rules, DM Review, 2001.