2004 ODTUG Business Rule Tool Shootout Report

Bonnie O’Neil

David C. Hay

Roland Berg

 

The 2004 Oracle Development Tool Users Group (ODTUG) Business Rule Tool Shootout evaluated the state of tools supporting the movement of the IT industry toward a business rule-driven approach to developing systems.

 

Four vendors submitted their tools for evaluation:

  1. Dulcian, Inc.
  2. Fair-Isaac
  3. ILOG
  4. LogicalApps

 

The vendors were provided with a list of questions to answer and were given approximately 3 hours each to demonstrate their products and answer the questions.  Realistically, it is impossible for any of the tool vendors to provide much more than a cursory overview of their products in this timeframe, but it is enough to get a feel for where they are going and how they fit into the market.

 

The Evaluators

The team of evaluators selected to review this year’s entries bring a variety of perspectives on business rules to the table.  

 

Ms. Bonnie O’Neil is a consultant and author who is one of the early adopters of the idea of business rules and rule-driven systems.  Her focus and specific interest is in the area of linguistics and the use of natural language or near natural language to collect rules.

 

Mr. David Hay is consultant and an internationally recognized author and data modeler.  He is best known for his book “Data Model Patterns.”  His specific area of interest relative to business rules is the structure of rules and the development of a data architecture for recording them.

 

Mr. Roland Berg is a consultant who has developed rule-driven systems using both commercial and “home-grown” rule engines.  His current interest is exploring the relationship between rule-driven approaches to software development and enterprise architecture.

 

Evaluation Criteria

Areas evaluated included:

 

Two of the most important questions that the vendors were asked were: “What problem is your product addressing?” and “Who is your user?”  These questions were the result of lessons learned during last years’ evaluation where it was difficult to compare tools because they each seemed to have a different focus.  By asking these questions up front the evaluators attempted to look at each tool and evaluate how well the tool addressed its vendor’s own stated purpose and target user community.

 

Product Classification

This year the participating products fell into two broad categories: 

  1. Rule Management Tools
  2. Rule-Based Development Tools.  

 

Generally speaking, rule management tools focus on the capture and organization of rules.  The rules are maintained in a centralized repository.  These tools tend to emphasize the process of collecting the rules and utilize complex grammars and/or rule templates to manage the collection of rules. The rules in the repository are made accessible to application systems either through embedded APIs, web services or some similar mechanism.  Application systems make use of these rule engines to determine what rules to enforce.   In this years’ evaluation this tool group was represented by Fair-Isaac’s Blaze Advisor and ILOG’s JRules tools. 

 

Rule-based development tools focus more on the creation of the final product, a working software system.  They take advantage of the underlying concepts of rule engines and leverage rule repositories to speed the development process through either generation of application code based on the rules or interpretation of the rule repository at the time of execution.    These tools take a more declarative approach to the specification of rules relying on diagramming tools and property sheets to specify many of the rules required to specify a system.  Dulcian’s Business Rule Information Manager (BRIM®) and LogicalApps’ AppsCreate® products represented this class of tool in the evaluation.

 

Both groups rely on the same basic architectures for deploying rules and applications preferring a multi-tiered approach where a middle-tier or backend server provides access to the rules contained in the repository.

 

The Evaluations

The following evaluation write-ups were compiled from notes made by the three person evaluation team after viewing each demo and reviewing the answers to the questions posed to each vendor.

 

A. Rule Management Tools

The first category consisted of the Rule Management tools group.

1. Fair-Isaac’s Blaze Advisor

Blaze Advisor is the senior statesman of the tools evaluated.  The management tools demonstrate a very well evolved and polished user interface.  Blaze recognizes the need to support a variety of rule types and scenarios.  This ability to support a variety of rule types ensures that the user will have the right type of rules available to solve a particular problem.  It has the ability to specify rules as IF/THEN statements, decision trees, decision tables, scoring algorithms and the like.  Power users specify the rules using structured English.  Power users can create templates that managers can use to specify the particulars of rules.

 

The rules are generated into XML files that are then read into applications.  This product does not create screens or database access.  It can communicate via XML, web services, or other techniques.  Applications can use the API to get at rules, and rules can use API, web services or XML to retrieve information from other systems to use for rules.

 

The tool uses a flow chart for rule set flows.  Each box consists of a process that can contain one or more rules. Triangles represented branches. You can pass the cursor over a triangle to get the name of a test or over a line to get name of a branch.  You click on the triangle to get the nature of the test.

 

Blaze Advisor features a debugger which allows you to see the sequence of execution of rules. You can also get a log that shows this in detail, along with a screen that lets you sort the rules by the resources used.

 

2. ILOG’s JRules

ILOG views its product as a Business Rules Management System (BRMS).  The objective is to provide comprehensive support for all participants in the BR-lifecycle including developers, policy managers and system administrators.

 

Key to ILOG’s product offering is a commitment to support enterprise-level integration by supporting a broad range of architectural approaches.

 

JRules is a collection of three components:  a development tool, a rule repository and a rule engine.  Developers use the development tool to create the rule configuration environment for the policy managers.  Policy managers use templates created by the developers to enter rules into the repository.  The engine is a class library is integrated into the system being developed.  It provides access to the information contained in the rule repository.

 

The organization of rules in the repository is determined by the business analysts and policy managers. 

 

JRules uses templates to cleverly map rules to object methods in the application.  Verbs in the templates map to methods in objects while nouns map to the method parameters. 

 

The iLog Builder is used to deploy, validate and test rules.  Rule flow shows the execution sequence of the rules.

 

JRules has a good rule debugging capability that allows you to step through the execution of the rules.

 

Since ILOG’s offering is tied to the Java environment, it requires skilled Java programmers to set the environment up initially.  Once the environment is set up business analysts and policy managers can manage the rules.  This approach relies on the developers to define the rule templates.  It is important to emphasize this point because it means that the developers are determining what types of rules are supported in your business.  This activity must be guided by the policy managers and business analysts in order to ensure that the correct rule types are implemented.

 

B. Application Development Tools

The second category consisted of the Rule-Based Development tools group.

3. Dulcian’s BRIM®

BRIM is the most comprehensive of the tools evaluated.  Its roots are as both a rule repository and database design tool.  It has extended its capabilities to support user interface, functional, and presentation rules as well.  When you have entered your rules and pressed the “big purple button” you get a database and rule repository that comes vary close to fully describing the system being developed.

 

Dulcian has chosen to avoid generating actual user interface layer application code and leaves integration of the user interface to the development team.  Development of a user interface layer for a BRIM project is very different than developing a traditional application.  Rather than coding program logic directly in your application, you are implementing a framework that exposes the rules as defined in the repository to the end-user.  This is a very different way of thinking for most developers and it is probably one of the biggest risks in adopting a rule-driven approach.  In fairness, this is true when you are integrating most of the rule engines into your applications.

 

Because it is a very comprehensive tool, BRIM can be a bit intimidating while you are becoming familiar with its capabilities and use.  Rules are specified in the context of either structural (class) diagrams or process (activity) diagrams that are extensions of familiar Unified Modeling Language (UML) diagrams.  Each object in a diagram whether it is a class, relationship, state or flow has a set of associated property sheets where rules are specified.   This approach should be familiar to most technical people who have used some form of integrated development environment.  The downside of this approach is that the BRIM view of what constitutes a business rule is somewhat different from that of most of the other vendors or what a functional person, the traditional owners of the business rules would expect.  This gives rise to the question: “Where are the business rules?”  The rules are there but finding them requires that you think in terms of the tool.

 

BRIM demonstrated the most significant changes over the last year.  The addition of two major pieces of functionality dramatically increased the power of this product.  First was the General Application Rule Processor (GARP) which provides a mechanism to define and process user interface rules via the repository. Second was the addition of a data mapping capability which allows you to define and generate external data interfaces from within the repository.  This is extremely valuable for data warehousing type projects or systems that need to interact with many external systems as data sources or data consumers.

 

BRIM is best suited to medium to large new projects.  Because the rule engine and data architecture are tightly coupled, use of BRIM as an external rules engine may not be appropriate.  Attempting to learn the tool on a project that is too small is likely to be too costly and will probably yield less than desirable results.  There is a point where the project doesn’t have enough mass to see benefit from the approach and results in getting an inappropriate view of the capabilities and value of the tool.

 

That said, due to the complexity of the tool, it is important that you retain an architect that has experience with the tool to guide you through your first project or two.  This will help ensure that you develop good business rule habits from the outset and increase your probability of success significantly.   Unfortunately, there are very few architects that have successfully implemented rule-driven approaches and there are even fewer that are familiar with BRIM.  

 

4. LogicalApps’ AppCreate

The people at LogicalApps are relative newcomers to the business rule arena and they entered the evaluation with an eye more toward learning how to improve their offering than to show off what they have done.  That said, they have the most application-centric approach of any of the tools and their focus is on delivering a working application.  They are in fact the only tool in the evaluation that actually produces working applications without further integration.

 

LogicalApps’ initial offering focuses on a declarative approach to application development that supports limited business rules directly.  The AppCreate tool focuses heavily on the end-user interface, an area largely ignored by most of the other tools.  It has a minimal, but functional, level of support for business rules internal to their application at this time.  That is not necessarily a bad thing.  There are appropriate hooks in place to allow you to augment their rule support with the capabilities of one of the other rule engine vendors while they work on expanding their capabilities.  This is actually a reasonable strategy as it allows them to target the weaknesses, and leverage the strengths, of the competition while they continue to make their offering more complete and robust.

 

The development environment is well thought out and is somewhat reminiscent of Oracle’s WebDB and HTML DB products.  The similarity stops when you delve deeper into the tool to begin specifying the business rules that drive the application.  The end result is a J2EE-compliant web application that is suitable for general use.

 

The team demonstrating the product was enthusiastic, knowledgeable, and very forthcoming with information about where they have been, where they were at the time of the evaluation and where they were trying to move their product. 

 

These guys are worth watching.  They have made remarkable progress in the small amount of time that they have been looking at this issue.

 

The Future:  Thoughts, Observations and Conclusions

When organizations choose to adopt a business rule-driven philosophy, they are undertaking a monumental task.  The task requires a very dramatic change in perspective at all levels of the organization. An incomplete transition at any level can cause the whole effort to fail.

 

Still there is tremendous value in anything that helps us get better requirements established up front and supports the ability to respond quickly to changes in, or refinement of, those requirements.  That alone may make the effort worthwhile.

 

Are the tools today ready for prime-time?  In general, yes.  The tools are maturing rapidly and when used properly will allow you to implement a rule-driven system.   They are a tremendous improvement over what was available just 2 or 3 years ago. But they all have shortcomings and, at least in the opinion of these evaluators, they are providing incomplete solutions when you look at the big picture. 

 

While each of the tools evaluated emphasizes the value of collecting and maintaining an organization’s business rules in a centralized repository, each of the tools evaluated is limited in some way in the types of rules that they can record.  If we evaluate the tools relative to the Zachman Framework, which emphasizes the documentation of all of the details of the who, what, when, where, why and how of the business, we find that none of the tools fully support all of the “rules” required to make an organization function.  They completely ignore those things that do not get automated.  The result is an incomplete architectural definition of the organization, and an incomplete set of rules.

 

These tools represent a big step forward in centralizing the collection and management of the rules of an organization, but it is in fact only a baby step.  If your goal is to achieve a level of understanding of your organization that will allow your business to respond intelligently to change you must strive to document the architecture of your organization completely.  A rule-driven approach will not do that for you.  It will only provide a portion of the information required.

 

If you are a mature organization with good development practices, a solid team and big budget then a rule-driven approach can significantly reduce your software life cycle costs by allowing you to change many aspects of your system without having to re-write code.  This allows change to happen more rapidly and with less destabilization of the product as a result of the change. However, do not expect this approach to solve problems rooted in lack of discipline, poor requirements and/or poor processes within your organization.  The business rules approach will only magnify these problems.  If your organization suffers from these problems, you should address them first and then move towards a rule-driven environment.

 

These tools are very expensive and most, if not all, will require additional expense in terms of training, consulting and an investment of time to transition your organization to a rule driven approach.  The cost of entry, which is somewhere in the mid-hundred thousands, may be too high for most small to medium-sized companies.  Cost not withstanding, the possible benefits of adopting the rules approach may still justify the cost of entry.        

 

There are three significant risks in undertaking a rule-driven approach to systems:

  1. First, you must have the organizational discipline to faithfully adhere to the approach.  If just one developer implements rule logic through application code rather than specifying it in the repository your entire effort has been undermined.  This is serious because you now have rules hidden in the code rather than exposed through the repository.  In essence you are back where you started. 
  2. Second, the ability to maintain the rules and perform impact analysis needs to be greatly improved.  Experience has shown us that while most of the tools provide some method of searching for a rule or analyzing impact, many of the rules are difficult to find and can be easily overlooked.  This is especially true of the tools that take a more declarative approach embedding many rules in property sheets and object-specific attributes which require you to know how and where a particular rule is recorded in the tool in order to find it. 
  3. Finally, many rules can not be implemented automatically by a code generator and unavoidably require implementation by a developer.  For these rules, there is some level of risk that the rule as implemented will be inconsistent with the rule as stated.  Errors of this type are very difficult to track down because most tools provide a mechanism for discovering rules but not for searching code.  In some cases these problems are actually easier to find in traditionally developed applications where you know the problem can be found in a particular “piece of code.”

 

In spite of these shortcomings, organizations that can successfully transition to rule-driven approaches will be on the road to having a better understanding of the automated portions of their businesses and reduced life-cycle costs within those systems and that is a good thing.   

 

We enjoyed having the opportunity to work with the vendors and review their tools.  We hope that they take our comments and observations and use them to make their products even more valuable to their customers.