2003 ODTUG Business Rule Tools Shootout Report

The Oracle Development Tools User Group (ODTUG) invited a number of vendors to participate in a Business Rule Tools Shootout, the results of which would be presented at the 2003 ODTUG Business Rules Symposium held on June 21, 2003 in Miami Beach, FL. The purpose of the shootout was to identify the capabilities of the various tools available in the marketplace and evaluate the results.

The shootout was conducted in two phases:

1. Each of the participating vendors was asked to complete a case study using their tool and prepare a report detailing how their tool addressed the evaluation criteria for the shootout.
2. Each vendor was given up to four hours to demonstrate their solution to the case study in a web demo for the evaluation team in order to showcase the capabilities of their tools, and answer questions.

The following report details the process, information gathered, and results of the Shootout. A presentation based on this information was given at the Symposium by Roland Berg and Bonnie O’Neil.

I. The Evaluation Team

Roland Berg, ThinkSpark

Bonnie O’Neill, Westridge Consulting

Gladys Lam, BR Solutions

David Hay, Essential Strategies, Inc.

Joseph Hudicka, Information Architecture Team

Mary Cauble, Southwest Texas State University

The evaluation team consisted of six industry experts each with experience in the development of rule-based systems, requirements analysis, rule specification, and enterprise architecture. Each of the experts brought with them a slightly different point of view based upon his or her individual experiences in identifying and documenting business rules and the development of rule-based systems.

Joining this group of experts on the evaluation team was a group of IT professionals from Southwest Texas State University who developed the case study. The addition of this group to the evaluation team added the valuable perspectives of technical people trying to solve a specific business problem who were new to the idea of rule-based systems. Their view of how well each of the tools was able to satisfy their business requirement helped ensure that the evaluation remained true to the central question: “How good are these tools at solving real-world business problems?"

II. The Case Study

The case study used for this shootout makes it unique compared to most industry evaluations. Many case studies used for this type of evaluation are oversimplified, contrived and really do not accurately reflect how the tools would be used to solve real-world problems.

The case study used for the shootout is an actual project underway at Southwest Texas State University. The University was in the process of defining business requirements and modeling solutions to the problem when they were asked to participate in the shootout by being the focus of the case study. Their participation in this case study brought several elements of realism to the evaluation of the tools. First, it helped to focus attention on common problems that exist in the real-world. Second, the information included in the case study is representative of the amount of information an organization would have about a problem when they were evaluating tools to address the problem.

The case study involves a project initiation and tracking system. The primary users of the system are customers of the Southwest Texas State University IT shop and the members of the IT shop themselves. When someone involved with the University wants to initiate a project, they enter it into the job costing system. The project request is then reviewed by a committee. If the review committee feels that the project is appropriate, the request is handed over to a technical staff member so that the level of effort required to complete the task can be estimated. This estimate includes both the manpower and physical assets necessary to complete the project. After the estimate is complete, the review committee determines whether or not the task should be undertaken. At this point, the request is closed or a project is created.

When a project is created, a project manager is assigned. The project manager partitions the tasks and assigns the appropriate resources. While the project is ongoing, staff members working on the project charge time to the tasks to which they are assigned. They also record any vacation or sick time on their timesheets. At any point in the process, team members or customers involved with the project or the work request, can record notes about the project or its tasks.

Information Provided to Vendors and Shootout Deliverables

Each of the vendors received an information package describing the case study. This package included a definition of the evaluation criteria, a system requirements document, an entity-relationship diagram, and detailed definitions of the entities and attributes contained in the diagram. The vendors were told to use the case study documentation provided as a guide. The data model provided was merely intended to communicate the business problem and their solution did not need to implement the model exactly as shown. This gave the vendors the flexibility necessary to implement the case study in a manner that would best highlight the capabilities of their tools. The vendors had between four and five weeks to complete the case studies and prepare the reports.

The following deliverables were requested as part of the shootout:

• A written report detailing how their product addressed each of the evaluation criteria.
• An implementation of the case study to the best of their ability given the capabilities of their products.
• A record of the time required to complete the case study
• Vendor demonstration over the web to demonstrate the products themselves as well as the solutions to the case study problem.

A portion of the information provided to the vendors is presented in the Appendix.

III. Product Classification and Review Criteria

The Business Rules Approach to system development involves several important premises:

• Systems using the classic, historical SDLC become “legacy” systems when requirements gathering stops; hence business rules are an attempt to extend requirements gathering in various ways to produce more flexible systems.
• Systems are generally written in a programming language not understandable to business people, so the business rules approach is an attempt to bridge the communication gap between business people and programmers in some way.
• Systems are typically written in a non-modular fashion. Even more modular systems do not do a good job of grouping “nuggets of information” that business people can manage well. The business rules approach attempts to group business rules in such a way that they can be managed by business people at some level.

The members of the evaluation team were asked to rate each of the products against each of the evaluation criteria, prepare a report commenting on each of the individual products, and document their overall impressions of the products relative to each other.

One of the most interesting points to come out of the demonstrations and evaluations is that the definition of what constitutes a business rule varies somewhat depending upon the perspective with which the problem was approached. This became evident in two different areas. First, the differing perspectives and approaches of the vendors make it appear that they are actually solving different problems. Second, the backgrounds and experiences of each of the evaluators had a similar influence.

The products can be loosely classified into three groups:

1. The first group addressed the problem from the business perspective with user interfaces focused on gathering business rules in as non-technical a way as possible. This group gathered rules using somewhat natural language and provided templates to guide the person defining the rules through the process.
2. The second group took a more technical, engineering-based approach using diagrammers and property sheets to specify business rules. This group required the greatest technical knowledge on the part of the person defining the rules.
3. The last group took a hybrid approach. The process of rule collection and definition attempted to be friendly to the business person while the implementation of the rules required more technical knowledge requiring some level of programming expertise.

Three of the products are external rule engines which interface with the system through the database, business, or user interface layers. One of the products integrates tightly with the Oracle database.

As a result of the differences in strategy and technical approaches of the four tools evaluated, it is difficult to conclude that any one product is better than the others. In fact, each of the products provides a viable solution for addressing the definition and implementation of business rules. Each of the tools has strengths and weaknesses that affect its appropriateness for use in a particular environment.

The selection of an appropriate tool must be based on the types of people using the tool, their skill levels, the overall objective of the project, and the architectural constraints placed on the solution. As a result, this discussion will not only focus on the capabilities of each of the tools, it will attempt to characterize the environments in which each of the tools will perform best based upon the opinions of the evaluators.

IV. Tool Evaluation

This report is organized by product. Each product section contains a short descriptive narrative and comments from the reviewers.

Most reviewers found it difficult to assess certain criteria given the short length of time allotted to complete the evaluations. For example, it is difficult to get a feel for performance or even thoroughness of documentation. Most, if not all, of the reviewers did not get a chance to look at any documentation, other than what was provided to us by the vendors, namely an overview of the shootout requirements and the answers to our case study. Therefore, ratings were based only on the materials submitted and may or may not reflect the tool’s overall documentation.

It was also difficult to rate performance, since there was no specific information about either how long the demonstrated solution took to build or how fast the application runs in a production environment. One of the criteria concerned industry standardization. Since there is no standard representation of rules, the comments were based upon how easy it would be for a non-technical (but semantically literate) business person to understand the rules.

Due to the fact that each tool is trying to solve a different problem, each tool needs to be judged with its individual goals in mind. Most of the reviewers seemed to favor tools in the first category (user interfaces focused on gathering business rules in a non-technical way).

1. Fair Isaac’s Blaze Advisor™

Blaze Advisor is a part of the Fair Isaac Business Science™ suite of Enterprise Decision Management tools. Blaze Advisor is a business rules management tool that provides the ability to use expert judgment to implement and deploy business rules to make use of policies, practices and procedures in operational business decisions.

A. General Overview

Of the four products evaluated for this shootout, Blaze Advisor is the most mature. It has been out in the marketplace for a long time (formerly Neuron Data), and it appears to be quite robust. It also appears to do quite well for impact analysis and conflict resolution. Complex rules seem to be supported by Advisor. Browsing capability for rules seems to be thorough in Blaze. Especially noteworthy are the decision trees and decision tables. Advisor supports a central rules engine storing rules in different formats.

One reviewer expressed concerns regarding Advisor’s complexity. It appears to be able to be set up only by Java programmers. It is not clear whether you can build an application in its entirety without traditional programming or whether you must build the front-end using Java. Apparently a Java-like language is used. Although Blaze Advisor is a powerful tool, power is often bought with a price—complexity. Ease of use is often sacrificed.

It is not known how well the tool holds up to managing a large-scale application. It would be nice to see a published benchmark dealing with number of rules, number of concurrent users and database size. It is also not known how Advisor would handle integration with legacy systems. For example, how does Advisor perform if you already have an application and you would like to implement rule management using a phased approach?

The purpose of having multiple reviewers was to get different views and perspectives. The comments help to explain the reviewers’ thought processes. In some cases, different interpretations of the requirements themselves may account for the difference in evaluation.

Comments

Not all of the reviewers expressed comments for each requirement. The chart below lists the comments provided:

Req#	Requirement	Rater 2 Comments	Rater 4 Comments	Rater Comments
1	Gathering & reporting of analysis rules	Intuitive, structured interface	Seems to be pretty good; a strong point	· Decision table and decision tree very helpful · Natural language also easy to use
2	Generation of the database	Rule engine does not require Oracle RDBMS	Generates a rule base	Complex
3	Generation of database triggers	Independent rules engine can be accessed by triggers.	With the caveat that I don’t think placing rules in triggers is a good practice, the tool offers a way to do it (although clumsy)	Does not automatically generates trigger
4	Generation of process application code		Strength of the product	No support of workflow process
5	How security is handled in the tool		Seems to be pretty good	More flexible
6	Configuration management	Ideal rule set navigation.	Seems to be pretty good	Rules organized in rule sets
7	Integration with database	Tight integration without dependency on database.	Good but complex	Supports more DBMS (jdbc); however, more work to integrate
8	Industry standard rule representation	Working closely with OMG	With the caveat that there is no standard rule representation at the moment, tool provides various ways to see a rule like decision trees	Natural language and decision tables
9	Rule re-use		Very good	Rules can be reused within an application
10	Performance		Seems to have nice features like event rules
11	Scalability for complex processes		Seems to be robust in this area
12	Learning curve	Natural language support in addition to technical notation.	Seems to be high	Natural language
13	Reporting from repository	Reporting includes conflict identification in addition to standard repository outputs.	Seems to be pretty good	Decision trees and tables, rule sets
14	Open API	Limited explanation of API	A strength of their tool
15	Level of effort required	No development savings projected, though maintenance is drastically improved.	Very high; appears to require a programmer to get it started
16	Scalability of number of users	Benchmarks not provided.	Don’t know	Limitation
17	Development required	Requirements are suited by typical developer pc	Complicated	Flexible
18	Deployment required	Highly diverse deployment platform support.	Complicated	Flexible
19	Ease in changing rules		Pretty good, but some “rules” are functions; In my opinion, functions are not rules	Rules in one place
20	Organization & visibility of rules	Ability to seek rules by context.	Seems to be pretty good, but complex
21	Impact Analysis		This is an area where they traditionally shine; nice navigation	Rule management capabilities
22	Rule interaction/conflict	Limited explanation	Conflict resolution is another Advisor strong point. Very good	Some detection at analysis time
23	Potential for integration with other systems		I’m not sure of all the details here, but because their infrastructure is so technical, I would presume integration is pretty good
24	Description of architectural approach	Diagrams require a bit more explanation.	Seems sound, but very technical	Rule-centric, rule integrated architecture
25	Impact of tool on SDLC	Requires modification of SDLC	Pretty good but technical; not totally convinced that a business person can really write functions	User front-end can support some business analysis activities
26	Documentation of tool	On-line doc is a plus.	Stuff they provided to us was pretty good
27	Different types of rules supported		Technical, AI-based rule types and functions. Business people would have a hard time identifying with them	More flexible

2. Oracle’s CDM RuleFrame

CDM RuleFrame is a framework for the structured implementation of business rules that supports the development of logical three-tier internet applications. CDM RuleFrame is a combination of concepts, process, software architecture and utilities that is supposed to enhance productivity and implement the business logic tier.

A. General Overview

This tool was developed in a “skunkworks” fashion by Oracle consultants out in the field who saw the potential of the business rules approach and wanted to shoehorn Designer in order to take advantage of it. It was pseudo-productized, but not marketed at all by Oracle and included as a free addition to Designer. RuleFrame enables developers to manage rules in the Designer repository as repository objects. The forward-looking Oracle consultants who developed RuleFrame saw the potential of the business rules approach for building flexible applications, and wanted to see Designer organized in this way. RuleFrame is a great step forward in accomplishing this.

RuleFrame by no means enables rule modification by business people. Code must be manually entered. There is no code generation functionality. If the ultimate goal is to provide a tool that can be used by business people to capture their rules themselves without programming, CDM RuleFrame does not meet this goal.

Designer itself is a very powerful product but difficult to begin to use (high learning curve). If a development shop is already using Designer and would like to extend it to begin to track business rules, RuleFrame is one way to accomplish this. However, this process requires writing the business rules as code, as well as considerable maintenance to ensure that the relationships are properly in place. For developers skilled in Designer, this is an alternative that leverages the existing investment in the tool to provide some rules management.

Comments

Req#	Req Name	Rater 4 Comments	Rater 5 Comments
1	Gathering & reporting of analysis rules	Good means of using Designer to use repository to store rules	Need to use Oracle classification scheme
2	Generation of the database	Very database centric	Oracle only
3	Generation of database triggers	With the caveat that I don’t think placing rules in triggers is a good practice, the tool offers a way to do it, much better than Blaze Advisor	Generates automatically
4	Generation of process application code	You must add the code yourself; the product provides templates only	Oracle Workflow
5	How security is handled in the tool	Uses Oracle DBMS security and that of Designer	Oracle
6	Configuration management	Same as that of Designer	Events are generated but some rules require manual functions
7	Integration with database	This is the main strength of this tool	Oracle only
8	Industry standard rule representation	With the caveat that there is no standard rule representation at the moment, rules are just documentation and don’t execute; the code that you write executes	System code
9	Rule re-use	Targets code re-use; not totally clear if this is accomplished the way they intended	Only data rules are reused
10	Performance	Code performs based on how you write it
11	Scalability for complex processes	Not really clear
12	Learning curve	Very high if not Designer literate; even Designer aficionados will need to learn the rule categories which may seem obtuse	DBMS and code
13	Reporting from repository	Very good; this is what the tool was designed for, repository management	Data constraint rules no rule management capabilities
14	Open API	As good as Designer’s
15	Level of effort required	Must write code	?
16	Scalability of number of users	Not sure	Unlimited
17	Development required	Must write code	Limited - Oracle only
18	Deployment required	After code is written, generate an application using Designer
19	Ease in changing rules	Must change the code itself	Code changes
20	Organization & visibility of rules	Good visibility within Designer
21	Impact Analysis	From what I can see, not very good; this is because the tool really wasn’t designed for this	Does not support rule management properties
22	Rule interaction/conflict	Does not provide a vehicle for this (at least that I could see)	Not supported
23	Potential for integration with other systems	As good as Designer
24	Description of architectural approach	Good shoe-horning of existing product, but does not solve the overall goal	Data centric architecture
25	Impact of tool on SDLC		Systems developer
26	Documentation of tool	Provided limited documentation answering our case study; not very descriptive	More general Oracle usage than RuleFrame usage
27	Different types of rules supported	Types of rules are based on technical, database-oriented categories and are difficult to grasp. RuleFrame does provide help to guide users into deciding which category each rule should be	Limited

3. Dulcian, Inc.’s Business Rules Information Manager (BRIM^®)

BRIM^® enables developers to generate a complete system with little hand coding. Business rules are articulated and placed into a repository. From the repository, the database and code are automatically generated. Generic applications provide immediate ability to use the system without any traditional application development. They also allow you to validate the quality of the business rules. Once these rules are validated, core applications can be built faster than in traditional systems.

A. General Overview

This tool is solving a different problem than any of the other tools evaluated. It deals with the business semantics very differently. Rules are represented in a UML model and the process model (which is an enhanced UML process model) but are not represented in a grammar format. The premise behind the tool is that such grammatical sentences will become unmanageable because there are too many in any system to be represented in natural language in a way that is useful.

This tool essentially generates an application completely, including the database structure. It is very innovative, and seems to defy characterization and categorization. It is very exciting in its potential. The problem it is solving is that of coping with rapid changes in the business rules in an organization. BRIM^® allows for total rule flexibility. This was very impressive! One reviewer said of all the tools, this one seemed to have the greatest future potential.

Classification of the rules is done by workflow. The relationship between business rules and workflow has been recognized in the industry, but coupling them completely as BRIM^® does is new. Most business rule analysts see rules as a separate component to the workflow. BRIM^® does not really attempt to deal with business semantics, apart from the workflow or process flow diagram. Analysis rules are essentially “documentation” (like use cases) and would have to be mapped manually to the implementation rules. It is not clear how this would be accomplished.

Comments

Req#	Req Name	Rater 2 Comments	Rater 4 Comments	Rater 5 Comments
1	Gathering & reporting of analysis rules	Assumes manual translation of analysis to technical rule syntax without consideration for language based translation. More suited to highly technical professionals.	Not really geared towards analysis rules. They are essentially documentation	Complex
2	Generation of the database	Based on Oracle and generates highly efficient back-end implementation.	Excellent	Seems to only support Oracle
3	Generation of database triggers	Achieves remarkable performance for trigger based implementation.	With the caveat that I don’t think placing rules in triggers is a good practice, BRIM generates database-oriented stuff including specialized triggers.	Supports some triggers
4	Generation of process application code		Excellent. This is what BRIM does best.	Some support of workflow process
5	How security is handled in the tool		Seems to be very comprehensive	Use Oracle structure
6	Configuration management		Seems to be comprehensive but technical; can be mismanaged	Rules constraint in data. Rules can be duplicated.
7	Integration with database		Excellent	Oracle only
8	Industry standard rule representation	Based loosely on PL/SQL, not a rule based syntax.	With the caveat that there is no standard rule representation at the moment, the rule semantics are difficult and embedded in the process flow diagram. Does use UML	System syntax
9	Rule re-use	Rules are atomic and easily shared.	Can be done but must be “coded” this way	Reuse at code level (not rule level)
10	Performance	Highly optimized for Oracle, not available for other databases.	Has a good story but needs to be objectively evaluated (we did not have time)
11	Scalability for complex processes	Has demonstrated ability to process thousands of rules.	Seems like it would be very scalable
12	Learning curve	Highly technical professionals will grasp the tool fairly quickly, but end users will struggle.	Seems to have a steep learning curve since the semantic is not pseudo English	Complex syntax
13	Reporting from repository		Not sure; didn’t really see any reports (left the demo early)	No rule management capabilities
14	Open API	Report did not expand on this capability.	Seems to have this
15	Level of effort required	While the effort may not be significant, navigation is challenging and therefore more apt to mistakes.	Seems to be fairly complex for a business person to understand how to specify a rule
16	Scalability of number of users		Probably good scalability, as you would get in traditional Oracle app
17	Development required	Heavy development requirements for client pc's	Difficult for a business person, not very intuitive paradigm	Limited
18	Deployment required	Easily incorporated into existing or newly implemented Oracle platforms.	Probably easy to deploy
19	Ease in changing rules	Navigation to find a specific rule is difficult.	Very flexible, can change rules easily	System syntax, rules maybe replicated
20	Organization & visibility of rules	see 22.	Not clear how the organization of these rules would go over to business people; I’m thinking not that great
21	Impact Analysis		Not convinced that this is a non-issue	Doesn’t support rule management
22	Rule interaction/conflict	Alerts for potential impact would be nice enhancement.	Not convinced that this is a non-issue	Detection at generation time
23	Potential for integration with other systems	Provided other systems are also Oracle-based.	I am thinking that this point refers to “can I construct a rule in the tool and have a previously existing app call that rule?” I don’t get the sense that this would be done well in BRIM.
24	Description of architectural approach		Good, sophisticated architecture; thorough coverage of it	Data-centric architecture
25	Impact of tool on SDLC	Scope Creep is surfaced but no recommendation for determining how detailed rules should get.	Seems to support our goal of continuous requirements gathering	System developer tool
26	Documentation of tool		Documentation for the Case Study was very thorough
27	Different types of rules supported		Classifications based on position in the workflow and not on business categories	Limited

4. ESI’s Logist

Logist is a decision-making system that enables organizations to capture human knowledge and thought processes in the form of computerized rules. Logist allows organizations to build applications that process knowledge and expose it to the users of the organization, to customers, vendors, and partners, via a GUI or WUI front-end.

A. General Overview

This tool seemed to be the favorite of the reviewers. Logist has a great user interface, and presents the easiest method for changing a rule of the tools reviewed. It uses a lexicon and is very linguistically oriented, expressing rules in structured English and providing a template to edit rules so that the user need not be a linguist to use it effectively. The wizard guides the user in selecting lexical elements with which to construct the rule. It has wonderful navigation and impact analysis facilities, enabling the user to find the rule in several different ways through rule families (rule group and rule class) and a tree structure. You can also see all of the rules using a specific building block. Alerts are very nice, providing management by exception. The alerts are built-in, and criteria for yellow and red alerts are set by the rules. For example, a rule may specify a yellow alert if a certain numeric threshold is reached. When the application is running and a yellow alert is reached, the user can drill down into why the alert was shown and the content of the rule.

Rules are built using grammar templates, and the grammar is extensible. Definitions of terms are provided by the user, and these terms can then be reused as components in other rules. If the rule is classified as a definition, it is executed first.

Logist also offers version control, check in and check out. Older versions of the rule base can be displayed, and you can see the differences in a specific rule. Logist provides the ability to logically partition the rule base into runtime files. You may not want to include all the dependent rules in a runtime file. Caution should be exercised since this can lead to problems.

Comments

Req#	Req Name	Rater 2 Comments	Rater 4 Comments	Rater 5 Comments
1	Gathering & reporting of analysis rules		Excellent—strong point of tool	I am very impressed with the traceability support.
2	Generation of the database			Export schema
3	Generation of database triggers		With the caveat that I don’t think placing rules in triggers is a good practice, the tool offers a way to do it (although clumsy)	Use plug-in components
4	Generation of process application code			No support of workflow process
5	How security is handled in the tool			More flexible
6	Configuration management			Rules resides in one place
7	Integration with database	Real time or batch mode rule assessment		Supports more DBMS however more work to integrate
8	Industry standard rule representation	Exploring standards, but have not advocated any particular rule set yet.	With the caveat that there is no standard rule representation at the moment,	Template form
9	Rule re-use		Lexical element reuse—Great	Same rule is reused in runtime
10	Performance	Built-in load balancing	Don’t know
11	Scalability for complex processes	No benchmarks	Don’t know
12	Learning curve	Training time is acceptable.	Seems to be very easy to learn; template-driven	Template
13	Reporting from repository	Flexible user-defined report interface.	Good	Traceability capability. Rule management capabilities
14	Open API	Cross-platform API availability.	Seems robust—it is probably possible to hook rules into legacy apps
15	Level of effort required	Not convinced that this is simple task.	Seems easy to learn
16	Scalability of number of users	Hardware bound.	Don’t know	Limitation
17	Development required	Too general.	Seems easy to use	Flexible
18	Deployment required		Not sure
19	Ease in changing rules	This interface was dynamite.	Strong point of tool	Rules in one place
20	Organization & visibility of rules	Probably the best interface for navigating through rule base.	Excellent
21	Impact Analysis		Great navigation	Rule management properties can be added in meta model
22	Rule interaction/conflict	Rule relationship graphs were neat.	Excellent; relationship graphs nice	Detection at analysis time
23	Potential for integration with other systems		Seems to be able to integrate due to API
24	Description of architectural approach		Very thorough	Rule- centric, rule integrated architecture
25	Impact of tool on SDLC	Conforms to SDLC	Seems to accomplish our goal: users can add/modify rules, simplifying SDLC	Supports some business analysis activities
26	Documentation of tool	Seems to be mostly document based as opposed to context.
27	Different types of rules supported		Seems to support most types	More flexible

V. Conclusions

The review committee appreciated the cooperation from all of the vendors for their participation in the shootout and for allowing us to provide a case study for all of them to solve. Three of the vendors actually provided solutions to the case study. The fourth vendor gave us a thorough demo but said that their main developer was ill and so they could not complete the case study. All vendors provided written responses to our questions. All provided us with a demonstration, and several provided two or more demonstrations to accommodate all of the reviewers.

To summarize there are three goals that the business rule approach and the tools evaluated are trying to achieve:

1. Provide a means of extending requirements gathering indefinitely to produce flexible systems
2. Provide a means of bridging the gap between business people and IT professionals
3. Provide an easy means to group “nuggets of information” in order to manage that information effectively

It should be emphasized that one clear conclusion stemming from the review of the participating tools is that the tools are solving similar problems from different points of view and with different objectives. It is therefore impossible to state that one product is necessarily better than the others. Each product has a viable position within the business rule product space.

At this point in the evolution of tools supporting rule-based development, all of the tools ultimately must drill down to some level of actual programming (writing code) either in proprietary languages or common languages such as Java or PL/SQL. While most of the tools offer some form of code writing assistant, they still require programming and therefore need some level of technical proficiency on the part of the user. All of the tools reviewed require skilled analysts/users and require formal training in order to get the most from them. This means that rule specification has not yet reached the point where the task can be passed to the business person.

At this point in time, Fair Isaac’s Blaze Advisor and ESI Logist come closest to achieving all three goals, however, both are external rule engines that bolt into and support the application system being developed. Both tools provide mature, highly polished user interfaces that, while complex, appear to be understandable and usable. They are an excellent choice for those wanting to document their business rules and migrate existing systems toward a more business rule-driven approach as well as for the development of new systems.

Oracle RuleFrame doesn’t attempt to bridge the gap between business users and IT professionals, but it helps Designer manage information more effectively to produce more flexible systems (points one and three). RuleFrame focuses on the technical user. It does, however, provide an extensive rule classification scheme that is useful in helping the technical user develop an understanding of the types of rules and how they should be supported in a database. Unfortunately RuleFrame neglects process-oriented rules entirely.

BRIM^® is perhaps the most intriguing of the tools available. Focused on providing the ability to build systems faster, it takes what appears to be a somewhat more technical approach than the other tools using Unified Modeling Language (UML) class diagrams and activity diagrams to model and collect the business rules. These diagrammers are more familiar to the technical person but they can also be easily learned and understood by the business person. BRIM^® also takes an interesting approach in that it allows the business user to define the rules while the technical can implement those rules through the same interface. This approach has real promise in that it does not require the business user to become a programmer. It allows the business person to focus on the specification of rules while leaving the programming to the technical staff. This may be the best balance that can be achieved at this point. The developer is able to see the business person’s specification of the rule in the same screen that is used to implement the rule. BRIM^® has added flexibility in that its functionality can be extended to support any analysis methodology in use by the organization, whether this consists of text based requirements documents, use cases, or the yet undiscovered silver bullet methodology. The extension of the BRIM^® repository in this manner provides a means of tracing the implementation of business rules back to the analysis work leading to their discovery. A comprehensive and extensive tool, BRIM^® is best suited to developing new systems, although it can also be used to extend existing systems with a little creativity.

All of the tools evaluated are still developing and maturing. Exciting times are ahead for us as we begin to incorporate them into our environments. It will be fun to see them respond to new challenges in the future.

Appendix – Information for Vendors

Due to length considerations, the entity and attribute descriptions have not been included. A copy of these can be requested from info@dulcian.com .
ODTUG 2003—Business Rules Tools Product Shootout

A. Overview

Business rules products support some portion of the business rules of a project. This evaluation will help us to ascertain which rules your product supports and how it supports them. We are tying to understand how the business rules are implemented and how your product can best be used to build a system. The goal is to determine how each product can support a specific list of business rules common to many organizations. The case study will provide you with a small system and a reasonable number of rules to implement.

B. Case Study Description

The case study for the business rules product shootout is a Work Request Processing system. It is an actual work-in-progress provided by the information technology department of a university. It represents a reengineering project that is underway at the university.

Business Problem to be Solved

The university wants to track work requests, group them into specific projects, and track the time and money resources allocated to them.

The business problem has two aspects:

1. The system is used to identify projects based upon work requests created by university members and establish cost estimates based on the resources necessary to complete the project.
2. Once a project has been scheduled, the system is used to assign resources and actually book time against the project’s tasks.

General Description

The system allows any member of the university community to request some type of project. Members include employees, students, and alumni.

Once a request is entered into the system, it is reviewed by a committee to decide whether or not to create a project based upon the request. The committee sets the priority for the project.

When a project is created, a project manager is assigned and the tasks/subtasks necessary to complete the project are identified. Some projects have a limit on the types of tasks that they can contain. Resource requirements can be recorded against the project or the task.

Resources are assigned to the tasks and, as work is performed, timesheets are created to record the work applied to a task/project.

This environment is very typical of one into which a business rules tool may be introduced. The documentation being provided is excerpted from actual project documentation. It consists of the following:

· A simple requirements document, which provides a description of the high-level business requirements and rules

· An entity-relationship diagram that provides a conceptual view of the planned system

· An entity-definition report, which provides definitions for all of the entities in the model

· An attribute report, which defines the attributes shown in the model

Business rules may be extracted from any of these documents.

C. Product Shootout Requirements

It is not necessary to implement the specific data model provided. It is necessary to provide support for all of the business rules identified in the Work Request System Requirements. You should build some or all of a system to support these requirements.

The evaluation of the system will be done by knowledgeable technical people, so you do not need to spend a great deal of time or effort creating an elaborate user interface design. The evaluation team is really more interested in the way in which the business rules are supported.

You also do not necessarily need to complete the system to be evaluated well in the product shootout. The goal is to communicate to the evaluation team how your tool works. We suspect that the more system you build the easier it will be to evaluate your product, but if time pressure is an issue, it is better to build a portion of the system well than an entire system badly.

The requirements in the Work Request System Information zip file are adapted from an actual system that is currently in development. The data model included is only to be taken as a suggestion. It was one modeler’s logical data model based on the requirements. Do not feel obligated to build to this model. It is included only to help convey one method of building the system.

D. Evaluation Criteria

The following is a list of the criteria that will be used in evaluating the deliverables submitted (Word document and demo):

1. System-created supports gathering and reporting of analysis rules
2. System-created supports generation of the database
3. System-created supports generation of database triggers
4. System-created supports generation of process application code
5. Security—how it is handled in tool
6. Configuration management —how it is handled in tool
7. Integration with database
8. Industry standard rule representation
9. Rule re-use
10. Performance
11. Scalability for complex processes
12. Learning curve required to use system
13. Reporting from repository (documentation of built system)
14. Open API
15. Level of effort required
16. Scalability of number of users
17. Development required—hardware/software
18. Deployment required—hardware/software
19. Ease in changing rules
20. Organization and visibility of rules (How easy is it to find a specific rule?)
21. Impact analysis (What organizational impact might the rules have?)
22. Rule interaction/conflict
23. Potential for integration with other systems
24. Description of architectural approach
25. Impact of tool on SDLC
26. Documentation of tool
27. Different types of rules supported (list them)

E. Deliverables

1. Written report: Create a Word document addressing each of the criteria listed above. This will help the team to understand your approach for each area.

2. Working system: You do not need to deliver the system itself to the evaluation team. We will schedule web demos for the team. If you do not have a facility for this, we suggest looking into the Go to My PC software or a similar product. You will need to support up to 8 different evaluation team members viewing your presentation.

F. Deadlines

The Written Report should be delivered to Caryl Lee Fisher on or before February 28, 2003. Demos of the Working System will be scheduled between March 3–April 25, 2003.

Questions regarding the product shootout should be e-mailed to Caryl Lee Fisher at cfisher@dulcian.com.

Work Request System Requirements

Overview

Work requests are initiated by a user. The service request is entered by the user and published to the identified list of stakeholders. All work requests must be reviewed, approved and prioritized by a committee. Authorized personnel create projects from sets of approved service requests.

Projects are divided into tasks and sub-task by authorized personnel. They can also give these tasks statues, priorities, time estimates, resource estimates and resource assignments. Authorized personnel can estimate time and resource needs at the project, task and sub-task level. Estimated staffing needs are based on job codes, while estimated equipment needs are based on equipment types. Assigned resources may include staff and/or equipment.

A person doing work on a project fills out a timesheet logging hours against specific tasks.

A person may do work that is not applied to any project.

Detail and summary reports can be generated on demand from the Job Costing system for service requests, project management and timekeeping.

REF	REQUIREMENTS
	Service Request
1.0	A service request may be entered by any user.
2.0	A service request must have an originating person and a sponsoring department.
3.0	A service request must contain: Originator’s name, job title, phone number and department Date submitted Date desired Service request title Detailed description of problem or opportunity Goal Objectives Stakeholders (Person or department who is directly impacted by the current problem or potentially impacted by its solution.) Name Job title (if applicable) Organization Phone number
4.0	Completed service requests must be published to their stakeholders.
5.0	Any user can give feedback on any service request.
6.0	Service request must be approved and prioritized by committee before becoming a project
7.0	Cancellation and rejection of a Service request must be allowed at any time prior to approval.
7.1	Service request can only be cancelled by the originating person or sponsoring department.
7.2	Service request can only be rejected by the originating person, sponsoring department or approval committee.
8.0	Projects originate from approved Service Requests and may only be created by authorized personnel.
8.1	Projects cannot exist without at least 1 associated service request.

	Project Management
1.0	Allow for resource (staffing and equipment; i.e., hardware, software, etc.) planning and assignments based on estimated and actual needs.
1.1	Allow for the assignments of staff who report to multiple supervisors and organizations.
1.2	Allow for multiple task assignments of staff.
2.0	Project planning must track projected staff assignments based on job description.
3.0	Project planning must track equipment; i.e., hardware, software, etc. assignments based on category.
4.0	Task estimated time cannot exceed the estimated time of the related project.
5.0	Tasks can be broken into sub-tasks.
5.1	Sub-task estimated time cannot exceed the estimated time of the parent task.
6.0	Tasks or sub-tasks may only be created by authorized personnel.
7.0	A person cannot be assigned a task activity without first being associated with an Organization.
7.1	A Project Manager must have authority over an individual to assign that individual to a task.
8.0	The assignment of tasks must allow for either a restricted (individual) or non-restricted (any available) assignment.
9.0	Equipment; i.e., hardware, software, etc. data will be supplied by a Materials Management system.
10.0	Resources assigned to a task must have appropriate characteristics/skill sets.
11.0	The types of tasks created under a project are to be restricted to those task types appropriate to the project.
12.0	Project manager must be able to request automatic notifications based on task status and time expended v. time projected.
13.0	Project manager should be able to define task or sub-task estimates as hard (cannot exceed) or soft can exceed). The system should notify the project manager if hard constraints are violated.
	Timekeeping
1.0	Compensatory, vacation and sick time balances will be supplied by Human Resource.
2.0	A person can only charge time to an active task and not to projects (enhancement, development, research) or non-project type activities (administrative and maintenance).
3.0	Must allow for the entry of timekeeping comments regarding assignments.
4.0	Vacation/Sick/Compensatory time taken may not exceed available balances.
5.0	Time charged to a task in excess of time projected for the task must be approved by the project manager.
6.0	Total hours in excess of 60 should result in notification to the worker’s supervisor. (Vacation, sick and holiday time do not count in this calculation)
7.0	Total hours less than the norm for the period should result in notification to the worker’s supervisor. (Vacation, sick and holiday time do count in this calculation)
	Reports
1.0	Must be able to group both summary and detail reports by any or all of the following: · Service Request · Project · Task · Organization · Person · Date Range · Category (Administrative, Maintenance, Enhancement, Research, Development)
	General
1.0	Staff may have multiple job descriptions.