This page details what to do when the results of the OE are not what we “were expecting”.
I do not see the impact on the change of scope/constraints on the results
Possible cause: All the needed calculation steps are not executed.
Solution: When the scope, the business rules or the objectives are changed, the run step will have to be executed again to compute the new results. After that, executing the results step is needed to update the visualizations.
Some results are really strange (massive negative margins etc.)
Check: Check the source data; often we are missing a filtering step removing bogus inputs or data that do not fit the scope of the problem. Ensure that you are working on the correct subset of data and if the behavior persists, look into the tables to find the explanation with the ID of the involved points.
The target objectives are not reached
Check: Look at the information tooltip next to the finished OE job. If it says that “Some value finders are still moving fast with unsatisfied neighbor” it means that the OE needs more steps to find the solution. This is especially true if a large portion of the value finders is moving fast.
Solution: Change the number accordingly and start a new run.
Possible cause 1: The constraint is not tuned enough.
Solution: Increase the priority of the constraint, check its target value and the minimum steps of the Value Finder.
Possible cause 2: Over-constrained problem: the OE job says that “Some value finders are very close to their boundaries and may be stuck there” it means that the solution of the problem is outside its research space.
We can still pinpoint what are these problematic constraints and what are the elements affected.
We can, if it makes business sense, adjust other constraints to make for room for the optimization.
Define with the client what are the most important constraints to satisfy.
Note: Do not hesitate to test some runs with more lenient constraints: people will often try to put more constraints than needed, by trying to predict the impact of a modification somewhere and applying their expected results on the intermediate steps from the product the target aggregation. More often than not it will only restrict the optimization space for no gain.
Possible cause 3: Over-constrained problem from conflicting objectives. Two or more constraints push the system in irreconcilables directions, and it settled at the best compromise it could. For example, a constraint to reduce the list price and a constraint to increase its margin.
Check: The “Unacceptable” lines in the results tables to validate that other constraints are in conflict with the one you are working on.
Solution: With feedback from the domain, decide if you have to modify some constraints or to add an exception for specific agents (e.g. not optimizing for some specific product).
The constraints are satisfied but the solution is not “good enough”
Possible cause: The objectives/constraints are not precise enough.
Solution: Fine tune the objectives precision (acceptable_delta, target…) using the previous results.
Possible cause: Some domain rules are not modeled yet. It can be the case when a client forgets to state some constraints they consider as obvious, such as “price cannot drop” or “margin must increase”.
Solution: Loop with the client to ensure that the domain is fully modelled and that the constraints have the correct type (e.g. target instead of threshold).