Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Scenario:

Has anybody else experienced timeouts in This article explains how timeouts are handled in a logic and in libraries which call each other? Right now I have lots of business logic in Groovy libraries, but because of this error I have had to set the timeouts to max. When a Groovy library logic is called, the timeout seems to be summed into some global cache which records the amount of time spent inside that element during the current execution thread. The better approach should be that the timeout is only applicable per-call to the element. Maybe easier would be to only obey the parent elements timeout, disregard all sub-elements timeouts.

Explanation:

Let me add some info on how timeouts technically work. Timeouts are essentially safety catches against a bad logic (such as accidental infinite loops). Now the JVM does not have something like a setting "run this thread, but only for x seconds". So we need to apply some tricks to make this work. The trick here is that we inject (at compile time) some code that throws an exception if the system time has elapsed the start time + timeout. Now the different logic elements are essentially Groovy script classes. So there is no easy sharing of a script-global "start time" variable in all thinkable cases. So we resort to using a start time variable per element (normal or lib element) which is initialized at the class instantiation. So essentially a private script class member variable which is set to the system time in the constructor of the script class. The engine "resets" that start time for lib elements by re-instantiating the lib elements before executing every normal formula element. Normal elements are only instantiated once (hence the topic of "timer starts running when element is executed first time" when you call previous elements further down).

There were some recent improvements with regards to better reporting on which exact element a timeout happened, and that a timeout change also leads to recompilation (to make it effective), but the fundamentals have not changed. And unless someone comes up with a way to have this safety latch implemented with a single timeout value on element level without compromising the effectiveness it will probably stay that way.

Hint: The suggestion "simply do not compile in timeouts for lib elements" does not work as then an infinite loop in a lib would not be caught... And no, from a system perspective we cannot trust any groovy code to be "safe":

  • The executed element timeout is always honored.

  • Library elements timeouts are ignored. What matters is the time the logic element is executed, including library calls.

  • Elements timeouts are always capped by the maximum timeout configured at the cluster level.

    • This setting defaults to 900s (15 min).

    • It can be increased by Pricefx Support via the setting formulaEngine.script.maxTimeoutInSec.

  • Special cases:

    • Since 10.0: PA Data Loads, PA Distributed Calculation, DMModel Calculations and Model Calculations completely ignore the timeouts in their logics but not in their libraries.

    • Since 10.2: PA Data Loads, PA Distributed Calculation, DMModel Calculations and Model Calculations completely ignore the timeouts in their logics and in the libraries they use.

    • Before 10.2: Elements timeout is set when the logic is pushed/saved to the partition. So if you increase the maximum timeout in your cluster, you need to resave your logic.

    • If an element just calls a library and the library has a higher timeout than the element, then the library element timeout is honored.