Trying to execute code after optimization completes

General Discussions
1

Is there any state or coding technique such that I can determine when the last strategy run of an optimization (and hence the optimization) has completed ?

I capture various metrics per strategy during a backtest and/optimization and want to perform a final calculation ( and close a custom log file) once completed

Thanks in advance

2

I do not know of a straightforward way to do this in NinjaTrader. Part of the reason for this is that backtest iterations in Strategy Analyzer have no guaranteed execution order because they run in parallel on multiple threads.

The solution will be to use a couple of static data members in your strategy and make a copy of the default optimizer where you modify its OnOptimize method.

The following code is not tested, but it will point you in the right direction.

Start by adding a couple of static fields to your strategy class along with a method to increment the counter.

// In your strategy

public static int OptIterationCount = 0;
public static int OptIterationCountMax = 0;
private static object _lockObject = new object();
private bool _isLastIteration = false;

protected override OnStateChange()
{
   if (State == State.Configure)
   {
      _IsLastIteration = false;
      IncrementOptIterationCount();
      // Or if you want to run code after the last iteration finishes, you could call
      // IncrementOptIterationCount() in OnBarUpdate on the penultimate or final bar instead
      // of here.
   }
}

public void IncrementOptIterationCount()
{
   lock (_lockObject)
   {
      OptIterationCount++;
      if (OptIterationCount == OptIterationCountMax)
      {
         _isLastIteration = true;
      }
   }
}

The @DefaultOptimizer class has this for the OnOptimize method.

   protected override void OnOptimize() => Iterate(0);

Make a copy of the @DefaultOptimizer so you can edit it. Give your new optimizer class a new class name and a new Name in State.SetDefaults.

In your new optimizer class, the OnOptimize method runs exactly once for your optimization. (If you optimize instruments it runs once per instrument.)

Whatever code you add before the call to Iterate(0) will run at the start of the optimization.

Iterate(0) iterates over all possible parameter combinations and submits a batch of iterations asynchronously, meaning the code execution could return immediately before most of the backtests are even done submitting. If you have a lot of batches (batch size determined by NT and the number of Logical Processors on your machine) then execution will return immediately after submitting the last batch of iterations, before they have all completed.

After Iterate(0), call WaitForIterationsCompleted(); to wait until all iterations have completed. Then whatever code runs after that you’ll know that all the backtests have finished and you really are executing at the end of the optimization.

   protected override void OnOptimize()
   {
      // Initialize your static fields at the start of the optimization.
      YourStrategyClassName.OptIterationCount = 0;
      YourStrategyClassName.OptIterationCountMax = NumberOfIterations;

      Iterate(0);

      WaitForIterationsCompleted();

      // Reset the static fields on your strategy.
      YourStrategyClassName.OptIterationCount = 0;
      YourStrategyClassName.OptIterationCountMax = NumberOfIterations;
   }

In the above code, I

  • initialize the static fields on the strategy (NumberOfIterations is calculated in OnStateChange which runs before OnOptimize.)
  • submit the iterations
  • wait for all iterations to complete
  • clear the static fields so they’re ready for the next optimization (or whatever other cleanup code you want here.

Hope this helps. Good luck.

3

Hi Steve,
Thanks so much for the excellent feedback. I’ve done much of what you’ve mentioned above, ie static fields in the strategy, locks where needed (I have a custom streamwriter log file that all the strategies update…) etc.

Trying not to do too deep a dive into c#, but short of adding attributes to the strategy/using reflection , is there any way to expose the custom static fields and/or custom methods I have in the strategies? ie if I created a cleanup() method in my strategy, any way to inform the optimizer of such a method? That way the optimizer could just call .cleanup() after its iterated thru all the strategy instances.

Thanks in advance

David

4

Yes.

protected override void OnOptimize()
{
  bool stratIsOfMyType = false;
  if (Strategies[0] is MyStrategyClassType)
  {
      stratIsOfMyType = true;
  }
   
  // Initialize your static fields at the start of the optimization.
  YourStrategyClassName.OptIterationCount = 0;
  YourStrategyClassName.OptIterationCountMax = NumberOfIterations;

  Iterate(0);

  WaitForIterationsCompleted();

  if (stratisOfMyType)
  {
      // Should probably call a static method. See note below.
      MyStrategyClassType.Cleanup();
  }

}

Your cleanup method should generally be a static method because this strategy instance is not the same as the instance that actually executes. If you want to do some kind of cleanup for each strategy iteration that executes you’ll need to do it in the strategy code or log whatever information you need from each strategy iteration into a static variable that your cleanup method can then access and handle.

1 Like
5

Hey Steve,

Thanks again for excellent feedback. Very much appreciated

David

1 Like