Increase absolute tolerance to 1e-4

[casella]

Reading from the report headings, the current tolerance for testing is set at 2% relative, 1e-6 absolute. This is problematic for signals going trhough zero, e.g. Modelica.Electrical.Analog.Examples.CompareTransformers.

@AnHeuermann for the MSL we set this to 1e-4 to avoid false negative. Can you please update the absolute tolerance for the verification to that value?

Thanks!

[AnHeuermann]

This is a tricky one. Going from absolute tolerance $\tau_{abs} = 10^{-6}$ to $10^{-4}$ doesn't solve this. The relative error $\tau_{rel}$ is around 3 to 4% while the basicTransformer.i2 is far enough away from zero. It's also not an interpolation issue. I use the dense output of the solver to interpolate with the same order as the ODE integrator at the reference time points. The saved CSV is using different time point, the reference results aren't considered at that location.

Maybe the solver is the issue here. I'm using a low tolerance solver Rodas5P (5th order A-stable stiffly stable Rosenbrock method with a stiff-aware 4th order interpolant. Has improved stability in the adaptive time stepping embedding).

For default tolerances, AutoTsit5(Rosenbrock23()) is a good choice. For lower tolerances, using AutoVern7 or AutoVern9 with Rodas4, KenCarp4, or Rodas5P can all be good choices depending on the problem. For very large systems (>1000 ODEs?), consider using lsoda.

From https://docs.sciml.ai/DiffEqDocs/stable/solvers/ode_solve/.

Changing that one is difficutl. The ODE is in mass matrix formulation, so not all solvers are compatible. And since it is an ODE I can't use the DAE solver IDA from Sundials.jl directly.

Or we need to use a different way to compare two trajectories, e.g. by using modelica-tools/csv-compare. But that is not straight forward to do.

[casella]

Hmm, I'm trying to figure out what's wrong here, but it seems the difference is only at the very beginning of the simulation, but I can't zoom in enough to really understand what is going on.

Maybe the reference result doesn't have enough time resolution there, but on the other hand its' a lot of data points. We're discussing in MAP-Lib how to reduce the size of the (huge) reference files for those models, by using only averaged variables for the verification of the Modelica example model, and then adding an extra simulation of the same model in ModelicaTest with a much shorter StopTime.

This will have to wait for a while.

Maybe you can manually try to run a much shorter transient, e.g. with StopTime = 0.1, with a shorter reporting interval, both with OMC and Julia, and manually see if the result match better in that interval.