Release Notes

pyMOR 0.5 (January 17, 2019)

After more than two years of development, we are proud to announce the release of pyMOR 0.5! Highlights of this release are support for Python 3, bindings for the NGSolve finite element library, new linear algebra algorithms, various VectorArray usability improvements, as well as a redesign of pyMOR’s projection algorithms based on RuleTables.

Especially we would like to highlight the addition of various system-theoretic reduction methods such as Balanced Truncation or IRKA. All algorithms are implemented in terms of pyMOR’s Operator and VectorArray interfaces, allowing their application to any model implemented using one of the PDE solver supported by pyMOR. In particular, no import of the system matrices is required.

Over 1,500 single commits have entered this release. For a full list of changes see here.

pyMOR 0.5 contains contributions by Linus Balicki, Julia Brunken and Christoph Lehrenfeld. See here for more details.

Release highlights

Python 3 support

pyMOR is now compatible with Python 3.5 or greater. Since the use of Python 3 is now standard in the scientific computing community and security updates for Python 2 will stop in less than a year (https://pythonclock.org), we decided to no longer support Python 2 and make pyMOR 0.5 a Python 3-only release. Switching to Python 3 also allows us to leverage newer language features such as the @ binary operator for concatenation of Operators, keyword-only arguments or improved support for asynchronous programming.

System-theoretic MOR methods

With 386 commits, [#464] added systems-theoretic methods to pyMOR. Module pymor.discretizations.iosys contains new discretization classes for input-output systems, e.g. LTISystem, SecondOrderSystem and TransferFunction. At present, methods related to these classes mainly focus on continuous-time, non-parametric systems.

Since matrix equation solvers are important tools in many system-theoretic methods, support for Lyapunov, Riccati and Sylvester equations has been added in pymor.algorithms.lyapunov, pymor.algorithms.riccati and pymor.algorithms.sylvester. A generic low-rank ADI (Alternating Direction Implicit) solver for Lyapunov equations is implemented in pymor.algorithms.lradi. Furthermore, bindings to low-rank and dense solvers for Lyapunov and Riccati equations from SciPy, Slycot and Py-M.E.S.S. are provided in pymor.bindings.scipy, pymor.bindings.slycot and pymor.bindings.pymess. A generic Schur decomposition-based solver for sparse-dense Sylvester equations is implemented in pymor.algorithms.sylvester.

Balancing Truncation (BT) and Iterative Rational Krylov Algorithm (IRKA) are implemented in BTReductor and IRKAReductor. LQG and Bounded Real variants of BT are also available (LQGBTReductor, BRBTReductor). Bitangential Hermite interpolation (used in IRKA) is implemented in LTI_BHIReductor. Two-Sided Iteration Algorithm (TSIA), a method related to IRKA, is implemented in TSIAReductor.

Several structure-preserving MOR methods for second-order systems have been implemented. Balancing-based MOR methods are implemented in pymor.reductors.sobt, bitangential Hermite interpolation in SO_BHIReductor and Second-Order Reduced IRKA (SOR-IRKA) in SOR_IRKAReductor.

For more general transfer functions, MOR methods which return LTISystems are also available. Bitangential Hermite interpolation is implemented in TFInterpReductor and Transfer Function IRKA (TF-IRKA) in TF_IRKAReductor.

Usage examples can be found in the heat and string_equation demo scripts.

NGSolve support

We now ship bindings for the NGSolve finite element library. Wrapper classes for VectorArrays and matrix-based Operators can be found in the pymor.bindings.ngsolve module. A usage example can be found in the thermalblock_simple demo script.

New linear algebra algorithms

pyMOR now includes an implementation of the HAPOD algorithm for fast distributed or incremental computation of the Proper Orthogonal Decomposition (pymor.algorithms.hapod). The code allows for arbitrary sub-POD trees, on-the-fly snapshot generation and shared memory parallelization via concurrent.futures. A basic usage example can be found in the hapod demo script.

In addition, the Gram-Schmidt biorthogonalization algorithm has been included in pymor.algorithms.gram_schmidt.

VectorArray improvements

VectorArrays in pyMOR have undergone several usability improvements:

  • The somewhat dubious concept of a subtype has been superseded by the concept of VectorSpaces which act as factories for VectorArrays. In particular, instead of a subtype, VectorSpaces can now hold meaningful attributes (e.g. the dimension) which are required to construct VectorArrays contained in the space. The id attribute allows to differentiate between technically identical but mathematically different spaces [#323].
  • VectorArrays can now be indexed to select a subset of vectors to operate on. In contrast to advanced indexing in NumPy, indexing a VectorArray will always return a view onto the original array data [#299].
  • New methods with clear semantics have been introduced for the conversion of VectorArrays to (to_numpy) and from (from_numpy) NumPy arrays [#446].
  • Inner products between VectorArrays w.r.t. to a given inner product Operator or their norm w.r.t. such an operator can now easily be computed by passing the Operator as the optional product argument to the new inner and norm methods [#407].
  • The components method of VectorArrays has been renamed to the more intuitive name dofs [#414].
  • The l2_norm2 and norm2 have been introduced to compute the squared vector norms [#237].

RuleTable based algorithms

In pyMOR 0.5, projection algorithms are implemented via recursively applied tables of transformation rules. This replaces the previous inheritance-based approach. In particular, the projected method to perform a (Petrov-)Galerkin projection of an arbitrary Operator has been removed and replaced by a free project function. Rule-based algorithms are implemented by deriving from the RuleTable base class [#367], [#408].

This approach has several advantages:

  • Rules can match based on the class of the object, but also on more general conditions, e.g. the name of the Operator or being linear and non-parametric.
  • The entire mathematical algorithm can be specified in a single file even when the definition of the possible classes the algorithm can be applied to is scattered over various files.
  • The precedence of rules is directly apparent from the definition of the RuleTable.
  • Generic rules (e.g. the projection of a linear non-parametric Operator by simply applying the basis) can be easily scheduled to take precedence over more specific rules.
  • Users can implement or modify RuleTables without modification of the classes shipped with pyMOR.

Additional new features

  • Reduction algorithms are now implemented using mutable reductor objects, e.g. GenericRBReductor, which store and extend the reduced bases onto which the model is projected. The only return value of the reductor’s reduce method is now the reduced discretization. Instead of a separate reconstructor, the reductor’s reconstruct method can be used to reconstruct a high-dimensional state-space representation. Additional reduction data (e.g. used to speed up repeated reductions in greedy algorithms) is now managed by the reductor [#375].

  • Linear combinations and concatenations of Operators can now easily be formed using arithmetic operators [#421].

  • The handling of complex numbers in pyMOR is now more consistent. See [#458], [#362], [#447] for details. As a consequence of these changes, the rhs Operator in StationaryDiscretization is now a vector-like Operator instead of a functional.

  • The analytical problems and discretizers of pyMOR’s discretization toolbox have been reorganized and improved. All problems are now implemented as instances of StationaryProblem or InstationaryProblem, which allows an easy exchange of data Functions of a predefined problem with user-defined Functions. Affine decomposition of Functions is now represented by specifying a LincombFunction as the respective data function [#312], [#316], [#318], [#337].

  • The pymor.core.config module allows simple run-time checking of the availability of optional dependencies and their versions [#339].

  • Packaging improvements

    A compiler toolchain is no longer necessary to install pyMOR as we are now distributing binary wheels for releases through the Python Package Index (PyPI). Using the extras_require mechanism the user can select to install either a minimal set:

    pip install pymor
    

    or almost all, including optional, dependencies:

    pip install pymor[full]
    

    A docker image containing all of the discretization packages pyMOR has bindings to is available for demonstration and development purposes:

    docker run -it pymor/demo:0.5 pymor-demo -h
    docker run -it pymor/demo:0.5 pymor-demo thermalblock --fenics 2 2 5 5
    

Backward incompatible changes

  • dim_outer has been removed from the grid interface [#277].
  • All wrapper code for interfacing with external PDE libraries or equation solvers has been moved to the pymor.bindings package. For instance, FenicsMatrixOperator can now be found in the pymor.bindings.fenics module. [#353]
  • The source and range arguments of the constructor of ZeroOperator have been swapped to comply with related function signatures [#415].
  • The identifiers discretization, rb_discretization, ei_discretization have been replaced by d, rd, ei_d throughout pyMOR [#416].
  • The _matrix attribute of NumpyMatrixOperator has been renamed to matrix [#436]. If matrix holds a NumPy array this array is automatically made read-only to prevent accidental modification of the Operator [#462].
  • The BoundaryType class has been removed in favor of simple strings [#305].
  • The complicated and unused mapping of local parameter component names to global names has been removed [#306].

Further notable improvements

pyMOR 0.4 (September 28, 2016)

With the pyMOR 0.4 release we have changed the copyright of pyMOR to

Copyright 2013-2016 pyMOR developers and contributors. All rights reserved.

Moreover, we have added a Contribution guideline to help new users with starting to contribute to pyMOR. Over 800 single commits have entered this release. For a full list of changes see here. pyMOR 0.4 contains contributions by Andreas Buhr, Michael Laier, Falk Meyer, Petar Mlinarić and Michael Schaefer. See here for more details.

Release highlights

FEniCS and deal.II support

pyMOR now includes wrapper classes for integrating PDE solvers written with the dolfin library of the FEniCS project. For a usage example, see pymordemos.thermalblock_simple.discretize_fenics. Experimental support for deal.II can be found in the pymor-deal.II repository of the pyMOR GitHub organization.

Parallelization of pyMOR’s reduction algorithms

We have added a parallelization framework to pyMOR which allows parallel execution of reduction algorithms based on a simple WorkerPool interface [#14]. The greedy [#155] and ei_greedy algorithms [#162] have been refactored to utilize this interface. Two WorkerPool implementations are shipped with pyMOR: IPythonPool utilizes the parallel computing features of IPython, allowing parallel algorithm execution in large heterogeneous clusters of computing nodes. MPIPool can be used to benefit from existing MPI-based parallel HPC computing architectures [#161].

Support classes for MPI distributed external PDE solvers

While pyMOR’s VectorArray, Operator and Discretization interfaces are agnostic to the concrete (parallel) implementation of the corresponding objects in the PDE solver, external solvers are often integrated by creating wrapper classes directly corresponding to the solvers data structures. However, when the solver is executed in an MPI distributed context, these wrapper classes will then only correspond to the rank-local data of a distributed VectorArray or Operator.

To facilitate the integration of MPI parallel solvers, we have added MPI helper classes [#163] in pymor.vectorarrays.mpi, pymor.operators.mpi and pymor.discretizations.mpi that allow an automatic wrapping of existing sequential bindings for MPI distributed use. These wrapper classes are based on a simple event loop provided by pymor.tools.mpi, which is used in the interface methods of the wrapper classes to dispatch into MPI distributed execution of the corresponding methods on the underlying MPI distributed objects.

The resulting objects can be used on MPI rank 0 (including interactive Python sessions) without any further changes to pyMOR or the user code. For an example, see pymordemos.thermalblock_simple.discretize_fenics.

New reduction algorithms

  • adaptive_greedy uses adaptive parameter training set refinement according to [HDO11] to prevent overfitting of the reduced model to the training set [#213].
  • reduce_parabolic reduces linear parabolic problems using reduce_generic_rb and assembles an error estimator similar to [GP05], [HO08]. The parabolic_mor demo contains a simple sample application using this reductor [#190].
  • The estimate_image and estimate_image_hierarchical algorithms can be used to find an as small as possible space in which the images of a given list of operators for a given source space are contained for all possible parameters mu. For possible applications, see reduce_residual which now uses estimate_image_hierarchical for Petrov-Galerkin projection of the residual operator [#223].

Copy-on-write semantics for VectorArrays

The copy method of the VectorArray interface is now assumed to have copy-on-write semantics. I.e., the returned VectorArray will contain a reference to the same data as the original array, and the actual data will only be copied when one of the arrays is changed. Both NumpyVectorArray and ListVectorArray have been updated accordingly [#55]. As a main benefit of this approach, immutable objects having a VectorArray as an attribute now can safely create copies of the passed VectorArrays (to ensure the immutability of their state) without having to worry about unnecessarily increased memory consumption.

Improvements to pyMOR’s discretizaion tookit

  • An unstructured triangular Grid is now provided by UnstructuredTriangleGrid. Such a Grid can be obtained using the discretize_gmsh method, which can parse Gmsh output files. Moreover, this method can generate Gmsh input files to create unstructured meshes for an arbitrary PolygonalDomain [#9].
  • Basic support for parabolic problems has been added. The discretize_parabolic_cg and discretize_parabolic_fv methods can be used to build continuous finite element or finite volume Discretizations from a given pymor.analyticalproblems.parabolic.ParabolicProblem. The parabolic demo demonstrates the use of these methods [#189].
  • The pymor.discretizers.disk module contains methods to create stationary and instationary affinely decomposed Discretizations from matrix data files and an .ini file defining the given problem.
  • EllipticProblems can now also contain advection and reaction terms in addition to the diffusion part. discretize_elliptic_cg has been extended accordingly [#211].
  • The continuous Galerkin module has been extended to support Robin boundary conditions [#110].
  • BitmapFunction allows to use grayscale image data as data Functions [#194].
  • For the visualization of time-dependent data, the colorbars can now be rescaled with each new frame [#91].

Caching improvements

  • state id generation is now based on deterministic pickling. In previous version of pyMOR, the state id of immutable objects was computed from the state ids of the parameters passed to the object’s __init__ method. This approach was complicated and error-prone. Instead, we now compute the state id as a hash of a deterministic serialization of the object’s state. While this approach is more robust, it is also slightly more expensive. However, due to the object’s immutability, the state id only has to be computed once, and state ids are now only required for storing results in persistent cache regions (see below). Computing such results will usually be much more expensive than the state id calculation [#106].
  • CacheRegions now have a persistent attribute indicating whether the cache data will be kept between program runs. For persistent cache regions the state id of the object for which the cached method is called has to be computed to obtain a unique persistent id for the given object. For non-persistent regions the object’s uid can be used instead. pymor.core.cache_regions now by default contains 'memory', 'disk' and 'persistent' cache regions [#182], [#121] .
  • defaults can now be marked to not affect state id computation. In previous version of pyMOR, changing any default value caused a change of the state id pyMOR’s defaults dictionary, leading to cache misses. While this in general is desirable, as, for instance, changed linear solver default error tolerances might lead to different solutions for the same Discretization object, it is clear for many I/O related defaults, that these will not affect the outcome of any computation. For these defaults, the defaults decorator now accepts a sid_ignore parameter, to exclude these defaults from state id computation, preventing changes of these defaults causing cache misses [#81].
  • As an alternative to using the @cached decorator, cached_method_call can be used to cache the results of a function call. This is now used in solve to enable parsing of the input parameter before it enters the cache key calculation [#231].

Additional new features

Backward incompatible changes

pyMOR 0.3 (March 2, 2015)

  • Introduction of the vector space concept for even simpler integration with external solvers.
  • Addition of a generic Newton algorithm.
  • Support for Jacobian evaluation of empirically interpolated operators.
  • Greatly improved performance of the EI-Greedy algorithm. Addition of the DEIM algorithm.
  • A new algorithm for residual operator projection and a new, numerically stable a posteriori error estimator for stationary coercive problems based on this algorithm. (cf. A. Buhr, C. Engwer, M. Ohlberger, S. Rave, ‘A numerically stable a posteriori error estimator for reduced basis approximations of elliptic equations’, proceedings of WCCM 2014, Barcelona, 2014.)
  • A new, easy to use mechanism for setting and accessing default values.
  • Serialization via the pickle module is now possible for each class in pyMOR. (See the new ‘analyze_pickle’ demo.)
  • Addition of generic iterative linear solvers which can be used in conjunction with any operator satisfying pyMOR’s operator interface. Support for least squares solvers and PyAMG (http://www.pyamg.org/).
  • An improved SQLite-based cache backend.
  • Improvements to the built-in discretizations: support for bilinear finite elements and addition of a finite volume diffusion operator.
  • Test coverage has been raised from 46% to 75%.

Over 500 single commits have entered this release. A full list of all changes can be obtained under the following address: https://github.com/pymor/pymor/compare/0.2.2…0.3.0