3.1. Software architecture and data flow

The software architecture consists of the following principal parts:

  • interface specification language, from which actual code is generated (to be added)

  • user-facing libraries that allow to invoke multiple implementations of numerical algorithms via interfaces

  • libraries for different languages that handle data marshalling to the dispatch library that forwards the data to the requested implementation

  • adapters for implementations

  • actual implementations of the numerical algorithms

Fig. 3.1.1 shows the principal components of the software architecture, the boundary between the code that is visible to the user and the data flow.

Software architecture of the Open Interfaces project

Fig. 3.1.1 Software architecture of the Open Interfaces project.

3.1.1. Project folder structure

The principal parts of the project are top-level directories:

interfaces/ # Language-independent interface specifications
    qeq.idl
    linsolve.idl
oif/        # Data marshalling and dispatch
oif_impl/   # Example implementation of interfaces

Note that we explicitly separate interfaces and implementations, so that the implementations can be developed distributedly. For example, we provide an interface for an optimization procedure and the authors of optimization packages could provide the adapters (implement the interface) for their packages.

3.1.1.1. Language-independent interface specifications

Language-independent interface specifications are not yet implemented. However, it is assumed that in the development of the project they will be organized as follows:

interfaces/
    qeq.idl      # Interface for solvers of quadratic equation
    linsolve.idl # Interface for solving systems of linear algebraic equations

Interfaces are declared in files with Interface Declaration Language syntax. Consequently, they are used to generate language-specific code for these interfaces.

3.1.1.2. User-facing library

The user interacts with liboif, the library that provides

  • language-specific interfaces that the user can invoke with given implementation

  • data marshalling to pass data between different languages

  • invocation of the requested implementation

The structure is the following:

oif/
    include/
        oif/
            # interface that bridge_LANG.c must implement
            bridge_api.h
            # definition of constants and types
            api.h
            # definition of supporting C functions for users
            c_bindings.h

     dispatch/
         # Implementation of the dispatch functionality
         oif_dispatch.c

     c/  # Bindings to liboif for C users
         src/
             # Implementation of auxiliary functions
             oif_c_bindings.c

    # Bindings to liboif for Python users
    python/
        oif/
            __init__.py
            core.py

    # later everything is autogenerated in here
    interfaces/
       c/
          include/
             qeq.h
          src/
             qeq.c
       python/
          oif/
             __init__.py
             interfaces/
                __init__.py
                qeq.py

Directory oif contains:

  • generated language-specific modules that are used directly by user to solve a computational problem, who also specifies the implementation as a string identifier

  • language bindings that accept data from the previous layer and convert to interoperable form

  • dispatch library that accepts data from the bindings layer, and dispatches them to the given implementation

3.1.1.3. Interface implementations

The structure is the following:

oif_impl/
   python/
      python_dispatch.c
      oif/
         __init__.py
         impl/ (later all auto-generated)
            __init__.py
            qeq.py  (abstract methods to implement (abstract class)
   c/ (later all auto-generated)
      include/
         qeq.h
      src/
         qeq_dispatch.c

   examples/
      qeq_c_solver/
         qeq_c_solver.conf  (which library to load and how to init)
         qeq.c
      qeq_py_solver/
         qeq_py_solver.conf  (which library to load and how to init)
         qeq_solver/
            __init__.py
            solver.py
      qeq_py_solver2/   # New implementation in Python
         qeq_py_solver2.conf  (which library to load and how to init)
         super_qeq_solver/
            __init__.py
            solver.py

Here there are three principal layers:

  • Language-specific bridge libraries that implement interface from bridge_api.h and marshall data to the types of the concrete language

  • Interfaces for the computational problems: abstract classes in Python and header files in C, etc.

  • Configuration files and actual implementations. Configuration files provide the means to the dispatch library to find the actual implementation.

Right now, C implementations are separate dynamic shared objects that are loaded by the C dispatch library. Alternative could be to build a single large shared library for all C implementations, however, it is debatable.

3.1.1.4. Installed structure

The structure is the following:

PREFIX/
share/
    oif_impl/
        qeq/
        qeq_c_solver.conf
        qeq_py_solver.conf
        qeq_py_solver2.conf
    lib/
        oif_impl/
            liboif_dispatch.so
            liboif_dispatch_python.so
            liboif_dispatch_c.so
            qeq/
                qeq_c_solver.so
                qeq_c_solver2.so
        python3.11/
            site-packages/
                oif_impl/
                    qeq_solver/
                        __init__.py
                        solver.py
                    super_qeq_solver/
                        __init__.py
                        solver.py

3.1.2. Configuration files

Configuration file might look like

PREFIX/lib/oif_impl/python_dispatch.so
super_qeq_solver.solver solve

where the first line specifies the dispatch library for the language of the requested implementation. Second line specifies information that must be passed to this language dispatch library so that it can do the actual function call.

For example, for Python it can be dot-separated path to the module and, whitespace separated, the function to invoke.

For C, it can be the path to the shared library to load and the name of the actual function to call.

3.1.3. Other technical details

The questions of how shared libraries will be discovered by the linker, how Python modules are discovered, are left now unspecified.

Temporary solution is to use LD_LIBRARY_PATH and PYTHONPATH environment variables during the early development stage.