How it works¶

The picture above displays the relationships among the software components of the JEODPP infrastructure that constitute the building blocks of the interactive library. Basically, there are three different environments where the interactive library software components are running:

the browser where users interact with Jupyter notebooks (upper part of the picture);

the Docker container that executes notebook Python code (left part of the picture);

the service Dockers that expose tile-based processing through HTTP calls (right part of the picture).

On the browser client, JavaScript and HTML code parts are running. A derived version of LeafLet JavaScript library is executed on the client site: its called IPyLeafLet and it is a porting of the LeafLet library to the Jupyter world. Its role is to provide a Map display where interactive zoom and pan functions are available. The Map mainly works with TileLayers, that is a layer that can be queried to a web service by using the so called SlippyMap.

The Docker container displayed on the left of the picture is a lightweight virtualization that operates server-side inside the JEODPP hardware infrastructure. It is a process that is activated for each user that connects to the Jupyter service. Its main role is to execute remotely the Python code that the user writes in the Jupyter notebook cells on the browser client. This component establishes a communication channel between the server and the client so that JavaScript code on the browser page and Python code on the Docker notebook container can exchange messages, request of code execution, results of execution, events, etc. On the notebook Docker container, an instance of the Interactive Library is running. The library is developed in C/C++ language and it is made available to the Python level through the automatic wrapping services of the SWIG software tool. In this way the Python code written by the users can call the inter module, create instances of classes like Map (Map), ImageCollection (ImageCollection), ImageProcess (ImageProcess), or VectorLayer (VectorLayer). Usually an instance on ImageCollection is created to filter the image dataset and the specific images to work with. An ImageProcess instance (using the method ImageCollection.process()) is then derived from the ImageCollection. The ImageProcess represents a processing chain made up of operators that can be applied to transform the original data in something useful for the specific analysis that the user wants to perform. Inside the notebook Docker container the processing chains are defined but not executed: this means that the processing chains requested by the user are only defined and described in a JSON string that is saved inside a PostGIS database. When the processing chain is added as a layer to the Map, the library passes to the IPyLeafLet instance a unique identifier of the processing chain. When the Map updates the display, it calls the web service to get the tiles by passing this unique identifier inside the HTTP requests.

The web service Dockers are executed in parallel inside many nodes of the JEODPP infrastructure. This ensures good performance even when many users are visualizing processed data on their browser. Every web service Docker runs an instance of Apache Gunicorn which is a WSGI HTTP server based on Python. The C/C++ interactive library is called by Python code to effectively execute the processing chains requested by the users: when a request for a tile from the Map is received by a gunicorn thread, the unique identifier passed in the HTTP request is used to retrieve the description of the processing chain from the PostGIS database. The content of the processing chain is interpreted and an instance of an ImageProcess object is created. It is this instance that creates the output tile by reading the source data from the EOS storage, applying in parallel all the operators inside the chain, packing the result in a PNG image that is sent back to the browser client. At this level, the Interactive library makes calls to JIPlib library and to GDAL to work on raster data and to Mapnik and OGR library to work on vector datasets.