LPP » INSTRU » QLop » SciQLOP

h1. SciQlop Status

"Priorities : Google doc link":https://docs.google.com/spreadsheets/d/1fIQDHRDhKJtTxnRRDlmInM25QFDaEuZ8gctZ523m6wU/edit#gid=0

h2. Scientific Objectives and Performances

|_.Functionnality |_.Status(% done)|

|SciQLOP should be portable, and an executable/setup should be available for Linux, Mac, Windows|{background:orange;color:white}. 50|

|Installing SciQLOP must be easy, must not require to read a documentation or install dependencies|{background:orange;color:white}. 50|

|When opened SciQLOP should by default reload the previous state (data products, plots, plugins, etc.)|{background:red;color:white}. 0|

|SciQLOP comes with good user documentation, including galleries, examples and  tutorials including video tutorials.|{background:orange;color:white}. 50|

|SciQLOP should propose the same functionalities on all platforms.|{background:orange;color:white}. 50|

|SciQLOP should propose the same performances on all platforms.|{background:orange;color:white}. 50|

|SciQLOP's GUI should remain light, beautiful and intuitive|{background:orange;color:white}. 50|

|Exploring databases, browsing data should be easier/faster than with other existing softwares|{background:orange;color:white}. 90|

|SciQLOP should provide efficient and transparent data browsing, access to user python routines, collaborative cataloging features with and without machine learning|{background:red;color:white}. 0|

|SciQLOP should remain responsive when plotting 10 millions of points on a standard machine.|{background:orange;color:white}. 90|

|SciQLOP should have a limited list of data type it can handle and group them by common traits (scalars, vectors, image...).|{background:green;color:white}. 100|

|SciQLOP may be able to provide features depending on the data type, source, unit or any condition/property the user may define or program.|{background:orange;color:white}. 50|

|Each modification of the data between the source (server) and the plot should be documented|{background:orange;color:white}. ?|

|SciQLOP should be used by students as a formation tool : provide users with knowledge toolkits associated to used quantities|{background:orange;color:white}. ?|

|SciQLOP should provide users with easy access to documentations on data, missions, instruments, etc.|{background:orange;color:white}. 90|

|SciQLOP should allow users to access wikipedia plasma/mission-related articles and suggest users to edit/add content.|{background:red;color:white}. 0|

|SciQLOP should allow users to save and restore sessions.|{background:red;color:white}. 0|

|SciQLOP user session should contain all data to restore its previous state.|{background:red;color:white}. 0|

h2. Code redistribution

|_.Functionnality |_.Status(% done)|

|SciQLOP source code will be  GPLv2.|{background:green;color:white}. 100|

|SciQLOP source folder should contain the files:README, INSTALL, COPYING, CHANGELOG.|{background:red;color:white}. 0|

|All SciQLOP source files should contain GPLv2 header.|{background:green;color:white}. 100|

|All SciQLOP dependencies should be compatible with GPLv2.|{background:orange;color:white}. 90|

h2. Code versioning

|_.Functionnality |_.Status(% done)|

|SciQLOP source code modifications should have a link between features or bug corrections and code revisions|{background:orange;color:white}. 50|

|SciQLOP source code should be hosted on the laboratory server hephaistos1 with the mercurial version system.|{background:green;color:white}. 100|

h2.Code Writing

|_.Functionnality |_.Status(% done)|

|SciQLOP developer's documentation, roadmap, issues, etc. will be done on the Redmine application : https://hephaistos.lpp.polytechnique.fr/redmine/projects/sciqlop|{background:green;color:white}. 100|

|Unit test are developed for all modules and performed after each important merge|{background:orange;color:white}. 90|

|SciQLOP's code is homogeneous in its syntax and philosophy to guarantee easy maintainability.|{background:green;color:white}. 100|

|SciQLOP's code is self-explanatory, comments are used to explain goals, methods rather than instructions. Comments record development flow (hacks, todos, etc.)|{background:orange;color:white}. 95|

|SciQLOP's code is based on the Qt framework and plotting capabilities use the QtCharts API.|{background:green;color:white}. 100|

h2. SciQLop Modules

|_.Functionnality |_.Status(% done)|

|{background:grey;color:white}. Core      |{background:grey}. |

|{background:#ddd;color:grey}.Database     |{background:#ddd}. |

|All the native data types SciQLOP should be handled|{background:green;color:white}. 100|

|Database module should allow to associate commands/functions on data/data types from GUI.|{background:orange;color:white}. 50|

|Database module should provide a way to view loaded products.|{background:green;color:white}. 100|

|Database module GUI should show if a product is in use and by which module(Plot5, plugin2, PythonContext...).|{background:red;color:white}. 0|

|Database module GUI should show which plugin provide the data .|{background:red;color:white}. 0|

|Origin of the data should be associated with metadata. Ex: if product comes from a file, display the path and file metadata (dates, size, permissions etc.). If from datadownloader, display which one, i.e. amda, cdaweb etc.|{background:red;color:white}. 0|

|Data products are accessible through a tree dynamically built with a filter|{background:green;color:white}. 100|

|Users can easily access metadata on data products (mission, resolution, unit, etc.) by interacting with the tree|{background:orange;color:white}. 95|

|Default tree representations are proposed to the user - TBD|?|

|Caveat and other useful information usually present in headers are available from the database's GUI|{background:red;color:white}. 0|

|Database representation should highlight data products when selected on plots|{background:red;color:white}. 0|

|Any data imported into the user session, whether it is from a file, or from the Space data Module must be added to the database|{background:green;color:white}. 0|

|Garbage collector - TBD|?|

|Each data product in the database acquires a unique identifier|{background:green;color:white}. 100|

|Data products loaded from  into the database are not mutable.|{background:green;color:white}. 100|

|Users should be able to delete a product from the database through contextual menu||

|Deleting a product from the Database checks whether the product is still in use. If so warns the user that all plots will be updated accordingly|{background:orange;color:white}. 95 don't warn|

|Multiple occurences of the same dataproducts (i.e. database entry differing by their UID, and possibly time interval) are grouped under the same data product name in the tree and appear by their UID/plot|{background:red;color:white}. 0|

|Database also show the currently defined variables in the python session, in a separate tab. Although python variables, like data products, contain data, they differ by being mutable and not associated with rich metadata such as origin, etc|{background:red;color:white}. 0|

|Users can manually add a python function to the database in the context menu|{background:red;color:white}. 0|

|{background:#ddd;color:grey}.Data Downloader     |{background:#ddd}. |

|Data downloader should be a singleton.|{background:green;color:white}. 100|

|Data downloader may implement most protocols.(HTTP, FTP, WebDav...)|?|

|Data downloader should handle proxy server.|{background:green;color:white}. 100|

|Data downloader should know when data asked by the user is locally accessible already.|{background:red;color:white}. 0|

|{background:#ddd;color:grey}.Python Engine     |{background:#ddd}. |

|SciQLOP functionalities are scriptable through the ipython terminal. ||

|Users can interact with the database through python, he may be able to pull data from database to python or push data from python to database.|{background:red;color:white}. 0|

|Users can interact with any plot with python|{background:red;color:white}. 0|

|User can grab data from plot or plot data with python|{background:red;color:white}. 0|

|Data in the python terminal is implemented as nimpy array and DataFrames objects, with time as index|{background:red;color:white}. 0|

|{background:#ddd;color:grey}.GUI Manager    |{background:#ddd}. |

|GUI manager should be a singleton|{background:red;color:white}. 0|

|GUI manager may provide an interface to populate menus from any module|{background:red;color:white}. 0|

|{background:#ddd;color:grey}.Catalogs & Community functionalities      |{background:#ddd}. |

|{background:grey;color:white}.Plot      |{background:grey}. |

|{background:#ddd;color:grey}.General      |{background:#ddd}. |

|SciQLOP should be able to plot all kinds of data relevant to space plasma missions(scalar, vector, tensor, spectrogram||

|Each plot element (context : colorbars, ticks, titles, labels etc. ; data : lines, style, dots, etc.) should be clickable and customizable||

|Controls on figure items should be identical in all kinds of plots (e.g. changing title, labels etc. is done identically for all kinds of plots).||

|A plot “style” (context and data style) can be copy/pasted onto another plot ||

|Users may define custom plot styles and save/reload them as desired||

|Plots should provide visualization of available metadata (instrument modes, etc.)||

|Each plot/view should provide the context menu associated to the data type.||

|Each plot/view should allow the user to select a portion of data and perform specific operations on it, such as labelling it, building new plots or applying data analysis methods on it (e.g. get statistical quantities such as mean/std, etc.).||

|Data can be selected by dragging a zone onto the plot: Time interval as rectangle for time series for example. A box in 3d plot/view for example.||

|For time dependent data the plot/view may ask for more data when the user scrolls to the borders of the plot/view. ||

|Users should experience no significant lag related to downloading data associated with scrolling windows||

|Each plot/view should allow the integration of custom controls (user defined gui widgets) provided by the current plotted data. The control may be provided as a QWidget.||

|Each plot/view should be easily duplicable.||

|Figures/plots can be linked together or not.|{background:green;color:white}. 100|

|Linked plots will zoom/unzoom, scroll together (horizontally)|{background:green;color:white}. 100|

|A Lens effect enables the user to zoom on a part of the data without changing the plot range (like a lens effect on Gimp-like software). This can be made by adding a plot window overlapped onto the plot.||

|Any plot should come with a default legend, customizable by the user||

|Any plot can be dragged to a folder and exported as an image (eps, png, jpg)||

|User can select multiple plots holding shift key and drag them onto a folder to export images. SciQLOP ask the user whether multiple images or a summary plot should be done.||

|Mouse wheel can be used to scroll in time when the figure is selected.||

|Plot adjustments controls appear on specific locations on mouse hover. This keeps the interface clear but still customizable.||

|{background:#ddd;color:grey}.Scalar Time series      |{background:#ddd}. |

|Contextual menu should give users rapid access to statistical functionalities such as mean(), std().||

|Data holes are handled in line style mode as a line segment.||

|Line width can be adjusting by clicking+mouse wheel||

|{background:#ddd;color:grey}.Vector Time series      |{background:#ddd}. |

|SciQLOP can plot one or more components of a vector as a function of time||

|SciQLOP is aware the quantity is part of a vector. If Bx is plotted it knows which products represent By and Bz and can easily add them on the plot.||

|The contextual menu on a vector plot should allow users to easily transform their vector into another basis, user-provided and generic basis such as GSE, GSM, etc. or MVA.||

|Vectors displayed on linked plots will be represented in the same basis dynamically (one change of basis on one of the linked plot will change all of them)||

|Vectors can be plotted in 3D or 2D-projection as 3D vectors as a function of time. This representation will be useful for visualizing aspects specific to vectors such as change of direction (e.g. waves & discontinuities) not easily viewable in the time series format.||

|{background:#ddd;color:grey}.Spectrograms      |{background:#ddd}. |

|SciQLOP should enable the same plot/context functionalities for all spectrograms, no matter the nature of the data (particles or waves), this include color range, colormaps, etc.||

|Characteristic frequencies should be overplottable as time series.||

|Using WAMP for theoretical wave frequencies and damping should be possible||

|Plasma eigenmodes visualization toolkit can be called from the spectrograms contextual menu||

|Data holes are handled by showing the background color/texture||

|Colormap by clicking on the colorbar, color range can easily be adjusted from cursors appearing on the colorbar on mouse hover||

|{background:#ddd;color:grey}.Orbits    |{background:#ddd}. |

|SciQLOP provides the user with three 2D projections of the magnetosphere/heliosphere and its key regions||

|Heliosphere/Magnetospheric plot use different models (Tsyganenko, shue, Parker Spiral, etc.) easily interchangeable by the user.||

|Orbits of the satellites from which data is currently being viewed appears overplotted on the three 2D magnetospheric/heliosphere projections||

|The position of the spacecraft for the considered intervals must be clearly visible on the trajectory with a colored portion of the orbit, the color being associated to a particular plot (or set of linked plots).||

|Changing the interval range of a spacecraft on its orbit must change the plot time range accordingly and dynamically.||

|SciQLOP should offer an interoperability with CDPP/3DView to view orbits in 3D||

|{background:grey;color:white}.SpaceData      |{background:grey}. |

|{background:#ddd;color:grey}.General    |{background:#ddd}. |

|Satellites Data provider should provide an access to CDAWeb data over REST protocol.||

|Satellites Data provider may allow advanced pattern filtering for fast data retrieval||

|Satellites Data provider should allow generic search/filtering among data fields.||

|Satellites Data provider search/filter should accept regular expressions||

|{background:grey;color:white}.App      |{background:grey}. |

|{background:grey;color:white}.Machine Learning     |{background:grey}. |