What is MOTEUR
MOTEUR is a workflow management system developed by the Modalis Team
, in particular Tristan Glatard
and Johan Montagnat
For more information check:
For advanced details:
What are Workflows
In simple words, workflows are pipelines where several components (programs) are combined to perform a given analysis task. Workflows can be very simple, composed of only one program, or very large and complex (for example, such as a workflow for analysis of functional MRI data
Workflow technology offers several facilities that are attractive to implement applications on grids. For example, with MOTEUR independent tasks are automatically performed in parallel and the complexity of grid job submission is hidden from the programmer. The Generic Application Service Wrapper (GASW
) is used to automatically wrap non-interactive command line linux applications (=programs) to run them as jobs on the grid.
For more info please read/follow this tutorial
Executing workflows on the e-BioInfra
Using workflows to implement and execute applications on the e-BioInfra involves the following steps (see Figure 1
- describe a workflow in the Simple Conceptual Unified Flow Language (SCUFL). This description is stored in a file using XML. It specifies all workflow inputs (parameters and files); workflow outputs (results), the workflow components (programs to be executed), and how they are connected to each other.
- specify the workflow inputs using the GUI of a special plugin developed for the VBrowser ( MOTEUR plugin).
- start the workflow from the plugin, which then contacts MOTEUR.This requires a valid grid proxy.
- MOTEUR executes the workflow on the grid using the gLite middleware.
- The grid jobs run on some grid node and store results on grid storage (LFC).
- The user monitors the progress of workflows via HTML pages and of jobs using the Job Monitoring plugin
- The workflow is complete when all boxes show in blue or in red.
Figure 1 - Sequence diagram for workflow execution on e-BioInfra with MOTEUR
For more detailed information about workflow execution, generated files and monitoring of individual jobs, read the MOTEUR advanced page
MOTEUR Plugin for the VBrowser
The VBrowser MOTEUR plugin provides a GUI to specify inputs for a workflow and run it on the grid with MOTEUR
. If the VBrowser is correcty configured, this plugin is automatically started when the user left-clicks on a SCUFL document (suffix .scufl).
plug-in is distributed with the VBrowser and needs to be configured by the user after installation as explained in the VBrowser page
Workflow and job monitoring pages
Figure 2 - Workflow (left) and job (right) monitoring pages generated by MOTEUR
MOTEUR maintains monitoring information about all executed workflows available the web server of the e-BioInfra server.
illustrates the monitoring page for the complete workflow. The boxes indicate workflow component corresponding to programs that are executed as grid jobs. The color of the boxes indicate status:
- grey: component not active yet
- green: jobs are running (or have been at least submitted)
- red: all jobs completed, some with error
- blue: all jobs completed ok
NOTE: If the same box needs to be executed several times for different input values, MOTEUR will start individual jobs for each one of them and show the box in blue only when all the jobs are completed.
Clicking on the boxes will open the job monitoring page (figure 2-right
The job monitoring page is composed of three sections:
- Summary: simplified view of all jobs
- Waiting: Jobs that have yet to start running.
- Running: Jobs that are currently executing.
- Successfully completed: Jobs that have completed without error.
- Failed: Jobs that have either failed to run, or have run and failed completing.
- Job graph: all possible states of a job. Each bar shows how many jobs are in that particular state.
- Job details for each job that has been submitted by MOTEUR for this workflow: the current state, the time it reached this state, and the id of the job. NOTE: job ids can be used with gLite commands to directly obtain more information about the job.