XtremWeb-HEP : middleware for distributed data processing

This software was developed (or is under development) within the higher education and research community. Its stability can vary (see fields below) and its working state is not guaranteed.

Web site: http://www.xtremweb-hep.org/spip.php?rubrique13
System:
Current version: 7.6.4 - 12/12/2011
License(s): GPL
Status:
Support: maintained, ongoing development
Designer(s): Oleg LODYGENSKY
Contact designer(s): xtremweb (at) lal.in2p3.fr
Laboratory, service:

Higher Edu - Research dev card

Creation or important update: 03/01/12
Minor correction: 03/01/12
Index card author: Etienne Urbah (LAL IN2P3)
Theme leader: Geneviève Romier (Institut des Grilles et du Cloud)

Keywords

General software features

’XtremWeb-HEP’ is a middleware for Distributed Data Processing (grids) :
– It permits Administrators to :
- manage various Users and Applications, by providing adequate access rights to them,
- catalog various Data and Computing Resources :
· PC farms managed by an IT department,
· PC grids contributed by volunteer citizens,
– It permits Users to submit Jobs referencing these Applications,
– From Job descriptions, it dynamically deploys and executes these Applications on available Computing Resources, then it provides the results to authorized Users,
– It protects the Computing Resources running Mac OS X by starting the Application inside MAC OS X Sandbox,
– For the access to data, it permits the usage of HTTP, HTTPS, and any URI scheme whose driver is provided by the User.

Secured three tiers Architecture. Scheduler and data repository managed by a software administrator on a server; Client installed on the machine of each User (for ex. scientist); Worker installed on the resource of each contributor.

Soon will come in production the version of XtremWeb-HEP additionally managing the submission of complete virtual machines for execution inside VirtualBox.
Interoperability with other grid middleware stacks :
– XtremWeb-HEP accepts X509 certificates and proxies for user management, in particular those of the DEGISCO international project.
– XtremWeb-HEP integrates a bridge permitting suitable XtremWeb-HEP jobs to be accepted by the gLite middleware in order to be executed by the EGI European infrastructure.
– On the other way, thanks to the 3G Bridge of the EDGI European project, the resources gathered by XtremWeb-HEP are available for the many users of the EGI infrastructure (gLite, ARC and Unicore middleware stacks).
Domain, Infrastructures, Documentation and Maintenance :
– In spite of its name, XtremWeb-HEP is used way beyond High Energy Physics : Biology, ADN Research, Mathematics, Physics of Solids, Signal Processing.
– XtremWeb-HEP is powering at least 2 production grids (For each grid, look at the 'Statistics' page) :
- http://www.xtremweb-hep.org/lal/xw_lal/
- http://xw.lri.fr:4330/XWHEP
– XtremWeb-HEP has a complete up to date set of user manuals, presented at http://www.xtremweb-hep.org/spip.php?rubrique16
– XtremWeb-HEP is maintained by the software team presented at http://www.xtremweb-hep.org/spip.php?rubrique35 and is strongly supported by Institut des Grilles et du Cloud, INRIA, ENS Lyon, GRID5000

Context in which the software is used

Distributed Data Processing
Distributed Computing
Resource Sharing
Computing Grid (PC Grid)
Job Submission

Publications related to software

Hybrid Distributed Computing Infrastructure Experiments in Grid5000 : Supporting QoS in Desktop Grids with Cloud Resources http://users.lal.in2p3.fr/lodygens/gc/g5k.pdf
G. Fedak, S. Delamare, O. Lodygensky. Grid 5000 School, Reims, France - April 18-21, 2011
Extending the EGEE grid with XtremWeb-HEP Desktop Grid http://users.lal.in2p3.fr/lodygens/gc/PCGrid2010.pdf
H. He, G. Fedak, P. Kacsuk, Z. Farkas, Z. Balaton, O. Lodygensky, E. Urbah, G. Caillat, F. Aurajo, A. Emmen. 4th Workshop on Desktop Grids and Volunteer Computing Systems, Melbourne, Australia - May 17-20, 2010
EDGeS : Bridging EGEE to BOINC and XtremWeb http://users.lal.in2p3.fr/lodygens/gc/EDGeS-Bridgi...
E. Urbah, P. Kacsuk, Z. Farkas, G. Fedak, G. Kecskemeti, O. Lodygensky, A. Marosi, Z. Balaton, G. Caillat, G. Gombas, A. Kornafeld, J. Kovacs, H. He, and R. Lovas. JoGC Journal of Grid Computing, Volume 7, Number 3, 2009.