Latest news
Email
Usage
The EGEE traces reported in this portal are produced by the gLite middleware.
The repository will be updated by extension, mainly weekly.
To download traces, go to the query page, and fill the registration form.
(Note: if you are retrieving big files - up to multiple GB in size - please be patient as the system may take time to present them to you for download. You only need to click once).
If you plan to exploit these traces for publications, please have a look at the Terms of Use.
Overview of gLite
EGEE combines globally-distributed computational and storage resources into a single production infrastructure available to EGEE users. Each participating site configures, runs, and maintains a batch system containing its computational resources and makes those resources available to the grid via a gatekeeper.
The gLite middleware deployed on the EGEE infrastructure integrates the sites' computing resources through the Workload Management System (WMS). The WMS is a set of middleware-level services responsible for the distribution and management of jobs. The site computational resources present a common interface to the WMS, the Computing Element (CE) service.
The core of the WMS is the Workload Manager which accepts jobs from users and dispatches them to computational resources based on the users requirements on one hand, and the characteristics (hardware, software, localization) and state of the resources on the other hand. The WM is implemented as a distributed set of resource brokers, with some tens of them currently installed; all the brokers get an approximately consistent view of the resource availability through the grid information system.
The Logging and Bookkeeping service (LB) tracks jobs in terms of event(important points of job life, e.g. submission, finding a matching CE, starting execution etc.) gathered from various WMS components as well as CEs.
For more information, consult:
E. Laure et al, Programming the grid with gLite
the gLite web site
The traces
There are two categories of traces. EGEE-wide traces, provided by the Real-Time Monitor project, are a summary of the lifecycle of jobs. All jobs submitted through gLite are reported. The other traces have been collected at the GRIF/LAL site of EGEE. Depending on their nature, they are more or less exhaustive.
The Information System
The Information System provides detailed information to the other services about the static and dynamic status of the grid infrastructure and services. The content of the information system is the Glue information model (Glue 1.3, and OSG Glue WG.), which is the current basis for interoperability between the EGEE grid and other grids. The information system is conceptually unique, even if its implementation is distributed. Thus the traces cover the whole EGEE infrastructure.
The Logging and Bookeeping
The LB logs most of the events in a jobs lifecycle, as provided by the various services of gLite. The service is job-centric: any event is assigned to a unique Grid job. Upon creation each job is assigned a unique, virtually non-recyclable job identifier (JobId) in an URL form, which is used in all logged events. The events are processed to give a higher level view on the job states (e.g. Submitted, Running and Done when the jobs starts and stops execution, or Transfer from a WMS component to another one ), and records various attributes (e.g. submission file in JDL language , destination CE name, job exit code, etc.). The logging process is highly reliable, within the limits of an asynchronous implementation. Each service is associated to a particular resource broker; thus, each log covers those jobs that were managed by this particular broker.
The Real Time Monitor (RTM) traces
They include the timestamps of the major events in a job lifecycle. Thus, they are a summary of the L&B data. The RTM data cover all the EGEE jobs managed by the gLite middleware.
The batch system traces
They report on the job events related to the batch system of a site. Scheduling and memory consumption are recorded. The Torque logs cover a batch system, which is often associated to a site.
The internal logs
They cover the following gLite services: wmproxy, CondorG, jobcontroller, logmonitor, workloadManager. They describe the fine details of the WMS.