Grid Data Access

How to implement my Grid data access system?

Access to data on the Grid is offered by services. These services allows the user to browse directories and list files on remote Grid systems. Currently g-Eclipse has implementation for:

gLite middleware:
- GridFTP services
- SRM 2.0 services
- LFC service
GRIA middleware:
- Gria data service
Amazon Web Service:
- S3 service
Others:
- Sftp

UI Components

The Connection Wizard allows to define new remote connection.

Important Extension points

Core extension points

org.eclipse.core.filesystem.filesystems: This extension point provides implementation of the Eclipse File System (EFS) API. Developer needs to specify file system's URI scheme tag and file system class extending org.eclipse.core.filesystem.provider.FileSystem (see below).
eu.geclipse.core.filesystem.transferService: Extension point for defining new transfer service for specified data services. If there exists service-specific transfer type, then it should be implemented using this extension point.

UI extension point

eu.geclipse.ui.efs: This extension point is used to split URI of the connection into tokens( host, port, path etc. ) important for given connection. This description is then used in g-Eclipse new connection wizard to present the user all requested and optional fields when creating new connection.

Interfaces to implement

org.eclipse.core.filesystem.provider.FileSystem: Since g-Eclipse file system mechanism bases on EFS, to implement Grid access you need to extend FileSystem abstract class.
org.eclipse.core.filesystem.provider.FileStore: Abstract Eclipse class you should extend when implementing new Grid data access service. File store represents single file in target file system.
eu.geclipse.core.model.ITransferService: If you decide to provide your file system with dedicated transfer service, you will need to create a transfer service class implementing this interface.

Sample implementations

As stated in the beginning g-Eclipse already contains several implementations of Grid data access services. They can be used as examples for future developers of different file system in g-Eclipse. Below you can find references to each implementation of org.eclipse.core.filesystem.provider.FileStore and, if it exists, to dedicated transfer service for given file system.

GridFTP service - implementation of basic grid access service for gLite, similar to basic File Transfer Protocol. Uses globus proxy to authenticate. Also gridFTP dedicated transfer was created, which speeds up transferring of data between two gridFTP connections.
- eu.geclipse.efs.gridftp.GridFile - gridFTP file store
- eu.geclipse.efs.gridftp.TransferParty - gridFTP file transfer
SRM service - Storage Resource Management (SRM) implementation for gLite middleware. g-Eclipse currently supports version 2 of the SRM services.
- eu.geclipse.efs.srm.SRMStore - SRM file store
LFC service - implementation of LCG (Large Hadron Collidor Computing Grid) File Catalog (LFC) for gLite middleware. This file system uses information about replicas (see eu.geclipse.efs.lgp.IReplicableFileSystem and eu.geclipse.efs.lgp.IReplicableStore) to show available resources to the user. User can manage his replicas using g-Eclipse.
- eu.geclipse.efs.lgp.LFCStore - LFC file store
GRIA data service - implementation of the GRIA data service for GRIA middleware. GRIA data service consists of several data stagers, which can contain a blob of data or be empty. You can manage permissions for your data stagers.
- eu.geclipse.efs.gria.GriaStore - GRIA file store
S3 data service - access for Amazon Web Service data services.
- eu.geclipse.aws.s3.internal.fileSystem.S3FileStore
Sftp service - SSH File Transfer Protocol implementation. Allows to access remote data using reliable data stream.
- eu.geclipse.efs.sftp.SFTPFileStore - Sftp file store