Edinburgh Data-Intensive Research - Internship student project

Connecting Rapid with the jclouds multi-cloud framework

Jano.van.Hemert — Mon, 22 Mar 2010 15:17:14 +0000

Principal goal: to extend Rapid, a tool for developing web portals for scientific computing, to operate with jclouds.

This is project is part of the Google Summer of Code 2010 (see http://www.omii.ac.uk/wiki/RapidJclouds)

Rapid is a unique way of quickly designing and delivering web portal interfaces to applications that require computational resources, such as utility computing infrastructures or high-performance computing facilities. It focuses on the requirements of the end-user by designing customised user interfaces for domain-specific applications that allow users to achieve particular tasks from the comfort of their own web browser.
Project Goals

Currently, Rapid can submit to several job submission engines, such as Sun Grid Engine, Condor and PBS. In this project you will build a bridge between Rapid and jclouds. The aim is to enable Rapid to submit compute jobs to the different cloud frameworks supported by jclouds. Then evaluate your solution by submitting a compute job to Amazon's Elastic Cloud (EC2) and to the experimental cloud resource set up at the University of Edinburgh that uses Eucalyptus.

Project Description

Currently, Rapid works with Grid and High-Performance Computing infrastructure. It is your task to adapt Rapid so as it can be used to generate intuitive interfaces that submit jobs to several cloud infrastructures. Examples of these infrastructures are Amazon's Elastic Cloud 2, Eucalyptus, Rackspace, Linode and GoGrid. Preferably you will look into existing solutions that can handle several of these infrastructures at ones via a standard library, such as for example libcloud.

Cloud provides several advantages over other distributed computing approaches such as Grid and high-performance computing. However, it also brings several problems, such as expensive data movement and the potential of wasting resources if virtual machines run idle. In this project you extend Rapid's set of modules by integrating with jcloud so that we can start submitting to several cloud vendors.

Project Requirements
* Learn how Rapid is used.
* Learn how Rapid works internally, especially its modules to communicate with various computing infrastructures
* Plan and design how to add one or more modules for jcloud
* Write the code with documentation
* Transfer the code to us

Supervisors @ NeSC:

Jano.van.Hemert

Jos.Koetsier

Project status:

Finished

Degree level:

Background:

Java, XML

Subject areas:

e-Science

Projects:

Rapid

Student project type:

Internship student project

Extension of Rapid to the Hadoop Framework

Jano.van.Hemert — Mon, 22 Mar 2010 15:12:42 +0000

Student:

Harika Yasa

Principal goal: to extend Rapid, a tool for developing web portals for scientific computing, to operate with Apache Hadoop.

This is project is part of the Google Summer of Code 2010 (see http://www.omii.ac.uk/wiki/RapidHadoop)

To add a module to Rapid, a technology to quickly create web portal interfaces to execute applications on remote compute resources, that allows it to communicate with the Apache Hadoop framework. Currently, Rapid can submit to several job submission engines, such as Sun Grid Engine, Condor and PBS. You will extend Rapid with code that will facilitate the submission to, monitoring of and handling of data with the Hadoop framework.

Project Description

Cloud provides several advantages over other distributed computing approaches such as Grid and high-performance computing. However, it also brings several problems, such as expensive data movement and the potential of wasting resources if virtual machines run idle. In this project you investigate solutions that involve Apache Hadoop to better organise the computation so as to make efficient use of compute resource.

Project Requirements
* Learn how Rapid is used.
* Learn how Rapid works internally, especially its modules to communicate with various computing infrastructures
* Plan and design how to add a module for Apache Hadoop
* Write the code with documentation
* Transfer the code to us

Supervisors @ NeSC:

Jano.van.Hemert

Jos.Koetsier

Project status:

Finished

Degree level:

Background:

Java, XML

Subject areas:

e-Science

Projects:

Rapid

Student project type:

Internship student project

Google Summer of Code 2009: Generating User Interfaces to the Cloud

Jano.van.Hemert — Sat, 21 Mar 2009 21:21:20 +0000

Student:

Raviteja Dodda

You will install an Open Source Cloud solution, such as EUCALYPTUS or Hadoop. Then, after experimenting a bit with Rapid, you will add at least one more job submission and monitoring engine to Rapid that connects to such a cloud infrastructure.

This project is available through the participation of OMII-UK in the Google Summer of Code 2009 programme. To be accepted as a student for the project depends on the supervisors (here called mentors), OMII-UK and Google. Apply here http://socghop.appspot.com/

Deadline: 19:00 UTC on April 3rd

Supervisors @ NeSC:

Jano.van.Hemert

Jos.Koetsier

Project status:

Finished

Degree level:

Background:

Knowledge of Java is required. A bit of experience with XML is useful.

Subject areas:

e-Science

WWW Tools and Programming

Projects:

Rapid

Student project type:

Internship student project

References:

Rapid: http://research.nesc.ac.uk/rapid/ Google Summer of Code 2009: http://socghop.appspot.com/ OMII-UK: http://www.omii-uk.ac.uk/ EUCALYPTUS: http://eucalyptus.cs.ucsb.edu/ Hadoop: http://hadoop.apache.org/