Edinburgh Data-Intensive Research - progressPosts

RapidSeis: Rapid Portals for Seismological Waveform Data

Jano.van.Hemert — Thu, 19 Nov 2009 16:10:22 +0000

Final project report for JISC, with links to all individually created deliverables and progress posts. RapidSeis has produced a scientific gateway via a web portal that allows seismologist to pick up data from Orfeus—the central repository for earthquake data in Europe—and then run several analyses on these data. Advanced users can also create new analyses and share these with all the other users.

When an analysis is run, data is extracted from the earthquake data stores in Europe and transferred to the University of Liverpool where the analysis is performed on a computer cluster. Once an analysis finishes, its results in the form of waveforms is send to the portal for the user to view and download.

* Screenshots or diagram of prototype: http://research.nesc.ac.uk/node/493
* Link to working prototype: http://193.52.21.80/jetspeed/portal/ needs login, so screencast provided next http://research.nesc.ac.uk/node/492
* Link to end user documentation: http://research.nesc.ac.uk/files/RapidTutorial-20090908.pdf
* Link to code repository or API: http://forge.nesc.ac.uk/projects/jos/
* Link to technical documentation: http://forge.nesc.ac.uk/download.php/455/Manual-0.9.2.pdf
* Date prototype was launched: 2009/11/13
* Project Team Names, Emails and Organisations: Jano I. van Hemert, j.vanhemert@ed.ac.uk - University of Edinburgh; Jos Koetsier, jkoetsie@staffmail.ed.ac.uk - University of Edinburgh; Andy Heath, aeh@liverpool.ac.uk - University of Liverpool; Alessandro Spinuso, Alessandro.Spinuso@knmi.nl - Royal Netherlands Meteorological Institute; Andreas Rietbrock, a.rietbrock@liverpool.ac.uk - University of Liverpool; Torild van Eck, torild.van.eck@knmi.nl - Royal Netherlands Meteorological Institute
* Project Website: http://research.nesc.ac.uk/rapid/
* PIMS entry: https://pims.jisc.ac.uk/projects/view/1340
* Table of Content for Project Posts
- RapidSeis workflow explained http://research.nesc.ac.uk/node/493
- Screencast: Rapid Portals for Seismological Waveform Data http://research.nesc.ac.uk/node/492
- Tutorial for Rapid http://research.nesc.ac.uk/node/487
- RapidSeis Workshop http://research.nesc.ac.uk/node/485
- Rapid portals improve teaching in practical sessions http://research.nesc.ac.uk/node/483
- Apache's Virtual File system is part of Rapid's architecture http://research.nesc.ac.uk/node/481
- Unit-testing code is especially important in code-generating projects http://research.nesc.ac.uk/node/482
- Apache's Maven for management of code structure? http://research.nesc.ac.uk/node/480
- Rapid prevents a disaster in the classroom http://research.nesc.ac.uk/node/479
- Rapid Tutorial at Advanced Distributed Services Summer School http://research.nesc.ac.uk/node/465
- Happy end user of Rapid gives praise http://research.nesc.ac.uk/node/442

Topic of this submission:

JISC:

Research topics:

Data-intensive computing

Collaborative environments

Intuitive interfaces

Projects:

Rapid

RapidSeis workflow explained

Jano.van.Hemert — Thu, 19 Nov 2009 16:02:16 +0000

RapidSeis consists of two porlets that run within the NERIES portal. Both assume the user logged in has already filled his/her cart with seismological data.

The Editor Portlet allows to (1) save and load source code of analysis plugins for the Seismological Waveform eXplorer application. When a load is performed, the code is loaded from a server that holds the personal account, which in turn asks (2) the description of the plugin, which is stored on the computing infrastructure. Whenever the user compiles (3) code, this builds a binary of the plugin, which is stored on the computing infrastructure (where it is needed when execution is requested). Plugins can be made public and thus shared with everyone that has access to the portal.

The Execution Portlet allows users to load (4) plugins from the account server. For each plugin the user interface is generated dynamically in the browser. Once parameters are set and the user requests (5) execution, the computing infrastructure gets the data in the user's cart (6) and runs the plugin on these data. The results are stored (7) on a web server in the form of HTML pages that contain aligned waveforms. These are then loaded by the portlet to allow the user to inspect and download them.

Topic of this submission:

JISC:

Projects:

Screencast: Rapid Portals for Seismological Waveform Data

Jano.van.Hemert — Thu, 19 Nov 2009 15:44:33 +0000

Welcome to the RapidSeis demo. This portal was created using Rapid and enables the use of Seismic Data eXplorer within the web portal of the Network of Research Infrastructures for European Seismology.

Topic of this submission:

JISC:

Research topics:

Projects:

Tutorial for Rapid

Jano.van.Hemert — Wed, 18 Nov 2009 15:28:41 +0000

This brief tutorial covers the installation of Rapid and the components it depends on. It explains how to get a simple portlet running and then increases the complexity of this portlet to include more functionality by exploiting Rapid's features.

Topic of this submission:

JISC:

Projects:

Attachment	Size
RapidTutorial-20090908.pdf	290.14 KB

RapidSeis Workshop

Jano.van.Hemert — Sat, 31 Oct 2009 11:12:05 +0000

At the meeting we will present the result of the RapidSeis project, a collaboration between ORFEUS, the UK National e-Science Centre and the University of Liverpool. Over the past six months, this project has created a system that facilitates running waveform analysis on data from ORFEUS where the computation is performed on remote compute resources provided by the University of Liverpool. The system is used via a secure web portal, which means no software needs to be installed by end-users; they only need access to a common web browser. Even visualization will be embedded in the portal.

The portal enables two modes of operation. First, it allows registered users to choose an analysis from a catalogue of analyses and then run this analysis on data chosen from the data hosted by ORFEUS. Second, it enables more advanced users to create new analyses and add these to the catalogue for others to use.

November 12th
• 13:00 Welcome/Lunch in the staff common room, Department of Earth and Ocean Sciences.
• 14:00 - 14:30 Luca Trani
• 14:30 - 15:00 Robert Barsch
• 15:00 - 15:30 Marco Olivieri
15:30 - 16:00 Coffee break
• 16:00 -16:30 Jano van Hemert
• 16:30 - 17:00 Paul Denton
18:30 Meet in the Philharmonic Pub for drinks, followed by dinner at 20:00 at Gusto Restaurant at the Albert Dock: http://www.gustorestaurants.uk.com/restaurants/gusto-liverpool

November 13th
• 09:00 - 10:45 RapidSeis - Andy Heath, Jos Koetsier, Luca Trani
10:45 - 11.15 Coffee break
• 11:15 - 12:30 RapidSeis
12.30 Lunch
• 14:00 Wrap up
15:00 Finish

Attendees (from left to right in the picture): Moritz Beyreuther (LMU), Robert Barsch (LMU), Paul Denton (BGS), Jano van Hemert (NeSC), Andreas Rietbrock (Liverpool), Andy Heath (Liverpool),Marco Olivieri (ETHZ), Alessandro Spinuso (Orfeus - facsimile), Luca Trani (Orfeus), Alexandra Lodge (Liverpool), Jos Koetsier (NeSC), Linus Kamb (EMSC)

Location:

Department of Earth & Ocean Sciences, University of Liverpool, UK

Dates:

12 Nov 2009 to 13 Nov 2009

Web site:

http://pcwww.liv.ac.uk/~aeh/Meetings/rapidseisws.htm

Research topics:

Data-intensive computing

Projects:

JISC:

Rapid portals improve teaching in practical sessions

Jano.van.Hemert — Thu, 29 Oct 2009 11:28:59 +0000

Written by Nick Funnel (2nd year PhD in the School of Chemistry).

The lab has been much easier to teach this year.

Last year the students had to edit Z-matrices for geometry optimization calculations, which have to be located from separate files and navigate between a PC and a workstation. This also required them to deal with UNIX in order to complete the lab. The vast majority of students struggled, having never dealt with issues like this before, and a significant proportion of these students were unhappy just using computers in general.

As a result, most of the calculations that were submitted did not work—most likely due to an input error but these were difficult to spot. Eventually, the answers often had to be provided to the student from a calculation that been previously run. Sorting out these problems was very time consuming and so there was no time to effectively teach the students the chemical theory behind the experiment and most students saw it as a waste of time.

This year, most of these problems have been solved—students no longer have to manage lots of separate files or use UNIX. Although the students still have to edit Z-matrices, there are clear instructions accompanying them and there is more time to explain the format of them to the student.

So far there has been no problem running any of the calculations. Students are working from one webpage and by effectively ‘clicking next’ all the time they are given more time to think about the theory behind what they are doing rather than just trying to get the calculations to run.

Topic of this submission:

JISC:

Projects:

Apache's Virtual File system is part of Rapid's architecture

Jano.van.Hemert — Mon, 19 Oct 2009 15:21:00 +0000

Apache's Virtual File system helped us to access different file systems transparently. The Rapid project implemented a new 'GSIFTP' (Grid Security Infrastructure File Transfer Protocol) module, which adds Grid connectivity to the set of supported file systems. This sub-project of Rapid, spawned the commons-vfs-grid module (http://sourceforge.net/projects/commonsvfsgrid/) which now supports GSIFTP, SRB (Storage Resource Broker) and iRods (https://www.irods.org).

Topic of this submission:

JISC:

Projects:

Unit-testing code is especially important in code-generating projects

Jano.van.Hemert — Mon, 19 Oct 2009 15:20:21 +0000

The use of testing frameworks, such as Junit is extremely valuable. It takes quite a bit of discipline to write new unit tests for each new piece of 'real' code that is written, but the benefits in the end in terms of stability, confidence and debugging make the extra overhead more than worth it. This is especially useful in a project such as Rapid, where code is being generated. Testing in the 'traditional' way, by examining or running generated code is extremely difficult and by using testing frameworks we can actually determine correctness of Rapid, before running any generated code.

Topic of this submission:

JISC:

Projects:

Apache's Maven for management of code structure?

Jano.van.Hemert — Mon, 19 Oct 2009 15:20:16 +0000

Apache's Maven project management tool (http://www.apache.org) helped the Rapid team structure code modules in a 'standardised' way. It helps managing dependencies and facilitates distributing the final product as dependencies are automatically downloaded and do not need to be packaged separately. However, the learning curve is rather steep and even now we sometimes have to go back to ANT (http://ant.apache.org/) to accomplish certain tasks.

Topic of this submission:

JISC:

Projects:

Rapid prevents a disaster in the classroom

Jano.van.Hemert — Wed, 14 Oct 2009 10:34:04 +0000

For the first time, chemists at the University of Edinburgh are using a portal developed with Rapid to teach 140 undergraduates computational chemistry.

The first week of the semester disaster struck, the primary server with all the software died. A new server was brought in, but its configuration is different and that would have meant lecture notes had to be altered, instructers would have to be briefed and practical sessions would have to be rescheduled.

Now for the good news. The portal was adjusted, using Rapid, in a few hours to match the new configuration. As the instructors and students were used to the portal, nothing bad happened. They kept going, blissfully unaware of what happened.

This is not the primary selling point of our technology, but definitely something we will include in our next sales pitch.

Topic of this submission:

JISC:

Projects: