Edinburgh Data-Intensive Research - Success story

Congratulations to Gary McGilvary on his PhD

Malcolm.Atkinson — Wed, 10 Sep 2014 07:56:08 +0000

We should all congratulate Gary on completing his corrections and on managing remote production of his thesis, "Ad hoc Cloud Computing", while in California, interning at Apple. It demonstrates the remarkable power of the Internet to arrange action at a distance! His thesis is now available here and should be read.

Gary's research, "Ad hoc Cloud Computing" is a much more remarkable and sustained demonstration showing how the unused power of an organisation's computers or altruistically offered computers can be harnessed effectively and made much more usable. He explored every step needed to make it easy to set up Volunteer Clouds, i.e., how to provide and manage a consistent and reliable Cloud running over unreliable, heterogeneous volunteer computers. He devised a technically elegant and practical approach to every stage, building on BOINC. He measured these carefully to demonstrate their power and efficiency.

But you don't want to hear about it from me, you should read his thesis, which I am glad to report is on this site, http://research.nesc.ac.uk/node/994. Please download it and have a good read; I enjoyed it every time I read it! His examiners primary concern was that more people should read and understand it, as they thought it was excellent research that should be widely applied.

Topic of this submission:

Success story

Research topics:

New conceptual models for systems

Surfing for earthquakes

Jano.van.Hemert — Fri, 20 Aug 2010 18:33:50 +0000

A better understanding of the ground beneath our feet will result from research by seismologists and Rapid—a group of computer scientists at the University of Edinburgh. The Earth's structure controls how earthquakes travel and the damage they can cause. A clear picture of this structure would be extremely valuable to earthquake planners, but it requires the analysis of huge amounts of data. The Rapid team developed a system that performs the seismologists' data-crunching, and have made it easy to use by relying on an interface familiar to all scientists – a web browser.

Seismologists measure vibrations in the Earth at hundreds of observatories across Europe which allows them to study earthquakes as they travel across countries and continents. By measuring the speed and strength of the vibrations at different sites, deductions can be made about the type of ground they have travelled through. From this information, the structure of the Earth can be constructed. The problem with earthquakes is that they don't occur when and where you need them.

Earthquakes aren't the only things that cause vibrations: road traffic, waves pounding on the beach and even wind and thunder can cause detectable vibrations. These vibrations – known as noise – may lack the strength of earthquakes, but they compensate by being available in huge numbers. If enough noise is analysed, it is possible to build up information about the Earth's structure. The analysis is not without problems "You can use noise to analyse the Earth's structure, but you need to analyse huge amounts of data and that's nearly impossible on standard [computers]" explained Andreas Rietbrock, who helped develop the new system with the Rapid team and is Professor of Seismology at the University of Liverpool.

The Orfeus foundation collects seismic data from around Europe and makes it available for analysis through websites like the Earthquake Data Portal (www.seismicportal.eu). Only a few organisations have the resources and technical know-how needed to process this vast store of data. Orfeus asked the Rapid team to develop a system that would allow any seismologist to analyse seismic data using powerful computers located around Europe. "We don't want [seismologists] to have to study how to access [remote] computer power and data" said Torild van Eck, Secretary General of Orfeus "Rapid is, for us, a tool to hide the tricky part of getting, steering and manipulating data".

The Rapid team have developed a reputation for helping scientists use their data, and have worked with everyone from chemists to medics and biologists to engineers. "It's been great working with the seismologists, because as a community they're very open to trying out new ways of working. And they have really pushed the boundaries of our technology" said Jano van Hemert, leader of the Rapid team. For Orfeus, the team developed a web portal. This takes all the complex computing needed for seismic analysis and hides it behind a standard web browser. By presenting all of the analysis tools in such a familiar environment, any seismologist—even the most technophobic ones—can use the system. One of the first applications for the Rapid web portal is to allow seismologists to study noise for the analysis of the Earth's structure. Rapid will build on this work with help from a grant from the UK's Natural Environment Research Council, which has provided funding to explore whether it is possible to predict earthquakes and volcanic eruptions.

The Rapid web portal allows even the smallest seismology groups to perform the kind of analysis that was previously limited to organisations that could afford their own supercomputers. By making this analysis easy, Rapid and Orfeus have brought complex research programmes into the hands of many more seismologists. More seismologists working together means that results are produced faster, and that means we could soon benefit from a better understanding of the ground beneath our feet.

(source: http://www.eurekalert.org/pub_releases/2010-08/uoe-sfe082010.php)

Topic of this submission:

Success story

Research topics:

Data-intensive computing

Collaborative environments

Intuitive interfaces

Projects:

Rapid

A 2-minute video introduction to Rapid

Jano.van.Hemert — Sat, 12 Dec 2009 12:55:09 +0000

A video where the Rapid-team introduces their technology to develop web portals for computational science.

(source: http://gridtalk-project.blogspot.com/2009/12/rapid-demo-at-all-handsieee...)

Topic of this submission:

Success story

Research topics:

Intuitive interfaces

Projects:

Rapid

Rapid portals improve teaching in practical sessions

Jano.van.Hemert — Thu, 29 Oct 2009 11:28:59 +0000

Written by Nick Funnel (2nd year PhD in the School of Chemistry).

The lab has been much easier to teach this year.

Last year the students had to edit Z-matrices for geometry optimization calculations, which have to be located from separate files and navigate between a PC and a workstation. This also required them to deal with UNIX in order to complete the lab. The vast majority of students struggled, having never dealt with issues like this before, and a significant proportion of these students were unhappy just using computers in general.

As a result, most of the calculations that were submitted did not work—most likely due to an input error but these were difficult to spot. Eventually, the answers often had to be provided to the student from a calculation that been previously run. Sorting out these problems was very time consuming and so there was no time to effectively teach the students the chemical theory behind the experiment and most students saw it as a waste of time.

This year, most of these problems have been solved—students no longer have to manage lots of separate files or use UNIX. Although the students still have to edit Z-matrices, there are clear instructions accompanying them and there is more time to explain the format of them to the student.

So far there has been no problem running any of the calculations. Students are working from one webpage and by effectively ‘clicking next’ all the time they are given more time to think about the theory behind what they are doing rather than just trying to get the calculations to run.

Topic of this submission:

JISC:

Projects:

Rapid prevents a disaster in the classroom

Jano.van.Hemert — Wed, 14 Oct 2009 10:34:04 +0000

For the first time, chemists at the University of Edinburgh are using a portal developed with Rapid to teach 140 undergraduates computational chemistry.

The first week of the semester disaster struck, the primary server with all the software died. A new server was brought in, but its configuration is different and that would have meant lecture notes had to be altered, instructers would have to be briefed and practical sessions would have to be rescheduled.

Now for the good news. The portal was adjusted, using Rapid, in a few hours to match the new configuration. As the instructors and students were used to the portal, nothing bad happened. They kept going, blissfully unaware of what happened.

This is not the primary selling point of our technology, but definitely something we will include in our next sales pitch.

Topic of this submission:

JISC:

Projects:

Rapid portlets are a hit with chemists

Jano.van.Hemert — Wed, 05 Aug 2009 14:10:34 +0000

Portlets make inaccessible technology accessible, because they run from within a browser—a familiar interface for even the most technophobic researcher. To encourage the use of portlets, it is necessary for them to be easy to develop. This led OMII-UK to fund Rapid, an easy-to-use portlet development tool.

Chemists from the Universities of Edinburgh and St. Andrews (EaStCHEM) have recently used the software to create portlets that have allowed access to computational- chemistry software by over 140 students.

EaStCHEM use computational- chemistry software, such as Gaussian 03, for research and teaching activities. However, technological barriers prevented many chemists from using the software. To overcome these barriers, the Rapid project showed EaStCHEM how to create portlets with Rapid. This training led to one chemist creating four portlets without the help of a software developer.

One of the four portlets is now used to teach over 140 chemistry students. This user-friendly portlet hides the complexity of the tutorial (such as, command line options, authentication protocols and job submission commands) from the student, so that they can concentrate on learning the software. Importantly, the only complexity that isn’t hidden, is the code needed to learn how to drive the computational-chemistry software. Learning this code is a skill that the students will need throughout their careers as chemists.

Portlets do not just make applications more user friendly. Future work with EaStCHEM will make more resources available to the chemists, such as those supplied by the National Grid Service. This will overcome difficulties in securing time on the resources available within the universities and will harness more computational power for the chemists.

Elsewhere, Rapid is being used in a wide range of areas, including brain imaging, seismology, and fire safety engineering.

Rapid is being used to design, build and deliver a portal to help brain imaging experts perform tasks using tools developed by the SFC Brain Imaging Research Centre.

Likewise, the Rapid Portals for Seismological Waveform Data project aims to encourage the seismology community to use an application that analyses seismic waveform data.

Potential users tend to be put off by using the waveform application because they are daunted by the prospect of installing and even understanding the application. They also tend to worry about the difficulty in transporting large amounts of data needed for analysis, and how to visualise the results.

Self-contained web portlets will be directly embedded in the community gateway and link to the data available in the Orfeus Data Center; the primary European centre for this kind of data. All the data and computing will be managed through Rapid, and the results presented via an existing web portal.

The development tool is also being used in the Firegrid project at the University of Edinburgh. FireGrid aims to establish a cross-disciplinary collaborative community to pursue fundamental research for developing real time emergency response systems, using the Grid, beginning with fire emergencies.

By Simon Hettrick and Gillian Law (taken from NeSC Newsletter, August 2009)

Topic of this submission:

Success story

Projects:

Rapid