This is a static HTML version of an old Drupal site. The site is no longer maintained and could be deleted at any point. It is only here for historical interest.
Data-intensive computing
Systems and problems that include huge data volumes and complex patterns of integration and interaction.
The presentation was given to the 41st Workshop of the International School of Geophysics, Erice, Italy, "A Roadmap for Earth Science in Europe: The next generation of Geophysical Research Infrastructures". The meeting was organised by EPOS, ORFEUS, COOPEUS and NERA. I was supposed to talk about massive data processing strategies, but as you will see, I saw that many other frequently occurring impediments to success with data were more dominant, so I tried to draw attention to these issues.
Date and time:
Thursday, 29 August, 2013 - 10:30
Location:
‘Ettore Majorana’ Foundation and Centre for Scientific Culture, Erice, Sicily
Congratulations to Gary, who as his hard-won internship in CERN approaches its end, has won a prize in the lightening presentations given by the 20 interns at CERN yesterday. His talk "The Implementation of OpenStack Cinder and Integration with NetApp and Ceph" was one of the three talks awarded a prize: http://openlab.web.cern.ch/news/openlab-summer-students-present-their-pr...
Researchers often need to use workflows that have been developed by other experts in their field to handle specific parts of their work. Sooner or later they find that they want to use workflows from multiple sources that are written in different languages. Enacting multi-lingual workflows (or meta workflows) has been pioneered in a group of European projects. The next step is to be able to change them when they don’t do exactly what you want. But that is not easy if you need to learn a different editor for each workflow language.
Molecular simulations are indispensable methods in areas like material science, structural biology, and drug design. These methods address data-intensive and compute-intensive problems, which demand high-performance computing to allow data analysis in an acceptable time. The project MoSGrid (Molecular Simulation Grid) offers a workflow-enabled grid portal allowing access to molecular simulation tools on distributed resources in an intuitive manner. Users are able to exchange workflows and data via repositories and, thus, to exchange knowledge about the specific application domain.
Large-scale distributed workflow systems for science are nowadays expected to operate in a consistent, predictable way as well as to promote collaboration between researchers or within groups in a unified way. In this talk we will discuss the VERCE Information Registry, which is designed to provide a consistent view of the VERCE ecosystem for seismology along with related architectural requirements, assumptions and interactions with other components.
Date and time:
Thursday, 20 June, 2013 - 11:40
Location:
Open Science Data Cloud Workshop 2013, Edinburgh, UK
Typical of the digital revolution, seismology is changing rapidly: the number of deployed seismometers grows rapidly, their performance and connectivity improve and seismologists employ data from other sources, such as LIDAR, satellite images and GPS. But their computational behavior is also changing: they used to just focus on earthquakes, today the use all of the continuous waveform. Just what does this mean for the science and the computational infrastructure to support it? The VERCE project is pioneering this approach and will provide examples during the talk.
Date and time:
Wednesday, 19 June, 2013 - 11:00
Location:
Open Science Data Cloud Workshop 2013, Edinburgh, UK
One of the main objectives of the VERCE project (Virtual Earthquake and Seismology Research Community in Europe) is to provide scientists with a unified, Europe-wide, computing environment able to support data-intensive scientific computation. This talk will be mainly about our approach to designing and building this infrastructure. More specifically, I will present the current computing environment, the rationale for designing our solutions around the workflow paradigm as well as the basic components of the architecture and their interactions.
Date and time:
Monday, 10 June, 2013 - 14:00
Location:
CISA Seminar, School of Informatics, University of Edinburgh, UK
Diversity, in every dimension, is a key attribute of today’s data bonanza. Our research takes a holistic view, embracing this diversity and the consequent intricate interactions between users and systems. We created the Dispel data-streaming language to describe complex computation patterns at high levels of abstraction, while providing meta-information for optimisation. Provenance and contextual information must be harnessed to achieve autonomous execution, data placement, energy efficiency and reliability.
Modern seismologists are presented with increasing amounts of data that may help them better understand the Earth's structure and systems. However: 1) they have to access these data from globally distributed sites via different transfer protocols and security mechanisms; 2) to analyse these data they need to access remote powerful computing facilities; 3) their experiments result in yet more data that need to be shared with scientific communities around the world.
PICO presentation:
Across Europe there are a large number of rock deformation laboratories, each which runs many experiments. Similarly
there are a large number of theoretical rock physicists who develop constitutive and computational models
both for rock deformation and changes in geophysical properties. Here we consider how to open up opportunities
for sharing experimental data in a way that is integrated with multiple hypothesis testing. We present a prototype
One of the objectives of the VERCE project (Virtual Earthquake and Seismology Research Community in Europe – http://www.verce.eu/) is to provide scientists with a unified, Europe-wide, computing environment able to support
data-intensive scientific computation. The term “data-intensive” is used to characterise computation that either
requires or generates large volumes of data, or that its data access patterns are complex due to algorithmic or
We are delighted to welcome Sandra back, some of you may remember her previous visits from the University of Tübingen, where she recently defended her PhD. We all congratulate her on that well deserved success. Her email address remains, sandra.gesing@uni-tuebingen.de. She can be found in room IF5.22. We will be working together on a graphical editor for DISPEL that also forms a proof-of-concept for a really versatile web-based generic workflow editor.
Drs Andre Luckow and Shantenu Jha from Rutgers University
This will be a talk in two parts: In the first part, we will motivate via a series of applications the concept of a scalable and general-purpose Pilot-Job. We will discuss the P* Model of Pilot-Jobs and present BigJob a SAGA-based Pilot-Job. In the second part of the talk, we focus on extension of the Pilot-Job concept to the challenge of "Big Data". Science that involves and depends upon large amounts of data, also requires overcoming various challenges, including managing large-scale data distribution and co-placement/scheduling with computing resources.
