Batch and Cloud overview. Andrew McNab University of Manchester GridPP and LHCb
|
|
- Madeleine Hoover
- 8 years ago
- Views:
Transcription
1 Batch and Cloud overview Andrew McNab University of Manchester GridPP and LHCb
2 Overview Assumptions Batch systems The Grid Pilot Frameworks DIRAC Virtual Machines Vac Vcycle Tier-2 Evolution Containers Event servers 2
3 Basic assumptions You have a series of pieces of computation that need doing From more than one person in your collaboration? With different priorities? With different requirements? You want to execute these pieces of computation on remote resources Remote not the machine you edit files on Accessing remote resources means you can use what is available May allow you to scale your computation up by orders of magnitude These pieces of computation need to read or write data Where they run? Somewhere remote from that too? Piece of computation sounds like a job, but not necessarily 3
4 Batch systems Traditionally all this has been done with batch queuing systems Long and noble history going back to first mainframes, when operators physically loaded people s stacks of punched cards into readers Many systems descended from or influenced by NASA NQS have the command qsub to submit a file that defines the job to be run. Simplest way of running remote batch jobs is to do something like: scp job.sh me@somewhere.ac.uk: ; ssh me@somewhere.ac.uk qsub job.sh In the 1990s CERN built a system called SHIFT that did just that If you have access to one big system somewhere else, you may be doing that yourself today However, it s very very unscalable, as it relies on setting up ssh access knowing that somewhere.ac.uk has resources you can run on 4
5 The Grid (Here The Grid is the infrastructure of European Grid Infrastructure (EGI) and Worldwide LHC Computing Grid (WLCG)) The Grid gives access to remote batch systems in a much more scalable fashion Uses X.509 and VOMS certificates for identifying users and services and for virtual organization (VO) membership An information system to discover suitable resources Lots of operational support (tickets, mailing lists, monitoring) Makes it possible to automate the submission of jobs Originally via a Broker or Workload Management System Now via the client submitting the jobs directly (That sounds like a backward step but it s not) 5
6 The Grid CREAM or ARC CE & batch queues Underlies the infrastructure run by EGI/WLCG Job Grid Site Originally proposed by EU DataGrid in 2000 WMS Broker (deprecated) Central agents & services Job factory User and production jobs 6
7 The Grid with Pilot Jobs The Grid + pilot jobs is the dominant model for running HEP jobs. Well established, and gives access to resources around the world. CREAM or ARC CE & batch queues Pilot Job. Runs Job Agent to fetch from TQ Pilot jobs Grid Site This late binding model allows the virtual organization to decide priorities itself. It sidelines the batch system s scheduling of time slots: each type of pilot job are all the same irrespective of the contents. Requests for real jobs Central agents & services Director (pilot factory) Matcher & Task Queue User and production jobs 7
8 Pilot frameworks Pilot job concept was introduced by LHCb s DIRAC project in 2003 DIRAC now spun-off as a separate project and used by other experiments, and provided by infrastructures like EGI and GridPP GridPP DIRAC service is intended to support experiments and VOs without their own pilot framework ATLAS and CMS also use pilot frameworks and have done some work on using ATLAS s BigPanda Pilot framework becomes a higher level batch system Can be used to hide the details of the underlying batch system, or even whether jobs are running inside VMs at cloud sites The virtual organization can monitor and manage all of the resources available to it centrally 8
9 Virtual grid sites Cloud systems like OpenStack allow virtualization of fabric What s new is that they expose (defacto?) standard APIs for the creation and management of virtual machines by external users Bringing up a machine can be delegated towards user communities Rather than using PXE+Kickstart+Puppet on bare metal Zeroth order virtualization is just to provide a Grid with Pilot Jobs site, on VMs Potential to use staff more efficiently: one big pool of hardware and hypervisors managed all together rather than separate clusters However, can take this a step further and have the experiment managing the cloud resources as a virtual grid site 9
10 Virtual Grid with Pilot Jobs site Gatekeeper (ARC? Condor?) Cloud Site Batch VM. Runs Job Agent to fetch from TQ Experiment creates its own conventional grid site on the cloud resources. Transparent to existing central services, and user/production job submitters. (Or via VM factory) CloudScheduler (ATLAS) and elastiq (ALICE) are implementations of this model. Requests for real jobs Central agents & services VM factory Pilot factory Matcher & Task Queue User and production jobs 10
11 Experiment creates VMs directly? Cloud Site VM. Runs Job Agent to fetch from TQ Experiment creates VMs instead of pilot jobs. Job Agent or pilot client runs as normal inside. CMS glidein WMS works this way for cloud sites: looks at demand and creates VMs to join Condor pool Requests for real jobs Central agents & services VM factory Matcher & Task Queue User and production jobs 11
12 Further simplification: Vacuum model Following the CHEP 2013 paper: The Vacuum model can be defined as a scenario in which virtual machines are created and contextualized for experiments by the resource provider itself. The contextualization procedures are supplied in advance by the experiments and launch clients within the virtual machines to obtain work from the experiments' central queue of tasks. ( Running jobs in the vacuum, A McNab et al 2014 J. Phys.: Conf. Ser ) Loosely coupled, late binding approach in the spirit of pilot frameworks For the experiments, VMs appear by spontaneous production in the vacuum Like virtual particles in the physical vacuum: they appear, potentially interact, and then disappear Three implementations so far: Vac, Vcycle, HTCondor Vacuum 12
13 Vac - the first Vacuum system Pilot VM. Runs Job Agent to fetch from TQ Vacuum site Since we have the pilot framework, we can do something really simple Strip the system right down and have each physical host at the site create the VMs itself. Requests for real jobs Instead of being created by the experiments, the virtual machines appear spontaneously out of the vacuum at sites. Use same VMs as on Cloud sites Central agents & services Matcher & Task Queue User and production jobs 13
14 Vcycle Site Vcycle VM factory Cloud Site Pilot VM. Runs Job Agent to fetch from TQ Apply Vac ideas to Clouds Vcycle implements an external VM factory that manages VMs. Can be run centrally by experiment or by site itself or by a third party Third party Vcycle VM factory Requests for real jobs VMs started and monitored by Vcycle, but not managed in detail ( black boxes ) Central agents & services Vcycle VM factory No direct communication between Vcycle and task queue Matcher & Task Queue User and production jobs 14
15 Pilot VMs Vac, Vcycle, HTCondor Vacuum assume the VMs have a defined lifecycle Need a boot image and user_data file with contextualisation Provided by experiment centrally from an HTTPS web server Virtual disks and boot media defined and VM started The VM runs and its state is monitored VM executes shutdown -h when finished or if no more work available Maybe also update a heartbeat file and so stalled or overruning VMs are killed shutdown_message file can be used to say why the VM shut down Experiments VMs are a lot simpler for the site to handle than running worker nodes LHC experiments and GridPP use CernVM-FS wide-area filesystem to provide VMs with operating system files and experiment software 15
16 Some implications of virtualization Decouples operating system versioning requirements of experiments and sites Jobs see an identical environment across sites Simplifies many aspects of security model (escalation much harder) Many opportunities for other simplifications (eg Vacuum) For the sites, VMs make WLCG systems seem a lot more normal Especially if unusual middleware is only inside the VMs When the Grid started, several sites had problems (or in fact failed) trying to run shared clusters with local IT services eg shared PBS cluster for HEP and other sciences, with same WN configuration for everyone Virtualization gives us a second chance to try this where appropriate 16
17 GridPP s Tier-2 Evolution project In the plan for GridPP5, we proposed radically simplifying what s needed to run a university grid site (a Tier-2 site ) using VMs We re working through all the requirements to be able to run purely VM-based sites Getting VMs signed off by experiments for all Tier-2 workloads Making sure monitoring, accounting, ticketing etc all support VM-based sites and the software we re using to manage them (Also trying to simplify the storage side) This is partly in response to reduced staff funding But it also dramatically lowers the effort required to create and run a Tier-2 style site to run jobs in GridPP DIRAC VMs and the other VM flavours we support 17
18 Containers Linux containers can provide an alternative (often a drop-in alternative) to most of the places I ve talked about VMs Can use containers to provide a protected virtual environment of libraries, files etc to payload jobs Container as VM So we should talk about Logical Machines to be general Other more lightweight models are possible, more focussed on the environment seen by particular processes or sets of processes CernVM base-image we re using it being adapted to work as a Docker container Again using CernVM-FS to get OS and experiment files Support for this will be added to Vac 18
19 Micro jobs and event servers So far we ve talked about jobs lasting hours or days Push model of early Grid became the pull model of Pilot Jobs But much finer grained pull models also exist Notably the ATLAS experiment s Event Service Still have something that looks like a job that runs for hours or days But now takes smaller packages of work from a central service Every few minutes? Makes it much easier to solve the Masonry Problem of packing work into the available time and space provided by a fixed lifetime job But the cost is potentially a lot more network connections and opportunities bottlenecks 19
20 Summary Broad trends towards automation and decoupling jobs from details of the sites they run at Automation gives opportunities for accessing a lot more resources As easy to run at 300 sites once you can automatically pick 1 of 3 But running at 300 sites is more complicated when things go wrong Pilot job frameworks abstract the specifics of how the job got there GridPP DIRAC service intended for experiments and VOs without their own pilot framework A lot of activity in using virtualized and now containerized resources GridPP aims to enable sites to run purely using VMs if they choose Event servers and micro jobs offer new opportunities to make more efficient use of job slots 20
21 Extra slides 21
22 The Masonry Problem... 22
23 The Masonry Problem Maximum length job, starting just before advance notice given Max length Max length job Wasted resources Max length job Short job Short job Max length Shorter job Max length job Short job Start of advance notice Hard deadline for jobs to finish 23
24 OpenStack cloud site architecture 24
Running real jobs in virtual machines. Andrew McNab University of Manchester
Running real jobs in virtual machines Andrew McNab University of Manchester Overview The Grid we have now Strategies for starting VMs Fabric / Cloud / Vac Hypervisors Disk images Contextualization Pilot
More informationCHEP 2013. Cloud Bursting with glideinwms Means to satisfy ever increasing computing needs for Scientific Workflows
CHEP 2013 Cloud Bursting with glideinwms Means to satisfy ever increasing computing needs for Scientific Workflows by I. Sfiligoi 1, P. Mhashilkar 2, A. Tiradani 2, B. Holzman 2, K. Larson 2 and M. Rynge
More informationHTCondor within the European Grid & in the Cloud
HTCondor within the European Grid & in the Cloud Andrew Lahiff STFC Rutherford Appleton Laboratory HEPiX 2015 Spring Workshop, Oxford The Grid Introduction Computing element requirements Job submission
More informationCloud Accounting. Laurence Field IT/SDC 22/05/2014
Cloud Accounting Laurence Field IT/SDC 22/05/2014 Helix Nebula Pathfinder project Development and exploitation Cloud Computing Infrastructure Divided into supply and demand Three flagship applications
More informationContext-aware cloud computing for HEP
Department of Physics and Astronomy, University of Victoria, Victoria, British Columbia, Canada V8W 2Y2 E-mail: rsobie@uvic.ca The use of cloud computing is increasing in the field of high-energy physics
More informationStatus and Evolution of ATLAS Workload Management System PanDA
Status and Evolution of ATLAS Workload Management System PanDA Univ. of Texas at Arlington GRID 2012, Dubna Outline Overview PanDA design PanDA performance Recent Improvements Future Plans Why PanDA The
More informationWM Technical Evolution Group Report
WM Technical Evolution Group Report Davide Salomoni, INFN for the WM TEG February 7, 2012 The WM TEG, organization Mailing list, wlcg-teg-workload-mgmt@cern.ch 48 people currently subscribed; representation
More informationCernVM Online and Cloud Gateway a uniform interface for CernVM contextualization and deployment
CernVM Online and Cloud Gateway a uniform interface for CernVM contextualization and deployment George Lestaris - Ioannis Charalampidis D. Berzano, J. Blomer, P. Buncic, G. Ganis and R. Meusel PH-SFT /
More informationIntegration of Virtualized Workernodes in Batch Queueing Systems The ViBatch Concept
Integration of Virtualized Workernodes in Batch Queueing Systems, Dr. Armin Scheurer, Oliver Oberst, Prof. Günter Quast INSTITUT FÜR EXPERIMENTELLE KERNPHYSIK FAKULTÄT FÜR PHYSIK KIT University of the
More informationThe Virtualization Practice
The Virtualization Practice White Paper: Managing Applications in Docker Containers Bernd Harzog Analyst Virtualization and Cloud Performance Management October 2014 Abstract Docker has captured the attention
More informationHTCondor at the RAL Tier-1
HTCondor at the RAL Tier-1 Andrew Lahiff, Alastair Dewhurst, John Kelly, Ian Collier, James Adams STFC Rutherford Appleton Laboratory HTCondor Week 2014 Outline Overview of HTCondor at RAL Monitoring Multi-core
More informationClusters in the Cloud
Clusters in the Cloud Dr. Paul Coddington, Deputy Director Dr. Shunde Zhang, Compu:ng Specialist eresearch SA October 2014 Use Cases Make the cloud easier to use for compute jobs Par:cularly for users
More informationSingle Sign-In User Centered Computing for High Energy Physics
Single Sign-In User Centered Computing for High Energy Physics Max Fischer, Oliver Oberst, Günter Quast, Marian Zvada ISGC 2013: March 19-22, 2013 Taipei, Taiwan INSTITUT FÜR EXPERIMENTELLE KERNPHYSIK
More informationPoS(EGICF12-EMITC2)110
User-centric monitoring of the analysis and production activities within the ATLAS and CMS Virtual Organisations using the Experiment Dashboard system Julia Andreeva E-mail: Julia.Andreeva@cern.ch Mattia
More informationDynamic Resource Provisioning with HTCondor in the Cloud
Dynamic Resource Provisioning with HTCondor in the Cloud Ryan Taylor Frank Berghaus 1 Overview Review of Condor + Cloud Scheduler system Condor job slot configuration Dynamic slot creation Automatic slot
More informationWork Environment. David Tur HPC Expert. HPC Users Training September, 18th 2015
Work Environment David Tur HPC Expert HPC Users Training September, 18th 2015 1. Atlas Cluster: Accessing and using resources 2. Software Overview 3. Job Scheduler 1. Accessing Resources DIPC technicians
More informationPoS(EGICF12-EMITC2)005
, Or: How One HEP Experiment Is Evaluating Strategies to Incorporate The Cloud into the Existing Grid Infrastructure Daniel Colin van der Ster 1 E-mail: daniel.colin.vanderster@cern.ch Fernando Harald
More informationRyu SDN Framework What weʼ ve learned Where weʼ ll go
Ryu SDN Framework What weʼ ve learned Where weʼ ll go FUJITA Tomonori NTT Software Innovation Center Ryu Project lead 2014.11.14 NTT Ryu team goal Change the networking industry by Open Source Software
More informationA Web-based Portal to Access and Manage WNoDeS Virtualized Cloud Resources
A Web-based Portal to Access and Manage WNoDeS Virtualized Cloud Resources Davide Salomoni 1, Daniele Andreotti 1, Luca Cestari 2, Guido Potena 2, Peter Solagna 3 1 INFN-CNAF, Bologna, Italy 2 University
More informationMonitoring the Grid at local, national, and global levels
Home Search Collections Journals About Contact us My IOPscience Monitoring the Grid at local, national, and global levels This content has been downloaded from IOPscience. Please scroll down to see the
More informationThe Definitive Guide To Docker Containers
The Definitive Guide To Docker Containers EXECUTIVE SUMMARY THE DEFINITIVE GUIDE TO DOCKER CONTAINERS Executive Summary We are in a new technology age software is dramatically changing. The era of off
More informationCloud Computing Architecture with OpenNebula HPC Cloud Use Cases
NASA Ames NASA Advanced Supercomputing (NAS) Division California, May 24th, 2012 Cloud Computing Architecture with OpenNebula HPC Cloud Use Cases Ignacio M. Llorente Project Director OpenNebula Project.
More informationDynamic Extension of a Virtualized Cluster by using Cloud Resources CHEP 2012
Dynamic Extension of a Virtualized Cluster by using Cloud Resources CHEP 2012 Thomas Hauth,, Günter Quast IEKP KIT University of the State of Baden-Wuerttemberg and National Research Center of the Helmholtz
More informationBuilding a Volunteer Cloud
Building a Volunteer Cloud Ben Segal, Predrag Buncic, David Garcia Quintas / CERN Daniel Lombrana Gonzalez / University of Extremadura Artem Harutyunyan / Yerevan Physics Institute Jarno Rantala / Tampere
More informationManaging a tier-2 computer centre with a private cloud infrastructure
Managing a tier-2 computer centre with a private cloud infrastructure Stefano Bagnasco 1, Dario Berzano 1,2,3, Riccardo Brunetti 1,4, Stefano Lusso 1 and Sara Vallero 1,2 1 Istituto Nazionale di Fisica
More informationATLAS job monitoring in the Dashboard Framework
ATLAS job monitoring in the Dashboard Framework J Andreeva 1, S Campana 1, E Karavakis 1, L Kokoszkiewicz 1, P Saiz 1, L Sargsyan 2, J Schovancova 3, D Tuckett 1 on behalf of the ATLAS Collaboration 1
More informationINTRODUCTION TO CLOUD MANAGEMENT
CONFIGURING AND MANAGING A PRIVATE CLOUD WITH ORACLE ENTERPRISE MANAGER 12C Kai Yu, Dell Inc. INTRODUCTION TO CLOUD MANAGEMENT Oracle cloud supports several types of resource service models: Infrastructure
More informationVirtualisation Cloud Computing at the RAL Tier 1. Ian Collier STFC RAL Tier 1 HEPiX, Bologna, 18 th April 2013
Virtualisation Cloud Computing at the RAL Tier 1 Ian Collier STFC RAL Tier 1 HEPiX, Bologna, 18 th April 2013 Virtualisation @ RAL Context at RAL Hyper-V Services Platform Scientific Computing Department
More informationCERN local High Availability solutions and experiences. Thorsten Kleinwort CERN IT/FIO WLCG Tier 2 workshop CERN 16.06.2006
CERN local High Availability solutions and experiences Thorsten Kleinwort CERN IT/FIO WLCG Tier 2 workshop CERN 16.06.2006 1 Introduction Different h/w used for GRID services Various techniques & First
More informationHEP Data-Intensive Distributed Cloud Computing System Requirements Specification Document
HEP Data-Intensive Distributed Cloud Computing System Requirements Specification Document CANARIE NEP-101 Project University of Victoria HEP Computing Group December 18, 2013 Version 1.0 1 Revision History
More informationCMS Experience Provisioning Cloud Resources with GlideinWMS. Anthony Tiradani HTCondor Week 2015 20 May 2015
CMS Experience Provisioning Cloud Resources with GlideinWMS Anthony Tiradani Week 2015 20 May 2015 glideinwms Quick Facts glideinwms is an open- source Fermilab CompuJng Sector product driven by CMS Heavy
More informationOASIS: a data and software distribution service for Open Science Grid
OASIS: a data and software distribution service for Open Science Grid B. Bockelman 1, J. Caballero Bejar 2, J. De Stefano 2, J. Hover 2, R. Quick 3, S. Teige 3 1 University of Nebraska-Lincoln, Lincoln,
More informationVirtualization, Grid, Cloud: Integration Paths for Scientific Computing
Virtualization, Grid, Cloud: Integration Paths for Scientific Computing Or, where and how will my efficient large-scale computing applications be executed? D. Salomoni, INFN Tier-1 Computing Manager Davide.Salomoni@cnaf.infn.it
More informationSolution for private cloud computing
The CC1 system Solution for private cloud computing 1 Outline What is CC1? Features Technical details Use cases By scientist By HEP experiment System requirements and installation How to get it? 2 What
More informationCloud services for the Fermilab scientific stakeholders
Cloud services for the Fermilab scientific stakeholders S Timm 1, G Garzoglio 1, P Mhashilkar 1*, J Boyd 1, G Bernabeu 1, N Sharma 1, N Peregonow 1, H Kim 1, S Noh 2,, S Palur 3, and I Raicu 3 1 Scientific
More informationVirtualization. (and cloud computing at CERN)
Virtualization (and cloud computing at CERN) Ulrich Schwickerath Special thanks: Sebastien Goasguen Belmiro Moreira, Ewan Roche, Romain Wartel See also related presentations: CloudViews2010 conference,
More informationThe Evolution of Cloud Computing in ATLAS
The Evolution of Cloud Computing in ATLAS Ryan Taylor on behalf of the ATLAS collaboration CHEP 2015 Evolution of Cloud Computing in ATLAS 1 Outline Cloud Usage and IaaS Resource Management Software Services
More informationInfrastructure as a Service
Infrastructure as a Service Jose Castro Leon CERN IT/OIS Cloud Computing On-Demand Self-Service Scalability and Efficiency Resource Pooling Rapid elasticity 2 Infrastructure as a Service Objectives 90%
More informationYAN, Tian. On behalf of distributed computing group. Institute of High Energy Physics (IHEP), CAS, China. CHEP-2015, Apr. 13-17th, OIST, Okinawa
YAN, Tian On behalf of distributed computing group Institute of High Energy Physics (IHEP), CAS, China CHEP-2015, Apr. 13-17th, OIST, Okinawa Distributed computing for BESIII Other experiments wish to
More informationInternational Symposium on Grid Computing 2009 April 23th, Academia Sinica, Taipei, Taiwan
International Symposium on Grid Computing 2009 April 23th, Academia Sinica, Taipei, Taiwan New resource provision paradigms for Grid Infrastructures: Virtualization and Cloud Ruben Santiago Montero Distributed
More informationKVM, OpenStack, and the Open Cloud
KVM, OpenStack, and the Open Cloud Adam Jollans, IBM Southern California Linux Expo February 2015 1 Agenda A Brief History of VirtualizaJon KVM Architecture OpenStack Architecture KVM and OpenStack Case
More informationCloud Computing PES. (and virtualization at CERN) Cloud Computing. GridKa School 2011, Karlsruhe. Disclaimer: largely personal view of things
PES Cloud Computing Cloud Computing (and virtualization at CERN) Ulrich Schwickerath et al With special thanks to the many contributors to this presentation! GridKa School 2011, Karlsruhe CERN IT Department
More informationExperiences with the GLUE information schema in the LCG/EGEE production Grid
Experiences with the GLUE information schema in the LCG/EGEE production Grid Stephen Burke, Sergio Andreozzi and Laurence Field CHEP07, Victoria, Canada www.eu-egee.org EGEE and glite are registered trademarks
More informationLinux A first-class citizen in Windows Azure. Bruno Terkaly bterkaly@microsoft.com Principal Software Engineer Mobile/Cloud/Startup/Enterprise
Linux A first-class citizen in Windows Azure Bruno Terkaly bterkaly@microsoft.com Principal Software Engineer Mobile/Cloud/Startup/Enterprise 1 First, I am software developer (C/C++, ASM, C#, Java, Node.js,
More informationIntegration of Virtualized Worker Nodes in Standard-Batch-Systems CHEP 2009 Prague Oliver Oberst
Integration of Virtualized Worker Nodes in Standard-Batch-Systems CHEP 2009 Prague Oliver Oberst Outline General Description of Virtualization / Virtualization Solutions Shared HPC Infrastructure Virtualization
More informationNew resource provision paradigms for Grid Infrastructures: Virtualization and Cloud
CISCO NerdLunch Series November 7, 2008 San Jose, CA New resource provision paradigms for Grid Infrastructures: Virtualization and Cloud Ruben Santiago Montero Distributed Systems Architecture Research
More informationAdding IaaS Clouds to the ATLAS Computing Grid
Adding IaaS Clouds to the ATLAS Computing Grid Ashok Agarwal, Frank Berghaus, Andre Charbonneau, Mike Chester, Asoka de Silva, Ian Gable, Joanna Huang, Colin Leavett-Brown, Michael Paterson, Randall Sobie,
More informationPotential of Virtualization Technology for Long-term Data Preservation
Potential of Virtualization Technology for Long-term Data Preservation J Blomer on behalf of the CernVM Team jblomer@cern.ch CERN PH-SFT 1 / 12 Introduction Potential of Virtualization Technology Preserve
More informationKVM, OpenStack, and the Open Cloud
KVM, OpenStack, and the Open Cloud Adam Jollans, IBM & Mike Kadera, Intel CloudOpen Europe - October 13, 2014 13Oct14 Open VirtualizaGon Alliance 1 Agenda A Brief History of VirtualizaGon KVM Architecture
More informationLSKA 2010 Survey Report Job Scheduler
LSKA 2010 Survey Report Job Scheduler Graduate Institute of Communication Engineering {r98942067, r98942112}@ntu.edu.tw March 31, 2010 1. Motivation Recently, the computing becomes much more complex. However,
More informationE-mail: guido.negri@cern.ch, shank@bu.edu, dario.barberis@cern.ch, kors.bos@cern.ch, alexei.klimentov@cern.ch, massimo.lamanna@cern.
*a, J. Shank b, D. Barberis c, K. Bos d, A. Klimentov e and M. Lamanna a a CERN Switzerland b Boston University c Università & INFN Genova d NIKHEF Amsterdam e BNL Brookhaven National Laboratories E-mail:
More informationSistemi Operativi e Reti. Cloud Computing
1 Sistemi Operativi e Reti Cloud Computing Facoltà di Scienze Matematiche Fisiche e Naturali Corso di Laurea Magistrale in Informatica Osvaldo Gervasi ogervasi@computer.org 2 Introduction Technologies
More informationAn Efficient Use of Virtualization in Grid/Cloud Environments. Supervised by: Elisa Heymann Miquel A. Senar
An Efficient Use of Virtualization in Grid/Cloud Environments. Arindam Choudhury Supervised by: Elisa Heymann Miquel A. Senar Index Introduction Motivation Objective State of Art Proposed Solution Experimentations
More informationInfrastructure Clouds for Science and Education: Platform Tools
Infrastructure Clouds for Science and Education: Platform Tools Kate Keahey, Renato J. Figueiredo, John Bresnahan, Mike Wilde, David LaBissoniere Argonne National Laboratory Computation Institute, University
More informationClodoaldo Barrera Chief Technical Strategist IBM System Storage. Making a successful transition to Software Defined Storage
Clodoaldo Barrera Chief Technical Strategist IBM System Storage Making a successful transition to Software Defined Storage Open Server Summit Santa Clara Nov 2014 Data at the core of everything Data is
More informationPlug-and-play Virtual Appliance Clusters Running Hadoop. Dr. Renato Figueiredo ACIS Lab - University of Florida
Plug-and-play Virtual Appliance Clusters Running Hadoop Dr. Renato Figueiredo ACIS Lab - University of Florida Advanced Computing and Information Systems laboratory Introduction You have so far learned
More informationNT1: An example for future EISCAT_3D data centre and archiving?
March 10, 2015 1 NT1: An example for future EISCAT_3D data centre and archiving? John White NeIC xx March 10, 2015 2 Introduction High Energy Physics and Computing Worldwide LHC Computing Grid Nordic Tier
More informationEvolution of Database Replication Technologies for WLCG
Home Search Collections Journals About Contact us My IOPscience Evolution of Database Replication Technologies for WLCG This content has been downloaded from IOPscience. Please scroll down to see the full
More informationBuild & Manage Clouds with Red Hat Cloud Infrastructure Products. TONI WILLBERG Solution Architect Red Hat toni@redhat.com
Build & Manage Clouds with Red Hat Cloud Infrastructure Products TONI WILLBERG Solution Architect Red Hat toni@redhat.com AGENDA Cloud Concepts Market Overview Evolution to Cloud Workloads Evolution to
More informationConstructing your Private Cloud Strategies for IT Administrators & Decision Makers
Constructing your Private Cloud Strategies for IT Administrators & Decision Makers Greg Shields Senior Partner and Principal Technologist, Concentrated Technology, LLC http://concentratedtech.com Virtualization
More informationDocker : devops, shared registries, HPC and emerging use cases. François Moreews & Olivier Sallou
Docker : devops, shared registries, HPC and emerging use cases François Moreews & Olivier Sallou Presentation Docker is an open-source engine to easily create lightweight, portable, self-sufficient containers
More informationData Centers and Cloud Computing. Data Centers. MGHPCC Data Center. Inside a Data Center
Data Centers and Cloud Computing Intro. to Data centers Virtualization Basics Intro. to Cloud Computing Data Centers Large server and storage farms 1000s of servers Many TBs or PBs of data Used by Enterprises
More informationDeploying a distributed data storage system on the UK National Grid Service using federated SRB
Deploying a distributed data storage system on the UK National Grid Service using federated SRB Manandhar A.S., Kleese K., Berrisford P., Brown G.D. CCLRC e-science Center Abstract As Grid enabled applications
More informationHigh Performance Computing OpenStack Options. September 22, 2015
High Performance Computing OpenStack PRESENTATION TITLE GOES HERE Options September 22, 2015 Today s Presenters Glyn Bowden, SNIA Cloud Storage Initiative Board HP Helion Professional Services Alex McDonald,
More information<Insert Picture Here> Private Cloud with Fusion Middleware
Private Cloud with Fusion Middleware Duško Vukmanović Principal Sales Consultant, Oracle dusko.vukmanovic@oracle.com The following is intended to outline our general product direction.
More informationMobile Cloud Computing T-110.5121 Open Source IaaS
Mobile Cloud Computing T-110.5121 Open Source IaaS Tommi Mäkelä, Otaniemi Evolution Mainframe Centralized computation and storage, thin clients Dedicated hardware, software, experienced staff High capital
More informationIntegrating a heterogeneous and shared Linux cluster into grids
Integrating a heterogeneous and shared Linux cluster into grids 1,2 1 1,2 1 V. Büge, U. Felzmann, C. Jung, U. Kerzel, 1 1 1 M. Kreps, G. Quast, A. Vest 1 2 DPG Frühjahrstagung March 28 31, 2006 Dortmund
More informationRunning Oracle Databases in a z Systems Cloud environment
Running Oracle Databases in a z Systems Cloud environment Sam Amsavelu samvelu@us.ibm.com ISV & Channels Technical Sales - Oracle IBM Advanced Technical Skills (ATS), America Technical University/Symposia
More informationPrivate Cloud Management
Private Cloud Management Speaker Systems Engineer Unified Data Center & Cloud Team Germany Juni 2016 Agenda Cisco Enterprise Cloud Suite Two Speeds of Applications DevOps Starting Point into PaaS Cloud
More informationExperience with Server Self Service Center (S3C)
Experience with Server Self Service Center (S3C) Juraj Sucik, Sebastian Bukowiec IT Department, CERN, CH-1211 Genève 23, Switzerland E-mail: juraj.sucik@cern.ch, sebastian.bukowiec@cern.ch Abstract. CERN
More informationTechnology Insight Series
Evaluating Storage Technologies for Virtual Server Environments Russ Fellows June, 2010 Technology Insight Series Evaluator Group Copyright 2010 Evaluator Group, Inc. All rights reserved Executive Summary
More informationHow To Use Openstack At Cern
IPv6 on OpenStack Feature Parity is a Tricky Question Today s Sequence Quick Review of OpenStack Is OpenStack IPv6 Ready? Case Study: CERN s use of OpenStack Takeaways Today s Sequence Quick Review of
More informationComputing at the HL-LHC
Computing at the HL-LHC Predrag Buncic on behalf of the Trigger/DAQ/Offline/Computing Preparatory Group ALICE: Pierre Vande Vyvre, Thorsten Kollegger, Predrag Buncic; ATLAS: David Rousseau, Benedetto Gorini,
More informationCloud Computing. A new kind of developers? Presentation by. Nick Barcet nick.barcet@canonical.com
Cloud Computing A new kind of developers? Presentation by Nick Barcet nick.barcet@canonical.com www.canonical.com July 2011 Cloud computing stack Salesforce.com, GoogleDocs, Office, etc... GoogleApps,
More informationBuilding Docker Cloud Services with Virtuozzo
Building Docker Cloud Services with Virtuozzo Improving security and performance of application containers services in the cloud EXECUTIVE SUMMARY Application containers, and Docker in particular, are
More informationDevelopment of nosql data storage for the ATLAS PanDA Monitoring System
Development of nosql data storage for the ATLAS PanDA Monitoring System M.Potekhin Brookhaven National Laboratory, Upton, NY11973, USA E-mail: potekhin@bnl.gov Abstract. For several years the PanDA Workload
More informationStackato PaaS Architecture: How it works and why.
Stackato PaaS Architecture: How it works and why. White Paper Published in 2012 Stackato PaaS Architecture: How it works and why. Stackato is software for creating a private Platform-as-a-Service (PaaS).
More informationwww.see-grid-sci.eu Regional SEE-GRID-SCI Training for Site Administrators Institute of Physics Belgrade March 5-6, 2009
SEE-GRID-SCI Virtualization and Grid Computing with XEN www.see-grid-sci.eu Regional SEE-GRID-SCI Training for Site Administrators Institute of Physics Belgrade March 5-6, 2009 Milan Potocnik University
More informationManjrasoft Market Oriented Cloud Computing Platform
Manjrasoft Market Oriented Cloud Computing Platform Innovative Solutions for 3D Rendering Aneka is a market oriented Cloud development and management platform with rapid application development and workload
More informationCloud and Virtualization to Support Grid Infrastructures
ESAC GRID Workshop '08 ESAC, Villafranca del Castillo, Spain 11-12 December 2008 Cloud and Virtualization to Support Grid Infrastructures Distributed Systems Architecture Research Group Universidad Complutense
More informationData Centers and Cloud Computing. Data Centers
Data Centers and Cloud Computing Intro. to Data centers Virtualization Basics Intro. to Cloud Computing 1 Data Centers Large server and storage farms 1000s of servers Many TBs or PBs of data Used by Enterprises
More informationCloud Computing for Control Systems CERN Openlab Summer Student Program 9/9/2011 ARSALAAN AHMED SHAIKH
Cloud Computing for Control Systems CERN Openlab Summer Student Program 9/9/2011 ARSALAAN AHMED SHAIKH CONTENTS Introduction... 4 System Components... 4 OpenNebula Cloud Management Toolkit... 4 VMware
More informationPES. Batch virtualization and Cloud computing. Part 1: Batch virtualization. Batch virtualization and Cloud computing
Batch virtualization and Cloud computing Batch virtualization and Cloud computing Part 1: Batch virtualization Tony Cass, Sebastien Goasguen, Belmiro Moreira, Ewan Roche, Ulrich Schwickerath, Romain Wartel
More informationGlobal Grid User Support - GGUS - in the LCG & EGEE environment
Global Grid User Support - GGUS - in the LCG & EGEE environment Torsten Antoni (torsten.antoni@iwr.fzk.de) Why Support? New support groups Network layer Resource centers CIC / GOC / etc. more to come New
More informationAneka Dynamic Provisioning
MANJRASOFT PTY LTD Aneka Aneka 2.0 Manjrasoft 10/22/2010 This document describes the dynamic provisioning features implemented in Aneka and how it is possible to leverage dynamic resources for scaling
More informationGlobal Headquarters: 5 Speen Street Framingham, MA 01701 USA P.508.872.8200 F.508.935.4015 www.idc.com
Global Headquarters: 5 Speen Street Framingham, MA 01701 USA P.508.872.8200 F.508.935.4015 www.idc.com W H I T E P A P E R O r a c l e V i r t u a l N e t w o r k i n g D e l i v e r i n g F a b r i c
More informationManjrasoft Market Oriented Cloud Computing Platform
Manjrasoft Market Oriented Cloud Computing Platform Aneka Aneka is a market oriented Cloud development and management platform with rapid application development and workload distribution capabilities.
More informationComputer Science. About PaaS Security. Donghoon Kim Henry E. Schaffer Mladen A. Vouk
About PaaS Security Donghoon Kim Henry E. Schaffer Mladen A. Vouk North Carolina State University, USA May 21, 2015 @ ICACON 2015 Outline Introduction Background Contribution PaaS Vulnerabilities and Countermeasures
More informationGrid Scheduling Dictionary of Terms and Keywords
Grid Scheduling Dictionary Working Group M. Roehrig, Sandia National Laboratories W. Ziegler, Fraunhofer-Institute for Algorithms and Scientific Computing Document: Category: Informational June 2002 Status
More informationScience+ Large Hadron Cancer & Frank Wurthwein Virus Hunting
Shared Computing Driving Discovery: From the Large Hadron Collider to Virus Hunting Frank Würthwein Professor of Physics University of California San Diego February 14th, 2015 The Science of the LHC The
More informationThis presentation provides an overview of the architecture of the IBM Workload Deployer product.
This presentation provides an overview of the architecture of the IBM Workload Deployer product. Page 1 of 17 This presentation starts with an overview of the appliance components and then provides more
More informationVirtualization, SDN and NFV
Virtualization, SDN and NFV HOW DO THEY FIT TOGETHER? Traditional networks lack the flexibility to keep pace with dynamic computing and storage needs of today s data centers. In order to implement changes,
More informationAn objective comparison test of workload management systems
An objective comparison test of workload management systems Igor Sfiligoi 1 and Burt Holzman 1 1 Fermi National Accelerator Laboratory, Batavia, IL 60510, USA E-mail: sfiligoi@fnal.gov Abstract. The Grid
More informationBig Data Trends and HDFS Evolution
Big Data Trends and HDFS Evolution Sanjay Radia Founder & Architect Hortonworks Inc Page 1 Hello Founder, Hortonworks Part of the Hadoop team at Yahoo! since 2007 Chief Architect of Hadoop Core at Yahoo!
More informationEnterprise Storage Solution for Hyper-V Private Cloud and VDI Deployments using Sanbolic s Melio Cloud Software Suite April 2011
Enterprise Storage Solution for Hyper-V Private Cloud and VDI Deployments using Sanbolic s Melio Cloud Software Suite April 2011 Executive Summary Large enterprise Hyper-V deployments with a large number
More information7 Myths about Backup & DR in Virtual Environments
NEW White Paper 7 Myths about Backup & DR in Virtual Environments by Eric Siebert, VMware vexpert Eric Siebert VMware vexpert Eric Siebert is an IT industry veteran, speaker, author and blogger with more
More informationCloudy Middleware MARK LITTLE <MLITTLE@REDHAT.COM> TOBIAS KUNZE <TKUNZE@REDHAT.COM>
Cloudy Middleware MARK LITTLE TOBIAS KUNZE About Mark Little Sr Director of Engineering, Red Hat Tobias Kunze PaaS Architect, Red Hat CTO/Co-founder of Makara 2
More informationGlobal Grid User Support - GGUS - start up schedule
Global Grid User Support - GGUS - start up schedule GDB Meeting 2004-07 07-13 Concept Target: 24 7 support via time difference and 3 support teams Currently: GGUS FZK GGUS ASCC Planned: GGUS USA Support
More informationSUSE Cloud 2.0. Pete Chadwick. Douglas Jarvis. Senior Product Manager pchadwick@suse.com. Product Marketing Manager djarvis@suse.
SUSE Cloud 2.0 Pete Chadwick Douglas Jarvis Senior Product Manager pchadwick@suse.com Product Marketing Manager djarvis@suse.com SUSE Cloud SUSE Cloud is an open source software solution based on OpenStack
More informationKVM, OpenStack and the Open Cloud SUSECon November 2015
KVM, OpenStack and the Open Cloud SUSECon November 2015 Adam Jollans Program Director, Linux & Open Virtualization Strategy IBM Agenda A Brief History of Virtualization KVM Architecture OpenStack Architecture
More information