DSS. Data & Storage Services. Cloud storage performance and first experience from prototype services at CERN
|
|
|
- Gillian Day
- 8 years ago
- Views:
Transcription
1 Data & Storage Cloud storage performance and first experience from prototype services at CERN Maitane Zotes Resines, Seppo S. Heikkila, Dirk Duellmann, Geoffray Adde, Rainer Toebbicke, CERN James Hughes, Huawei Lu Wang, IHEP 20th International Conference on Computing in High Energy and Nuclear Physics (CHEP) 14th of October 2013 Amsterdam, The Netherlands
2 Introduction Motivation Cloud storage market is growing fast CERN uses custom made storage solutions Question Are cloud storages able to meet the High Energy Physics (HEP) data storage requirements? Method Evaluate scalability and fault-tolerance Test with real applications 2
3 Huawei cloud storage setup Located physically at CERN Storage nodes Front-end nodes 3 Storage nodes
4 Huawei cloud storage setup 384 disks 768 TB Storage nodes S3 compatible 4 Front-end nodes Buckets divide the name space Storage nodes
5 Storage nodes Each blade has eight storage nodes One chassis has two blades (16 disks) 5
6 Benchmarking scalability Speed vs concurrent clients 6
7 Benchmarking scalability Small (4kB) & big (100MB) files Speed vs concurrent clients Enough network capacity & client machines 7
8 Download throughput scalability 100MB file downloads Scales up to bandwidth limit 8
9 Download metadata scalability 4kB file downloads a e Lin 9 c s r ng i l a Front-end performance limit reached
10 Upload metadata scalability 4kB file uploads More buckets required, as expected 10 Scales: up to 2,500 files/s
11 Front-end scalability 4kB file downloads Linear scaling
12 Raw performance summary Metadata (4kB) performance 2,500 files/second upload 25,000 files/second download Throughput (100MB) performance 20Gbit network fully utilized Front-end scalability Each front-end downloads 3500 files/s 12
13 Recovery after powering off a chassis Chassis down Chassis up 16 disks down 13 Time (seconds)
14 Recovery after powering off a chassis Chassis down No failures, only delays Chassis up First storage nodes respond to ping All storage nodes in use 14 Time (seconds)
15 CVMFS introduction What is CVMFS (CernVM File System) Read only cached file system to deliver software Widely used in WLCG (Worldwide LHC Computing Grid) Mounted by users and files are downloaded on demand 15
16 CVMFS introduction What is CVMFS (CernVM File System) Read only cached file system to deliver software Widely used in WLCG (Worldwide LHC Computing Grid) Mounted by users and files are downloaded on demand CVMFS challenges Publishing new software should be fast (upload tens of thousands of files) Files should be accessed with HTTP protocol 16
17 File system with Huawei back-end Implementation Files are uploaded to multiple buckets in the cloud storage Files are downloaded with unified name space
18 File system with Huawei back-end Implementation Files are uploaded to multiple buckets in the cloud storage Files are downloaded with unified name space Result Full publish procedure tested to work using 30,000 small files Upload speed 1200 files/second (with 240 threads) 18
19 CVMFS partial uploads Uploading 10,000 files (of average size 10kB) Motivation: only part of the files change between releases 19
20 Conclusion Raw performance Upload and download scalability demonstrated Additional front-end nodes increased linearly the performance Fault tolerance: powering off a chassis Transparent disk failure recovery demonstrated File system with cloud storage back-end Full publishing procedure tested Uploading of only new files feature tested 20
21 Conclusion Raw performance Upload and download scalability demonstrated Additional front-end nodes increased linearly the performance Fault tolerance: powering off a chassis Transparent disk failure recovery demonstrated File system with cloud storage back-end Full publishing procedure tested Uploading of only new files feature tested Dank je wel. Questions? seppo.heikkila@cern.ch 21
Using S3 cloud storage with ROOT and CernVMFS. Maria Arsuaga-Rios Seppo Heikkila Dirk Duellmann Rene Meusel Jakob Blomer Ben Couturier
Using S3 cloud storage with ROOT and CernVMFS Maria Arsuaga-Rios Seppo Heikkila Dirk Duellmann Rene Meusel Jakob Blomer Ben Couturier INDEX Huawei cloud storages at CERN Old vs. new Huawei UDS comparative
CERN Cloud Storage Evaluation Geoffray Adde, Dirk Duellmann, Maitane Zotes CERN IT
SS Data & Storage CERN Cloud Storage Evaluation Geoffray Adde, Dirk Duellmann, Maitane Zotes CERN IT HEPiX Fall 2012 Workshop October 15-19, 2012 Institute of High Energy Physics, Beijing, China SS Outline
Development of Monitoring and Analysis Tools for the Huawei Cloud Storage
Development of Monitoring and Analysis Tools for the Huawei Cloud Storage September 2014 Author: Veronia Bahaa Supervisors: Maria Arsuaga-Rios Seppo S. Heikkila CERN openlab Summer Student Report 2014
files without borders
files without borders exploring Internet-connected storage for research Fabio Hernandez fabio@in2p3.fr IN2P3 / CNRS computing center, Lyon, France FJPPL compu+ng workshop, Lyon, March 11th 2015 2 Preamble
Improved metrics collection and correlation for the CERN cloud storage test framework
Improved metrics collection and correlation for the CERN cloud storage test framework September 2013 Author: Carolina Lindqvist Supervisors: Maitane Zotes Seppo Heikkila CERN openlab Summer Student Report
Alternative models to distribute VO specific software to WLCG sites: a prototype set up at PIC
EGEE and glite are registered trademarks Enabling Grids for E-sciencE Alternative models to distribute VO specific software to WLCG sites: a prototype set up at PIC Elisa Lanciotti, Arnau Bria, Gonzalo
Storage strategy and cloud storage evaluations at CERN Dirk Duellmann, CERN IT
SS Data & Storage Storage strategy and cloud storage evaluations at CERN Dirk Duellmann, CERN IT (with slides from Andreas Peters and Jan Iven) 5th International Conference "Distributed Computing and Grid-technologies
Shoal: IaaS Cloud Cache Publisher
University of Victoria Faculty of Engineering Winter 2013 Work Term Report Shoal: IaaS Cloud Cache Publisher Department of Physics University of Victoria Victoria, BC Mike Chester V00711672 Work Term 3
HEP Compu*ng in a Context- Aware Cloud Environment
HEP Compu*ng in a Context- Aware Cloud Environment Randall Sobie A.Charbonneau F.Berghaus R.Desmarais I.Gable C.LeaveC- Brown M.Paterson R.Taylor InsItute of ParIcle Physics University of Victoria and
Leveraging BlobSeer to boost up the deployment and execution of Hadoop applications in Nimbus cloud environments on Grid 5000
Leveraging BlobSeer to boost up the deployment and execution of Hadoop applications in Nimbus cloud environments on Grid 5000 Alexandra Carpen-Amarie Diana Moise Bogdan Nicolae KerData Team, INRIA Outline
Status and Evolution of ATLAS Workload Management System PanDA
Status and Evolution of ATLAS Workload Management System PanDA Univ. of Texas at Arlington GRID 2012, Dubna Outline Overview PanDA design PanDA performance Recent Improvements Future Plans Why PanDA The
DSS. Diskpool and cloud storage benchmarks used in IT-DSS. Data & Storage Services. Geoffray ADDE
DSS Data & Diskpool and cloud storage benchmarks used in IT-DSS CERN IT Department CH-1211 Geneva 23 Switzerland www.cern.ch/it Geoffray ADDE DSS Outline I- A rational approach to storage systems evaluation
The Evolution of Cloud Computing in ATLAS
The Evolution of Cloud Computing in ATLAS Ryan Taylor on behalf of the ATLAS collaboration CHEP 2015 Evolution of Cloud Computing in ATLAS 1 Outline Cloud Usage and IaaS Resource Management Software Services
Distributed Computing for CEPC. YAN Tian On Behalf of Distributed Computing Group, CC, IHEP for 4 th CEPC Collaboration Meeting, Sep.
Distributed Computing for CEPC YAN Tian On Behalf of Distributed Computing Group, CC, IHEP for 4 th CEPC Collaboration Meeting, Sep. 12-13, 2014 1 Outline Introduction Experience of BES-DIRAC Distributed
Distributed File System. MCSN N. Tonellotto Complements of Distributed Enabling Platforms
Distributed File System 1 How do we get data to the workers? NAS Compute Nodes SAN 2 Distributed File System Don t move data to workers move workers to the data! Store data on the local disks of nodes
IPv6 Traffic Analysis and Storage
Report from HEPiX 2012: Network, Security and Storage david.gutierrez@cern.ch Geneva, November 16th Network and Security Network traffic analysis Updates on DC Networks IPv6 Ciber-security updates Federated
Cloud Storage. Parallels. Performance Benchmark Results. White Paper. www.parallels.com
Parallels Cloud Storage White Paper Performance Benchmark Results www.parallels.com Table of Contents Executive Summary... 3 Architecture Overview... 3 Key Features... 4 No Special Hardware Requirements...
www.basho.com Technical Overview Simple, Scalable, Object Storage Software
www.basho.com Technical Overview Simple, Scalable, Object Storage Software Table of Contents Table of Contents... 1 Introduction & Overview... 1 Architecture... 2 How it Works... 2 APIs and Interfaces...
Data and Storage Services
Data and Storage Services G. Cancio, D. Duellmann, J. Iven, M. Lamanna, A. Pace, A.J. Peters, R.Toebbicke CERN IT Department CH-1211 Genève 23 Switzerland www.cern.ch/it CERN IT Department CH-1211 Genève
Comparison of the Frontier Distributed Database Caching System with NoSQL Databases
Comparison of the Frontier Distributed Database Caching System with NoSQL Databases Dave Dykstra dwd@fnal.gov Fermilab is operated by the Fermi Research Alliance, LLC under contract No. DE-AC02-07CH11359
Introduction to Windows Azure Cloud Computing Futures Group, Microsoft Research Roger Barga, Jared Jackson,Nelson Araujo, Dennis Gannon, Wei Lu, and
Introduction to Windows Azure Cloud Computing Futures Group, Microsoft Research Roger Barga, Jared Jackson,Nelson Araujo, Dennis Gannon, Wei Lu, and Jaliya Ekanayake Range in size from edge facilities
PARALLELS CLOUD STORAGE
PARALLELS CLOUD STORAGE Performance Benchmark Results 1 Table of Contents Executive Summary... Error! Bookmark not defined. Architecture Overview... 3 Key Features... 5 No Special Hardware Requirements...
Hadoop Architecture. Part 1
Hadoop Architecture Part 1 Node, Rack and Cluster: A node is simply a computer, typically non-enterprise, commodity hardware for nodes that contain data. Consider we have Node 1.Then we can add more nodes,
In Memory Accelerator for MongoDB
In Memory Accelerator for MongoDB Yakov Zhdanov, Director R&D GridGain Systems GridGain: In Memory Computing Leader 5 years in production 100s of customers & users Starts every 10 secs worldwide Over 15,000,000
Summer Student Project Report
Summer Student Project Report Dimitris Kalimeris National and Kapodistrian University of Athens June September 2014 Abstract This report will outline two projects that were done as part of a three months
How swift is your Swift? Ning Zhang, OpenStack Engineer at Zmanda Chander Kant, CEO at Zmanda
How swift is your Swift? Ning Zhang, OpenStack Engineer at Zmanda Chander Kant, CEO at Zmanda 1 Outline Build a cost-efficient Swift cluster with expected performance Background & Problem Solution Experiments
Network Infrastructure Services CS848 Project
Quality of Service Guarantees for Cloud Services CS848 Project presentation by Alexey Karyakin David R. Cheriton School of Computer Science University of Waterloo March 2010 Outline 1. Performance of cloud
A High-Performance Storage System for the LHCb Experiment Juan Manuel Caicedo Carvajal, Jean-Christophe Garnier, Niko Neufeld, and Rainer Schwemmer
658 IEEE TRANSACTIONS ON NUCLEAR SCIENCE, VOL. 57, NO. 2, APRIL 2010 A High-Performance Storage System for the LHCb Experiment Juan Manuel Caicedo Carvajal, Jean-Christophe Garnier, Niko Neufeld, and Rainer
Avid ISIS 7000. www.avid.com
Avid ISIS 7000 www.avid.com Table of Contents Overview... 3 Avid ISIS Technology Overview... 6 ISIS Storage Blade... 6 ISIS Switch Blade... 7 ISIS System Director... 7 ISIS Client Software... 8 ISIS Redundant
Prototyping a file sharing and synchronisation platform with owncloud
Data & Storage Services Prototyping a file sharing and synchronisation platform with owncloud CERN IT Department CH-1211 Geneva 23 Switzerland www.cern.ch/it Jakub T. Moscicki Massimo Lamanna CERN IT-
Big Data Processing Experience in the ATLAS Experiment
Big Data Processing Experience in the ATLAS Experiment A. on behalf of the ATLAS Collabora5on Interna5onal Symposium on Grids and Clouds (ISGC) 2014 March 23-28, 2014 Academia Sinica, Taipei, Taiwan Introduction
Key Messages of Enterprise Cluster NAS Huawei OceanStor N8500
Messages of Enterprise Cluster NAS Huawei OceanStor Messages of Enterprise Cluster NAS 1. High performance and high reliability, addressing bid data challenges High performance: In the SPEC benchmark test,
Scala Storage Scale-Out Clustered Storage White Paper
White Paper Scala Storage Scale-Out Clustered Storage White Paper Chapter 1 Introduction... 3 Capacity - Explosive Growth of Unstructured Data... 3 Performance - Cluster Computing... 3 Chapter 2 Current
DSS. High performance storage pools for LHC. Data & Storage Services. Łukasz Janyst. on behalf of the CERN IT-DSS group
DSS High performance storage pools for LHC Łukasz Janyst on behalf of the CERN IT-DSS group CERN IT Department CH-1211 Genève 23 Switzerland www.cern.ch/it Introduction The goal of EOS is to provide a
Evolution of Database Replication Technologies for WLCG
Home Search Collections Journals About Contact us My IOPscience Evolution of Database Replication Technologies for WLCG This content has been downloaded from IOPscience. Please scroll down to see the full
Dynamic Extension of a Virtualized Cluster by using Cloud Resources CHEP 2012
Dynamic Extension of a Virtualized Cluster by using Cloud Resources CHEP 2012 Thomas Hauth,, Günter Quast IEKP KIT University of the State of Baden-Wuerttemberg and National Research Center of the Helmholtz
Solution for private cloud computing
The CC1 system Solution for private cloud computing 1 Outline What is CC1? Features Technical details Use cases By scientist By HEP experiment System requirements and installation How to get it? 2 What
Solbox Cloud Storage Acceleration
DATA SHEET Solbox Cloud Storage Acceleration Today s ongoing and rapidly-accelerating growth in data comes at the same time that organizations of all sizes are focused on cost deduction. Cloud storage
Multi-Channel Clustered Web Application Servers
THE AMERICAN UNIVERSITY IN CAIRO SCHOOL OF SCIENCES AND ENGINEERING Multi-Channel Clustered Web Application Servers A Masters Thesis Department of Computer Science and Engineering Status Report Seminar
BENCHMARKING CLOUD DATABASES CASE STUDY on HBASE, HADOOP and CASSANDRA USING YCSB
BENCHMARKING CLOUD DATABASES CASE STUDY on HBASE, HADOOP and CASSANDRA USING YCSB Planet Size Data!? Gartner s 10 key IT trends for 2012 unstructured data will grow some 80% over the course of the next
SPC BENCHMARK 1 EXECUTIVE SUMMARY 3PAR INC. 3PARINSERV T800 STORAGE SERVER SPC-1 V1.10.1
SPC BENCHMARK 1 EXECUTIVE SUMMARY 3PAR INC. 3PARINSERV T800 STORAGE SERVER SPC-1 V1.10.1 Submitted for Review: September 2, 2008 Submission Identifier: A00069 EXECUTIVE SUMMARY Page 2 of 7 EXECUTIVE SUMMARY
Chapter 7. Using Hadoop Cluster and MapReduce
Chapter 7 Using Hadoop Cluster and MapReduce Modeling and Prototyping of RMS for QoS Oriented Grid Page 152 7. Using Hadoop Cluster and MapReduce for Big Data Problems The size of the databases used in
Maginatics Cloud Storage Platform for Elastic NAS Workloads
Maginatics Cloud Storage Platform for Elastic NAS Workloads Optimized for Cloud Maginatics Cloud Storage Platform () is the first solution optimized for the cloud. It provides lower cost, easier administration,
ASPERA HIGH-SPEED TRANSFER SOFTWARE. Moving the world s data at maximum speed
ASPERA HIGH-SPEED TRANSFER SOFTWARE Moving the world s data at maximum speed PRESENTERS AND AGENDA PRESENTER John Heaton Aspera Director of Sales Engineering john@asperasoft.com AGENDA How Cloud is used
GridFTP GUI: An Easy and Efficient Way to Transfer Data in Grid
GridFTP GUI: An Easy and Efficient Way to Transfer Data in Grid Wantao Liu 1,2 Raj Kettimuthu 2,3, Brian Tieman 3, Ravi Madduri 2,3, Bo Li 1, and Ian Foster 2,3 1 Beihang University, Beijing, China 2 The
Integration of Network Performance Monitoring Data at FTS3
Integration of Network Performance Monitoring Data at FTS3 July-August 2013 Author: Rocío Rama Ballesteros Supervisor(s): Michail Salichos Alejandro Álvarez CERN openlab Summer Student Report 2013 Project
Testing & Assuring Mobile End User Experience Before Production. Neotys
Testing & Assuring Mobile End User Experience Before Production Neotys Agenda Introduction The challenges Best practices NeoLoad mobile capabilities Mobile devices are used more and more At Home In 2014,
YAN, Tian. On behalf of distributed computing group. Institute of High Energy Physics (IHEP), CAS, China. CHEP-2015, Apr. 13-17th, OIST, Okinawa
YAN, Tian On behalf of distributed computing group Institute of High Energy Physics (IHEP), CAS, China CHEP-2015, Apr. 13-17th, OIST, Okinawa Distributed computing for BESIII Other experiments wish to
Business-centric Storage FUJITSU Hyperscale Storage System ETERNUS CD10000
Business-centric Storage FUJITSU Hyperscale Storage System ETERNUS CD10000 Clear the way for new business opportunities. Unlock the power of data. Overcoming storage limitations Unpredictable data growth
The Availability of Commercial Storage Clouds
The Availability of Commercial Storage Clouds Literature Study Introduction to e-science infrastructure 2008-2009 Arjan Borst ccn 0478199 Grid Computing - University of Amsterdam Software Engineer - WireITup
Aspera Direct-to-Cloud Storage WHITE PAPER
Transport Direct-to-Cloud Storage and Support for Third Party April 2014 WHITE PAPER TABLE OF CONTENTS OVERVIEW 3 1 - THE PROBLEM 3 2 - A FUNDAMENTAL SOLUTION - ASPERA DIRECT-TO-CLOUD TRANSPORT 5 3 - VALIDATION
Analisi di un servizio SRM: StoRM
27 November 2007 General Parallel File System (GPFS) The StoRM service Deployment configuration Authorization and ACLs Conclusions. Definition of terms Definition of terms 1/2 Distributed File System The
Testing of several distributed file-system (HadoopFS, CEPH and GlusterFS) for supporting the HEP experiments analisys. Giacinto DONVITO INFN-Bari
Testing of several distributed file-system (HadoopFS, CEPH and GlusterFS) for supporting the HEP experiments analisys. Giacinto DONVITO INFN-Bari 1 Agenda Introduction on the objective of the test activities
Direct NFS - Design considerations for next-gen NAS appliances optimized for database workloads Akshay Shah Gurmeet Goindi Oracle
Direct NFS - Design considerations for next-gen NAS appliances optimized for database workloads Akshay Shah Gurmeet Goindi Oracle Agenda Introduction Database Architecture Direct NFS Client NFS Server
Protect Data... in the Cloud
QUASICOM Private Cloud Backups with ExaGrid Deduplication Disk Arrays Martin Lui Senior Solution Consultant Quasicom Systems Limited Protect Data...... in the Cloud 1 Mobile Computing Users work with their
IT of SPIM Data Storage and Compression. EMBO Course - August 27th! Jeff Oegema, Peter Steinbach, Oscar Gonzalez
IT of SPIM Data Storage and Compression EMBO Course - August 27th Jeff Oegema, Peter Steinbach, Oscar Gonzalez 1 Talk Outline Introduction and the IT Team SPIM Data Flow Capture, Compression, and the Data
AWS EC2 Experience: HAProxy
AWS EC2 Experience: HAProxy The Reliable, High Performance TCP/HTTP Load Balancer Enkhjin Bayarsaikhan Diego Montero 2012-05- 30 SDS - CANS@upc 1 Outline MoMvaMon 2012-05- 30 SDS - CANS@upc 2 The Problem
HPC Storage Solutions at transtec. Parallel NFS with Panasas ActiveStor
HPC Storage Solutions at transtec Parallel NFS with Panasas ActiveStor HIGH PERFORMANCE COMPUTING AT TRANSTEC More than 30 Years of Experience in Scientific Computing 1980: transtec founded, a reseller
Establishing Applicability of SSDs to LHC Tier-2 Hardware Configuration
Establishing Applicability of SSDs to LHC Tier-2 Hardware Configuration A CHEP 2010 presentation by: Sam Skipsey and The GridPP Storage Group With particular acknowledgments to: Wahid Bhimji (go see his
Enabling multi-cloud resources at CERN within the Helix Nebula project. D. Giordano (CERN IT-SDC) HEPiX Spring 2014 Workshop 23 May 2014
Enabling multi-cloud resources at CERN within the Helix Nebula project D. Giordano (CERN IT-) HEPiX Spring 2014 Workshop This document produced by Members of the Helix Nebula consortium is licensed under
POWER ALL GLOBAL FILE SYSTEM (PGFS)
POWER ALL GLOBAL FILE SYSTEM (PGFS) Defining next generation of global storage grid Power All Networks Ltd. Technical Whitepaper April 2008, version 1.01 Table of Content 1. Introduction.. 3 2. Paradigm
Introduction to HDFS. Prasanth Kothuri, CERN
Prasanth Kothuri, CERN 2 What s HDFS HDFS is a distributed file system that is fault tolerant, scalable and extremely easy to expand. HDFS is the primary distributed storage for Hadoop applications. Hadoop
Product Innovation Award - 2013: Frost & Sullivan
2013 PRODUCT INNOVATION AWARD IN CLOUD INFRASTRUCTURE RECIPIENT HUAWEI TECHNOLOGIES CO., LTD Frost & Sullivan s Global Research Platform Frost & Sullivan is in its 51th year in business with a global research
Write a technical report Present your results Write a workshop/conference paper (optional) Could be a real system, simulation and/or theoretical
Identify a problem Review approaches to the problem Propose a novel approach to the problem Define, design, prototype an implementation to evaluate your approach Could be a real system, simulation and/or
Investigation of storage options for scientific computing on Grid and Cloud facilities
Investigation of storage options for scientific computing on Grid and Cloud facilities Overview Context Test Bed Lustre Evaluation Standard benchmarks Application-based benchmark HEPiX Storage Group report
HP reference configuration for entry-level SAS Grid Manager solutions
HP reference configuration for entry-level SAS Grid Manager solutions Up to 864 simultaneous SAS jobs and more than 3 GB/s I/O throughput Technical white paper Table of contents Executive summary... 2
MagFS: The Ideal File System for the Cloud
: The Ideal File System for the Cloud is the first true file system for the cloud. It provides lower cost, easier administration, and better scalability and performance than any alternative in-cloud file
Hadoop IST 734 SS CHUNG
Hadoop IST 734 SS CHUNG Introduction What is Big Data?? Bulk Amount Unstructured Lots of Applications which need to handle huge amount of data (in terms of 500+ TB per day) If a regular machine need to
CISCO WIDE AREA APPLICATION SERVICES (WAAS) OPTIMIZATIONS FOR EMC AVAMAR
PERFORMANCE BRIEF CISCO WIDE AREA APPLICATION SERVICES (WAAS) OPTIMIZATIONS FOR EMC AVAMAR INTRODUCTION Enterprise organizations face numerous challenges when delivering applications and protecting critical
Scaling Objectivity Database Performance with Panasas Scale-Out NAS Storage
White Paper Scaling Objectivity Database Performance with Panasas Scale-Out NAS Storage A Benchmark Report August 211 Background Objectivity/DB uses a powerful distributed processing architecture to manage
Context-aware cloud computing for HEP
Department of Physics and Astronomy, University of Victoria, Victoria, British Columbia, Canada V8W 2Y2 E-mail: rsobie@uvic.ca The use of cloud computing is increasing in the field of high-energy physics
Patrick Fuhrmann. The DESY Storage Cloud
The DESY Storage Cloud Patrick Fuhrmann The DESY Storage Cloud Hamburg, 2/3/2015 for the DESY CLOUD TEAM Content > Motivation > Preparation > Collaborations and publications > What do you get right now?
Private Cloud Migration
W H I T E P A P E R Infrastructure Performance Analytics Private Cloud Migration Infrastructure Performance Validation Use Case October 2012 Table of Contents Introduction 3 Model of the Private Cloud
Content Repository Benchmark Loading 100 million documents
Content Repository Benchmark Loading 100 million documents Goal The goal of this benchmark is to prove that the Content Repository is scalable enough to be called Enterprise Content Repository. To achieve
dcache, list of topics
dcache, list of topics EGI Meeting on H2020 Patrick Fuhrmann dcache EIG Meeting Patrick Fuhrmann 22 October 2013 1 Content The project structure Project funding, customers and contacts Current work areas
Dynamic Resource Provisioning with HTCondor in the Cloud
Dynamic Resource Provisioning with HTCondor in the Cloud Ryan Taylor Frank Berghaus 1 Overview Review of Condor + Cloud Scheduler system Condor job slot configuration Dynamic slot creation Automatic slot
Running a typical ROOT HEP analysis on Hadoop/MapReduce. Stefano Alberto Russo Michele Pinamonti Marina Cobal
Running a typical ROOT HEP analysis on Hadoop/MapReduce Stefano Alberto Russo Michele Pinamonti Marina Cobal CHEP 2013 Amsterdam 14-18/10/2013 Topics The Hadoop/MapReduce model Hadoop and High Energy Physics
Amazon Cloud Storage Options
Amazon Cloud Storage Options Table of Contents 1. Overview of AWS Storage Options 02 2. Why you should use the AWS Storage 02 3. How to get Data into the AWS.03 4. Types of AWS Storage Options.03 5. Object
A Survey on Cloud Storage Systems
A Survey on Cloud Storage Systems Team : Xiaoming Xiaogang Adarsh Abhijeet Pranav Motivations No Taxonomy Detailed Survey for users Starting point for researchers Taxonomy Category Definition Example Instance
Investigation of Storage Systems for use in Grid Applications
Investigation of Storage Systems for use in Grid Applications Overview Test bed and testing methodology Tests of general performance Tests of the metadata servers Tests of root-based applications HEPiX
Cloud Computing PES. (and virtualization at CERN) Cloud Computing. GridKa School 2011, Karlsruhe. Disclaimer: largely personal view of things
PES Cloud Computing Cloud Computing (and virtualization at CERN) Ulrich Schwickerath et al With special thanks to the many contributors to this presentation! GridKa School 2011, Karlsruhe CERN IT Department
OSG Hadoop is packaged into rpms for SL4, SL5 by Caltech BeStMan, gridftp backend
Hadoop on HEPiX storage test bed at FZK Artem Trunov Karlsruhe Institute of Technology Karlsruhe, Germany KIT The cooperation of Forschungszentrum Karlsruhe GmbH und Universität Karlsruhe (TH) www.kit.edu
Faster Cloud Backup: How It's Done
TECH BRIEF Faster Cloud Backup: How It's Done What You Need to Know One of the main concerns we hear from customers is around the ability to transmit large amounts of data over the Internet with their
Scientific Computing Data Management Visions
Scientific Computing Data Management Visions ELI-Tango Workshop Szeged, 24-25 February 2015 Péter Szász Group Leader Scientific Computing Group ELI-ALPS Scientific Computing Group Responsibilities Data
owncloud Enterprise Edition on IBM Infrastructure
owncloud Enterprise Edition on IBM Infrastructure A Performance and Sizing Study for Large User Number Scenarios Dr. Oliver Oberst IBM Frank Karlitschek owncloud Page 1 of 10 Introduction One aspect of
Clusters in the Cloud
Clusters in the Cloud Dr. Paul Coddington, Deputy Director Dr. Shunde Zhang, Compu:ng Specialist eresearch SA October 2014 Use Cases Make the cloud easier to use for compute jobs Par:cularly for users
A Service for Data-Intensive Computations on Virtual Clusters
A Service for Data-Intensive Computations on Virtual Clusters Executing Preservation Strategies at Scale Rainer Schmidt, Christian Sadilek, and Ross King rainer.schmidt@arcs.ac.at Planets Project Permanent
WHITE PAPER BRENT WELCH NOVEMBER
BACKUP WHITE PAPER BRENT WELCH NOVEMBER 2006 WHITE PAPER: BACKUP TABLE OF CONTENTS Backup Overview 3 Background on Backup Applications 3 Backup Illustration 4 Media Agents & Keeping Tape Drives Busy 5
The CMS openstack, opportunistic, overlay, online-cluster Cloud (CMSooooCloud)"
The CMS openstack, opportunistic, overlay, online-cluster Cloud (CMSooooCloud)" J.A. Coarasa " CERN, Geneva, Switzerland" for the CMS TriDAS group." " CHEP2013, 14-18 October 2013, Amsterdam, The Netherlands
Boost Database Performance with the Cisco UCS Storage Accelerator
Boost Database Performance with the Cisco UCS Storage Accelerator Performance Brief February 213 Highlights Industry-leading Performance and Scalability Offloading full or partial database structures to
A Digital Fountain Approach to Reliable Distribution of Bulk Data
A Digital Fountain Approach to Reliable Distribution of Bulk Data John Byers, ICSI Michael Luby, ICSI Michael Mitzenmacher, Compaq SRC Ashu Rege, ICSI Application: Software Distribution New release of
BookKeeper. Flavio Junqueira Yahoo! Research, Barcelona. Hadoop in China 2011
BookKeeper Flavio Junqueira Yahoo! Research, Barcelona Hadoop in China 2011 What s BookKeeper? Shared storage for writing fast sequences of byte arrays Data is replicated Writes are striped Many processes
安 瑞 科 技 物 聯 網 對 應 用 交 付 器 (ADC) 的 需 求 及 應 用 實 例 徐 乃 丁 博 士 研 發 副 總 裁 / 技 術 長
安 瑞 科 技 物 聯 網 對 應 用 交 付 器 (ADC) 的 需 求 及 應 用 實 例 徐 乃 丁 博 士 研 發 副 總 裁 / 技 術 長 Internet of Things needs Application Delivery Controller (ADC) But Internet of Things demands a new class of networking equipment,
Building Storage Clouds for Online Applications A Case for Optimized Object Storage
Building Storage Clouds for Online Applications A Case for Optimized Object Storage Agenda Introduction: storage facts and trends Call for more online storage! AmpliStor: Optimized Object Storage Cost
Chapter 18: Database System Architectures. Centralized Systems
Chapter 18: Database System Architectures! Centralized Systems! Client--Server Systems! Parallel Systems! Distributed Systems! Network Types 18.1 Centralized Systems! Run on a single computer system and
SharePoint Performance Optimization
White Paper AX Series SharePoint Performance Optimization September 2011 WP_SharePoint_091511.1 TABLE OF CONTENTS 1 Introduction... 2 2 Executive Overview... 2 3 SSL Offload... 4 4 Connection Reuse...
The safer, easier way to help you pass any IT exams. Exam : E20-895. Backup Recovery - Avamar Expert Exam for Implementation Engineers.
http://www.51- pass.com Exam : E20-895 Title : Backup Recovery - Avamar Expert Exam for Implementation Engineers Version : Demo 1 / 7 1.An EMC Avamar customer is currently using a 2 TB Avamar Virtual Edition
Virtual machine interface. Operating system. Physical machine interface
Software Concepts User applications Operating system Hardware Virtual machine interface Physical machine interface Operating system: Interface between users and hardware Implements a virtual machine that