Data analysis requires new approaches in many domains for evaluating tools and techniques, particularly when the data sets grow large and more complex. Evaluation–as–a– service (EaaS) was coined as a term to represent evaluation approaches based on APIs, virtual machines or source code submission, different from the common paradigm of evaluating techniques on a distributed test collection, tasks and submitted results files. Such new approaches become necessary when data sets become extremely large, contain confidential information or might change quickly over time. The workshop on cloud–based evaluation (CBE) took place in Boston, MA, USA on November 5, 2015 and explored several approaches for data analysis evaluation and frameworks in this field. The objective was to include several stakeholders from academic partners, companies to funding agencies to cover various interests and viewpoints in the discussion of evaluation infrastructures. The workshop focused on the biomedical domain but the results are easily applicable to many domains of information analysis and retrieval.
Allan Hanbury is Professor for Data Intelligence at the TU Wien, Austria, and Faculty Member of the Complexity Science Hub. He is initiator of the Austrian ICT Lighthouse Project, as well as participates in Data Market Austria, which is creating a Data-Services Ecosystem in Austria. He was scientific coordinator of the EU-funded Khresmoi Integrated Project on medical and health information search and analysis, and is co-founder of contextflow, the spin-off company commercialising the radiology image search technology developed in the Khresmoi project. He also coordinated the EU-funded VISCERAL project on evaluation of algorithms on big data, and the EU-funded KConnect project on technology for analysing medical text.
His areas of research include Data Science, Information Retrieval, Semantic Analysis and Search, Information Retrieval Evaluation, Recommender Systems, Data Mining and Machine Learning.
Andrew Trister is a Clinical Research Professor in radiation medicine at the School of Radiation Medicine.
Antonio Criminisi joined the Machine Learning and Perception group at Microsoft Research Cambridge in June 2000 as Visiting Researcher. In February 2001, he moved to the Interactive Visual Media Group in Redmond (WA, USA) as a Post-Doctorate Researcher. In October 2002, he moved back to the Machine Learning and Perception Group in Cambridge as Researcher. Antonio’s current research interests are in the area of medical image analysis, object category recognition, image and video analysis and editing, one-to-one teleconferencing, 3D reconstruction from single and multiple images with application to virtual reality, forensic science and history of art.
Arno Klein is the Child Mind Institute’s Director of Innovative Technologies, Joseph Healey Scholar, and Research Scientist in the Center for the Developing Brain. He is developing a Sensors and Wearables Program to study mental illness and offer potential interventions.
Dr. Klein builds upon recent experience as the scientific lead on the mPower Parkinson mobile health research study launched last year by Apple. The mPower study is an example of how mobile devices can vastly scale up open medical research, in this case by tracking symptoms of Parkinson disease from iPhone sensors in thousands of participants. Dr. Klein created an open source software pipeline for analyzing sensor data and designed interactive visualizations to present results to patients, clinicians and researchers.
Artm Mamonov is a Software Engineer at MGH & BWH Center for Clinical Data Science developing scalable, web-based applications for annotation of medical images and reports. He also works on visualization of results of Machine Learning algorithms and integration of these results into medical workflows.
David Kennedy serves as Vice President and General Manager of Teledyne Isco, Inc. Mr. Kennedy began his professional career with Rohm and Haas Co. in Philadelphia, Pa., in industrial research and development. He received BS and PhD degrees in chemistry from Iowa State University in 1965 and 1969, respectively.
C.E.O. and Founder of Omics Data Automation Inc. Gans holds 39 U.S. Patents across multiple disciplines including bioinformatics, genomics, medical imaging, processor architecture, and enterprise computing. Previously a Senior Principal Engineer at Intel; At Intel Gans was the Chief Architect who developed the Xeon multi-core architecture. Led the pathfinding team which cut compute time for Whole Genome Sequencing from a week to a day on Intel Architecture. Developed, designed and spearheaded the Intel Collaborative Cancer Cloud project which was a complete new approach to sharing large medical data files securely between medical centers across North America. Currently Gans and his team have developed a framework and tools to enable medical data automation, aggregation, and analysis to enable the practice of precision medicine.
Dr. Müller is a professor in computer science at the HES-SO and titular prof. in medicine at the University of Geneva.
Dr. Kalpathy-Cramer is an Associate Professor of Radiology, Harvard Medical School and an Assistant in Neuroscience, Massachusetts General Hospital
Jin H. Paik is the Program Director and Senior Researcher at the Crowd Innovation Laboratory (CIL)/NASA Tournament Laboratory (NTL) at Harvard University. In his role, he serves as the lab’s general manager. He works to develop strategic vision at the lab and directs project and research activities. He oversees the development of open innovation projects through partnerships with NASA, federal government agencies, partner academic institutions, and research institutes. He advises organizations and firms on adoption innovation strategies and prize-based contests for solving complex problems. He has worked extensively on programs ranging from improvements NASA’s International Space Station to machine learning uses for medical imaging. He currently works to shape impact on precision medicine through contest development with the Harvard Medical School and the Broad Institute. Prior to joining the CIL team, he worked at the Harvard Kennedy School and Mathematica Policy Research. He holds a bachelor’s degree from the University of Michigan and a master’s degree from Harvard University.
Keyvan Farahani is the program director for Image-Guided Interventions, Cancer Imaging Program, at the National Cancer Institute. He’s responsible for the development of NCI research initiatives that address diagnosis and treatment of cancer through integration of advanced imaging and minimally invasive interventions, including nanotechnologies. He has led NCI initiatives in oncologic IGI focused on small business development, early phase clinical trials, and image-guided drug delivery research. Since 2013, in collaboration with national and international academic groups, Keyvan has led the organization of many computational challenges related to imaging, digital pathology, and radiomics of cancer, conducted through international scientific societies. He chairs the NCI Quantitative Imaging Network’s Task Force on Challenge and Collaborative Projects. He obtained his Ph.D. in biomedical physics from the University of California at Los Angeles in 1993
Nina Preuss is an experienced project manager of scientific, technical and non-profit projects, software analysis, selection, and modification, event logistics, quality improvement and process management, strategic planning, sales and marketing.
Dr. Rinat Sergeev is Senior Data Scientist & Scientific Advisor at the Crowd Innovation Lab/NASA Tournament Lab at Harvard University. Rinat works as a head of data science team, and a lead science and technical expert on exploring and utilizing crowdsourcing approaches in application to the data science and algorithmic challenges, coming from NASA, Business, or Academia. In his role, Rinat provides full guidance and support on the way of the project from learning the area and formulating the problem, to controlling the challenge execution and analysing it’s outcome, working closely with all the parties involved. Rinat received his PhD in Quantum Mechanics in Ioffe Institute, Saint Petersburg. Following his innate curiosity, he pursued challenges in a variety of academic fields, from Semiconductors to Immunology and Epidemiology. His research interests include conceptual analysis, analytical approaches and models in multiple areas. His personal interests include Math puzzles, strategic games and politics.
Dr. Norman is a senior program officer in Quantitative Sciences at the Bill & Melinda Gates Foundation. She currently co-leads the Knowledge Integration (Ki) initiative on the Integrated Development team in Global Health. Thea comes to the foundation with more than 18 years of experience in science and business leadership in startup biotechnology, most recently in “big data” open science working at the leading edge of digital and mobile health. While in biotech, Thea co-invented two molecules that subsequently received FDA approval - one for people (Linaclotide: first in class for IBS) and one for dairy cows (Pegbovigrastim: first in class for mastitis). On the business development side, Thea helped sign collaborations and led the resulting scientific alliances with: Bristol-Myers Squibb, Eli Lilly and Company, Merck Inc, Merck-Serono, Pfizer Inc., Google, and IBM. Thea holds a Ph.D. in Chemistry (UC Berkeley) and her doctoral work was followed by two post-docs focused on combinatorial chemistry (UC Berkeley) and then molecular biology and yeast genetics (UCSF).
Sign in below to view this resource. If you do not have an account, you can register for free.