ARCH-COMP20 Repeatability Evaluation Report

9 pages•Published: September 25, 2020

Abstract

This report presents the results of the repeatability evaluation for the 4th International Competition on Verifying Continuous and Hybrid Systems (ARCH-COMP’20). The competition took place as part of the workshop Applied Verification for Continuous and Hybrid Systems (ARCH) in 2020, affiliated with the IFAC World Congress. In its fourth edition, twenty-eight tools submitted artifacts through a Git repository for the repeatability evaluation, applied to solve benchmark problems for seven competition categories. The majority of participants adhered to the requirements for this year’s repeatability evaluation, namely to submit scripts to automatically install and execute tools in containerized virtual environments (specifically Dockerfiles to execute within Docker), and several categories used performance evaluation information from a common execution platform. The repeatability results represent a snapshot of the current landscape of tools and the types of benchmarks for which they are particularly suited and for which others may repeat their analyses. Due to the diversity of problems in verification of continuous and hybrid systems, as well as basing on standard practice in repeatability evaluations, we evaluate the tools with pass and/or failing being repeatable.

Keyphrases: artifact evaluation, hybrid systems, reachability, repeatability, verification

In: Goran Frehse and Matthias Althoff (editors). ARCH20. 7th International Workshop on Applied Verification of Continuous and Hybrid Systems (ARCH20), vol 74, pages 175-183.

Links:	https://easychair.org/publications/paper/3W11
	https://doi.org/10.29007/8dp4

BibTeX entry

@inproceedings{ARCH20:ARCH_COMP20_Repeatability_Evaluation,
  author    = {Taylor T Johnson},
  title     = {ARCH-COMP20 Repeatability Evaluation Report},
  booktitle = {ARCH20. 7th International Workshop on Applied Verification of Continuous and Hybrid Systems (ARCH20)},
  editor    = {Goran Frehse and Matthias Althoff},
  series    = {EPiC Series in Computing},
  volume    = {74},
  publisher = {EasyChair},
  bibsource = {EasyChair, https://easychair.org},
  issn      = {2398-7340},
  url       = {/publications/paper/3W11},
  doi       = {10.29007/8dp4},
  pages     = {175-183},
  year      = {2020}}

Download PDF Open PDF in browser