A European Benchmarking Framework for hybrid quantum-classical computing

Expected Outcome:Enhanced evidence-based decision making through comprehensive system comparison that improves the procurement process for quantum-accelerated supercomputers (hybrid HPC-QC). This will enable more informed choices regarding the acquisition of new systems and upgrades of existing onesOverall improved operation and fine-tuning of hybrid HPC-QC systems leading to improved performance, throughput and energy optimization, and improved end-user experienceEnhancing the competitiveness of EU quantum suppliers through transparent benchmarking of industrially relevant workflowsA unified, extensible and well-documented benchmarking framework to easily accommodate new, community-contributed benchmarks with common standards, versioning and controlA well-maintained and continuously updated benchmarking suite for QC- and hybrid HPC-QC workflows with open-source reference implementations for every benchmark, SDK-agnostic specifications and sample datasetsActive contributions to European and global standardisation on QC-, and hybrid HPC-QC benchmarking, certification and sustainability via standardisation bodies such as IEEE P7130/P7131, CEN-CENELEC JTC22-WG3, IEC/ISO JTC3 etc. Scope:A.) Development of a benchmarking framework addressing the following aspects:offer a fine grained and fair comparison methodology among different systems and integration approaches, i.e. all benchmarks, benchmark run rules[1] and benchmark submission rules must be designed to ensure reproducibility, repeatability and replicability of metrics on the same system, ensuring fairness and comparability of metrics across different systemsdefine and continuously update a precise and relevant performance metrics cataloguestandardise all benchmarking input- and output formatscollect and report all benchmarking results while offering statistically sound result analysesensure that all benchmarks are executable on the respective target environment(s)offer a standardized structured workflow capturing