Relevance Filtering Sample Clauses

Relevance Filtering. Rodin's external provers (PP, newPP, and sometimes also ML) tend to perform poorly in the presence of irrelevant hypotheses. For PP and newPP the user can still manually select the hypotheses he considers relevant, but that is a tedious and error-prone process, in particular for large models. Several heuristics for selecting relevant hypotheses have been proposed in the literature[2] [3] [4] [5] . The relevance filter plug-in implements these and other heuristics, and provides a default configuration that has been shown to be almost optimal on a given collection of models from different domains[6] . The relevance filter plug-in has also significantly increased the number of automatically discharged proof obligations on models of deployment partners, which had not been used for fine tuning the heuristics.

Relevance Filtering. The relevance filter heuristics we have considered do not work out of the box - their parameters need to be carefully adjusted. The major design decision concerned how to carry out the process of fine tuning. We started with an ad-hoc benchmark containing models of several problem domains and aimed for maximizing the number of automatically discharged proof obligations among this benchmark while minimizing the amount of time spent for proving. We experimented with different filter configurations, i.e., combinations of heuristics, heuristic parameters, provers (PP, newPP, or ML) and prover timeouts. Finally, the parameters and timeouts were chosen such that • the number of automatically discharged proof obligations is almost maximal among all considered filter configurations, and • decreasing the timeouts would significantly decrease the number of automatically discharged proof obligations. To rebut criticism of overfitting, we tested the final filter configuration on a validation benchmark (based on deployment partners' models), which was chosen independently from the benchmark used for fine-tuning. We observed that the final filter configuration significantly increases the number of automatically discharged proof obligations among the validation benchmark in comparison to not using relevance filtering.