Verified software development in proof assistants

Table of Contents

Use cases of proof assistants

All use cases can be grouped to:

  1. formalizing mathematics, mathematical foundations, physics, economics, etc
  2. verification of algorithms, models, specifications, software, hardware, certification, etc
  3. control of AI results, improvement of AI results, etc
  4. software development, hardware development, etc

4-th - software development is in the focus of this article. It is not just proving, verifying of some properties of some theories, models, algorithms, etc - IT IS CREATION OF SOFTWARE. Yes, theorem provers, proof assistants (lets call them PA) can be used to... WRITE SOFTWARE... and they are already used today -

you can start with "The Best New Programming Language is a Proof Assistant".

How does it happen? Mostly - via compilation and extraction:

                  special language
 (sometimes allows traditional programming language)
                         |
                         v
                  proof assistant
                developing software
                         |
           .-------------+-------------.
           |                           |
           v                           v
      compilation                  extraction
          to                           to
      executable                some programming
                                    language

Some of PA support compilation, some - extraction, some - both. Some of them can execute the PA code directly (eg, Why3), some of them understand more than 1 input language, including traditional programming languages (Why3 is such PA!).

The ultimate idea is that you get very safe, verified software which is safe not bcs it uses runtime mechanisms like automatic memory control, garbage collectors, resource managers existing in high level languages like Java, Python, etc, and working in runtime, opposite: the code can be very low level: assembler, C with manual control of all resources but safe enough bcs it was verified at "COMPILE" TIME and it was qualified as correct.

I will try to classify them all in the context of software development using AI, so maybe tables are not fully completed - check always the documentation of PA - they often add new features - example, I just checked the backend for Rocq (Coq) called "CertiCoq" and voila - it supports extraction to C subset (Clight) and... WebAssembly:

CertiRocq (formerly CertiCoq) is a compiler for Gallina, the specification language of the Rocq Prover. CertiRocq targets WebAssembly and Clight, a subset of the C language that can be compiled with any C compiler, including the CompCert verified compiler.

Supported languages

Notes on the table:

  • Proof assistant: primarily an interactive theorem prover, dependent type theory, higher-order logic system.
  • Verification language: primarily a programming language in which specifications and proofs are attached to executable programs.
  • Extraction: computational content is extracted from verified definitions, proofs, normally erasing logical, ghost material.
  • Code generation: verified, executable definitions are translated to another programming language.
  • Compilation: the system compiles its own language to executable code, rather than necessarily producing a human-oriented target-language program.
  • Proof, executable connection: how directly the generated executable implementation is connected to the object that was formally verified.
System Category Source programming language Target extracted language(s) Verification style Extraction, generation mechanism Proof, executable connection
Rocq (Coq) Proof assistant Gallina OCaml, Haskell, Scheme, JSON CIC, dependent type theory Program extraction Direct extraction: computational content of Gallina terms, proofs is extracted; proofs can be erased
Lean 4 Proof assistant + programming language Lean C, then native executable Dependent type theory Compiler, code generation Direct: Lean executable definitions are compiled; proof, specification material can be erased
Isabelle/HOL Proof assistant Isabelle/HOL SML, OCaml, Haskell, Scala Higher-order logic Isabelle code generator Direct for executable HOL definitions: logical definitions are translated to executable code
Agda Proof assistant, dependently typed language Agda Haskell, JavaScript Dependent type theory Compiler backends Direct computational compilation; irrelevant fields, arguments can be erased
F* Proof-oriented programming language F* OCaml, F#; C/Rust via Pulse + KaRaMeL; assembly via Vale Dependent, refinement types, SMT, effect system Extraction, code generation Strong: verified F* programs can be extracted; low-level paths provide additional verification machinery
Why3 Deductive verification platform WhyML, MicroPython, MicroC, MLCFG, Coma OCaml, C, Java Hoare logic, SMT-backed deductive verification Why3 extract Strong: WhyML program is verified and then extracted; C has a restricted supported subset
Dafny Verification language Dafny C#, Java, JavaScript, Go, Python, Rust, C++ SMT-based contracts, invariants, termination Source-to-source compiler Strong but different from classical extraction: verified Dafny program is translated to target source
ACL2 Theorem prover, programming environment ACL2, Common Lisp C, Java First-order logic, executable Lisp ATC, ATJ Very strong: ATC is explicitly proof-generating C code generation
Viper Verification infrastructure Viper intermediate language Primarily verification IR rather than a general-purpose extraction target Permission-based separation logic Verification-oriented translation Not primarily an extraction system
Vale Verification language, framework Vale Assembly, low-level code SMT-backed low-level verification Verified assembly generation Very strong: designed for verified low-level cryptographic, security code
K Framework Semantics, verification framework K definitions Depends on backend; LLVM, JVM, JavaScript ecosystems Reachability logic, executable semantics Semantics-driven code generation, execution Varies: primarily semantics and verification rather than classical extraction
Event-B, Rodin Formal method Event-B Usually manually implemented code Set theory, refinement Refinement, code-generation extensions Indirect: formal model is refined toward implementation
B Method, Atelier B Formal method B, Event-B family Ada, C, etc. depending on toolchain Set theory, refinement, proof obligations Refinement + code generation Strong refinement connection, but not classical proof-term extraction
PVS Proof assistant PVS specification language Lisp, generated executable forms Higher-order logic, dependent types Code generation, evaluation Limited compared with Rocq: primarily a specification, proof environment
HOL4 Proof assistant Standard ML + HOL SML Higher-order logic Code generation, evaluation Direct for executable HOL definitions
HOL Light Proof assistant OCaml + HOL OCaml-oriented execution Higher-order logic Evaluation, derived executable definitions More proof, evaluation oriented than multi-target extraction
NuPRL Proof assistant Computational type theory Lisp, extracted computational content historically Constructive type theory Program extraction Direct computational interpretation
Minlog Proof assistant Minlog language Scheme Constructive type theory Program extraction Direct: realizers, computational content extracted from constructive proofs
Isabelle/UTP etc. Proof framework Isabelle theories SML, OCaml, Haskell, Scala when definitions are executable HOL, object logics Isabelle code generator Depends on the object logic and executable definitions

PA extracting the largest number of languages: Dafny, Why3, F*, Isabelle... Dafny is especially unusual because it has a large number of source-language backends.

Which PA system can eventually give me code in language X?

Target Systems
C F* + Pulse + KaRaMeL, Why3, ACL2/ATC, Stainless/GenC, Dafny (limited/rudimentary backend), Vale-related workflows
C++ Dafny
C# Dafny
Java Dafny, Why3, ACL2/ATJ
JavaScript Dafny, Agda
Python Dafny
Go Dafny
Rust F* + Pulse + KaRaMeL, Dafny, Verus (native Rust)
OCaml Rocq, Isabelle, F*, Why3
Haskell Rocq, Isabelle, Agda
Scheme Rocq, Minlog
SML Isabelle, HOL4
Scala Isabelle
F# F*
WebAssembly F* ecosystem + low-level toolchains
Assembly Vale + F* ecosystem

So, the most popular languages for extraction are Ocaml and C.

Classical extraction systems

Use-case: "I prove a theorem, define a constructive algorithm, and the system removes the proof-only material and gives me executable source code." - then the particularly relevant systems are:

System Classical extraction? Main targets
Rocq (Coq) YES OCaml, Haskell, Scheme
Agda YES-ish, compilation of computational content Haskell, JavaScript
Minlog YES Scheme
NuPRL YES Historically Lisp-oriented computational extraction
Isabelle/HOL Code generation rather than Coq-style extraction SML, OCaml, Haskell, Scala
Lean Compilation rather than classical extraction Native executable via compiler or C backend
F* Extraction, code generation OCaml, F#, C, Rust through ecosystem
Why3 Program extraction OCaml, C

Rocq is particularly clear-cut: its extraction mechanism explicitly supports OCaml, Haskell and Scheme.

Why3 is similarly explicit: the command why3 extract generates OCaml, and its C driver can generate C for a supported subset of WhyML.

Verified output vs verified source

Level Meaning Examples
L0 No executable-code generation Viper-like verification infrastructure
L1 Compile verified source normally Verus, SPARK, Liquid Haskell
L2 Generate executable source from verified definitions Dafny, Stainless, Isabelle, Why3
L3 Proof, program extraction Rocq, Minlog, constructive type-theory systems
L4 Proof-producing low-level generation ACL2, ATC, parts of F* with KaRaMeL/Vale

Specific runtime

The key distinction is between standalone generated code and code that depends on a language-specific runtime/library. Also, "no special runtime" does not necessarily mean "zero dependencies": e.g. generated C may use the C standard library or a small support library.

System Typical target Special runtime required? Generated code usable independently? Notes
Rocq/Coq OCaml, Haskell, Scheme - Usually no Rocq runtime Yes Extracted program is ordinary target-language code. It may depend on ordinary target libraries.
Lean 4 C/native + Lean runtime generally required No, not completely Lean's generated native code relies on Lean's runtime support for things such as memory management and runtime representations.
Isabelle/HOL SML, OCaml, Haskell, Scala - Generally no Isabelle runtime Yes Generated code is intended to be executable independently of Isabelle; target-language libraries/runtime still apply.
Agda Haskell, JavaScript + Target-language/runtime libraries Mostly yes Haskell output runs under GHC/Haskell runtime; JS output needs the normal JS environment and any generated support. No Agda theorem-prover runtime is normally required.
F* OCaml, F#, C, Rust - Depends on backend Often yes OCaml/F# can be ordinary target programs; C/Rust extraction can use generated/support libraries, depending on the F\*/KaRaMeL setup.
Why3 OCaml, C - Usually no Why3 runtime Yes One of the cleaner cases: extraction produces normal OCaml/C code. External libraries can of course be required.
Dafny C#, Java, JS, Go, Python, etc. + Yes, commonly Usually no Generated programs use Dafny runtime/support libraries for Dafny types and operations. This is one of the clearest examples of runtime-dependent generation.
ACL2/ATC C - Small/support runtime may be needed depending on features Largely yes ATC is specifically designed to generate C; generated code can be very close to conventional standalone C.
ACL2/ATJ Java ± ACL2-specific support may be required Not always Depends on which ACL2 constructs/features are translated.
Stainless Scala, C ± Backend/runtime support depends on generated features Often yes Generated Scala is normal Scala; GenC can require Stainless support/runtime for some constructs.
Verus Rust - No Verus runtime Yes `spec`/`proof`/ghost material is erased; executable Rust is compiled normally. This is a particularly clean model.
SPARK Ada - No SPARK runtime in the conceptual sense Yes SPARK is a subset/methodology over Ada. Verified Ada is compiled using normal Ada toolchains/runtime as appropriate.
Liquid Haskell Haskell - No Liquid Haskell runtime Yes Refinements disappear; resulting program is ordinary Haskell.
Vale Assembly - No Vale runtime for the generated assembly itself Yes Designed specifically for low-level code; generated/verified routines can be incorporated into normal systems.
K Framework Depends on backend ± Usually backend-specific Depends K is primarily semantics/execution/verification infrastructure rather than a simple extraction pipeline.
Event-B/Rodin Various ± Depends on code-generation toolchain Depends Refinement/code-generation approach rather than classical proof extraction.
B Method Ada, C, etc. ± Depends on toolchain Usually yes Generated implementation can become ordinary target-language code.
PVS Various - Not primarily an extraction system Depends Code generation is not its central use case.
HOL4 SML - No HOL runtime normally required Yes Generated SML can be compiled/executed as ordinary SML.
Minlog Scheme - No Minlog runtime in the classical model Yes Constructive proof → extracted Scheme program.
NuPRL Lisp-oriented ± Historically implementation-dependent Depends More of a historical/academic example of computational extraction.

Also they can be grouped in this sense as:

Category Systems What you get
A. Truly independent extraction Rocq, Why3, Isabelle, Minlog, HOL4 Ordinary target-language source; no theorem-prover runtime is fundamentally part of the executable
B. Normal language compilation, proofs erased Verus, SPARK, Liquid Haskell, Vale The executable is essentially an ordinary Rust/Ada/Haskell/assembly program
C. Generated code + project/runtime support Dafny, Lean, some F_/Stainless configurations, ACL2/ATJ Generated source is usable, but some language/runtime support is normally carried along
D. Backend/toolchain dependent K, Event-B, B, PVS Whether the final artifact is standalone depends substantially on the particular backend/toolchain

The interesting cases

Dafny

This is the model you're probably thinking about:

       Dafny source
            |
     <verification> ----> Z3, ...
            |
      Dafny compiler
            |
   .--------+--------.
   |        |        |           Dafny runtime
  C#      Java     Python ...          |
   |        |        |                 |
   `--------+--------'                 |
            |                          |
            +--------------------------'
            |
            v
        executable

That's an important distinction if your objective is "I want to give the generated source to somebody who doesn't have Dafny installed."

They generally don't need the Dafny compiler, but they may need the Dafny runtime/support package.

Rocq (Coq)

Rocq is much closer to what I'd call classical extraction:

         Gallina
            |
    <theorems proving>
            |
       <extraction>
            |
    +-------+-------+
    |       |       |
    v       v       v
 OCaml   Haskell   Scheme
    |       |       |
    v       v       v
<compile w/o dependencies>

The resulting OCaml, for example, doesn't conceptually need Coq/Rocq to execute. The theorem prover is not part of the runtime architecture.

That is a major advantage if your goal is deployment.

Why3

Why3 is another particularly interesting one:

              WhyML
                |
       +--------+--------+
       |                 |
  verification       extraction
       |                 |
  SMT solvers       +----+----+
                    |    |    |
                 OCaml   C   Java

The verification infrastructure is not needed to execute the resulting program. That makes it quite attractive for the: "Prove it here, deploy it somewhere completely unrelated." model.

Verus

Verus is perhaps even cleaner conceptually, because executable code is already Rust:

fn verified_algorithm(...) -> ... {
    ...
}

----- alongside -----

spec fn ...
proof fn ...

The specification/proof portions are ghost material.

Conceptually:

     Rust + proofs
          |
    Verus verifier
          |
          v
   erase ghost code
          |
          v
     ordinary Rust
          |
        rustc
          |
          v
    native binary

There is no Verus runtime that the deployed program needs. That's a very different architecture from Dafny.

Garbage collection

If your concern is not merely "special verifier runtime" but also "I want predictable systems code with no GC", the classification changes substantially.

System/target Typical GC? Suitable for GC-free systems code?
F* -> C No tracing GC required by C itself YES
Why3 -> C No YES
ACL2 -> C No YES
Verus -> Rust No tracing GC YES
Vale -> assembly No YES
SPARK -> Ada No tracing GC normally YES
Rocq -> OCaml Yes Usually no for hard real-time/GC-free requirements
Isabelle -> OCaml Yes Usually no
F* -> OCaml Yes Usually no
Rocq -> Haskell Yes Usually no
Agda -> Haskell Yes Usually no
Liquid Haskell Yes Usually no
Dafny -> C# Yes (.NET GC) Usually no
Dafny -> Java Yes (JVM GC) Usually no
Dafny -> Python Yes/managed memory No for this requirement
Lean native Runtime-managed Depends heavily on application, but not equivalent to bare C

Important distinction - 4 different questions:

  1. Does the target language normally have a garbage collector?
  2. Does the generated program require a GC?
  3. Does the generated program require a special runtime supplied by the verifier?
  4. Can the generated program use manual, ownership, region-based memory management instead?

These must not be conflated.

For example, C has no built-in GC:

C program
    |
    +--> malloc/free
    |
    +--> custom allocator
    |
    +--> tracing GC library

And all 3 are perfectly legitimate C programs.

Therefore: "generates C" does not automatically means "does not use GC".

Also distinct "GC added by PA" and "GC pre-existing in the target language", e.g.:

                 WhyML
                   |
              verification
                   |
          +--------+--------+
          |                 |
        OCaml               C
          |                 |
      OCaml GC      explicit C memory
          |                 |
          v                 v
      executable        executable

So, for Why3 -> OCaml you naturally get OCaml's GC, bcs you cannot remove the OCaml's GC from OCaml. But for extraction of C - you don't get any GC from Why3.

F* (Low*, KaRaMeL)

F* is one of the most interesting systems for this particular question.

Low* is explicitly designed to model C-like low-level computation. Its libraries model:

  • stack allocation
  • heap allocation
  • C arrays
  • machine integers
  • C memory
  • system-level C functionality

and KaRaMeL erases proofs before generating C (comparing to Why3 which can keep runtime asserts, if you need such ones).

Conceptually:

F*/Low*
    |
    | proofs specifications
    |
    v
verification
    |
    | erase proofs
    |
    v
Low-level implementation
    |
    v
    C
    |
    +--> stack
    |
    +--> heap
    |
    +--> explicit allocation
    |
    +--> no GC required

This is fundamentally different from:

high-level functional language
             |
             v
             C
             |
             v
    generated GC runtime

Low* is specifically trying to model the C memory behavior rather than hide it behind a tracing collector.

However, F* as a whole should not be labeled "no GC" bcs:

F*
 |
 +--> OCaml backend       -> OCaml GC
 |
 +--> F# backend          -> .NET GC
 |
 +--> Low*/KaRaMeL -> C   -> can be GC-free
 |
 +--> other backends      -> depends

So F* can be treated as "backend-dependent" classification.

Dafny

Dafny is almost the opposite example. Supposing Dafny -> C++ it does not follow that you get "normal manual-memory C++". Dafny has its own runtime, support infrastructure, and generated C++ can depend on that infrastructure.

Therefore:

  Dafny
   |
   v
  C++
   |
   +--> Dafny runtime
   |      |
   |      +--> runtime representations
   |      |
   |      +--> collections
   |      |
   |      +--> memory management
   |
   v
program

The generated C++ can therefore effectively have managed-memory semantics even though C++ itself has no mandatory tracing GC.

Isabelle/HOL

Isabelle's code generator produces SML, OCaml, Haskell and Scala.

Therefore the memory-management story is inherited largely from the target:

Isabelle/HOL
 |
 +--> SML      -> SML runtime/GC
 |
 +--> OCaml    -> OCaml GC
 |
 +--> Haskell  -> GHC GC
 |
 +--> Scala    -> JVM GC

So Isabelle doesn't need an its own "Isabelle GC", but the generated program is certainly not necessarily GC-free.

Agda

Agda is similar:

Agda
 |
 +--> Haskell     -> GHC GC
 |
 +--> JavaScript  -> JS engine GC

Lean

Lean is different.

Lean's executable model includes a runtime system, and its compiler targets native code through its compilation infrastructure.

A more accurate model for Lean is:

    Lean
     |
     v
Lean compiler
     |
     v
native code
     |
     +--> Lean runtime
              |
              +--> runtime representations
              |
              +--> memory management
              |
              +--> GC/runtime machinery

ACL2/ATC

ACL2's ATC is interesting because it generates C and is explicitly designed as a proof-generating C code generator.

The conceptual model is:

       ACL2
        |
        | proof
        |
        v
       ATC
        |
        v
        C
        |
        v
ordinary C compiler

ATC is much closer to: "generate C that follows the proved semantics" than to: "generate C plus a large managed-language runtime".

So GC is not normally intrinsic for ACL2.

Of course, nothing prevents the generated C program from linking against a GC library if the application requires one.

Vale

Vale operates at a much lower level - Vale is a DSL/toolchain for verified ASSEMBLY (ARM, x86?) for F* and Dafny, see more. The project today does not look as too alive though.

Its generated/verified code can be incorporated into systems software without a mandatory tracing GC.

But, just as with C or assembly generally:

"no mandatory GC"
       !=
"GC impossible"

You can implement a GC in assembly.

The relevant question is whether the Vale-generated code requires one. Normally, no.

HOL4

HOL4 can generate SML, so its extracted code uses GC of SML (not specially generated by HOL4).

A better overall classification

Category Systems GC status
GC-free by design for the main low-level target Verus → Rust, F*/Low* → C, Why3 → C, SPARK → Ada, Vale → assembly No tracing GC required by the normal architecture
Generated target inherits a GC Rocq → OCaml, Rocq → Haskell, Isabelle → OCaml/Haskell/SML/Scala, Agda → Haskell, Liquid Haskell → Haskell, HOL4 → SML, Minlog → Scheme GC normally present
Special runtime/managed execution model Dafny, Lean GC/managed memory can be part of runtime architecture
Backend-dependent F*, Stainless, ACL2, B/Event-B, K, PVS Must inspect particular backend
Not primarily executable-code generators Viper, some PVS/K workflows GC question is not directly applicable
Can deliberately verify a GC itself Rocq/CertiCoq ecosystem, verified-GC projects GC can itself be a formally verified component

There are actually 3 independent axes:

                       GENERATED PROGRAM
                              |
             +----------------+----------------+
             |                                 |
          managed                          explicit
             |                               memory
             |                                 |
       +-----+-----+                    +------+------+
       |           |                    |             |
      GC       RC/other              malloc,       ownership,
       |                              free         regions
       |
       v
runtime support

So "GC or no GC" is not the whole story.

For systems programming, this it a richer classification:

Memory model Examples Main characteristic
Tracing GC Dafny managed targets, Lean runtime, OCaml, Haskell, JVM/Scala Automatic object reclamation
Reference counting Possible in generated/runtime code Deterministic reclamation but reference cycles need handling
Ownership/borrowing Verus/Rust Compile-time resource management
Manual allocation Low*/C, Why3/C, ordinary C Programmer/program proof controls allocation/deallocation
Regions/arenas Low-level verified systems, some functional implementations Lifetime is controlled by region
Stack/static SPARK, Low* portions No general heap required
Hybrid F*, Stainless, custom runtimes Different types/data structures can use different strategies
Verified GC CertiCoq/verified-GC research GC exists, but its behavior can itself be formally verified

The surprising case: C + verified GC

This is why the CertiCoq ecosystem is particularly interesting.

It is possible to have:

               Rocq
                 |
           proof/program
                 |
             CertiCoq
                 |
                 v
             generated C
                 |
        +--------+--------+
        |                 |
   application        verified GC
        |                 |
        +--------+--------+
                 |
                 v
           machine code

So there is nothing contradictory about:

      "C code"
         +
"garbage collection"
         +
"formally verified"

C doesn't prohibit GC; it simply doesn't provide one as part of the language.

Reduce the enormous table:

System Target GC mandatory? Special runtime mandatory? Explicit memory possible? Ownership/regions possible? Particularly suitable for systems code
F*/Low* C NO NO in the Low/C model YES YES/modeled regions YES
Why3 C NO NO YES Limited/model-dependent YES
ACL2/ATC C NO normally NO normally YES C-style YES
Verus Rust NO NO Controlled by Rust YES YES
SPARK Ada NO normally NO YES YES YES
Vale Assembly NO NO YES Explicit low-level YES
Dafny C++/other Potentially/runtime-managed YES support runtime Abstracted by runtime Not primary model Depends
Rocq/CertiCoq C Can be YES Runtime/support may be required YES Possible YES, especially research/verified-runtime scenarios
F -> OCaml OCaml YES NO F runtime necessarily Abstracted Not the Low model Less suitable for GC-free systems code
Rocq -> OCaml OCaml YES NO Coq runtime Abstracted Not normally the goal Excellent verified functional code
Why3 -> OCaml OCaml YES NO Why3 runtime Abstracted Not normally the goal Excellent verified functional code
Isabelle -> OCaml/Haskell/SML Managed functional targets YES NO Isabelle runtime Abstracted Not normally the goal Excellent verified functional code
Agda -> Haskell Haskell YES NO Agda runtime Abstracted Not normally the goal Excellent dependent functional code

Bottom line

So, yes: there absolutely are provers/formal-verification systems where the generated language is C/C++ but the resulting program can still have a garbage collector.

The cleanest conceptual distinction is:

                   C/C++
                     |
          +----------+----------+
          |                     |
       GC-based             GC-free
          |                     |
   runtime/collector       malloc/free
          |                 ownership
          |                  regions
          |                  arenas
          |                     |
   CertiCoq-style         Low*/KaRaMeL
   / Dafny-style             Why3/C
                            ACL2/ATC

CertiRocq-generated code normally relies on a CertiRocq runtime, and that runtime includes a generational garbage collector.

See: "Memory Model and Garbage Collection - CertiRocq"

And: "Certifying graph-manipulating C programs via localizations within data structures"