Internals of Arithmetic Reasoning in Z3

Tutorial at Alpine Verification Meeting, Timisoara, 2025 https://z3prover.github.io/slides/avm.html

Nikolaj Bjørner
Microsoft Research
Lev Nachmanson
Microsoft Research

Contents

Arithmetic - a fundamental SMT theory

Frog

F*

EverParse

linear and non-linear arithmetic

Certora

  • 256 bit non-linear arithmetic

OpenZeppelinImage

Verus

  • linear arithmetic

AutoVerus

Several other

  • Dafny, Boogie, Viper
  • Dafny - first customer of non-linear arithmetic (Gröbner)
  • Verdis - ZK for prime fields

Many Arithmetic Theories

ArithmeticTheories

Other Fragments

Cardinality, Pseudo-Boolean, finite range LIA theories


Modular arithmetic, finite fields


Bi-linear real arithmetic $\vec{x}A\vec{y} \leq \vec{b}$


Non-unit two-variable per inequality $ax + by \leq c$


At most one (unit) positive variable per inequality (Horn) $x - 3y - 2z \leq 2$. What is the non-unit complexity?


max-atom: $\max(x,y) + k \geq z$. In NP $\cap$ co-NP, local strategy/policy iteration (Costan et al., 2005; Gawlitza and Seidl, 2007; Bezem et al., 2008; Karpenkov et al., 2016; Karpenkov, 2017)


Knapsack theories: all variables with positive coefficients, except for one inequality where all variables have negative coefficients.

Theoretical vs Practical Complexity

  • Conjunctions:
    • RDL, IDL solved by
      • Floyd-Warshal ($V^3$),
      • Bellman-Ford ($V\cdot E$).
    • LRA - in P
    • LIA - NP complete
    • Finite Fields ~ Bit Vectors - NEXPTIME
    • NIA - undecidable
    • Quantified NIA - highly undecidable
  • Clauses:
    • $\geq$ NP hard

CDCL(Arith) by example

BasicTheory6

Solver Architecture (N. S. Bjørner and Nachmanson, 2024)

Arch

LRA - Linear Real Arithmetic

LPArch

LRA

What: Variables range over reals (rationals). Addition, multiplication by constant

How: Dual Simplex

Why: A basis for everything more

\[\begin{mdmathpre}%mdk \mdmathindent{2}\mathid{x},~\mathid{y},~\mathid{z}~\in \mathcal{R}\\ \mdmathindent{2}0~<~\mathid{x}~<~\mathid{y}~<~\mathid{z}~\land \mathid{x}~+~\mathid{y}~+~\mathid{z}~=~1\\ \\ \mdmathindent{2}\mathid{x}~=~1/6,~\mathid{y}~=~1/3,~\mathid{z}~=~1/2 \end{mdmathpre}%mdk \]

Farkas and Dual Simplex (Dutertre and Moura, 2006)

SMT solving based on Dual Simplex maintain a tableau of the form:

\[\begin{array}{ll} x_i = \sum_{x_j \in \nonbasicvars} a_{ij} x_j & x_i \in \basicvars \\ l_j \leq x_j \leq u_j & x_j \in \nonbasicvars \cup \basicvars \\ val : x_j \mapsto {\mathcal R} & \forall x_j \in \nonbasicvars \\ nval : x_i \mapsto \sum_{x_j \in \nonbasicvars} a_{ij} val(x_j) & \forall x_i \in \basicvars \end{array} \]

where $\basicvars$ are basic and $\nonbasicvars$ are non-basic variables.

Compiling into tableau

  x, y = Reals('x y')
  solve([x >= 0, Or(x + y <= 2, x + 2*y >= 6), 
                 Or(x + y >= 2, x + 2*y > 4)])

Introduce slacks to define terms

\[\begin{mdmathpre}%mdk \mdmathindent{2}\mathid{s}_1~=~\mathid{x}~+~\mathid{y},~\mathid{s}_2~=~\mathid{x}~+~2\mathid{y},~\mathid{s}_1,~\mathid{s}_2~\in \basicvars,~\mathid{x},~\mathid{y}~\in \nonbasicvars \end{mdmathpre}%mdk \]

Formula using slacks:

\[\begin{mdmathpre}%mdk \mdmathindent{2}\mathid{x}~\geq 0,~(\mathid{s}_1~\leq 2~\vee \mathid{s}_2~\geq 6),~(\mathid{s}_1~\geq 2~\vee \mathid{s}_2~>~4) \end{mdmathpre}%mdk \]

Constraints := Tableau Equalities + bounds

Equalities are global definitions

\[\begin{mdmathpre}%mdk \mdmathindent{2}\mathid{s}_1~=~\mathid{x}~+~\mathid{y},~\mathid{s}_2~=~\mathid{x}~+~2\mathid{y},~\mathid{s}_1,~\mathid{s}_2~\in \basicvars,~\mathid{x},~\mathid{y}~\in \nonbasicvars \end{mdmathpre}%mdk \]

Bounds (e.g., $s_1 \leq 2$) are added/deleted during search.

Infeasible Tableau

A tableau is infeasible if there is a row $x_i = \sum_{x_j \in \nonbasicvars} a_{ij} x_j$ such that

\[\begin{array}{lll} & a_{ij} < 0 \implies val(x_j) = l_j, & \forall x_j \\ & a_{ij} > 0 \implies val(x_j) = u_j, & \forall x_j \\ & nval(x_i) < l_i % \\ %or \\ % & a_{ij} > 0 \implies val(x_j) = l_j, & \forall x_j \\ % & a_{ij} < 0 \implies val(x_j) = u_j, & \forall x_j \\ % & nval(x_i) > u_i \end{array} \]

conflict explanation: literals that are used for tight bounds.

LRA is Convex

Convexity: $A\vec{x} \leq b \implies x_1 = y_1 \lor x_2 = y_2$, then $A\vec{x} \leq b \implies x_i = y_i$ for $i = 1$ or $i = 2$.

  • Suppose $val_1$ is a solution to $A\vec{x} \leq b$, then $val_1(x_i) = val_1(y_i)$ for $i = 1$.
  • Similar for solution $val_2$ but with $i = 2$.
  • Set $val_3 := (val_1 + val_2) / 2$.
  • Then $A \cdot val_3(\vec{x}) \leq b$ too and $val_3$ has to agree with either $val_1$ or $val_2$ on implied equality.

LRA - next

  • Z3 uses bignum (infinite precision arithmetic), always.

    • A show-stopper for LP centric applications (Marabou, traffic engineering)
    • Many scenarios can be solved at fixed precision or floats with guarantees (SCIP).

  • Z3 also contains difference logic solvers. They are used sparingly for SMTLIB categories IDL/RDL.

  • Integration of policy/strategy iteration?

Strategy/Policy iteration (Costan et al., 2005)

Goal is to find minimal $x_i$ such that:

\[\begin{mdmathpre}%mdk \mdmathindent{3}\mathid{x}_\mathid{i}~\geq \mathid{e}_\mathid{i}~~\mathid{for}~\mathid{each}~\mathid{x}_\mathid{i}\\ \mdmathindent{3}\mathid{where}~\mathid{e}~::=~\mathid{x}_\mathid{j}~|~\mathid{a}\cdot \mathid{e}~|~\mathid{e}~+~\mathid{e}~|~\min(\mathid{e},\mathid{e})~|~\max(\mathid{e},\mathid{e})\\ \mdmathindent{3}\mathid{where}~\mathid{a}~>~0~\mathid{is}~\mathid{a}~\mathid{constant} \end{mdmathpre}%mdk \]


Policy Initially $\pi$ replaces $\min(e_1,e_2)$ by $e_1$ (arbitrary choice).


Solve the resulting unit-Horn system with solution $M$.


Evaluate each $\min(e_1, e_2)$ subterm wrt. $M$ to check if it preserves $\min$.


Repeat For each $\min(e_1, e_2)$, if $M(e_1) > M(e_2)$, but $\pi(\min(e_1,e_2)) = e_1$, then update $\pi(\min(e_1,e_2)) := e_2$.

Bounds and Equality Propagation

BoundsArch

Bounds

What: Extract Boolean propagation axioms from arithmetic.

How: Index bounds atoms that are created and infer bounds from tableau.

Why: It is much more efficient to have the SAT solver propagate (binary) clauses than theory reasoning.

\[\begin{mdmathpre}%mdk \mdmathindent{3}(\mathid{x}~\geq 1~\lor \mathid{P})~\land (\mathid{x}~\leq 0~\lor \mathid{Q})~\land \ldots \end{mdmathpre}%mdk \]

If $x \geq 1$ is asserted, then directly propagate $x \leq 0$ to false.

LP Bounds Propagation

  • For atoms $x \geq 3, x \geq 2, x \geq 1$ add axioms $x \geq 3 \implies x \geq 2,\ \ \ \ x \geq 2 \implies x \geq 1$.

  • Infer new bounds of variables: $x - 2y = 0, y \geq 1 \implies x \geq 2$.

    • For every tableau row $r$,
      • For every variable $x$ in row $r$,
        • Check if bounds of other variables strengthen current variable bound,
        • Use implied bound for T-propagation of bounds atoms.

Equalities

What: Extract equalities between variables and propagate equalities to other theories.

How: By cheap tests on the tableau.

Why: Useful to propagate congruences for arithmetic variables used under non arithmetical function symbols.

\[\begin{mdmathpre}%mdk \mdmathindent{4}\mathid{x}~\leq \mathid{y}~+~1~\land \mathid{z}~\geq \mathid{y}~+~1~\land \mathid{z}~\leq \mathid{x}~\land \mathid{p}(\mathid{x})~\land \neg \mathid{p}(\mathid{z})\\ \\ \mdmathindent{4}\leadsto \mathid{x}~=~\mathid{z}~~\\ \mdmathindent{4}\leadsto \mathid{p}(\mathid{x})~\land \neg \mathid{p}(\mathid{x}) \end{mdmathpre}%mdk \]

Finding implied equalities

\[\begin{mdmathpre}%mdk \mdmathindent{3}\mathid{x}~+~\mathid{u}~\leq \mathid{z}~\land \mathid{z}~-~1~\leq \mathid{y}~\land \mathid{y}~\leq \mathid{x}~\land 1~\leq \mathid{u}~\leq 1 \end{mdmathpre}%mdk \]

Solved tableau:

\[\begin{mdmathpre}%mdk \mdmathindent{4}\mathid{x}~=~\mathid{z}~-~\mathid{u}~-~\mathid{s}_1,~\mathid{y}~=~\mathid{z}~-~\mathid{u}~-~\mathid{s}_2,~\\ \mdmathindent{4}1~\leq \mathid{u}~\leq 1,~0~\leq \mathid{s}_1~\leq 0,~0~\leq \mathid{s}_2~\leq 0 \end{mdmathpre}%mdk \]

assignment $z = 0, x = y = -1$. Necessary condition for $x = y$. Sufficient (incomplete) condition:

  • $x$ is basic, and the tableau has row $x - y + \alpha = 0$,
  • $x, y$ are connected through a non-basic variable $z$ in a pair of the tableau rows in one of the following forms
    1. $x - z + \alpha = 0, y - z + \alpha' = 0$,
    2. $x + z + \alpha = 0, y + z + \alpha' = 0$,

$\alpha, \alpha'$ are linear combinations of fixed variables. We never have to calculate $\alpha$!

Complete Equality Propagation

Will this method find all implied equalities?

Propagating Equality Assumptions

If $x, y$ are shared variables and $val(x) = val(y)$, but the equality between $x, y$ has not been propagated. Then we can assume the equality $x \simeq y$ atom. It is satisfied in the theory of arithmetic, and the equality atom will propagate to other theories that have to reconcile on $x = y$.

\[\begin{mdmathpre}%mdk \mdmathindent{2}0~\leq \mathid{x},~\mathid{y},~\mathid{z}~\leq 1~\land \mathid{x},~\mathid{y},~\mathid{z}~\in \mathcal{Z}~\land \\ \mdmathindent{2}\mathid{P}(\mathid{x})~\land \neg \mathid{P}(\mathid{y})~\land \mathid{Q}(\mathid{y})~\land \neg \mathid{Q}(\mathid{z})~\land \neg \mathid{S}(\mathid{x})~\land \mathid{S}(\mathid{z}) \end{mdmathpre}%mdk \]

If $val(x) = val(y) = 0$ then assume $x \simeq y$: conflicts with $P(x) \land \neg P(y)$.

Propagating Equality Assumptions (II)

  • Arithmetic solver: $x = y = 0, z = 1$

  • Add atom $x \simeq y$ to search space and assign it to true.

  • EUF solver $x \simeq y \land P(x) \land \neg P(y) \land Q(y) \land \neg Q(z) \land \neg S(x) \land S(z)$

  • Conflict: $x \simeq y \land P(x) \land \neg P(y)$

  • Lemma $x \not\simeq y$

  • Arithmetic solver: $x = 0, y = z = 1$.

  • New lemma $y \not\simeq z\ldots$

Propagation - next

  • Z3 exposes solve_for API function to extract all equalities through Gaussian elimination.

    • If variables are equal modulo current $val$ it performs a local solver call to check if disequality is feasible.

  • Integrate stronger inequality propagation eagerly?

    • Such as bounds tightening.

Special Arithmetic

Difference Arithmetic

(set-logic QF_IDL) ; optional in Z3
(declare-fun t11 () Int)
(declare-fun t12 () Int)
(declare-fun t21 () Int)
(declare-fun t22 () Int)
(declare-fun t31 () Int)
(declare-fun t32 () Int)

(assert (and (>= t11 0) ...))
(assert (and (>= t21 0) ...))
(assert (and (>= t31 0) (>= t32 (+ t31 2)) (<= (+ t32 3) 8)))
(assert (or ... (>= t21 (+ t11 2))))
(assert (or (>= t21 (+ t31 2)) ...))
(check-sat)
(get-model) ; display the model

Bellman-Ford

Solve difference logic using graph $O(E\cdot V)$ Bellman-Ford network flow algorithm. Negative cycle $\Rightarrow$ unsat.

BellmanFord

Convexity and Complexity

  • IDL poly time but is not convex. $0 \leq x, y, z \leq 1$: two out of three have to be equal.

  • UF is poly time.

  • Quiz [Vaughan Pratt]: The combination IDL+UF is NP hard.

Bit-vectors as arithmetic (Zohar et al., 2022)

Add bit-vector operators together with on-demand expansion.

\[\begin{mdmathpre}%mdk \mdmathindent{3}\mathsf{bvand}(\mathid{x},~\mathid{y},~\mathid{N})~~\mbox{bit-wise conjunction of $x, y$ defined for $[0, 2^N-1]$}\\ \mdmathindent{3}\displaystyle \mathsf{bvand}(\sum_\mathid{i}~2^\mathid{i}~\mathid{x}_\mathid{i},~\sum_\mathid{i}~2^\mathid{i}~\mathid{y}_\mathid{i},~\mathid{N})~=~\sum_{\mathid{i}~=~0}^{\mathid{N}-1}~2^\mathid{i}~(\mathid{x}_\mathid{i}\cdot \mathid{y}_\mathid{i})~ \end{mdmathpre}%mdk \]

  • Native arithmetic operators and decision procedures for bit-wise and, shift left, shift right logical and shift right arithmetical.

  • Quiz: express bitwise negation.

Bit-vectors as arithmetic in Z3.

  • Eager axioms:

    • $0 \leq \mathsf{bvand}(x, y, N) < 2^N$
    • $0 \leq x \implies \mathsf{bvand}(x, y, N) \leq x$.

  • On-demand axioms - the rest

    • based on current LIA model.

Pseudo-Booleans

  • Pseudo-Boolean constraints are treated as a theory.
  • There are two implementations: one in the main SMT core, and one as part of a tuned SAT solver.
  • The PB theory solvers perform Boolean propagation on cardinality and PB constraints.
  • Conflict analysis enabled by hijacking the SAT solver conflict analysis process.

EUF or Arithmetic?

An instance from a plant capacity problem:

  • Number of Stations = O(1K)
  • Number of Tasks = O(10K)
  • Up to O(10) different operators per station

Direct MIP-style encoding: $t_{i,s,op}$ - Task $i$ is at station $s$ attended by operator $op$.

\[\begin{mdmathpre}%mdk \mdmathindent{3}10\mathid{K}~\times 1\mathid{K}~\times 10~=~100\mathid{M}~\mbox{variables} \end{mdmathpre}%mdk \]

Instead use EUF $\mathit{station}(i) = s, \mathit{operator}(i) = op$, $\mathit{taskof}(s, op) = i$.

Scales encoding, but loses propagation efficiency.

Claim: Ackerman reduction simulate indicator variables.

LIA - Linear Integer Arithmetic

LIAArch

LIA

What: Variables range over Integers. Addition, multiplication by constant.

How: Conservative: If unsat in LRA then unsat in LIA. Cuts, Branch and Bound.

Why: A practical basis for program verification (few verification systems use bit-vectors) and many applications.

\[\begin{mdmathpre}%mdk \mdmathindent{1}\mathid{x},~\mathid{y},~\mathid{z}~\mathid{in}~\mathcal{Z}\\ \mdmathindent{1}0~<~\mathid{x}~<~\mathid{y}~<~\mathid{z}~\land \mathid{x}~+~\mathid{y}~+~\mathid{z}~=~1~\leadsto \mathid{unsat}\\ \mdmathindent{1}2\mathid{x}~+~6\mathid{y}~+~3~=~0~\leadsto \mathid{unsat} \end{mdmathpre}%mdk \]

From LRA to LIA

When can a solution to LRA be used for LIA?

  • Patching and Cubes: If assignment from LRA is already integral or can be fixed.

  • GCD Test and bounds tightening: Detect when there are no integer solutions.

  • Cuts and branches: Block current LRA solution.

Patching (Moura and Bjørner, 2008)

\[\begin{mdmathpre}%mdk \mdmathindent{1}\mathid{y}~-~\frac{1}{2}~\mathid{x}~=~0~~~~&~~~~~~~~~~~~\\ \mdmathindent{1}\mathid{z}~-~\frac{1}{3}~\mathid{x}~=~0~~~~&~~~~~~~~~~~\\ 3~\leq \mathid{x}~\leq 10,~-3~\leq \mathid{y}~\leq 4,~-4~\leq \mathid{z}~\leq 12~\\~ \end{mdmathpre}%mdk \]

\[\begin{mdmathpre}%mdk \mdmathindent{2}\mbox{Initial assignment:}~~~~~~~~&~~~\mathid{x}~:=~3,~\mathid{y}~:=~\frac{3}{2},~\mathid{z}~:=~1~\\ \mdmathindent{2}\mbox{Move $x$ from $3$ to $8$:}~~&~~~\mathid{x}~:=~8,~\mathid{y}~:=~4,~\mathid{z}~:=~\frac{8}{3}\\ \mdmathindent{2}\mbox{Move $x$ from $3$ to $6$:}~~&~~~\mathid{x}~:=~6,~\mathid{y}~:=~3,~\mathid{z}~:=~2 \end{mdmathpre}%mdk \]

Patching (N. Bjørner and Nachmanson, 2018)

CAV2024slide

Cubes (Bromberger and Weidenbach, 2016,Bromberger and Weidenbach (2017))

Proposition: If $Ax \leq b - \frac{1}{2} \onenorm{A}$ has a solution over the reals, then $Ax \leq b$ has an integer solution obtained by rounding.

Example: Given

\[3x + y \leq 9 \wedge - 3y \leq -2 \]

Solve instead

\[3x + y \leq \underbrace{9 - \frac{1}{2}(3 + 1)}_{7} \wedge -3y \leq \underbrace{-2 - \frac{1}{2}3}_{-3.5} \]

Real solution $y = \frac{7}{6}, x = \frac{35}{18}$. Then $y = 1, x = 2$ is an integer solution.

Cubes (N. Bjørner and Nachmanson, 2018)

Observation: One can often avoid strengthening inequalities.


bounds on variables $x \leq 4$.


differences between variables $x - y \leq 6$ need not be strengthened to $x - y \leq 5$.


octagon inequalities $x + y \leq 5$ can be strengthened to $x + y < 5$, or solutions can be patched.


unit horn inequalities $x \geq 5y + 6z + 3$

GCD Tests

Basic GCD: $5/6x + 3/6y + z + 5/6u = 0$ is infeasible, if

  • There are fixed variables: $2 \leq x \leq 2, -1 \leq u \leq -1$
  • After replacing fixed variables by values: $5 + 3(y + 2z) = 0$
  • $3 \not\uparrow 5$.

Extended GCD: $a x + b y + c = 0$ is infeasible if

  • $a < b$
  • $lb \leq x \leq ub$ is bounded

  • $\lfloor \frac{a \cdot ub}{\gcd(b,c)} \rfloor < \lceil \frac{a \cdot lb}{\gcd(b,c)} \rceil$

Example:

  • $3x + 7y + 14 = 0$, $1 \leq x \leq 2$
  • $0 = \lfloor \frac{3\cdot 2}{7} \rfloor < \lceil \frac{3 \cdot 1}{7} \rceil = 1$.

Gomory cuts

  • Z3's implementation is based on (Dutertre and Moura, 2006).
  • Applies generally to mixed integer problems.
  • Some value to finding cuts with small coefficients.

Bounds Tightening

\[\begin{mdmathpre}%mdk \mdmathindent{9}&~~2\mathid{x}~+~4\mathid{y}~+~\mathid{z}~\leq 5,~\mathid{z}~\leq 2~\\ \implies &~\mbox{by replacing $z$ with upper bound}\\ \mdmathindent{9}&~2\mathid{x}~+~4\mathid{y}~\leq 3~\\ \implies &~\mbox{by rounding upper bound}\\ \mdmathindent{9}&~2\mathid{x}~+~4\mathid{y}~\leq 2~\\ \implies &~\mbox{by adding back $z$}~\\ \mdmathindent{9}&~2\mathid{x}~+~4\mathid{y}~+~\mathid{z}~\leq 4 \end{mdmathpre}%mdk \]

Cuts from Hermite Matrices (Dillig et al., 2009; Christ and Hoenicke, 2015)

Constraints: $Ax \leq b$.


Assume $A$ is square and a tight non-integral solution $Ax_0 = b$.


Find Hermite $H$, unimodular $U$, s.t. $AU = H$. $\left(\begin{array}{rrrr}1 & 0 & 0 & 0 \\-1 & 2 & 0 & 0 \\-3 & -1 & 3 & 0 \\-1 & 0 & -2 & 4 \\\end{array}\right)$


Then $Ax \leq b \Leftrightarrow \exists y \ . \ Hy \leq b \land y = U^{-1}x$


Then $y_0 = U^{-1}x_0$ is not integral either.


Branch on some non-integral $y_{0i}$: $y_i \leq \lfloor y_{0i} \rfloor$.

Diophantine Solutions and Cuts (Griggio, 2012)

Bring tableau into integer solved form $x := \vec{a}\vec{y}$ with $\vec{a}$ integer (also used by Pugh's Omega test):

\[\begin{mdmathpre}%mdk \mdmathindent{7}&~3\mathid{x}~+~14\mathid{y}~-~7~=~0~\\ \equiv &~\\ \mdmathindent{7}&~3~(\mathid{x}~+~(14~\idiv 3)~\mathid{y}~-~(7~\idiv 3))\\ \mdmathindent{7}&~+~(14~\;\mbox{mod}\;~3)\mathid{y}~-~(7~\;\mbox{mod}\;~3)~=~0\\ \equiv &~\\ \mdmathindent{7}&~3~(\mathid{x}~+~4~\mathid{y}~-~2)~+~2~\mathid{y}~-~1~=~0\\ \simeq &~\\ \mdmathindent{7}&~3~\mathid{z}~+~2~\mathid{y}~-~1~=~0~~~~~~~~~~~~~~~~~~&~\mathid{fresh}\;~\mathid{z}\\ \mdmathindent{7}&~\mathid{x}~:=~\mathid{z}~-~4~\mathid{y}~-~2 \end{mdmathpre}%mdk \]
  • Detect GCD conflicts during normalization.
  • Cuts and bounds tightening on solved form.

Branching

Example 1.
LRA solver assigns $val(x) := \frac{1}{2}$, for integer $x$. Create branch atom $x \leq 0$.

When $x \leq 0$ is assigned, upper bound $x \leq 0$ is added to tableau.

When $\neg(x \leq 0)$ is assigned, then lower bound $1 \leq x$ is added to tableau.

LIA - next

  • Opportunities for micro-tuning are endless.

  • Smart scheduling for LIA (and NRA/NIA) end-game solvers?

  • LIA solver for small integers.

  • IntSAT/CutSAT not part of Z3.

NRA - Non-linear Real Arithmetic

NRAArch

NRA

What: Variables range over Reals. Addition, unrestricted multiplication.

How: Incremental linearization, CAD.

Why: A basis for solving also NIA and interesting self-contained uses pop up, such as Economy.

\[\begin{mdmathpre}%mdk \mdmathindent{2}\mathid{x},~\mathid{y},~\mathid{z}~\in \mathit{R}\\ \mdmathindent{2}\mathid{x}~>~1~\land \mathid{y}~>~1~\land \mathid{x}\cdot \mathid{y}~<~1~\leadsto \mathid{unsat} \end{mdmathpre}%mdk \]

Tableau with monomial definitions

Subset of variables are monomials $\mulvars$:

\[\begin{array}{ll} x_i = \sum_{x_j \in \nonbasicvars} a_{ij} x_j & x_i \in \basicvars \\ m_i := \prod_{x_{ij}} x_{ij} & m_i \in \mulvars \subset \nonbasicvars \cup \basicvars \\ l_j \leq x_j \leq u_j & x_j \in \nonbasicvars \cup \basicvars \\ val : x_j \mapsto {\mathcal R} & \forall x_j \in \nonbasicvars \\ nval : x_i \mapsto \sum_{x_j \in \nonbasicvars} a_{ij} val(x_j) & \forall x_i \in \basicvars \end{array} \]

Z3 tracks some equalities: If $m := xy$ and $x = -z$ is known, then $m := -zy$ is also used for lemmas.

NL Patching

Suppose $ m := x \cdot x \cdot y$ is in tableau, but $val(m) \neq val(x)^2 val(y)$.

  • Try patch $val(m) := val(x) \cdot val(x) \cdot val(y)$,
  • Try patch $val(y) := val(m) / (val(x) \cdot val(x))$,
  • Try patch $val(x) := \pm r / val(y)$, when $val(m) = r^2$ for a rational $r$.

Patching uses LP patch lookahead.

NL Bounds propagation using Interval arithmetic

Monomial propagation: $m := x\cdot x\cdot y$

\[\begin{mdmathpre}%mdk \mdmathindent{2}\mathid{y}~\geq 3~\implies \mathid{m}~\geq 0~~~~~~~~~~~~~~~~~&~\mbox{by}~&~\mathid{m}~\in [-\infty,\infty]^2~\cdot [3,~\infty]\\ \\ \mdmathindent{2}\mathid{x}~\geq 1,~\mathid{y}~\geq 3~\implies \mathid{m}~\geq 3~~~~~~~&~\mbox{by}~&~\mathid{m}~\in [1,\infty]^2[3,\infty]\\ \\ \mdmathindent{2}\mathid{m}~\leq 12,~\mathid{y}~\geq 3~\implies -2~\leq \mathid{x}~\leq 2~&~\mbox{by}~&~\mathid{x}~\in \pm\sqrt{(-\infty,12]~/~[3,\infty)}. \end{mdmathpre}%mdk \]

NL Bounds propagation on terms

Recall: A term is a slack variable introduced as short-hand for arithmetic expression.

Interval arithmetic isn't distributive:

Let $x \geq 2, y \geq -1, z \geq 2$

Then $xy + xz \in [2,\infty)\cdot[-1,\infty) + [2,\infty)\cdot[2,\infty) = (-\infty,\infty)$

But $x(y + z) \in [2,\infty)([-1,\infty) + [2,\infty)) = [2,\infty)$

So z3 explores different variants of distributing multiplication over terms.

Gröbner basis saturation

  • $V := \{ m_i, x_{ij} \mid m_i := \prod_{ij} x_{ij}, val(m_i) \neq \prod_{ij} val(x_{ij}) \}$.
  • $\mathcal{R} := \emptyset$
  • Ignore $r_i : x + by = 0$, $x$ is unbounded.
  • Add $r_i : ax + by = 0$ to $\mathcal{R}$, and add $y$ to $V$, if $x \in V$.
  • Run bounded Gröbner saturation based on $\mathcal{R}$.
  • Propagate new equations, detect infeasible equations
    • using interval bounds propagation
    • GCD tests
    • Euclidian tests

Multi-terminal ZDDs for polynomials

Design goals:

  • single instance store
  • polynomial simplification is built-in

PDD

Incremental Linearization (Cimatti et al., 2018a)

\[\begin{mdmathpre}%mdk \mdmathindent{2}\mathid{x}~=~0~~~~~&~\implies &~\mathid{xy}~=~0~\\ \mdmathindent{2}\mathid{x}~=~\pm 1~&~\implies &~\mathid{xy}~=~\pm \mathid{y}\\ \mdmathindent{2}\mathid{x}~\geq 1~~&~\implies &~\mathid{xy}~\geq \mathid{y}\\ \mdmathindent{2}\mathid{x}~\geq 0,~\mathid{y}~\geq 0~&~\implies &~\mathid{xy}~\geq 0 \end{mdmathpre}%mdk \]

NLSAT (Jovanovic and Moura, 2012)

nlsat

NRA - next

  • Integration of linearization with NLSAT (Promies et al., 2025)

  • $\delta$-satisfiability to leverage numeric methods.

  • Postitiv Stellensatz (ground and quantifiers)

NIA - Non-linear Integer Arithmetic

NRAArch

NIA

What: Variables range over Integers. Addition, unrestricted multiplication.

How: Methods for NRA + LIA + $\epsilon$

Why: The tougher spot in verification tools, such as F* and Certora.

\[\begin{mdmathpre}%mdk \mdmathindent{2}\mathid{x},~\mathid{y},~\mathid{z}~\in \mathit{Z}\\ \mdmathindent{2}\mathid{x}~>~1~\land \mathid{y}~>~1~\land \mathid{x}\cdot \mathid{y}~<~3~\leadsto \mathid{unsat} \end{mdmathpre}%mdk \]

NIA - Theoretical Difficulty and Practical Scope

  • QF_NIA is undecidable (Davis et al., 1961; Matiyasevich, 1970): there is no complete proof system for UNSAT.

  • QF_NIA used by ZK/Smart Contract auditors for bounded integers $[0, 2^{256}($.

  • Partial solution by $NIA := LIA + NRA$

NIA := LIA + NRA

NLSAT Branch and Bound (Jovanović, 2017)

  • Idea: Augment NLSAT (complete for NRA) post-check for integers.

  • If solution to $x \in \mathcal{Z}$ is not integer, but in interval $(k, k+1), k \in \mathcal{Z}$, then

  • Add axiom $x \leq k \lor x \geq k + 1$.

  • Run NLSAT again.

Incremental Linearization for Integers (Cimatti et al., 2018a)

  • Makes the biggest difference for solving NIA in Z3.
  • Creates constraints for violated monomial assignments $val(xy) \neq val(x)val(y)$.
  • Example, tangent lemma:
\[\begin{mdmathpre}%mdk \mathid{x}~>~\mathid{a},~\mathid{y}~>~\mathid{b}~&~\implies &~(\mathid{x}~-~\mathid{a})~\cdot (\mathid{y}~-~\mathid{b})~>~0\\ \mdmathindent{13}&~\implies &~\mathid{xy}~-~\mathid{ay}~-~\mathid{bx}~+~\mathid{ab}~>~0 \end{mdmathpre}%mdk \]

Suppose $val(xy) < val(x)\cdot val(y)$. Then setting $a := val(x) - 1$, $b := val(y) -1$ ensures that $val(xy) - a\cdot val(y) - b\cdot val(x) - a\cdot b \leq 0$.

Pseudo-linear monomials

\[\begin{mdmathpre}%mdk \mdmathindent{4}&~\mathid{x}~=~\mathid{val}(\mathid{x})~\implies \mathid{x}\cdot \mathid{y}~=~\mathid{val}(\mathid{x})\cdot \mathid{y}\\ \mathid{if}~~&~\\ \mdmathindent{4}&~\mathid{lo}~\leq \mathid{x}~\leq \mathid{hi},~\mathid{z}~\in \mathcal{Z}\\ \mdmathindent{4}&~\mathid{hi}~-~\mathid{lo}~\leq 4~~~ \end{mdmathpre}%mdk \]

But NIA > LIA + NRA

Euclidian Lemmas (I)

Consistency checks on equations produced by Gröbner saturation.

\[\begin{mdmathpre}%mdk \\ \mdmathindent{1}\mathid{x}~\mathid{y}~-~\mathid{z}~=~0~&~\implies &~\mathid{x}~\uparrow \mathid{z}~~~&~~\mathid{x},~\mathid{y},~\mathid{z}~\in \mathid{Z}\\ \mdmathindent{13}&~\implies &~(\mathid{x}~>~\mathid{z}~\geq 0~\implies \mathid{z}~=~0)~&~\clubsuit\\ \mdmathindent{13}&~\implies &~(\mathid{x}~<~\mathid{z}~\leq 0~\implies \mathid{z}~=~0)~&~\spadesuit\\ \mdmathindent{13}&~\implies &~(2\mathid{x}~>~\mathid{z}~\geq \mathid{x}~>~0~\implies \mathid{z}~=~\mathid{x})~&~\heartsuit\\ \mdmathindent{13}&~\implies &~\ldots \end{mdmathpre}%mdk \]

Create lemma if current assignment to $x, y, z$ violates $\clubsuit$ or $\spadesuit$. To not introduce divergence, z3 skips lemmas of the form $\heartsuit$.

Euclidian Lemmas (II)

NIAParity

NIAParitySolution

NIA - next

  • Identifying practical inference algorithms $X$ where $NIA = NRA + LIA + X + Y$.

  • Local search

  • Still very fragile, some can be mitigated by parallelism

NIAscatter

Extra Credit

LIA*

LIAstarIntro

LIA* - non-starter approach

LIAstarNonstarter RuzicaMunch

LIA* - mile-high

LIAstarMileHigh

LIA* - under approxiations

Grow under-approximation: $U^* \implies F_2^*$:

Initially $U := \emptyset, U^* := (0,0)$

Maintain, $U = \bigcup_i a_i + \lambda B_i$ under-approximates $F_2$ and set $U^* := \exists \mu\lambda (x = \sum_i \mu_i a_i + \lambda_i B_i) \land \bigwedge_i (\mu_i = 0 \implies \lambda_i = 0)$.

  • If $U^* \land F_1$ is SAT done. Otherwise,
  • Let $x_1, y_1: U^*(x_0, y_0) \land F_2(x_1, y_1) \land \neg U^*(x_0 + x_1, y_0 + y_1)$.
  • Add $(x_1, y_1)$ to $U$.
  • Reduce vectors $a_i, B_i$ using new element.

LIA* over-approximation (I)

LIAstar

LIA* over-approximation (II)

Suppose

\[\begin{mdmathpre}%mdk \mdmathindent{6}\mathid{U}^*(\mathid{x}_0,~\mathid{y}_0)~\land \mathid{F}_2(\mathid{x}_1,~\mathid{y}_1)~\land \mathid{F}_2(\mathid{x}_2,~\mathid{y}_2)~&~\\ \land \mathid{F}_1(\mathid{x}_0~+~\mathid{x}_1~+~\mathid{x}_2,~\mathid{y}_0~+~\mathid{y}_1~+~\mathid{y}_2)~~~~~~~~~~~~~~~~~&~\mbox{is unsat} \end{mdmathpre}%mdk \]

Compute interpolant $I$:

\[\begin{mdmathpre}%mdk \mathid{U}^*(\mathid{x}_0,~\mathid{y}_0)~\land \mathid{F}_2(\mathid{x}_1,~\mathid{y}_1)~&~\implies &~\mathid{I}(\mathid{x}_0~+~\mathid{x}_1,~\mathid{y}_0~+~\mathid{y}_1)~\\ \mathid{I}(\mathid{x},~\mathid{y})~&~\implies &~(\mathid{F}_2(\mathid{x}_2,~\mathid{y}_2)~\implies \neg \mathid{F}_1(\mathid{x}~+~\mathid{x}_2,~\mathid{y}~+~\mathid{y}_2)) \end{mdmathpre}%mdk \]

Initially $O^* := \top$

Add conjunctions from I to $O^*$ that are inductive, that is:

\[\begin{mdmathpre}%mdk \mathid{O}^*(\mathid{x},~\mathid{y})~\land \mathid{F}_2(\mathid{x}_1,~\mathid{y}_1)~\implies \mathid{O}^*(\mathid{x}~+~\mathid{x}_1,~\mathid{y}~+~\mathid{y}_1) \end{mdmathpre}%mdk \]

Warm Start

You can set the initial value of variables. 

 (declare-const x Int)
 (set-initial-value x 10)
 (push)
 (assert (> x 0))
 (check-sat)
 (eval x)
 ; 10

It is not always possible to enforce: If a benchmark is translated to SAT, initialization is partial.

References

Marc Bezem, Robert Nieuwenhuis, and Enric Rodríguez-Carbonell. “The Max-Atom Problem and Its Relevance.” In Logic for Programming, Artificial Intelligence, and Reasoning, 15th   International Conference, LPAR 2008, Doha, Qatar, November 22-27, 2008. Proceedings, 47–61. 2008. doi:10.1007/978-3-540-89439-1_4. 🔎
Nikolaj Bjørner, and Lev Nachmanson. “Theorem Recycling for Theorem Proving.” In Vampire 2017. Proceedings of the 4th Vampire Workshop, edited by Laura Kovács and Andrei Voronkov}, 53:1–8. EPiC Series in Computing. EasyChair. 2018. doi:10.29007/r58f. 🔎
Nikolaj S. Bjørner, and Lev Nachmanson. “Arithmetic Solving in Z3.” In Computer Aided Verification - 36th International Conference, CAV   2024, Montreal, QC, Canada, July 24-27, 2024, Proceedings, Part I, edited by Arie Gurfinkel and Vijay Ganesh, 14681:26–41. Lecture Notes in Computer Science. Springer. 2024. doi:10.1007/978-3-031-65627-9_2. 🔎
Martin Bromberger, and Christoph Weidenbach. “Fast Cube Tests for LIA Constraint Solving.” In IJCAR. 2016. doi:10.1007/978-3-319-40229-1_9. 🔎
Martin Bromberger, and Christoph Weidenbach. “New Techniques for Linear Arithmetic: Cubes and Equalities.” Formal Methods in System Design 51 (3): 433–461. 2017. doi:10.1007/s10703-017-0278-7. 🔎
Jürgen Christ, and Jochen Hoenicke. “Cutting the Mix.” In CAV. 2015. doi:10.1007/978-3-319-21668-3_3. 🔎
Alessandro Cimatti, Alberto Griggio, Ahmed Irfan, Marco Roveri, and Roberto Sebastiani. “Experimenting on Solving Nonlinear Integer Arithmetic with Incremental   Linearization.” In SAT. 2018. doi:10.1007/978-3-319-94144-8_23. 🔎
Alessandro Cimatti, Alberto Griggio, Ahmed Irfan, Marco Roveri, and Roberto Sebastiani. “Experimenting on Solving Nonlinear Integer Arithmetic with Incremental Linearization.” In Theory and Applications of Satisfiability Testing – SAT 2018, 10929:383–398. Lecture Notes in Computer Science. Springer. 2018. doi:10.1007/978-3-319-94144-8_23. 🔎
Alexandru Costan, Stephane Gaubert, Eric Goubault, Matthieu Martel, and Sylvie Putot. “A Policy Iteration Algorithm for Computing Fixed Points in Static   Analysis of Programs,” in @DBLP:conf/cav/2005, 3576:462–475. 2005. doi:10.1007/11513988_46. 🔎
Martin Davis, Hilary Putnam, and Julia Robinson. “The Decision Problem for Exponential Diophantine Equations.” Annals of Mathematics 74 (3). Princeton University Press: 425–436. 1961. doi:10.2307/1970295. 🔎
Isil Dillig, Thomas Dillig, and Alex Aiken. “Cuts from Proofs: A Complete and Practical Technique for Solving   Linear Inequalities over Integers.” In CAV. 2009. doi:10.1007/978-3-642-02658-4_20. 🔎
B. Dutertre, and L. de Moura. “A Fast Linear-Arithmetic Solver for DPLL(T).” In CAV. 2006. 🔎
Thomas Gawlitza, and Helmut Seidl. “Precise Relational Invariants Through Strategy Iteration,” in @DBLP:conf/csl/2007, 4646:23–40. 2007. doi:10.1007/978-3-540-74915-8_6. 🔎
Alberto Griggio. “A Practical Approach to Satisability Modulo Linear Integer Arithmetic.” J. Satisf. Boolean Model. Comput. 8 (1/2): 1–27. 2012. doi:10.3233/SAT190086. 🔎
Dejan Jovanović. “Solving Nonlinear Integer Arithmetic with MCSAT.” In Proceedings of the 18th International Conference on Verification, Model Checking, and Abstract Interpretation (VMCAI), 1–20. Springer. 2017. https://​yices.​csl.​sri.​com/​papers/​vmcai2017.​pdf. 🔎
Dejan Jovanovic, and Leonardo Mendonça de Moura. “Solving Non-Linear Arithmetic.” In Automated Reasoning - 6th International Joint Conference, IJCAR   2012, Manchester, UK, June 26-29, 2012. Proceedings, 339–354. 2012. doi:10.1007/978-3-642-31365-3_27. 🔎
Egor George Karpenkov. “Finding Inductive Invariants Using Satisfiability modulo Theories   and Convex Optimization. (Recherche D’invariants Inductifs Par Satisfiabilité modulo Théorie et Optimisation Convexe).” Phdthesis, Grenoble Alpes University, France. 2017. https://​tel.​archives-​ouvertes.​fr/​tel-​01681555. 🔎
Egor George Karpenkov, David Monniaux, and Philipp Wendler. “Program Analysis with Local Policy Iteration,” in @DBLP:conf/vmcai/2016, 9583:127–146. 2016. doi:10.1007/978-3-662-49122-5_6. 🔎
Yuri V. Matiyasevich. “Enumerable Sets Are Diophantine.” Doklady Akademii Nauk SSSR 191: 279–282. 1970. Translated in Soviet Math. Dokl. 11 (1970), 354–358. 🔎
Leonardo Mendonça de Moura, and Nikolaj Bjørner. “Model-Based Theory Combination.” Electron. Notes Theor. Comput. Sci. 198 (2): 37–49. 2008. doi:10.1016/j.entcs.2008.04.079. 🔎
Valentin Promies, Jasper Nalbach, Erika, and Paul Wagner. “More Is Less: Adding Polynomials for Faster Explanations in NLSAT.” In Automated Deduction - CADE 30 - 30th International Conference on   Automated Deduction, Stuttgart, Germany, July 28-31, 2025, Proceedings, edited by Clark W. Barrett and Uwe Waldmann, 15943:116–135. Lecture Notes in Computer Science. Springer. 2025. doi:10.1007/978-3-031-99984-0_7. 🔎
Yoni Zohar, Ahmed Irfan, Makai Mann, Aina Niemetz, Andres Nötzli, Mathias Preiner, Andrew Reynolds, Clark W. Barrett, and Cesare Tinelli. “Bit-Precise Reasoning via Int-Blasting.” In Verification, Model Checking, and Abstract Interpretation - 23rd International   Conference, VMCAI 2022, Philadelphia, PA, USA, January 16-18, 2022, Proceedings, edited by Bernd Finkbeiner and Thomas Wies, 13182:496–518. Lecture Notes in Computer Science. Springer. 2022. doi:10.1007/978-3-030-94583-1_24. 🔎