Recent advances in SMT

Programming Z3

System Overview

SAT

  from z3 import *
  Tie, Shirt = Bools('Tie Shirt')
  s = Solver()
  s.add(Or(Tie, Shirt), 
        Or(Not(Tie), Shirt), 
        Or(Not(Tie), Not(Shirt)))
  print(s.check())
  print(s.model())

SMT

  Z = IntSort()
  f = Function('f', Z, Z)
  x, y, z = Ints('x y z')
  A = Array('A',Z,Z)
  fml = Implies(x + 2 == y, f(Store(A, x, 3)[y - 2]) == f(y - x + 1))
  solve(Not(fml))

SMT-LIB and Python

 (set-logic QF_LIA)
 (declare-const x Int)
 (declare-const y Int)
 (assert (> (+ (mod x 4) (* 3 (div y 2))) (- x y)))
 (check-sat)

Python version:

 solve((x % 4) + 3 * (y / 2) > x - y)

Python and SMT-LIB

It is also possible to extract an SMT-LIB2 representation of a solver state.

from z3 import *
x, y = Ints('x y')
s = Solver()
s.add((x % 4) + 3 * (y / 2) > x - y)
print(s.sexpr())

produces the output

(declare-fun y () Int)
(declare-fun x () Int)
(assert (> (+ (mod x 4) (* 3 (div y 2))) (- x y)))

Accessing Expressions

x = Int('x')
y = Int('y')
n = x + y >= 3
print("num args: ", n.num_args())
print("children: ", n.children())
print("1st child:", n.arg(0))
print("2nd child:", n.arg(1))
print("operator: ", n.decl())
print("op name:  ", n.decl().name())

('num args: ', 2)
('children: ', [x + y, 3])
('1st child:', x + y)
('2nd child:', 3)
('operator: ', >=)
('op name:  ', '>=')

Quantifiers

solve(y == x + 1, ForAll([y], Implies(y <= 0, x < y)))

• Is this satisfiable?

Lambdas

m, m1 = Array('m', Z, Z), Array('m1', Z, Z)
def memset(lo,hi,y,m):
    return Lambda([x], If(And(lo <= x, x <= hi), y, Select(m, x)))
solve([m1 == memset(1, 700, z, m), Select(m1, 6) != z])

Q = Array('Q', Z, B)
prove(Implies(ForAll(Q, Implies(Select(Q, x), Select(Q, y))), x == y))

SMT Theories

SAT + Theories

• Mainstream SMT solver architecture based on co-processor architecture to CDCL.

• Alternatives:

  • Model Constructing SAT - instantiate values of non-Booleans
  • MIP(T) - MIP solver is main engine
  • Super-position

CDCL(T) by example

CDCL(T) by example (2)

CDCL(T) by example (3)

CDCL(T) by example (4)

CDCL(T) by example (5)

CDCL(T) by example (6)

EUF

  S = DeclareSort('S')
  f = Function('f', S, S)
  x = Const('x', S)
  solve(f(f(x)) == x, f(f(f(x))) == x)
  solve(f(f(x)) == x, f(f(f(x))) == x, f(x) != x)

Arithmetic

  x, y = Reals('x y')
  solve([x >= 0, Or(x + y <= 2, x + 2*y >= 6), 
                 Or(x + y >= 2, x + 2*y > 4)])

Bit-vectors

def is_power_of_two(x):
    return And(x != 0, 0 == (x & (x - 1)))
x = BitVec('x', 4)
prove(is_power_of_two(x) == Or([x == 2**i for i in range(4)]))

The absolute value of a variable can be obtained using addition and xor with a sign bit.

v = BitVec('v',32)
mask = v >> 31
prove(If(v > 0, v, -v) == (v + mask) ^ mask)

Algebraic Datatypes

Tree = Datatype('Tree')
Tree.declare('Empty')
Tree.declare('Node', ('left', Tree), ('data', Z), ('right', Tree))
Tree = Tree.create()
t = Const('t', Tree)
solve(t != Tree.Empty)

may produce the solution

[t = Node(Empty, 0, Empty)]

Similarly, one can prove that a tree cannot be a part of itself.

prove(t != Tree.Node(t, 0, t))

Floating Point Arithmetic

Floating points are bit-vectors with an interpretation specified by the IEEE floating point standard.

from z3 import *
x = FP('x', FPSort(3, 4))
y = FP('y', FPSort(3, 4))
solve(10 * x == y, x != 0)

[y = -0.0, x = -0.0]

Sequences and Strings

s, t, u = Strings('s t u')
prove(Implies(And(PrefixOf(s, t), SuffixOf(u, t), 
                  Length(t) == Length(s) + Length(u)), 
             t == Concat(s, u)))

One can concatenate single elements to a sequence as units:

s, t = Consts('s t', SeqSort(IntSort()))
solve(Concat(s, Unit(IntVal(2))) == Concat(Unit(IntVal(1)), t))
prove(Concat(s, Unit(IntVal(2))) != Concat(Unit(IntVal(1)), s))

Interfacing with Solvers

Scopes

p, q, r = Bools('p q r')
s = Solver()
s.add(Implies(p, q))
s.add(Not(q))
print(s.check())
s.push()
s.add(p)
print(s.check())
s.pop()
print(s.check())

Assumptions

Alternative to scopes, it is possible to check satisfiability under the assumption of a set of literals.

s.add(Implies(p, q))
s.add(Not(q))
print(s.check(p))

Cores

p, q, r, v = Bools('p q r v')
s = Solver()
s.add(Not(q))
s.assert_and_track(q, p)
s.assert_and_track(r, v)
print(s.check())
print(s.unsat_core())

Models

When s.check() returns sat Z3 can provide a model.

f = Function('f', Z, Z)
x, y = Ints('x y')
s.add(f(x) > y, f(f(y)) == y)
print(s.check())
print(s.model())

A possible model for s is:

[y = 0, x = 2, f = [0 -> 3, 3 -> 0, else -> 1]]

Using Solvers

Blocking Previous Models

I would like to enumerate more than one model.

Blocking Previous Models (II)

def block_model(s):
    m = s.model()
    s.add(Or([ f() != m[f] for f in m.decls() if f.arity() == 0]))           

Maximal Satisfying Subsets

I am given a solver s and set ps of formulas.

• Find a largest possible subset of ps that is satisfiable with s.

Maximal Satisfying Subsets (II)

def tt(s, f): 
    return is_true(s.model().eval(f))

def get_mss(s, ps):
    if sat != s.check():
       return []
    mss = { q for q in ps if tt(s, q) }
    return get_mss(s, mss, ps)

def get_mss(s, mss, ps):
    ps = ps - mss
    backbones = set([])
    while len(ps) > 0:
       p = ps.pop()
       if sat == s.check(mss | backbones | { p }):
          mss = mss | { p } | { q for q in ps if tt(s, q) }
          ps  = ps - mss
       else:
          backbones = backbones | { Not(p) }
    return mss          

All Cores and Correction Sets

I am given a solver s and set ps of formulas.

• Enumerate all unsatisfiable subsets of ps

• Enumerate all maximal satisfiable subsets of ps.

All Cores and Correction Sets (II)

Idea:

• use a map solver to enumerate substs of ps that

  • neither contain a core

  • nore is contained is a satisfiable subset

• Given a core

  • Add negation to map.

• Given a satisfiable subset

  • Add parts of ps that are not in subset
  • Known as correction set

All Cores and Correction Sets (III)

def ff(s, p): 
    return is_false(s.model().eval(p))

def marco(s, ps):
    map = Solver()
    while map.check() == sat:
        seed = {p for p in ps if not ff(map, p)}
        if s.check(seed) == sat:
           mss = get_mss(s, seed, ps)
           map.add(Or(ps - mss))
           yield "MSS", mss
        else:
           mus = s.unsat_core()
           map.add(Not(And(mus)))
           yield "MUS", mus

Propositional Interpolation

I am given solvers A and B

• A.add(B.assertions()) is unsat

• A, B share propositional atoms xs

• Compute reverse interpolant I

  • Implies(B, I)

  • Implies(A, Not(I))

Propositional Interpolation (II)

def mk_lit(m, x):
    if is_true(m.eval(x)):
       return x
    else:
       return Not(x)

def pogo(A, B, xs):   
    while sat == A.check():
       m = A.model()
       L = [mk_lit(m, x) for x in xs]
       if unsat == B.check(L):
          notL = Not(And(B.unsat_core()))
          yield notL
          A.add(notL)
       else:
          print("expecting unsat")
          break

Propositional Interpolation (III)

A = SolverFor("QF_FD")
B = SolverFor("QF_FD")
a1, a2, b1, b2, x1, x2 = Bools('a1 a2 b1 b2 x1 x2')
A.add(a1 == x1, a2 != a1, a2 != x2)
B.add(b1 == x1, b2 != b1, b2 == x2)
print(list(pogo(A, B, [x1, x2])))

    [Not(And(Not(x2), Not(x1))), Not(And(x2, x1))]

SAT Core

QF_FD

from z3 import *
s = SolverFor("QF_FD")
Color, (red, green, blue) = EnumSort('Color', ['red','green','blue'])
clr = Const('clr', Color)
u, v = BitVecs('u v', 32)
s.add(u >= v,
      If(v > u + 1, clr != red, clr != green),
      clr == green,
      AtLeast(u + v <= 3, v <= 20, u <= 10, 2))
print(s.check())
print(s.model())

is satisfiable, and a possible model is:

[v = 4, u = 2147483647, clr = green]

Cardinality and Pseudo-Boolean Constraints

p, q, r, u = Bools('p q r u')
solve(AtMost(p, q, r, 1), u, 
      Implies(u, AtLeast(And(p, r), Or(p, q), r, 2)))

Branching

1. Literal selection: Which choice to make?

2. Phase selection: Which direction to make the choice?
   
   
   

Cube and Conquer arguably a main development in scaling SAT.

What do SMT solvers do?

Backdoors

Fix subset of variables, such that reduced problem is easy (polynomial).

Literal Selection

• Lots in MIP community for branch and bound [1]

  • Most fractional branch on with current solution furthest away from integral

  • Progress towards optimal:

    • .

• CSP:

  • branch on variable with smallest current domain.

  • branch on variable with smallest ratio of constraint occurrences to domain size.

• Not much I know of in SMT, mostly piggyback on VSIDS (+ new friends), Lookahead or “steal” from MIP.

Phase selection

• Yices, MathSAT - model-based branching:

  • Candidate model

  • Branches

    • ,
    • .

• Strings [3] - Theory-aware branching.

  • Branches :
    • where is fresh
    • , where is fresh

The Cube, the cloud, and Z3

Cube and Conquer

s = SolverFor("QF_FD")
s.add()
s.set("sat.restart.max", 100)
def cube_and_conquer(s):
    for cube in s.cube():
       if len(cube) == 0:
          return unknown
       if is_true(cube[0]):
          return sat     
       is_sat = s.check(cube):
       if is_sat == unknown:
          s1 = s.translate(s.ctx)
          s1.add(cube)
          is_sat = cube_and_conquer(s1)
       if is_sat != unsat:
          return is_sat
    return unsat

Arithmetical theories

• Arithmetic is everywhere

• Cyber-physical systems, DNNs, axiomatic economy.

Linear Arithmetic

(set-logic QF_IDL) ; optional in Z3
(declare-fun t11 () Int)
(declare-fun t12 () Int)
(declare-fun t21 () Int)
(declare-fun t22 () Int)
(declare-fun t31 () Int)
(declare-fun t32 () Int)

(assert (and (>= t11 0) (>= t12 (+ t11 2)) (<= (+ t12 1) 8)))
(assert (and (>= t21 0) (>= t22 (+ t21 3)) (<= (+ t22 1) 8)))
(assert (and (>= t31 0) (>= t32 (+ t31 2)) (<= (+ t32 3) 8)))
(assert (or (>= t11 (+ t21 3)) (>= t21 (+ t11 2))))
(assert (or (>= t11 (+ t31 2)) (>= t31 (+ t11 2))))
(assert (or (>= t21 (+ t31 2)) (>= t31 (+ t21 3))))
(assert (or (>= t12 (+ t22 1)) (>= t22 (+ t12 1))))
(assert (or (>= t12 (+ t32 3)) (>= t32 (+ t12 1))))
(assert (or (>= t22 (+ t32 3)) (>= t32 (+ t22 1))))
(check-sat)
(get-model) ; display the model  

Linear Arithmetic Example

Some Arithmetical Theories

Algorithmic Fragments of Arithmetic

Non-linear Arithmetic [19]

NLSAT demo

NIA

Hilbert's 10th.

• Z3 falls back to incomplete heuristics
• NIA over bounded integers may get translated to bit-vector constraints
• Z3 and Yices [17]: NLSAT + branching
  • thus, create
  • for
• MathSAT [9]: incremental linearization
  • Follows from

Other theories

Cardinality and Pseudo-Boolean theories (2-3 other talks here)

Modular arithmetic (linear, LIAMC and non-linear with Groebner)

Bi-linear real arithmetic

Non-unit two-variable per inequality

At most one (unit) positive variable per inequality (Horn) . What is the non-unit complexity?

max-atom: . In NP co-NP, local strategy/policy iteration [4, 10, 15, 20, 21]

Knapsack theories: all variables with positive coefficients, except for one inequality where all variables have negative coefficients.

Bellman-Ford - a selection

(set-logic QF_IDL) ; optional in Z3
(declare-fun t11 () Int)
(declare-fun t12 () Int)
(declare-fun t21 () Int)
(declare-fun t22 () Int)
(declare-fun t31 () Int)
(declare-fun t32 () Int)

(assert (and (>= t11 0) ...))
(assert (and (>= t21 0) ...))
(assert (and (>= t31 0) (>= t32 (+ t31 2)) (<= (+ t32 3) 8)))
(assert (or ... (>= t21 (+ t11 2))))
(assert (or (>= t21 (+ t31 2)) ...))
(check-sat)
(get-model) ; display the model  

Bellman-Ford

Solve difference logic using graph Bellman-Ford network flow algorithm. Negative cycle unsat.

General form Simplex [12]

General form and

Only bounds (e.g., ) are asserted during search.

From Definitions to a Tableau

- are basic (dependent)
- are non-basic

Pivoting

• Value of non-basic variable can be chosen between and .
• Value of basic variable is a function of non-basic variable values.
• Pivoting swaps basic and non-basic variables.
  • used to get values of basic variables within bounds.

Pivoting example

- are basic (dependent)
- are non-basic

Initial values:

Bounds :

, make non-basic.
- are basic (dependent)
- are non-basic

Cubes [6]

Proposition: If has a solution over the reals, then has an integer solution obtained by rounding.

Example: Given

Solve instead

Real solution . Then is an integer solution.

Cubes [5]

Observation: One can often avoid strengthening inequalities.

bounds on variables .

differences between variables need not be strengthened to .

octagon inequalities can be strengthened to , or solutions can be patched.

unit horn inequalities

Cuts from Hermite Matrices [8, 11]

Constraints: .

Assume is square and a tight non-integral solution .

Find Hermite , unimodular , s.t. .

Then

Then is not integral either.

Branch on some non-integral : .

Arithmetic for DNN analysis

• ReLu [22]:

  • Dual Simplex based on floats (sacrifice numeric stability).
  • Extra inequalities solved by local search before branching.

• SHERLOCK [13]: Help Gurobi (MIP) solver using local search.

• Lipschitz bounded functions [24]:

  • Check that is unsat.
  • Exploit Lipschitz bound . Limits samples in .
  • , where is error tolerance.

• An opportunity to revisit SMT and global optimization?

Strategy/Policy iteration [10]

Goal is to find minimal such that:

Policy Initially replaces by (arbitrary choice).

Solve the resulting unit-Horn system with solution .

Evaluate each subterm wrt. to check if it preserves .

Repeat For each , if , but , then update .

Combining Theories

In practice, we need a combination of theories.

A theory is a set (potentially infinite) of first-order sentences.

Main questions: Is the union of two theories consistent? Given a solvers for and , how can we build a solver for ?

Combining Theories In a Nutshell

Is there an effective over shared signature, that when embeddable into implies is consistent?

A Combination History

Disjoint Theories

• Two theories are disjoint if they do not share function/constant and predicate symbols. is the only exception.

• Example:

  • The theories of arithmetic and arrays are disjoint.

  • Arithmetic symbols: .

  • Array symbols:

Purification

becomes

Purification (2)

becomes

Stably Infinite Theories

• A theory is stably infinite if every satisfiable QFF is satisfiable in an infinite model.

• EUF and arithmetic are stably infinite.

• Bit-vectors are not

Stably Infinite Theories - Result

Beyond Disjointness - Amalgamation [16]

Amalgamation [16]

1. there is a universal theory in the shared signature that is contained in both and
   • is universal = axioms of use only .
2. admits model-completion
   
   • Every model of embeds into .
   • admits quantifier elimination.
3. every model of embeds into a model of
   • iff
4. is effectively locally finite.
   • For every set of constants , there is a finite, computable, set over s.t. .

Signature non-disjoint theories

Ingredient: establish homomorphism from theory to , such that consistency of is reduced to consistency of .

Local Theory Extensions, Bridge Functions, Surjective Homomorphisms [25], [14], [28], [27], [7], [2]

Nelson-Oppen combination

Let and be consistent, stably infinite theories over disjoint (countable) signatures. Assume satisfiability of conjunction of literals can be decided in and time respectively. Then

1. The combined theory is consistent and stably infinite.

2. Satisfiability of quantifier free conjunction of literals can be decided in .

3. If and are convex, then so is and satisfiability in is in .

Convexity

A theory is convex iff:

• For all finite sets of literals
• For every disjunction
  • iff
  • for some .

Convexity: Positive Results

• Every convex theory with non trivial models is stably infinite.

• Horn equational theories are convex.

  • Horn equations are formulas of the form .

Convexity: Negative Results

• Integer arithmetic is not convex.

  • implies .

• Real non-linear arithmetic

  • implies .

Combination of non-convex theories

• EUF is convex

• IDL is non-convex

• is NP-complete

  • Reduce 3-CNF to EUF+ IDL:
    • for each Boolean variable.
    • For each clause : .

A Reduction Approach to Combination [23, 29]

Theory Combination in Z3:

• Core Theories = Arithmetic + EUF + SAT

• Bit-vectors, Finite domains = SAT

• Other theories reduced to Core Theories

  • Arrays, Datatypes, ..: Instantiate Theory Axioms
    • each index , each occurrence .
    • add
    • add

Gentle, smooth, polite theory combinations [18]

• Filtering rules on when shared variables matter.

• are shared, but

  • occurs only in

  • occurs only in

• We don't care if or .

Combining Theories in Practice

Propagate all implied equalities.

• Deterministic Nelson-Oppen.

• Complete only for convex theories.

• It may be expensive for some theories.

Delayed Theory Combination.

• Deterministic Nelson-Oppen.

• Create set of interface equalities between shared variables.

• Use SAT solver to guess the partition.

• Disadvantage: the number of additional equalities is quadratic in the number of shared variables.

Combining Theories in Practice (2)

Common to these methods is that they are pessimistic about which equalities are propagated.

Model-based Theory Combination:

• Optimistic approach.

• Use a candidate model for one of the theories and propagate all equalities implied by the candidate model hedging that other theories will agree.

• If not, use backtracking on choices to fix the model.

• It is cheaper to enumerate equalities that are implied in a particular model than of all models.

Model-Based Theory Combination - Example

purify

Model-Based Theory Combination (1)

Model-Based Theory Combination (2)

Model-Based Theory Combination (3)

Model-Based Theory Combination (4)

Model-Based Theory Combination (5)

Model-Based Theory Combination (6)

Model-Based Theory Combination (7)