code.test

Fact-based custom testing.

code.test is the custom test runner used throughout foundation-base. It reads fact forms, arrow assertions, namespace metadata, and runner options rather than delegating to standard clojure.test.

1    Motivation

The repository uses tests as examples as well as verification. fact forms can be imported into docs, linked back to source, and executed with targeted selectors. This is why new docs should prefer runnable fact examples when dependencies are local.

2    How to use it

Run focused namespaces with lein test :only some.namespace-test or families with lein test :with "[std]". Conditional environments should use fact:global skips so missing tools skip cleanly instead of failing setup.

lein test :only code.doc-test
lein test :with "[code]"

3    Internal usage

code.test is used by normal tests under test/, language tests under test-lang/, and documentation source files under src-doc/documentation/. The docs generator can render facts as examples, so a fact can serve both test and explanation.

4    Walkthrough

4.1    Writing a fact

Tests are written with the fact macro and the => arrow. A fact can contain multiple input => expected pairs, plain setup forms, and string descriptions.

a basic fact with the arrow assertion

(+ 1 2)
=> 3

(str "hello" " " "world")
=> "hello world"

4.2    Checkers

code.test provides matchers that go beyond equality. throws expects an exception, satisfies accepts a predicate or type, and contains checks nested data loosely.

throws catches exceptions

(/ 1 0)
=> (throws)

satisfies accepts predicates and classes

42
=> (satisfies even?)

"hello"
=> (satisfies string?)

contains checks a subset of a map

{:a 1 :b 2 :c 3}
=> (contains {:a 1 :b even?})

4.3    Inspecting facts

Facts are registered when the namespace loads. fact:list returns the ids in the current namespace, and fact:all retrieves facts from any loaded namespace.

list ids in the current namespace

(set (keys (t/fact:list)))
=> set?

retrieve all facts for a namespace

(t/fact:all 'code.project-test)
=> map?

4.4    End-to-end: running a focused namespace

Use task/run to execute a single test namespace, or run-errored to retry only the tests that failed in the last run.

run a focused test namespace

(task/run 'code.project-test)
=> map?

5    API

+listeners+ ^

NONE
(def +listeners+
  (listener/install-listeners))
link

-main ^

[& args]
Added 3.0

main entry point for leiningen

v 3.0
(defn -main
  ([& args]
   (let [opts    (process-test-args args)
         project (project/project)
         {:keys [throw failed timeout] :as stats}
         (cond
           (:files opts)
           (run (resolve-files (:files opts) project)
                (dissoc opts :files)
                project)

           (:report opts)
           (run:report (:report opts)
                       (dissoc opts :report)
                       project)

           :else
           (run (cond (:ns opts) (:ns opts) :else :all)
                (dissoc opts :ns)
                project))
         res (+ (or throw 0) (or failed 0) (or timeout 0))]
      (if (get opts :no-exit)
        res
        (System/exit res)))))
link
(task/-main)

=> ^

NONE
(def => '=>)
link

all ^

[& cks]
Added 3.0

checker that allows `and` composition of checkers

v 3.0
(defn all
  ([& cks]
   (let [cks (mapv common/->checker cks)]
     (common/checker
      {:tag :all
       :doc "Checks if the result matches all of the checkers"
       :fn (fn [res]
             (->> cks
                  (map #(common/verify % res))
                  (every? common/succeeded?)))
       :expect cks}))))
link
(mapv (all even? #(< 3 %))
      [1 2 3 4 5])
=> [false false false true false]

any ^

[& cks]
Added 3.0

checker that allows `or` composition of checkers

v 3.0
(defn any
  ([& cks]
   (let [cks (mapv common/->checker cks)]
     (common/checker
      {:tag :any
       :doc "Checks if the result matches any of the checkers"
       :fn (fn [res]
             (or (->> cks
                      (map #(common/verify % res))
                      (some common/succeeded?))
                 false))
       :expect cks}))))
link
(mapv (any even? 1)
      [1 2 3 4 5])
=> [true true false true false]

anything ^

[x]
Added 3.0

a checker that returns true for any value

v 3.0
(defn anything
  ([x]
   ((satisfies f/T) x)))
link
(anything nil) => true

(anything [:hello :world]) => true

approx ^

[v] [v threshold]
Added 3.0

checker that allows approximate verifications

v 3.0
(defn approx
  ([v]
   (approx v 0.001))
  ([v threshold]
   (checker
    {:tag :approx
     :doc "Checks if the result is approximately the given value"
     :fn (fn [res] (< (- v threshold) (res/result-data res) (+ v threshold)))
     :expect v})))
link
((approx 1) 1.000001) => true

((approx 1) 1.1) => false

((approx 1 0.0000001) 1.001) => false

capture ^

[] [checker] [checker sym]
Added 3.0

adds a form to capture test input

v 3.0
(defmacro capture
  ([]
   `(capture nil ~'$0))
  ([checker]
   `(capture ~checker ~'$0))
  ([checker sym]
   `(fn [~'x]
      (intern *ns* (quote ~(with-meta sym {:dynamic true})) ~'x)
      (or ~(if checker `(~checker ~'x) true)))))
link

contains ^

[x & modifiers]
Added 3.0

checker for maps and vectors

v 3.0
(defn contains
  ([x & modifiers]
   (cond (map? x)
         (contains-map x)

         (set? x)
         (contains-set x)

         (seqable? x)
         (contains-vector (seq x) (set modifiers))

         (sequential? x)
         (contains-vector x (set modifiers))

         :else
         (throw (ex-info "Cannot create contains checker" {:input [x modifiers]})))))
link
((contains {:a odd? :b even?}) {:a 1 :b 4})
=> true

((contains {:a 1 :b even?}) {:a 2 :b 4})
=> false

((contains [1 2 3]) [1 2 3 4])
=> true

((contains [1 3]) [1 2 3 4])
=> false

((contains [1 3] :gaps-ok) [1 2 3 4])
=> true

((contains [3 1] :gaps-ok) [1 2 3 4])
=> false

((contains [3 1] :in-any-order) [1 2 3 4])
=> false

((contains [3 1 2] :in-any-order) [1 2 3 4])
=> true

((contains [3 1] :in-any-order :gaps-ok) [1 2 3 4])
=> true

contains-in ^

[x]
Added 3.0

shorthand for checking nested maps and vectors

v 3.0
(defmacro contains-in
  ([x]
   "A macro for nested checking of data in the `contains` form" (cond (map? x)
                                                                      `(contains ~(collection/map-vals (fn [v] `(contains-in ~v)) x))
                                                                      (set? x)
                                                                      `(contains ~(mapv (fn [v] `(contains-in ~v)) x)
                                                                                 :in-any-order)

                                                                      (vector? x)
                                                                      `(contains ~(mapv (fn [v] `(contains-in ~v)) x))
                                                                      :else x)))
link
((contains-in {:a {:b {:c odd?}}}) {:a {:b {:c 1 :d 2}}})
=> true

((contains-in [odd? {:a {:b even?}}]) [3 {:a {:b 4 :c 5}}])
=> true

exactly ^

[v] [v function]
Added 3.0

checker that allows exact verifications

v 3.0
(defn exactly
  ([v]
   (exactly v identity))
  ([v function]
   (checker
    {:tag :exactly
     :doc "Checks if the result exactly satisfies the condition"
     :fn (fn [res] (= (function (res/result-data res)) v))
     :expect v})))
link
((exactly 1) 1) => true

((exactly Long) 1) => false

((exactly number?) 1) => false

fact ^

[desc & body]
Added 3.0

top level macro for test definitions

v 3.0
(defmacro fact
  ([desc & body]
   (let [{:keys [id eval] :as meta} (clojure.core/meta &form)
         [meta body] (fact-prepare-core desc body meta)
         fpkg (install-fact meta body)]
     (if id
       (intern (:ns fpkg) id fpkg))
     (if (and context/*eval-mode*
              (not (false? eval)))
       (fact-eval fpkg)))))
link

fact:all ^

[] [ns]
Added 3.0

returns all facts in namespace

v 3.0
(defmacro fact:all
  ([]
   `(rt/all-facts))
  ([ns]
   `(rt/all-facts (quote ~ns))))
link

fact:exec ^

[] [id] [ns id]
Added 3.0

runs main hook for fact form

v 3.0
(defmacro fact:exec
  ([]
   `((fact:get)))
  ([id]
   `((fact:get ~id)))
  ([ns id]
   `((fact:get ~ns ~id))))
link

fact:get ^

[] [id] [ns id]
Added 3.0

gets elements of the current fact

v 3.0
(defmacro fact:get
  ([]
   (let [m    (meta &form)
         {:keys [id]} (rt/find-fact m)]
     (rt/get-fact (quote ~id))))
  ([id]
   `(fact:get ~(env/ns-sym) ~id))
  ([ns id]
   (if (symbol? id)
     `(rt/get-fact (quote ~ns) (quote ~id))
     `(let [fpkg# (rt/find-fact (quote ~ns) {:source (quote ~id)})]
        (rt/get-fact (quote ~ns) (:id fpkg#))))))
link

fact:global ^

[] [cmd & args]
Added 3.0

fact global getter and setter

v 3.0
(defmacro fact:global
  ([]
   `(fact:global-fn))
  ([cmd & args]
   `(fact:global-fn (quote ~cmd)
                    ~@(map #(list `quote %) args))))
link
(ctx/with-new-context {}
  (fact:global :set {:a 1})
  (fact:global :get))
=> {:a 1}

fact:list ^

[]
Added 3.0

lists all facts in namespace

v 3.0
(defn fact:list
  ([]
   (rt/list-facts)))
link
(ctx/with-new-context {:registry (atom {'my.ns {:facts {'a {:id 'a :line 1}}}})}
  (binding [*ns* (create-ns 'my.ns)]
    (fact:list)))
=> '(a)

fact:missing ^

[] [ns]
Added 3.0

returns all missing facts for a given namespace

v 3.0
(defn fact:missing
  ([]
   (let [ns (-> (str (env/ns-sym))
                (clojure.string/replace "-test$" "")
                (symbol))]
     (fact:missing ns)))
  ([ns]
   (->> (clojure.set/difference
         (set (map #(symbol (name ns) (name %))
                   (keys (ns-interns ns))))
         (set (map :refer (vals (rt/all-facts)))))
        (map (comp symbol name))
        sort)))
link

fact:ns ^

[& forms]
Added 3.0

provides a macro for managing namespace-related operations within facts

v 3.0
(defmacro fact:ns
  ([& forms]
   `(fact:ns-fn (quote ~forms))))
link
;; (code.test.manage/fact:ns (:link my.ns))
;; => [[:link]]
true => true

fact:ns-load ^

[ns]
Added 3.0

loads a test namespace

v 3.0
(defn fact:ns-load
  ([ns]
   (project/in-context
    (executive/load-namespace ns))))
link
(with-redefs [project/in-context (fn [f] (f))
              executive/load-namespace (fn [ns & _] ns)]
  (fact:ns-load 'my.ns))
=> 'my.ns

fact:purge ^

[]
Added 3.0

purges all facts in namespace

v 3.0
(defn fact:purge
  ([]
   (rt/purge-facts)))
link
(ctx/with-new-context {:registry (atom {'my.ns {:facts {:a 1}}})}
  (binding [*ns* (create-ns 'my.ns)]
    (fact:purge)
    (rt/all-facts 'my.ns)))
=> nil

fact:remove ^

[]
Added 3.0

removes the current fact

v 3.0
(defmacro fact:remove
  ([]
   `(rt/run-op (meta &form) :remove)))
link

fact:setup ^

[]
Added 3.0

runs setup hook for current fact

v 3.0
(defmacro fact:setup
  ([]
   `(rt/run-op ~(meta &form) :setup)))
link

fact:setup? ^

[]
Added 3.0

checks if setup hook has been ran

v 3.0
(defmacro fact:setup?
  ([]
   `(rt/run-op ~(meta &form) :setup?)))
link

fact:symbol ^

[]
Added 3.0

gets the current fact symbol

v 3.0
(defmacro fact:symbol
  ([]
   `(rt/run-op (meta &form) :symbol)))
link

fact:teardown ^

[]
Added 3.0

runs teardown hook for current fact

v 3.0
(defmacro fact:teardown
  ([]
   `(rt/run-op (meta &form) :teardown)))
link

fact:template ^

[desc & body]
Added 3.0

adds a template to the file

v 3.0
(defmacro fact:template
  ([desc & body]
   `(binding [context/*eval-mode* false]
      ~(with-meta
         `(fact ~desc ~@body)
         (assoc (meta &form) :eval false)))))
link

is-not ^

[ck] [ck function]
Added 3.0

checker that allows negative composition of checkers

v 3.0
(defn is-not
  ([ck]
   (is-not ck identity))
  ([ck function]
   (let [ck (common/->checker ck)]
     (common/checker
      {:tag :is-not
       :doc "Checks if the result is not an outcome"
       :fn (fn [res]
             (not (ck (function res))))
       :expect ck}))))
link
(mapv (is-not even?)
      [1 2 3 4 5])
=> [true false true false true]

just ^

[x & modifiers]
Added 3.0

combination checker for both maps and vectors

v 3.0
(defn just
  ([x & modifiers]
   (cond (map? x)
         (just-map x)

         (set? x)
         (just-set x)

         (vector? x)
         (just-vector x (set modifiers))

         :else
         (throw (ex-info "Cannot create just checker" {:input [x modifiers]})))))
link
((just {:a odd? :b even?}) {:a 1 :b 4})
=> true

((just {:a 1 :b even?}) {:a 1 :b 2 :c 3})
=> false

((just [1 2 3 4]) [1 2 3 4])
=> true

((just [1 2 3]) [1 2 3 4])
=> false

((just [3 2 4 1] :in-any-order) [1 2 3 4])
=> true

just-in ^

[x]
Added 3.0

shorthand for exactly checking nested maps and vectors

v 3.0
(defmacro just-in
  ([x]
   "A macro for nested checking of data in the `just` form" (cond (map? x)
                                                                  `(just ~(collection/map-vals (fn [v] `(just-in ~v)) x))

                                                                  (set? x)
                                                                  `(just ~(mapv (fn [v] `(just-in ~v)) x)
                                                                         :in-any-order)

                                                                  (vector? x)
                                                                  `(just ~(mapv (fn [v] `(just-in ~v)) x))

                                                                  :else x)))
link
((just-in {:a {:b {:c odd?}}}) {:a {:b {:c 1 :d 2}}})
=> false

((just-in [odd? {:a {:b even?}}]) [3 {:a {:b 4}}])

((just-in [odd? {:a {:b even?}}]) [3 {:a {:b 4}}])
=> true

print-options ^

[] [opts]
Added 3.0

output options for test results

v 3.0
(defn print-options
  ([] (print-options :help))
  ([opts]
   (cond (set? opts)
         (alter-var-root #'context/*print*
                         (constantly opts))

         (= :help opts)
         #{:help :current :default :disable :all}

         (= :current opts) context/*print*

         (= :default opts)
         (alter-var-root #'context/*print*
                         (constantly #{:print-throw :print-failed :print-timeout :print-bulk}))

         (= :disable opts)
         (alter-var-root #'context/*print* (constantly #{}))

         (= :all opts)
         #{:print-throw
           :print-success
           :print-timeout
           :print-facts
           :print-facts-success
           :print-failed
           :print-bulk})))
link
(task/print-options)
=> #{:disable :default :all :current :help}

(task/print-options :default)
=> #{:print-failed :print-bulk :print-timeout :print-throw}

(task/print-options :all)
#{:print-failed
  :print-bulk
  :print-facts-success
  :print-timeout
  :print-facts
  :print-throw
  :print-success}

run ^

[] [ns] [ns params] [ns params project] [ns params lookup project]
Added 3.0

executes all tests, optionally filtering by list or specific namespace

v 3.0
(invoke/definvoke run
  [:task {:template :test
          :main   {:fn executive/run-namespace}
          :params {:title "TEST PROJECT"}}])
link
(task/run :list)

(task/run 'std.lib.foundation)
;; {:files 1, :thrown 0, :facts 8, :checks 18, :passed 18, :failed 0}
=> map?

run-errored ^

[]
Added 3.0

runs only the tests that have errored

v 3.0
(defn run-errored
  ([]
   (let [latest @executive/+latest+]
     (-> (clojure.set/union (set (:errored latest))
                  (set (map (comp :ns :meta) (:failed latest)))
                  (set (map (comp :ns :meta) (:throw latest)))
                  (set (map (comp :ns :meta) (:timeout latest))))
         (run)))))
link
(task/run-errored)

run:current ^

[] [ns] [ns params] [ns params project] [ns params lookup project]
Added 4.0

runs the current namespace

v 4.0
(invoke/definvoke run:current
  [:task {:template :test
          :main   {:fn executive/run-current}
          :params {:title "TEST CURRENT"}}])
link
(:type (deref #'task/run:current))
=> :test

(get-in (deref #'task/run:current) [:params :title])
=> "TEST CURRENT"

(sequential? (task/run:current :list))
=> true

run:interrupt ^

[]
Added 4.0

interrupts the test

v 4.0
(defn run:interrupt
  []
  (alter-var-root #'std.task.process/*interrupt*
                  (fn [_] true)))
link
(with-redefs [std.task.process/*interrupt* false]
  (task/run:interrupt)
  std.task.process/*interrupt*)
=> true

run:load ^

[] [ns] [ns params] [ns params project] [ns params lookup project]
Added 3.0

load test namespace

v 3.0
(invoke/definvoke run:load
  [:task {:template :test
          :main   {:fn executive/load-namespace}
          :item   {:output identity}
          :params {:title "TEST LOADED"}}])
link
(:type (deref #'task/run:load))
=> :test

(get-in (deref #'task/run:load) [:params :title])
=> "TEST LOADED"

(sequential? (task/run:load :list))
=> true

run:report ^

[path] [path params] [path params project]
Added 4.1

reruns failed tests recorded in a saved report

v 4.1
(defn run:report
  ([path]
   (run:report path {}))
  ([path params]
   (run:report path params (project/project)))
  ([path params project]
   (let [path   (str path)
         report (executive/load-report path)
         failed (executive/report-failed-facts report)
         nss    (vec (sort (keys failed)))]
     (when (empty? nss)
       (throw (ex-info "No failed tests found in report" {:path path})))
     (run nss (assoc params :filter failed) project))))
link
(let [calls (atom nil)]
  (with-redefs [code.test.base.executive/load-report (fn [path]
                                                       {:path path})
                code.test.base.executive/report-failed-facts (fn [report]
                                                               {'my.ns '#{my.fn}})
                task/run (fn [nss params project]
                           (reset! calls {:nss nss :params params :project project})
                           {:passed 1})]
    (task/run:report "/tmp/report.edn" {} {:root "/tmp"})
    @calls))
=> (contains {:nss '[my.ns]
              :params (contains {:filter {'my.ns '#{my.fn}}})
              :project (contains {:root "/tmp"})})

run:test ^

[] [ns] [ns params] [ns params project] [ns params lookup project]
Added 3.0

executes tests that are currently loaded in the runtime

v 3.0
(invoke/definvoke run:test
  [:task {:template :test
          :main   {:fn executive/test-namespace}
          :params {:title "TEST EVAL"}}])
link
(:type (deref #'task/run:test))
=> :test

(get-in (deref #'task/run:test) [:params :title])
=> "TEST EVAL"

(sequential? (task/run:test :list))
=> true

run:unload ^

[] [ns] [ns params] [ns params project] [ns params lookup project]
Added 3.0

unloads the test namespace

v 3.0
(invoke/definvoke run:unload
  [:task {:template :test
          :main   {:fn executive/unload-namespace}
          :item   {:output identity}
          :params {:title "TEST UNLOADED"}}])
link
(:type (deref #'task/run:unload))
=> :test

(get-in (deref #'task/run:unload) [:params :title])
=> "TEST UNLOADED"

(sequential? (task/run:unload :list))
=> true

satisfies ^

[v] [v function]
Added 3.0

checker that allows loose verifications

v 3.0
(defn satisfies
  ([v]
   (satisfies v identity))
  ([v function]
   (checker
    {:tag :satisfies
     :doc "Checks if the result can satisfy the condition:"
     :fn (fn [res]
           (let [data (function (res/result-data res))]
             (cond (= data v) true

                   (class? v) (instance? v data)

                   (and (f/comparable? v data)
                        (zero? (compare v data)))
                   true

                   (map? v) (= (into {} data) v)

                   (vector? v) (= data v)

                   (ifn? v) (boolean (v data))

                   (f/regexp? v)
                   (cond (f/regexp? data)
                         (= (.pattern ^Pattern v)
                            (.pattern ^Pattern data))

                         (string? data)
                         (boolean (re-find v data))

                         :else false)

                   :else false)))
     :expect v})))
link
((satisfies 1) 1) => true

((satisfies Long) 1) => true

((satisfies number?) 1) => true

((satisfies #{1 2 3}) 1) => true

((satisfies [1 2 3]) 1) => false

((satisfies number?) "e") => false

((satisfies #"hello") #"hello") => true

stores ^

[v]
Added 3.0

a checker that looks into a ref object

v 3.0
(defn stores
  ([v]
   (checker
    {:tag :stores
     :doc "Checks if the result is a ref with "
     :fn (fn [res]
           (if (f/ideref? res)
             ((satisfies v) (deref res))
             false))
     :expect v})))
link
((stores 1) (volatile! 1)) => true

((stores 1) 1) => false

throws ^

[] [e] [e msg]
Added 3.0

checker that determines if an exception has been thrown

v 3.0
(defn throws
  ([]  (throws Throwable))
  ([e] (throws e nil))
  ([e msg]
   (checker
    {:tag :throws
     :doc "Checks if an exception has been thrown"
     :fn (fn [{:keys [data status]}]
           (and (= :exception status)
                (instance? e data)
                (if msg
                  (= msg (.getMessage ^Throwable data))
                  true)))
     :expect {:exception e :message msg}})))
link
((throws Exception "Hello There")
 (res/result
  {:status :exception
   :data (Exception. "Hello There")}))
=> true

throws-info ^

[] [m]
Added 3.0

checker that determines if an `ex-info` has been thrown

v 3.0
(defn throws-info
  ([]  (throws-info {}))
  ([m]
   (common/checker
    {:tag :raises
     :doc "Checks if an issue has been raised"
     :fn (fn [{:keys [^Throwable data status]}]
           (and (= :exception status)
                (instance? clojure.lang.ExceptionInfo data)
                ((contains m) (ex-data data))))
     :expect {:exception clojure.lang.ExceptionInfo :data m}})))
link
((throws-info {:a "hello" :b "there"})
 (process/evaluate {:form '(throw (ex-info "hello" {:a "hello" :b "there"}))}))
=> true

with-new-context ^

[m & body]
Added 4.1

with-new-context binds dynamic variables

v 4.1
(defmacro with-new-context
  [m & body]
  `(let [ctx# (merge (new-context)
                     ~m)]
     (binding [*eval-fact*       (:eval-fact ctx#)
               *eval-mode*       (:eval-mode ctx#)
               *eval-meta*       (:eval-meta ctx#)
               *eval-global*     (:eval-global ctx#)
               *eval-check*      (:eval-check ctx#)
               *eval-current-ns* (:eval-current-ns ctx#)
               *run-id*          (:run-id ctx#)
               *registry*        (:registry ctx#)
               *accumulator*     (:accumulator ctx#)
               *errors*          (:errors ctx#)
               *settings*        (:settings ctx#)
               *root*            (:root ctx#)
               *results*         (:results ctx#)
               *timeout-ns-global* (:timeout-ns ctx#)
               *print*           (:print ctx#)]
       ~@body)))
link
(context/with-new-context {:eval-mode true}
  context/*eval-mode*) => true

(context/with-new-context {:print #{:a}}
  context/*print*) => #{:a}

-main ^

[& args]
Added 3.0

main entry point for leiningen

v 3.0
(defn -main
  ([& args]
   (let [opts    (process-test-args args)
         project (project/project)
         {:keys [throw failed timeout] :as stats}
         (cond
           (:files opts)
           (run (resolve-files (:files opts) project)
                (dissoc opts :files)
                project)

           (:report opts)
           (run:report (:report opts)
                       (dissoc opts :report)
                       project)

           :else
           (run (cond (:ns opts) (:ns opts) :else :all)
                (dissoc opts :ns)
                project))
         res (+ (or throw 0) (or failed 0) (or timeout 0))]
      (if (get opts :no-exit)
        res
        (System/exit res)))))
link
(task/-main)

display-errors ^

[data]

NONE
(defn- display-errors
  [data]
  (let [errors (concat (executive/retrieve-line :failed data)
                       (executive/retrieve-line :throw data)
                       (executive/retrieve-line :timeout data))
        cnt  (count (:passed data))]
    (if (empty? errors)
      (res/result {:status :highlight
                   :data  (format "passed (%s)" cnt)})
      (res/result {:status :error
                   :data (format "passed (%s), errors: #{%s}"
                                 cnt
                                 (common/joinl ", " (mapv #(str (first %) ":" (second %))
                                                       errors)))}))))
link

file-paths ^

[files]

NONE
(defn- file-paths
  [files]
  (cond
    (nil? files)
    []
    
    (string? files)
    (->> (str/split files #"s+")
         (remove empty?)
         vec)
    
    (or (symbol? files)
        (keyword? files))
    [(str files)]
    
    (coll? files)
    (->> files
         (mapcat file-paths)
         vec)
    
    :else
    [(str files)]))
link

print-options ^

[] [opts]
Added 3.0

output options for test results

v 3.0
(defn print-options
  ([] (print-options :help))
  ([opts]
   (cond (set? opts)
         (alter-var-root #'context/*print*
                         (constantly opts))

         (= :help opts)
         #{:help :current :default :disable :all}

         (= :current opts) context/*print*

         (= :default opts)
         (alter-var-root #'context/*print*
                         (constantly #{:print-throw :print-failed :print-timeout :print-bulk}))

         (= :disable opts)
         (alter-var-root #'context/*print* (constantly #{}))

         (= :all opts)
         #{:print-throw
           :print-success
           :print-timeout
           :print-facts
           :print-facts-success
           :print-failed
           :print-bulk})))
link
(task/print-options)
=> #{:disable :default :all :current :help}

(task/print-options :default)
=> #{:print-failed :print-bulk :print-timeout :print-throw}

(task/print-options :all)
#{:print-failed
  :print-bulk
  :print-facts-success
  :print-timeout
  :print-facts
  :print-throw
  :print-success}

process-test-args ^

[args]
Added 4.1

parses code.test CLI args, including multiple :only namespaces

v 4.1
(defn process-test-args
  [args]
  (task/process-ns-args (task/collapse-only args)))
link
(task/process-test-args [":only" "foo"])
=> {:ns '[foo]}

(task/process-test-args [":only" "foo" "bar" "baz"])
=> {:ns '[foo bar baz]}

(task/process-test-args [":only" "foo" "bar" ":timeout" "100"])
=> {:ns '[foo bar] :timeout 100}

resolve-files ^

[files] [files project]
Added 4.1

resolves source and test file paths to test namespaces

v 4.1
(defn resolve-files
  ([files]
   (resolve-files files (project/project)))
  ([files project]
   (->> (file-paths files)
        (mapv (fn [path]
                (let [[ns] (project/lookup-ns path)]
                  (when-not ns
                    (throw (ex-info "Cannot resolve namespace from file"
                                    {:file path})))
                  (if (= :test (project/file-type path))
                    ns
                    (project/test-ns ns)))))
        (distinct)
        vec)))
link
(task/resolve-files ["src/code/project.clj"
                     "test/std/task_test.clj"]
                    (project/project))
=> ['code.project-test
    'std.task-test]

(task/resolve-files "test/code/project_test.clj test/code/test/task_test.clj"
                    (project/project))
=> ['code.project-test
    'code.test.task-test]

retrieve-fn ^

[kw]

NONE
(defn- retrieve-fn [kw]
  (fn [data]
    (not-empty
     (->> (executive/retrieve-line kw data)
          (mapv #(str (first %) ":" (second %)))))))
link

run ^

[] [ns] [ns params] [ns params project] [ns params lookup project]
Added 3.0

executes all tests, optionally filtering by list or specific namespace

v 3.0
(invoke/definvoke run
  [:task {:template :test
          :main   {:fn executive/run-namespace}
          :params {:title "TEST PROJECT"}}])
link
(task/run :list)

(task/run 'std.lib.foundation)
;; {:files 1, :thrown 0, :facts 8, :checks 18, :passed 18, :failed 0}
=> map?

run-errored ^

[]
Added 3.0

runs only the tests that have errored

v 3.0
(defn run-errored
  ([]
   (let [latest @executive/+latest+]
     (-> (clojure.set/union (set (:errored latest))
                  (set (map (comp :ns :meta) (:failed latest)))
                  (set (map (comp :ns :meta) (:throw latest)))
                  (set (map (comp :ns :meta) (:timeout latest))))
         (run)))))
link
(task/run-errored)

run:current ^

[] [ns] [ns params] [ns params project] [ns params lookup project]
Added 4.0

runs the current namespace

v 4.0
(invoke/definvoke run:current
  [:task {:template :test
          :main   {:fn executive/run-current}
          :params {:title "TEST CURRENT"}}])
link
(:type (deref #'task/run:current))
=> :test

(get-in (deref #'task/run:current) [:params :title])
=> "TEST CURRENT"

(sequential? (task/run:current :list))
=> true

run:interrupt ^

[]
Added 4.0

interrupts the test

v 4.0
(defn run:interrupt
  []
  (alter-var-root #'std.task.process/*interrupt*
                  (fn [_] true)))
link
(with-redefs [std.task.process/*interrupt* false]
  (task/run:interrupt)
  std.task.process/*interrupt*)
=> true

run:load ^

[] [ns] [ns params] [ns params project] [ns params lookup project]
Added 3.0

load test namespace

v 3.0
(invoke/definvoke run:load
  [:task {:template :test
          :main   {:fn executive/load-namespace}
          :item   {:output identity}
          :params {:title "TEST LOADED"}}])
link
(:type (deref #'task/run:load))
=> :test

(get-in (deref #'task/run:load) [:params :title])
=> "TEST LOADED"

(sequential? (task/run:load :list))
=> true

run:report ^

[path] [path params] [path params project]
Added 4.1

reruns failed tests recorded in a saved report

v 4.1
(defn run:report
  ([path]
   (run:report path {}))
  ([path params]
   (run:report path params (project/project)))
  ([path params project]
   (let [path   (str path)
         report (executive/load-report path)
         failed (executive/report-failed-facts report)
         nss    (vec (sort (keys failed)))]
     (when (empty? nss)
       (throw (ex-info "No failed tests found in report" {:path path})))
     (run nss (assoc params :filter failed) project))))
link
(let [calls (atom nil)]
  (with-redefs [code.test.base.executive/load-report (fn [path]
                                                       {:path path})
                code.test.base.executive/report-failed-facts (fn [report]
                                                               {'my.ns '#{my.fn}})
                task/run (fn [nss params project]
                           (reset! calls {:nss nss :params params :project project})
                           {:passed 1})]
    (task/run:report "/tmp/report.edn" {} {:root "/tmp"})
    @calls))
=> (contains {:nss '[my.ns]
              :params (contains {:filter {'my.ns '#{my.fn}}})
              :project (contains {:root "/tmp"})})

run:test ^

[] [ns] [ns params] [ns params project] [ns params lookup project]
Added 3.0

executes tests that are currently loaded in the runtime

v 3.0
(invoke/definvoke run:test
  [:task {:template :test
          :main   {:fn executive/test-namespace}
          :params {:title "TEST EVAL"}}])
link
(:type (deref #'task/run:test))
=> :test

(get-in (deref #'task/run:test) [:params :title])
=> "TEST EVAL"

(sequential? (task/run:test :list))
=> true

run:unload ^

[] [ns] [ns params] [ns params project] [ns params lookup project]
Added 3.0

unloads the test namespace

v 3.0
(invoke/definvoke run:unload
  [:task {:template :test
          :main   {:fn executive/unload-namespace}
          :item   {:output identity}
          :params {:title "TEST UNLOADED"}}])
link
(:type (deref #'task/run:unload))
=> :test

(get-in (deref #'task/run:unload) [:params :title])
=> "TEST UNLOADED"

(sequential? (task/run:unload :list))
=> true

test-lookup ^

[project]

NONE
(defn- test-lookup [project]
  (project/all-files (:test-paths project)
                     {}
                     project))
link

fact:global ^

[] [cmd & args]
Added 3.0

fact global getter and setter

v 3.0
(defmacro fact:global
  ([]
   `(fact:global-fn))
  ([cmd & args]
   `(fact:global-fn (quote ~cmd)
                    ~@(map #(list `quote %) args))))
link
(ctx/with-new-context {}
  (fact:global :set {:a 1})
  (fact:global :get))
=> {:a 1}

fact:global-fn ^

[] [cmd & args]
Added 3.0

global getter and setter

v 3.0
(defn fact:global-fn
  ([]
   (fact:global-fn :get))
  ([cmd & args]
   (let [ns (env/ns-sym)]
     (cond (map? cmd)
           (fact:global-map ns cmd)

           (symbol? cmd)
           (let [[sym cmd] [cmd (or (first args)
                                    :init)]
                 data (rt/get-global ns :component sym)]
           (case cmd
               :purge    (fact:global-fn :remove [:component sym])
               :create   (intern ns (with-meta sym {:dynamic true})
                                 (rt/eval-in-ns ns (:create data)))
               :prelim   (if (:prelim data)
                           (rt/eval-in-ns ns ((:prelim data) sym))
                           (rt/eval-in-ns ns sym))
               :setup    (if (:setup data)
                           (rt/eval-in-ns ns ((:setup data) sym))
                           (rt/eval-in-ns ns sym))
               :teardown (if (:teardown data)
                           (rt/eval-in-ns ns ((:teardown data) sym))
                           (rt/eval-in-ns ns sym))
               :init     (do (fact:global-fn sym :create)
                             (fact:global-fn sym :setup))))

           :else
           (case cmd
             :get      (apply rt/get-global ns args)
             :set      (rt/set-global ns (first args))
             :purge    (rt/set-global ns nil)
             :update   (rt/update-global ns (first args))
             :remove   (rt/update-global ns
                                         (fn [m]
                                           (collection/dissoc-nested m (first args))))
             :prelim   (if context/*eval-mode*
                         (rt/eval-in-ns ns (collection/seqify (rt/get-global ns :prelim))))
             :setup    (if context/*eval-mode*
                         (rt/eval-in-ns ns (collection/seqify (rt/get-global ns :setup))))
             :teardown (if context/*eval-mode*
                         (rt/eval-in-ns ns (collection/seqify (rt/get-global ns :teardown))))
             :list     (keys (rt/get-global ns :component))
             (rt/eval-in-ns ns (collection/seqify (rt/get-global ns cmd))))))))
link
(ctx/with-new-context {}
  (fact:global-fn :set {:a 1})
  (fact:global-fn :get))
=> {:a 1}

fact:global-map ^

[ns {:keys [import unimport remove component], :as m}]
Added 3.0

sets and gets the global map

v 3.0
(defn fact:global-map
  ([ns {:keys [import unimport remove component] :as m}]
   (let [unimported (-> unimport rt/get-global :component keys)
         component (or (f/->> (mapv (fn [x]
                                      (cond (vector? x)
                                            (-> (first x) rt/get-global :component
                                                (select-keys (second x)))
                                            :else
                                            (-> x rt/get-global :component)))
                                    import)
                              (apply merge)
                              (apply dissoc % (concat unimported remove))
                              (merge % component))
                       {})
         m (-> (dissoc m :unimport :import :remove)
               (assoc :component component))]
     (rt/update-global ns #(collection/merge-nested % m)))))
link
(ctx/with-new-context {}
  (fact:global-map *ns* {:a 1})
  (rt/get-global *ns*))
=> (contains {:a 1})

fact:ns ^

[& forms]
Added 3.0

provides a macro for managing namespace-related operations within facts

v 3.0
(defmacro fact:ns
  ([& forms]
   `(fact:ns-fn (quote ~forms))))
link
;; (code.test.manage/fact:ns (:link my.ns))
;; => [[:link]]
true => true

fact:ns-alias ^

[ns]
Added 3.0

imports all aliases into current namespace

v 3.0
(defn fact:ns-alias
  ([ns]
   (let [aliases (->> (ns-aliases ns)
                      (mapv (fn [[sym ^clojure.lang.Namespace ns]]
                              [(.getName ns) :as sym])))]
     (do (apply require aliases)
         aliases))))
link
(with-redefs [ns-aliases (fn [_] {'alias (find-ns 'clojure.core)})
              clojure.core/require (fn [& _] nil)]
  (fact:ns-alias *ns*))
=> [['clojure.core :as 'alias]]

fact:ns-fn ^

[forms]
Added 3.0

fact ns getter and setter

v 3.0
(defn fact:ns-fn
  ([forms]
   (mapv (fn [[tag & inputs]]
           (let [fact-fn (case tag
                           :link     rt/add-link
                           :unlink   rt/remove-link
                           :load     (fn [ns]
                                       (rt/add-link ns)
                                       (fact:ns-load ns))
                           :unload   (fn [ns]
                                       (fact:ns-unload)
                                       (rt/remove-link ns))
                           :alias    fact:ns-alias
                           :unalias  fact:ns-unalias
                           :intern   fact:ns-intern
                           :unintern fact:ns-unintern
                           :import   fact:ns-import
                           :unimport fact:ns-unimport
                           :clone    (fn [ns]
                                       (fact:ns-import ns)
                                       (fact:ns-intern ns))
                           :unclone  (fn [ns]
                                       (fact:ns-unintern ns)
                                       (fact:ns-unimport ns))
                           :global   fact:global-fn)]
             (mapv fact-fn inputs)))
         forms)))
link
(with-redefs [rt/add-link (fn [_] :link)]
  (fact:ns-fn '((:link my.ns))))
=> [[:link]]

fact:ns-import ^

[ns]
Added 3.0

loads, imports and aliases current namespace

v 3.0
(defn fact:ns-import
  ([ns]
   (let [sym (if (vector? ns)
               (first ns)
               ns)]
     (rt/add-link sym)
     (fact:ns-load sym)
     (fact:ns-alias sym)
     (fact:global-fn {:import [ns]}))))
link
(with-redefs [rt/add-link (fn [_] nil)
              fact:ns-load (fn [_] nil)
              fact:ns-alias (fn [_] nil)
              fact:global-fn (fn [_] nil)]
  (fact:ns-import 'my.ns))
=> nil

fact:ns-intern ^

[ns]
Added 3.0

imports all interns into current namespace

v 3.0
(defn fact:ns-intern
  ([ns]
   (mapv (partial apply f/intern-var (env/ns-sym))
         (ns-interns ns))))
link
(with-redefs [ns-interns (fn [_] {'sym #'print})
              f/intern-var (fn [& _] nil)]
  (fact:ns-intern *ns*))
=> [nil]

fact:ns-load ^

[ns]
Added 3.0

loads a test namespace

v 3.0
(defn fact:ns-load
  ([ns]
   (project/in-context
    (executive/load-namespace ns))))
link
(with-redefs [project/in-context (fn [f] (f))
              executive/load-namespace (fn [ns & _] ns)]
  (fact:ns-load 'my.ns))
=> 'my.ns

fact:ns-unalias ^

[ns]
Added 3.0

removes all aliases from current namespace

v 3.0
(defn fact:ns-unalias
  ([ns]
   (let [aliases (->> (ns-aliases ns)
                      (mapv (fn [[sym ^clojure.lang.Namespace ns]]
                              [(.getName ns) :as sym])))]
     (doseq [alias aliases]
       (ns-unalias (env/ns-sym) (last alias)))
     aliases)))
link
(with-redefs [ns-aliases (fn [_] {'alias (find-ns 'clojure.core)})
              clojure.core/ns-unalias (fn [& _] nil)]
  (fact:ns-unalias *ns*))
=> [['clojure.core :as 'alias]]

fact:ns-unimport ^

[ns]
Added 3.0

unload, unimports and unalias current namespace

v 3.0
(defn fact:ns-unimport
  ([ns]
   (fact:global-fn {:unimport [ns]})
   (fact:ns-unalias ns)
   (fact:ns-unload ns)
   (rt/remove-link ns)))
link
(with-redefs [fact:global-fn (fn [_] nil)
              fact:ns-unalias (fn [_] nil)
              fact:ns-unload (fn [_] nil)
              rt/remove-link (fn [_] nil)]
  (fact:ns-unimport 'my.ns))
=> nil

fact:ns-unintern ^

[ns]
Added 3.0

removes all interns into current namespace

v 3.0
(defn fact:ns-unintern
  ([ns]
   (let [isyms (sort (keys (ns-interns ns)))]
     (doseq [sym isyms]
       (ns-unmap (env/ns-sym) sym))
     isyms)))
link
(with-redefs [ns-interns (fn [_] {'sym #'print})
              clojure.core/ns-unmap (fn [& _] nil)]
  (fact:ns-unintern *ns*))
=> ['sym]

fact:ns-unload ^

[ns & more]
Added 3.0

unloads a test namespace

v 3.0
(defn fact:ns-unload
  ([ns & more]
   (project/in-context
    (executive/unload-namespace ns))))
link
(with-redefs [project/in-context (fn [f] (f))
              executive/unload-namespace (fn [ns & _] ns)]
  (fact:ns-unload 'my.ns))
=> 'my.ns

6    code.test Guide

code.test is a custom testing framework that provides a robust alternative to clojure.test. It emphasizes clear facts, rich assertions, and integrated management tools.

6.1    Core Concepts

  • Fact: The unit of test execution. Defined via fact.n- Assertion: Uses the => arrow to compare results.n- Checker: Objects that encapsulate validation logic (e.g., throws, contains).

6.2    Usage

6.2.1    Basic Structure

(ns my.ns-test\n  (:require [code.test :refer [fact =>]]))\n\n(fact \"test description\"\n  (expression) => expected-value)

6.2.2    Scenarios

6.2.2.1    1. Testing Exceptions

When testing for exceptions, you often need to verify not just the type, but the message or the data (in ex-info).

(fact \"exception handling\"\n  ;; 1. Check exception type\n  (/ 1 0) => (throws ArithmeticException)\n\n  ;; 2. Check exception type and message\n  (throw (Exception. \"Hello\")) => (throws Exception \"Hello\")\n\n  ;; 3. Check ex-info data using nested checkers\n  ;; Note: The verification of `ex-info` data often requires capturing the exception\n  ;; or using a custom predicate if you need to check the data map deep inside.\n\n  (throw (ex-info \"Error\" {:code 500}))\n  => (throws clojure.lang.ExceptionInfo)\n)

To strictly check ex-info data, you can use a custom predicate or throws-info if available (check coll/throws-info in code.test.checker.collection).

(require '[code.test.checker.collection :as coll])\n\n(fact \"ex-info check\"\n  (throw (ex-info \"msg\" {:a 1}))\n  => (coll/throws-info {:a 1}))

6.2.2.2    2. Complex Data Validation

Use contains, contains-in, and just for detailed map/collection verification.

(fact \"complex data\"\n  (def m {:user {:name \"Bob\" :age 30 :roles [:admin]}})\n\n  ;; Partial match on map\n  m => (contains {:user (contains {:name \"Bob\"})})\n\n  ;; Nested match\n  m => (contains-in [:user :roles] [:admin])\n\n  ;; Exact match (ignoring order for sets/maps where applicable)\n  {:a 1 :b 2} => (just {:b 2 :a 1}))

6.2.2.3    3. Skipping and Focusing Tests

You can control test execution using metadata on the fact form.

;; Skip this test during normal runs\n(fact \"long running test\"\n  {:tag :integration}\n  (Thread/sleep 1000) => nil)\n\n;; Prevent evaluation at definition time (if configured)\n(fact \"pending test\"\n  {:eval false}\n  (future-implementation) => true)

To run tagged tests, you would typically filter them via the runner (e.g., lein test :tag integration).

6.2.2.4    4. Side Effects and Mocking

Use with-redefs to mock functions. Since fact executes in its own scope, these redefinitions are contained.

(defn external-call [] :real)\n\n(fact \"mocking external calls\"\n  (with-redefs [external-call (constantly :mocked)]\n    (external-call) => :mocked)\n\n  ;; Original is restored\n  (external-call) => :real)

6.2.2.5    5. Logic Checkers

Combine checkers for flexible validation.

(require '[code.test.checker.logic :as logic])\n\n(fact \"logic combinations\"\n  10 => (logic/all number? pos?)\n  10 => (logic/any 10 20 30)\n  10 => (logic/is-not neg?))

6.2.2.6    6. Capturing Values for Debugging

You can use the capture checker to inspect intermediate values during test development.

(require '[code.test.checker.common :as common])\n\n(fact \"capturing\"\n  (+ 1 2) => (common/capture common/anything my-var))\n\n;; After running, `my-var` will hold the value 3 in the test namespace.

6.2.2.7    7. Attaching Debug Forms to Failing Expressions

For expressions that are hard to diagnose when they throw or fail an assertion,nyou can attach a debug form via metadata. If the expression errors or its =>ncheck fails, the debug form is evaluated and its result is printed in thenfailure report.

(fact \"debug on throw\"\n  ^{:on-error '(prn :troubleshooting)}\n  (risky-op))\n\n(fact \"debug on failed assertion\"\n  ^{:on-error '(l/with:print (compute-context))}\n  (computed-value) => expected-value)

Important: the value of :on-error must be a quoted form. Metadata mapnvalues are evaluated when the fact is read, so an unquoted form would runneven when the test passes. The debug form runs only when the main expressionnthrows an exception or when the assertion returns false. Its result appearsnin the THROW or FAILED output under an On Error: label.

6.2.3    Running Tests

  • All: lein testn- Namespace: lein test :only my.nsn- Pattern: lein test :in my.pkgn- Save REPL rerun helper: lein test :only my.ns :save-run true writes .hara/runs/run-<timestamp>.run.edn with a copy-pastable code.test/run form.n- Re-run failures: The runner typically outputs instructions or you can use code.manage to focus on failures.

Related

Architecturedocs

how parse, collect, link, and render fit together

code.docdocs

compiler, themes, and authoring primitives

code.frameworkdocs

source and test analysis foundation

code.managedocs

analysis, scaffolding, location, and refactoring tasks

code.mcpdocs

agent-facing MCP tool surface

code.projectdocs

project metadata and file lookup

code.querydocs

structural code queries and transformations

code.tooldocs

build, java, maven, and measurement tooling