1    Introduction

1.1    Overview

std.lib.system is part of the standard foundation library set. This page collects the public API reference for the namespace.

2    Walkthrough

2.1    Primitives and system predicates

primitive? recognises basic scalar values, and system? checks whether a value satisfies the ISystem protocol.

detect primitive values

(primitive? 1)
=> true

(primitive? "hello")
=> true

(primitive? {:a 1})
=> false

a plain map is not a system

(system? {})
=> false

2.2    System arrays

array builds a component array from a vector of configs. array? checks the result, and info-array summarises the array for display.

construct and inspect a component array

(let [arr (array {:constructor identity} [{:id 1} {:id 2}])]
  (array? arr)
  => true

  (count arr)
  => 2

  (vec arr)
  => [{:id 1} {:id 2}])

2.3    Topology helpers

Topologies can be written in short form and expanded with long-form. From the expanded form you can extract dependencies and exposed keys.

expand a topology to long form

(long-form {:db    [identity]
            :cache [identity :db]})
=> (contains {:db    (contains {:type :build
                                :constructor identity
                                :dependencies []})
              :cache (contains {:type :build
                                :constructor identity
                                :dependencies [:db]})})

extract dependencies and exposed keys

(let [topo (long-form {:db     [identity]
                       :cache  [identity :db]
                       :public {:expose identity :in :cache}})]
  (get-dependencies topo)
  => {:db [] :cache [:db] :public [:cache]}

  (get-exposed topo)
  => [:public])

2.4    Partial systems

valid-subcomponents and subsystem let you work with a subset of a larger system. These are useful for tests and for starting only the components required by a particular feature.

find valid subcomponents for a partial system

(let [topo {:a [identity]
            :b [identity :a]
            :c [identity :b]
            :d [identity]}]
  (valid-subcomponents topo [:c])
  => (contains [:a :b :c] :in-any-order))

2.5    End-to-end: build, start, and inspect a system

A complete workflow defines a topology, constructs a system, starts it, checks its health, inspects its info, and stops it.

build and lifecycle a small system

(let [topo  {:db    [identity]
             :cache [identity :db]}
      sys   (system topo
                    {:db    {:host "localhost"}
                     :cache {:ttl 60}})
      sys   (start-system sys)]
  (keys sys)
  => (contains [:db :cache] :in-any-order)

  (health-system sys)
  => {:status :ok}

  (:db (info-system sys))
  => {:host "localhost"}

  (keys (stop-system sys))
  => (contains [:db :cache] :in-any-order))

3    API

array ^

[{:keys [constructor defaults]} config]
Added 3.0

constructs a system array

v 3.0
(defn array
  ([{:keys [constructor defaults]} config]
   (if (vector? config)
     (let [defaults (coll/merge-nested (meta config) defaults)]
       (ComponentArray. (mapv (fn [entry]
                                (if (map? entry)
                                  (constructor (coll/merge-nested defaults entry))
                                  entry))
                              config)))
     (throw (ex-info (str "Config " config " has to be a vector.") {:config config})))))
link
(array {:constructor map->Database} [{:id 1} {:id 2}])
=> any?

array? ^

[x]
Added 3.0

checks if object is a system array

v 3.0
(defn array?
  ([x]
   (instance? ComponentArray x)))
link
(array? (array {:constructor map->Database} [{}]))
=> true

primitive? ^

[x]
Added 3.0

checks if a component is a primitive type

v 3.0
(defn primitive?
  ([x]
   (or (string? x)
       (number? x)
       (boolean? x)
       (h/regexp? x)
       (uuid? x)
       (uri? x)
       (h/url? x))))
link
(primitive? 1) => true

(primitive? {}) => false

scaffold:all ^

[]
Added 3.0

lists all running scaffolds

v 3.0
(defn scaffold:all
  ([]
   (sort @*running*)))
link

scaffold:clear ^

[] [ns]
Added 3.0

clears the current system

v 3.0
(defn scaffold:clear
  ([]
   (scaffold:clear (env/ns-sym)))
  ([ns]
   (let [{:keys [instance]} (scaffold:current ns)]
     (if (deref instance)
       (do (swap! *running* disj (str ns))
           (reset! instance nil))
       (throw (ex-info "Already cleared" {:instance (deref instance)}))))))
link

scaffold:create ^

[] [ns]
Added 3.0

creates a system

v 3.0
(defn scaffold:create
  ([] (scaffold:create (env/ns-sym)))
  ([ns]
   (let [{:keys [topology config options]} (scaffold:current ns)
         config (if (fn? config) (config) config)]
     (type/system topology config options))))
link

scaffold:current ^

[] [ns]
Added 3.0

returns the current scaffold

v 3.0
(defn scaffold:current
  ([]
   (scaffold:current (env/ns-sym)))
  ([ns]
   (get @*registry* (keyword (str ns)))))
link

scaffold:deregister ^

[] [ns]
Added 3.0

deregisters a scaffold in the namespace

v 3.0
(defn scaffold:deregister
  ([]
   (scaffold:deregister (env/ns-sym)))
  ([ns]
   (let [ns-key (keyword (str ns))]
     (swap! *registry* dissoc ns-key)
     ns-key)))
link
(resolve 'scaffold:deregister) => var?

scaffold:new ^

[] [ns]
Added 3.0

creates and starts a system

v 3.0
(defn scaffold:new
  ([]
   (scaffold:new (env/ns-sym)))
  ([ns]
   (let [sys (scaffold:create ns)]
     (component/start sys))))
link

scaffold:register ^

[] [m] [ns {:keys [instance topology config options wrap], :or {instance (env/ns-get ns "*instance*"), topology (env/ns-get ns "*topology*"), config (env/ns-get ns "config"), options {}, wrap identity}}]
Added 3.0

registers a scaffold in the namespace

v 3.0
(defn scaffold:register
  ([]
   (scaffold:register {}))
  ([m]
   (scaffold:register (env/ns-sym) m))
  ([ns {:keys [instance topology config options wrap]
        :or {instance  (env/ns-get ns "*instance*")
             topology  (env/ns-get ns "*topology*")
             config    (env/ns-get ns "config")
             options   {}
             wrap      identity}}]
   (let [ns-key (keyword (str ns))]
     (swap! *registry* assoc ns-key {:instance instance
                                     :topology topology
                                     :config config
                                     :options options
                                     :wrap wrap})
     ns-key)))
link
(resolve 'scaffold:register) => var?

(scaffold:register)

scaffold:registered ^

[]
Added 3.0

lists all registered scaffolds

v 3.0
(defn scaffold:registered
  ([]
   (vals @*registry*)))
link

scaffold:restart ^

[] [ns]
Added 3.0

restarts the system

v 3.0
(defn scaffold:restart
  ([]
   (scaffold:restart (env/ns-sym)))
  ([ns]
   (let [{:keys [instance]} (scaffold:current ns)]
     (if (deref instance) (scaffold:stop ns))
     (scaffold:start ns))))
link

scaffold:start ^

[] [ns]
Added 3.0

starts the system

v 3.0
(defn scaffold:start
  ([]
   (scaffold:start (env/ns-sym)))
  ([ns]
   (let [{:keys [instance]} (scaffold:current ns)]
     (if instance
       (if-not (deref instance)
         #_(log/block {:log/label "START SYSTEM"
                       :log/state (str ns)
                       :meter/reflect true
                       :console/retain true
                       :console/style {:header {:label {:text :yellow}
                                                :position {:display {:default false}}
                                                :date  {:text :yellow}}}})
         (let [sys (scaffold:new ns)]
           (reset! instance sys)
           (swap! *running* conj (str ns))
           sys)
         (throw (ex-info "Already started" {:instance (keys (deref instance))})))))))
link

scaffold:stop ^

[] [ns]
Added 3.0

stops the system

v 3.0
(defn scaffold:stop
  ([]
   (scaffold:stop (env/ns-sym)))
  ([ns]
   (let [{:keys [instance]} (scaffold:current ns)]
     (if (and instance (deref instance))
       #_(log/block {:log/label "STOP SYSTEM"
                     :log/state (str ns)
                     :meter/reflect true
                     :console/retain true
                     :console/style {:header {:label {:text :yellow}
                                              :position {:display {:default false}}
                                              :date  {:text :yellow}}}})
       (do (component/stop @instance)
           (swap! *running* disj (str ns))
           nil)))))
link

scaffold:stop-all ^

[]
Added 3.0

kills all running scaffolds

v 3.0
(defn scaffold:stop-all
  ([]
   (mapv (fn [ns] (scaffold:stop ns))
         @*running*)))
link

subsystem ^

[system keys]
Added 3.0

returns the subsystem given certain keys

v 3.0
(defn subsystem
  ([system keys]
   (let [subkeys (system-subkeys system keys)
         {:keys [build order dependencies]} (meta system)]
     (with-meta (type/map->ComponentSystem (select-keys system subkeys))
       {:build (select-keys build subkeys)
        :order (filter subkeys order)
        :dependencies (select-keys build subkeys)}))))
link
(subsystem +sys+ #{:entry})
=> any?

system ^

[topology config] [topology config {:keys [partial? tag display notify], :as opts}]
Added 3.0

creates a system of components

v 3.0
(defn system
  ([topology config]
   (system topology config {:partial false}))
  ([topology config {:keys [partial? tag display notify] :as opts}]
   (let [build   (topology/long-form topology)
         expose  (topology/get-exposed build)
         dependencies (topology/get-dependencies build)
         order (sort/topological-sort dependencies)
         initial  (apply dissoc build expose)]
     (-> (reduce-kv (fn [sys k {:keys [constructor compile defaults] :as build}]
                      (let [cfg (get config k)]
                        (assoc sys k (cond (= compile :array)
                                           (array/array (update build :defaults merge defaults)
                                                        cfg)

                                           :else
                                           (constructor (coll/merge-nested defaults cfg))))))
                    (ComponentSystem.)
                    initial)
         (with-meta (merge {:build   build
                            :order   order
                            :dependencies dependencies}
                           opts))))))
link
;; The topology specifies how the system is linked
(def topo {:db        [map->Database]
           :files     [[map->Filesystem]]
           :catalogs  [[map->Catalog] [:files {:type :element :as :fs}] :db]})

;; The configuration customises the system
(def cfg  {:db     {:type :basic
                    :host "localhost"
                    :port 8080}
           :files [{:path "/app/local/1"}
                   {:path "/app/local/2"}]
           :catalogs [{:id 1}
                      {:id 2}]})

;; `system` will build it and calling `start` initiates it
(def sys (-> (system topo cfg) component/start))

;; Check that the `:db` entry has started
(:db sys)
=> (just {:status "started",
          :type :basic,
          :port 8080,
          :host "localhost"})

;; Check the first `:files` entry has started
(-> sys :files first)
=> (just {:status "started",
          :path "/app/local/1"})

;; Check that the second `:store` entry has started
(->> sys :catalogs second)
=> (contains-in {:id 2
                 :status "started"
                 :db {:status "started",
                      :type :basic,
                      :port 8080,
                      :host "localhost"}
                 :fs {:path "/app/local/2", :status "started"}})

system? ^

[obj]
Added 3.0

checks if a component extends ISystem

v 3.0
(defn system?
  ([obj]
   (satisfies? ISystem obj)))
link
(system? 1)
=> false

valid-subcomponents ^

[full-topology keys]
Added 3.0

returns only the components that will work (for partial systems)

v 3.0
(defn valid-subcomponents
  ([full-topology keys]
   (let [expose-keys (topology/get-exposed full-topology)
         valid-keys (set (concat expose-keys keys))
         sub-keys (->> full-topology
                       topology/get-dependencies
                       topology/all-dependencies
                       (coll/map-entries (fn [[k v]] [k (conj v k)])))]
     (reduce-kv (fn [arr k v]
                  (if (set/superset? valid-keys v)
                    (conj arr k)
                    arr))
                []
                sub-keys))))
link
(valid-subcomponents
 (topology/long-form {:model  [identity]
                      :tag    [{:expose :tag} :model]
                      :kramer [identity :tag]})
 [:model])
=> [:model :tag]

wait ^

[system key] [system key {:keys [success error failure max-retries timeout], :as callback}]
Added 3.0

wait for a system entry to come online

v 3.0
(defn wait
  ([system key]
   (wait system key *callback*))
  ([system key {:keys [success error failure max-retries timeout] :as callback}]
   (loop [current-retries 0]
     (if (component/with [sub (subsystem system #{key})]
           (= :ok (:status (component/health (get sub key)))))
       (if success (success system key))
       (if (or (nil? max-retries)
               (< current-retries max-retries))
         (do
           (Thread/sleep (or timeout *timeout*))
           (if error (error system key))
           (recur (inc current-retries)))
         (when failure
           (failure system key)
           (throw (ex-info "Max retries reached" {:max-retries max-retries}))))))))
link
(resolve 'wait) => var?

wait-for ^

[system keys] [system keys {:keys [start final], :as callback}]
Added 3.0

wait for all system entries to come online

v 3.0
(defn wait-for
  ([system keys]
   (wait-for system keys *callback*))
  ([system keys {:keys [start final] :as callback}]
   (let [invalid (remove #(contains? system %) keys)]
     (if-not (empty? invalid)
       (throw (ex-info "Invalid keys" {:invalid invalid
                                       :options (clojure.core/keys system)}))))
   (if start (start))
   (doseq [key keys]
     (wait system key callback))
   (if final (final))))
link
(resolve 'wait-for) => var?

4    std.lib.system: A Comprehensive Summary

The std.lib.system module provides a powerful and flexible framework for defining, building, and managing complex systems composed of various components. It introduces a declarative way to define system topologies, manage component lifecycles (start, stop, health checks), and handle dependencies between components. This module is crucial for orchestrating application startup, shutdown, and dynamic reconfiguration within the foundation-base project.

The module is organized into several sub-namespaces:

4.1    std.lib.system.common

This namespace defines basic predicates and interfaces for system components.

  • ISystem Protocol: A marker protocol for system components.n system? [obj]: Checks if an object implements the ISystem protocol.n primitive? [x]: Checks if an object is a primitive Clojure type (string, number, boolean, regex, UUID, URI, URL).

4.2    std.lib.system.array

This namespace provides utilities for managing arrays of components, treating them as a single component.

  • ComponentArray Deftype: A record that wraps a vector of components, allowing it to be treated as a single component. It implements clojure.lang.Seqable, clojure.lang.IObj, clojure.lang.IMeta, clojure.lang.Counted, clojure.lang.Indexed, and std.protocol.component/IComponent and IComponentQuery.n info-array [arr]: Returns information about the elements within a ComponentArray.n health-array [carr]: Returns the health status of a ComponentArray, aggregating the health of its individual components.n start-array [carr]: Starts all components within a ComponentArray.n stop-array [carr]: Stops all components within a ComponentArray.n array [opts config]: Constructs a ComponentArray from a configuration, applying a constructor and defaults to each element.n array? [x]: Checks if an object is a ComponentArray.

4.3    std.lib.system.display

This namespace is currently empty, suggesting it's a placeholder for future display-related functionalities for system components.

4.4    std.lib.system.partial

This namespace provides functionalities for working with partial systems, allowing for the selection of subcomponents and waiting for their readiness.

  • *timeout*: A dynamic var for the default timeout in milliseconds for wait.n *callback*: A dynamic var for default callback functions used in wait.n valid-subcomponents [full-topology keys]: Filters a topology to return only components that are valid subcomponents given a set of keys and exposed components.n system-subkeys [system keys]: Recursively finds all subcomponents that are dependencies of the given keys within a system.n subsystem [system keys]: Extracts a subsystem from a larger system, including only the specified keys and their dependencies.n wait [system key & [callback]]: Waits for a specific component (key) within a system to become healthy, with retry logic and configurable callbacks.n wait-for [system keys & [callback]]: Waits for all specified keys within a system to become healthy.

4.5    std.lib.system.scaffold

This namespace provides a scaffolding mechanism for managing system instances, particularly for testing and development environments. It allows for registering, creating, starting, stopping, and restarting system configurations.

  • *registry*: A global atom storing registered scaffold configurations.n *running*: A global atom tracking currently running scaffold instances.n *timeout*: A dynamic var for the default timeout.n scaffold:register [& [ns m]]: Registers a system configuration (topology, config, instance var) for a given namespace.n scaffold:deregister [& [ns]]: Deregisters a system configuration.n scaffold:current [& [ns]]: Returns the registered scaffold configuration for a namespace.n scaffold:create [& [ns]]: Creates a new system instance from a registered scaffold configuration.n scaffold:new [& [ns]]: Creates and starts a new system instance from a registered scaffold.n scaffold:stop [& [ns]]: Stops a running system instance.n scaffold:start [& [ns]]: Starts a system instance.n scaffold:clear [& [ns]]: Clears (stops and unsets) a running system instance.n scaffold:restart [& [ns]]: Restarts a running system instance.n scaffold:registered []: Lists all registered scaffold configurations.n scaffold:all []: Lists all currently running scaffold instances.n scaffold:stop-all []: Stops all running scaffold instances.

4.6    std.lib.system.type

This namespace defines the core ComponentSystem record and the logic for building, starting, and stopping complex systems based on a declarative topology.

  • ComponentSystem Deftype: The central record for a system, which is essentially a map of components. It implements std.lib.system.common/ISystem, std.protocol.component/IComponent, and std.protocol.track/ITrack.n *stop-fn*: A dynamic var that can be bound to component/stop or component/kill to control how components are stopped.n info-system [sys]: Returns detailed information about the components within a system.n health-system [system]: Returns the overall health status of a system, aggregating the health of its components.n remote?-system [system]: Checks if any component in the system is a remote resource.n system-string [sys]: Provides a string representation of a system.n system [topology config & [opts]]: The main function for creating a ComponentSystem. It takes a topology (declarative definition of components and their dependencies), config (component-specific configurations), and opts. It uses topology/long-form to expand the topology and sort/topological-sort to determine the startup order.n system-import [component system import]: Imports a component from the system into another component (e.g., an array of components).n system-expose [component system opts]: Exposes a subcomponent from the system based on options.n start-system [system]: Starts all components in the system according to their topological order, handling imports, exposes, and lifecycle hooks.n system-deport [component import]: Deports (removes) an imported component from another component.n stop-system [system]: Stops all components in the system in reverse topological order, handling deports and lifecycle hooks.n kill-system [system]: Forcefully stops all components in the system by binding *stop-fn* to component/kill.

4.7    std.lib.system (Facade Namespace)

This namespace acts as a facade, re-exporting key functions from its sub-namespaces for convenience.

  • h/intern-in: Interns various functions from common, array, type, partial, and scaffold into this namespace.

Overall Importance:

The std.lib.system module is a cornerstone of the foundation-base project's architecture, providing a robust and declarative way to manage complex applications. Its key contributions include:

  • Declarative System Definition: Allows developers to define application structure and dependencies in a clear, concise, and extensible manner.n Automated Lifecycle Management: Handles the intricate details of starting, stopping, and managing the health of interconnected components, including dependency resolution.n Modularity and Reusability: Promotes the creation of reusable components that can be easily integrated into different systems.n Dynamic Configuration: Supports flexible configuration of components and their interactions.n Testing and Development Support: The scaffolding mechanism simplifies the setup and teardown of test environments.n* Extensibility: The protocol-driven design allows for easy integration of new component types and custom lifecycle behaviors.

By offering these comprehensive system management capabilities, std.lib.system significantly enhances the foundation-base project's ability to build, deploy, and maintain sophisticated multi-language applications with greater reliability and efficiency.

Related

jvm.artifactdocs

artifact coordinates and paths

jvm.classloaderdocs

classpath and classloaders

jvm.depsdocs

JVM dependency resolution

jvm.monitordocs

JVM metrics

jvm.namespacedocs

namespace management

jvm.protocoldocs

JVM extension protocols

jvm.reflectdocs

Java reflection

jvm.requiredocs

dependency-aware requiring