1    Introduction

1.1    Overview

std.lib.walk is part of the standard foundation library set. This page collects the public API reference for the namespace.

2    Walkthrough

2.1    Traversal

postwalk and prewalk apply a function to every node of a nested data structure. postwalk visits children first; prewalk visits the parent first.

increment every number in a nested structure

(postwalk (fn [x] (if (number? x) (inc x) x))
          {:a [1 2 {:b 3}]})
=> {:a [2 3 {:b 4}]}

(prewalk (fn [x] (if (number? x) (inc x) x))
         {:a [1 2 {:b 3}]})
=> {:a [2 3 {:b 4}]}

2.2    Key conversion

These helpers recursively convert map keys between keywords and strings, including snake_case and spear-case variants.

convert string keys to keywords

(keywordize-keys {"a" 1 "b" {"c" 2}})
=> {:a 1 :b {:c 2}}

convert keyword keys to strings

(stringify-keys {:a 1 :b {:c 2}})
=> {"a" 1 "b" {"c" 2}}

convert between snake_case and spear-case

(keyword-spearify-keys {"a_b_c" [{"e_f_g" 1}]})
=> {:a-b-c [{:e-f-g 1}]}

(string-snakify-keys {:a-b-c [{:e-f-g 1}]})
=> {"a_b_c" [{"e_f_g" 1}]}

2.3    Replacement

prewalk-replace and postwalk-replace substitute matching forms throughout a tree. They are handy for templating and symbolic substitution.

replace symbols throughout a form

(prewalk-replace {'x 10 'y 20}
                 '(+ x (* y x)))
=> '(+ 10 (* 20 10))

(postwalk-replace {'x 10 'y 20}
                  '(+ x (* y x)))
=> '(+ 10 (* 20 10))

2.4    Searching

walk:find collects every node satisfying a predicate, and walk:keep collects the non-nil results of applying a function to each node.

find nested vectors starting with an even number

(walk:find (fn [x]
             (and (vector? x)
                  (even? (first x))))
           [[1] [[3 [4 [6]]]]])
=> #{[4 [6]] [6]}

keep transformed odd numbers

(walk:keep (fn [x]
             (if (odd? x)
               (+ 10 x)))
           [[1] [[3 [4 [6]]]]])
=> #{11 13}

2.5    End-to-end: normalise external JSON data

A common pipeline is to keywordize keys from an external source, transform the values, then stringify the keys again for serialisation.

keywordize, transform, and stringify nested data

(->> {"user_name" "Ada"
      "user_age"  36
      "address"   {"street_name" "Maple"}}
     keywordize-keys
     (postwalk (fn [x] (if (string? x) (.toUpperCase x) x)))
     stringify-keys)
=> {"user_name" "ADA"
    "user_age"  36
    "address"   {"street_name" "MAPLE"}}

3    API

keyword-spearify-keys ^

[m]
Added 4.0

recursively transfroms all map keys to spearcase

v 4.0
(defn keyword-spearify-keys
  ([m]
   (let [f (fn [[k v]] (if (string? k) [(keyword (.replaceAll ^String k "_" "-")) v] [k v]))]
    ;; only apply to maps
     (postwalk (fn [x] (if (map? x) (into {} (map f x)) x)) m))))
link
(keyword-spearify-keys  {"a_b_c" [{"e_f_g" 1}]})
=> {:a-b-c [{:e-f-g 1}]}

keywordize-keys ^

[m]
Added 3.0

recursively transforms all map keys from strings to keywords.

v 3.0
(defn keywordize-keys
  ([m]
   (let [f (fn [[k v]] (if (or (string? k)
                               (symbol? k))
                         [(keyword k) v] [k v]))]
    ;; only apply to maps
     (postwalk (fn [x] (if (map? x) (into {} (map f x)) x)) m))))
link
(resolve 'keywordize-keys) => var?

macroexpand-all ^

[form]
Added 3.0

recursively performs all possible macroexpansions in form.

v 3.0
(defn macroexpand-all
  ([form]
   (prewalk (fn [x] (if (seq? x) (macroexpand x) x)) form)))
link

postwalk ^

[f form]
Added 3.0

performs a depth-first, post-order traversal of form

v 3.0
(defn postwalk
  ([f form]
   (walk (partial postwalk f) f form)))
link
(resolve 'postwalk) => var?

postwalk-replace ^

[smap form]
Added 3.0

recursively transforms form by replacing keys in smap with their values.

v 3.0
(defn postwalk-replace
  ([smap form]
   (postwalk (fn [x] (if (contains? smap x) (smap x) x)) form)))
link

prewalk ^

[f form]
Added 3.0

like postwalk, but does pre-order traversal.

v 3.0
(defn prewalk
  ([f form]
   (walk (partial prewalk f) identity (f form))))
link
(resolve 'prewalk) => var?

prewalk-replace ^

[smap form]
Added 3.0

recursively transforms form by replacing keys in smap with their values.

v 3.0
(defn prewalk-replace
  ([smap form]
   (prewalk (fn [x] (if (contains? smap x) (smap x) x)) form)))
link

string-snakify-keys ^

[m]
Added 4.0

recursively transforms keyword to string keys

v 4.0
(defn string-snakify-keys
  ([m]
   (let [f (fn [[k v]] (if (keyword? k) [(.replaceAll ^String (name k) "-" "_") v] [k v]))]
    ;; only apply to maps
     (postwalk (fn [x] (if (map? x) (into {} (map f x)) x)) m))))
link
(string-snakify-keys
 {:a-b-c [{:e-f-g 1}]})
=> {"a_b_c" [{"e_f_g" 1}]}

stringify-keys ^

[m]
Added 3.0

recursively transforms all map keys from keywords to strings.

v 3.0
(defn stringify-keys
  ([m]
   (let [f (fn [[k v]] (if (keyword? k) [(name k) v] [k v]))]
    ;; only apply to maps
     (postwalk (fn [x] (if (map? x) (into {} (map f x)) x)) m))))
link

walk ^

[inner outer form]
Added 3.0

traverses form, an arbitrary data structure

v 3.0
(defn walk
  [inner outer form]
  (cond    
    (list? form) (outer (with-meta (apply list (map inner form)) (meta form)))

    (instance? clojure.lang.IMapEntry form) (outer (vec (map inner form)))
    (seq? form) (outer (doall (with-meta (map inner form) (meta form))))

    (set? form) (outer (with-meta (into (empty form) (map inner form)) (meta form)))
    (map? form) (outer (with-meta (into (empty form) (map inner form)) (meta form)))
    (vector? form) (outer (with-meta (mapv inner form) (meta form)))

    (instance? clojure.lang.IRecord form)
    (outer (reduce (fn [r x] (conj r (inner x))) form form))
    (coll? form) (outer (into (empty form) (map inner form)))
    :else (outer form)))
link
(resolve 'walk) => var?

walk:contains ^

[pred form]
Added 4.0

recursively walks form to check for containment

v 4.0
(defn walk:contains
  ([pred form]
   (let [found (volatile! false)]
     (prewalk (fn [x]
                (if (try (pred x) (catch Throwable t))
                  (vreset! found true))
                x)
              form)
     @found)))
link

walk:find ^

[pred form]
Added 4.0

recursively walks to find all matching forms

v 4.0
(defn walk:find
  ([pred form]
   (let [found (volatile! #{})]
     (postwalk (fn [x]
                 (if (try (pred x) (catch Throwable t))
                   (vswap! found conj x))
                 x)
               form)
     @found)))
link
(walk:find (fn [x]
             (and (vector? x)
                  (even? (first x))))
           [[1] [[3 [4 [6]]]]])
=> #{[4 [6]]
     [6]}

walk:keep ^

[f form]
Added 4.0

recursively walks and keeps all processed forms

v 4.0
(defn walk:keep
  ([f form]
   (let [found (volatile! #{})]
     (postwalk (fn [x]
                 (try (let [val (f x)]
                        (if val (vswap! found conj val)))
                      (catch Throwable t))
                 x)
               form)
     @found)))
link
(walk:keep (fn [x]
             (if (odd? x)
               (+ 10 x)))
           [[1] [[3 [4 [6]]]]])
=> #{13 11}

4    std.lib.walk: A Comprehensive Summary

The std.lib.walk namespace provides a set of powerful functions for traversing and transforming arbitrary Clojure data structures. It extends the core clojure.walk functionality with additional utilities for key transformation, macro expansion, and searching within nested data. This module is fundamental for tasks involving deep inspection, modification, or normalization of complex data.

Key Features and Concepts:

  1. Core Traversal Functions:n walk: The foundational function for traversing data structures. It takes inner and outer functions, which are applied to the children and the form itself, respectively. It handles various collection types (lists, maps, vectors, sets, records).n postwalk: Performs a depth-first, post-order traversal. The f function is applied to the form after its children have been processed.n prewalk: Performs a depth-first, pre-order traversal. The f function is applied to the form before its children have been processed.nn2. Key Transformation Utilities:n keywordize-keys: Recursively transforms all map keys from strings or symbols to keywords.n keyword-spearify-keys: Recursively transforms string keys with underscores (e.g., "abc") to kebab-case keywords (e.g., :a-b-c).n stringify-keys: Recursively transforms all map keys from keywords to strings.n string-snakify-keys: Recursively transforms kebab-case keyword keys (e.g., :a-b-c) to snake-case strings (e.g., "abc"). These are particularly useful for interoperability with systems that prefer different key naming conventions (e.g., JSON APIs, databases).nn3. Replacement and Macro Expansion:n prewalk-replace: Recursively replaces elements in a form based on a provided substitution map, performing a pre-order traversal.n postwalk-replace: Similar to prewalk-replace, but performs a post-order traversal.n macroexpand-all: Recursively performs all possible macroexpansions within a given form, which is invaluable for inspecting the fully expanded code generated by macros.nn4. Searching and Filtering:n walk:contains: Recursively walks a form to check if any element satisfies a given predicate.n walk:find: Recursively walks a form to find all elements that satisfy a given predicate, returning them as a set.n * walk:keep: Recursively walks a form, applies a function f to each element, and collects all non-nil results into a set.

Usage and Importance:

The std.lib.walk module is a cornerstone for data manipulation and code analysis within the foundation-base project. It enables:

  • Data Normalization: Easily convert data between different key naming conventions (e.g., snake_case strings to kebab-case keywords) for consistent internal representation or external API compatibility.n Code Transformation: Perform complex transformations on Clojure code (represented as data) for tasks like static analysis, refactoring, or code generation.n Deep Inspection: Efficiently search for specific patterns or values within deeply nested data structures without writing custom recursive logic.n* Meta-programming: The macroexpand-all function is crucial for understanding and debugging complex macros, which are heavily used in the foundation-base project for transpilation and runtime management.

By providing these versatile tools, std.lib.walk significantly enhances the project's ability to process, analyze, and transform both data and code, contributing to its overall flexibility and power.

Related

jvm.artifactdocs

artifact coordinates and paths

jvm.classloaderdocs

classpath and classloaders

jvm.depsdocs

JVM dependency resolution

jvm.monitordocs

JVM metrics

jvm.namespacedocs

namespace management

jvm.protocoldocs

JVM extension protocols

jvm.reflectdocs

Java reflection

jvm.requiredocs

dependency-aware requiring