dds/scribblings/utils.scrbl

#lang scribble/manual
@(require scribble/example racket/sandbox
	  (for-label typed/racket/base graph
	  	     "../utils.rkt"
		     (only-in typed/graph Graph)
		     (only-in racket/set set)
		     (only-in racket/stream stream->list stream-take)
		     (only-in typed/racket/class class super-new new send)))

@(define utils-evaluator
  (parameterize ([sandbox-output 'string]
  		 [sandbox-error-output 'string]
		 [sandbox-memory-limit 500])
   (make-evaluator 'typed/racket #:requires '("utils.rkt"))))

@(define-syntax-rule (ex . args)
  (examples #:eval utils-evaluator . args))

@(define-syntax-rule (deftypeform . args)
  (defform #:kind "type" . args))

@(define-syntax-rule (deftype . args)
  (defidform #:kind "polymorphic type" . args))

@title[#:tag "utils"]{dds/utils: Various Utilities}

@defmodule[dds/utils]

This module defines miscellaneous utilities, supporting the other modules of
the package: evaluating sexps, manipulating lists,
@hyperlink["https://orgmode.org/"]{Org-mode} interoperability, etc.

@section{Base types}

@deftype[Variable]{

Any Racket symbol.  Designates a variable in a discrete dynamical network.

}

@deftypeform[(VariableMapping A)]{

An immutable mapping (a hash table) assigning elements of type @racket[A] to
the variables.

}

@defform[(NonemptyListof a)]{

A @racket[(Listof a)] which contains at least one element.

}

@section{Type utilities}

Typed Racket's @racket[cast] should only be used as a last resort, because it
installs two contracts which may have a significant performance penalty.
See
@hyperlink["https://racket.discourse.group/t/managing-cast-performance-penalty/905"]{this
discussion} for more details.  The best approach is to use
@hyperlink["https://docs.racket-lang.org/ts-guide/occurrence-typing.html"]{occurrence
typing} either via a direct @racket[if] check using a predicate, or using
@racket[assert].

@defform[(assert-type expr type)]{

@racket[assert] that the type of @racket[expr] is @racket[type].

@ex[
(define some-number : Any 1)
(assert-type some-number Integer)
(assert-type some-number Positive-Integer)
(eval:error (assert-type some-number Zero1))
]}

@deftogether[(@defform[(for/first/typed type-ann (for-clause ...) expr ...+)])
              @defform[(for*/first/typed type-ann (for-clause ...) expr ...+)]]{

Typed versions of @racket[for/first] and @racket[for*/first].

@ex[
(for/first/typed : (Option Integer)
                 ([i (in-range 1 10)]
                  #:when (zero? (modulo i 5)))
  (* i 3))
(for*/first/typed : (Option (Pairof Integer Integer))
                  ([i (in-range 1 10)]
                   [j (in-range 1 10)]
                   #:when (> (+ i j) 5))
  (cons i j))
]

The implementation of these macros is a simplified version the definition of
@hyperlink["https://github.com/racket/typed-racket/blob/9d3264c97aa63b6a7163a219937b88a612add8ab/typed-racket-lib/typed-racket/base-env/prims.rkt#L512"]{@racket[define-for/acc:-variant]}.

}

@defform[(define/abstract/error method-name args ...)]{

In a typed class, defines a public method @racket[method-name] with the
arguments @racket[args] and with the body announcing that this method
is abstract.

@ex[
(define my-abstract-class%
  (class object%
    (super-new)

    (: abstract-increment (-> Integer Integer))
    (define/abstract/error (abstract-increment x))))

(define obj (new my-abstract-class%))
(eval:error (send obj abstract-increment 1))
]}

@defform[(relax-arg-type/any name arg-type)]{

Defines a unary anonymous function whose argument type is @racket[Any], and
which calls @racket[name], with the argument coerced to @racket[arg-type].

@ex[
(relax-arg-type/any add1 Number)
]

The main use of this macro is to allow easily passing different one-argument
functions as arguments of the type @racket[(-> Any Any)].  See for example
@racket[update-graph].

}

@section{Hashtable injection}

This section defines some utilities to streamline the usage of hash tables
mapping symbols to values.  The goal is essentially to avoid having to write
explicit @racket[hash-ref] calls.

@defproc[(eval-with [ht (VariableMapping Any)] [expr Any]) AnyValues]{

Temporarily injects the mappings from the given hash table as bindings in
a namespace including @racket[racket/base] and then evaluates the expression.

@ex[
(let ([ht (hash 'a 1 'b 1)])
  (eval-with ht '(+ b a 1)))
]

The local bindings from the current lexical scope are not
conserved.  Therefore, the following outputs an error about a
missing identifier:

@ex[
(eval:error
(let ([ht (hash 'a 1 'b 1)]
      [z 1])
  (eval-with ht '(+ b z a 1)))
)]}

@defproc[(eval1-with [ht (VariableMapping Any)] [expr Any]) Any]{

Like @racket[eval-with], but returns only the first value computed by
@racket[expr].

@ex[
(let ([ht (hash 'a 1 'b 1)])
  (eval1-with ht '(+ b a 1)))
]}

@defform[(auto-hash-ref/explicit stx)
         #:contracts ([stx (VariableMapping A)])]{

Given a @racket[VariableMapping] and a sequence of symbols, binds these symbols
to the values they are associated with in the hash table, then puts the body in
the context of these bindings.

@ex[
(define env #hash((a . 1) (b . 2)))
(auto-hash-ref/explicit (env a b) (+ a (* 2 b)))
]

Note that only one expression can be supplied in the body.

}

@defform[(auto-hash-ref/: stx)
	 #:contracts ([stx (VariableMapping A)])]{

Given an expression and a @racket[VariableMapping], looks up the symbols with
a leading semicolon and binds them to the value they are associated with in the
hash table.

@ex[
(define env #hash((a . 1) (b . 2)))
(auto-hash-ref/: env (+ :a (* 2 :b)))
]

Thus the symbol @racket[:a] is matched to the key @racket['a] in the
hash table.

Note that only one expression can be supplied in the body.

}

@deftogether[(@defform*[((lambda/: body) (lambda/: type body))]
              @defform*[((λ/: body) (λ/: type body))])]{

Defines an anonymous function with the body @racket[body], taking a hash table
as an argument, and applying @racket[auto-hash-ref/:] to @racket[body] in the
context of this hash table.

@ex[
(let ([ht (hash 'a 1 'b 2)])
  ((λ/: (+ :a :b)) ht))
]

If the optional annotation @racket[type] is specified, the only argument of the
resulting lambda will be of type @racket[type].

@ex[
(let ([ht (hash 'a 1 'b 2)])
  ((λ/: (HashTable Symbol Natural) (+ :a :b)) ht))
]}

@defform*[((define/: name body)
           (define/: name type body))]{

A shortcut for @racket[(define name (lambda/: body))], with the optional
type annotation.

@ex[
(let ([ht (hash 'a 1 'b 2)])
  (: f (-> (HashTable Symbol Natural) Natural))
  (define/: f (+ :a :b))
  (f ht))
]}

@section{Analysis of quoted expressions}

@defproc[(extract-symbols [form Any]) (Listof Symbol)]{

Produces a list of symbols appearing in the quoted expression
passed in the first argument.

@ex[
(extract-symbols '(1 (2 3) x (y z 3)))
]

}

@section{Org-mode interoperability}

Org-mode supports laying out the output of code blocks as tables, which is very
practical for various variable mappings (e.g., states).  However, when the hash
table maps variables to lists, Org-mode will create a column per list element,
which may or may not be the desired effect.  This section defines some
utilities for nicer interoperation with Org-mode tables.  It also defines some
shortcuts to reduce the number of words to type when using dds with Org-mode.
See
@hyperlink["https://git.marvid.fr/scolobb/dds/src/branch/master/example/example.org"]{example.org}
for examples of usage.

@defproc[(any->string [x Any]) String]{

Converts any value to string by calling @racket[display] on it and capturing
the result in a string.

@ex[
(any->string '(a 1 (x y)))
]}

@defproc[(stringify-variable-mapping [ht (VariableMapping Any)]) (VariableMapping String)]{

Converts all the values of a @racket[VariableMapping] to string.

@ex[
(stringify-variable-mapping (hash 'a '(and a b) 'b '(not b)))
]}

@defproc[(string->any [str String]) Any]{

Reads any value from string.

@ex[
(string->any "(or b (not a))")
]}

@defproc[(map-sexp [func (-> Any Any)] [sexp Any]) Any]{

Given a @racket[Sexp], applies the @racket[func] to any object which is not
a list.

@racket[map-sexp] will not check whether @racket[func] is indeed applicable to
every non-list element of @racket[sexp].  If this is not the case, a contract
violation for func will be generated.

@ex[
(map-sexp (λ (x) (add1 (cast x Number))) '(1 2 (4 10) 3))
]}

@defproc*[([(read-org-sexp [str String]) Any]
	   [(unorg [str String]) Any])]{

Reads a @racket[sexp] from a string produced by Org-mode for a named table.

@racket[unorg] is a shortcut for @racket[read-org-sexp].

@ex[
(unorg "(#t \"#t\" \"#t \" '(1 2 \"#f\"))")
]}

@defform[(GeneralPair A B)]{

A @racket[(Pair A B)] or a @racket[(List A B)].

}

@defproc[(unstringify-pairs [pairs (Listof (GeneralPair String Any))])
         (Listof (GeneralPair Symbol Any))]{

Given a list of pairs of strings and some other values (possibly strings),
converts the first element of each pair to a string, and reads the second
element with @racket[string->any] or keeps it as is if it is not a string.

@ex[
(unstringify-pairs '(("a" . 1) ("b" . "(and a (not b))")))
]}

@defproc*[([(read-org-variable-mapping [str String]) (VariableMapping Any)]
           [(unorgv [str String]) (VariableMapping Any)])]{

Reads a @racket[VariableMapping] from a string, such as the one which Org-mode
produces from tables.

@racket[unorgv] is a synonym of @racket[read-org-variable-mapping].

@ex[
(read-org-variable-mapping
  "((\"a\" . \"(and a b)\") (\"b\" . \"(or b (not a))\"))")
]}

@defproc[(read-symbol-list (str String)) (Listof Symbol)]{

Reads a list of symbols from a string.

@ex[
(read-symbol-list "a b c")
]}

@defproc[(drop-first-last (str String)) String]{

Removes the first and the last symbol of a given string.

Useful for removing the parentheses in string representations of lists.

@ex[
(drop-first-last "(a b)")
]}

@defproc[(list-sets->list-strings (lst (Listof (Setof Any)))) (Listof String)]{

Converts a list of sets of symbols to a list of strings containing
those symbols.

@ex[
(list-sets->list-strings (list (set 'x 'y) (set 'z) (set) (set 't)))
]}

@section[#:tag "utils_Pretty_printing"]{Pretty printing}

@defproc[(pretty-print-set [s (U (Setof Any) (Listof Any))]) String]{

Pretty prints a set by listing its elements in alphabetic order.

@ex[
(pretty-print-set (set 'a 'b 1))
(pretty-print-set (list 'a 'b 1))
]}

@defproc[(pretty-print-set-sets [ms (U (Listof (Setof Any)) (Setof (Setof Any)))])
				String]{

Pretty-prints a set of sets of symbols.

Typically used for pretty-printing the annotations on the edges of
a state graph.

@ex[
(pretty-print-set-sets (set (set 'a 'b) (set 'c)))
(pretty-print-set-sets (list (set 'a 'b) (set 'c)))
]}

@section{Additional graph utilities}

All examples in this section depend on @racket[typed/graph]:

@ex[
(require typed/graph)
]

@defproc[(dotit [graph Graph]) Void]{

Typesets the graph via @racket[graphviz] and @racket[display]s it.

@ex[
(dotit (weighted-graph/directed '((1 a b) (2 b c))))
]}

@defproc[(update-vertices/unweighted [graph Graph] [func (-> Any Any)]) Graph]{

Applies a transformation to every vertex in the unweighted graph and returns
the new graph.

If the transformation function maps two vertices to the same values, these
vertices will be merged in the resulting graph.  The transformation function
may be called multiple times for the same vertex.

This function does not rely on @racket[rename-vertex!], so it can be used to
permute vertex labels.

@ex[
(define g (directed-graph '((a b) (b c))))
(define (double-labels [x : Any])
  (define x-str (symbol->string (cast x Symbol)))
  (string->symbol (string-append x-str x-str)))
(dotit (update-vertices/unweighted g double-labels))
]}

@defproc[(update-graph [graph Graph]
		       [#:v-func v-func (-> Any Any) identity]
		       [#:e-func e-func (-> Any Any) identity])
		       Graph]{

Given a (directed) graph, apply the transformation @racket[v-func] to every
vertex label and, if the graph is a weighted graph, the transformation
@racket[e-func] to every edge label.  Both transformations default to identity
functions.  If @racket[graph] is an weighted graph, the result is a weighted
graph.  If @racket[graph] is an unweighted graph, the result is an
unweighted graph.

@ex[
(define g (weighted-graph/directed '((10 a b) (11 b c))))
(define (double-labels [x : Any])
  (define x-str (symbol->string (cast x Symbol)))
  (string->symbol (string-append x-str x-str)))
(define (double-edges [x : Any])
  (* 2 (cast x Number)))
(dotit (update-graph g #:v-func double-labels #:e-func double-edges))
]}

@section{Additional list and hash map utilities}

@defproc[(collect-by-key [keys (Listof a)] [vals (Listof b)])
			 (Values (Listof a) (Listof (Listof b)))]{

Given a list of keys and the corresponding values, collects all the values
associated to any given key and returns a list of keys without duplicates, and
a list containing the corresponding list of values.

If @racket[keys] can be treated as edges (i.e. pairs of vertices), the results
produced by this function are suitable for graph constructors.

@ex[
(collect-by-key '(a b a) '(1 2 3))
]}

@defproc[(collect-by-key/sets [keys (Listof a)] [vals (Listof b)])
			      (Values (Listof a) (Listof (Setof b)))]{

Like @racket[collect-by-key], but produce a list of sets instead of a list
of lists.

@ex[
(collect-by-key/sets '(a b a) '(1 2 3))
]}

@defproc[(ht-values/list->set [ht (HashTable a (Listof b))])
			      (HashTable a (Setof b))]{

Converts the values of a hash table from lists to sets.

@ex[
(ht-values/list->set #hash((a . (1 1))))
]}

@defproc[(hash->list/ordered [ht (HashTable a b)])
			     (Listof (Pairof a b))]{


Returns the key-value pairs of a given hash table in the order in which the
hash table orders them for @racket[hash-map].

@bold{TODO:} Remove after Typed Racket has caught up with Racket 8.4, in which
@racket[hash->list] gets a new optional argument @racket[try-order?].

@ex[
(hash->list/ordered #hash((b . 1) (a . 1)))
]}

@defproc[(hash-replace-keys/ordered [ht (Immutable-HashTable (K1 V))]
				    [new-keys (Listof K2)])
				    (Immutable-HashTable K2 V)]{

Replaces the keys in @racket[ht] by the keys in @racket[new-keys].

The key-value pairs of the hash table @racket[ht] are processed in the
order produced by @racket[hash-map] with @racket[#:try-order?] set to
@racket[#t].

@ex[
(hash-replace-keys/ordered (hash 'a 1 'b 2) '(x y))
]}

@defproc[(multi-split-at [lists (Listof (Listof a))]
			 [pos Integer])
			 (Values (Listof (Listof a)) (Listof (Listof a)))]{

Given a list of lists, splits every single list at the given position, and then
returns two lists of lists: one consisting of the first halves, and the one
consisting of the second halves.

@ex[
(multi-split-at '((1 2 3) (a b c)) 2)
]}

@defproc[(lists-transpose [lists (List (Listof a) ... a)])
			  (Listof (List a ... a))]{

Transposes a list of lists.  The length of the resulting list is the length of
the shortest list in @racket[lists].

This function is essentially @racket[in-parallel], wrapped in
a couple conversions.

@ex[
(lists-transpose '((a b) (1 2)))
(lists-transpose '((a b) (1 2 3) (#t)))
]

As of 2022-04-07, Typed Racket cannot convert the type of
@racket[lists-transpose] to a contract.  The @seclink["utils/untyped"]{untyped
submodule} provides a version of this function which can be used in
untyped code.

}

@defproc[(append-lists [lsts (Listof (List (Listof a) (Listof a)))])
		       (Listof (Listof a))]{

@racket[lsts] is a list of rows, in which each row is split in two halves.
The function returns the list of the same rows, with the two halves appended.

@ex[
(append-lists '(((1 2) (a b))
                ((3 4) (c d))))
]}

@section{Randomness}

@defproc*[([(in-random) (Sequenceof Flonum)]
	   [(in-random [k Integer]) (Sequenceof Nonnegative-Fixnum)]
	   [(in-random [min Integer] [max Integer]) (Sequenceof Nonnegative-Fixnum)])]{

Generates a stream of (inexact) random numbers.  The meaning of the arguments
is the same as for the function @racket[random]:

@itemlist[

@item{@racket[(in-random)] — a stream of random inexact numbers between
0 and 1,}

@item{@racket[(in-random k)] — a stream of random exact integers in the range
@racket[0] to @racket[k]-1.}

@item{@racket[(in-random min max)] — a stream of random exact integers the
range @racket[min] to @racket[max]-1.}

]

@ex[
(require typed/racket/stream)
(stream->list (stream-take (in-random) 5))
(stream->list (stream-take (in-random 10) 5))
(stream->list (stream-take (in-random 5 10) 5))
]}

@section{Additional stream utilities}

@defproc[(cartesian-product-2/stream [s1 (Sequenceof a)]
				     [s2 (Sequenceof b)])
				     (Sequenceof (Pair a b))]{

Generates a stream containing all the pairs of the elements from @racket[s1]
and @racket[s2].  The elements of @racket[s2] are enumerated in order for every
element of the @racket[s1], taken in order as well.

@ex[
(require typed/racket/stream)
(stream->list (cartesian-product-2/stream (in-range 1 5) '(a b)))
]

The streams can be infinite.  If the second stream is infinite, only the first
element of @racket[s1] will be enumerated.

@ex[
(stream->list (stream-take (cartesian-product-2/stream '(a b) (in-naturals)) 10))
]}

@defproc[(cartesian-product/stream [ss (Listof (Sequenceof a))])
				   (Sequenceof (Listof a))]{

Generates a stream containing all the elements of the Cartesian product between
the streams of @racket[ss].

This function relies on @racket[cartesian-product-2/stream] to build the
Cartesian product, so it has the same properties with respect to the order in
which the streams are enumerated.

Union types can be used to build the Cartesian product of streams containing
values of different types.

@ex[
(stream->list (cartesian-product/stream (list (in-range 3) (in-range 4 6) '(a b))))
]}

@section{Boolean operations}

@defproc[(boolean-power [n Integer])
			(Listof (Listof Boolean))]{

Returns the @racket[n]-th Cartesian power of the Boolean domain.

@ex[
(boolean-power 2)
]}

@defproc[(boolean-power/stream [n Integer])
			       (Sequenceof (Listof Boolean))]{

Like @racket[boolean-power], but returns a stream.

@ex[
(stream->list (boolean-power/stream 2))
]}

@defproc[(any->01 [x Any]) (U Zero One)]{

Converts any non-@racket[#f] value to 1 and @racket[#f] to 0.

@ex[
(any->01 #t)
(any->01 #f)
(any->01 'hello)
]}

@defproc[(01->boolean [x (U Zero One)]) Boolean]{

Converts 0 to @racket[#f] and 1 to @racket[#t].

@ex[
(01->boolean 0)
(01->boolean 1)
]}

@section[#:tag "utils/untyped"]{Untyped definitions}

@defmodule[(submod dds/utils untyped)]

@(require (for-label (only-in racket/contract/base listof any/c)))

This submodule contains some functions whose types cannot be converted to
contracts by Typed Racket.

@(define utils-evaluator/untyped
  (parameterize ([sandbox-output 'string]
  		 [sandbox-error-output 'string]
		 [sandbox-memory-limit 50])
   (make-evaluator 'racket #:requires '((submod "utils.rkt" untyped)))))

@(define-syntax-rule (ex/untyped . args)
  (examples #:eval utils-evaluator/untyped . args))

@defproc[(lists-transpose [lists (listof (listof any/c))])
			  (listof (listof any/c))]{

Transposes a list of lists.  The length of the resulting list is the length of
the shortest list in @racket[lists].

This function is essentially @racket[in-parallel], wrapped in
a couple conversions.

@ex/untyped[
(lists-transpose '((a b) (1 2)))
(lists-transpose '((a b) (1 2 3) (#t)))
]}