Distinguish between "rendered" and "unrendered" Tensors.
There are now three types of `Tensor`:
- `Tensor Value a`: rendered value
- `Tensor Ref a`: rendered reference
- `Tensor Build a` : unrendered value
The extra bookkeeping makes it easier to track (and enforce) which tensors are
rendered or not. For examples where this has been confusing in the past, see
With this change, pure ops look similar to before, returning `Tensor Build`
instead of `Tensor Value`. "Stateful" (monadic) ops are unchanged. For
example:
add :: OneOf [..] t => Tensor v'1 t -> Tensor v'2 t -> Tensor Build t
assign :: (MonadBuild m, TensorType t)
=> Tensor Ref t -> Tensor v'2 t -> m (Tensor Ref t)
The `gradients` function now requires that the variables over which it's
differentiating are pre-rendered:
gradients :: (..., Rendered v2) => Tensor v1 a -> [Tensor v2 a]
-> m [Tensor Value a]
(`Rendered v2` means that `v2` is either a `Ref` or a `Value`.)
Additionally, the implementation of `gradients` now takes care to render every
intermediate value when performing the reverse accumulation. I suspect this
fixes an exponential blowup for complicated expressions.
Each op `foo :: ...` now has a corresponding `foo' :: OpParams -> ...`
which lets you set optional attributes. `OpParams` is currently a type alias for
`OpDef -> OpDef`. In the future we should consider more type safety, e.g.,
using type-level strings and OverloadedLabels for optional attributes.
I used it to replace a few manual `buildOp`s in our code with the codegenerated
ops, now that it's easier to set attributes. I also removed `tensorAttr` and
`named` since it's now possible to set those op attributes directly.
Although this clutters up the API a bit, I think it's simpler than using type
classes to implement optional arguments (as in, for example, `Text.Printf`) --
especially in terms of type inference with the rest of the library.
This change adds a class that both `Build` and `Session` are instances of:
class MonadBuild m where
build :: Build a -> m a
All stateful ops (generated and manually written) now have a signature that returns
an instance of `MonadBuild` (rather than just `Build`). For example:
assign_ :: (MonadBuild m, TensorType t)
=> Tensor Ref t -> Tensor v t -> m (Tensor Ref t)
This lets us remove a bunch of spurious calls to `build` in user code. It also
lets us replace the pattern `buildAnd run foo` with the simpler pattern `foo >>= run`
(or `run =<< foo`, which is sometimes nicer when foo is a complicated expression).
I went ahead and deleted `buildAnd` altogether since it seems to lead to
confusion; in particular a few tests had `buildAnd run . pure` which is
actually equivalent to just `run`.
Adds a new type `ListOf` which wraps a heterogeneous list; for example,
`ListOf (Tensor Value) '[Int32, Float]` represents a list of two
elements: a tensor of int32s and a tensor of floats.
Also changes the `Queue2` type (which suppored pairs of tensors) to
`Queue` (which supports arbitrary lists).
This change allows us to reenable the rest of the ResourceHandle ops, and
future-proofs us against more being added. It removes the custom logic that
assumed there was a "dtype" attribute to guess what the type parameter is
(which wasn't true in general.)
When we switch to ResourceHandle (e.g., for queues and variables) we can add
parameters to the wrapper types like "Queue" on a case-by-case basis.
Add support for logging to tensorboard
Based on @gnezdo's internal version with some differences:
* Uses a pure haskell implementation of EventWriter instead of FFI.
* Special `buildAnd*` functions were dropped in favor of using
`mergeAllSummaries :: Build SummaryTensor` with the normal
`build` function.
Also removes all the ghc-8-specific logic in the .cabal files.
ghc-8 has issues with deeply nested tuples of constraints. We can
work around it by:
- Changing TensorTypes to a regular class. This required FlexibleContexts.
(But we'll probably need it anyway when we support heterogeneous tensor
lists.)
- Specializing NoneOf for long type lists.
For more details, see: https://ghc.haskell.org/trac/ghc/ticket/12175.
Also added 'directory' to tensorflow-core-ops' dependencies since it's used
in the Setup script.
One more step towards fixing #38.
Two issues:
- The definition of `\\` was missing parentheses. It was probably a bug
that this used to worked in ghc-7.10.
- Set `-fconstraint-solver-iterations=0` to work around
https://ghc.haskell.org/trac/ghc/ticket/12175. It looks like we can
trigger that bug when defining a significantly complicated op. Specifically,
our type shenanigans ("OneOf") along with lens setters (for OpDef) seem
to confuse GHC.
Still TODO: automate testing of different ghc versions to prevent a regression.
* No longer need to hide ResourceHandle ops
* Blacklisted not supported TensorArrayV2
* Ownership of feed tensors changed (1f0c5119a0230c5160d45496175b9256f097e144)
