Hamilton quirk register¶
Every place fnloom deliberately differs from Apache Hamilton 1.90.0, what it does instead, and the test that pins it. fnloom fixes Hamilton's quirks by default, but each change needs a reason that holds for any user, not just one workload.
Just moving code over?
Coming from Hamilton summarises the rows that affect your code. This page is the full reference.
What the dispositions mean:
- KEEP: fnloom does what Hamilton does.
- FIX: deliberately different. Rows marked Planned describe a feature fnloom doesn't have yet.
- REFUSE: Hamilton silently accepts something dubious; fnloom raises.
- DROP: not carried over.
- DEFER: to be decided later.
Evidence. The behaviours below were first reported by an earlier investigation (2026-10-22) against Hamilton 1.90.0
and partly confirmed by reading its source. Rows Q4, Q5, Q6, Q7, Q10, Q11 to Q13, Q16, Q19, Q21 to Q48, Q50 to Q117
have differential tests in tests/differential/test_deliberate_differences.py, each marked deliberate("<id>"), that
assert Hamilton's behaviour and fnloom's side by side. Q23 is also covered by fnloom-only
tests (tests/test_hooks.py, tests/test_binding.py and the executor's unit tests), Q3 by tests/test_driver.py,
and Q49, a KEEP row, by tests/test_caching.py. Every other row is not yet re-run for this repository and becomes a
differential test before it is relied on. Source line numbers are deliberately omitted.
| ID | Hamilton behaviour | Disposition | fnloom behaviour |
|---|---|---|---|
| Q1 | Ships no py.typed; decorator __call__ is unannotated, so decorated functions lose their signature for type checkers. |
FIX | Typed package with py.typed; decorators return the function typed as itself. |
| Q2 | No public graph-inspection API; callers reach into Driver.graph, FunctionGraph and Node internals. |
FIX | Driver.nodes: a read-only mapping, sorted by name, of a frozen NodeView per node and input, with its name, resolved type, dependencies, is_input, origin function, module, params, tags and docstring. The node's own callable is not exposed: a prepared plan replaces the replay it was used for. |
| Q3 | Every execute re-walks the subgraph, re-validates inputs and the executor, and generates a run id. |
FIX | Compile once per (outputs, input names, override names); prepare(...) returns a reusable plan and execute uses it, keeping the 128 most recently used plans per driver. |
| Q4 | An input name that is not an input of the graph is accepted silently. | REFUSE | ValueError naming the culprit. |
| Q5 | A misspelled overrides key is ignored. |
REFUSE | ValueError. |
| Q6 | A bare str passed as final_vars is split into one output per character. |
REFUSE | TypeError. |
| Q7 | An overrides key naming an input is honoured when that input is also supplied and refused ("Required input … not provided") when it is not. |
FIX | Pass-through: an override may name any node, input or derived; it beats a supplied input and satisfies a missing one. |
| Q8 | The order in which inputs are listed is not specified. | FIX | One deterministic order everywhere: listing APIs are sorted by node name; a result dict is in requested order, each output once (first occurrence), as in Hamilton. |
| Q10 | Edge and input type checks can only be switched off together, by an adapter (NoEdgeAndInputTypeChecking) that also changes other behaviour. |
FIX | Edge checks always on at build; input and override checks on by default; Builder.without_input_type_checks() turns off only those, and a mismatched edge still fails the build. Pinned by test_q10_opt_out_is_explicit, and by tests/test_type_checks.py. |
| Q11 | AsyncDriver defaults to a pandas DataFrame result; the sync Builder defaults to a dict, and async_driver.Builder gives a dict with its keys sorted, whatever order they were requested in. |
FIX | Dict everywhere: AsyncDriver.execute returns what Driver.execute does, each output once, in the order first requested. |
| Q12 | async_driver.Builder.build is async, so it cannot be cached the usual way. |
FIX | Construction is always synchronous: Builder.build_async() returns the AsyncDriver itself, and only execute and AsyncPreparedPlan.run are coroutines. prepare is synchronous too. |
| Q13 | AsyncDriver uses asyncio.iscoroutinefunction, deprecated and scheduled for removal in Python 3.16: on 3.14 every node it runs warns DeprecationWarning. |
FIX | inspect.iscoroutinefunction: no warning, and nothing to break on 3.16. |
| Q14 | Extension autoload imports heavy libraries (xgboost, sklearn, …) on import hamilton unless disabled by an environment variable. |
DROP | No autoload. |
| Q15 | Every failed execute logs a community-promotion banner at ERROR level. |
DROP | No banner. |
| Q16 | F17: the shipped cache re-hashes source on every execute, so a hit can cost more than recomputing. | FIX | Code versions are made once, at build, from code objects rather than source: a run reads no code, so rebuild the driver after editing it. |
| Q19 | Driver.execute returns different types depending on result builder and adapters. A result builder gets the outputs in request order, inputs and overrides requested as outputs included, and after-graph gets its built result. A builder that raises fails the run, and after-graph gets the raw dict. raw_execute skips the builder and fires every hook. The AsyncDriver silently drops a ResultBuilder. |
FIX | execute returns a dict on both drivers, unless Builder.with_result_builder(build) sets a callable. build gets the raw outputs as a dict in request order, inputs and overrides requested as outputs included, as in Hamilton. execute, PreparedPlan.run and their async forms return what it returns. Builder, Driver, AsyncDriver and both plans take its type as a parameter that defaults to dict[str, object], so a type checker sees the built type. RunEnd.results is always the raw dict. A builder that raises fails the run: after_run gets ok=False, its error and the raw results, as in Hamilton. Driver.raw_execute and PreparedPlan.run_raw, with their async forms, fire every hook and skip the builder, as Hamilton's raw_execute does. The AsyncDriver uses the builder. See Result builders. |
| Q20 | Depending on the package pulls in the whole runtime and its dependencies. | FIX | No required dependency beyond the interpreter. |
| Q21 | An adapter implementing the input-check hook replaces Hamilton's default check rather than adding to it: one whose validate_input returns False refuses a correct x=1, and one returning True lets x="s" reach the node. None passes whatever the hook says. The edge check is OR'ed with the default, so an adapter can only loosen an edge. |
FIX | Builder.with_value_check(check) adds a check, cumulative with others: a value passes when the default check passes, or else when any custom check returns a true value for (declared_type, value). A custom check is called only for a value the default refuses, so a value that fits costs nothing more, and it can admit but never refuse. It applies to inputs, overrides, extracted fields, output checks (Q87) and validate, never to edges, which compare types (Q10). An exception it raises is raised as it is. See Type checks. |
| Q22 | Override values bypass type checking and fail inside the node, or flow on as the wrong type. | FIX | Override values are checked against the node's declared type, like inputs, and a mismatch is a ValueError before any node runs. Builder.without_override_type_checks() turns off only the override checks, keeping input checks on (without_input_type_checks() turns off both). Pinned by test_q22_override_type_checked, and by tests/test_type_checks.py. |
| Q23 | A hook that raises aborts the run and skips the other adapters' hooks for that event. A raising before-graph hook skips after-graph too. A raising after-node or after-graph hook masks a node's error, which survives only as the hook error's __context__. |
FIX | Every hook sees an event before its errors are raised. An error in before_run stops the run, and after_run(ok=False) still fires. One in before_node stops it before the node is called, and that node gets no after_node, as in Hamilton. One in after_node(ok=True) stops it and discards the node's value; one in after_run(ok=True) replaces the result, as in Hamilton. One error is raised as itself. Several are one ExceptionGroup: the node's own exception first, if a node failed, then the hook errors in event order, and within an event in hook order. A node's error is never masked. See Hooks. |
| Q24 | Two adapters cannot implement the same lifecycle method, such as do_build_result or do_node_execute: two result builders are a ValueError at build(), naming the method, not the adapters. Hooks and validators layer. |
KEEP / FIX | KEEP for the result builder: a second with_result_builder is a ValueError naming both builders, raised by the call rather than at build(). Hooks layer too (Q84). Node wrappers compose as an ordered chain, any number per driver (Q111). |
| Q25 | The graph walk is recursive, so a chain of about 1,000 dependent nodes raises RecursionError (verified at 1,000 and 5,000 on 1.90.0). |
FIX | No recursion over graph depth anywhere; a 5,000-node chain runs. |
| Q26 | Execution order follows a recursive depth-first walk from the requested outputs, so it depends on the order the outputs are listed in and on parameter order, and it is visible to hooks and side effects. | FIX | Topological order with ties broken by node name, independent of output order. |
| Q27 | A dependency cycle is not detected up front: the recursive walk overflows and raises RecursionError. |
FIX | ValueError naming the cycle, raised by Graph::build, that is by fnloom.build and Builder.build, before any request. |
| Q28 | what_is_upstream_of and what_is_downstream_of return HamiltonNode objects. |
FIX | The same nodes, the named ones included, as sorted names; Driver.nodes[name] gives each one's typed view (Q2). |
| Q29 | A positional-only, *args or **kwargs parameter is accepted and treated as an input of that name. |
REFUSE | ValueError at build: fnloom calls a node with one keyword argument per parameter name, and none of these kinds binds such an argument as a plain parameter. |
| Q30 | final_vars given functions maps each to its __name__, blindly: a @parameterize function becomes a name no node has, a forged function is accepted, and a function of a submodule is refused. |
FIX | A function means the node it makes in this driver, matched by identity with the node's origin, never by name. A plain function is its node and an @extract_* function its producer. When several nodes share the function (an alias alias = base, or a row named like it), the one named like the function is its node. Otherwise, such as for a @parameterize one, it is a TypeError listing its rows. A function no node comes from (another module, an unselected @config variant, a lambda) is a ValueError. Works in execute, prepare and both drivers; the result is keyed by name. Every culprit of a request is listed once, sorted, in one message: a TypeError if any is malformed, else a ValueError. Pinned by the test_q30_* differential tests, which also pin Hamilton's side, and by tests/test_validate.py. |
| Q31 | execute(final_vars, overrides, display_graph, inputs) takes overrides second, so a positional mapping is read as overrides. |
FIX | inputs and overrides are keyword-only on execute, prepare and PreparedPlan.run; a positional mapping is a TypeError. |
| Q32 | An async def or generator function is accepted as a node, and the sync driver returns the un-awaited coroutine or the generator object as its value. |
REFUSE | ValueError at build naming the function: calling these does not run the body to a value. For an async def, build() says to use build_async(), whose driver awaits it; a generator, sync or async, is refused by both. |
| Q33 | A @parameterize row naming a parameter the function does not have raises InvalidDecoratorException, an Exception subclass that is not a ValueError, when the decorator is applied. |
FIX | ValueError, also when the decorator is applied, listing every unknown row parameter, sorted, like every other malformed-function error. |
| Q34 | Every function node carries a module tag naming its module, beside the tags the author wrote; inputs carry no tags ({}). |
FIX | A node's tags are only what the author wrote with @tag; its module is NodeView.module. Inputs carry no tags ({}) and their module is None, so the test compares inputs as {} on both sides. |
| Q35 | A @parameterize row binding a parameter that has a default to source("x") makes x optional, but when x is not supplied the node raises KeyError instead of using the default. |
FIX | The keyword is left out, so the function's own default applies, exactly as for a plain optional parameter. |
| Q36 | @tag accepts a list of strings as a tag value, as well as a string. |
REFUSE | A tag value must be a str, because NodeView.tags is a Mapping[str, str], so a tag always reads as one string; anything else is a TypeError when the decorator is applied, naming the tag. |
| Q37 | tuple[int, ...] is not treated as a generic: the input check rejects (1, 2), and a node returning tuple[int, ...] does not feed a Sequence[int] parameter, because the ... is compared as a second argument. |
FIX | tuple[T, ...] takes a tuple, as list[T] takes a list (its items unchecked, as in Hamilton), and feeds Sequence[T]: across kinds its one element type is what is compared. |
| Q38 | A NewType over int takes no value (its values are plain ints at runtime, so even 1 is rejected), and a node returning it feeds neither int nor any type int would feed. |
FIX | A NewType answers as its supertype: 1 fits it, and it feeds whatever its supertype feeds. Only the NewType itself (or Any) feeds a parameter annotated with it, as in Hamilton. |
| Q39 | A node returning Literal[1, 2] feeds only Literal[1, 2] and Any, not int, object or a union holding int. |
FIX | A Literal producer answers as the types of its values: it feeds a parameter when each value's type does. |
| Q40 | A node returning a union (int \| None, int \| str) feeds only that same union and Any, not even object. |
FIX | A union producer feeds a parameter when every member does, so int \| None feeds object and int \| str \| None; a union parameter still takes a producer that feeds one member. |
| Q41 | Annotated[T, ...] is checked by exact type in the input check, so True fits int but not Annotated[int, "m"], and Child() does not fit Annotated[Base, "m"]. |
FIX | Metadata never changes an answer: Annotated[T, ...] is checked as T, on both the input and the edge check. |
| Q42 | A parameter annotated with a plain (not runtime-checkable) Protocol and fed by a node returning a class makes the build raise the TypeError of issubclass, which names no node; a value that conforms to a runtime_checkable protocol is rejected by the input check. |
FIX | A plain protocol cannot be checked at runtime, so it is no match (False), never an error; a runtime_checkable protocol takes any value that isinstance accepts. A plain-protocol input still takes no value, and not None either (Q79). |
| Q43 | The input check reads the first item of a value declared Iterable[T] to check it, so a generator or other iterator supplied as an input loses that item before the node runs, and is accepted or rejected by it. |
FIX | An iterator is never looked inside: it is checked by its kind alone (it fits Iterable[T] and Iterator[T], not list[T]), and reaches the node with every item. |
| Q44 | A PEP 695 alias (type Price = float) is compared as itself: the input check takes no float for a Price input, and a node returning float feeding a Price parameter, or returning Price feeding a float one, fails the build; the opt-out is the only way through, and it drops every check. |
FIX | An alias is checked as what it names, read through any number of aliases, on the input and the edge check. A subscripted generic alias (Pair[int] for type Pair[T] = tuple[T, T]) is read with its arguments put in when its value is one of its type parameters or a generic over exactly those parameters, in order; any other generic alias answers as before. An alias nested inside another generic's arguments (list[Price] fed by list[float]) is compared as written on the edge check. Aliases naming each other are compared as written. Every alias's value is evaluated at build, so an alias over a name imported only for type checkers is unresolved, as any such annotation is (Q81). |
| Q45 | Mapping[str, int] takes no dict, Callable[[int], int] takes no function, and likewise every other parameterized abstract collection outside the Sequence and Iterable families (MutableMapping, Set, Collection, MutableSequence, ...): the input check refuses every value but None. |
FIX | A parameterized collections.abc type outside those two families is checked by its kind alone, with isinstance, its items or signature unchecked; Sequence[T] and Iterable[T] keep the first-element check, and an iterator is still never read (Q43). The edge check already agrees with Hamilton on these forms (dict[str, int] feeds Mapping[str, int]) and is unchanged. |
| Q46 | F1, F2: the cache stores results by a hash of the value that ignores its type, so 1 and "1" share an entry: a node returning "1" gets back another node's 1 on the next run. |
FIX | A result is stored by its node's key, which names the node, and an input's fingerprint carries its exact type and full structure: no two nodes, and no two values of different types, share an entry. |
| Q47 | F3: a value it cannot hash nested inside a container (a __slots__ object in a list) is hashed as a constant, so the run hits after that object changed. |
FIX | A value that cannot be fingerprinted at any depth makes the nodes reading it uncacheable: they run, and explain_cache gives the reason. |
| Q48 | F4: a node's code version is its own source only, so an edit to a helper it calls, or to a module constant it reads, still hits the old result. | FIX | The code version covers the function's code objects, the helpers and classes it reaches, the constants it reads, its defaults, closures and library versions; a global that may change makes the node uncacheable unless @cache(version=...) is given. |
| Q49 | F6: a recompute run does not replace the stored value, so the next default run serves the value from before the recompute. |
KEEP | The same value: a node's policy is part of its key, so default and recompute never share an entry (a recompute result is not stored at all), and a default run after a recompute serves the earlier default entries for the node and everything downstream, consistently, never a recompute result beside downstream entries keyed before it. A node that reads the outside world must stay recompute (purity is declared). Pinned by test_a_policy_switch_never_mixes_key_spaces in tests/test_caching.py. |
| Q50 | F10, F11: ignore is set on the producer and drops it from every consumer's key, so a consumer hits after the producer's value changed; and the producer's result is still pickled, so a node producing a lock or a client fails the run with TypeError. |
FIX | @cache(behavior="disable") on the producer never stores or versions its result; @cache(ignore=("name",)) on a consumer leaves only that parameter out of that consumer's key, so it hits, while every other consumer still reads the producer. |
| Q51 | F12: a truncated or corrupt stored result fails the run with the unpickling error. | FIX | Entries are written atomically, and any entry that cannot be read is a miss: it is removed with a CacheWarning and the node recomputes. |
| Q52 | Something that is not a module, such as a function, passed to Builder.with_modules builds an empty graph, without error: the mistake surfaces at execute as an unknown output. |
REFUSE | build, Builder.with_modules and the Builder constructor raise TypeError: expected a module, got function 'f', naming the first item that is not a module (any types.ModuleType is one). |
| Q53 | A module that contributes no node builds an empty part of the graph, without error: a package whose __init__ only re-exports functions defined in its submodules, a module of private helpers, or a hand-built types.ModuleType whose functions' __module__ names another module. The user finds out at execute, as an unknown output (Hamilton issue #1281). |
REFUSE | Build raises one ValueError naming every such module and each function it holds with why it is not a node: 'f' is defined in 'pkg.impl' (a re-export, a node of the module that defines it) or '_g' is private; a module with no functions at all says so. Building from no modules at all is still an empty driver. |
| Q54 | @cache(format="json") stores a dict through a JSON saver that loads a tuple inside it back as a list, so the first hit returns a different value than the node computed, with no warning. |
FIX | The built-in json format refuses any value that would not load back as the same types: a tuple, a set, a key that is not a str, a subclass such as an enum. The write is skipped with a CacheWarning, and the node runs again. |
| Q55 | @cache(format=...) naming a format no data adapter registers runs the node, then fails the run with KeyError, on every run. |
REFUSE | Formats are resolved against the store when the driver is built. An unknown name, a Format unlike the store's of the same name, or a store that takes no formats is a ValueError naming each node. A MemoryCache holds values, never bytes, so it takes any format and ignores it. fnloom has no dependencies, so only pickle and json are built in: Hamilton's pandas-backed names such as parquet and csv are given as a fnloom.Format, or added to the store with DiskCache(formats=...). |
| Q56 | Every config key becomes an input node: a parameter of the same name receives the config value, the key can be requested as an output, inputs= naming it is a ValueError ("present in both config and inputs"), and a key named like a function silently replaces that function with the config value. |
FIX | Config shapes the graph and nothing else: a key is never a node, an input, a parameter value or an output. A parameter named like a config key is an ordinary input, supplied in inputs=; when it is missing, the missing-input error adds a line saying it is a config key, and config is not passed to nodes. A function named like a key stays a node. Driver.config is a read-only copy of the config. |
| Q57 | A config key the config does not have reads as None, so @config.when_not(mode="a"), when_not_in and when(mode=None) match an empty config. A node no variant matches builds as a required input, and the run fails at execute with Required input foo not provided (Hamilton issue #1213). |
FIX | A condition on a missing key is false, whichever form it takes. A node no variant matches and with no @config.default is a ValueError at build when anything reads it: no variant of 'foo' matches the config {'mode': 'c'}; its variants need mode == 'a' (foo__a), mode == 'b' (foo__b); add @config.default or a variant for it, every such node listed, sorted. A node nothing reads is absent, and requesting it raises the same reason. A name that only a variant the config did not select gives, a row or an extracted field, is refused the same way, never made an input: no selected variant gives 'r2': the config {'mode': 'z'} selects rows__other for 'rows', and only rows__a gives 'r2'. An @extract_fields() naming no fields gives its TypedDict's keys; when an unselected variant's return annotation does not evaluate at build, its fields are unknown and are not refused. |
| Q58 | Two variants of one node matching the config fail the build with Cannot define function foo more than once, naming the function added second, not the one already there. A config-decorated function defined twice in one module keeps the last definition silently. |
REFUSE | Two matching variants are a ValueError naming both functions, sorted; two @config.defaults for one node are refused the same way, whatever the config. A config-decorated function defined twice at the top level of its module is a ValueError naming it and the lines of each definition, read from the module's source at build; a module without source skips the check. A function bound under two names, or a module given twice, is one variant. |
| Q59 | @config.when without parentheses turns the function into a config object and the node silently disappears; a positional string is taken as the node name; when_in(mode="ab") tests substring membership, and a non-iterable fails at build with a bare TypeError; a name ending in __ raises InvalidDecoratorException with a message cut off mid-sentence; and when_not_in(name=...) reads name as a config key while the other forms read it as the node name. |
REFUSE | Each is refused when decorating: a bare @config.when, a positional string, a str, bytes or non-iterable for when_in and when_not_in, a name ending in __ and a when* with no condition are a TypeError or ValueError saying what is wrong. name= is the node name on all four forms; a condition on a key called name goes in a positional mapping, config.when({"name": "x"}). |
| Q60 | Building renames the user's function in place (foo__a.__name__ becomes 'foo', with __original_name__ and __config_decorated__ added), and every node from a config-decorated function gets a hamilton.config tag naming its keys. |
FIX | The function is never changed, and no tag is added: a node's tags are only what the author wrote. NodeView.config holds the conditions that selected the node, empty for a plain function or a @config.default. |
| Q61 | @extract_fields checks nothing it declares: a field of the wrong type reaches its consumers, and {"a": str} on a dict[str, int] return builds. A field missing at run raises InvalidDecoratorException. Names given beside a mapping are dropped silently, a name a TypedDict return lacks raises a bare KeyError, a Mapping[str, T] return is refused, and two @parameterize rows under it fail only at build. An input declared as a TypedDict makes the input check raise TypeError. |
FIX | Each field's value is checked on every run against its declared type, with the input check's matcher, even when input checks are off: a wrong type is a ValueError naming the producer, the field and both types. A declared type the return annotation can't feed fails the build, naming the field. A missing field is a ValueError naming the fields the producer returned, sorted. A mapping with extra names, a name the TypedDict lacks and two @parameterize rows are a ValueError when decorating. Mapping[str, T] returns are accepted. A TypedDict type, for a field or an input, takes any dict, its keys unchecked. |
| Q62 | fill_with writes the fill into the dict the producer returned, so the producer's own output changes, and so does a dict it returned from a module global. fill_with=None means no fill: a missing field still raises. |
FIX | The fill is seen only by the field's node: the producer's value is never changed. None is a fill like any other; leaving fill_with out means no fill. |
| Q63 | @tag on an @extract_fields or @extract_columns function tags only the field or column nodes, not the producer, in either order. |
FIX | The tags land on the producer and on every field or column node. |
| Q64 | @extract_columns needs a registered dataframe extension: with extensions not autoloaded, a pd.DataFrame producer fails at decoration with "does not output a type we know about". A dict return fails the same way even with the pandas extension loaded. A column named twice, as "a", "a" or "a", ("a", "doc"), fails the build with Cannot define function a more than once. fill_with writes the column into the producer's DataFrame, and with a (name, doc) column it adds a column named by the whole tuple, then fails with No such column. A missing column builds and fails at run with InvalidDecoratorException. A name the frame holds twice gives a DataFrame, and no column value is checked. |
FIX | extract_columns is generic: it splits any value with in and [], so it needs no extension, and a mapping works. A column's type is pandas.Series for a pandas.DataFrame return, polars.Series for a polars.DataFrame, each read by module and name without importing either, and Any otherwise; each value is checked against it on every run, so a duplicated name in the frame is a ValueError. A column name given more than once, in any form, is a ValueError when decorating, listing every repeat, sorted. A missing column is a ValueError at run naming the producer and the columns it returned, sorted. fill_with never changes the producer's value: it is a full column for a pandas or polars frame, the value itself for a mapping, and a TypeError at run naming any other type; a (name, doc) column fills by its name. |
| Q65 | The AsyncDriver runs a sync node inline on the event loop, so the loop stops while it runs: an awaiting sibling, and every other coroutine on the loop, waits for it to finish. |
FIX | A sync node runs in a worker thread of the loop's default executor, through loop.run_in_executor in the caller's context, so the loop keeps running while it does. build_async(sync_nodes="inline") runs sync nodes on the loop instead, for a graph of cheap nodes, where a thread costs more than the node. A generated extract_* node always runs on the loop. |
| Q66 | When a node fails, the AsyncDriver raises its exception and leaves its running siblings running: they finish, unobserved, after execute has raised. |
FIX | The first node to fail cancels every node still running, once, and waits for each task to take its cancellation, as a cancelled caller does; a later cancellation of the caller neither cancels a node again nor cuts the wait short. A node's own CancelledError is raised as itself. A sync node in a thread can't be interrupted, so the run doesn't wait for it and drops its result. The failing node's own exception is raised once, with the sync driver's note naming the node, and no ExceptionGroup, so except KeyError works as it does on Driver. |
| Q67 | AsyncDriver.execute checks no input or override type. A missing input raises NotImplementedError: x was expected to be passed in but was not., and an unsupplied input requested as an output raises KeyError. |
FIX | The AsyncDriver checks a request as Driver does, with the same errors: input and override types, unknown names, a missing input as ValueError naming what needs it, and an unsupplied input requested as an output. |
| Q68 | with_cache(...) on the async driver's builder fails at build, with ValueError: Multiple adapters cannot (currently) implement the same lifecycle method. Sync methods: ['do_node_execute']. Async methods: []. The async driver can't cache, and the message doesn't say so. |
FIX | Builder.with_cache(...).build_async() caches as Driver does: the same keys, stores, @cache policies and explain_cache, so a cached result is shared between the two drivers. A node is stored as it succeeds; a node that fails, or that a failure cancels, stores nothing. A recompute node runs first in every run, awaited when it's an async def. Store calls run on the event loop's thread. |
| Q69 | The AsyncDriver doesn't await an async def under @parameterize: each row's value is the un-awaited coroutine object, since the row's wrapper is a sync function. A plain async def is awaited. |
FIX | Every async def node is awaited, a row of one included: a row's call returns its function's coroutine, and the driver awaits it. |
| Q70 | list_available_variables, graph_modules and what_is_the_path_between are Hamilton's inspection. graph_modules keeps a module given twice (a build allows that only with allow_module_overrides). what_is_the_path_between(a, b) includes both endpoints and returns a set's order, which changes with PYTHONHASHSEED; for an unknown name it names only the first one, upstream before downstream. Nothing lists the inputs with their consumers, or the nodes nothing consumes. |
FIX | Driver.inputs() maps each input to an InputView (type, whether some consumer requires it, its consumers), in name order; Driver.outputs() lists the nodes nothing consumes, sorted; Driver.modules holds each module once, in build order. what_is_the_path_between includes both endpoints and takes an input as the upstream one (as the downstream one only when both are that input, since nothing feeds an input), as Hamilton does, but returns topological order with ties broken by name, on every seed, and its ValueError names every unknown endpoint, sorted. AsyncDriver has all four. |
| Q71 | validate_execution(final_vars, overrides=None, inputs=None) takes overrides before inputs, returns None or raises, and raises unknown outputs on their own, before any input problem. It refuses a function in final_vars that execute accepts (TypeError: sequence item 0: expected str instance, function found). A missing input names every consumer in the graph, requested or not, and an unknown input is ignored. No message suggests a name: a misspelled input key leaves the real input missing, and the error never says what was meant. |
FIX | Driver.validate(final_vars, *, inputs=None, overrides=None) returns every problem execute would raise, one line each, sorted, and [] for a valid request; it runs nothing and keeps no plan. Each line is the line execute raises for that problem on its own: unknown outputs, inputs and overrides, missing inputs, input and override types, and outputs no @config.when* variant gave. A missing input names only the consumers this request plans. A function in final_vars works, and one no node comes from is a line, as execute raises it. A malformed request raises a TypeError: a bare str, an item that is neither a name nor a function, a function that makes several nodes, or inputs or overrides that is not a mapping. AsyncDriver.validate is the same, and synchronous. Every unknown name in an error gains (did you mean 'x'?) for the closest node within difflib.get_close_matches cutoff 0.8. In validate only, a missing input also gains the closest supplied name that is no node of the graph; execute raises that unknown input first, on its own. |
| Q72 | No graph is built from a list of functions. ad_hoc_utils.create_temporary_module copies each function, sharing its code and globals, with __module__ set to a new module that it registers in sys.modules; the node is the copy, so the original function in final_vars is refused. A function whose name starts with _ is refused. A lambda is accepted: unannotated, the build fails on its missing return annotation; annotated, its node is named <lambda>. |
FIX | Builder.with_functions(*fns) uses the functions themselves: nothing is copied, no module is made or registered, a node's origin is its function and its module the function's own __module__, so the original in final_vars means its node. A private name is accepted. A lambda is refused unless named with fnloom.node(fn, name=...). |
| Q73 | __all__ is ignored: every public function a module defines is a node, listed or not. @hamilton_exclude (the public name is an instance of configuration.hamilton_exclude) keeps a function out, but only written bare: @hamilton_exclude() raises TypeError: NodeTransformLifecycle.__call__() missing 1 required positional argument: 'fn'. |
FIX | A module that defines __all__ makes nodes of only the functions it names, by attribute name: a @config variant foo__a must be listed as foo__a. Names it lists that are no function the module defines, such as re-exports and constants, are ignored, and __all__ never makes a private function a node. An __all__ that is not a list or tuple of str is a TypeError at build naming every such module, sorted. @fnloom.exclude works bare and called, records the mark and returns the function itself (None, a class or any other non-function is a TypeError), stacks with the other decorators in either order, and beats __all__: a function both names and marks is excluded, with no error. A left-out function is never read, so its annotations may be missing or name a TYPE_CHECKING-only name, as with hamilton_exclude. A missing input named like a left-out function gains the line 'x' is a function in module 'm' not listed in its __all__ (or marked @exclude, or both joined by and), each name's lines sorted so they don't depend on module order; for a @config variant it names the function, 'price__us' (node 'price') is a function …, and the error for a node no variant gave names the variants left out too. The empty-module error (Q53) gives both reasons. Which function a final_vars entry means is decided by identity (Q30), so an excluded function is never a node and is refused as one no node comes from. |
| Q74 | with_modules takes modules only: no builder option namespaces or prefixes a module's nodes. A name two modules define fails the build unless allow_module_overrides() is on, and then the later module's function silently wins, for every reader. The only namespacing is @subdag, whose nodes are named <function>.<name>. |
FIX | Builder.with_modules(*modules, namespace="ns") names each node of those modules ns.<name>. Inside the namespace, a parameter reads ns.x when the namespace defines x, else plain x. One module may be given under several namespaces. Nothing overrides: a name defined twice fails the build. |
| Q75 | An overrides key naming an input is refused ("Required input … not provided") when some node the plan computes needs that input and it is not also supplied (an override only prunes below the overridden node). A misspelled parameter or source("...") builds, and is an input nothing supplies; no setting declares the inputs a graph may have. |
FIX | Both are opt-in, so the defaults stay as Q7 and Hamilton's build. Builder.with_strict_overrides() refuses that override as Hamilton does, settled when a request is prepared and never on a run of a prepared plan; the ValueError lists every such input, sorted, with the nodes that need it, and validate returns the same lines. Builder.with_declared_inputs(*names), cumulative, fails the build for any input of the graph not declared and for any declared name that is no input, each with the closest node or declared name (Q71's suggestion), in one sorted message. Both settings are carried by build_async, pickling and copying. |
| Q76 | A node with the same name as a parameter that has a default silently replaces the default, and a supplied input of that name is ignored. Two functions giving one input different defaults each keep their own; a caller's value reaches both. Neither says anything. | REFUSE | Both fail the build, with one ValueError listing every case, sorted. A parameter with a default that reads a computed node (by its own name, or by source("...") in a row; a value() has no edge) names the parameter, the consumer and the node. An input that two or more consumers give defaults that are not equal (==, with two NaNs equal; a result that is not a bool or numpy's bool, an error, or two objects without __eq__ count as different) names every such consumer with its default. @fnloom.defaults(shadowed=("cost",), differing=("shared",)) on the consumer opts a parameter out: the node still wins, and each consumer keeps its default. differing leaves the opted-out consumers out of the comparison, so naming it on each consumer but one is enough. Applying defaults twice merges the names. A name that is no parameter with a default is a ValueError when decorating, listing every one. |
| Q77 | An input or override naming a parameter that every value() binding supplies is silently dropped, even a wrong-typed one. When another function has a parameter of that name, it is a real input: the caller's value feeds that function alone and the value() rows keep their constant. |
REFUSE | Nothing reading the name: the input or override is refused as an unknown name (Q4, Q5). Another node reading it: kept, as Hamilton does, and tested on both sides. |
| Q78 | An input nobody supplied, requested as an output, returns None for it: {'x': None}. An input with a default in its consumer reads None too, not the default. |
REFUSE | ValueError: missing inputs: 'x' (requested as an output). The default of a consumer's parameter belongs to that consumer, so it's never an input's value. |
| Q79 | None passes every input check, whatever the type: a node declared x: int runs with None. |
FIX | None passes only a type that admits it: Optional[T], T \| None, object, Any, None, Literal[None], and a TypeVar that is free or whose bound or a constraint admits it. For any other, such as int, it is a mismatch: a ValueError, and no node runs. |
| Q80 | A fixed-length tuple type refuses every tuple, a correct one included: (1, 'a') for tuple[int, str], () for tuple[()]. |
FIX | A fixed-length tuple takes a tuple of its length whose items each fit: (1, 'a') fits tuple[int, str], and ('a', 1) and (1,) do not; tuple[()] takes only (). |
| Q81 | An annotation naming something that does not exist at run time, such as an import under TYPE_CHECKING, fails the build with a NameError that names no node. |
FIX | The build succeeds, and the annotation is an opaque type, unresolved 'Decimal', its text as written. An edge matches it only from or to the same text, or Any; any other edge with it fails the build, naming the text. A value checked against it, as an input or override, raises a TypeError saying to import it at run time. |
| Q82 | @parameterize sugar is checked loosely. A {ghost} in the function's docstring prints as ghost, a group(...) parameter prints as its name, and a stray brace or a positional field is a raw ValueError. A (bindings, doc) row of 1 item is a raw IndexError, one of 3 drops the extra, and a non-str doc is kept. parameterize_sources(p=5) imports, then fails with a raw TypeError when the graph is used. parameterize_values with {} builds no node, and a 3-item key is a raw unpacking ValueError. A group source is optional when its parameter has a default, and a partly supplied group fails with a raw KeyError. A mis-annotated group parameter can list itself twice in the error, for example a positional group on an int parameter. |
FIX | The docstring is a template, checked when decorating: a placeholder the row lacks, a positional field or a stray brace is a ValueError naming the row, and a group(...) prints its items joined (x, 3). A row of 1 or 3 items, or with a non-str doc, is a TypeError. A non-str source is a TypeError when decorating, {} is a ValueError, and a key that is no (name, doc) is a TypeError. A group source is required even when its parameter has a default: a missing one is a ValueError naming the input. A group parameter is listed once. |
| Q83 | A driver keeps the functions it was built with. After importlib.reload it runs the old code, a function the edit added is an unknown node, and the reloaded function in final_vars means the old node, by name. A node's version reads the source file, so after an edit it already differs from the code the driver runs. Only a new Builder builds again. |
FIX | Driver.rebuild() and AsyncDriver.rebuild() build again from the driver's builder, with every setting, and see the reloaded module. An existing driver keeps its code, as Hamilton's does, and a reloaded function in final_vars is no node of it (Q30). Code versions are read at build, so an edit misses the cache only after rebuild. A function given to with_functions stays the object given: after a reload, rebuild of a builder that also has its module raises a ValueError saying the function is stale after a reload, and to give the reloaded function to a new builder. |
| Q84 | Lifecycle adapters get each event as keyword arguments. The after-graph hook gets no outputs, inputs or overrides, and execution_path is a list made from a set, so its order changes with PYTHONHASHSEED. with_adapters accepts any object and ignores one that is no adapter, and calls an instance given twice once. No signature is checked: a hook that can't take the keywords fails at its first call, and an async def hook on the sync driver is never awaited. |
FIX | A fnloom.Hook has four methods, before_run, before_node, after_node and after_run, each a no-op until a subclass overrides it, and each gets one frozen event: RunStart, NodeStart, NodeEnd or RunEnd. RunEnd repeats the request, and RunStart.nodes lists the nodes the run reports, in the order it reports them. Builder.with_hooks(*hooks) is cumulative, and every event calls the hooks in the order given, after-events too, as Hamilton does. Checked when attached: a non-Hook is a TypeError, the same instance twice a ValueError, and an overridden method that can't take one positional event, or is an async def, a TypeError naming it. Only the methods a hook overrides are called. |
| Q85 | With .with_cache(), node hooks fire on a cache hit exactly as if the node ran: the same node_kwargs and result, and for every node upstream of the hit too, each loaded from the store, though no node function runs. No hook argument says whether a node was a hit; the outcome is only in dr.cache.logs(). |
FIX | On a cached driver, every node event carries the node's CacheOutcome as cache: hit, miss or uncacheable, with the reason explain_cache gives. A hit gets before_node and after_node, so hooks that pair them still pair, with inputs=None, since nothing upstream is loaded, and the loaded value as result. Hits report first, in plan order, since they are loaded before any node runs; then the nodes the run computes. A node only hits need doesn't run, so it reports nothing. RunStart.nodes lists exactly these. A recompute node whose result keys another node runs before anything is looked up, so it and what it needs report first, and RunStart.nodes then lists them and every other node the plan could report. A node whose after_node raised is not stored. The AsyncDriver fires the same events. See Hooks. |
| Q86 | The AsyncDriver calls a sync hook when it schedules a node, not when the node runs: node_kwargs hold the asyncio.Tasks of the node's dependencies, after-node gets the node's Task as its result, and it fires before the node has run, so it reports success=True for a node that then fails. Its after-node events come in scheduling order, not the order nodes finish. It silently drops a ResultBuilder subclass, so execute returns the dict. |
FIX | An AsyncDriver fires the sync driver's events, through the same dispatcher and with the same rules. before_node fires as a node starts, and after_node once its value or error is in, with real values: the node's arguments, its result, or ok=False and its own exception. Concurrent nodes report in the order they start and finish. Every hook runs on the event loop's thread, never in a worker thread, so a hook needs no lock for one run. A hook that raises fails the run as a failing node does: every node still running is cancelled (Q66), and a node the failure cancels gets no after_node. RunStart.nodes lists the plan's nodes in plan order, since concurrent nodes have no fixed event order; on a driver with a cache, it lists the hits first, then the nodes the run computes in plan order, as Q85 says. A missing or mistyped input fails after before_run, as on Driver. A result builder is used as on Driver, where it runs before after_run (Q19). See Hooks and With a cache. |
| Q87 | No node's result is checked by default (H6.14). The opt-in FunctionInputOutputTypeChecker checks with isinstance rules of its own: it refuses (1, "a") for tuple[int, str] and accepts (1,), raises IndexError on tuple[()], and raises a raw TypeError for Callable[[int], int], frozenset[str], type[int], Iterator[int], NewTypes and htypes.column, discarding a correct result. It refuses 1 for float. It checks a failing node's None result, so the node's own error surfaces as TypeError: Node fails returned a result of type <class 'NoneType'>, the real error only in __context__. |
FIX | No result is checked by default either (KEEP). Builder.with_output_type_checks() checks each node's result against its return annotation by the input checks' rules: None only for a type that admits it (Q79), fixed tuples by length and item (Q80), NewTypes and generics by value, deep checks and numeric widening when they are on, and custom value checks (Q21). A mismatch fails the node with TypeError: x: returned str, expected int, so after_node gets ok=False and that error; a node that raises keeps its own error. An async def node's awaited result is checked. The checks are made at build, around the nodes that need one: a node returning Any or object gets none. An annotation that did not resolve is Q81's TypeError. See Writing nodes. |
| Q88 | A failing node is always logged at ERROR on hamilton.execution.graph_functions, with a banner and the node's inputs, each value whose repr is over 50 characters replaced by its first 50 and ...; then a second ERROR on hamilton.driver asks the reader to join Hamilton's Slack. The first carries exc_info; a caller can only silence either by configuring those loggers. |
FIX | fnloom logs nothing by default: a library that logs every failure, with a support banner, makes noise its caller can't turn off. fnloom.LogFailures(logger=None, *, level=logging.ERROR) is an opt-in Hook that logs each failed node (ok=False) on logging.getLogger("fnloom"), or the logger given: node 'boom' failed; inputs: a=2, big=[0, 1, 2, ..., with each input's repr cut to 50 characters and ended with ..., sorted by name, and the node's error as exc_info. It logs nothing on success or for a cache hit. It works on both drivers. See Hooks. |
| Q89 | PandasDataFrameResult builds a frame by rules of its own. A DataFrame output is flattened into output.column columns, and index-type warnings are logged. The rest goes to pandas.DataFrame(outputs), so Series of different lengths or indexes are outer-joined on the index and padded with NaN, turning integers into floats without a word. |
DROP | fnloom ships no pandas or polars builder, and depends on neither (design §10). with_result_builder(pandas.DataFrame) or with_result_builder(polars.DataFrame) builds a frame by the library's own rules: no column is flattened and no index warning is logged. pandas still outer-joins Series, so the documented recipe refuses outputs on different indexes instead of padding them. See Result builders. |
| Q90 | display_all_functions, display_upstream_of, display_downstream_of and visualize_path_between return a graphviz.Digraph, built from a set of nodes, so the order of nodes and edges in its DOT text changes with PYTHONHASHSEED. display_downstream_of also draws each drawn node's parents, alike. visualize_path_between names only the first unknown endpoint, and display_upstream_of() with no names draws an empty graph. |
FIX | draw(), draw_upstream_of(*names), draw_downstream_of(*names) and draw_path_between(upstream, downstream, context=True) on both drivers return a fnloom.Drawing: plain data, its nodes in topological order (ties broken by name) and its edges in the order of their nodes, so to_dot() is the same text under every hash seed. Downstream and path drawings add each drawn node's other dependencies, as Hamilton's do, marked context; context=False is Hamilton's strict_path_visualization. Every unknown name is named, sorted, in one ValueError; no names, or a non-str, is a TypeError; no path is a ValueError, as in Hamilton. A bad orient is a ValueError; Hamilton writes it as given. See Drawing. |
| Q91 | A node's label shows its type by the bare name of its origin: list[int] is list, dict[str, list[int]] is dict, int \| None is Union, Literal['a'] is Literal. A @config.when node's label is name: key, without the value it matched. |
FIX | A label shows the whole annotation, as inspect.formatannotation writes it, cut to 79 characters and … past 80 as Hamilton cuts at 80, and an unresolved name (Q81) as written. A @config.when* node's label adds its conditions: mode == 'fast', mode in ['slow', 'slower']. A display_name tag replaces the name in the label, as in Hamilton; the node's id stays its name. |
| Q92 | visualize_execution(final_vars, inputs=, overrides=, bypass_validation=False) draws the nodes a run computes, the outputs and overrides filled in their own colours, after validating the inputs (bypass_validation skips it, and the error's text says so). An input requested as an output is not drawn as an output: it is a row in its consumer's box of inputs, unmarked. |
FIX | draw_execution(final_vars, *, inputs=None, overrides=None, validate=True) draws what execute would compute, as Hamilton's does, outputs marked output and overrides override, nothing only an override needed, and an input requested as an output drawn and marked output. With validate, every problem Driver.validate finds is raised in one ValueError; validate=False draws as if every required input were supplied, and still refuses unknown names. Functions in final_vars are taken, as execute takes them (Q30). |
| Q93 | Inputs are not nodes in a drawing: each consumer gets a box of its own listing the inputs it reads, so an input read twice is drawn twice; deduplicate_inputs merges only consumers reading the identical set. Every config key is drawn as a node in every drawing, whether a drawn node used it or not. |
FIX | Each input is one node with a dashed outline, and an edge to each node that reads it. Config is never drawn, since a key is never a node (Q56); the conditions a variant was chosen by are on its label (Q91). |
| Q94 | Drawing needs the graphviz Python package. Without it every drawing method returns None, writes nothing and logs a full traceback at ERROR, advising pip install "sf-hamilton[visualization]", the old distribution's name; no error reaches the caller. Writing a .dot path gives Graphviz's laid-out DOT, and the source DOT is written only with keep_dot=True, to a file with no suffix. |
FIX | DOT and Mermaid text need nothing: Drawing.to_dot() and to_mermaid() are fnloom's own, and fnloom adds no dependency. Drawing.render(path) writes .dot and .mmd as that text, and any other format through Graphviz's dot executable; without it, FileNotFoundError saying how to install Graphviz, and when dot fails, RuntimeError with its message. _repr_svg_ shows the drawing in Jupyter when dot is installed, and the text otherwise. |
| Q95 | export_execution(final_vars, inputs=, overrides=) returns json.dumps({"nodes": [...]}) for a planned run: each node's name, tags, type, dependencies, source, docstring and version, sorted by name. An input carries the source and version of a function that reads it. Types are the bare names Q91 describes. The config is not exported, so the document can't say which @config.when variant was chosen, and an overridden node is not marked. There is no format version. A function in final_vars is a TypeError, though visualize_execution takes one, and a function Python has no source for, such as one made by exec, makes it raise OSError. There is no whole-graph export. |
FIX | Driver.export(final_vars=None, *, inputs=None, overrides=None, validate=True), on both drivers, returns a JSON-ready dict: {"format": "fnloom.export/1", "fnloom", "graph_version", "config", "request", "nodes"}. With no final_vars it describes the whole graph; with them, the run draw_execution draws, checked the same way. Nodes come in drawing order, each with its kind, whole type, parameters and what each reads or is bound to, tags, doc, function, namespace, config conditions, source (null when Python has none), code version (Q96), marks (output, override) and output checks. An input has no source or version, and says whether it is required and who reads it. No value is exported: request lists names, and config values and value() bindings are their repr. Functions in final_vars are taken (Q30). See Exporting. |
| Q96 | A node's version is a hash of its function's source text, so an edited helper it calls or a changed module constant it reads leaves it unchanged (K = 1 to K = 2 keeps it). An input's version is a consumer's. The graph's version hashes the sorted node versions alone, without names, types or wiring. |
FIX | A function node's version is its code version, the one the result cache keys on (Plan 4): its code, defaults, closures and the globals it reads, helpers and constants included. A function with no code version, such as one reading a mutable global, exports version: null and unversioned saying why. A node's version covers its output checks too, as the result cache's key does. graph_version hashes every node's name, kind, type, parameter bindings (a value() by its version), config conditions, checks and version, so a rename, a rewired edge, a changed type or a changed constant changes it; it is null when a node is unversioned or a bound value has no fingerprint, since such a change could not be seen. |
| Q97 | The hamilton CLI (build, diff, validate, version, view) needs typer. It takes modules as files and compiles each file's text alone, so a module importing its sibling (from helper import f) fails with ModuleNotFoundError, and a dotted module name is not accepted. Config comes only from a --context file; without one, every @config.when node silently disappears from what it builds, versions and draws. Inputs and overrides come only from that file too. There is no command to draw part of a graph, draw a run, or export. |
FIX | The fnloom command (also python -m fnloom) uses only the standard library: draw, export, validate and diff. A module is a dotted name, imported with the current folder on sys.path, or a .py path, loaded under its stem with its folder on sys.path, so a sibling import works, and refused when a module of that name is already imported from elsewhere; MODULE@NS namespaces it (Q74). --config, --input and --override take KEY=VALUE (JSON, or else a plain string), --outputs names a run's outputs, and --context reads all four from a JSON or TOML file, the flags winning. A @config.when node is built exactly as Builder builds it: when its key is missing it is left out if nothing reads it, and is otherwise a ValueError naming the key (Q57); --config supplies the key. draw writes DOT or Mermaid to stdout, or any format to -o, for the whole graph, --upstream, --downstream, --path or --execution. See The CLI. |
| Q98 | A failing CLI command logs {'command': ..., 'success': False, 'message': {'error': "<class 'ValueError'>", 'details': ...}}, a Python repr, to stderr and exits 1; with --json-out, nothing at all reaches stdout. validate prints the context back on success. diff compares node versions between the files and their state at a git reference: an input takes its consumer's version (Q96), so an input is reported as edited when its consumer is, and diff --view always fails in 1.90.0, as visualize_diff passes no config. |
FIX | stdout carries only the result: DOT, Mermaid, JSON, or validate's problems, one per line. Every error goes to stderr as fnloom: TypeName: message, with status 1, and a closed pipe ends the command quietly; a command line that can't be run is argparse's usage error, status 2. validate prints nothing and exits 0 for a valid request, else every problem and status 1. diff OLD NEW compares two fnloom export documents, not git revisions: added, removed, changed with the fields that differ, and unversioned; an input is compared as an input, so it never changes because a consumer did. Its status is 0 for no change, 1 for a change and 2 for a file that is no export. |
| Q99 | @check_output and @check_output_custom turn a node a into three: a_raw (the function), one a_<check>_validator node per check, returning a ValidationResult, and a final a returning a_raw's value. All are listed, requestable and hooked. Results are read only as the validator nodes' values, found by their hamilton.data_quality.* tags. |
FIX | A check runs inside its node, after the function returns: no node is added, and the node keeps its name and place. A failed fail check is the node's own failure, so after_node gets ok=False and the error; a node that raises is never checked, as in Hamilton. Every check's result, passed or not, is on NodeEnd.checks as a fnloom.CheckResult (name, importance, passes, message, diagnostics). See Checking values. |
| Q100 | importance defaults to "warn", which logs a WARNING and goes on. "fail" raises DataValidationError, which subclasses Exception, not ValueError; its only argument is a list of messages, so str(error) is the list's repr. |
FIX | importance defaults to "fail". A failure raises fnloom.ValidationError, a ValueError, with one line per failed check (a: check range failed: 12 is not in [0, 10]), its node, and the failed results. All of a node's checks run first, then every failed fail check is raised together, as in Hamilton. A failed warn check logs one WARNING per run on the fnloom logger, through logging only, and the run goes on. |
| Q101 | @check_output's checks are resolved against the return annotation: pandas Series checks (range, values_in, data_type, NaN and statistics checks), numbers.Real and str checks, and allow_none for anything. A generic or union annotation (list[int], int \| None) crashes the build with a raw TypeError: issubclass() arg 1 must be a class; an argument with no matching check is a ValueError at build. data_type is an exact isinstance. |
FIX | check_output(range=, values_in=, data_type=, allow_none=, contains=, does_not_contain=, allow_nans=, max_fraction_nans=, importance=, field=) reads any value: a scalar, or each item of a value that iterates, numpy and pandas values through their own vectorised methods, with no import. A str, bytes or mapping is one value, and a one-shot iterator is never consumed: refused at build when the annotation says so, a TypeError at run otherwise. data_type checks by value_matches, so generics and unions work. A set given to values_in is sorted, so its order never follows the hash seed; a member whose == gives no bool, as an array's, is a TypeError when decorating, and an item like that is not a member. None counts as missing for the NaN checks, as pandas counts it. A check that can't read the return annotation is a TypeError at build naming the node; bad arguments are refused when decorating. allow_none=True lets None pass every check of the node. A node annotated as a one-shot iterator, or a union with one, can't carry checks, and a one-shot value at run is a TypeError before any check reads it. The statistics and pandera checks are documented custom checks. |
| Q102 | @check_output_custom takes DataValidator subclasses. applies_to is never called, validate sees only the node's value, and returning a bool fails later with AttributeError: 'bool' object has no attribute 'passes'. An async validator on the sync driver builds, then fails at run with AttributeError. |
FIX | A custom check is a plain function or a fnloom.Validator. A function takes the value, and an inputs parameter for the node's arguments by parameter name, and returns True, None, False, a message str or a CheckResult. A Validator's validate(value, inputs) returns the same, and its applies_to(declared) is called at build, where False is a TypeError naming the node. An async def check is a TypeError when decorating. A check that raises fails the node with its own error, noting in check <name>, whatever its importance. |
| Q103 | A second @check_output on one function fails the build (both make <name>_raw). Which value is checked follows decorator order: above or below @extract_fields checks the fields or the dict, above or below @pipe_output the value after or before the steps, and a @mutate step is never checked. An overridden final node isn't checked; a cache hit isn't checked again, and its warning isn't logged again. |
FIX | Checks stack in any number and order, and run in the order written, top first. A check is on the function's value after its @pipe_output and @mutate steps; field="p" puts it on an extracted field, whatever the decorator order. Every @parameterize row and @config.when variant is checked, as in Hamilton. An overridden node isn't checked, and a cache hit isn't checked again or logged again, as in Hamilton and Q87; a value failing a fail check is never stored. The checks are part of the node's cache key, so a check added or changed later misses rather than returning a value it never checked. |
| Q104 | @pipe_input and @pipe_output lower each step(...) to a node named <fn>.with_<step>, with _1, _2 for repeats, or .named's name; @pipe_output runs the function as <fn>.raw and adds an identity node <fn>. Repeat numbers count only the steps the config kept, so f.with_add_1 names a different step under another config. User tags go on the final node only, and @pipe_output sets the final node's module tag to hamilton.function_modifiers.macros. |
KEEP / FIX | KEEP the names and shapes: each step is an ordinary node, requestable, typed, cached and hooked; source, value and bare literals bind a step's other parameters; tags go on the final node. FIX: repeat numbers count every step written, and <fn>.raw is made even when the config keeps no step, so names never change with the config, and every node's module is its own function's. A step's node stands for no function in a request. Not there: Hamilton's namespace= on @pipe_input and .named, and @pipe_output(on_output=...); a step's node is always <node>.<label>, and .on_output goes on each step. See Steps. |
| Q105 | A step parameter with a default, left unbound, becomes a graph input of its own name, not namespaced: every step in any pipe or module with that parameter shares one input, which can collide with a node. | FIX | A defaulted step parameter left unbound keeps its default and is never a graph input. A required one left unbound is a TypeError when decorating, as in Hamilton. |
| Q106 | on_input naming no parameter is ignored, and the first parameter is fed. @pipe_input on a function with no parameters fails the build with a raw IndexError. The head of a @pipe_input chain is not type checked: the graph input takes the first step's annotation. An .on_output field the function doesn't produce is ignored. |
FIX | Each is refused: on_input naming no parameter, and a function with no parameters, are a TypeError when decorating; the first step must take the parameter's own annotation, by type_matches, and an unknown .on_output field is a ValueError, both at build. A step with no parameters, *args, an unknown binding or a missing annotation is a TypeError when decorating, as in Hamilton. |
| Q107 | @mutate(*targets, **bindings) appends a step to each target's @pipe_output, or wraps it in one, in definition order; apply_to(fn, **bindings) replaces the bindings for one target; the mutator is no node. A target in another module is refused with NotSameModuleError, whose message points at a setting that doesn't exist. A target that is no node, or an .on_output field the target doesn't produce, is silently ignored. A string target fails with AttributeError. Applying a mutator twice, as re-running a notebook cell does, adds its step twice. A mutated value escapes the target's @check_output. |
KEEP / FIX | KEEP the shape: the steps follow the target's own @pipe_output steps, in the order the mutators are defined; apply_to bindings replace the mutator's; the mutator is no node, even a public one. FIX: a target in another module works when both modules are built, and a module of mutators alone builds; a target no built module defines is a ValueError at build naming the mutator and the target; a string or a lambda is a TypeError when decorating; a redefinition of a mutator, as re-running a notebook cell makes, replaces its step, while mutators sharing one function's code, made in a loop or a factory, each add theirs, as in Hamilton; one mutator bound the same and applied twice, or named twice as a target, adds its step once, its bindings matched by identity or fingerprint, never by ==; the target's checks see the mutated value (Q103). |
| Q108 | @pipe_input with @parameterize or @inject fails the build: Cannot call NodeExpander ... on more than one node. @pipe_output with @extract_fields checks the steps' types against the whole dict or each field, by decorator order. |
FIX | @pipe_input with @parameterize gives each row its own chain, named under the row. @pipe_output on a function with extracted fields applies to each field, or to those .on_output names, in any decorator order. |
| Q109 | @inject(param=source/value/group) is a one-row @parameterize that keeps the function's name; its docstring is templated with the values. A bare value is refused, and @inject with @parameterize on one function is refused when decorating. @inject with @pipe_input fails the build. |
KEEP / FIX | KEEP: fnloom.inject(**bindings) keeps the function's name, templates its docstring, refuses a bare value, and refuses @parameterize on the same function, in either order. FIX: with @pipe_input, the one row gets its chain (Q108). |
| Q110 | @resolve(when=ResolveAt.CONFIG_AVAILABLE, decorate_with=...) applies the decorators decorate_with returns for the config keys its parameters name, a defaulted parameter an optional key. It needs hamilton.enable_power_user_mode=True in the config; without the key at all it raises a raw KeyError. A missing required key is an InvalidDecoratorException naming the decorator object; a resolved @parameterize beside a written one fails with too many values to unpack. Config keys a resolver reads become config nodes. |
KEEP / FIX | KEEP the shape: fnloom.resolve(decorate_with), whose parameters are config keys, a defaulted one optional, returning a decorator or a sequence of them, the first outermost, applied as if written. FIX: no power-user mode; a missing required key is a ValueError at build naming the function and the key; a value that is no decorator, or one that changes which functions are nodes (@config.when*, @exclude, @mutate, @resolve), is a TypeError at build; @mutate is refused before it writes, however a resolver comes to apply it, so a refused build leaves every function as it was. The decorators go on a copy of the function made for each build, so builds with other configs never see each other's, and the copy stands for the function in requests and views. Config never becomes a node (Q56). Its known limit: a resolver that imports a module defining a @mutate is refused too, so import that module at the top of the file instead. |
| Q111 | A NodeExecutionMethod adapter's run_to_execute_node replaces how a node is called; it may call node_callable(**node_kwargs) any number of times or not at all. Only one may be given per driver: a second is a ValueError at build naming do_node_execute. For generated nodes, such as @check_output's and @pipe_output's, it is handed Hamilton's own internal callables. |
KEEP / FIX | KEEP: a wrapper may run its node any number of times, or return a value of its own. FIX: Builder.with_node_wrappers(*wrappers) takes any number, composed as an ordered chain, the first outermost; each is wrapper(node, call, inputs), with the node's NodeView, a call() running the rest of the chain and the node, and its arguments by parameter name; every node that runs a function of the user's is wrapped, a step's included, and an extracted field or a @pipe_output node passing its last step's value on is not. Hooks fire outside the chain and output checks run inside it, except a @pipe_output node's, which run on the unwrapped node passing its last step's value on, so no wrapper sees them fail. A cache hit runs no wrapper, and wrappers are not part of the cache key. A non-callable or async def wrapper, or one that can't take the three arguments, is a TypeError from with_node_wrappers; build_async refuses an async def node while wrappers are set. Wrappers are carried by pickling, copy, rebuild() and build_async. See Node wrappers. |
| Q112 | @subdag(module, inputs={...}, config={...}) reuses a module under the decorated function's name: each inner node is <fn>.<name>, and the function body, reading inner nodes by their names, is the final node. inputs= feeds that copy a source() or value() of its own. An unused source() is silently dropped and an unused value() becomes an orphan node; an input the module defines itself is a "defined more than once" error. A misspelled final-function parameter becomes a namespaced input, a namespaced input passed at run is silently ignored, and a value() inside config= puts the wrapper object into the node. @parameterized_subdag makes one @subdag per row. |
FIX / DROP | FIX: Builder.with_modules(*modules, namespace="eu", bind={"gross": source("eu_gross")}) feeds a namespace (Q74) its own inputs. Each key is a name the namespace's nodes read that none of its modules defines; a source("x") reads x outside the namespace (an input, an outer node, or another namespace's node by its full name), a value(v) is a constant. A key no selected node of the namespace reads, or one the namespace defines, is a ValueError at build naming every key and its namespace; bind= without namespace=, a binding that is no source or value, or a key that is no identifier is refused by with_modules, as is binding a key again otherwise. Bindings show as each reader's dependencies, are in the export, graph_version and the cache key as a value() parameter's are (a value holding a set keyed the same on every hash seed), and are carried by pickling, copy and rebuild(). The final node is an ordinary function reading source("eu.doubled"), so a misspelling is a missing input. DROP: config= per copy (a config key per copy instead) and @parameterized_subdag (a loop of with_modules calls) (Coming from Hamilton). |
| Q113 | @does(replacing_fn, **mapping) makes one node, typed and named after the decorated function with an empty body, that calls replacing_fn with the arguments renamed. Nothing compares the two signatures' types, so a str comes back from a node typed int. The empty-body check compares bytecode, so on 3.14 a body of ... or a docstring passes too. A mapping to a missing parameter is a raw KeyError when decorating. A public replacing function in the same module is a node too. |
DROP | Not planned. The fnloom way is to call the function: def total(x: int, y: int) -> int: return _sum_all(a=x, b=y) is named, typed and cached as written. fnloom.node(fn, name=..., bind=...) makes the node from the function itself (Coming from Hamilton). |
| Q114 | @schema.output(("a", "int"), ...) records one tag, hamilton.internal.schema_output = "a=int". Nothing checks it at run time. At build it refuses any type that isn't a registered dataframe type, so with no extensions loaded it refuses even pandas.DataFrame. It never parses its field strings, and its only reader, the visualizer, expects : where it writes =. |
DROP | Not planned. A check that runs is @check_output_custom with a function testing the columns, which fails with a ValidationError by default (Q99 to Q102). Metadata alone is @tag(schema=...) (Coming from Hamilton). |
| Q115 | @dataloader() splits a function returning tuple[T, dict] into <fn>.loader, holding the whole tuple, and <fn>, holding the data. @datasaver() only adds tags, and only on a function annotated exactly -> dict. Neither checks the return shape at run, so a loader returning a list yields its first element. @parameterize can't stack on @dataloader. The tags are read by the visualizer's styling, the MLflow plugin and the UI tracker. |
DROP | Not planned. A loader is a function returning its data; metadata becomes its own node through @extract_fields({"data": ..., "metadata": dict}), and @tag(kind="loader") marks it. A saver is a node, or an after_run hook (Q84) (Coming from Hamilton). |
| Q116 | @load_from.<format>(...) replaces a parameter with two nodes, <fn>.load_data.<param> and <fn>.select_data.<param>, through a registry of adapters (json, file, pickle, literal and environment with no extensions). @save_to.<format>(...) adds a saver node, save.<fn> by default, that writes only when requested. Materializers (to, from_, Driver.materialize, Builder.with_materializers) do the same per call or per driver. A loader silently shadows a node named like its parameter. The loader node's declared type has its tuple backwards. Loaded values aren't type-checked. With two stacked @save_tos and no target_, the outer one saves the inner saver's metadata. A registered GraphExecutionHook hides materialize's real error behind an UnboundLocalError. |
DROP | Not planned: materializers are a design §2 non-goal. Loading is a node, its annotation checked against each reader's at build and its value at run with with_output_type_checks() (Q87). Saving is an after_run hook given RunEnd.results (Q84), or a saver node requested with the outputs. A @config.when variant chooses the source (Coming from Hamilton). |
| Q117 | A StaticValidator adapter's run_to_validate_node and run_to_validate_graph return (bool, message). They run once at build and again on each materialize, and a failure is a ValidationException(Exception) aggregated per node. A class given instead of an instance is accepted and never runs. A None message crashes with a TypeError. Graph checks are skipped once a node check fails. Inputs are checked as nodes. The count in the message counts nodes, not errors. |
DROP | Not planned. The fnloom way is a function over Driver.nodes, run right after build(), that raises one ValueError listing every problem (Coming from Hamilton). To be reconsidered if checks carried through rebuild() are asked for. |
Not yet decided¶
Array shape/dtype contracts and length alignment for array-valued inputs were found against one consumer's numpy
workloads, not against Hamilton in general. They are out of core and may return as an opt-in policy once a general-user
case exists. Numeric widening (an int for float, numpy scalars for int and float) is the opt-in
Builder.with_numeric_widening(); see Type checks.