Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

--check causes <<loop>> for basic expressions #1016

Open
runeksvendsen opened this issue Dec 28, 2021 · 12 comments
Open

--check causes <<loop>> for basic expressions #1016

runeksvendsen opened this issue Dec 28, 2021 · 12 comments

Comments

@runeksvendsen
Copy link

I get a <<loop>> when trying to evaluate many simple Nix expressions using --check, e.g. data/nix/tests/lang/eval-okay-let.nix:

./result/bin/hnix -v5 --trace --check ./data/nix/tests/lang/eval-okay-let.nix +RTS -xc
*** Exception (reporting due to +RTS -xc): (THUNK_STATIC), stack trace:
  Data.Text.Internal.IO.readTextDevice,
  called from Nix.Utils.readFile,
  called from Nix.Parser.parseFromFileEx,
  called from Nix.Parser.parseNixFileLoc,
  called from Nix.Standard.runWithBasicEffects,
  called from Main.main
*** Exception (reporting due to +RTS -xc): (THUNK_STATIC), stack trace:
  Nix.Eval.evalBinds,
  called from Nix.Eval.evalSelect,
  called from Nix.Eval.eval,
  called from Data.Fix.foldFix,
  called from Nix.Type.Infer.inferTop,
  called from Nix.Standard.runWithBasicEffects,
  called from Main.main
*** Exception (reporting due to +RTS -xc): (THUNK_STATIC), stack trace:
  Nix.Eval.evalBinds,
  called from Nix.Eval.evalSelect,
  called from Nix.Eval.eval,
  called from Data.Fix.foldFix,
  called from Nix.Type.Infer.inferTop,
  called from Nix.Standard.runWithBasicEffects,
  called from Main.main
*** Exception (reporting due to +RTS -xc): (THUNK_STATIC), stack trace:
  Nix.Eval.evalBinds,
  called from Nix.Eval.evalSelect,
  called from Nix.Eval.eval,
  called from Data.Fix.foldFix,
  called from Nix.Type.Infer.inferTop,
  called from Nix.Standard.runWithBasicEffects,
  called from Main.main
hnix: <<loop>>

This applies to all of the following expressions:

data/nix/tests/build-hook.nix
data/nix/tests/check-refs.nix
data/nix/tests/check-reqs.nix
data/nix/tests/check.nix
data/nix/tests/config.nix
data/nix/tests/dependencies.nix
data/nix/tests/export-graph.nix
data/nix/tests/filter-source.nix
data/nix/tests/fixed.nix
data/nix/tests/gc-concurrent.nix
data/nix/tests/gc-runtime.nix
data/nix/tests/hash-check.nix
data/nix/tests/import-derivation.nix
data/nix/tests/lang/eval-fail-assert.nix
data/nix/tests/lang/eval-fail-blackhole.nix
data/nix/tests/lang/eval-fail-remove.nix
data/nix/tests/lang/eval-fail-scope-5.nix
data/nix/tests/lang/eval-okay-arithmetic.nix
data/nix/tests/lang/eval-okay-attrnames.nix
data/nix/tests/lang/eval-okay-attrs.nix
data/nix/tests/lang/eval-okay-attrs2.nix
data/nix/tests/lang/eval-okay-attrs5.nix
data/nix/tests/lang/eval-okay-builtins.nix
data/nix/tests/lang/eval-okay-closure.nix
data/nix/tests/lang/eval-okay-concatmap.nix
data/nix/tests/lang/eval-okay-context-introspection.nix
data/nix/tests/lang/eval-okay-deepseq.nix
data/nix/tests/lang/eval-okay-delayed-with-inherit.nix
data/nix/tests/lang/eval-okay-delayed-with.nix
data/nix/tests/lang/eval-okay-elem.nix
data/nix/tests/lang/eval-okay-eq-derivations.nix
data/nix/tests/lang/eval-okay-eq.nix
data/nix/tests/lang/eval-okay-filter.nix
data/nix/tests/lang/eval-okay-flatten.nix
data/nix/tests/lang/eval-okay-functionargs.nix
data/nix/tests/lang/eval-okay-getattrpos.nix
data/nix/tests/lang/eval-okay-import.nix
data/nix/tests/lang/eval-okay-let.nix
data/nix/tests/lang/eval-okay-list.nix
data/nix/tests/lang/eval-okay-listtoattrs.nix
data/nix/tests/lang/eval-okay-map.nix
data/nix/tests/lang/eval-okay-mapattrs.nix
data/nix/tests/lang/eval-okay-overrides.nix
data/nix/tests/lang/eval-okay-partition.nix
data/nix/tests/lang/eval-okay-patterns.nix
data/nix/tests/lang/eval-okay-redefine-builtin.nix
data/nix/tests/lang/eval-okay-regex-match.nix
data/nix/tests/lang/eval-okay-remove.nix
data/nix/tests/lang/eval-okay-scope-2.nix
data/nix/tests/lang/eval-okay-scope-3.nix
data/nix/tests/lang/eval-okay-scope-4.nix
data/nix/tests/lang/eval-okay-scope-6.nix
data/nix/tests/lang/eval-okay-scope-7.nix
data/nix/tests/lang/eval-okay-search-path.nix
data/nix/tests/lang/eval-okay-toxml2.nix
data/nix/tests/lang/eval-okay-versions.nix
data/nix/tests/lang/eval-okay-with.nix
data/nix/tests/lang/lib.nix
data/nix/tests/lang/parse-fail-dup-attrs-2.nix
data/nix/tests/lang/parse-fail-dup-attrs-3.nix
data/nix/tests/lang/parse-fail-dup-attrs-7.nix
data/nix/tests/lang/parse-fail-regression-20060610.nix
data/nix/tests/lang/parse-okay-crlf.nix
data/nix/tests/multiple-outputs.nix
data/nix/tests/nar-access.nix
data/nix/tests/nix-copy-closure.nix
data/nix/tests/parallel.nix
data/nix/tests/remote-builds.nix
data/nix/tests/run.nix
data/nix/tests/search.nix
data/nix/tests/secure-drv-outputs.nix
data/nix/tests/shell.nix
data/nix/tests/simple.nix
data/nix/tests/structured-attrs.nix
data/nix/tests/timeout.nix
data/nix/tests/user-envs.nix
tests/eval-compare/builtins.appendContext.nix
tests/eval-compare/builtins.eq-bottom-00.nix
tests/eval-compare/builtins.fetchurl-01.nix
tests/eval-compare/builtins.lessThan-01.nix
tests/eval-compare/builtins.replaceStrings-01.nix
tests/eval-compare/builtins.toJSON.nix
tests/eval-compare/ellipsis.nix
tests/files/attrs.nix
tests/files/callLibs.nix
tests/files/hello2.nix
tests/files/lint.nix
tests/files/test.nix

Is --check supposed to work?

I'm on master, building hnix using nix-build using the "Full debug info" options on macOS.

@Anton-Latukha
Copy link
Collaborator

Thanks for the report.

The initial hunch is that I kind of went wild along here & swapped cons into snocs in some places, which looking back probably shouldn't, but I had some ideas to find optimizations in mind, thou not finished that. Those changes may cause infinite loops. But also there are many other things that may cause it.

A bunch of reports need bisecting. Thankfully commits are good for that & https://github.com/Anton-Latukha/git-bisect-master-pr. Would look into it when would make a release from master.
You are free to look into it.

The code that processes the --check starts here:

hnix/main/Main.hs

Lines 119 to 128 in d3d4356

when isCheck $
do
expr' <- liftIO $ reduceExpr mpath expr
either
(\ err -> errorWithoutStackTrace $ "Type error: " <> ppShow err)
(liftIO . putStrLn . (<>) "Type of expression: " .
ppShow . maybeToMonoid . Map.lookup @VarName @[Scheme] "it" . coerce
)
$ HM.inferTop mempty $ curry one "it" $ stripAnnotation expr'

@runeksvendsen
Copy link
Author

Thank you @Anton-Latukha, I will look into it

@runeksvendsen

This comment has been minimized.

@runeksvendsen

This comment has been minimized.

@Anton-Latukha

This comment has been minimized.

@Anton-Latukha
Copy link
Collaborator

Anton-Latukha commented Jan 17, 2022

Currently seems it was "always" present. (at least, an old bug). It is present in 0.12.0.1 version.
One of the things that prevented the travel back in versions is hnix-store frivolous open bounds, saltine 0.2 released & changed the internal modules, which expectedly broke the hnix-store-core compilations. So I went back in time and made a number of Hackage revisions for hnix-store-core bounds. During the next lookup into/release of hnix-store, one of the things would be to set those bounds haskell-nix/hnix-store#175.

Currently waiting for when Hackage would update with the new snapshot.

@Anton-Latukha
Copy link
Collaborator

Well, it goes beyond 0.9.1.

@Anton-Latukha
Copy link
Collaborator

Before 0.8.0.

@Anton-Latukha
Copy link
Collaborator

Anton-Latukha commented Jan 17, 2022

It is pre 0.7.1 at this point it essentially - it means it is before I become a maintainer.

Before, 0.6.1 - it is a long time ago.

@Anton-Latukha
Copy link
Collaborator

Anton-Latukha commented Jan 17, 2022

Syndrome is:

Works great:

cabal run hnix -- --check --expr '
                               let {
                                 x = "foo";
                                 y = "bar";
                               }
                               '
Up to date
Type of expression: [ Forall [ TV "a" ] (TVar (TV "a")) ]
rec {
  x = "foo";
  y = "bar";
  }.body⏎       

Referencing set - nope:

cabal run hnix -- --check --expr '
                               let {
                                 x = "foo";
                                 y = "bar";
                                 body = x + y;
                               }
                               '
Up to date
hnix: <<loop>>

@runeksvendsen
Copy link
Author

@Anton-Latukha thank you for looking into this.

I did a bit of debugging. The <<loop>> occurs in the type inference stage inside Nix.Scope.lookupVarReader when Nix.Scope.lexicalScopes is evaluated for the current scope. I inserted some traceM statements to deduce this.

The following diff against 4321e7d

diff --git a/main/Main.hs b/main/Main.hs
index 378fa468..00f487bd 100644
--- a/main/Main.hs
+++ b/main/Main.hs
@@ -32,6 +32,7 @@ import           Prettyprinter           hiding ( list )
 import           Prettyprinter.Render.Text      ( renderIO )
 import qualified Repl
 import           Nix.Eval
+import qualified Debug.Trace

 main :: IO ()
 main =
@@ -124,7 +125,7 @@ main' opts@Options{..} = runWithBasicEffectsIO opts execContentsFilesOrRepl
                 (liftIO . putStrLn . (<>) "Type of expression: " .
                   ppShow . maybeToMonoid . Map.lookup @VarName @[Scheme] "it" . coerce
                 )
-                $ HM.inferTop mempty $ curry one "it" $ stripAnnotation expr'
+                $ HM.inferTop mempty $ (\expr'' -> ("inferTop: " <> show expr) `Debug.Trace.trace` expr'') $ curry one "it" $ stripAnnotation expr'

                 -- liftIO $ putStrLn $ runST $
                 --     runLintM opts . renderSymbolic =<< lint opts expr
diff --git a/src/Nix/Scope.hs b/src/Nix/Scope.hs
index d9de14a3..70248725 100644
--- a/src/Nix/Scope.hs
+++ b/src/Nix/Scope.hs
@@ -10,6 +10,7 @@ import qualified Data.HashMap.Lazy             as M
 import qualified Text.Show
 import           Lens.Family2
 import           Nix.Expr.Types
+import qualified Debug.Trace

 --  2021-07-19: NOTE: Scopes can gain from sequentiality, HashMap (aka AttrSet) may not be proper to it.
 newtype Scope a = Scope (AttrSet a)
@@ -110,6 +111,11 @@ lookupVarReader
   -> m (Maybe a)
 lookupVarReader k =
   do
+    Debug.Trace.traceM $ "lookupVarReader: " <> show k
+    lexicalScopes' <- asks $ lexicalScopes @m . view hasLens
+    Debug.Trace.traceM $ "lexicalScopes length: " <> show (length lexicalScopes')
+    Debug.Trace.traceM $ "lexicalScopes: " <> show (lexicalScopes' :: [Scope a])
+
     mres <- asks $ scopeLookup k . lexicalScopes @m . view hasLens

     maybe

will produce this output:

$ cabal run hnix -- --check --expr '
                               let {
                                 x = 2;
                                 y = 1;
                                 body = x + y;
                               }
                               '
Up to date
lookupVarReader: VarName "x"
lexicalScopes length: 0
lexicalScopes: []
lookupVarReader: VarName "y"
lexicalScopes length: 0
lexicalScopes: []
lexicalScopes length: 0
lexicalScopes: []
lexicalScopes length: 0
lexicalScopes: []
inferTop: Fix (Compose (AnnUnit {annotation = SrcSpan {spanBegin = SourcePos {sourceName = "<string>", sourceLine = Pos 2, sourceColumn = Pos 32}, spanEnd = SourcePos {sourceName = "<string>", sourceLine = Pos 6, sourceColumn = Pos 33}}, annotated = NSelect Nothing (Fix (Compose (AnnUnit {annotation = SrcSpan {spanBegin = SourcePos {sourceName = "<string>", sourceLine = Pos 2, sourceColumn = Pos 36}, spanEnd = SourcePos {sourceName = "<string>", sourceLine = Pos 6, sourceColumn = Pos 33}}, annotated = NSet Recursive [NamedVar (StaticKey (VarName "x") :| []) (Fix (Compose (AnnUnit {annotation = SrcSpan {spanBegin = SourcePos {sourceName = "<string>", sourceLine = Pos 3, sourceColumn = Pos 38}, spanEnd = SourcePos {sourceName = "<string>", sourceLine = Pos 3, sourceColumn = Pos 39}}, annotated = NConstant (NInt 2)}))) (SourcePos {sourceName = "<string>", sourceLine = Pos 3, sourceColumn = Pos 34}),NamedVar (StaticKey (VarName "y") :| []) (Fix (Compose (AnnUnit {annotation = SrcSpan {spanBegin = SourcePos {sourceName = "<string>", sourceLine = Pos 4, sourceColumn = Pos 38}, spanEnd = SourcePos {sourceName = "<string>", sourceLine = Pos 4, sourceColumn = Pos 39}}, annotated = NConstant (NInt 1)}))) (SourcePos {sourceName = "<string>", sourceLine = Pos 4, sourceColumn = Pos 34}),NamedVar (StaticKey (VarName "body") :| []) (Fix (Compose (AnnUnit {annotation = SrcSpan {spanBegin = SourcePos {sourceName = "<string>", sourceLine = Pos 5, sourceColumn = Pos 41}, spanEnd = SourcePos {sourceName = "<string>", sourceLine = Pos 5, sourceColumn = Pos 46}}, annotated = NBinary NPlus (Fix (Compose (AnnUnit {annotation = SrcSpan {spanBegin = SourcePos {sourceName = "<string>", sourceLine = Pos 5, sourceColumn = Pos 41}, spanEnd = SourcePos {sourceName = "<string>", sourceLine = Pos 5, sourceColumn = Pos 42}}, annotated = NSym (VarName "x")}))) (Fix (Compose (AnnUnit {annotation = SrcSpan {spanBegin = SourcePos {sourceName = "<string>", sourceLine = Pos 5, sourceColumn = Pos 45}, spanEnd = SourcePos {sourceName = "<string>", sourceLine = Pos 5, sourceColumn = Pos 46}}, annotated = NSym (VarName "y")})))}))) (SourcePos {sourceName = "<string>", sourceLine = Pos 5, sourceColumn = Pos 34})]}))) (StaticKey (VarName "body") :| [])}))
lookupVarReader: VarName "x"
lexicalScopes length: 1
hnix: <<loop>>

So first a pass runs prior to inferTop which uses lookupVarReader without issues to produce the NExprLoc value. Then, inferTop runs and ultimate calls lookupVarReader for the x variable. Here, the length of lexicalScopes is equal to 1 and it crashes with a <<loop>> when trying to show the lexicalScopes value.

@Anton-Latukha
Copy link
Collaborator

Anton-Latukha commented Feb 10, 2022

Looked into the lookupVarReader code semantically.

So far not found the cause, but the process definitely approaches closer.

If to use only the dynamic scope search - the process terminates:

cabal run hnix -- --check --expr '
                                                                   let {
                                                                     x = "foo";
                                                                     y = "bar";
                                                                     body = x + y;
                                                                   }
                                                                   '
...
Type of expression: [ Forall [ TV "a" ] (TVar (TV "a")) ]

rec {
  x = "foo";
  y = "bar";
  body = x + y;
  }.body

Soon would ship the simplified version of the code & it would be easier to look further.
Maybe/probably in asks $ view hasLens for lexical scope the lens happens to be an infinite one, that is a guess, but since lexicalScopes is simple getter & body is involved in the lens...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants