Skip to content

more system dependencies for language:r #339

Closed

Description

@lorenzwalthert

Some hook updates fail, e.g. this PR: lorenzwalthert/precommit#685. Looking at the diff, there are only two package versions updated and it seems they previously had recursive dependencies on R packages that depended on some system dependencies (like {V8}), but no code path actually checked for these dependencies previously. This apparently changed. In the pre-commit.ci build log, we get:

build attempt 1...
    => error
    [INFO] Installing environment for https://github.com/lorenzwalthert/precommit.
    [INFO] Once installed this environment will be reused.
    [INFO] This may take a few minutes...
    An unexpected error has occurred: CalledProcessError: command: ('/opt/r/bin/Rscript', '--vanilla', '/tmp/tmpx51lm3ap/script.R')
    return code: 1
    stdout:
        # Bootstrapping renv 1.2.3 ---------------------------------------------------
        - Downloading renv ... OK
        - Installing renv  ... OK
    
        - None of the packages recorded in the lockfile are currently installed.
        - Use `renv::restore()` to restore the project library.
        The following required system packages are not installed:
        - libcurl4-openssl-dev  [required by curl]
        - libnode-dev           [required by V8]
        - libxml2-dev           [required by xml2]
        - pandoc                [required by knitr]
        The R packages depending on these system packages may fail to install.
    
        An administrator can install these packages with:
        - sudo apt install libcurl4-openssl-dev libnode-dev libxml2-dev pandoc
    
        The following package(s) will be updated:
    

And further down:

    stderr:
        Warning messages:
        1: In file(file, "rt") :
          cannot open file '/opt/r/lib/R/doc/CRAN_mirrors.csv': No such file or directory
        2: In value[[3L]](cond) : cannot open the connection
        Error: error testing if 'V8' can be loaded [error code 1]
        /opt/r/lib/R/bin/R --vanilla -s -f '/tmp/RtmpN2MWVa/renv-install-3c21e7ad06e.R'
        ===============================================================================
    
        Error in dyn.load(file, DLLpath = DLLpath, ...) : 
          unable to load shared object '/pc/clone/LpTyOK7wR2u4Ro8izrEMqA/renv-default/renv/staging/1/V8/libs/V8.so':
          libnode.so.109: cannot open shared object file: No such file or directory
        Calls: loadNamespace -> library.dynam -> dyn.load
        Execution halted
        In addition: Warning messages:
        1: In file(file, "rt") :
          cannot open file '/opt/r/lib/R/doc/CRAN_mirrors.csv': No such file or directory
        2: In value[[3L]](cond) : cannot open the connection
        Execution halted
    Check the log at /pc/pre-commit.log

Can I send a PR to add these system dependencies similar to #334?

Activity

  1. asottile commented on Jul 28, 2026

    @asottile
    Member

    v8 at the very least is very very heavy. I'm not really inclined to add it :S

  2. lorenzwalthert commented on Jul 28, 2026

    @lorenzwalthert
    ContributorAuthor

    I see 😀. I don't think we need to add v8. We need to at least add libnode-dev, which seems half an MB, to solve the error above (assuming no more errors come up from the other missing system dependencies).

  3. asottile commented on Jul 28, 2026

    @asottile
    Member

    yeah but it depends on libnode which is v8

  4. MLopez-Ibanez commented on Aug 4, 2026

    @MLopez-Ibanez

    What in precommit requires V8? It seems mostly used for stuff that requires a web browser.

  5. MLopez-Ibanez commented on Aug 4, 2026

    @MLopez-Ibanez

    jsonvalidate is now in Suggest for the development version of precommit and it imports V8.

  6. asottile commented on Aug 4, 2026

    @asottile
    Member

    it's not pre-commit itself -- it's the R package rlang (?)

  7. MLopez-Ibanez commented on Aug 4, 2026

    @MLopez-Ibanez

    How does rlang depend on V8?

  8. asottile commented on Aug 4, 2026

    @asottile
    Member

    I haven't a clue lol

  9. MLopez-Ibanez commented on Aug 4, 2026

    @MLopez-Ibanez
  10. MLopez-Ibanez commented on Aug 4, 2026

    @MLopez-Ibanez
  11. MLopez-Ibanez commented on Aug 4, 2026

    @MLopez-Ibanez
  12. asottile commented on Aug 4, 2026

    @asottile
    Member

    please don't spread the nonsense of the hallucinating misinformation machine

  13. MLopez-Ibanez commented on Aug 4, 2026

    @MLopez-Ibanez
  14. lorenzwalthert commented on Aug 4, 2026

    @lorenzwalthert
    ContributorAuthor

    @MLopez-Ibanez I don't think the output of Claude or ChatGPT is helpful here. We appreciate it if you voice your interest in a solution here, e.g. through an upvote or a short comment, but pasting output of LLMs here without having the right prompt context won't help us solve the problem. To ensure it's not a problem in pre-commit.ci's image or caching mechanism, I triggered a new pre-commit.ci cache built with the same tag that used to work (lorenzwalthert/precommit#686), and it still succeeded. This means indeed, changing {rlang}, {Rcpp} or {xfun} version here causes a problem with V8 somehow. How I don't see. But my idea was to indeed remove {jsonvalidate} as a dependency here and see if I can make the tests pass.

    Before suggesting to replace an R dependency with a python tool, I advise you to check how the dependency is used. It is used in the renv-lockfile-validate hook, which should be enough to see that we can't simply call an external tool.

    If you don't want to rely on pre-commit.ci, you could use https://pre-commit.ci/lite.html where you control the compute environment that is used and hence just install the required system library.

  15. MLopez-Ibanez commented on Aug 4, 2026

    @MLopez-Ibanez

    You are right: I should have checked how jsonvalidate is used. I'll leave this link here: jeroen/V8#204 and stop trying to help. I can wait until you guys figure out how to fix it. Thanks for pre-commit.ci and the R hooks!

  16. lorenzwalthert commented on Sep 28, 2026

    @lorenzwalthert
    ContributorAuthor

    I see if you don't want to add V8, there might be another way out. Seems like according to jeroen/V8#204, updating R could help. We are on 4.4, current release is 4.6.1. Would you consider updating the R image? This is a good thing to do anyways, since it's almost 18 months old.

  17. asottile commented on Sep 29, 2026

    @asottile
    Member

    yeah! seems like a good idea -- the build for that is here if you want to take a stab at it: https://github.com/pre-commit-ci/runner-image/tree/main/languages/R

  18. lorenzwalthert commented on Sep 29, 2026

    @lorenzwalthert
    ContributorAuthor

    Seems like we just have to update R version and SHA in the build script and then you need to run it and create a new GitHub release?

  19. asottile commented on Oct 8, 2026

    @asottile
    Member

    R has been upgraded to 4.6.1

  20. lorenzwalthert commented on Oct 10, 2026

    @lorenzwalthert
    ContributorAuthor

    Ok, this actually doesn't work: While {V8} is now properly installed, we fail at {fs}:

     ✔ fs 2.1.0                                 [installed binary]
            Traceback (most recent calls last):
            10: renv::restore()
             9: renv_restore_run_actions(project, diff, current, lockfile, rebuild, 
                    strict, descriptions)
             8: renv_graph_install(descriptions)
             7: handle(pkg, list(success = success, output = output, elapsed = elapsed))
             6: renv_graph_install_finalize(entry$record, entry$prepared, installdir, 
                    project, linkable)
             5: tryCatch(renv_install_test(package), error = function(err) {
                    unlink(installpath, recursive = TRUE)
                    stop(err)
                })
             4: tryCatchList(expr, classes, parentenv, handlers)
             3: tryCatchOne(expr, names, parentenv, handlers[[1L]])
             2: value[[3L]](cond)
             1: stop(err)
            ?25h
        stderr:
            During startup - Warning message:
            package ‘stats’ in options("defaultPackages") was not found 
            Warning messages:
            1: In file(file, "rt") :
              cannot open file '/opt/r/lib/R/doc/CRAN_mirrors.csv': No such file or directory
            2: In value[[3L]](cond) : cannot open the connection
            Error: error testing if 'fs' can be loaded [error code 1]
            /opt/r/lib/R/bin/R --vanilla -s -f '/tmp/RtmplS4OMs/renv-install-96d6145c1.R'
            =============================================================================
        
            During startup - Warning message:
            package ‘stats’ in options("defaultPackages") was not found 
            Error in dyn.load(file, DLLpath = DLLpath, ...) : 
              unable to load shared object '/opt/r/lib/R/library/stats/libs/stats.so':
              libgfortran.so.5: cannot open shared object file: No such file or directory
            Calls: loadNamespace ... asNamespace -> loadNamespace -> library.dynam -> dyn.load
            Execution halted
            In addition: Warning messages:
            1: In file(file, "rt") :
              cannot open file '/opt/r/lib/R/doc/CRAN_mirrors.csv': No such file or directory
            2: In value[[3L]](cond) : cannot open the connection
            Execution halted
        Check the log at /pc/pre-commit.log
    

    Based on lorenzwalthert/precommit@52c59a2: https://results.pre-commit.ci/run/github/186313720/1791632394.HwVdS-5LTGOZATf4xSUrCQ

    Not sure if it's because we removed the system libraries liblapack3 and libblas3, but I have a hard time reproducing the error from CI locally.

    Do all repose with R hooks now use the latest image and will fail if they have {fs} dependency? As an immediate remedy: Can we roll back to the previous version? Sorry I did not think about testing much.

  21. lorenzwalthert commented on Oct 10, 2026

    @lorenzwalthert
    ContributorAuthor

    I am investigating and I think the solution is to add transitive system dependencies we dropped with in our PR. Will post an update soon.

  22. lorenzwalthert commented on Oct 10, 2026

    @lorenzwalthert
    ContributorAuthor

    This is fixed with #348. Unfortunately, an unrelated conda test failure blocks the merge.

  23. lorenzwalthert commented on Oct 12, 2026

    @lorenzwalthert
    ContributorAuthor

    Closed with #348.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions