Configuration#
This guide is to help administrators configure RSPM for your organizations specific use cases.
Prerequisites#
If you haven't already, follow the instructions for installing and licensing RSPM.
Your user must be a member of the rstudio-pm group:
Terminal
sudo usermod -aG rstudio-pm <USER>
Replace <USER> with the desired RSPM admin user. After running this command, restart your shell.
Finally, add the rspm binary in one of two ways:
- Set up an alias:
alias rspm='/opt/rstudio-pm/bin/rspm' - Add the
rspmCLI binary to your system path:export PATH=/opt/rstudio-pm/bin:$PATH
Overview#
The right sidebar contains quick start guides. Follow the ones that are best suited for your use case(s).
For example, most basic RSPM installations will:
You can always add additional package support in the future.
Once you have configured the server, share the URL to the web interface with your users. Users can set up R, Python, or RStudio to use RSPM by following the instructions included in each repository's Setup page.
Tip
RStudio Workbench, formerly RStudio Server Pro, can be configured to use RSPM without requiring user setup. For more information, see the configuration instructions.
Serving CRAN Packages#
A common use case for RSPM is making CRAN packages available in environments with restricted internet access.
To make the CRAN packages available:
- Ensure that RSPM has the appropriate metadata using the
synccommand. RSPM pulls packages and metadata from the RStudio CRAN service. - Create a repository and subscribe it to the built-in source named "cran".
Terminal
<< # Create a repository:
$ rspm create repo --name=prod-cran --description='Access CRAN packages'
<< # Subscribe the repository to the cran source:
$ rspm subscribe --repo=prod-cran --source=cran
<< # Initiate a sync:
$ rspm sync --type=cran
Future updates occur on a schedule dictated in the configuration file. For more information, see the Updates from CRAN section.
After completing these steps, the prod-cran repository is available in the web interface.
Serving CRAN Snapshots#
If you prefer to make CRAN packages available for only specific dates, you can use a cran-snapshot source. For example:
Terminal
<< # Initiate a sync:
$ rspm sync --type=cran
<< # List available snapshot dates (increase the count to see more options)
$ rspm list cran snapshots --count=25
<< # Create source from desired date:
$ rspm create source --name=cran-by-date --type=cran-snapshot --snapshot=2020-07-09
<< # Create a repository:
$ rspm create repo --name=prod-cran --description='Access CRAN packages by date'
<< # Subscribe a repository to the cran-snapshot source:
$ rspm subscribe --repo=prod-cran --source=cran-by-date
<< # To update in the future:
$ rspm update --source=cran-by-date --snapshot=2020-10-06 --commit
Serving Curated CRAN Subsets#
If you prefer to make only a subset of CRAN packages available, you can use a curated-cran source. Curated CRAN sources automatically identify the dependencies required for a set of top-level R packages. You can also specify which CRAN date to use.
Terminal
<< # Ensure you have CRAN metadata:
$ rspm sync --type=cran
<< # Create the curated-cran source from a specific starting date:
$ rspm create source --name=subset --type=curated-cran --snapshot=2020-07-06
<< # Specify the top-level packages you want to add:
$ rspm add --packages=ggplot2,shiny --source=subset
The result will contain information on all the packages that will be added. The proposal
can be saved to a CSV file using the csv-out flag. The required dependencies
for the named packages are automatically discovered and included. Optionally use the
--include-suggests flag to also discover and add suggested packages.
This action will add the following packages:
Name Version Path License Needs Compilation Dependency
BH 1.66.0-1 BSL-1.0 no true
crayon 1.3.4 MIT + file LICENSE no true
digest 0.6.15 GPL (>= 2) yes true
htmltools 0.3.6 GPL (>= 2) no true
httpuv 1.4.3 GPL (>= 2) | file LICENSE yes true
ISLR 1.2 GPL-2 no false
jsonlite 1.5 MIT + file LICENSE yes true
later 0.7.3 GPL (>= 2) yes true
... ... ... ... ...
To commit the changes, follow the instructions:
Terminal
<< # Commit the top-level packages you want to add:
$ rspm add --packages=ggplot2,shiny --source=subset --snapshot=2020-07-06 --commit
In order to ensure consistency, the entire source must be updated at once, not individual packages.
Terminal
<< # Update packages in a curated-cran source:
$ rspm update --source=subset --snapshot=2020-10-07
A preview of the changes is presented:
This action will add or archive the following packages:
Name Version Path License Needs Compilation Dependency Action
backports 1.1.10 GPL-2 | GPL-3 yes true add
callr 3.4.4 MIT + file LICENSE no true add
glue 1.4.2 MIT + file LICENSE yes true add
jsonlite 1.7.1 MIT + file LICENSE yes true add
Rcpp 1.0.5 GPL (>= 2) yes true add
rlang 0.4.7 GPL-3 yes true add
tibble 3.0.3 MIT + file LICENSE yes true add
vctrs 0.3.4 GPL-3 yes true add
withr 2.3.0 GPL (>= 2) no true add
backports 1.1.8 GPL-2 | GPL-3 yes archive
callr 3.4.3 MIT + file LICENSE no archive
glue 1.4.1 MIT + file LICENSE yes archive
jsonlite 1.7.0 MIT + file LICENSE yes archive
...
To commit the changes:
Terminal
<< # Update packages in a curated-cran source:
$ rspm update --source=subset --snapshpot=2020-10-07 --commit
Finally, be sure to subscribe a repository to the source to make the packages available to users:
Terminal
<< # Create a repository:
$ rspm create repo --name=prod-cran --description='Access curated CRAN packages by date'
<< # Subscribe a repository to the curated-cran source:
$ rspm subscribe --repo=prod-cran --source=subset
Tip
Curated CRAN repositories also support adding a large number of packages that are
specified in a file. To do this, create a file containing one package name per line.
For example, /tmp/packages.csv:
plumber
shiny
ISLR
Then use the add command, this time using the --file-in flag:
Terminal
$ rspm add --file-in='/tmp/packages.csv' --source=subset
Distributing Local Packages#
Many teams have a handful of internally built packages. If your internal packages are tracked in Git, then refer to the Serving Git Packages section.
If your internal packages are already built, they can be added as Local packages:
- Create the bundled version of each package: R packages - Package Structure
Note
If you are unfamiliar with building the bundled version of a package, then reach out to the R developer that maintains the package.
- Copy the resulting tar files to the RSPM server:
Terminal
<< # Create a local source:
$ rspm create source --name=prod-internal-src
<< # Add each local package tar file to the source:
<< # The tar file must be rwx by the user running the CLI and the account running
<< # RSPM (rstudio-pm by default)
$ rspm add --source=prod-internal-src --path='/path/to/package_1.0.tar.gz'
<< # Create a repository:
$ rspm create repo --name=prod-internal --description='Stable releases of our internal packages'
<< # Subscribe the repository to the source:
$ rspm subscribe --repo=prod-internal --source=prod-internal-src
RSPM automatically supports multiple versions of each package. When the R developers are ready for the next release of a package, simply run:
Terminal
rspm add --source=prod-internal-src --path='/path/to/package_2.0.tar.gz'
RSPM ensures that version 2.0 is the default for new installations.
Note
If you wish to use an older version, RSPM keeps version 1.0 in the repository's archive.
Most internal packages depend on packages from CRAN. In this case, the easiest option is to create a repository that subscribes to both a local source and CRAN.
Tip
Repositories can subscribe to multiple sources if you want users to be able to install internal packages and CRAN packages from a single place. See rspm subscribe --help.
Serving Git Packages#
The previous configuration uses a local source and requires manual steps to add and update packages. If your organization uses Git to store internal R packages, then you can automate this process using a Git source.
Git sources require a valid R installation. For more information, see the Building R Packages section.
Terminal
<< # Create a Git source:
$ rspm create source --type=git --name=prod-internal-src
<< # Create a Git builder, configured to surface tagged commits:
$ rspm create git-builder --url=https://bitbucket.example.com/r-pkg.git --source=prod-internal-src --build-trigger=tags
<< # Check the status and learn more about the build:
$ rspm list git-builds --source=prod-internal-src --name=r-pkg
<< # Read the build logs using the ID from the previous
<< # command output or from the "Activity" log in the UI:
$ rspm logs --transaction-id=[ID]
<< # Create a repository and subscribe it to the source:
$ rspm create repo --name=prod-git --description='Stable releases of our internal packages'
$ rspm subscribe --source=prod-internal-src --repo=prod-git
Packages can be built using Git endpoints accessed via HTTP(s) or SSH URLs:
https://github.com/user/repo.git vs. git@github.com:user/repo.git.
Note
For Git systems with non-standard ports, RSPM also supports an SSH schema that can be used
to specify the port. In practice this looks like: ssh://git@github.com:22/rstudio/rstudio.git. See the
SSH Protocol section of
Git on the Server - The Protocols for more
information.
If the Git URL uses SSH, then it requires an SSH key for authentication. In this case, import the key before
using the create command.
SSH keys are not required to use a passphrase, but a secure key with passphrase is recommended.
Terminal
<< # Import the SSH key:
<< # passphrase file should just be text file with passphrase for key (avoid leaving in bash history):
$ rspm import --name=read-r-pkg --path=/path/to/ssh/key --passphrase-path=/path/to/passphrase/file
<< # Optionally, remove the key from disk:
<< # rm /path/to/ssh/key
<< # Create the Git builder
$ rspm create git-builder --url=user@bitbucket.example.com/r-pkg.git --source=prod-internal-src --build-trigger=tags --ssh-key=read-r-pkg
For more information on Git sources and SSH key security, see the Git Sources and SSH Key Security sections.
Serving Bioconductor Packages#
RSPM can serve Bioconductor packages from two types of repositories:
- Bioconductor repositories provide multiple versions of Bioconductor side by side and serve as a Bioconductor mirror. Packages are installed from these repositories using the BiocManager package. This is the most common way to make Bioconductor packages available.
- R repositories provide a single version of Bioconductor in a CRAN-like repository.
Packages are installed from these repositories using
install.packages. Unlike Bioconductor repositories, R repositories allow CRAN, Local, and Git packages to be served alongside Bioconductor packages in the same repository.
The appropriate repository type to use depends on how your organization uses Bioconductor.
Bioconductor Repositories#
To create a Bioconductor repository, run the following commands:
Note
All Bioconductor versions will be synced and automatically added to the repository.
Terminal
<< # Create a Bioconductor repository named 'bioconductor':
$ rspm create repo --type=bioconductor --name=bioconductor
<< Repository: bioconductor - Bioconductor
<< # Initiate a Bioconductor sync:
$ rspm sync --type=bioconductor
<< ...
<< Initiated Bioconductor synchronization for bioc-3.11-bioc. Depending on how much data has been previously synchronized, this could take a while. Actions will appear in the Package Manager UI as they are completed.
<< Initiated Bioconductor synchronization for bioc-3.11-data/annotation. Depending on how much data has been previously synchronized, this could take a while. Actions will appear in the Package Manager UI as they are completed.
<< ...
<< Bioconductor synchronization complete.
We also recommend creating a repository that provides compatible CRAN snapshots for past Bioconductor releases. To provide CRAN snapshots, create a repository that subscribes only to CRAN:
Terminal
<< # Check if an existing repository already subscribes to CRAN only:
$ rspm list
<< ...
<< prod-cran - R
<< - cran (CRAN)
<< # If not, run the following commands:
<< # Create a repository named 'prod-cran':
$ rspm create repo --name=prod-cran --description='Access CRAN packages'
<< # Subscribe the repository to the CRAN source:
$ rspm subscribe --repo=prod-cran --source=cran
<< # Initiate a CRAN sync:
$ rspm sync --type=cran
The Bioconductor repository should now be available in the web interface and ready to use. If a CRAN repository was created, the Setup page for the Bioconductor repository will provide instructions for configuring an appropriate CRAN snapshot in the Using a CRAN Snapshot section.
R Repositories#
Alternatively, you can serve a single version of Bioconductor in an R repository.
First, list the available Bioconductor versions and create a Bioconductor source for the desired version:
Terminal
<< # List available Bioconductor versions:
$ rspm list bioconductor versions
<< Available Bioconductor versions:
<< 3.12
<< 3.11
<< 3.10
<< ...
<< # Create a source for Bioconductor 3.11:
$ rspm create source --type=bioconductor --version=3.11
<< Source 'bioc-3.11':
<< Type: Bioconductor
Then, create an R repository, subscribe it to the source, and sync the source:
Terminal
<< # Create a repository for Bioconductor 3.11 named 'bioconductor-3.11':
$ rspm create repo --name=bioconductor-3.11 --description='Access Bioconductor 3.11 packages'
<< # Subscribe the repository to the Bioconductor source:
$ rspm subscribe --repo=bioconductor-3.11 --source=bioc-3.11
<< # Initiate a sync for the Bioconductor source:
$ rspm sync --type=bioconductor --source=bioc-3.11
<< Initiated Bioconductor synchronization for bioc-3.11-bioc. Depending on how much data has been previously synchronized, this could take a while. Actions will appear in the Package Manager UI as they are completed.
<< Initiated Bioconductor synchronization for bioc-3.11-data/annotation. Depending on how much data has been previously synchronized, this could take a while. Actions will appear in the Package Manager UI as they are completed.
<< ...
<< Bioconductor synchronization for bioc-3.11 complete.
For past Bioconductor releases, we also recommend serving CRAN packages from this repository that are compatible with the Bioconductor version. To add the appropriate CRAN packages:
- Find the last date that the Bioconductor release was current. This is usually the date of the next Bioconductor release, which can be found here.
- Find a CRAN snapshot that coincides with this date using the
list cran snapshotscommand. - Create a CRAN snapshot source and add it to the repository.
For example, to add CRAN packages for Bioconductor 3.11 (current until October 28, 2020):
Terminal
<< # Ensure CRAN metadata is available:
$ rspm sync --type=cran
<< # List available CRAN snapshots:
$ rspm list cran snapshots
<< ...
<< 2020-10-29
<< 2020-10-28
<< 2020-10-27
<< # Create a CRAN snapshot source for October 28, 2020:
$ rspm create source --type=cran-snapshot --snapshot=2020-10-28 --name=cran-2020-10-28
<< # Subscribe the CRAN snapshot source:
$ rspm subscribe --repo=bioconductor-3.11 --source=cran-2020-10-28
<< Repository: bioconductor-3.11
<< Sources:
<< --bioc-3.11 (Bioconductor)
<< --cran-2020-10-28 (CRAN Snapshot)
Python and Mirroring PyPI (Beta)#
Another common use case for RSPM is making PyPI (the Python Package Index) packages available in environments with restricted internet access.
To make the PyPI packages available:
- Ensure that RSPM has the appropriate metadata using the sync command. RSPM pulls packages and metadata from the RStudio PyPI service.
- Create a repository and subscribe it to the built-in source named "PyPI".
Terminal
<< # Create a Python repository:
$ rspm create repo --name=pypi --type=python --description='Access PyPI packages'
<< Repository: pypi - Python
<< # Subscribe the repository to the PyPI source:
$ rspm subscribe --repo=pypi --source=pypi
<< Repository: pypi
<< Sources:
<< --pypi (Python)
<< # Initiate a PyPI sync:
$ rspm sync --type=pypi
<< Initiated PyPI synchronization for pypi. Depending on how much data has been previously synchronized, this could take a while. Actions will appear in the Package Manager UI as they are completed.
<< Snapshots for pypi: 0 / 34 [----------------------------------------------------------------------------------------------------------------------------------]
<< Packages in pypi snapshot: 14127 / 231734 [======>-------------------------------------------------------------------------------------------------------] 5m9
Tip
Configure additional storage before enabling PyPI packages to handle the large metadata and package downloads.
For more information on RSPM's Python capabilities see the Python section of the documentation.