REP: 111
Title: Multiple Package Manager Support for Rosdep
Version: $Revision: 83 $
Last-Modified: $Date: 2011-01-17 19:02:13 -0800 (Mon, 17 Jan 2011) $
Author: Tully Foote
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 28-June-2011
ROS-Version: 1.6
Post-History: 30-Jun-2011
Abstract
========
This REP add support to the rosdep YAML format [1]_ to support
multiple package managers on an individual platform. This includes a
a new 'source' package manager that replacees the previous mechanisms
for performing source-based installs.
Rationale
=========
Previously, the rosdep format has enabled two modes of installation of
system dependencies: the platform's preferred package manager, or a
custom bash script. Both of these installation mechanisms were
limiting.
Using only a single package manager limits integration options on
platforms like OS X, where there are multiple mechanisms available for
installing system libraries (e.g. MacPorts, distribute/easy_install).
For example, for Python libraries, it can be beneficial to use
distribute, which works with the default Python installation on OS X
10.6, instead of using the MacPorts Python libraries, which require
additional setup by the user.
Similarly, the bash script specification was brittle and provided
limited information to rosdep itself. The scripts were required to
be idempotent because rosdep was unable to detect whether or not the
dependency was already installed.
The goal of this REP is to make rosdep more robust and flexible, as
well as enable better support of platforms other than Ubuntu.
Motivation
==========
Existing Implementation
-----------------------
The current implementation of rosdep in ROS Diamondback [2]_ supports the
following structures:
* Simple
::
ROSDEP_NAME:
OS_NAME1: PACKAGE_ARGUMENTS_A
OS_NAME2: PACKAGE_ARGUMENTS_A
* Versioned
::
ROSDEP_NAME:
OS_NAME1:
OS_VERSION1: PACKAGE_ARGUMENTS_A
OS_VERSION2: PACKAGE_ARGUMENTS_A2
The names above resolve as follows:
* ``ROSDEP_NAME`` is the name referred to by manifest files. Examples: ``log4cxx``, ``gtest``.
* ``OS_NAME`` is the name of an OS. Examples: ``ubuntu``, ``fedora``, ``debian``, ``windows``, ``macports``.
* ``OS_VERSION`` (''optional'') is the name of specific versions in the OS. Examples: ``'10.04'``, ``squeeze``. If no ``OS_VERSION`` is specified, the rule is assumed to apply to all versions.
* ``PACKAGE_ARGUMENT`` has two interpretations. If it is a single line, it is expected to be a list of packages to pass to the package manager for the OS. If this field is a multi-line script, it is an idempotent bash script that is executed to install the dependency.
Example:
::
python:
ubuntu: python-dev
fedora: python-devel
macports: py26-scipy
boost:
ubuntu:
'11.04': libboost.42-all-dev
'10.10': libboost.42-all-dev
Support for Multiple Package Managers
-------------------------------------
For many Linux-based platforms, there are dominant package managers
that are the main mechanism for deploying software, such as deb/apt
for Debian-based systems and rpm/yum for RedHat-based systems.
However, for non-Linux platforms, as well as for cross-platform
language environments (e.g. Python, Java, Ruby), there exist other
software deployment mechanisms that are commonly used. There are also
platforms like OS X and Windows, where no package manager has a
dominant role.
On OS X, rosdep currently uses MacPorts as the default package
manager. While MacPorts does provide most of the libraries necessary
for integrating ROS on an OS X system, the current implementation
limits the flexibility of integration options. One area of
integration we wish to improve is Python. The MacPorts Python
integration requires a separate Python tree from the default OS X
Python interpreter. This has proven to be very brittle for users and
a common are of misconfiguration. Now that OS X now deploys with a
Python version that is compatible with ROS, we wish to explore using
distribute/easy_install for Python libraries instead.
Richer information from scripting interface
-------------------------------------------
The current support for bash scripts requires these scripts to be
idempotent: they must detect the presence of the required dependency
and install it if it is not present. There is no mechanism for rosdep
to determine for itself whether the required dependency is installed,
nor is there any mechanism for removing dependencies installed via
this mechanism. This creates confusion on the part of the user as
there is no mechanism for analyzing whether or not these dependencies
are installed correctly. Creating more structure for source-based
installations will enable simpler installation scripts, better error
reporting, and more advanced features (e.g. uninstall) in the future.
Integration with OS X
----------------------
OS X dependencies are specified generally as ``macports`` and are not
versioned to a particular OS X release. OS X releases frequently have
significant differences as Apple does not maintain much backwards
compatibility.
The current ROS OS X integration suffers from 32-bit/64-bit
compatibility problems due to the OS X 10.6 release. The current
rosdep specification also does not allow integrators to take advantage
of the builtin Python 2.5+ interpeter that was introduced in the OS X
10.5 release.
Another limitation of the OS X integration is that the rosdep syntax
did not allow for specification of MacPorts "variants" for individual
packages. The rosdep syntax interpreted the package argument value as
a list of packages, which was not flexible enough for specifying
variants on specific packages.
OS version identifiers
----------------------
The current OS version identifiers use the release number (e.g. 11.04)
instead of the release codename (e.g. lucid). Using the release
number has been a frequent source of bugs as YAML interprets ``11.04``
as a floating point number instead of a string. This requires
explicit string escaping when specifying rules (e.g. ``'11.04'``).
Specification
=============
The new specification for rosdep introduces a ``PACKAGE_MANAGER``
key in the YAML specification, which is a child key of the ``OS_NAME``.
rosdep no longer interprets the ``PACKAGE_MANAGER_ARGUMENTS`` and instead
passes them to the specified ``PACKAGE_MANAGER`` handler.
* Simple
::
ROSDEP_NAME:
OS_NAME1:
PACKAGE_MANAGER1:
PACKAGE_ARGUMENTS_A
OS_NAME2:
PACKAGE_MANAGER2:
PACKAGE_ARGUMENTS_A
* Versioned
::
ROSDEP_NAME:
OS_NAME1:
OS_VERSION1:
PACKAGE_MANAGER1:
PACKAGE_ARGUMENTS_A
OS_VERSION2:
PACKAGE_MANAGER2:
PACKAGE_ARGUMENTS_A2
The names above resolve as follows:
* ``ROSDEP_NAME`` is the name referred to by manifest files. Examples: ``log4cxx`` or ``gtest``.
* ``OS_NAME`` is the name of an OS. Examples: ``ubuntu``, ``osx``, ``fedora``, ``debian``, or ``windows``.
* ``OS_VERSION`` (*optional*) is the name of specific versions in the OS. Examples: ``'10.04'`` or ``squeeze``. If no ``OS_VERSION`` is specified, the rule is assumed to apply to all versions.
* ``PACKAGE_MANAGER`` (*optional in ROS Electric, required in ROS Fuerte*) is the name of the package manager to use for this rosdep. Examples: ``apt``, ``easy_install``, ``macports``.
* ``PACKAGE_ARGUMENT`` is free-form YAML that is be passed to the handler for the specified ``PACKAGE_MANAGER``.
Example
-------
For Ubuntu the default package manager is apt. The new syntax is:
::
rosdep_name:
ubuntu:
apt:
debian-package-name other-debian-package-name
or versioned as follows:
::
rosdep_name:
ubuntu:
lucid:
apt:
debian-package-name other-debian-package-name
OS version identifiers
----------------------
OS version identifiers will be standardized onto one-word codenames
for each distro.
For example:
* debian: squeeze
* ubuntu: lucid, maverick, natty, oneiric
* osx: snow, lion
Changes to OS X
---------------
The `OS_NAME` key for OS X will be changed to `osx` and will start
including `OS_VERSION` keys that represent the version number of the
OS X releases (e.g. `lion`). The default package is ``macports``,
which can be used as a `PACKAGE_MANAGER` field.
::
osx:
lion:
macports: bzip2
Disambiguation of OS_VERSION and PACKAGE_MANAGER
------------------------------------------------
For backwards compatibility, the ``PACKAGE_MANAGER`` is allowed to be
optional in the ROS Electric case. As both ``PACKAGE_MANAGER`` and
``OS_VERSION`` are optional, this creates an ambiguious case where
either ``OS_VERSION`` or ``PACKAGE_MANAGER`` is specified, but not
both.
In this ambiguous case, rosdep first interprets the key as a
``PACKAGE_MANAGER``. If this test fails, it will be interpreted as an
``OS_VERSION``. Developers should exercise caution in keeping
``OS_VERSION`` and ``PACKAGE_MANAGER`` keys globally distinct.
Dependencies
------------
Although not specified in this REP, developers of new package managers
for rosdep are encouraged to include a dependency specification in
their ``PACKAGE_ARGUMENTS`` values. This dependency specification
should enable rosdep rules to specify dependencies on other rosdep
rules. This, for example, would enable an ``easy_install`` package to
also depend on a ``apt``-based install. One use case of this is
rosinstall, which can be installed via ``easy_install``, but also
implicitly requires non-Python libraries for Git, Subversion,
Mercurial, and other non-``easy_install`` packages.
This advisory does not apply to package managers that are builtin to a
particular platform, such as apt and yum.
Internally, the rosdep library will provide APIs for developers to
trigger installation of these dependencies.
New source-based package manager
--------------------------------
A new source-based package manager is discussed in the motivation for
this REP. This specification of this new source-based package manager
will be described in a separate forthcoming REP.
Single package manager per rule
-------------------------------
rosdep rules are only allowed to specify a single package manager
to fulfill them.
Backwards Compatibility
=======================
All current rosdep.yaml files are compatible with the new proposed
syntax, though some features will be deprecated and the files should
be upgraded during the ROS Electric cycle to maintain compatibility.
Default Package Manager
-----------------------
If no package manager is specified, the previous default package
manager will be used.
Defaults:
* ubuntu: apt
* debian: apt
* mint: apt
* osx: macports
* freebsd: port
Bash scripts
------------
Multi-line values will still be treated as bash scripts in ROS
Electric. Support for this will be removed in ROS Fuerte.
Macports
--------
The ``macports`` rules will be supported during ROS Electric.
References and Footnotes
========================
.. [1] rosdep documentation
(
http://www.ros.org/wiki/rosdep)
.. [2] rosdep.yaml format in ROS Diamondback
(
http://www.ros.org/wiki/rosdep/rosdep.yaml/diamondback)
Copyright
=========
This document has been placed in the public domain.
?
..
Local Variables:
mode: indented-text
indent-tabs-mode: nil
sentence-end-double-space: t
fill-column: 70
coding: utf-8
End:
(text/plain)