Pulp operates with three main components that get installed potentially on different machines.
- This is the main application server that stores data and distributes content.
- This component runs on consumers and communicates with the server to provide remote content management.
- This is a command line component that comes as two pieces: admin-client, which manages the server; and consumer-client, which manages a consumer’s relationship to the server. admin-client can be run from any machine that can access the server’s REST API, but the consumer-client must be run on a consumer.
Additional steps are needed for upgrading Pulp 1.1 installations. More information can be found in the Pulp v1 Upgrades section of this guide.
Supported Operating Systems¶
- RHEL 6 & 7
- Fedora 19 & 20
- CentOS 6 & 7
- RHEL 5, 6, & 7
- Fedora 19 & 20
- CentOS 5, 6 & 7
- RHEL 6 & 7
- Fedora 19 & 20
- CentOS 6 & 7
- On RHEL 5, Pulp does not currently work with SELinux. SELinux must be set to Permissive or Disabled.
- The following ports must be open into the server:
- 80 for consumers to access repositories served over HTTP
- 443 for consumers to access repositories served over HTTPS
- 443 for clients (both admin and consumer) to access Pulp APIs
- 5672 for consumers to connect to the message bus if it is left unsecured
- 5671 for consumers to connect to the message bus if it is running over HTTPS
- The mod_python Apache module must be uninstalled or not loaded. Pulp uses mod_wsgi which conflicts with mod_python and will cause the server to fail.
MongoDB is known to have serious limitations on 32-bit architectures. It is strongly advised that you run MongoDB on a 64-bit architecture.
The MongoDB database can easily grow to 10GB or more in size, which vastly exceeds the amount of data actually stored in the database. This is normal (but admittedly surprising) behavior for MongoDB. As such, make sure you allocate plenty of storage within /var/lib/mongodb.
- Download the appropriate repo definition file from the Pulp repository:
For RHEL and CentOS systems, the EPEL repositories are required. Following commands will add the appropriate repositories for RHEL6 and RHEL7 respectively:
$ sudo rpm -Uvh https://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm
$ sudo rpm -Uvh https://dl.fedoraproject.org/pub/epel/7/x86_64/e/epel-release-7-2.noarch.rpm
For RHEL 5 systems, subscribe to the following RHN channels:
- MRG Messaging v. 1
- MRG Messaging Base v. 1
- Qpid RPMs are not available in the default CentOS repositories for CentOS releases 6.2 and earlier. Instructions on building those RPMs can be found at Building QPID RPMs.
You must provide a running MongoDB instance for Pulp to use. You can use the same host that you will run Pulp on, or you can give MongoDB its own separate host if you like. You can even use MongoDB replica sets if you’d like to have higher availability. For yum based systems, you can install MongoDB with this command:
$ sudo yum install mongodb-server
You need mongodb-server with version >= 2.4 installed for Pulp server. After installing MongoDB, you should configure it to start at boot and start it. For Upstart based systems:
$ sudo service mongod start $ sudo chkconfig mongod on
For systemd based systems:
$ sudo systemctl enable mongod $ sudo systemctl start mongod
On new MongoDB installations, MongoDB takes some time to preallocate large files and will not accept connections until it finishes. When this happens, Pulp will wait for MongoDB to become available before starting.
You must also provide a message bus for Pulp to use. Pulp will work with Qpid or RabbitMQ, but is tested with Qpid, and uses Qpid by default. This can be on the same host that you will run Pulp on, or elsewhere as you please. To install Qpid on a yum based system, use this command:
$ sudo yum install qpid-cpp-server qpid-cpp-server-store
In environments that use Qpid, the qpid-cpp-server-store package provides durability, a feature that saves broker state if the broker is restarted. This is a required feature for the correct operation of Pulp. Qpid provides a higher performance durability package named qpid-cpp-server-linearstore which can be used instead of qpid-cpp-server-store, but may not be available on all versions of Qpid. If qpid-cpp-server-linearstore is available in your environment, consider uninstalling qpid-cpp-server-store and installing qpid-cpp-server-linearstore instead for improved broker performance. After installing this package, you will need to restart the Qpid broker to enable the durability feature.
For Pulp to operate with the Qpid broker, authentication needs to be either disabled or configured. To disable authentication add auth=no to the qpidd.conf file. Qpid 0.24 and higher places the config file is at /etc/qpid/qpidd.conf, and earlier Qpid versions place the config file at /etc/qpidd.conf. Qpid must be restarted after changes are made to qpidd.conf.
To leave broker authentication enabled, you will need to configure SASL with a username/password, and then configure Pulp to use that username/password. Refer to the Qpid docs on how to configure username/password authentication using SASL. Once the broker is configured, update Pulp according to the Pulp Broker Settings Guide.
The server can be optionally configured so that it will connect to the broker using SSL by following the steps defined in the Qpid SSL Configuration Guide. By default, Pulp does not expect to use SSL and will connect to the broker using a plain TCP connection to localhost.
After installing and configuring Qpid, you should configure it to start at boot and start it. For Upstart based systems:
$ sudo service qpidd start $ sudo chkconfig qpidd on
For systemd based systems:
$ sudo systemctl enable qpidd $ sudo systemctl start qpidd
Install the Pulp server, task workers, and their dependencies. For Pulp installations that use Qpid, install Pulp server using:
$ sudo yum groupinstall pulp-server-qpid
The Pulp team believes that Pulp’s webserver and Celery workers can be deployed across several machines (with load balancing for the HTTP requests), but this has not been formally tested by our Quality Engineering team. We encourage feedback if you have tried this, positive or negative. If you wish to try this, each host that participates in the distributed Pulp application will need to have access to a shared /var/lib/pulp filesystem, including the web servers and the task workers. It is important that the httpd and celery processes are run by users with identical UIDs and GIDs for permissions on the shared filesystem.
For RabbitMQ installations, install Pulp server without any Qpid specific libraries using sudo yum groupinstall pulp-server. You may need to install additional RabbitMQ dependencies manually.
Edit /etc/pulp/server.conf. Most defaults will work, but these are sections you might consider looking at before proceeding. Each section is documented in-line.
- email if you intend to have the server send email (off by default)
- database if your database resides on a different host or port
- messaging if your message broker for communication between Pulp components is on a different host or if you want to use SSL. For more information on this section refer to the Pulp Broker Settings Guide.
- tasks if your message broker for asynchronous tasks is on a different host or if you want to use SSL. For more information on this section refer to the Pulp Broker Settings Guide.
- security to provide your own SSL CA certificates, which is a good idea if you intend to use Pulp in production
- server if you want to change the server’s URL components, hostname, or default credentials
Initialize Pulp’s database. It is important that the broker is running before initializing Pulp’s database. It is also important to do this before starting Apache or any Pulp services. The database initialization needs to be run as the apache user, which can be done by running:
$ sudo -u apache pulp-manage-db
If Apache or Pulp services are already running, restart them after running the pulp-manage-db command.
It is recommended that you configure your web server to refuse SSLv3.0. In Apache, you can do this by editing /etc/httpd/conf.d/ssl.conf and configuring the SSLProtocol directive like this:`SSLProtocol all -SSLv2 -SSLv3`
Start Apache httpd and set it to start on boot. For Upstart based systems:
$ sudo service httpd start $ sudo chkconfig httpd on
For systemd based systems:
$ sudo systemctl enable httpd $ sudo systemctl start httpd
Pulp has a distributed task system that uses Celery. Begin by configuring, enabling and starting the Pulp workers. To configure the workers, edit /etc/default/pulp_workers. That file has inline comments that explain how to use each setting. After you’ve configured the workers, it’s time to enable and start them. For Upstart systems:
$ sudo chkconfig pulp_workers on $ sudo service pulp_workers start
For systemd systems:
$ sudo systemctl enable pulp_workers $ sudo systemctl start pulp_workers
The pulp_workers systemd unit does not actually correspond to the workers, but it runs a script that dynamically generates units for each worker, based on the configured concurrency level. You can check on the status of those generated workers by using the systemctl status command. The workers are named with the template pulp_worker-<number>, and they are numbered beginning with 0 and up to PULP_CONCURRENCY - 1. For example, you can use sudo systemctl status pulp_worker-1 to see how the second worker is doing.
There are two more services that need to be running, but it is important that these two only run once each (i.e., do not enable either of these on any more than one Pulp server).
pulp_celerybeat and pulp_resource_manager must both be singletons, so be sure that you only enable each of these on one host if you are experimenting with Pulp’s untested HA deployment. They do not have to run on the same host, however.
On some Pulp system, configure, start and enable the Celerybeat process. This process performs a job similar to a cron daemon for Pulp. Edit /etc/default/pulp_celerybeat to your liking, and then enable and start it. Again, do not enable this on more than one host. For Upstart:
$ sudo chkconfig pulp_celerybeat on $ sudo service pulp_celerybeat start
$ sudo systemctl enable pulp_celerybeat $ sudo systemctl start pulp_celerybeat
Lastly, one pulp_resource_manager process must be running in the installation. This process acts as a task router, deciding which worker should perform certain types of tasks. Apologies for the repetitive message, but it is important that this process only be enabled on one host. Edit /etc/default/pulp_resource_manager to your liking. Then, for upstart:
$ sudo chkconfig pulp_resource_manager on $ sudo service pulp_resource_manager start
$ sudo systemctl enable pulp_resource_manager $ sudo systemctl start pulp_resource_manager
The Pulp Admin Client is used for administrative commands on the Pulp server, such as the manipulation of repositories and content. The Pulp Admin Client can be run on any machine that can access the Pulp server’s REST API, including the server itself. It is not a requirement that the admin client be run on a machine that is configured as a Pulp consumer.
Pulp admin commands are accessed through the pulp-admin script.
- Install the Pulp admin client packages:
$ sudo yum groupinstall pulp-admin
- Update the admin client configuration to point to the Pulp server. Keep in mind that because of the SSL verification, this should be the fully qualified name of the server, even if it is the same machine (localhost will not work with the default apache generated SSL certificate). Regardless, the “host” setting below must match the “CN” value of the server’s HTTP SSL certificate. This change is made globally to the /etc/pulp/admin/admin.conf file, or for one user in ~/.pulp/admin.conf:
[server] host = localhost.localdomain
Consumer Client And Agent¶
The Pulp Consumer Client is present on all systems that wish to act as a consumer of a Pulp server. The Pulp Consumer Client provides the means for a system to register and configure itself with a Pulp server. Additionally, the Pulp Consumer Client runs an agent that will receive messages and commands from the Pulp server.
Pulp consumer commands are accessed through the pulp-consumer script. This script must be run as root to permit access to add references to the Pulp server’s repositories.
1. For environments that use Qpid, install the Pulp consumer client, agent packages, and Qpid specific consumer dependencies with one command by running:
$ sudo yum groupinstall pulp-consumer-qpid .. note:: For RabbitMQ installations, install the Pulp consumer client and agent packages without any Qpid specific dependencies using ``sudo yum groupinstall pulp-consumer``. You may need to install additional RabbitMQ dependencies manually including the ``python-gofer-amqplib`` package.
- Update the consumer client configuration to point to the Pulp server. Keep in mind that because of the SSL verification, this should be the fully qualified name of the server, even if it is the same machine (localhost will not work with the default Apache generated SSL certificate). Regardless, the “host” setting below must match the “CN” value of the server’s HTTP SSL certificate. This change is made to the /etc/pulp/consumer/consumer.conf file:
[server] host = localhost.localdomain
The agent may be configured so that it will connect to the Qpid broker using SSL by following the steps defined in the Qpid SSL Configuration Guide. By default, the agent will connect using a plain TCP connection.
Set the agent to start at boot. For upstart:
$ sudo chkconfig goferd on $ sudo service goferd start
$sudo systemctl enable goferd $sudo systemctl start goferd
To try out Pulp, the default SSL configuration should work well. However, when deploying Pulp in production, you should supply your own SSL certificates.
In /etc/pulp/server.conf, find the [security] section. There is good documentation in-line, but make sure in particular that cacert and cakey point to the certificate and private key that you want Apache to use for HTTPS. Also make sure that Apache’s config in /etc/httpd/conf.d/pulp.conf matches these settings. If you plan to use Pulp’s consumer features, set ssl_ca_certificate.
If you want to use SSL with Qpid, see the Qpid SSL Configuration Guide.
Pulp Broker Settings¶
To configure Pulp to work with a non-default broker configuration read the Pulp Broker Settings Guide.
To configure Pulp for connecting to the MongoDB with username/password authentication, use the following steps: 1. Configure MongoDB for username password authentication. See MongoDB - Enable Authentication for details. 2. In /etc/pulp/server.conf, find the [database] section and edit the username and password values to match the user configured in step 1. 3. Restart the httpd service
$ sudo service httpd restart