Pulp notes:

Title:Pulp patching too notes
Author:Douglas O’Leary <dkoleary@olearycomputers.com>
Description:Information that’s different from the admittedly very good installation docs.
Date created:12/13/16
Date updated:12/21/17: updated for rhel7
Disclaimer:Standard: Use the information that follows at your own risk. If you screw up a system, don’t blame it on me...

Overview:

This is ver 2 of the docs. First one went into the installlation in some depth and basically rewrote the very good docs available at http://docs.pulpproject.org/user-guide/installation/f23-.html. This one’s not going to do that. I’ll follow the docs and make notes about where I had to deviate from them and why. I’ll leave the pulp people to keep their docs up to date because they do a whole lot better job of it than I do.

Commands:

pulp-admin –map:
Displays groups of valid command line arguments for pulp-admin. 354 lines long...
pulp-admin ${group} [ ${subcommadn} ] -help:
Displays help for a specific subcategory of pulp-admin commands.
pulp-admin [login -u ${u} [ -p ${p} ]] | [ logout ]:
Creates or deletes a pulp session. Required to execute pulp-admin commands but can be avoided by use of ~/.pulp/admin.conf
pulp-admin auth user create –login dkoleary –password ‘${pwd}’
Creates a user. Docs suggest not using --password ${pwd} option.
# pulp-admin auth role user [add|reomve] –role-id super-users –login dkoleary
Adds/removes dkoleary to super-users role

Lessons learned:

  • repoview cannot handle utf8 code in package metadata which results in repo publishing failures and, eventually, failed patching work. Details are available at https://pulp.plan.io/issues/2346 with an as yet globally unavailable patch to repoview available at https://pulp.plan.io/attachments/320. I’ve tested the patch and it seems to work with no identifiable side effects yet. Edit the /usr/bin/repoview file (after backing it up) adding the lines in the patch.

  • Required firewall ports:

    • 80 (http)
    • 443 (https)
    • 5671
    • 5672
  • (el6 only) Need to install the copr repo in order to install the pulp server otherwise, you’ll run into dependency issues:

    cd /etc/yum.repos.d
    wget https://copr.fedorainfracloud.org/coprs/g/qpid/qpid/repo/epel-6/group_qpid-qpid-epel-6.repo
    
  • Ensure host uses fqdn for hostname before installing pulp. ssl certs will become confused if not done.

  • Consumer script installation errors as defined below errors out if puppet is not installed. Either install it or remove pulp-puppet-handlers from the cli.

  • Disable ssl: not a good idea, but for testing, set verify_ssl: False in the following:

    • /etc/pulp/admin/admin.conf
    • /etc/pulp/consumer/consumer.conf
    • /etc/pulp/repo_auth.conf
    • /etc/pulp/nodes.conf # note: didn’t exist in my installation.
  • Default user/pwd: admin/admin Can be changed in /etc/pulp/server.conf in the [server] section:

    [server]
    default_login: Bog
    default_password: 5WHc69wHDWDaMSq
    
  • Creation of ~/.pulp/admin.conf with syntax below allows execution of pulp-admin commands w/o havinng to execute pulp-admin login:

    # cat ~/.pulp/admin.conf
    [auth]
    username: admin
    password: admin
    
  • Consumer registration must happen from consumer side.

  • To restart pulp services:

    • service httpd restart
    • service pulp_workers restart
    • service pulp_celerybeat restart
    • service pulp_resource_manager restart
  • If task is stuck in waiting and shows ‘unstarted’ as the ‘start time’, pulp-admin -vv repo task details --task-id=${long_hex}, restart

    services, particularly pulp_workers:

    # service pulp_workers restart
    celery init v10.0.
    Using config script: /etc/default/pulp_workers
    celery multi v3.1.11 (Cipater)
    > reserved_resource_worker-0@pulp.olearycomputers.com: DOWN
    > reserved_resource_worker-1@pulp.olearycomputers.com: DOWN
    > reserved_resource_worker-2@pulp.olearycomputers.com: DOWN
    > reserved_resource_worker-3@pulp.olearycomputers.com: DOWN
    celery multi v3.1.11 (Cipater)
    > Starting nodes...
            > reserved_resource_worker-0@pulp.olearycomputers.com: OK
            > reserved_resource_worker-1@pulp.olearycomputers.com: OK
            > reserved_resource_worker-2@pulp.olearycomputers.com: OK
            > reserved_resource_worker-3@pulp.olearycomputers.com: OK
    

Process:

  1. Install server
  2. Install admin client on server
  3. Install consumer client on server.
  4. install consumer client on additional clients.

Software Installation:

Once again, following the directions at http://docs.pulpproject.org/user-guide/installation/f23-.html for rhel7.

  1. Install repos on both clients and server:

  2. Create filesystems for /var/lib/mysql and /var/lib/mongodb:

    # pvcreate /dev/vdb
      Physical volume "/dev/vdb" successfully created.
    # vgextend vg00 /dev/vdb
      Volume group "vg00" successfully extended
    # lvcreate -L 20g -n mongdb vg00
      Logical volume "mongdb" created.
    # lvcreate -L 20g -n pulp vg00
      Logical volume "pulp" created.
    # mkfs.xfs /dev/vg00/mongdb
    # mkfs.xfs /dev/vg00/pulp
    # vi /etc/fstab
    # mkdir -p -m 755 /var/lib/{mongodb,pulp}
    # mount /var/lib/mongodb
    # mount /var/lib/pulp
    
  3. Install software per directions:

    • mongodb. Starting mongod will take some time.
    • qpid
    • pulp server
    • pulp extras
  4. Services that must be enabled/started. Note: follow the directions. There are tasks between enabling/starting some of these processes.

    • mongodb
    • qpidd
    • httpd
    • pulp_workers
    • pulp_celerybeat
    • pulp_resource_manager
  5. Install admin client. Doesn’t have to be server or consumer.

    • Install required repos if not already done:

      • epel (yum)
      • rhel-pulp.repo (wget)
    • Install packages:

      yum install pulp-admin-client \
          pulp-rpm-admin-extensions \
          pulp-puppet-admin-extensions \
          pulp-docker-admin-extensions
      
    • Update hostnane in /etc/pulp/admin/admin.conf:

      [server]
      host: pulp.olearycomputers.com
      
  6. Install pulp consumer on all servers that will be consumers:

    • Install required repos if not already done:

      • rhel-pulp.repo (OS version specific)
      • epel (yum) yum install epel-release
      • qpid (el6)
    • Install packages:

      yum install pulp-consumer-client \
          pulp-rpm-consumer-extensions \
          pulp-puppet-consumer-extensions \
          pulp-agent pulp-rpm-handlers \
          pulp-rpm-yumplugins \
          pulp-puppet-handlers \
          python-gofer-qpid
      
    • Update hostname in /etc/pulp/consumer/consumer.conf

    • Enable/start the service:

      chkconfig goferd on service goferd start

  7. In my little lab environment, disable ssl checking by adding verify_ssl: False to server section of /etc/pulp/admin/admin.conf

  8. Create an administrative account and enable it:

    # pulp-admin login -u ${root} -p ${root_pwd}
    # pulp-admin auth user create  --login dkoleary --password 'no-me-pwd'
    # pulp-admin auth role user add --role-id super-users --login dkoleary
    # mkdir -p -m 700 ~/.pulp
    # vi ~/.pulp/admin.conf
    # cat ~/.pulp/admin.conf
    # cat admin.conf
    [auth]
    username: dkoleary
    password: no-me-pwd
    
    [server]
    host: pulp.olearycomputers.com
    verify_ssl: False
    # pulp-admin logout
    # chmod 600 ~/.pulp/admin.conf
    # pulp-admin auth user list
    
  9. Open required firewall ports on the pulp server

  10. Create and sync repos.

Admin Client:

  • LL on ~/.pulp/admin.conf
  • Users:
    • Create users, roles, etc. Examples in command section.
    • Permissions. Create, read, update, delete, execute. Can be set on individual repos. Potentially useful.
    • Roles:
      • Default role: super-users
      • Rather than provide permissions to specific accounts, create roles which have permissions set, then add/delete users as needed.
      • Some users need to be in super-users to ensure maintainability:
# pulp-admin auth role user add --role-id super-users --login dkoleary

User [dkoleary] successfully added to role [super-users]

  • Groups: This will be the interesting one at work. Some interesting possiblilities just based on the reading:

    pulp-admin rpm consumer group package update \
        --name tzdata --consumer-group dev
    pulp-admin rpm consumer group update \
        --consumer-group qa
    
  • Repos:
    • rpm repo create doesn’t link a url with the pulp repo. Still need to find out how to do that.
    • I created centos6_base and centos6_updates for my testing.
    • I can group them together

Creating repo mirrors:

Using epel as the example:

  • Create the pulp repo:

    pulp-admin rpm repo create \
    --description 'Extra Packages for Enterprise Linux 7 - x86_64' \
    --display-name 'Extra Packages for Enterprise Linux 7 - x86_64' \
    --feed http://download.fedoraproject.org/pub/epel/7/x86_64 \
    --relative-url=epel_el7_x86_64 \
    --repo-id=epel_el7_x86_64 \
    --remove-missing=true \
    --retain-old-count=2 \
    --repoview=true
    
  • Run the initial sync and set a schedule:

    # pulp-admin rpm repo sync run --repo-id epel_el7_x86_64
    
  • Set a schedule for resyncing:

    # Sched=$(date -d "2AM tomorrow" +"%FT%T-600/P1D")
    # echo ${Sched}
    2017-12-22T02:00:00-600/P1D
    # pulp-admin rpm repo sync schedules create \
    --schedule ${Sched} --repo-id epel_el7_x86_64
    

Follow the same process for any other repo. Watch space utilization in /var/lib/pulp. Epel alone took upwards of 13 gigs. A good place to start for CentOS 7:

#----------------------------------
name='CentOS-7 (x86_64) - Base'
baseurl=http://mirror.centos.org/centos/7/os/x86_64/
#----------------------------------
name='CentOS-7 (x86_64) - Updates'
baseurl=http://mirror.centos.org/centos/7/updates/x86_64/
#----------------------------------
name='CentOS-7 (x86_64) - Extras'
baseurl=http://mirror.centos.org/centos/7/extras/x86_64/
#----------------------------------
name='CentOS-7 (x86_64) - Plus'
baseurl=http://mirror.centos.org/centos/7/centosplus/x86_64/

and for CentOS 6:

#----------------------------------
name='CentOS-6 (x86_64) - Base'
baseurl=http://mirror.centos.org/centos/6/os/x86_64/
#----------------------------------
name='CentOS-6 (x86_64) - Updates'
baseurl=http://mirror.centos.org/centos/6/updates/x86_64/
#----------------------------------
name='CentOS-6 (x86_64) - Extras'
baseurl=http://mirror.centos.org/centos/6/extras/x86_64/
#----------------------------------
name='CentOS-6 (x86_64) - Plus'
baseurl=http://mirror.centos.org/centos/6/centosplus/x86_64/
#----------------------------------
name='CentOS-6 (x86_64) - Contrib'
baseurl=http://mirror.centos.org/centos/6/contrib/x86_64/

Repositories:

  • Add –feed=${url} to link a pulp repo to an external one.
  • Process:
    • Create the repo
    • Sync the repo
    • register consumers
    • Bind repos: