Doing bulk IPA operations from the command-line can be inefficient because each command requires a round trip. So a loop like this can be rather slow:

for line in $(cat /etc/passwd); do
        IFS=' '
        username=$(echo $line|cut -f1 -d:)
        password=$(echo $line|cut -f2 -d:)
        uid=$(echo $line|cut -f3 -d:)  
        gid=$(echo $line|cut -f4 -d:)
        ...
        ipa user-add $username --first=NIS --last=USER --password --gidnumber=$gid --uid=$uid --gecos=$gecos --homedir=$homedir --shell=$shell --setattr userpassword={crypt}$password
done

There is a round trip for every user.

The obvious way to improve this is to reduce the number of round trips by using the IPA batch command. Here is the skeleton of a program to read /etc/passwd. It lacks a whole ton of error checking and may be filled with errors but it should illustrate how the batch command works.

This will batch the creation of 50 users at a time.

from ipalib import api
from ipalib import errors
import sys


def add_batch_operation(command, *args, **kw):
    batch_args.append({
        "method": command,
        "params": [args, kw],
    })


def flush_batch_operation():
    if not batch_args:
        return None

    kw = {}

    try:
        return api.Command['batch'](*batch_args, **kw)
    except errors.CCacheError as e:
        print(e)
        sys.exit(1)


api.bootstrap(context='batch')
api.finalize()
api.Backend.rpcclient.connect()

lineno = 0
batch_args = 0
count = 0
batch_args = list()
with open("/etc/passwd", "r") as passwd:
    for line in passwd:
        lineno += 1
        try:
            (login, password, uid, gid, gecos, homedir, shell) = \
                line.strip().split(':')
        except ValueError as ve:
            print("Malformed line %d: %s" % (lineno, ve))

        if gecos:
            try:
                first, last = gecos.split(' ', 1)
            except ValueError:
                print("Unable to parse gecos line %d" % lineno)
        else:
            print("Missing gecos line %d" % lineno)

        params = [login]
        kw = {
            'givenname': first,
            'sn': last,
            'cn': gecos,
            'userpassword': '{crypt}' + password,
            'gecos': gecos,
            'homedirectory': homedir,
            'loginshell': shell,
        }

        add_batch_operation('user_add', *params, **kw)
        count += 1

        if count % 50 == 0:
            print("%d entries" % count)
            results = flush_batch_operation()
            for result in results.get('results'):
                if result.get('error') != None:
                    print(result.get('error'))
            batch_args = list()

flush_batch_operation()
Advertisements

certmonger CA subsystem renewal

The CA subsystem certificate (OCSP, Audit, etc) are renewed directly against dogtag rather than being processed through IPA like the Apache and 389-ds server certificates are.

certmonger does the renewa by issuing a request like this:

GET /ca/ee/ca/profileSubmit?profileId=caServerCert&serial_num=5&renewal=true&xml=true&requestor_name=IPA

The serial number value comes from the current certificate being tracked by certmonger. Dogtag will generate its own CSR based on the template values currently in LDAP, cn=5,ou=ca,ou=requests,o=ipaca

Migration and User Private Groups

When adding an IPA user they are typically created with a User-Private Group (UPG). This is a group of the same name, with the same GID. It is treated specially in that it cannot have members and does not typically appears in group searches using the IPA API (unless the private option is included).

Migrating from another LDAP source, including another IPA server, does not create UPGs. There are a number of reasons for this:

  1. It can be expensive to be sure that no groups reference any given user
  2. What to do if one group cannot be made into a UPG.  There is no interactive mode so it is either skip it, add it as a non-UPG or drop the group members and add it as a UPG.

We took the easy way out and don’t convert any. There is an RFE to be able to do this during migration.

This came up recently on the freeipa-users list and I thought about what it would take to convert a group back into a UPG.

My first solution was a rather compex set of ldapmodify operations.

One to update the group:

$ kinit admin
$ ldapmodify -Y GSSAPI
dn: cn=test,cn=groups,cn=accounts,dc=example,dc=com
changetype: modify
add: objectclass
objectClass: mepManagedEntry
-
add: mepManagedBy
mepManagedBy: uid=test,cn=users,cn=accounts,dc=example,dc=com
- 
delete: objectclass
objectClass: ipausergroup
-
delete: objectclass
objectClass: groupofnames
-
delete: objectclass
objectClass: nestedgroup

^D

And one to update the user:

$ ldapmodify -Y GSSAPI
dn: uid=test,cn=users,cn=accounts,dc=example,dc=com
changetype: modify
add: objectclass
objectClass: mepOriginEntry
-
add: mepManagedEntry
mepManagedEntry: cn=test,cn=groups,cn=accounts,dc=example,dc=com

^D

This seemed cumbersome especially if there are a lot of groups to convert. It also doesn’t consider the case where a group has a member so is nonconvertible. An ObjectclassViolation LDAP error would be thrown in that case.

So I poked at the group-detach command and came up with this. If you drop this file, attach.py, into /usr/lib/python-*/site-packages/ipaserver/plugins and restart Apache you’ll have the group-attach command:

import six

from ipalib import Str
from ipalib.plugable import Registry
from .baseldap import (
    pkey_to_value,
    LDAPQuery
)
from ipalib import _, ngettext
from ipalib import errors
from ipalib import output

register = Registry()


@register()
class group_attach(LDAPQuery):
    __doc__ = _('Attach a managed group to a user.')

    takes_parms = (
        Str('user',
            doc=_('User to attach group to'),
            flags=['no_create', 'no_update', 'no_search'],
        )
    )

    has_output = output.standard_value
    msg_summary = _('Attached group "%(value)s" to user "%(value)s"')

    def execute(self, *keys, **options):
        """
        This requires updating both the user and the group. We first need to
        verify that both the user and group can be updated, then we go
        about our work. We don't want a situation where only the user or
        group can be modified and we're left in a bad state.
        """
        ldap = self.obj.backend

        group_dn = self.obj.get_dn(*keys, **options)
        user_dn = self.api.Object['user'].get_dn(*keys)

        try:
            user_attrs = ldap.get_entry(user_dn)
        except errors.NotFound:
            raise self.obj.handle_not_found(*keys)
        is_managed = self.obj.has_objectclass(
            user_attrs['objectclass'], 'mepmanagedentry'
        )
        if (not ldap.can_write(user_dn, "objectclass") or
                not ldap.can_write(user_dn, "mepManagedEntry")
                and is_managed):
            raise errors.ACIError(
                info=_('not allowed to modify user entries')
            )

        group_attrs = ldap.get_entry(group_dn)
        is_managed = self.obj.has_objectclass(
            group_attrs['objectclass'], 'mepmanagedby'
        )
        if (not ldap.can_write(group_dn, "objectclass") or
                not ldap.can_write(group_dn, "mepManagedBy")
                and is_managed):
            raise errors.ACIError(
                info=_('not allowed to modify group entries')
            )

        objectclasses = user_attrs['objectclass']
        if 'meporiginentry' in [x.lower() for x in objectclasses]:
            raise errors.ACIError(
                info=_('The user is already attached to a group')
            )

        group_attrs = ldap.get_entry(group_dn)
        objectclasses = group_attrs['objectclass']
        if 'mepmanagedentry' in [x.lower() for x in objectclasses]:
            raise errors.ACIError(
                info=_('The group is already managed')
            )
        if group_attrs.get('member'):
            raise errors.ACIError(
                info=_('The group has members')
            )

        for objectclass in ('ipausergroup', 'groupofnames', 'nestedgroup'):
            try:
                i = objectclasses.index(objectclass)
            except ValueError:
                # this should never happen
                pass
            del objectclasses[i]

        objectclasses.append('mepManagedEntry')

        group_attrs['mepManagedBy'] = user_dn
        group_attrs['objectclass'] = objectclasses
        ldap.update_entry(group_attrs)

        try:
            user_attrs['objectclass'].append('mepOriginEntry')
            user_attrs['mepManagedEntry'] = group_dn
            ldap.update_entry(user_attrs)
        except ValueError:
            # Somehow the user isn't managed, let it pass for now. We'll
            # let the group throw "Not managed".
            pass

        return dict(
            result=True,
            value=pkey_to_value(keys[0], options),
        )

This leaves some things to be desired, notably the exceptions are ACIError rather than something perhaps more relevant.

$ ipa group-detach test
--------------------------------------
Detached group "test" from user "test"
--------------------------------------

$ ipa group-attach test
------------------------------------
Attached group "test" to user "test"
------------------------------------

There be dragons. I have barely tested this, just enough to scratch the itch of my curiosity.

Developers should learn to love the IPA lite-server

If you’re trying to debug an issue in a plugin then the lite-server is for you. It has a number of advantages:

  • It runs in-tree which means you don’t need to commit, build code, re-install, etc
  • Or worse, avoid directly editing files in /usr/lib/python3.6/*
  • It is very pdb friendly
  • Auto-reloads modified python code
  • It doesn’t run as root

You’ll need two sessions to your IPA master. In one you run the lite-server via:

$ export KRB5CCNAME=~/.ipa/ccache
$ kinit admin
$ make lite-server

In the second we run the client. You’ll need to say which configuration to use:

$ export IPA_CONFDIR=~/.ipa

Now copy the installed configuration there:

$ cp /etc/ipa/default.conf ~/.ipa
$ cp /etc/ipa/ca.crt ~/.ipa

Edit ~/.ipa/default.conf and change the xmlrpc_uri to:

http://localhost:8888/ipa/xml

Now you can run your command locally:

$ kinit admin
$ PYTHONPATH=. python3 ./ipa user-show admin

And if something isn’t working right, stick pdb in ipaserver/plugins/user.py in the show pre_callback¬† command and re-run (notice that the lite-server picks up the change automatically):

$ PYTHONPATH=. python3 ./ipa user-show admin

And in the lite-server session:

> /home/rcrit/redhat/freeipa/ipaserver/plugins/user.py(858)pre_callback()
-> return dn
(Pdb)

 

certmonger D-Bus introspection

I’m looking to do some certificate work related to certmonger and was thinking D-Bus would be a good way to get the data (freeIPA does something similar). The Using D-Bus Introspection blog post was key for me to figure out what certmonger could provide (without digging too much into the code).

I ended up running:

dbus-send --system --dest=org.fedorahosted.certmonger \
--type=method_call --print-reply \
/org/fedorahosted/certmonger \
org.freedesktop.DBus.Introspectable.Introspect

This provided me the list of interfaces I needed. First I started with getting the current requests:

dbus-send --system --dest=org.fedorahosted.certmonger \
--type=method_call --print-reply \
/org/fedorahosted/certmonger \
org.fedorahosted.certmonger.get_requests

Then you can pick or iterate through the requests to get the information you want. Here is how to get the serial number:

dbus-send --system --dest=org.fedorahosted.certmonger \
--type=method_call --print-reply \
/org/fedorahosted/certmonger/requests/Request1 \
org.freedesktop.DBus.Properties.Get \
string:org.fedorahosted.certmonger.request string:serial

You can find a list of possible values in src/tdbus.h

Peer certificate cannot be authenticated with given CA certificates

I’ve seen this twice now so I’m documenting the fix so I don’t have to go hunting again.

Basically it starts as a typical “oh crap my certs expired” question on #freeipa or freeipa-users. Sadly the usual things don’t seem to help (go back in time).

The last time this happened there was the added twist that the renewal master was gone so we had to first reconfigure a replica to do the renewal (you do have more than one CA right? RIGHT?)

Anyway, he would persistently get the Peer certificate cannot be authenticated message. We tried:

  • Confirming that ipaCert was the correct value in the IPA RA entry in LDAP
  • The CA is up and running: curl --cacert /etc/ipa/ca.crt -v https://`hostname`:8443/ca/ee/ca/getCertChain
  • Ensuring that certutil -L -d /etc/pki/pki-tomcat/alias-n 'caSigningCert cert-pki-ca' -a and cat /etc/ipa/ca.crt are the same cert

Then I remembered a fellow Red Hatter had reported a similar issue and discovered that the fix was to reset the NSS trust flags in the Apache NSS database (which certmonger uses).

# SSL_DIR=/etc/httpd/alias/ curl -v -o /dev/null --cacert /etc/ipa/ca.crt https://`hostname`:8443/ca/agent/ca/profileReview

You should get client certificate not found. If you don’t then try this:

# certutil -M -d /etc/httpd/alias -n 'EXAMPLE.COM IPA CA' -t ,,
# certutil -M -d /etc/httpd/alias -n 'EXAMPLE.COM IPA CA' -t CT,C,C

See https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahosted.org/thread/XSMWWPJU2VRUIGE6SRAHYAJF7BYBCNOE/

Setting up AD for winsync testing

It had literally been years since I had to setup an AD test environment to do basic winsync testing. I found some scraggly notes and decided to transcribe them here for posterity. They were written for AD 2003 and things for 2008 are a bit different but I still found it fairly easy to figure out (in 2008 there is less need to go to the Start menu).

I don’t in fact remember what a lot of these notes do so don’t kill the messenger.

Start with an AD 2008 instance by following http://www.freeipa.org/page/Setting_up_Active_Directory_domain_for_testing_purposes

Once that is booted:

  1. Change the hostname
  2. My Computer -> right click -> Properties -> Computer Name -> Change = win2003
  3. REBOOT
  4. Manage your Server
    1. Add or remove a role -> Next [Preliminary Steps]
    2. Custom -> Domain Controller
    3. Domain controller for a new domain
    4. Domain in a new forest
    5. Fill DNS name for new domain: example.com
  5. If conflict select Install and Configure DNS on this server
  6. REBOOT
  7. Start -> Control Panel -> Add or Remove Programs
    1. Add/Remove Windows Components
    2. Certificate Services, yes to the question
    3. Next
    4. Enterprise root CA
    5. AD CA for the common name
    6. Accept other defaults
    7. Ok about IIS
  8. REBOOT (or wait a little while for certs to issue)
  9. Start -> Admin Tools -> Certificate Authority
    1. Certificate Authority -> AD CA -> Issued Certificates
    2. Select the cert, double click
    3. Certificate Path
    4. Select AD CA, view certificate
    5. Details
    6. Copy to file
    7. Base 64-encoded x509 (.cer)
  10. Install WinSCP
  11. Copy cert to IPA

Now on the IPA master the agreement can be created:

# ipa-replica-manage connect win2003.example.com –winsync –cacert=/home/rcrit/adca.cer -v –no-lookup –binddn ‘cn=administrator,cn=users,dc=example,dc=com’ –bindpw <AD pw> –passsync <something>

As I recall I tended to put the AD hostname into /etc/hosts (hence the –no-lookup).

Frustrated rantings of a developer