Duplicate Management Resources

Hi Brian,

Just watched the Skill Builder, which was very apropos as our nascent consortium had just begun discussing duplicate management.

When previously we were a single organization, we built out some pretty complicated duplicate management add-ons: with the help of Consulting Services we build an "auto-merge" process, and I have padded that with tools to allow our users to prioritize potential duplicates of important customers.

One thing I wanted to offer: there's a question in the Skill Builder about cases where you know that two customers are not the same, but the parameters for duplicate identification keep finding them.  We had a number of actual cases of this, and I fixed it by creating an Attribute that could be placed on one or both records that allowed you to enter the customer number of the other record.  I then have a procedure that I run after I run the standard identify potential duplicates procedure the finds those matches and deletes them from T_POTENTIAL_DUPS.  The procedure that does this is also what I use to pare down that list to just high-priority customers (mostly based on Constituency) so that those duplicates are attended to first, and also so that the users are not overwhelmed by the sheer number.

Currently we are keeping the constituents for our two organizations distinct; I don't know how many other consortia do this.  I will be adding a new component to my current procedure (for trimming T_POTENTIAL_DUPS) to identify if the keep and delete are from different organizations, and will remove those matches as well.

--Gawain

I'm fascinated about your work regarding circumstances where you know that two records tagged as potential duplicates are not the same; I know that Mindee Waltz (Past Member) has mentioned the prevalence of multiple records with the same name (due to fathers naming their sons as Jr., II, III, IV, etc.) in the past, so I bet there would be some interest in further discussion among this group.

With regard to the question of constituents being distinct between organizations, our situation in Tulsa is such that, with eight organizations in the shared database (and a relatively small overall patron-base), we opt to merge records when we know that the duplicate is legitimate versus allowing for multiple "versions" of the same patron, used individually by our member organizations. It does introduce some additional questions with regard to how our organizations can maintain their preferred point of contact with a shared record, especially when that contact point is not tagged as "primary" for the record.

Wow, lost my whole reply four times.  I'll try this again in a text editor, then copy and paste.

So, I'm looking at my procedure, and boy is it over-engineered (four sub-procedures!), but this is the heart of the "Not a Duplicate" screening:

I'm sorry, I can't insert the code.  If I try to do it via source it causes the form submission to fail, while the "insert code" button is still broken (if you click it the "Ok" button keeps scrolling off the page forever).

It's finding all the dups where one or the other customer account has a "Not a Duplicate" attribute that identifies the other one.  It then is putting it in a temporary table: you could just change this to be a straight delete, of course.  I tend to pre-pack operations into temporary tables so that I can choose to just output the temporary table for debugging/preview or instead us it in a join for the actual delete/insert/whatever.

I also check against the attribute before allowing merges.  This guards against manual merge submissions.

Remember that if you create an attribute like this you'll need to include updating it in your merge operations in LP_CONST_MERGE!

I feel the pain on reply issues here on the forums; I've chatted with Todd Lantry (he/him) about it in the past, so I'm tagging him in here for any insight/suggestions regarding the code insert.

We were strongly advised to just bung all the records together, as the core Tessitura functionality for managing separate constituencies for different organizations in a consortium is broken and Tessitura has no plans for fixing it.  Presumably there will be some kind of functionality replacement in v16, although I'm not sure how complete it will be.

However, being able to bring the new member on in a timely way meant keeping the records distinct so that they could do the onboarding work and get to the point where their users were trained enough and their management knowledgeable enough to be be able to handle shared records and to know what decisions to make about those complicating bits, like contact points.

Right now users just have to look for the org constituency and remember to not touch accounts from the other organization.  I'm currently just screening their customers altogether from our dup matching process, but we're working on updating that.  It will mean a duplicating the current auto-merge process, but with screens blocking our customers instead of theirs, and then a filter that will remove any potential dups between the organizations from T_POTENTIAL_DUPS.

Thanks for tagging me, Brian. Gawain Lavers I think you should be able to insert your code by pasting it in the rich text editor, without using the insert code or edit source options; that's one way we've found to get around the issues with those options. We have a task open to investigate the issue with the modal expanding off the screen (polls do that too, and have investigated it previously without luck), and putting code in the source editor triggers the WAF to think it's attempted code injection and it blocks it. We have made some adjustments and are continuing to work on that as well, while also maintaining security.

Another workaround is to attach the code file to your post, which would allow maintaining the code formatting. Thanks for letting us know and please be in touch with me if you find further issues posting.

Todd

Thanks Brian!