Scrubbing copy-down of TEST Data

Hello,

We'd like to scrub our copy-down so that it no longer contains personal information. We don't want to lose data as it is good for training and development; however, we want the personal information to be removed. Are there any conversion scripts we can utlize.

Thank you,

Lou Ann

  • I think I've had a conversation about this with TN staff before, and there was a suggestion that there could be some existing scripts (internal to the network, perhaps the RAMP team) that might be made available... but I don't think I heard anything further than that.

  • Since our Test environment has pretty much the same security as our Live environment, we only make limited modifications to it.  This also helps users doing testing, allowing them to easily find accounts that they know have interesting features for whatever it is that they are testing.

    We did have an elaborate script for mangling PII in our Development environment, when we had one.  We moved to RAMP at the same time that we upgraded to v11, and never wound up setting up a Dev after that, so the scripts I have are hopelessly out-of-date (no household handling, for instance).  I could dig them up, just to give a sense of all the tables I found PII or potential PII in (back then).

    I do have a set of scripts that I run whenever we do a Live->to->Test copy which make a bunch of changes.  Mostly these are to configuration variables (so that, for instance, confirmation emails sent from the Test environment are sent from different addresses than they are in Live), TNEW configuration is updated, etc.  I also do two things to customer accounts.  First, I mangle all of the login passwords in Test (except for customers marked with a "Testing Account" Constituency).  Second, to prevent testers from accidentally emailing customers, I mangle all email addresses (again, save those marked for testing).  Since our email service is from Google, I take the previous email address, replace the special characters with periods, then add them after a "+" to an internal IT mailing list email address, so that any actual mailing of a customer will come to us for review.

  • Thanks for replies and the offer Gawain. Having a Testing constituency or attribute is a good idea. For a procedure I was thinking:

    1) stripping login, emails, and phone

    2) modifying everyone's address to be same fake one

    3) mangling names and salutations

    4) running credit card deletion utility (not worrying about pledges as cc's are tokenized)

  • Depends on your usage: you'll not be able to run operations in Test that look at logins and emails, or anything where you are looking at different addresses in bulk (i.e. reporting on zip code).  I guess I think that wiping those things completely could make it harder to build and test reports, new criteria or utilities.  That's why I generally favored mangling, and in the case of addresses mangled out unique addresses and kept the City/Zip untouched.

    Are you a RAMP client?  Your credit cards will be wiped automatically.  This will also, I'm pretty sure, wipe out tokens, including tokens used by pledges.

    For the dev mangling we used Customer Number, so

    First Name: F[Customer #]

    Last Name: L[Customer #]

    Street 1: S1[Customer #]

    etc.