• Replies 11 replies
  • Subscribers 341 subscribers
  • Views 421 views
  • Users 0 members are here
Related

Using SQL to clean up CSI data

Brandi Cline
$organization

Has anyone had any luck using SQL to clean up CSI data that comes from TNEW custom forms?

We are using custom form data to collect information on our travel program participants when they register for a trip, which is great!  However, when you run a report on CSIs and export it, all the collected data comes across as one cell in excel.  I've used column splits in power query to separate everything out, but we'd love for the data to feed directly into a tableau, rather than being exported and then power queried.  I'm pretty new to SQL and completely self taught, so I didn't know if there was some way to make SQL do what the power query is doing.  The folks actually looking at and using the data don't use power query and only barely get into Tess for anything, so they currently can't get at the data themselves and certainly can't get it into a usable format.

Any advice would be appreciated! Slight smile

Thanks!

Parents
  • $organization

    TNEW has a Carriage Return (char(13)) as the delimiter 

    SELECT value
    FROM STRING_SPLIT((select notes from t_cust_activity where activity_no = 18300), char(13))

    took my entry that looks like 

    Contact_details: HEATH WILDER PO BOX 1092 BROADWAY, NSW (227) 2007 Phone_num: +61414310104 Dietary_req: test Comments: Line Item ID: 44567 Customer Number: 183407 Order Number: 43360 

    and turns it into ...

    Contact_details: HEATH WILDER PO BOX 1092 BROADWAY, NSW (227) 2007
    Phone_num: +61414310104
    Dietary_req: test
    Comments:
    Line Item ID: 44567
    Customer Number: 183407
    Order Number: 43360

    Hope that helps

    (That's an old address for anyone who is planning on sending me a cake)

  • $organization in reply to Heath Wilder

    Have you figured out a simple way to pivot a whole bunch of CSI’s up onto a single lives per csi?

  • $organization in reply to Heath Wilder

    I've made a local a local table-valued function that uses FT_SPLIT_LIST to also output the activity_no as well as ElementID and Element fields. You just use a cursor to move down and select the activity_no from T_CUST_ACTIVITY using a filter like activity_type, for example.

    With this local function, I can join it to itself as often as there are ElementID rows per activity_no, each join is a different ElementID. While this could be placed in a view, we have so many so I'm opting for a nightly procedure to populate a local table, instead. This will join back to T_CUST_ACTIVITY to tie back to customer_no. 

    I've done what Nick suggested at a past organisation, i.e. populate a table instead of using CSIs, however, that was for a brand new project. Another consideration is you can only pass up to 32 parameters (populate up to 32 fields) unless that has changed recently.

Reply
  • $organization in reply to Heath Wilder

    I've made a local a local table-valued function that uses FT_SPLIT_LIST to also output the activity_no as well as ElementID and Element fields. You just use a cursor to move down and select the activity_no from T_CUST_ACTIVITY using a filter like activity_type, for example.

    With this local function, I can join it to itself as often as there are ElementID rows per activity_no, each join is a different ElementID. While this could be placed in a view, we have so many so I'm opting for a nightly procedure to populate a local table, instead. This will join back to T_CUST_ACTIVITY to tie back to customer_no. 

    I've done what Nick suggested at a past organisation, i.e. populate a table instead of using CSIs, however, that was for a brand new project. Another consideration is you can only pass up to 32 parameters (populate up to 32 fields) unless that has changed recently.

Children