Help with SQL: attribute update/insert from local table

I have some trepidation about asking for critical review in a public forum, but here goes.

We have certain attributes which require periodic updating for most of the individual constituent records in our database (about 140,000).  We have stored procedures which create the new values and place them in local tables.  Deleting all of these attributes and recreating them seemed wasteful and burdensome for the database, so I attempted to write a procedure that marches through the local table, compares the fresh value to the one currently stored as an attribute, and chooses to either:

  1. insert a new attribute (if none exists),
  2. update an existing attribute with a fresh value, or
  3. leave it alone (if the fresh value is identical to the old one)

Writing loops in SQL is not my forte, but I produced something that seems to work.  My concern is that, running it in Test last night, it took 5.5 hours, and that was for one attribute out of eight!  If you have the skill and the time, please see below and tell me what I could do better.

Each local table has a field named seq that is sequentially numbered starting at 1.

USE [impresario]
GO
declare @seq int;
declare @maxseq int;
declare @customer_no int;
declare @keyword_no int;
set @keyword_no = 554
set @seq = 1
set @maxseq = (select max(seq) from lt_insight_final_mg_scores)
while @seq <= @maxseq
BEGIN
begin tran
set @customer_no = (select a.ID from lt_insight_final_mg_scores a where @seq = a.seq)
if not exists
(Select * from TX_CUST_KEYWORD k1
join lt_insight_final_mg_scores a1
on a1.ID = k1.customer_no
where k1.keyword_no = @keyword_no and k1.customer_no = @customer_no)
begin
insert into [dbo].[TX_CUST_KEYWORD] (customer_no, keyword_no, key_value)
values (@customer_no, @keyword_no,
(Select a1.MG_Score from lt_insight_final_mg_scores a1 where a1.ID = @customer_no))
end
if exists
(Select * from lt_insight_final_mg_scores a2
join TX_CUST_KEYWORD k2
on a2.ID = k2.customer_no
where a2.ID = @customer_no and k2.keyword_no = @keyword_no and a2.mg_score <> k2.key_value)
begin
update [dbo].[TX_CUST_KEYWORD]
set key_value = (Select a2.MG_Score from lt_insight_final_mg_scores a2 where a2.ID = @customer_no)
where customer_no = @customer_no and keyword_no = @keyword_no
end
commit tran
set @seq = @seq + 1;
END

Parents
  • Former Member
    Former Member $organization

    Hi Clarke,

    Try this code.  MERGE is much more efficient for this type of scenario when you want to update/insert/delete depending on the data. This works on the tables as a SET not as RBAR. I don't have your table so I can't test the code. Let me know if you have any questions.

    Help with using MERGE - https://www.mssqltips.com/sqlservertip/1704/using-merge-in-sql-server-to-insert-update-and-delete-at-the-same-time/.

    USE [impresario]
    GO

    DECLARE @keyword_no INT = 554;

    MERGE dbo.TX_CUST_KEYWORD  ckey
    USING LT_INSIGHT_FINAL_MG_SCORES fscr
        ON ckey.keyword_no = @keyword_no AND ckey.customer_no = fscr.customer_no
    WHEN MATCHED AND ckey.key_value <> fscr.MG_Score
        THEN UPDATE SET ckey.key_value = fscr.MG_Score
    WHEN NOT MATCHED BY ckey
        THEN INSERT (customer_no, keyword_no, key_value)
        VALUES (fscr.ID, @keyword_no, fscr.MG_Score

Reply
  • Former Member
    Former Member $organization

    Hi Clarke,

    Try this code.  MERGE is much more efficient for this type of scenario when you want to update/insert/delete depending on the data. This works on the tables as a SET not as RBAR. I don't have your table so I can't test the code. Let me know if you have any questions.

    Help with using MERGE - https://www.mssqltips.com/sqlservertip/1704/using-merge-in-sql-server-to-insert-update-and-delete-at-the-same-time/.

    USE [impresario]
    GO

    DECLARE @keyword_no INT = 554;

    MERGE dbo.TX_CUST_KEYWORD  ckey
    USING LT_INSIGHT_FINAL_MG_SCORES fscr
        ON ckey.keyword_no = @keyword_no AND ckey.customer_no = fscr.customer_no
    WHEN MATCHED AND ckey.key_value <> fscr.MG_Score
        THEN UPDATE SET ckey.key_value = fscr.MG_Score
    WHEN NOT MATCHED BY ckey
        THEN INSERT (customer_no, keyword_no, key_value)
        VALUES (fscr.ID, @keyword_no, fscr.MG_Score

Children
  • I will try merge, and will let you know how it goes.  Thanks, Debbie.

  • Wow, that is pretty slick! And it finished in under a minute!!  Thanks again, Debbie.

    I added a delete clause, since I can imagine that being useful someday, and discovered that "WHEN NOT MATCHED BY" expects either TARGET or SOURCE, not an alias for the target or the source.

    USE [impresario]
    GO

    DECLARE @keyword_no INT = 554;

    MERGE dbo.TX_CUST_KEYWORD as k1
    USING LT_INSIGHT_FINAL_MG_SCORES as a1
    ON (k1.keyword_no = @keyword_no AND k1.customer_no = a1.ID)
    WHEN MATCHED AND k1.key_value <> a1.MG_Score THEN
    UPDATE SET k1.key_value = a1.MG_Score
    WHEN NOT MATCHED BY TARGET THEN
    INSERT (customer_no, keyword_no, key_value)
    VALUES (a1.ID, @keyword_no, a1.MG_Score)
    WHEN NOT MATCHED BY SOURCE THEN
    DELETE;
    GO