Help with SQL: attribute update/insert from local table

I have some trepidation about asking for critical review in a public forum, but here goes.

We have certain attributes which require periodic updating for most of the individual constituent records in our database (about 140,000).  We have stored procedures which create the new values and place them in local tables.  Deleting all of these attributes and recreating them seemed wasteful and burdensome for the database, so I attempted to write a procedure that marches through the local table, compares the fresh value to the one currently stored as an attribute, and chooses to either:

  1. insert a new attribute (if none exists),
  2. update an existing attribute with a fresh value, or
  3. leave it alone (if the fresh value is identical to the old one)

Writing loops in SQL is not my forte, but I produced something that seems to work.  My concern is that, running it in Test last night, it took 5.5 hours, and that was for one attribute out of eight!  If you have the skill and the time, please see below and tell me what I could do better.

Each local table has a field named seq that is sequentially numbered starting at 1.

USE [impresario]
GO
declare @seq int;
declare @maxseq int;
declare @customer_no int;
declare @keyword_no int;
set @keyword_no = 554
set @seq = 1
set @maxseq = (select max(seq) from lt_insight_final_mg_scores)
while @seq <= @maxseq
BEGIN
begin tran
set @customer_no = (select a.ID from lt_insight_final_mg_scores a where @seq = a.seq)
if not exists
(Select * from TX_CUST_KEYWORD k1
join lt_insight_final_mg_scores a1
on a1.ID = k1.customer_no
where k1.keyword_no = @keyword_no and k1.customer_no = @customer_no)
begin
insert into [dbo].[TX_CUST_KEYWORD] (customer_no, keyword_no, key_value)
values (@customer_no, @keyword_no,
(Select a1.MG_Score from lt_insight_final_mg_scores a1 where a1.ID = @customer_no))
end
if exists
(Select * from lt_insight_final_mg_scores a2
join TX_CUST_KEYWORD k2
on a2.ID = k2.customer_no
where a2.ID = @customer_no and k2.keyword_no = @keyword_no and a2.mg_score <> k2.key_value)
begin
update [dbo].[TX_CUST_KEYWORD]
set key_value = (Select a2.MG_Score from lt_insight_final_mg_scores a2 where a2.ID = @customer_no)
where customer_no = @customer_no and keyword_no = @keyword_no
end
commit tran
set @seq = @seq + 1;
END

Parents
  • Hi Clarke -

    I'll try to take a deeper look at this later but I think you're causing a lot of spin in these two statements (part of your 'exists/not exists' logic):

    SELECT *
    FROM TX_CUST_KEYWORD k1...

    If you're only checking if the record is there already, you only need to return a single value here, like this:

    SELECT 1
    FROM TX_CUST_KEYWORD k1...

    The way you have it now, you'll return all of the rows in 'TX_CUST_KEYWORD' and all of the rows in 'lt_insight_final_mg_scores'.  I don't have that local table in my system (of course) but the multiplication of the number of rows from each table must be pretty big, I imagine.  When checking for existence, you should be safe with 'SELECT 1'.  HTH.

Reply
  • Hi Clarke -

    I'll try to take a deeper look at this later but I think you're causing a lot of spin in these two statements (part of your 'exists/not exists' logic):

    SELECT *
    FROM TX_CUST_KEYWORD k1...

    If you're only checking if the record is there already, you only need to return a single value here, like this:

    SELECT 1
    FROM TX_CUST_KEYWORD k1...

    The way you have it now, you'll return all of the rows in 'TX_CUST_KEYWORD' and all of the rows in 'lt_insight_final_mg_scores'.  I don't have that local table in my system (of course) but the multiplication of the number of rows from each table must be pretty big, I imagine.  When checking for existence, you should be safe with 'SELECT 1'.  HTH.

Children