<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" media="screen" href="/~d/styles/rss2full.xsl"?><?xml-stylesheet type="text/css" media="screen" href="http://feeds.exceptionalgeeks.com/~d/styles/itemcontent.css"?><rss xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:sy="http://purl.org/rss/1.0/modules/syndication/" version="2.0">
<channel>
	<title>Comments for BI-Curious</title>
	
	<link>http://exceptionalgeeks.com/bi-curious</link>
	<description>...Business Intelligence and Other Curiosities</description>
	<lastBuildDate>Wed, 17 Mar 2010 21:29:50 +0000</lastBuildDate>
	<generator>http://wordpress.org/?v=2.9.2</generator>
	<sy:updatePeriod>hourly</sy:updatePeriod>
	<sy:updateFrequency>1</sy:updateFrequency>
		<atom10:link xmlns:atom10="http://www.w3.org/2005/Atom" rel="self" type="application/rss+xml" href="http://feeds.exceptionalgeeks.com/BI-Curious/Comments" /><feedburner:info xmlns:feedburner="http://rssnamespace.org/feedburner/ext/1.0" uri="bi-curious/comments" /><atom10:link xmlns:atom10="http://www.w3.org/2005/Atom" rel="hub" href="http://pubsubhubbub.appspot.com/" /><item>
		<title>Comment on ETL Pattern: Staged Refresh by Natalya</title>
		<link>http://exceptionalgeeks.com/bi-curious/2010/02/12/etl-pattern-staged-refresh/comment-page-1/#comment-931</link>
		<dc:creator>Natalya</dc:creator>
		<pubDate>Wed, 17 Mar 2010 21:29:50 +0000</pubDate>
		<guid isPermaLink="false">http://exceptionalgeeks.com/bi-curious/2010/02/12/etl-pattern-staged-refresh/#comment-931</guid>
		<description>Thank you for your response!</description>
		<content:encoded><![CDATA[<p>Thank you for your response!</p>
]]></content:encoded>
	</item>
	<item>
		<title>Comment on ETL Pattern: Staged Refresh by Pierre LaFromboise</title>
		<link>http://exceptionalgeeks.com/bi-curious/2010/02/12/etl-pattern-staged-refresh/comment-page-1/#comment-930</link>
		<dc:creator>Pierre LaFromboise</dc:creator>
		<pubDate>Wed, 17 Mar 2010 20:03:48 +0000</pubDate>
		<guid isPermaLink="false">http://exceptionalgeeks.com/bi-curious/2010/02/12/etl-pattern-staged-refresh/#comment-930</guid>
		<description>Ultimately, the solution needs to be tailored to the situation and there isn’t going to be a blanket answer. However, there are a few things I would immediately look at when evaluating it. First, I’d try to understand why a full load is being performed. Generally a data warehouse is there for historical data. If the data is historical, why is it being refreshed in full each time?

There are legitimate reasons for doing a complete reload but more often than not I find that it is due to lazy design. It is just easier to reload everything than it is to detect changes. Maybe, if I get some time I will do some posts on common change detection patterns for SSIS.

Back to the design that you describe: I’d worry about the down time and possibility of error resulting in an empty table for end users. In the design described in the blog post the data is fully loaded and verified on the target system before the existing data is dropped. And, finally, the staging table is simply re-named to replace the target table. All of these procedures are very quick and reduce the chance for error. It isn’t perfect, but it is fairly bulletproof.

In your situation, it sounds as though the data is loaded onto one server/database (DW_Staging), verified, and finally moved to the target server/database (DW). Even if the databases co-exist on the same server it is going to take time to move data. And, this moving takes place after you have deleted all of the existing data. So, I would argue that it isn’t as good of a solution.

Anyhow, like I said, the solution is going to depends on the details of your situation. But, this should at least give you something to think about. Thanks for the comment and good luck!</description>
		<content:encoded><![CDATA[<p>Ultimately, the solution needs to be tailored to the situation and there isn’t going to be a blanket answer. However, there are a few things I would immediately look at when evaluating it. First, I’d try to understand why a full load is being performed. Generally a data warehouse is there for historical data. If the data is historical, why is it being refreshed in full each time?</p>
<p>There are legitimate reasons for doing a complete reload but more often than not I find that it is due to lazy design. It is just easier to reload everything than it is to detect changes. Maybe, if I get some time I will do some posts on common change detection patterns for SSIS.</p>
<p>Back to the design that you describe: I’d worry about the down time and possibility of error resulting in an empty table for end users. In the design described in the blog post the data is fully loaded and verified on the target system before the existing data is dropped. And, finally, the staging table is simply re-named to replace the target table. All of these procedures are very quick and reduce the chance for error. It isn’t perfect, but it is fairly bulletproof.</p>
<p>In your situation, it sounds as though the data is loaded onto one server/database (DW_Staging), verified, and finally moved to the target server/database (DW). Even if the databases co-exist on the same server it is going to take time to move data. And, this moving takes place after you have deleted all of the existing data. So, I would argue that it isn’t as good of a solution.</p>
<p>Anyhow, like I said, the solution is going to depends on the details of your situation. But, this should at least give you something to think about. Thanks for the comment and good luck!</p>
]]></content:encoded>
	</item>
	<item>
		<title>Comment on ETL Pattern: Staged Refresh by Natalya</title>
		<link>http://exceptionalgeeks.com/bi-curious/2010/02/12/etl-pattern-staged-refresh/comment-page-1/#comment-929</link>
		<dc:creator>Natalya</dc:creator>
		<pubDate>Wed, 17 Mar 2010 17:21:48 +0000</pubDate>
		<guid isPermaLink="false">http://exceptionalgeeks.com/bi-curious/2010/02/12/etl-pattern-staged-refresh/#comment-929</guid>
		<description>Pierre, first of all, thank you for great blog. I am new to BI, so there is a lot to learn.  I have a question about ETL pattern I have noticed at one of the client sites. There are 2 databases, DW and DW_Staging. First everything gets truncated in DW_Staging and gets reloaded, then there is a check for successfull load, after which they truncate everything in DW database and load it from DW_Staging.  I was wondering what your thought would be on this sort of design. Is it something that I should correct or essentially since it is working, leave it alone? What are implications?</description>
		<content:encoded><![CDATA[<p>Pierre, first of all, thank you for great blog. I am new to BI, so there is a lot to learn.  I have a question about ETL pattern I have noticed at one of the client sites. There are 2 databases, DW and DW_Staging. First everything gets truncated in DW_Staging and gets reloaded, then there is a check for successfull load, after which they truncate everything in DW database and load it from DW_Staging.  I was wondering what your thought would be on this sort of design. Is it something that I should correct or essentially since it is working, leave it alone? What are implications?</p>
]]></content:encoded>
	</item>
	<item>
		<title>Comment on 3 Screens, the Cloud, and Business Intelligence? – Part 2 by BeyeNETWORK: Raising the Bar on Business Analytics: Innovation … | Management Business Wisdom</title>
		<link>http://exceptionalgeeks.com/bi-curious/2010/03/16/3-screens-the-cloud-and-business-intelligence-%e2%80%93-part-2/comment-page-1/#comment-927</link>
		<dc:creator>BeyeNETWORK: Raising the Bar on Business Analytics: Innovation … | Management Business Wisdom</dc:creator>
		<pubDate>Wed, 17 Mar 2010 05:39:39 +0000</pubDate>
		<guid isPermaLink="false">http://exceptionalgeeks.com/bi-curious/?p=99#comment-927</guid>
		<description>[...] BI-Curious » 3 Screens, the Cloud, and Business Intelligence? – Part 2 [...]</description>
		<content:encoded><![CDATA[<p>[...] BI-Curious » 3 Screens, the Cloud, and Business Intelligence? – Part 2 [...]</p>
]]></content:encoded>
	</item>
	<item>
		<title>Comment on 3 Screens, the Cloud, and Business Intelligence? – Part 1 by BI-Curious » 3 Screens, the Cloud, and Business Intelligence? – Part 2</title>
		<link>http://exceptionalgeeks.com/bi-curious/2010/03/11/3-screens-the-cloud-and-business-intelligence-part-1/comment-page-1/#comment-926</link>
		<dc:creator>BI-Curious » 3 Screens, the Cloud, and Business Intelligence? – Part 2</dc:creator>
		<pubDate>Tue, 16 Mar 2010 23:03:59 +0000</pubDate>
		<guid isPermaLink="false">http://exceptionalgeeks.com/bi-curious/?p=90#comment-926</guid>
		<description>[...] as stated in the previous post, when Microsoft is referring to the three-screen concept they are talking about PC, mobile, and TV. [...]</description>
		<content:encoded><![CDATA[<p>[...] as stated in the previous post, when Microsoft is referring to the three-screen concept they are talking about PC, mobile, and TV. [...]</p>
]]></content:encoded>
	</item>
	<item>
		<title>Comment on BI Style Guides by Tweets that mention BI-Curious » BI Style Guides -- Topsy.com</title>
		<link>http://exceptionalgeeks.com/bi-curious/2010/01/21/bi-style-guides/comment-page-1/#comment-852</link>
		<dc:creator>Tweets that mention BI-Curious » BI Style Guides -- Topsy.com</dc:creator>
		<pubDate>Thu, 21 Jan 2010 22:14:39 +0000</pubDate>
		<guid isPermaLink="false">http://exceptionalgeeks.com/bi-curious/?p=67#comment-852</guid>
		<description>[...] This post was mentioned on Twitter by JD Wade, MrVitaminP. MrVitaminP said: New blog post: "BI Style Guides" - http://tr.im/L9l2 [...]</description>
		<content:encoded><![CDATA[<p>[...] This post was mentioned on Twitter by JD Wade, MrVitaminP. MrVitaminP said: New blog post: &#8220;BI Style Guides&#8221; &#8211; <a href="http://tr.im/L9l2" rel="nofollow">http://tr.im/L9l2</a> [...]</p>
]]></content:encoded>
	</item>
	<item>
		<title>Comment on BI Release Management by Agile Business Intelligence and Why It Matters « specialization is for insects</title>
		<link>http://exceptionalgeeks.com/bi-curious/2009/10/22/bi-release-management/comment-page-1/#comment-777</link>
		<dc:creator>Agile Business Intelligence and Why It Matters « specialization is for insects</dc:creator>
		<pubDate>Fri, 30 Oct 2009 15:51:47 +0000</pubDate>
		<guid isPermaLink="false">http://exceptionalgeeks.com/bi-curious/2009/10/22/bi-release-management/#comment-777</guid>
		<description>[...] the progress and relevance of agile business intelligence (http://agileintelligence.blogspot.com/, http://exceptionalgeeks.com/bi-curious/2009/10/22/bi-release-management/, http://www.analyticbridge.com/profiles/blogs/agile-business-intelligence, [...]</description>
		<content:encoded><![CDATA[<p>[...] the progress and relevance of agile business intelligence (<a href="http://agileintelligence.blogspot.com/" rel="nofollow">http://agileintelligence.blogspot.com/</a>, <a href="http://exceptionalgeeks.com/bi-curious/2009/10/22/bi-release-management/" rel="nofollow">http://exceptionalgeeks.com/bi-curious/2009/10/22/bi-release-management/</a>, <a href="http://www.analyticbridge.com/profiles/blogs/agile-business-intelligence" rel="nofollow">http://www.analyticbridge.com/profiles/blogs/agile-business-intelligence</a>, [...]</p>
]]></content:encoded>
	</item>
	<item>
		<title>Comment on Manage BI Like Software Development by BI-Curious » BI Revision/Version/Source Control</title>
		<link>http://exceptionalgeeks.com/bi-curious/2009/10/15/manage-bi-like-software-development/comment-page-1/#comment-775</link>
		<dc:creator>BI-Curious » BI Revision/Version/Source Control</dc:creator>
		<pubDate>Thu, 29 Oct 2009 00:26:35 +0000</pubDate>
		<guid isPermaLink="false">http://exceptionalgeeks.com/bi-curious/2009/10/15/manage-bi-like-software-development/#comment-775</guid>
		<description>[...] is a continuation on my previous rants on borrowing traditional software development concepts for a BI project. As the title [...]</description>
		<content:encoded><![CDATA[<p>[...] is a continuation on my previous rants on borrowing traditional software development concepts for a BI project. As the title [...]</p>
]]></content:encoded>
	</item>
	<item>
		<title>Comment on BI Release Management by BI-Curious » BI Revision/Version/Source Control</title>
		<link>http://exceptionalgeeks.com/bi-curious/2009/10/22/bi-release-management/comment-page-1/#comment-774</link>
		<dc:creator>BI-Curious » BI Revision/Version/Source Control</dc:creator>
		<pubDate>Wed, 28 Oct 2009 20:59:28 +0000</pubDate>
		<guid isPermaLink="false">http://exceptionalgeeks.com/bi-curious/2009/10/22/bi-release-management/#comment-774</guid>
		<description>[...] is a continuation on my previous rants on borrowing traditional software development concepts for a BI project. As the title suggests, the [...]</description>
		<content:encoded><![CDATA[<p>[...] is a continuation on my previous rants on borrowing traditional software development concepts for a BI project. As the title suggests, the [...]</p>
]]></content:encoded>
	</item>
	<item>
		<title>Comment on BI Release Management by uberVU - social comments</title>
		<link>http://exceptionalgeeks.com/bi-curious/2009/10/22/bi-release-management/comment-page-1/#comment-770</link>
		<dc:creator>uberVU - social comments</dc:creator>
		<pubDate>Sat, 24 Oct 2009 08:11:31 +0000</pubDate>
		<guid isPermaLink="false">http://exceptionalgeeks.com/bi-curious/2009/10/22/bi-release-management/#comment-770</guid>
		<description>&lt;strong&gt;Social comments and analytics for this post...&lt;/strong&gt;

This post was mentioned on Twitter by MrVitaminP: New blog post: "BI Release Management" - http://tr.im/CGXs...</description>
		<content:encoded><![CDATA[<p><strong>Social comments and analytics for this post&#8230;</strong></p>
<p>This post was mentioned on Twitter by MrVitaminP: New blog post: &#8220;BI Release Management&#8221; &#8211; <a href="http://tr.im/CGXs.." rel="nofollow">http://tr.im/CGXs..</a>.</p>
]]></content:encoded>
	</item>
</channel>
</rss><!-- Dynamic page generated in 3.309 seconds. --><!-- Cached page generated by WP-Super-Cache on 2010-03-20 16:37:11 --><!-- Compression = gzip -->
