SQL Server 2008 FILESTREAM and ADO.NET

August 6, 2010 11:45 by wjchristenson2

In my last post, I gave a tutorial of SQL Server 2008’s new FILESTREAM feature.  I described what it is, why it should be used for BLOB data storage, and released some TSQL scripts to enable FILESTREAM and create a table to hold some media.  In this post, I will show how to insert and read FILESTREAM data from a SQL 2008 Server using ADO.NET.

ADO.NET comes with the SqlClient data provider (System.Data.SqlClient).  It supports FILESTREAM operations by using the SqlFileStream class which resides in the System.Data.SqlTypes namespace.  As its name suggests, the SqlFileStream is a stream and thus inherits from System.IO.Stream.  The SqlFileStream will provide methods to allow us to read and write streams of data to the SQL Server.  I will show 2 examples: inserting and reading FILESTREAM data with ADO.NET (the SqlFileStream object).  The program I created allows the user to save and view .jpg images from the SQL Server using FILESTREAM.  Here is a picture of the final example program.


Overview:
The SqlFileStream requires 2 arguments which are supplied from the SQL Server: the read/write path to the server’s file and the transactional context/token.  These 2 items are acquired via a preliminary SQL query before the the SqlFileStream is employed to read or write to the NTFS file system.

Inserting FILESTREAM Data
I first INSERT a record with a NULL value for the varbinary(max) field.  I use this blank record to acquire the server file path and also return the current FILESTREAM transactional context/token.  After this is complete, I can employ the SqlFileStream to write to the server file from my local file.  Below is my stored proc I used along with the .NET code.

 

   1:  IF EXISTS (SELECT * FROM sysobjects WHERE type = 'P' AND name = 'usp_InsertDocument')
   2:      BEGIN
   3:          DROP PROCEDURE dbo.usp_InsertDocument
   4:      END
   5:  GO
   6:   
   7:  CREATE PROCEDURE dbo.usp_InsertDocument 
   8:      WITH ENCRYPTION
   9:  AS
  10:      SET NOCOUNT ON;
  11:   
  12:      DECLARE @DocumentID INT;
  13:   
  14:      --insert NULL first to acquire ID & the transaction context values
  15:      INSERT INTO dbo.Document (Document) VALUES (0x00);
  16:   
  17:      SELECT @DocumentID = SCOPE_IDENTITY();
  18:   
  19:      SELECT 
  20:          doc.DocumentID,
  21:          doc.DocumentGUID,
  22:          doc.Document.PathName(0) AS [PathName], 
  23:          GET_FILESTREAM_TRANSACTION_CONTEXT() AS [TransactionContext] 
  24:      FROM dbo.Document AS doc WITH (NOLOCK) 
  25:      WHERE doc.DocumentID = @DocumentID;
  26:   
  27:      SET NOCOUNT OFF;
  28:  GO

 

   1:      Private Sub InsertDocument(ByVal clientFilePath As String)
   2:          Dim dt As DataTable = New DataTable()
   3:   
   4:          Using conn As SqlConnection = New SqlConnection(_connectionString)
   5:              Dim reader As SqlDataReader = Nothing
   6:              Dim context() As Byte = Nothing
   7:              Dim serverFilePath As String = ""
   8:   
   9:              conn.Open()
  10:   
  11:              Dim transaction As SqlTransaction = conn.BeginTransaction()
  12:   
  13:              Try
  14:                  Dim cmd As SqlCommand = conn.CreateCommand()
  15:                  With cmd
  16:                      .Transaction = transaction
  17:                      .CommandType = CommandType.StoredProcedure
  18:                      .CommandText = "dbo.usp_InsertDocument"
  19:                  End With
  20:   
  21:                  reader = cmd.ExecuteReader()
  22:                  dt.Load(reader)
  23:   
  24:                  If dt.Rows.Count > 0 Then
  25:                      serverFilePath = dt.Rows(0)("PathName").ToString()
  26:                      context = DirectCast(dt.Rows(0)("TransactionContext"), Byte())
  27:   
  28:                      Using serverStream As SqlFileStream = New SqlFileStream(serverFilePath, context, FileAccess.Write)
  29:                          Using clientStream As FileStream = New FileStream(clientFilePath, FileMode.Open, FileAccess.Read)
  30:                              Dim bytesRead As Integer = 0, bufferSize As Integer = 4096
  31:                              Dim buffer As Byte() = New Byte(bufferSize - 1) {}
  32:   
  33:                              Do
  34:                                  bytesRead = clientStream.Read(buffer, 0, bufferSize)
  35:                                  If bytesRead = 0 Then
  36:                                      Exit Do
  37:                                  End If
  38:   
  39:                                  serverStream.Write(buffer, 0, bytesRead)
  40:                                  serverStream.Flush()
  41:                              Loop While True
  42:   
  43:                              clientStream.Close()
  44:                          End Using
  45:   
  46:                          serverStream.Close()
  47:                      End Using
  48:                  End If
  49:   
  50:                  transaction.Commit()
  51:              Catch ex As Exception
  52:                  Try
  53:                      'attempt to rollback the transaction
  54:                      transaction.Rollback()
  55:                  Catch ex2 As Exception
  56:                      'rollback was unsuccessful (connection may be closed)
  57:                  End Try
  58:   
  59:                  MessageBox.Show(ex.Message)
  60:              Finally
  61:                  If reader IsNot Nothing AndAlso Not reader.IsClosed Then reader.Close()
  62:              End Try
  63:          End Using
  64:   
  65:          LoadDocuments()
  66:      End Sub

 

Reading FILESTREAM Data
Similar to the insertion of a file, reading a file requires getting the server file path and then current FILESTREAM transactional context/token.  After those items are gathered from the SQL Server, we can connect to and stream the data anyway we like.  Below I streamed .jpg data from the SQL Server and loaded them into a WPF Image control for viewing purposes.

 

   1:  IF EXISTS (SELECT * FROM sysobjects WHERE type = 'P' AND name = 'usp_SelectDocument')
   2:      BEGIN
   3:          DROP PROCEDURE dbo.usp_SelectDocument
   4:      END
   5:  GO
   6:   
   7:  CREATE PROCEDURE dbo.usp_SelectDocument (
   8:      @DocumentID INT
   9:  )
  10:      WITH ENCRYPTION
  11:  AS
  12:      SET NOCOUNT ON;
  13:   
  14:      SELECT 
  15:          doc.DocumentID,
  16:          doc.DocumentGUID,
  17:          doc.Document.PathName(0) AS [PathName], 
  18:          GET_FILESTREAM_TRANSACTION_CONTEXT() AS [TransactionContext] 
  19:      FROM dbo.Document AS doc WITH (NOLOCK)
  20:      WHERE doc.DocumentID = @DocumentID;
  21:   
  22:      SET NOCOUNT OFF;
  23:  GO

 

   1:      Private Sub LoadDocument(ByVal documentID As Integer)
   2:          Dim dt As DataTable = New DataTable()
   3:   
   4:          Using conn As SqlConnection = New SqlConnection(_connectionString)
   5:              Dim reader As SqlDataReader = Nothing
   6:              Dim context() As Byte = Nothing
   7:              Dim serverFilePath As String = ""
   8:   
   9:              conn.Open()
  10:   
  11:              Dim transaction As SqlTransaction = conn.BeginTransaction()
  12:   
  13:              Try
  14:                  Dim cmd As SqlCommand = conn.CreateCommand()
  15:                  With cmd
  16:                      .Transaction = transaction
  17:                      .CommandType = CommandType.StoredProcedure
  18:                      .CommandText = "dbo.usp_SelectDocument"
  19:                      .Parameters.Add(New SqlParameter("@DocumentID", documentID))
  20:                  End With
  21:   
  22:                  reader = cmd.ExecuteReader()
  23:                  dt.Load(reader)
  24:   
  25:                  If dt.Rows.Count > 0 Then
  26:                      serverFilePath = dt.Rows(0)("PathName").ToString()
  27:                      context = DirectCast(dt.Rows(0)("TransactionContext"), Byte())
  28:   
  29:                      Using memStream As MemoryStream = New MemoryStream()
  30:                          Using serverStream As SqlFileStream = New SqlFileStream(serverFilePath, context, FileAccess.Read)
  31:   
  32:                              Dim bytesRead As Integer = 0, bufferSize As Integer = 4096
  33:                              Dim buffer As Byte() = New Byte(bufferSize - 1) {}
  34:   
  35:                              Do
  36:                                  bytesRead = serverStream.Read(buffer, 0, bufferSize)
  37:                                  If bytesRead = 0 Then
  38:                                      Exit Do
  39:                                  End If
  40:   
  41:                                  memStream.Write(buffer, 0, bytesRead)
  42:                                  memStream.Flush()
  43:                              Loop While True
  44:   
  45:                              serverStream.Close()
  46:                          End Using
  47:   
  48:                          memStream.Seek(0, SeekOrigin.Begin)
  49:   
  50:                          Dim bmpImage As BitmapImage = New BitmapImage()
  51:                          With bmpImage
  52:                              .BeginInit()
  53:                              .StreamSource = memStream
  54:                              .CreateOptions = BitmapCreateOptions.None
  55:                              .CacheOption = BitmapCacheOption.OnLoad
  56:                              .EndInit()
  57:                              .Freeze()
  58:                          End With
  59:   
  60:                          Me.imgView.Source = bmpImage
  61:   
  62:                          memStream.Close()
  63:                      End Using
  64:                  End If
  65:   
  66:                  transaction.Commit()
  67:              Catch ex As Exception
  68:              Try
  69:                  'attempt to rollback the transaction
  70:                  transaction.Rollback()
  71:              Catch ex2 As Exception
  72:                  'rollback was unsuccessful (connection may be closed)
  73:              End Try
  74:   
  75:              MessageBox.Show(ex.Message)
  76:          Finally
  77:              If reader IsNot Nothing AndAlso Not reader.IsClosed Then reader.Close()
  78:          End Try
  79:          End Using
  80:      End Sub

So there you have it.  The basics of writing and reading to a FILESTREAM on a SQL 2008 Server using ADO.NET.  Below you’ll find my scripts and .NET source code.

SqlFileStreamExample_Soln.zip (559.27 kb)


Silverlight 4 Final – Released 4/15/2010

April 18, 2010 10:59 by wjchristenson2

On Thursday 4/15/2010, Microsoft officially released the final version Silverlight 4.  Scott Guthrie gave a 60 minute keynote on Silverlight 4 a few days prior.  If you haven’t watched the keynote, I’d recommend doing so.  I was impressed with the new features and the demos provided.  Silverlight 4 includes a ton of new features. 

Here is a quick list of some of the new features that I found interesting:

  • Tooling – Multi-Targeting Silverlight 3 and 4.  You can also now design within VS 2010.
  • Printing API – You can now print from Silverlight.
  • Right-Click Event Handling – MouseRightButtonUp/Down events are now available.
  • Webcam & Microphone Access – SL can now use the client’s webcam(s) and microphone(s).
  • Mouse Wheel Support – The mouse wheel event was added in SL 3, but now TextBox, ComboBox, Calendar, DatePicker, and the ScrollViewer auto-scroll.  You no longer have to manually handle the event.
  • RichTextArea Control – Provides an area where users can create and edit text and specify bold/italic/underlined/etc text.
  • Google Chrome Support - M$ finally now supports Google Chrome.
  • Implicit Theming – Create a style for a targeted type and all of those types will implicitly use it.
  • Fluid UI Support in the ItemsControl – 3 new states: BeforeLoaded, Loaded, and Unloaded.
  • DataGrid Enhancements – Column relative width sizing refactored.
  • DataBinding Enhancements – Binding can now use StringFormat, TargetNullValue, FallbackValue (e.g. Dates no longer requires ValueConverters to format the date).
  • IDataErrorInfo – When this interface is implemented, it reports data validation errors that a UI can bind to.  Only one property is validated/reported on at a time.
  • INotifyDataErrorInfo – Allows developers to provide custom validation logic server-side via asynchronous routines.  Here is a video on the new data validation features.
  • Silverlight Drop Target – Drag and drop folders/files into your Silverlight application.
  • ViewBox – New control which is used to simplify the resizing of images.
  • Text Trimming – Auto adding of word ellipse (…)
  • Keyboard Access in Full Screen Mode – If SL is in full screen mode – all keyboard input is accepted in Silverlight as long as the application is trusted.
  • Network Authentication – You can now pass separate credentials when connecting with 3rd party service providers.
  • COM Interop – Silverlight can now create COM objects – however this only applies to trusted SL applications.
  • Notification (“Toast”) API – Notifications (like a new Outlook email as arrived) can now be used.
  • Local File Access – SL can now access the user’s “My” folders (e.g. My Documents, etc).  This requires trusted SL application.
  • Elevated Trust Applications – Many new features in SL4 require elevated trust.  This new feature will prompt the user to allow/fully trust the SL application when installed.
  • HTML Hosting with WebBrowser – You can now show and host HTML within your SL application.
  • Clipboard API – SL now has access to copy to and paste from the clipboard.

ASP.NET MVC First Impressions

February 21, 2010 19:06 by wjchristenson2

Today I decided to make a small ASP.NET MVC application which would add, edit, delete, and list US states which resided in a SQL Server Express database.  I am a veteran in ASP.NET Web Forms development, so I wanted to see what MVC is like.  I create business objects/models everyday, so understanding models was easy enough.  The tricky part for me was creating and understanding how controllers and views interact with each other.

First Impressions:
1)  I liked the clean separation between controller/view/models.  Basically this is required when using MVC.  I’ve already been accustomed to separating layers, so this wasn’t new to me, but seeing how the MVC Framework has this built in was refreshing.
2)  When I created a new MVC Web Application, it automatically prompted to create a Test project.  I thought that was pretty neat.  I’m not experienced in creating automated test yet, but I’m sure I’ll get there soon and just seeing the parallelism with MVC and testing was impressive.
3)  REST (Representational State Transfer) URLs was interesting.  Apparently you can configure how requests are mapped to controllers in the Global.asax.  I didn’t get into this much as I took the defaults, but simply editing a state looked like such: “/states/Edit/1”.  No more mapping to a template/resource.
4)  You can generate a view straight from the controller by right-clicking when you return an ActionResult.  Visual Studio will then prompt/ask you what kind of view you want to make and you can also have it create a strongly typed view so that you can access the model’s properties.  So the framework can assist in starting a view pretty quickly for you.
5)  Not having view state and post backs was really nice.  The rendered HTML was very clean.
6)  I was confused at times as to how the framework knew what to pass to controller routines on a form post and also how id’s work.  What if you have more than 1 property that makes up the id?  I imagine this would come with time.

Overall I enjoyed the initial experience and it seemed very structured.  I feel that one day doesn’t do it justice as MVC is a bit more complex than web forms.

Helpful Resources:
1)  Scott Guthrie’s NerdDinner.com Chapter:  Scott Guthrie wrote a chapter in Professional ASP.NET MVC 1.0 in which he walks you through step-by-step in creating an MVC web application.  You can download the chapter in PDF and also download the source code if needed.
2)  Simple E-Commerce Storefront Application: This is another Scott Guthrie example, however this example has some nice architecture diagrams that go along with it to explain what’s going on a bit more.

MVCTest_Soln.zip (891.39 kb)

ASP.NET MVC Overview Part 1 (General ASP.NET MVC Overview)
ASP.NET MVC Overview Part 2 (Advantages of MVC and Web Forms)
ASP.NET MVC First Impressions


ASP.NET MVC Overview Part 2

February 14, 2010 13:36 by wjchristenson2

In Part1, I discussed the basics of what the MVC pattern is and how Microsoft has incorporated MVC into ASP.NET.  In this segment, I am going to discuss some general advantages of ASP.NET Web Forms and ASP.NET MVC.  Neither flavor of ASP.NET will be the superior choice in all circumstances.  Therefore knowing the advantages to each will help.

I’ve found through my development career that there’s never a right or wrong way to do something.  Scott Guthrie posted recently the following which is very true, “Great developers using bad tools/frameworks can make great apps.”  The same principal applies to bad developers using great tools/frameworks… inevitably they will create bad applications.  Just because one framework may shine better in certain situations may not mean its better.  If the developers using the superior technology cannot grasp or develop it properly, its full potential will not be realized.  So keep these principals in mind when evaluating the two.

Advantages of ASP.NET MVC
1) Separation of model, view, controller results in reduction of complexity, promotes parallel development, and easier to maintain.
2) Enables full control over rendered HTML (leaner html rendering).
3) Works much better with automated testing as all core contracts are interface based.  You can run unit-tests without having to run the controllers under an ASP.NET process.
4) Does not use view state or postbacks (no view stage = quicker load times).
5) Framework is very extensible and pluggable.
6) Supports existing markup (aspx, master pages, ascx, inline expressions).
7) Supports existing session, caching, authentication, etc.
8) Easy integration with javascript frameworks.
9) Follows the design of a stateless web.


Advantages of ASP.NET Web Forms
1) Commonality between windows forms development in that it uses the event model.
2) Usage of view state and server forms helps manage state more easily.
3) Promotes RAD more so than MVC as more controls/components are available.
4) Less complex than MVC and generally requires less code.
5) More mature as it’s been around since the 1990’s.

ASP.NET MVC Overview Part 1 (General ASP.NET MVC Overview)
ASP.NET MVC Overview Part 2 (Advantages of MVC and Web Forms)
ASP.NET MVC First Impressions


ASP.NET MVC Overview Part 1

February 9, 2010 21:35 by wjchristenson2

Traditionally there has been only one approach to develop ASP.NET web applications; web forms.  Lately I’ve been hearing a lot of hype wrapped around this thing called MVC and the fact that it can be used as an approach to develop ASP.NET applications.  What is MVC?  MVC stands for Model-View-Controller.  It is a design pattern which separates an application into 3 logical tiers: domain, input, and presentation.  This isolation between layers yields independent development, testing, and maintenance of each.

Model – The model can be thought of as the business layer.  It performs application logic and is used to maintain the state of the application typically in a database.

View – A view is a component that is used to display the UI which can often be a direct representation of the model’s state.  For an example, the model may return a collection of workers from the database.  The View could display the workers in a GridView/DataGrid.

Controller – The controller ultimately handles and responds to user interaction.  It works with the model based on what the user desires and then renders the appropriate view accordingly.

The ASP.NET MVC Framework is Microsoft’s stab at at creating an ASP.NET programming environment that is centered around the MVC pattern.  So how is the request model different between MVC and traditional web forms?  An ASP.NET web form uses the postback to relay the request and the page’s code behind handles it.  In ASP.NET MVC, REST is used to communicate the request and MVC is used to process it.

REST stands for Representational State Transfer.  In ASP.NET MVC, instead of mapping URLs to ASPX files stored on disk you are mapping URLs to controllers (classes).  The controller will handle what view to render back to the end-user.

ASP.NET MVC Overview Part 1 (General ASP.NET MVC Overview)
ASP.NET MVC Overview Part 2 (Advantages of MVC and Web Forms)
ASP.NET MVC First Impressions


Configure WCF Service – Reuse Collection Types Issue

December 14, 2009 14:22 by wjchristenson2

In VS 2008, there is a problem with reusing collection types when you configure a WCF service and wish to “Reuse types in all refrenced assemblies”.  If you want to pass collections around via your WCF service, a Visual Studio will create a proxy class for each collection type regardless whether your collection type is included within referenced assemblies or not.  I will show you how to bypass this shortcoming.

First, locate the Reference.svcmap file for the WCF service reference you are having problems with.  If you can’t see it, ensure your project  has “Show All Files” enabled.  If you open the Reference.svcmap, you’ll find that it is written in XML.  Locate the CollectionMappings node.  Within CollectionMappings, add your collection type so when you update your service reference, it recognizes it as a known type and will not generate a proxy class for you.  Hope this helps.


   1:  <CollectionMappings>
   2:      <CollectionMapping TypeName="System.Collections.Generic.List`1" Category="List" />
   3:      <CollectionMapping TypeName="WpfApplication1.Objects.MyCollection" Category="List" />
   4:  </CollectionMappings>

Fiddler - Inspecting WCF Binary Encoded Messages

November 25, 2009 08:57 by wjchristenson2

Fiddler is the tool of choice when it comes to inspecting WCF messages that utilize an HTTP transport.  In a previous post I mentioned that HTTP WCF messages should be in encoded in a binary format to decrease their size.  The problem you run into when encoding your WCF messages is that when Fiddler attempts to inspect an encoded message, it’s not in a readable format.  To resolve this limitation, you can download a free plug-in which will add an inspector to Fiddler to decode binary WCF message.  Using this new inspector, you can inspect binary encoded WCF messages freely from within Fiddler.

http://code.msdn.microsoft.com/wcfbinaryinspector



Silverlight 3 WCF Binary Message Encoding

October 14, 2009 13:29 by wjchristenson2

Silverlight 3 offers us some new features when it comes to WCF web services.  In Silverlight 2, BasicHttpBinding was the only supported binding.  This essentially encodes your serialized objects in clear text and sends them over an HTTP transport.  Because the objects were sent as clear text, the message size could get out-of-hand.  When sending data across HTTP/Internet, you obviously want to decrease the size as this will improve performance of your client application.  Silverlight 3 offers the ability to create custom bindings which support the ability to encode your WCF web service messages as a binary format.

Binary encoding offers some serious performance gains over text encoding.  Personally I’ve seen 30% – 40% reduction in message size between the server and client when binary encoding is enabled.  Keep in mind that binary encoding is a WCF-specific feature.  Therefore if you have heterogeneous technologies wanting to consume your service, you’ll need to stick with BasicHttpBinding.  Here is an example of how to enable binary encoding for your WCF service:


   1:  <bindings>
   2:     <customBinding>
   3:        <binding name="binaryHttpBinding">
   4:           <binaryMessageEncoding />
   5:           <httpTransport />
   6:        </binding>
   7:     </customBinding>
   8:  </bindings>
   9:   
  10:  <endpoint address="" binding="customBinding" bindingConfiguration="binaryHttpBinding" contract="MyService" />

ExecuteNonQuery with “GO” Separators

September 25, 2009 08:26 by wjchristenson2

If you’ve ever tried to run a collection of SQL batches via the .NET DbCommand.ExecuteNonQuery method, you probably received an error similar to the following: “Incorrect Syntax near ‘GO’ …”.  DbCommand.ExecuteNonQuery is intended to execute a single SQL batch statement against a database.  The “GO” separator is a command that is used/understood by SQL Server Query Analyzer and Query Editor utilities to separate out SQL batches when ran.

There has been some workarounds that I’ve seen that involve regular expressions and string manipulation/splitting to break out the script with “GO” separators into a collection of SQL statements and run them each individually.  This can work, but you will run into situations where commented areas will contain the “GO” statement and other problems can/will arise.  Such situations would cause the workaround to fail.

The proper way to run scripts with the “GO” separator is to employ SQL Server Management Objects (SMOs).  Microsoft’s SQL Server 2008 Feature Pack contains links to download/install the SMO assemblies (.NET Framework object model) .  Developers can use SMOs to perform SQL Server management routines through their .NET application code.

Steps to Use SMOs:

1) Download and Install SMOs - Download the SQL Server Management Objects found on this page.  Run the installation package.  If you are running SQL Server 2005, you may need to search for the 2005 version as the link I gave is for 2008.

2) Add references - After the SMOs are installed,  we’ll need to add references to them in our project.  The SMO assemblies should appear under your .NET tab when adding references to your project.  If you can’t  find them there, the assemblies were installed to “C:\Program Files\Microsoft SQL Server\100\SDK\Assemblies” for me.  To run scripts with “GO” statement, we’ll need references to the following DLLs:
  a) Microsoft.SqlServer.ConnectionInfo.dll
  b) Microsoft.SqlServer.Management.Sdk.Sfc.dll
  c)  Microsoft.SqlServer.Smo.dll

3) Import the required namespaces and execute your SQL script with “GO” statements like below:


   1:  Imports System.Data.SqlClient
   2:  Imports Microsoft.SqlServer.Management.Smo
   3:  Imports Microsoft.SqlServer.Management.Common
   4:   
   5:  Class Window1
   6:   
   7:      Public Sub ExecuteSqlStatements(ByVal connectionString As String, ByVal sqlStatements As String)
   8:          If Not String.IsNullOrEmpty(sqlStatements) Then
   9:              Using mySqlConnection As SqlConnection = New SqlConnection(connectionString)
  10:                  Dim mySqlServer As Server = New Server(New ServerConnection(mySqlConnection))
  11:                  mySqlServer.ConnectionContext.ExecuteNonQuery(sqlStatements)
  12:              End Using
  13:          End If
  14:      End Sub
  15:   
  16:  End Class

SqlBatchExecNonQuery_Soln.zip (965.09 kb)