CodeBetter.Com
CodeBetter.Com
RSS 2.0 via Feedburner
           Do you Twitter? Follow us @CodeBetter

Brendan Tompkins [MVP]

Blog First. Ask Questions Later.

Transform an ADO.NET DataSet to ADO Recordset XML, Excel XML and more!

A couple of weeks ago, I complained about the KB code here KB316337.  What this code is supposed to do is transform a DataSet into ADO Recordset XML format.  I ended up porting the VB code to C# and also rewriting some of the code along the way...  It's too much code to show it all here so I've created a web project that you can download which contains the transformation in a utility class.

Using this utility class you can transform a DataSet into different formats, including ADO Recordset XML, Comma Delimited, Tab Delimited and Excel XML.  Others have posted about doing the same thing, notably Darrell Norton in a post here.   I've boiled it all down in to an easy to use utility class.  To use this, simply make the method call like so:

Util.ExportUtils.ExportDataSet(this.exampleDataSet1, "ADOXMLExport", Common.DataSetExportType.ADORecordSet, this.Context);

Notice that the third parameter is an enumeration specifying the export type. This enum looks like this:

public enum DataSetExportType
{
   XML,
   ExcelXML,
   TabDelimited,
   CommaDelimited,
   ADORecordSet
}

Why would anyone want to go from ADO.NET DataSet to an ADO Recordset?  Well, one big reason is to funnel  a .NET DataSet's data into an Office Web Component, such as a PivotTable and PivotChart.  This can be pretty powerful stuff. The example page included in the project lets you go from a DataGrid bound to a typed dataset, that looks like this:

To a Pivot Chart that looks like this:

Trust me, show this to a client and they'll do backflips with excitement!

If you're intereseted in seeing all of this in action, download and run the project here.  Don't open the ASPX page with the designer, because it messes up the <OBJECT> code for the OWCs. Also, to use the OWC, you'll need Office XP and you'll have to trust your local domain from within IE's trusted sites tab.  This was (quickly) thrown together for example purposes, so you may have to tweak it for your specific implementation, but it should get you started. Enjoy!

-Brendan

 



Comments

andy said:

Good looking code example.

One thing I noticed and I only noticed it because I struggled with the issue for a bit in a similar issue. If a field is Null then GetXML will not export it. If every value in a column is null then you will never see the column in the xml. http://support.microsoft.com/default.aspx?scid=kb;EN-US;Q317961

Not an issue for some things but if you are trying to get your data into excel or comma delimited then accounting for every column is critical. The best resolution I came up with was to make sure that there was never a null value in the dataset I was going to transform. Other people have mentioned using writexml.
# April 29, 2004 10:09 AM

Brendan Tompkins said:

Andy,

You're right about that. The DataSet's GetXML() method leaves out nodes that are null. I haven't had problems with this going into An ADO Recordset, but certainly for delimited files this becomes problematic. The problem with having some value, is what do you put into value fields? For example, what do you put into a null DateTime field? One thing I've done is convert a Typed DataSet into an untyped one, and put NO VALUE in for the nulls. This seems to work, but it a kludge, to say the least. Perhaps there should be a VERBOSE switch to the GetXML which could creat these empty nodes?
# April 29, 2004 12:24 PM

andy said:

There really isn't a clean solution. The underlying getxml should be fixed. Some of the things that I have done are
1. Replace null with default values of the columns. Of course this only works if the default value is set and the default value is not null.
2. If datatype is of no concern to me (exporting to a text file for instance) then convert all of the columns to string. Then just replace all nulls with a zero length string.
3. Kludge method(s) etc...

It's interesting to note that if you inspect the schema of the dataset that the minoccurs property is set to zero. Perhaps modifiying that might get a better output with the function getxml. I've never really tried it though.
# April 30, 2004 5:16 AM

Don Haas said:

I'm new to the XML thing. I'm having trouble with WriteXML. I think the problem is that the records in the dataset have columns that contain binary data. Can WriteXML handle this out of the box, or do I have to use the MSXML4 SDK w/COM Interop which exposes the dataType property? Is it as simple as using the SQL Server OLEDB provider instead of JET, or using the ADO.NET native SQL Server driver?

My project compiles with the DOM interfaces, and everything seems to be in place, but now I get an exception that led me to something about an application manifest for side by side execution on Windows XP. I created the manifest per the MSXML4 help, but now I'm getting a system.resource exception. Looks like when I upgrade from W2K Pro to XP Pro the msxml4.manifest wasn't copied to C:\WINNT\WinSxS\Manifests so I have no way of verifying the public key info. I'm assuming the manifest isn't there because MSXML4.dll was already registered by regsrv32. The interop assembly is not in the GAC neither.

Any insight would be greatly appreciated ! To make things even more interesting I'm doing all this in Delphi 8 . . .
# May 6, 2004 10:01 AM

Brendan Tompkins said:

Don,

You're using a DataSet, right? If so, there are two types, hexBinary, and base64Binary that you can use to store binary data. On your c# side it will be a byte array (System.Byte []) See this article here for more about the DataSet types:

http://www.xmlme.com/DatasetToXml.aspx?mid=45

AFAIK, this should serialize in and out okay, withouth having to resort to COM Interop although I've not done it myself.

-Brendan
# May 7, 2004 3:42 AM

John F said:


Does anyone else have a problem running this sample under Visual studio .NET 2003 - When I click any of the Export buttons, nothing happens, and the Click event doesn't even get called ? (according to my neophyte level debugging skills)

But when I click the pivot chart or pivot table buttons, their click event does get called (and the item is shown (either the table or chart control)

Anyone have a similar experience ?
# June 8, 2004 3:50 AM

John F said:

Regarding my problem running the sample

It seems it was caused by me NOT opening default.aspx in the designer (as Brendan warned against ) - but I found that after I opened it with the designer and clicked on all the buttons to drill down to their event code - it re-hooked up all the events and worked fine - even for the pivot and chart ...
# June 8, 2004 4:22 AM

Brian Surowiec said:

I Ran into the problem of null values in my dataset, so to combat that I just added this to any column that I knew allowed null values. Works perfectly now.

SELECT ISNULL(COLUMN_NAME, '') AS [NotNullColumn] FROM TABLE_NAME

or

SELECT ISNULL((SELECT…), '') AS [NotNullColumn] FROM TABLE_NAME
# June 22, 2004 10:22 AM

Mike Prasad said:

How can I export a dataset from a WinForms app to Excel (preferably) or CSV. My users are using Excel 2000 and cannot upgrade - my development box has Officre XP installed.
# June 29, 2004 3:24 AM

Brendan Tompkins said:

Mike, you should be able to use the export utils. Open the ASP.NET project, and move the classes that do the exporting to your windows app. Combine the output string with a FileDialog, and you should be set.
# June 29, 2004 4:06 AM

TJ Kennedy said:

I have problems with the maxlength generated by the .xsl file being "-1" for dates and for bit fields. Then the file read craps out. Anyone else have this problem. I can manipulate the doc and it works fine, but I need it to work without my monitoring the file all the time.

Any Thoughts??
# July 23, 2004 3:54 AM

b0p0046 said:

what does one do if one doesn't have WinXP? I have Visual Studio 2003, Office 2000, and just the MSDE 2000 RelA SQL server engine, and no money to invest in more.

I want to read Excel attachments with an Outlook email into my sql database.
# August 24, 2004 7:13 PM

g51015 said:

How can I get the ture format of number type column in excel.I found it is all string type in all xsl
# November 4, 2004 10:13 PM

robustengine said:

We are reading an Excel workbook using the OLEDB Approach. The excel contents are read into a DataSet.



The problem we are facing is that, the contents of a column are getting truncated to 255 characters. Any idea of how to resolve this issue? We need the entire contents of the cell.



We have tried using MAXSCANROWS option of Extended Properties of the EXCEL but it does not work.
# December 10, 2004 1:35 AM

techSage said:

Is anyone able to use "Export to Excel" from the PivotTable successfully? I get an error, a blank excel sheet, and an entry in log file stating:
XML ERROR in PivotTable
REASON: Missing Tag
ACTION: Aborting current XML group
FILE: F:\DOCUME~1\<user>\LOCALS~1\Temp\PivotTable62219.HTM
GROUP: PivotTable
TAG: PivotField

The referenced .HTM file is empty. <user> is my Windows username. Any ideas?
# December 28, 2004 6:41 AM

BlackTiger said:

I have HUGE problem with this kind of transformation. This method incorrecly shows columns containing spaces and other non-latin-literal symbols - renames it to something like "_x0020_"

"Field 1" -> "Field_x0020_1"

Can I fix it?

# January 22, 2005 9:36 AM

Duray AKAR said:

How about multiple tables in a dataset?

please see my challenge in my blog...

Any ideas?

# February 14, 2005 12:27 PM

Larry said:

To BlackTiger:

System.Xml.XmlConvert.DecodeName should help you out.

# February 25, 2005 9:57 AM

Brett Green said:

To g51015:

You can export the DataSet schema along with the data itself. The following example writes it to a file, but you can also write to any stream.

ds.WriteXml("\test.xml", XmlWriteMode.WriteSchema)

This writes the schema definition inline inside the XML file along with the data. Then you can change the XSL to take advantage of this information to dynamically set the types of your Excel cells.

I have rewritten the ExcelXML style sheet to do just that and can make it available here if its wanted. Just need to know who to send it to.

# March 10, 2005 11:10 AM

yamazed said:

Hi,

I am trying to do exactly the opposite of what you are doing... :-)

~yamazed
# March 16, 2005 5:48 PM

yamazed said:

Hi,

Is there an easy way to convert ADO RecordSet XML back to ADO.NET style? I redid this example in Windows form... It works great except I could not get the Export to Excel doesn't function properly and the PivotTable needs some tweaking.

~yamazed
# March 17, 2005 1:27 PM

Swirl said:

To Brett Green,

I would love to see your ExcelXML style sheet that dynamically set the data types of the ExcelXML cells using the WriteSchema of DataSet.
Can you post it to swirlobt[at]yahoo[dot]com?

Thanks!
# July 19, 2005 7:16 PM

IanS said:

Brian, your suggestion, didnt work for me for some reason, below:
SELECT ISNULL(COLUMN_NAME, '') AS [NotNullColumn] FROM TABLE_NAME

So I altered your idea and used the DateTime.MinValue and checked for it in my application:
SELECT ISNULL(MYDATE, '0001-01-01') as mydate FROM TABLE_NAME

OR in my case as I was serialising the dataset:
SELECT ISNULL(CONVERT(VARCHAR(19), MYDATE, 126), '0001-01-01T00:00:00') as mydate FROM TABLE_NAME

Thanks for the suggestion, though.
# August 17, 2005 8:29 AM

Harry said:

I am wundering if there is only this solution to go from a DATASET to EXCEL. I have found a support article on MS where they describe how to create a new excel file from a database connection (KB316934). But in my case, I do not have a database connection, I only have a Dataset, so how could I achieve the same as in the KB with a dataset... Any ideas?
# August 22, 2005 4:27 PM

Alex said:

Hi Brendan

Have you managed to find out a way to import Excel 2002 Spreadsheet XML into a dataset?
If so pray tell us!

See here:

http://groups.google.com/group/microsoft.public.dotnet.framework.aspnet/browse_thread/thread/1ee844091288f27a/375b44aff0907f9e?lnk=st&q=postings@alexshirley.com&rnum=1&

Many thanks!

Alex
# October 26, 2005 5:45 AM

Peter C said:

To Brett Green,


Re your post from Thursday, March 10, 2005 11:10 AM:

Would be very interested to check out the ExcelXML style sheet you created to take advantage of DataSet schema definitions to dynamically set the types of Excel cells. Thanks for the insights and for offering to make this available to the community.

Can you post it to pchipello[at]yahoo[dot]com?

Many Thanks,
Peter C
# December 7, 2005 11:39 AM

Trevor Benedict R said:

Thanks for the Code. I have changed the code to use a DataReader instead of a DataSet and the performance is much faster with records exceeding 20000+ against an Oracle Database.

Regards,

Trevor Benedict R
MCSD
# February 16, 2006 10:02 PM

Sarmad said:

Good
# August 1, 2006 7:30 AM

Ace Balasador said:

my client didnt do a backflip =(
# August 9, 2006 2:52 AM

OU said:

TechSage, did you ever sort out this issue?

XML ERROR in PivotTable

REASON: Missing Tag

ACTION: Aborting current XML group

FILE: F:\DOCUME~1\<user>\LOCALS~1\Temp\PivotTable338245.HTM

GROUP: PivotTable

TAG: PivotField

I get exactly the same error.

# October 4, 2006 10:35 AM

symwilson said:

Brett Green,

I would really appreciate yout ExcelXML style sheet that dynamically set the data types of the ExcelXML cells using the WriteSchema of DataSet.

Could you post it to symwilson[at]yahoo[dot]com please?

If only I had scrolled down 2 days ago I could have saved myself hours of hacking away!

Cheers

# October 31, 2006 2:08 PM

Yura said:

I've applied a decision of an issue with NULL values.

This class processes dataset and creates XmlDocument.

When you call GetXml method - you will receive a string with all columns of the DataSet

Public Class XmlBuilder

   Private _processData As DataSet

   Private _doc As XmlDocument

   Public Sub New(ByVal processData)

       _processData = processData

   End Sub

   Public Function GetXml() As String

       If _doc Is Nothing Then

           _doc = New XmlDocument

           Dim header As XmlDeclaration = _doc.CreateXmlDeclaration("1.0", Nothing, "yes")

           _doc.AppendChild(header)

           Dim rootNode As XmlElement = _doc.CreateElement("NewDataSet")

           _doc.AppendChild(rootNode)

           For Each row As DataRow In _processData.Tables(0).Rows

               Dim tableElement As XmlElement = Me.AddElement(rootNode, "Table", Nothing)

               For Each column As DataColumn In _processData.Tables(0).Columns

                   Dim columnElement As XmlElement = Me.AddElement(tableElement, column.Caption, row(column))

               Next

           Next

       End If

       Return _doc.InnerXml

   End Function

   Protected Function AddElement(ByVal parent As XmlElement, ByVal name As String, ByVal innerText As Object) As XmlElement

       Dim element As XmlElement = _doc.CreateElement(name)

       If (Not innerText Is Nothing) And (Not innerText Is DBNull.Value) Then

           element.InnerText = innerText.ToString()

       End If

       parent.AppendChild(element)

       Return element

   End Function

End Class

then change DataSetTransformation.Util.ExportUtils.ExportDataSet

method so that first parameter was string instead of DataSet, and replace ds.GetXml() there with first string parameter.

Here is new signature:

public static void ExportDataSet(String processedDataset, string fileName, DataSetExportType type, HttpContext context)

# December 8, 2006 4:55 AM

Roland said:

Thanks. Saved me the effort :)

# May 1, 2007 1:01 PM

Roland said:

Thanks. Saved me the effort :)

Might suggest you change (forgive c# translation)

XmlElement rootNode = _doc.CreateElement("NewDataSet");

to

XmlElement rootNode = _doc.CreateElement(_processData.DataSetName);

And

  XmlElement tableElement = this.AddElement(rootNode, "Table", null);

to

  XmlElement tableElement = this.AddElement(rootNode, _processData.Tables[0].TableName, null);

Thanks again

# May 1, 2007 1:05 PM

Andrew Guard said:

How do I go from .xls to DataSet? Save as MS DataSet within Office Excel would be good.

# July 24, 2008 1:56 PM

Leave a Comment

(required)  
(optional)
(required)  

Enter the numbers above:
Add

About Brendan Tompkins

Brendan has been programming with .NET since the first public beta and is owner and operator of Port Technology Services, a consultancy company providing .NET application development services to the Maritime industry. In July, 2007, he was awarded the Microsoft MVP award for ASP.NET. He's also a proud co-founder of failed .COM startup Intrinsigo, and has had a hand in the failure of numerous other businesses. He currently runs CodeBetter.Com and Devlicio.us, and lives in Norfolk, Virgina with his wife Tiara and son Ian.

View Brendan's profile on LinkedIn

Check out Devlicio.us!

Our Sponsors