Geeks With Blogs

News

  
Bill Evjen's Blog Code, Life and Community

I have been working in the area of Web services for some time and  I am on a project where I needed to distribute some extremely large datasets to some clients (Windows Forms). So, imagine an interface where you need to request information about an entity and you have a service where you can send in an array of entities to then get back an array of result sets about each one of the identifiers passed in. We had one such application and it worked quite nicely in that we built some ASP.NET 2.0 Web services to push out the SOAP messages to the client in just this scenario. It worked so well since we were only sending in 30 identifiers at a time but then we really decided to test it out when we put in 1,000 - 5,000 identifiers into one request. This in return was giving us back tremendously large SOAP messages and in many cases our Windows Forms application was simply timing out. How to approach this we thought. There are many methods of approach and we had to pick one and go with it as we had to support this scenario for our clients.

Our options included:

  • Using the Windows Communication Foundation and building services that pushed binary objects around instead of using SOAP.
  • Using .NET Remoting instead and using the binary over HTTP that it offers.
  • Using an HTTP compression class technique along with SOAP Extensions to compress the SOAP message before it went out across the wire.
  • Using the HTTP Compression feature found on the Windows Server 2003 box with our standard ASP.NET Web services
  • Using WSE 3.0 MTOM capabilities to encapsulate our messages into binary objects to send across the wire

There are a lot of options, but we quickly narrowed them down. First off, we are not able to use the WCF yet as we are unable to put this on the client at this point in time as we are supporting so many operating systems that are below the required Windows XP SP2. .NET Remoting ... maybe, but we really wanted to stick to the SOAP model for later routing purposes. Soap Extensions - a possibility, but not the easiest to implement and would require us to put a decompression class on the client. HTTP Compression - that's a possibility. WSE 3.0 - another interesting stance, but would require us to put the WSE 3.0 on the client - something we weren't too interested in doing.

So, we narrowed our tests down to .NET Remoting as well as HTTP Compression using Windows Server 2003. We built a test application that first did a comparison between .NET Remoting and standard ASP.NET 2.0 Web services with no compression enabled. The tests were interesting. First off when a single identifier was sent in and a tiny SOAP message was returned - there really wasn't that much difference between the two. The results were pretty much the same. Though, when sending in 1,000 identifiers .NET Remoting considerably outperformed ASP.NET 2.0 Web services. This was due to the fact that the large SOAP message was being sent as binary over the wire and the entire serialization and deserialization process was eliminated. The results were that .NET Remoting performed by shrinking the time it took to get the response to less than HALF that of the ASP.NET 2.0 Web services approach.

Then we turned on HTTP Compression on the Windows Server 2003 box. Instructions for this can be found at http://support.microsoft.com/?id=322603 but you want to do this for .asmx pages. Though, you will discover that enabling compression through IIS is not the only step you are required to take. In order for the HTTP message to be compressed on the server, you are going to need to get the client to ask for the message to be compressed. This means that the client will need to send a accept-encoding: gzip; in the HTTP header. By default, proxy classes don't do this for you. Instead, you are going to have to force the client to make this type of request by using the new EnableDecompression property in .NET 2.0.

Fundamental.Fundamentals_1_0 proxy = new Fundamental.Fundamentals_1_0();
proxy.EnableDecompression =
true;

After you instantiate the proxy class, you simply set the EnableDecompression property to true and you are then good to go - the proper HTTP header will now be included. Enabling our client to do this and sending in the 1,000 identifiers we were amazed that we could just about replicate the .NET Remoting performance and in some cases we could match it. This showed us that we could deliver our large SOAP messages over HTTP and get away with doing the littlest impact to the client that was possible (as far as new installs of any software or code changes to the client app).

Other scenarios may of course warrant different approaches - I'm just describing what worked for us in this case. It was a good lesson.

Posted on Wednesday, September 13, 2006 1:33 PM ASP.NET , Microsoft , MSDN , HOW-TOs | Back to top


Comments on this post: Web Services - Sending Large SOAP Messages

# re: Web Services - Sending Large SOAP Messages
Requesting Gravatar...
I too face the similar problem.What we did is to serialize the entrie dataset to byte array and transmits it through the wire.Before serializing we did convert the datasets to arrays and used gzip copression after serialization.But I feel the perfomance canbe improved,as we are all greedy by nature.

I consider .Net remoting,but fears about any setback .I don't know ,if the above stated technique ,could be adopted by remoting effectively.Anyway I am on the tests.Since we are using ASP.NET 1.1 ,I haven't considered WSE 3.0.

WSE2.0 has method to implement TCP channel in webservice.That also has to be checked out.

Any way nice article.if you have any suggestions,please.
Left by Ajith on Oct 08, 2006 9:21 PM

# re: Web Services - Sending Large SOAP Messages
Requesting Gravatar...
this is not enough. The http compression in HTTP 1.1 only works for the response... the request is not encoded.
Left by JF on Jan 25, 2007 9:36 AM

# re: Web Services - Sending Large SOAP Messages
Requesting Gravatar...
.NET 2.0 Docs say EnableDecompression is defaulted to true so if the server is setup for it it should work without explicitly setting it to true.
Left by PL on Apr 08, 2007 9:42 AM

# re: Web Services - Sending Large SOAP Messages
Requesting Gravatar...
I too face these similar problem. But on thing I want to know how can I ensure that the response received is compressed. i am using .Net 2.0 and IIS 5.0.
Left by Madhu on Apr 12, 2007 12:52 AM

# re: Web Services - Sending Large SOAP Messages
Requesting Gravatar...
I too face these similar problem. But on thing I want to know how can I ensure that the response received is compressed. i am using .Net 2.0 and IIS 5.0.
Left by Madhu on Apr 12, 2007 12:52 AM

# re: Web Services - Sending Large SOAP Messages
Requesting Gravatar...
Check out Fiddler at http://www.fiddlertool.com.
Left by Bill Evjen on Apr 19, 2007 4:49 AM

# re: Web Services - Sending Large SOAP Messages
Requesting Gravatar...
The http compression in HTTP 1.1 works for the response AND the request.

When using http 1.1 the connection will remain open normally between the client and web server. Both client and server know that compression is ok for each other having already established this on earlier communication on the same connection so both will compress messages where appropriate.

Works a treat. Yay!
Left by Kim on May 09, 2007 3:27 AM

# re: Web Services - Sending Large SOAP Messages
Requesting Gravatar...
THE EASIEST is to implement SOAP in general using SOAP with attachments (Java implementation server and client the best). Entirely forget .NET1.x. MS does not handle attachments - it needs data section as part of SOAP body, which creates a problem for SAAJ in Java.
Left by Roman on Jul 09, 2008 4:46 PM

# re: Web Services - Sending Large SOAP Messages
Requesting Gravatar...
I also had this problem after lot of research came to the conclusion that if the dataset is huge nothing can be done, even compression or remoting not solving the problem only option would be paging. Still web service technology has to improve lot in terms of performance.
Left by Raja.S on Jun 23, 2009 2:19 AM

Your comment:
 (will show your gravatar)


Copyright © Bill Evjen | Powered by: GeeksWithBlogs.net