Softwaremaker

Threading and Throttling differences between WCF and ASMX

I know I havent been posting deep technical stuff that I used to do. Contrary to what people think my current role entails, keeping abreast of the technology landscape is what I am supposed to do and what I enjoy and so when colleagues joked with me when was the last time I booted up Visual Studio, for example, I enjoyed seeing their shocked faces when I told them: "oh - just last night. why you asked ?" i-wink.

I dont dwell deep like I used to but I still code up decent projects which I implement within my own developmental testing environment (yes, I have one running latest versions of Active Directory, SharePoint, Exchange and all the other goodies), driving the houshold crazy when I think of new and different ways to document expenses, publish a Book or CD library, home automation projects using all sorts of different technologies (yes, that includes Ruby-on-Rails) or in my new Windows Mobile 6 Device. Of course, I admit I dont post topics deep like I used to. It is not so much the content but more so, the limiting factor of time.

Recently, I was involved in some internal technical discussions with regards to the issue of scale comparisions between _WCF_ and ASMX. Below are some discussions:

If you have a web service that is going to be IO bound, you would definitely want it to be scalable and almost every resource in the world tells you to implement ASP.NET asynchronous pattern (BeginSomething/EndSomething, etc) on it so to go easy on the thread pool. ASP.NET uses an IAsyncHttpHandler to handle the request, which means the worker threads are not blocked while the IO-bound operation executes somewhere else. Sounds good so far.

If you make a WCF version of it with webHttpBinding (which actually means you can invoked it AJAX-style) following the same async pattern for the methods, you may find that each invocation of the WCF service eats up two threads – one for its ASP.NET HttpModule.ProcessRequest and the other for the actual IO. Ouch! You may think that this means your WCF implementation may end up eating all threads reserved for ASP.NET, which would indeed scale down the server

Is this true OR are we missing the complete picture ?

While the scenarios explained above are reasonable observations, it doesnt paint the complete picture. WCF does perform better scalability than ASMX.

• Threading:
  For ASMX, when a request comes in, it would be queued up immediately for async ASMX. So the thread is released for that request and a new thread will pick up the work item later.

For WCF, when a request comes in, we queue it up in WCF and let an IO thread handle the request. At the same time, the request thread-pool thread is held to wait for the request to complete.

Yes, WCF uses more threads than async ASMX. But there is a reason for this. Using asynchronous ASMX is dangerous and not really a good practice (and I have hinted at this many times in the many Web Service/ASMX presentations I have done over the past few years). While it does well at what it is supposed to do, it does trick the developer into a "false sense of security". Essentially, if you dont know how the ASP.NET blackbox works, you may find yourself thrown against the car wall when you take a hidden, unsuspecting corner at high speeds. It does not provide enough throttling for client loads. Basically the server takes all items and queue them up for later processing. The server does not have a good throttling mechanism to control the number of work items. To everyone else, it seems that the server is quite friendly to all clients. However, if the number of clients is unbounded, this is really bad. First of all, the server working set would grow unlimited due to unlimited requests queued up. Secondly, many client requests would become obsolete when it’s picked up by the server from the queue. The latter accounts for a a good set of problematic scenarios I have come across in my past consulting gigs with regards to high-load and high-transactional ASMX asynchronous implementations before I joined the borg.

Think of it as a side of the brain (that tells you that you are about to be full) not functioning properly when you sit down at a buffet table. You eat and eat and eat without knowning when to stop and then your ingestion/digestion system starts kicking in, you actually hit the wall. Hard. Literally.

• Server Throughput
  When you measure scalability, the most important measurement is the server throughput. That is, how many requests the server can handle per time unit? For async ASMX, it would be pretty fast at the initial phase. However, like the ingestion/digestion analogy I was referring to above - Once the server is in a steady phase (as when CPU is fully loaded), the throughput will go down because the server capacity has reached. You can compare the data between async ASMX and sync ASMX over the long run to see what I mean.

Also you would see higher memory usage of the async approach.

• ASP.NET Throttling
  That said, ASP.NET does have a throttling mechanism that is used for sync ASMX, which is the threadpool thread limit. The number of threads used to handle requests are bounded (http://support.microsoft.com/kb/821268). WCF uses this fact to throttle incoming requests. You can always change the configuration settings to increase number of threads to be used to allow more work items to be queued up.

The max number of threads follows the following formula:
MaxWorkerThreads x #CPU – MinFreeThreads
This is 12 by default on a single-proc machine.

• Two-level Throttling for WCF
  WCF leverages the ASP.NET threadpool throttling to throttle client requests. At the same time, WCF has its own item queue throttling. The former is throttled by the setting mentioned in the immediate above point, while the latter is controlled by WCF throttling settings (maxConcurrentCalls etc). ASP.NET can automatically adjust threads based on CPU loads so that you would always get full load of the server.

In this way, you may experience client failures because the requests are rejected at ASP.NET layer beforehand. So you can increase the ASP.NET throttling to get better experience. But eventually you would still be bounded by the physical server capacity, no matter whether you use async ASMX, sync ASMX, or WCF as mentioned above.

There is improvement work done in .NET 3.0 SP1 and of course, .NET 3.5 (beta 2 here), with the use of prioritized item queues. Do expect even-better WCF performance even in some of the common scenarios. Fine tuning minWorkerThreads will even give us even better results.

Thanks to Wenlong for helping out with the guidance and explanation. The complete scenario and the design principles for it will be published in greater detail in a MSDN whitepaper later. Do watch out for it.

Please sell us solutions, SingNet

I have been a SingTel and SingNet customer for the longest time and do recommend friends and family over sometimes. While I have no complains about the service, I do have questions about some of their front-line staff operations and capabilities.

Another incident struck today.

I decided to sign up for the ADSL upgrade package from the current 3500kbps to the ADSL2 6000kbps after my contract is up for renewal. On the day of service activation, things just weren't going well. To cut a long story short, the downtime and service disruption was almost 3/4 of the day (which meant no www and email access == no work gets done) and my router power-supply cable failed today for some reason (due to age, I believe) and luckily, I had an old power-supply cable lying around which was put to good use. When the ADSL2 modem arrived late, I hooked it up, expecting a simple plug-n-play.

I was wrong.

The SingNet technical service staff was at a loss to help me. The only thing they could do was tell me that I could hook up my single machine (I have 8 at home) to the modem directly and things work fine. That is not a solution. Solve my problem, SingTel. Dont you know how many people in Singapore uses a Router to share broadband bandwidth amongst the household. If ADSL bandwidth was made for 1 machine and 1 user, we dont need anything more than 256mbps. Their technical staff did 1 step further and said they supported only 2 brands of routers with their modems. Fine - given that there are hundreds of models of routers anyways. I qualify myself as a technical person and asked if I could have the settings  of those "supported" routers so I know how to tweak and configure mine to work with the new modem.

Me: "Could I have those config settings of those routers so I can see how I can configure mine"
SingNet: "Err...We support 2 brands X and Y of routers but I am sorry, the settings are NOT in my knowledge base so I cannot help you there..."
Me: "OK. What else can I do ? If it doesnt work with my router, I cannot surf so tell me what I am paying for again ?"
SingNet: "I am sorry, Sir but those are the only 2 brands of integrated Router-modem we support"
Me: "Huh ? Integrated Router-modem ? Are you asking me to throw away my VPN Firewall router so I can use your modem ? Duh !"

So, we have this pipe of 6mpbs of bandwidth flowing through, we share. It makes sense and is cost-effective. So, while looking at the PO of the delivered modem, I saw the vendor who actually was being subcontracted by SingTel to sell those modems so I decided to give them a call and it was great that their support-line was open till 2100 hours. This would have never happened in the Australia, US or Canada. i-smile

It turns out that SingNet has only subcontracted to bundle their modems with their ADSL service offerings and the modem they are bundling is actually a router-modem !!! which explains why it cannot work with my current router as both services and DHCP will clash. To cut another long story short, I had to do some research, together with this vendor, to see how we can disable the router function and use this modem mainly as a bridge. There is no user iterface. Guess what I have to do ?

I have to telnet into the device with the proper credentials and command-lined these:

_{Administrator}=>ppp relay
{Administrator}[ppp relay]=>flush
{Administrator}[ppp relay]=>..
{Administrator}[ppp]=>flush
{Administrator}[ppp]=>:eth
{Administrator}[eth]=>flush
{Administrator}[eth]=>:atm
{Administrator}[atm]=>flush
{Administrator}[atm]=>phonebook
{Administrator}[atm phonebook]=>flush
{Administrator}[atm phonebook]=>add
name = pvcInternet
addr = 0.100
:atm phonebook add name=pvcInternet addr=0*100
{Administrator}[atm phonebook]=>..
{Administrator}[atm]=>ifadd
intf = atmInternet
:atm ifadd intf=atmInternet
{Administrator}[atm]=>ifconfig
intf = atmInternet
[dest] = pvcInternet
[qos] = default
[encaps] = llc
[retry] = 10
[fcs] = disabled
[ulp] = mac
:atm ifconfig intf=atmInternet dest=pvcInternet ulp=mac
{Administrator}[atm]=>ifattach
intf = atmInternet
:atm ifattach intf=atmInternet
{Administrator}[atm]=>..
{Administrator}=>eth bridge
{Administrator}[eth bridge]=>ifadd
intf = snbbridge
:eth bridge ifadd intf=snbbridge
{Administrator}[eth bridge]=>ifconfig
intf = snbbridge
[dest] = atmInternet
[portstate] = forwarding
[retry] = 10
[vlan] = default
[prioconfig] = disabled
[ipprec] = disabled
[priority] = 0
[regenprio] = 01234567
[ingressfiltering] = disabled
[acceptvlanonly] = disabled
[mcastfilter] = disabled
[dynvlan] = disabled
[igmpsnooping] = enabled
:eth bridge ifconfig intf=snbbridge dest=atmInternet
{Administrator}[eth bridge]=>ifattach
intf = snbbridge
:eth bridge ifattach intf=snbbridge
{Administrator}[eth bridge]=>saveall
{Administrator}[eth bridge]=>:
{Administrator}=>dhcp server config state=disabled
{Administrator}=>saveall
{Administrator}=>exit
 
After this exit, this particular router-modem functions as a bridge mode and I was able to post this blog.

WTF ?!?!?!?! Are they expecting home-users and lay-men to be typing these commands into a UNIX firmware ? I consider myself to be fairly technical and yet I barfed at the idea of doing this. Goodbye Civilization, Hello Stone Age ...

... which brings me to my main point ...

I bought into an upgrade path. My old modem was A modem, nothing more, nothing less. I used a router to connect to it. An upgrade path should be seamless. But SingNet decided to have a field day with me ...

• Their technical service staff have no idea it is a router modem. They kept using the term "Modem" only so I had to run around in circles figuring out why it doesnt work in the first place
• It gave me a router modem, with no user interface and documentation on how to use it, much less how to enable it to switch to a bridging mode. Obviously, an upgrade path is given to customers who have been on a certain older plan for x amount of time and obviously had their way of doing things. To share a ADSL connection, most people (if not all) use a router. By introducing a router modem into the picture, SingNet is forcing customers to throw their old router away. People like me have VPN rules, Firewall rules, Port Forwarding rules in this router and if it functions as a wireless access point as well, absolutely NO ONE would throw it away.

That is not a solution. Solve my problem, SingTel. Give customers options, SingNet. Follow the lead of UOB. A customer like me, and I am sure there MUST be hundreds of me in Singapore, at least, who were on the older plan for an x amount of time uses a current router. Then give me a pure modem, please. Give what you gave me today to those people who are looking for a router modem. Have choices. Choices are good. No choices are bad.

To make things worse, after I got it all set up, I ran multiple FTP test plus the famous SpeedTest and found my speed to be about an average of 2900kpbs. This is only a fraction of my promised 6000kbps and even worse than my older 3500kpbs plan. If this doesnt get fixed in time, I will make sure everyone in Singapore hears about this though formal and informal complaints through chat rooms / forums and the press

Gosh - when are our service providers ever going to get it right ?

Speaking in our Singapore Government Public Sector Day - 22nd Oct 2007

This is targetted at our Singapore Government Public Sector customers and obviously this blog post is targetted at those customers who are reading this post. i-smile.

I will be speaking at the above event, which will be held at our office premises. As to the topic I am speaking about, this area is something which I have been focusing of late and it really is all about architecture and the alignment of it through businesss and information technology.

Aligning Business and IT for Greater Corporate Agility
For long, IT and Business has communicated. While they may be speaking in the same language but are they speaking in the same "dialect”?  In this session, William Tay will share his field experience as well as explore Microsoft's vision for service-oriented architecture:

• What are the best implementation approaches so that instead of being led by the business, IT strategies are driven by business needs
• How Microsoft supports this vision by way of an integrated platform, etc.

If you are coming for my session at the Enterprise Architecture Summit 2007 (18th Oct 2007), which coincidentally, is also targetted at the Public Sector customers (regionally, of course), the above event on the 22nd Oct 2007 will be a good sequel to my Enterprise Architecture Summit pitch where I will be discussing in greater details on the value of the Microsoft platform to implement those business capabilities.