Rick,

Having some trouble with COM on a production server. System runs well in file mode but if it crashes, I need to restart it. I wondered if there is an app you or anyone uses or perhaps a PowerShell script that will check every minute to see if our exe is running.

West Wind Web Monitor is old but still works... I use it on all my servers.

+++ Rick ---

Are you running the .NET Handler? We have a client who also had issues with COM so switched them File mode under .NET a couple of years ago. We have found it to be quite robust with no need for monitoring. AFAIK, they've never had to do manual restarts. We've had the whole server go down on a few occasions, but after rebooting the servers will reload automatically the first time the site is hit (which was not the case under ISAPI).

--stein

Hi Stan,

Thanks for the response. We have a multi tenant setup on this particular clients install which prevents me from using .NET handler although I am working on some code to make my multi-tenant work with it. COM mode is noticeably faster as well, so not sure that running in File Mode is a good long term plan.

Did you ever discover why the COM mode is not working consistently? In my world it seems to work fine for weeks then for what seems like no reason the server OS starts rejecting requests (403 errors). One system admin I work with believes that DOS attacks cause IIS to start rejecting requests and either you get serious about blocking rogue bots before the server or increase the size of the server to deal with the traffic floods.

Anyway, thanks for the response.

FWIW, I run all the apps I still run with WWWC in COM and never have any problems. That said though all the apps are fast response apps, meaning requests are always fractional second requests. I think COM gets in trouble when you have COM servers getting killed due to timeouts or application crashes which is much more likely to occur when servers are under load and have long running requests that could pile up and saturate the COM service pool.

The problem in that case isn't so much Web Connection - which does its best to manage the servers and failures - but the fact that the COM system starts getting unstable when servers are not shut down properly. IOW, if you kill a COM object there's some left over resources that eventually pile up and end up killing the system. COM objects get killed when timeouts occur or if there is VFP hard crash in code.

There are some workarounds for that if you have applications that are prone to crash or time out. You can set up the IIS Application Pools to recycle on a schedule. You can set this cycle interval to say an hour or even half an hour and IIS will shut down the application pool which shuts down the Web Connection app with it, and then restarts it. IIS also safely shuts them down waiting for requests to finish in the existing AppPool, while starting a new one and routing requests to the new one. It's a relatively low impact way to force the IIS ApplicationPool process to clean itself up on a schedule and never let things get out of hand. It's nice with Web Connection because new AppPool startup time with a Web Connection application is negliable (unlike most ASP.NET apps) as the IIS/.NET framework parts are very small and lightweight.

I've never used this approach on my sites, because I've not needed it for my apps, but used it for several customers with great results.

+++ Rick ---

Did you ever discover why the COM mode is not working consistently?

No. The problems persisted, even after we retained Rick to optimize their server configuration. This is a vertical market app running on over 100 different sites - it runs fine under COM for everyone else. So we don't think it's a code issue. And as noted, it works for this customer in File mode. But if we switched to COM, we'd start seeing errors after just a few minutes (so this does not seem to be the issue of COM getting unstable over time that Rick refers to). I don't remember the exact errors - as I recall the usual manifestation was that the pages would simply stop rendering - you'd hit a page and it would come up blank.

We initially switched to File as a stopgap measure until we could fix the COM problem, but since the solution proved difficult and since everything seemed to work great in File mode, we just stayed with it. It's been over 2 years and AFAIK they haven't had any issues. They are running 6 server instances and response times appear to be fine.

--Stein

Stein,

If I recall you were doing something similar to Michael which is using the ISPAI dll with COM and mapping the same DLL to multiple sub-domains/applications. I suspect that that is what specifically is causing problems for IIS and the way COM objects are mapped into the address space since you essentially have many sub-applications mapped into a single DLL. As far as IIS and ASP.NET is concerned there are multiple AppDomains, talking to a single DLL which is very likely problematic as the DLL is mapped into multiple separate address spaces.

I do think that with the .NET object this would work better due to the much improved threading model (each instance gets its own dedicated and persisted thread). Except that that exact model (multiple apps - single DLL and VFP instances) doesn't work for sharing a single DLL and COM pool for many applications. .NET always maps a new DLL into an application, whereas with ISAPI the win32 DLL can be mapped globally. This means with .NET there's no way to avoid one or more FoxPro instances per individual application/domain/virtual.

File based doesn't have these issues since there's no COM marshalling and Win32 DLL mapping involved and because the .NET DLL is pretty lightweight it can also easily run a separate copy for each application without massive overhead of many VFP instances per application.

This may have all worked at some point in the past when ISAPI was the primary interface to IIS, but so much changed in the IIS infrastructure and support for COM was either reduced or changed. The old ISAPI DLL relied heavily on the COM subsystem itself.

The .NET Handler is much simpler and more efficient with COM due to the pinned thread per instance approach which avoids the main COM issue of thread marshalling.

AFAIK, for my own apps and those for customers COM is very stable even at massive loads (there are a couple of customers who are running multiple clusters of WC servers with many millions of hits a day) as long as used in the standard DLL per app way.

+++ Rick ---

Your theory sounds plausible, but it doesn't match the reality of this particular situation:

• We are just running a single app in one subdomain at this site, and..
• It is running under the .NET handler

It is true that we were using ISAPI when this problem first showed up. My initial response was to switch them to .NET, thinking that would fix things. Unfortunately the issues persisted. But the fact that they were running under .NET meant that switching to File Mode was a workable alternative, so we've just left things at that.

--stein

Then I don't know.

If the app was failing in ISAPI in COM, and also failing in .NET in COM then the problem points at an error in the COM server - perhaps differences due to the security model of running under COM vs. running standalone or some operation that triggers UI and causes the COM server to hang. The fact that ISAPI and .NET are so different in how the COM servers are loaded and if you saw the same behavior does suggest something in the server causing the issues.

All that said, COM and File based do have some differences especially when it comes to potential UI operations (most of that can be resolved with COM Unattended Mode (SYS(2335,0)) which fails COM servers on attempted UI operations which otherwise might hang the COM server.

I can't recall what the problem was at the time - it's been a long time. FWIW, I'm not sure if this was prior to or after some of the newer server loading features were implemented where the COM servers load but don't initialize until the first request which provides much better error handling that in the original versions which failed during COM Init() and thus couldn't provide error information.

IAC - for new development, if the plan is to run under COM for production I always recommend testing frequently under COM right from the start of development to make sure the application is running and works. Then if there is a problem due to a specific application coding issue, it's much easier to track down problems. Not an option obviously for old systems but important.

+++ Rick ---

I do think that issue predated your updated COM error handling. It's occurred to me to upgrade that client and then have them try switching back to COM to see if we get any better info. But since they're happily running in File Mode I've just let the situation slide.

FWIW, this app gets regularly tested in COM prior to releasing our quarterly updates. It's also running under COM at over 100 other customer sites. The problems that forced us into File Mode have only showed up at this particular client's site - part of what makes this situation so baffling.

--stein