We have a situation where a West Wind app we wrote has a memory leak. While we should try and fix the leak, so far that appears to be a very big job. So what we'd like to do instead (or in the interim), is to have a second instance start, and then "tell" the instance which was already running to shut down. Can you give us any advice as to how we could accomplish that?

Thanks Charles

Uh, how do you know - or more precisely, how does your application know which instance has a memory leak?

If you're running under COM killing a server will automatically start a new one on the next request. You can do that manually from the Admin page (on the bottom where you can see a list of running service along with the ability to kill each instance individually).

It's possible to detect I suppose - you can use .NET to get a process list and look at current memory usage then call the Kill() method to kill the process and then let the Module reload the class - but that's wrought with other issues in that the COM server won't quit cleanly and eventually after many process kills along this line will corrupt the COM system.

However, frankly it's probably just easier to shut down all running instances and just restart the pool. There's no 'signal' to shut down specific instances. In fact, COM instances can't even kill themselves - they have to be either killed or explicitly shut down through the Web Connection module/ISAPI since that's what's holding the COM reference.

My recommendation - if you know you need to restart an instance, restart the entire pool which can be done by simply firing an HTTP request.

+++ Rick ---

COM instances can't even kill themselves

even using _VFP.quit() ?

You can't QUIT a COM server, no... There's no entry point function that can 'exit' since COM instantiates a class which is held by the COM server (ie. the Web Connection Module or ISAPI handler).

I can't remember exactly what happens when you QUIT though - but for sure the COM reference doesn't go away.

+++ Rick ---

Rick

Thanks for the help here! I appreciate it.

We know we have a memory leak because the memused in the request log keeps going up, up, up. Eventually the program crashes. So we came up with a bat file which runs on the scheduled, determines the PID of the running exe, launches a second exe, and then does a taskkill on the first instanced. Crude, but worked. Then we discovered that by using taskkill, if the instance was in the middle of writing to tables, it never finishes, causing corrupt data. So that's why we want a more "elegant" way to shut it down.

We are running in File mode. I'd rather not switch to com, because the developer who worked on this is not able to help trouble shoot it if there are issues. I am a novice West-Wind developer. My plan is to rewrite the whole thing, and run the new version in COM. For now, we just need to solve the existing problem.

I don't think shut them all down and restating is viable, because there are anywhere from 2-6 active users all the time. If we shut down the exe gracefully and then started up another one I fear they would get timeouts in the interim. When we send in the request to shutdown, it takes a while (like 30-60 seconds). In saying this, I am wondering if the developer simply put in an inkey() to make it wait, and if so, we could find that and shorten the delay (or eliminate it).

It looks like most of your comments are geared towards us running in COM mode, which as I've said, we don't.

Any other thoughts? Thanks Charles

There's no graceful way to shutdown a running file based instance externally. Your only option is to TASKKILL the process externally. But if you're running in File mode you can shut down your server with a QUIT operation.

In that case it's easy to do one of the following:

• Add a request that can be called from a monitor that checks what memory usage is, and if it's over a certain level, starts a new instance (simply RUN or HTTP admin request to launch a new instance) and then runs QUIT. This may not be 100% reliable if you have multiple instances but if you hit it often enough it should catch memory usage eventually on all instances. This is the least intrusive approach.

• Do the same as above, but inject the check/restart code on a request that happens often but not very frequently. You can use EVALTIMER() to delay execution of that code until after the current response was returned.

• You can inject logic into the Web Connection timer code to check in specified intervals whether the memory usage is high and then do the start new instance/quit operation.

FWIW, none of the above work under COM because you can't

I'm thinking it would be nice to add a Web Connection admin call that allows to shut down a specific COM instance by process Id. something like:

http://localhost/wconnect/ShutDownInstance.wc?pid=1234

+++ Rick ---

When you say “Add a request that can be called from a monitor” are you talking about the West Wind web monitor? I believe we purchased that a long time ago but I have never used it. If it could be used for this, then I want to look into it.

Alternatively, we already have the ability to send in a request to quit. However, I do not know how to direct that request to the instance which has been running for a while and is approaching the point of crashing due to the memory leak.

In my mind, I can envision doing the following, but I do not know how.

Add code to the program to determine the PID it is running in when it starts and store that PID value to a global variable or application object.

Have the scheduled bat file:

1. determine the PID of the currently running instance
2. start a new instance
3. send in a quit request with a parameter containing the PID of the first instance
4. if the first instance gets this hit, have it compare the stored PID value with that of the passed parameter, and if they match, then quit

The problem with the above, is that the passed in request might be picked up by the second instance.

So I am back to how can we gracefully shut down the first instance?

Thanks Charles

As I said there's no way to gracefully shut down instances because the pool manager doesn't explicitly know what instance a request is going to. For Web Connection modules and COM there will be a new feature that provides that ability, but there's no real way to do this in file based.

You can go through each of the Web Connection processes running and check their memory usage.

if you're running file based you maybe able to do this via Win32 Messages - you can SendMessage() to a specific Window handle although I'm not 100% sure how you can get that externally

in .NET you can do:

	foreach (var process in System.Diagnostics.Process.GetProcesses())
	{
		if (process.MainWindowTitle.Contains("Web Connection"))
		{
			Console.WriteLine($"{process.MainWindowTitle} - " +
			"WH: {process.MainWindowHandle} - " +
			"PID: {process.Id} - " +
			"Mem: {process.PrivateMemorySize}");
			
			// WM_CLOSE sent
			process.CloseMainWindow();						
		}
	}

You can call this sort of code with wwdotnetbridge as well. The problem is that WM_CLOSE isn't going to kill the application because it's sitting a READ EVENTS loop, so you'd have to BINDEVENTS() in FoxPro to the Windows events and pick out the WM_CLOSE message, kill the event loop and then QUIT.

There are lots of ways to do this but they are all ugly. The way to deal with this is to fix the memory leak 😃

All doable but a lot of effort when you really could just restart the pool if one of hte instances goes bad.

+++ Rick ---

After giving this some more thought, it occurred to me that perhaps we can simply send in a request to shut down, and then once that instance has finished, launch a new one. We wrote a batch file to do this, the code is below. The problem is, once the request has been submitted, further requests get a "website is shutting down" response. My hope was that their requests would get held in a queue until the current instance quit, and would be processed by the new one. So now I am wondering if I can change the behaviour of the shutdown down request to work as described above, as opposed to responding with that message. The other thing which I'd like to change is the length of time it takes to shut down. It seems to be waiting 30 seconds. Please note that I am a West-Wind "newbie", and in fact not a strong VFP developer, so I am struggling with how to find where this code is in our app. I do not even know if this is something we coded, or is part of what comes with West-Wind. Any comments you might be able to share on this matter would be greatly appreciated.

Here's the batch file:

REM Set Variables set pName=webpics2.exe set pPath=C:\WW\WebPICS\WebPICSWA\

REM Find the process ID for /f "tokens=2" %%a in ('tasklist^|find /i "%pName%"') do (set PID=%%a)

REM Terminate the Process C:\WW\WebPICS\WebPICSWA\exhttpget9.exe "http://CHIOFT-PICS/webPICS/webpics2quit.p2so?&password=webpda"

rem pause

REM Loop to wait for the process to finish 😛ROCESS_CHECK tasklist /FI "PID eq %PID%" 2>NUL | find /I /N "%pName%">nul if %errorlevel%==0 goto PROCESS_CHECK goto PROCESS_START

😛ROCESS_START REM Start New Instance start %pPath%%pName%"

As I've said before - if you're running in file mode there's no control over the server instance from the server. That's why you should be running in COM. COM can manage instances and shutdown procedures because it has an actual reference to the instance and can control it remotely.

In file mode the only option for a controlled shutdown is to hook in the timer loop at certain intervals (not on every timer tick!) and check memory usage after the request has run. Then if the usage goes over a certain limit you can explicitly turn off the timer, CLEAR EVENTS, then QUIT to exit the current server cleanly. Before you do, you can use RUN or wwApi::CreateProcess() to launch a new instance of the same server that you just killed.

The code for this will be in the wcVisual class and the wwFileTimer class I believe. You can subclass the timer class and then set Server.cTimerClass on your new class during server startup in (or as your server properties).

This is off the cuff - I haven't tried this but something along these lines should work:

*** add a property to custom timer class 
tLastCheckTime = DateTime()

*** Override the Timer method
function Timer

*** Run default request checks
DoDefault()

*** Only do your check after 5 minutes
IF (THIS.tLastCheckTime + 300 < DateTime())
    THIS.Enabled = .F.

    *** Do your check - .f. if you need to restart
    IF (!THIS.CheckForMemoryLeak())
      THIS.StartNewInstance()  && your code to restart instance
      CLEAR EVENTS
      QUIT  
    ELSE
      THIS.Enabled = .T.
    ENDIF
    THIS.tLastCheckTime = DateTime()
ENDIF

ENDFUNC

Then in your server class' declaration:

cTimerClass = "MyCustomFileTimer"

+++ Rick ---

Get a list of instances of the running application from DOS. You can do this on startup, or before:

tasklist /svc /fi "imagename myprocess.exe" > mytextfile.txt

Have your app read the file to extract the PIDs from the text file, and then delete it.

Have your app check it's own pid (see http://www.news2news.com/vfp/?function=222) to exclude it, and write a tag file for all other PIDs using the PID's you want to kill.

On each hit in your running application, check to see if the tag file matching it's own PID exists. If it does, set a property, service the last request, and gracefully shut down. You don't want to shut down before processing the current request.

I'm not sure where in code to put the 'gracefully shut down' part. Whatever runs at the end of the request...

You don't need to run anything into a text file - there are a numerous APIs to read process lists - the easiest being the .NET Process class which provides all sorts of info, plus the ability to actually shut down applications (gracefully using the CloseApplication() and hard Kill()).

The issue is and remains that externally you can't gracefully shut down an application. Even if you send a request with a PID to the pool there's no guarantee that hte instance you're trying to close actually will handle that request.

I'm going to look into BINDEVENT() and handling shut down events properly which would actually be nice for a number of things (like shutting down dev applications running a Web Connection server with having the dreaded - Can't quit Visual Foxpro error message.

+++ Rick ---

Here's some more info in a blog post that discusses some of the issues that revolve around intercepting shutdown requests of Web Connection servers from Windows Messages.

• Shutting down a file-based Web Connection Instance through Windows

+++ Rick ---