Hi all,

I'm happy to announce that the West Wind site has been moved to a new server over the last days. It's been a long time coming and a bit of a scary step going from physical hardware to an all virtual machine setup running on a single hosted Virtual Machine.

You may have noticed a few bumps and edges in the last few days as services were down for small stretches of time and a few resources weren't hooked up but overall I think the process went really well.

Really impressed with Vultr

My server has been running for nearly 10 years on a physical, bare metal box that I've hosted at a local ISP. It's my own box and West Wind Servers have been with this local ISP for as long as I've been online in 1995. They provided great performance and a great price for my servers - online perf was always awesome.

But - it was time to move on and I moved my server over to Vultr which is a low cost hosting service, that provides very high performance VMs for what is comparably low prices. I've spent quite a bit of time comparing services and Vultr tended to come up at the top for the performance / price ration - by a long shot.

You can check them out here:

• Vultr Virtual Machine Hosting

Why Move? From Physical to Virtual

My Hood River ISP has been good to me and running the physical box and local ISP has always worked out great. However, as the machine is now approaching 10 years I realized it's time to get off that physical box. Pushing past 10 years for spinning (10k rpm Raptor) drives is really pushing it and it's surprising that nothing has blown up to date. Even replacing drives for some SSDs probably still wouldn't give me peace of mind - I'd still worry about the motherboard, power supplies, onboard chips etc. to go bad. I've been meaning to move off for a few years, but frankly had always been put off by the price/performance ratio of Virtual machine solutions.

In the past when I looked and compared machines performance for a 4 processor/8gb/100gb machine was somewhere in the $100 range, which is fine. However, performance of those setups tended to be much worse than my 10 year old server. I'd get somewhere around 60% of the perf I got on my physical box on VMs. I've been trying out Azure, AWS and Google hosting and on all of them using comparable CPU setups to my physical hardware tended to be twice as expensive as my physical hosting and roughly 40%-50% slower. IOW - a terrible price/performance ratio.

A couple years back I checked out Vultr and even back then the performance was good. For $96 I could get a comparable machine as my ancient server, which is roughly the same I paid for my hosted physical box.

So this year I finally set up a new machine again with the intention of moving over my two busiest applications which is my Weblog and this message board. Between these two there's about 3/4 of million requests fired.

I started with a lower VM than I was expecting to finally deploy on - 4 CPU/8gb/100gb for $56 ($40 + $16 for Windows). My thought was to set up on the smaller VM, then snapshot and restore onto a bigger VM once everything was set up and running.

Turns out after setting up all my applications and apps, I don't think I'll move to the higher tier as needed. Performance of the new server is great and memory usage is not actually pushing the 8gb limit. This even with SQL Server, MongoDb and 20+ apps running on the single box, some of which (Weblog mainly) are using a huge amount of memory for caching.

If I sound like a commercial - it's because honestly I'm impressed. I figured when I finally move my servers to VMs I'd take a performance hit, instead it looks like now I'm getting slightly better performance and ended up paying half of what I paid before.

Not bad, not bad at all!

Upgrading

All in all the process of moving my machine from one to the other took about a day. This is very much a legacy server - there are over 20 Web sites/virtuals configured on this machine so getting everything configured is quite a chore.

Chocolatey

My biggest savior is Chocolatey - Chocolatey lets you install most system level, tools and utilities through the Chocolatey package store, which makes it easy to automate installations of most of the stuff I need to get onto a machine.

To install Chocolatey and Chrome I use a batch file:

@powershell -NoProfile -ExecutionPolicy unrestricted -Command "iex ((new-object net.webclient).DownloadString('https://chocolatey.org/install.ps1'))" && SET PATH=%PATH%;%systemdrive%\chocolatey\bin

REM Apps
call choco install GoogleChrome -y

Then to install most common apps using another batch file:

call choco install disableuac

REM Apps
call choco install vscode -y
call choco install filezilla -y
call choco install 7zip -y
call choco install git -y
call choco install tortoisegit -y

call choco install xplorer2 -y
call choco install procexp -y
call choco install deletefiles -y
call choco install MarkdownMonster

call choco install filezilla.server -y

call choco install sql-server-express
call choco install sql-server-management-studio

which handles the most common installs for me.

Next IIS installation with this power shell script:

# This script installs IIS and the features required to
# run Web Connection.
#
# * Make sure you run this script from a Powershel Admin Prompt!
# * Make sure Powershell Execution Policy is bypassed to run these scripts:
# * YOU MAY HAVE TO RUN THIS COMMAND PRIOR TO RUNNING THIS SCRIPT!
Set-ExecutionPolicy Bypass -Scope Process

Enable-WindowsOptionalFeature -Online -FeatureName IIS-WebServerRole
Enable-WindowsOptionalFeature -Online -FeatureName IIS-WebServer
Enable-WindowsOptionalFeature -Online -FeatureName IIS-CommonHttpFeatures
Enable-WindowsOptionalFeature -Online -FeatureName IIS-HttpErrors
Enable-WindowsOptionalFeature -Online -FeatureName IIS-HttpRedirect
Enable-WindowsOptionalFeature -Online -FeatureName IIS-ApplicationDevelopment
Enable-WindowsOptionalFeature -Online -FeatureName IIS-NetFxExtensibility45
Enable-WindowsOptionalFeature -Online -FeatureName IIS-HealthAndDiagnostics
Enable-WindowsOptionalFeature -Online -FeatureName IIS-HttpLogging
Enable-WindowsOptionalFeature -Online -FeatureName IIS-LoggingLibraries
Enable-WindowsOptionalFeature -Online -FeatureName IIS-RequestMonitor
Enable-WindowsOptionalFeature -Online -FeatureName IIS-HttpTracing
Enable-WindowsOptionalFeature -Online -FeatureName IIS-Security
Enable-WindowsOptionalFeature -Online -FeatureName IIS-RequestFiltering
Enable-WindowsOptionalFeature -Online -FeatureName IIS-Performance
Enable-WindowsOptionalFeature -Online -FeatureName IIS-WebServerManagementTools
Enable-WindowsOptionalFeature -Online -FeatureName IIS-IIS6ManagementCompatibility
Enable-WindowsOptionalFeature -Online -FeatureName IIS-Metabase
Enable-WindowsOptionalFeature -Online -FeatureName IIS-ManagementConsole
Enable-WindowsOptionalFeature -Online -FeatureName IIS-BasicAuthentication
Enable-WindowsOptionalFeature -Online -FeatureName IIS-WindowsAuthentication
Enable-WindowsOptionalFeature -Online -FeatureName IIS-StaticContent
Enable-WindowsOptionalFeature -Online -FeatureName IIS-DefaultDocument
Enable-WindowsOptionalFeature -Online -FeatureName IIS-WebSockets
Enable-WindowsOptionalFeature -Online -FeatureName IIS-ApplicationInit
Enable-WindowsOptionalFeature -Online -FeatureName IIS-NetFxExtensibility45
Enable-WindowsOptionalFeature -Online -FeatureName IIS-ASPNET45
Enable-WindowsOptionalFeature -Online -FeatureName IIS-ISAPIExtensions
Enable-WindowsOptionalFeature -Online -FeatureName IIS-ISAPIFilter
Enable-WindowsOptionalFeature -Online -FeatureName IIS-HttpCompressionStatic

choco install urlrewrite -y
choco install webdeploy -y

And that gets me most of the things I needed to install and configure and ready to install actual applications.

Installing Web Connection Apps

Web Connection applications in the past were a bit tricky to install because you had to pretty much do most of the configuration manually. However with Version 6+ the new configuration features make it much easier to install apps.

I simply restored my app folders from backups. Specifically this message board is an example. It still uses FoxPro data so everything is fully self contained in a single folder structure which gets copied on the new server.

So for example to install this message board I can just:

• Create a new Web site in IIS
• Ensure wwthreads.ini points at the right site
• Run wwThreads CONFIG to setup IIS and permissions
• Run!

I had to make one change to wwthreads.ini file to change the IIS Site instance (/12/ below):

[ServerConfig]
Virtual=
ScriptMaps=wwt,wc,wcs,wcsx
IISPath=IIS://localhost/w3svc/12/root

No other changes required I can now just do:

c:\> wwThreads.exe config

which auto-configures IIS and sets up folder permissions and registers the COM server.

In order to test the server locally I also ran the new 6.21 feature to set up local admin logins:

c:\> console.exe disableloopbackcheck

At this point the server should be able to start and take requests. The app started and I was able to run the application in file mode first, then switched to COM.

Compiled Templates Gotcha

I did run into one tricky snag: Template pages would end up crashing with mysterious errors that resulted in no output errors. Apparently when I moved the templates in the Web Folder, the templates were compiled with a slightly different version of the FoxPro runtime (a patched version) which is incompatible. WWWC was trying to use the FXP files and crashing. Deleting the prg and fxp files and letting WWWC recreate them made everything work again.

I used roughly the same approach for the Web Connection and Web Connection Weblog sites which are similarily self-configuring. Other than the script compile issue, the whole process of getting each Web Connection app up and running took less than 5 minutes.

Yay!

.NET Applications

The .NET applications are even easier because there's literally nothing to install. Each app runs in its own Web site so the only thing that needs to be done is create a Web site and configure the Application pool.

I do make sure to install:

• .NET Full Framework (latest version)framework

For .NET Core Applications:

• .NET Core SDK
• .NET Core Hosting Pack

Apps would pretty much come up without any sort of configuration which is as you would expect for .NET applications.

SQL Server Databases

The biggest time consuming issue in all of these processes however is getting the SQL data migrated over. This involves making current backups and restoring them on the new machine and then making sure all the users are properly added to the Sql Server and to each of the individual databases. SQL Server Database configuration was easily the most time consuming process of the entire migration.

MongoDb Databases

I also have a couple of MongoDb applications that are running on my server. One nice thing about Mongo is that it's basically a local folder install. There are no dependencies elsewhere on the system so 'installing' merely means copying the binaries and firing it up and installing the service. The actual restore and startup for getting mongo to run took all of 5 minutes literally.

Perhaps the most time consuming part for me is to restore backups which is done with command line tooling for which the documentation really sucks. I have batch files for most of the operations I need to do thankfully, but of course there were Mongo version changes and the files failed to work as is.

Odds and ends

My server is running close to 20 Web sites/virtuals and so it's a bit of a struggle to find all the little details that might have gone unnoticed. If you find something that doesn't work please let me know either on this post or a new post. I'm sure there will be a few loose ends that are missing.

Housekeeping

I also took the opportunity to remove really old and irrevelant content. I don't think the old FoxPro 2.5 articles and tools really need to be online anymore 😃

Lots of duplicated content etc. was cleaned out as well as old saved downloads that now mostly come in through Chocolatey.

All of this mainly because the new server is configured with 100gb of disk space, which is not a ton. Since I now run everything on a single box including backing up locally before sending backups to an Azure Cold Storage Blob, 100gb feels a little cramped.

If I need to upgrade to a higher Vultr plan, it'll most likely be because 100gb was just not quite enough space.

As it is the server is running at 70% of disk capacity. It's going to be critical to really clean up temp files, logs and other stuff to ensure the server doesn't run out of space. Unfortunately, there's no support for adding 'local' drives to a machine - it's either get a bigger machine plan, or use WAN drive on a storage machine that's not guaranteed to be in proximity.

Vultr Nits

This is one of the disadvantages of Vultr: The amount of plans they have are not super fine grained. So the plan I have now is the $40 plan which is perfect except it would be nice to have a little more disk space. Instead the next leg up is double the price for quite a bit more hardware (16gb/8/200gb) which is priced fair, but just way more hardware than I really need. More drive options would help.

The other thing is the odd payment system they use where you basically buy a fixed amount of credit up front. Rather than billing you monthly you use your credits until they run out. Doesn't look like there's an auto-refill mechanism which is kind of lame.

But it's minor. It's easy enough to pre-pay for a year and forget about it - I haven't run low enough but I suspect they'll let you know when your account runs too low on funds to renew.

Summary: Hooray

So I'm excited and this takes a huge load of my back. The worry of a server crash has always been in the back of my mind because of the old server hardware. If it happened when I'm on Maui at the very least it'd mean jumping on a plane and flying to Oregon to retrieve drives.

With the VMs I can take daily or weekly snapshots of the VM and at least hardware failure isn't a thing with VMs since a hardware failure on Vultr's part just causes them to spin up the VM on a different machine.

I'm also really happy with performance - this is a bit unexpected and Vultr announcements seem to point at recent hardware updates that have likely made this possible. Even last year when I checked last perf was not quite up to what I wanted but now things look good.

A little more peace of mind!

Resources

• Vultr Hosting

+++ Rick ---

I totally agree with you about VULTR, Rick. Except for one glitch a few years back, they've been great.

I thought I'd add that they do notify you when your account is getting low. And, IIRC, they will automatically charge your account with the card on file if your balance drops below zero. However, I can't say that for certain because I've always recharged my account as soon as they notified me it was getting low.

What was the glitch?

+++ Rick ---

Hi Rick,
As always, thanks for sharing! Migrating to new hardware is never fun. Looks like you have automated many of the installs, sweet!!

One question. There's no additional charge for network traffic with Vultr? Maybe that's normal, but I always thought/assumed the hosting provider charged extra if you went over a certain amount of traffic. No?

Thanks again,
Steve

Yes there are bandwidth limits. The $56 account is 4tb, the $96 account is 5tb. Overage is 1 cent per gb, which comes to $10 per terrabyte.

Not sure how they measure this really, but in the last week I've used 65gb of bandwidth which is less than 2%. So I don't think the 4tb limit is going to be an issue. My site has a fair bit of downloads and the blog has a ton of large images it serves for banners and content. I was worried about bandwidth because I had no idea what I was using before, but it looks like that won't be a big issue.

Incidentally the biggest hit to bandwidth since installations where the inbound traffic of all the installer downloads which I'm just breaking over with outbound traffic now after a week 😃 Apparently they charge either for higher inbound or outbound traffic.

+++ Rick ---

Hi Rick

For sending emails, the Vultr doc recommends setting up IIS SMTP server (Step 6: Setting up SMTP email).

Do you send emails using this feature?

PMFJI, Thierry. I have started using the smtp2go service for outgoing emails. Highly recommended if all you need is sending outgoing mail and handling bouncebacks. I'm using that for most of my clients.

If you need something for incoming mail and mailing list management, I've found it beneficial to keep my mail server on a different IP address (if not a different server) to better cope with blacklists, security, etc.

No I use an external service: MailGun.

There's no way in hell I'd run my own mail server these days - it's a full time job trying to maintain decent email reputation for IPs to have reliable deliverability. For low volume MailGun (and others like SendGrid) are free.

+++ Rick ---

Thanks Michael and Rick for your insight

We use SendInBlue for bulk emails, another SMTP server from the hosting company for wConnect Admin e-mails (subject of my question). As it's running on the same network, the latter has no security constraint (no encryption, 'SSL', not even login/PW).

We'll probably use SendInBlue for all emails; I just need to figure out the server address and what protocol are supported.

Rick,

Mailgun API supports HTTP and SMTP; which protocol do you use for sending wConnect messages?

I use SMTP because existing code already uses that and it's more portable in case I switch to something else later. The Mailgun HTTP API is simple though so either would work without too much effort.

I did have some issues with Mailgun initially. They assign you an IP address that mail is sent from, and the IP was already blacklisted for some domains. Any mail sent to Microsoft domains would not go through. They assigned a new one after I complained, but it's important to check your initial mail traffic.

+++ Rick ---

Rick,

In the mailgun docs they have this c# example of sending a message. We use mailgun as a backup to loIp.sendmail() and I was curious what the secret is to getting an attachment added to a message in mailgun?

I assume we need to encode it somehow.

Thanks in advance if you have time for a suggestion on how to achieve.

You can't use the API with .SendMail(). But you should just use their SMTP interface to send emails.

+++ Rick ---

Can't is such a big word! I am using their api just fine now, its just the attachments that's not working. If I can figure it out i'll post the code here.

Thanks.

Yeah sure you can use the API - just not with .SendMail().

Why would you though? Using SMTP lets you switch between different providers - using the API you're locked into MailGun.

+++ Rick ---