Web Connection
Best way to transfer data using webconnect
Gravatar is a globally recognized avatar based on your email address. Best way to transfer data using webconnect
  Ron Jasper
  All
  Aug 27, 2025 @ 08:33am

Hi Rick, I'm having to send data from an inhouse server to the webconnection server. What's the best way to automate this (it has to be done every 5 minutes). There are about 28,000 records and the total size (including the dbc and individual dbf tables) is around 5 - 6 megabytes. Should I send JSON payloads or use some other method. I have to get the data to the webconnection server from the client's inhouse server. As always, thanks for your help.

Gravatar is a globally recognized avatar based on your email address. re: Best way to transfer data using webconnect
  Rick Strahl
  Ron Jasper
  Aug 27, 2025 @ 09:24am

For 5-6 megs of data JSON is probably fine. If it's much bigger you might want to send binary files as a Zip file:

  • Run a query to export the data
  • Package the dbf/fpt into a zip file
  • Upload
  • Unpack on the other end
  • load the file
  • Delete the temp files

While all that sounds complicated and slow it's actually pretty fast and likely much faster than serialization of a large data set.

The problems with JSON are:

  • JSON bloats size so the size'll likely be 1.5x or more larger
  • JSON exports fine, but for imports you have to provide the data structure in order to import back
  • You need to use CollectionToCursor() to convert back

+++ Rick ---

Gravatar is a globally recognized avatar based on your email address. re: Best way to transfer data using webconnect
  Ron Jasper
  Ron Jasper
  Aug 27, 2025 @ 11:16am

Thanks Rick, appreciate the advice.

© 1996-2025