FoxPro Programming
Need dependable method for downloading image files from web site
Gravatar is a globally recognized avatar based on your email address. Need dependable method for downloading image files from web site
  JCarls
  All
  Feb 3, 2021 @ 09:16am

I'm using Windows's URLDownloadToFile to allow my users to drag and drop images onto a form. It works well with disk images and sort of okay with web sites: I keep running into URLs that say "it's a JPG" or "it's a PNG" and then the %$#$%# server delivers an unusable WEBP file instead. This can occur even though I can pop a problem message instructing them to try using right-click and "save image" as an alternative.

Because URLDownloadToFile requires a target file name, the result is a download error, apparently caused by my supplying a file with a "bad" extension. I checked my West Wind help file, but could only find FTPGetFile() as an option, which I assume means I would need an FTP server to log into, not a web site address—and that seems too complicated. This functionality could be used on any web site that might have product images (my clients are interior designers).

What I need is to be able to use the dropped data to download whatever file the server decides to give me "as is," then check its extension after the fact. I can then give my users a less generic problem message.

Is there a download solution in West Wind that I'm missing or just don't have in my version? On a related note, does the latest version of CopyImage in wwwAPI handle WEBP images? Thanks!

Gravatar is a globally recognized avatar based on your email address. re: Need dependable method for downloading image files from web site
  Tore Bleken
  JCarls
  Feb 3, 2021 @ 10:36am

Maybe my function filetype.prg may be helpful? I mean, maybe you can download the file with a bogus extension, and change it afterwards with the help of this function?

Please note that the function is really old, and I don't use it anymore, so it may have to be updated.

Function filetype
   Lparameters lcData
   Local lcReturn,lcContents, lnHandle
   Local laDummy(1)
   If Pcount()=0 Or Vartype(lcData)#'C'
      lcReturn=''
   Else
      If Len(Trim(lcData))<1000 And Adir(laDummy,lcData)>0 && File
         lnHandle = fopen(lcData)
         lcContents = fread(lnHandle,50)
         fclose(lnHandle)
      Else && Memory variab?e
         lcContents=lcData
      Endif
      Do Case
         Case Len(lcContents)<4
            lcReturn=''
         Case Left(lcContents,3)=Chr(0xFF)+Chr(0xD8)+Chr(0xFF)
            lcReturn='JPG'
         Case Left(lcContents,3)='GIF'
            lcReturn='GIF'
         Case Substr(lcContents,42,3)='EMF'
            lcReturn='EMF'
         Case Left(lcContents,4)=Chr(0xd0)+Chr(0xcf)+Chr(0x11)+Chr(0xe0)
            lcReturn='XLS'
         Case Left(lcContents,4)=Chr(0xD7)+Chr(0xCD)+Chr(0xC6)+Chr(0x9A)
            lcReturn='WMF'
         Case Left(lcContents,4)=Chr(0x4D)+Chr(0x4D)+Chr(0x00)+Chr(0x2A)
            lcReturn='TIF'
         Case Left(lcContents,4)=Chr(0x89)+'PNG'
            lcReturn='PNG'
         Case Left(lcContents,5)=Chr(0xD0)+Chr(0xCF)+Chr(0x11)+Chr(0xE0)+Chr(0xA1)
            lcReturn='DOC'
         Case Left(lcContents,5)='%PDF-'
            lcReturn='PDF'
         Case Left(lcContents,2)='BM'
            lcReturn='BMP'
         Case Left(lcContents,3)='CWS' And Asc(Substr(lcContents,4,1))<16
            lcReturn='SWF'
         Case Left(lcContents,3)='FWS'  And Asc(Substr(lcContents,4,1))<16
            lcReturn='SWF'
         Otherwise
            lcReturn=''
      Endcase
   Endif
   Return lcReturn


Gravatar is a globally recognized avatar based on your email address. re: Need dependable method for downloading image files from web site
  Rick Strahl
  JCarls
  Feb 3, 2021 @ 11:58am

You need to look at the content type from the result HTTP headers and that tells you what type of content you've downloaded. You can't do that with UrlDownloadToFile. You can I think with XmlHttp COM object although I can't recall if you get access to the headers or not...

Using the wwHttp class in West Wind Client Tools though you can do something like this:

DO wwhttp  && load libs

loHttp = CREATEOBJECT("wwHttp")

*** Download directly to file (file parm passed - otherwise return as string)
loHttp.Get("https://markdownmonster.west-wind.com/Images/MarkdownMonsterLogo.jpg",;
           "c:\temp\image.jpg")

*** All Http Headers
? loHttp.cHttpHeaders 

*** Retrieve Content Type: image/jpeg
? loHttp.GetHttpHeader("Content-Type")

wwHttp is a clean FoxPro wrapper around the same WinHttp APIs used by UrlDownloadToFile and XmlHttp, but it provides a lot more control over every aspect of the HTTP request process.

Hope this helps,

+++ Rick ---

Gravatar is a globally recognized avatar based on your email address. re: Need dependable method for downloading image files from web site
  JCarls
  Tore Bleken
  Feb 3, 2021 @ 12:14pm

Thanks! That may be useful here if I can get the file to download. If I drag one of these WEBP files masquerading as a JPG or PNG to a file manager, it downloads as a WEBP file, but trying it with URLDownloadToFile returns the INET_E_DOWNLOAD_FAILURE value (-2146697208) which is supposed to signify a bad URL, and does not create a file.

Gravatar is a globally recognized avatar based on your email address. re: Need dependable method for downloading image files from web site
  JCarls
  Rick Strahl
  Feb 3, 2021 @ 12:23pm

Thanks! I'll check into that.

Not sure if this is meaningful: With some of these instances, URLDownloadToFile doesn't even create a file. It works fine with a URL in which the server does not substitute a WEBP file, but with the "says 'jpg' but gives you a 'webp'" URL, it returns a failure value and no file is created. I caught it in the debugger and changed the target file value to a ".webp" extension, and it still failed. Yet, I can drag the same web image directly to a file manager and the WEBP file is created.

Gravatar is a globally recognized avatar based on your email address. re: Need dependable method for downloading image files from web site
  Rick Strahl
  JCarls
  Feb 3, 2021 @ 12:28pm

The UrlDownloadFile function in the WinInet API is old - it probably doesn't know how to map the WEBP image type to a file type.

You're better off manually downloading and deciding based on the content type what to save it as.

The content type is up to the Web Server, so the Web Server decides what the content type is. It may do this with content sniffing or by file extension (I think IIS will always do it based on file extension but other servers may do something else).

+++ Rick ---

Gravatar is a globally recognized avatar based on your email address. re: Need dependable method for downloading image files from web site
  Tore Bleken
  JCarls
  Feb 3, 2021 @ 12:41pm

Maybe you can use this code, which I have used successfully for ages:

*GetDataFromURL.prg
LPARAMETERS pcUrlName
DECLARE INTEGER InternetOpen IN wininet.DLL STRING sAgent, ;
      INTEGER lAccessType, STRING sProxyName, ;
      STRING sProxyBypass, INTEGER lFlags

DECLARE INTEGER InternetOpenUrl IN wininet.DLL ;
   INTEGER hInternetSession, STRING sUrl, STRING sHeaders,;
   INTEGER lHeadersLength, INTEGER lFlags, INTEGER lContext

DECLARE INTEGER InternetReadFile IN wininet.DLL INTEGER hfile, ;
      STRING @sBuffer, INTEGER lNumberofBytesToRead, INTEGER @lBytesRead

DECLARE short InternetCloseHandle IN wininet.DLL INTEGER hInst

#DEFINE INTERNET_OPEN_TYPE_PRECONFIG 0
#DEFINE INTERNET_OPEN_TYPE_DIRECT 1
#DEFINE INTERNET_OPEN_TYPE_PROXY 3
#DEFINE SYNCHRONOUS 0
#DEFINE INTERNET_FLAG_RELOAD 2147483648
#DEFINE CR CHR(13)

local lsAgent, lhInternetSession, lhUrlFile, llOk, lnOk, lcRetVal, lcReadBuffer, lnBytesRead

* what application is using Internet services?
lsAgent = "VPF 5.0"

lhInternetSession = InternetOpen( lsAgent, INTERNET_OPEN_TYPE_PRECONFIG, ;
      '', '', SYNCHRONOUS)

* debugging line - uncomment to see session handle
* WAIT WINDOW "Internet session handle: " + LTRIM(STR(hInternetSession))

IF lhInternetSession = 0
   WAIT WINDOW "Internet session cannot be established" TIME 2
   RETURN .null.
ENDIF

lhUrlFile = InternetOpenUrl( lhInternetSession, pcUrlName, '', 0, ;
                             INTERNET_FLAG_RELOAD, 0)

* debugging line - uncomment to see URL handle
* WAIT WINDOW "URL Handle: " + LTRIM(STR(hUrlFile))

IF lhUrlFile = 0
   WAIT WINDOW "URL cannot be opened" Timeout 5
   RETURN .null.
ENDIF

lcRetVal = ""
llOk = .t.

DO WHILE llOK
   * set aside a big buffer
   lsReadBuffer = SPACE(32767)
   lnBytesRead = 0
   lnOK = InternetReadFile( lhUrlFile, @lsReadBuffer, LEN(lsReadBuffer), @lnBytesRead)

   if ( lnBytesRead > 0 )
      lcRetVal = lcRetVal + left( lsReadBuffer, lnBytesRead )
   endif

   * error trap - either a read failure or read past eof()
   llOk = ( lnOK = 1 ) and ( lnBytesRead > 0 )
ENDDO

* close all the handles we opened
InternetCloseHandle( lhUrlFile )
InternetCloseHandle( lhInternetSession )

* return the URL contents
RETURN lcRetVal

Gravatar is a globally recognized avatar based on your email address. re: Need dependable method for downloading image files from web site
  Rick Strahl
  Tore Bleken
  Feb 4, 2021 @ 03:17pm

That won't give you the headers to determine what type of file you're dealing with...

+++ Rick ---

© 1996-2021