Web Connection
How to include .key & .pem in wwHTTP request
Gravatar is a globally recognized avatar based on your email address. How to include .key & .pem in wwHTTP request
  Scott R
  All
  Jun 26, 2024 @ 01:08pm

Hi Rick,

We are using web connect as a rest API but we are needing to authenticate with another API and their authentication is whacky. They force you to create an SSL cert with their Certificate Authority and then you have to pass that cert with your calls.

The curl example they give is:

curl --cert <your_certificate>.pem --key <your_key>.key -d 'grant_type=client_credentials&client_id=<your_client_id>&client_secret=<your_client_secret>' https://accounts.adp.com/auth/oauth/v2/token

I asked CoPilot to convert the code to a js fetch equivalent as I can usually then convert it to wwHTTP from there. However, I am at a total lose for this one. The code it gave me was:

const yourCertificate = fs.readFileSync('<your_certificate>.pem');
const yourKey = fs.readFileSync('<your_key>.key');

const agent = new https.Agent({
  cert: yourCertificate,
  key: yourKey
});

fetch('https://accounts.adp.com/auth/oauth/v2/token', {
  method: 'POST',
  body: 'grant_type=client_credentials&client_id=<your_client_id>&client_secret=<your_client_secret>',
  headers: {
    'Content-Type': 'application/x-www-form-urlencoded'
  },
  agent: agent
})

I have no idea if the code it generated is right but how would I go about passing in the .key and .pem using wwHttp? I've tried the following (mostly out of hope it works) but haven't gotten anywhere...

oHttp.addPostKey('key', fileToStr(m.keyFile))

oHttp.addPostKey('key', m.keyFile, .T.)

oHttp.addHeader('key', fileToStr(m.keyFile))

Any thoughts?

Gravatar is a globally recognized avatar based on your email address. re: How to include .key & .pem in wwHTTP request
  Rick Strahl
  Scott R
  Jun 27, 2024 @ 08:50am

You're going to make me look that up, aren't you? 😄

Certficates are handled by the HTTP clients so you can't compare JavaScript to WinInet (what wwHttp uses under the covers). Windows has a very complex way to deal with certificates made worse that it uses its own non-standard (ie. non OpenSsl) certificate formats by default so certs need to be converted first for many operations.

wwHttp supports passing certificates that are registered in the certificate store by their certificate index which you can do with GetCertificates() (for the list) and (interactively) with SelectCertificateByIndex(). But that only works with registered certificates that are in the store, not loose certificates from disk and there's no support for that built in. It's supported in WinInet but it's an ugly and ill-documented process.

Offhand I don't know how to do this I need to look into it. I know we can get the certificate binaries easily enough through .NET and probably convert them, but not sure how to feed those to the WinInet APIs.

If this is a one off call it might be easier to run the entire request through .NET and handle the certification aquisitiona and HTTP call from there and call that from FoxPro...

+++ Rick ---

Gravatar is a globally recognized avatar based on your email address. re: How to include .key & .pem in wwHTTP request
  Scott R
  Rick Strahl
  Jun 27, 2024 @ 09:01am

Hi Rick,

I was wondering if that would be the case. I'll figure something out using another tool then as this is the only API I've ever come across that's requiring this. Thanks.

Gravatar is a globally recognized avatar based on your email address. re: How to include .key & .pem in wwHTTP request
  Rick Strahl
  Scott R
  Jun 27, 2024 @ 09:55am

If you can get the certificate into the store then you can use it with the existing wwHttp code - I think. To be honest I've never actually gone through this as I never had a valid use case with private certificates to test this on. Interesting that this never came up in the 25+ years that this library has been around.

Private certificates suck - it's an awful process to work with which is why it's not done much anymore. It was very popular with SOAP services, but luckily all that's mostly gone by the wayside. It's painful even if the platforms (like your JS code or .NET) support it.

+++ Rick ---

Gravatar is a globally recognized avatar based on your email address. re: How to include .key & .pem in wwHTTP request
  Rick Strahl
  Scott R
  Jun 27, 2024 @ 10:04am

This actually got me thinking about how dated wwHttp and WinInet is. But wwHttp has been super stable and reliable and has rich support of Http features over the years, it's just a few fringe things that are not supported. Adding those however is a major hurdle.

I've been thinking about rebuilding wwHttp using .NET Http stack which is modern and supports all features, but I'm more afraid of breaking things that are working well today.

+++ Rick ---

Gravatar is a globally recognized avatar based on your email address. re: How to include .key & .pem in wwHTTP request
  Scott R
  Rick Strahl
  Jun 27, 2024 @ 11:05am

Hi Rick,

We've been pretty happy with wwHTTP overall. Minor things like this we'll figure out a workaround. The downside with this private cert is we a group of servers in a round robin and so adding it to the cert store may or may not be super feasible.

Not sure changing wwHTTP up at this point would be worth it as it's not really missing any main features except for weird one of stuff like this and it has been very reliable.

On a side note, I saw a thread with you and someone else on problems with the comm servers after an app pool recycle. We have the same issue and when you connected in and looked at it with us a few years ago the 'answer' was to stop at 15 comm servers (we do a lot of sql connections and are average request is anywhere from .5-3 seconds depending on how much SQL querying we have to do). We still have occasional problems when the app pool recycles but it usually seems to pull out of it but every day we have multiple errors of 'Unable to retreive a com server from the app pool', etc. Does your webconnect version 8 help with that? Trying to decide if it's worth it to spend time updating right now (we're running version 6.21 right now)

Gravatar is a globally recognized avatar based on your email address. re: How to include .key & .pem in wwHTTP request
  Rick Strahl
  Scott R
  Jun 27, 2024 @ 07:17pm

Version 8.0 (just released) addresses the COM load/unload issues. A lot of work has gone into making that process faster and much more reliable over the last few weeks and I think that's paid off. The release post has more details:

With these changes you should be able to run more instances too as the load/unload is much quicker and servers start processing as soon as they become available (ie. not waiting on the entire pool to load first as before).

As to wwHttp - I agree but the code is very hard to maintain - not that there's a lot of maintenance these days, all the quirks have been worked out long ago, but still the WinInet code is complex and directly deals with the Win32 APIs. At some point I suspect this might not update to newer protocols like TLS 1.3 so that's a worry. Using .NET would be a lot simpler and much easier to maintain and bring things like Http 2.0, TLS 1.3, cert support, NTLM support and a few other things. We'll see... not an immediate concern.

+++ Rick ---

Gravatar is a globally recognized avatar based on your email address. re: How to include .key & .pem in wwHTTP request
  Richard Kaye
  Scott R
  Jun 28, 2024 @ 08:39am

Side note lurker here. 😃 The updates to the engine that Rick made in v8 have significantly improved the process of deploying updates as well as gracefully handling automated app pool cycling in IIS. Highly recommended. Obviously going from 6.x to 8 may involve a bit more work but it's well worth it in my humble opinion.

Gravatar is a globally recognized avatar based on your email address. re: How to include .key & .pem in wwHTTP request
  Rick Strahl
  Scott R
  Jun 28, 2024 @ 07:03pm

Upgrading from 6.x is a little more work, that from 7, but there are very few breaking changes. The big changes happened in the 6.x release - everything else has been mostly incremental. You can look at the v7 change log and specifically at the breaking changes (there's not many to look at).

Most of the changes have to with admin related features which don't affect existing applications running. But once you look through the features you might want to retro fit your project layout to use some of the new stuff as it makes development and deployment much more enjoyable and faster.

Note the new DLL will not work out of box with old servers due to a server interface change (although you can hack that).

+++ Rick ---

Gravatar is a globally recognized avatar based on your email address. re: How to include .key & .pem in wwHTTP request
  Scott R
  Rick Strahl
  Jul 2, 2024 @ 06:39am

Appreciate the info Rick & Richard. We'll look into it.

© 1996-2024