Hi Rick,

I've got a new API (using POST) to hit here and am running into inconsistent behavior with auth handling.

I prototyped my request using WebSurge. My tokens are coming from an AWS tokengen endpoint which I'm using successfully with earlier APIs in both my stage and production environments. The only header in Websurge is the authorization with the token. The body is very simple:

{
    "paidInvoices": [
        {
            "invoiceID": 12345678,
            "paid": true,
            "paidDate": "2024-12-04"
        }
    ]
}

I've created good requests that will succeed (200) as well as bad requests (400) that will fail in my websurge file. I run the tokengen endpoint call, integrate the token response into my test requests using VB inheritance, and run each request to completion (200 or 400). (Yes, I know about the token injection directives in Websurge and although I've been able to get that to cooperate in the past, not so much with this particular prototype. This is a problem I'll solve another day.)

Next step is building a test process method in my app. It's a very simple page that instantiates an object from my class which is based on wwJsonServiceClient, and invokes the appropriate method for this new endpoint from the class.

loMarkPaid=NEWOBJECT([iainvoices],[iapiinvoices.prg])
loMarkPaid.callMarkInvoicePaid(m.InvoiceID, m.Paid)
m.lcResult=TRANSFORM(m.loMarkPaid.cResponseData)

In the app environment, I am also adding a contentType directive and a 2nd custom header in addition to the token header:

.oHttp.cContentType=[application/json; charset=utf-8]  
.oHttp.AddHeader("X-Correlation-Id",GetUniqueId(16))  
.oHttp.AddHeader("Authorization",.oToken.Token)  

Considering the data in these requests I probably don't need the charset but force of habit has me putting that in. The response writes out cResponseData. I do have lSaveRequestData set to .t.

When I test this request in the context of my process class in my app environment, I'm always getting a 401. Removing the contentType declaration doesn't affect the outcome. When I review the content of cRequestData I can see the headers and the post buffer and other than the encoding process adding \r and \n it all looks proper. OTOH when I run the WWC Request Viewer I can't see the cExtraHeaders or the cPostBuffer values in the Raw Request page.

I'm assuming something is mangling the token data along the way but don't have a clue as to how I would test this hypothesis. Or maybe I'm just doing something else wrong. I have tried to simplify quite a bit based on our last conversations around using wwJsonServiceClient. I've also asked the internal team if they can see anything in their logs which might provide some clues but so far all they see is what I get; the 401. Thoughts?

TIA

First thing to check is that your logging is set correctly to capture the headers (and body). Make sure you're using the most verbose logging mode while testing (3 - Request and Response).

If you're not seeing what you're expecting between client and server:

When in doubt use a proxy like Fiddler Classic to check requests and see what's actually going over the wire.

This will let you trace this to the client or the server.

Tokens time out, so if you're using WebSurge and hardcoding tokens, you have to make sure you update the tokens frequently - whenever they are set to expire. In WebSurge you can add tokens to every request by adding a Replace Authorization value in the Session Configuration (ie. Bearer xxxx) and making sure that each request you want the header to be sent has an Authorization: header (it can be blank or have some dummy value - it'll be replaced). You can also call some endpoint to pick up the token and automatically set this (so you can add the AWS auth request to WebSurge, and pickup the token from there and then automatically update the value mentioned above for all requests). There are context menu options in the Http Header menu in WebSurge that make this easier to remember the names of values and 'default' values.

+++ Rick ---

Rough guess here that my HTTP object is not passing what I expect it to pass? Certainly, no auth header would return a 401 every time. I need to reexamine my assumptions about object scope.

Here's a class method snippet:

IF .getToken(.apiTokenUserName,.apitokenPassword)
	.oSerializer.PropertyNameOverrides=[paidInvoices,invoiceID,paidDate]
	.oInvoices=CREATEOBJECT([empty])
	.addInvoice(m.tnInvoiceID, m.tlPaid)
	m.lcJSON=.oSerializer.Serialize(.oInvoices)
	.oHttp.AddPostKey(m.lcJson)
	m.lcEndpoint=.apiAPIServiceURL+.apiAppPath+.apiHouseUserName;
		+.apiInvoice+.apiMarkPaid
	.oResponse=.callService(m.lcEndpoint, .oHttp, [POST])

The getToken method does just that. It uses CallService to get the token object. When that comes back successfully I add the headers and content type mentioned in my original message to the existing oHTTP object. addInvoice populates the object that holds the body payload. I add that as a post key and then send the request.

Because the proxy http object was first used to retrieve the token, do I need to reset it? Or create a new object to make the 2nd request?

Figured it out. In the immortal words of Emily Litella, "Never mind".

While I go spelunking in the help, is there a magic util that will take a formatted json string and turn it into a prettified html string? Ya know, like with br codes added to the end of each kvp.

Paste the JSON into VS Code.

Ctrl-A to select all, then Ctrl-K-F to format.

or if you're old school, create a utility with FoxPro:

do wwJsonSerializer
loSer = CREATEOBJECT("wwJsonSerializer")
_cliptext = loSer.FormatJson(_cliptext)
* ShowJson(_clipText)   && needs DO wwUtils

+++ Rick ---

Oh, I can absolutely roll my own in VFP. I was hoping you'd already done the work for me. 😃

I did! 😄

+++ Rick ---

I was looking for oSerializer.serializeToFormattedHTML(). 😃

You mean?

• wwJsonSerializer.Serialize(lvValue, llFormat)

LOL! RTFM! 😄

+++ Rick ---

RTFM? I should be in the proofreader's Hall of Fame for WW doc errata submissions. 😃

I take that formatted json string, toss it into a memvar and spit it out in a very vanilla webpage.

lcResult=TRANSFORM(m.loMarkPaid.cResponseData)
loResponse=m.lomarkpaid.oserializer.deserializejson(m.lcResult)
lcResult=m.lomarkpaid.oserializer.serialize(m.loResponse,.t.)
TEXT TO m.cHTML NOSHOW TEXTMERGE FLAGS 1 PRETEXT 1+2
	
	<div style="width:700px;margin:30px auto;">
	<<m.lcResult>>
	</div>
ENDTEXT
MakeHeader2([admi],[colors]) 
Response.Write(m.cHTML)
MakeFooter()

I see this:

Here's the page source:

<div style="width:700px;margin:30px auto;">
{
  "data": {
    "message": "Invoice 2250716 was marked unpaid.",
    "success": "true"
  },
  "errors": [],
  "httpstatus": 200,
  "locale": "en-us",
  "pageaccessexception": null,
  "time": 1733862337641
}

This is just a test page so it's really not critical in any way. It would be nice if it came out in an HTML page the same way it does in the memvar so that the human looking at it doesn't have to work too hard at parsing the response. Of course, I can easily STRTRAN all the CRLFs to br codes. So I did. Now I am ashamed for a different reason. 😉

That's because the browser tries to display it as HTML which loses all the line feeds. If you View Source you will see the raw data with the formatting I think.

You need to install a JSON Extension that will format JSON. I use JsonView. This is the Chromium version but I think there's also a FireFox version.

Yep. The source was one of the screenshots in my last message.

One extra line of good ol' VFP code does the trick; hence my embarrassment:

lcResult=STRTRAN(m.lomarkpaid.oserializer.serialize(m.loResponse,.t.),CRLF,[<br />])

I dunno why I thought a CRLF would be magically converted to a break. Wishful thinking I guess.

To your earlier note on the token capture stuff, I tried multiple variations on that and for some reason it just wouldn't cooperate. Which puzzled me as I have done it with other Websurge session scripts.

I put this in the header of the request that hits the token service.

Accept-Encoding: gzip,deflate
Content-Type: application/json
WebSurge-Request-CaptureAndSetAuthorizationBearerTokenFromJson: token

It populates the Replace Authorization value in the settings as expected.
Then in a request that requires the token I've tried several things.

Content-Type: application/json
WebSurge-Request-InjectJsonBearerToken: token
Authorization: 

and

Content-Type: application/json
WebSurge-Request-InjectJsonToken: token
Authorization: 

and

Content-Type: application/json
Authorization: Bearer

The first two return CheckSite Error: The format of value '' is invalid. The last returns a 401. If anything obvious jumps out at you about why the token injection isn't working and you care to comment let me know. Otherwise I'll figure out how to use Fiddler some day when I have leisure time on my hands. I've got all the new API use cases tested so probably won't have to revisit my Websurge script for this any time soon.

Does token match a key in your JSON data? The token value has to be an actual key name.

+++ Rick ---

Yep. Here's the whole request out of Websurge, sanitized for my protection.

PUT https://<my secret token endpoint> HTTP/2.0
Accept-Encoding: gzip,deflate
Content-Type: application/json
WebSurge-Request-CaptureAndSetAuthorizationBearerTokenFromJson: token

{
"username": "my secret name", 
"password": "my secret password"
}


HTTP/1.1 200 OK
Content-Type: application/json
Content-Length: 1006
Date: Thu, 12 Dec 2024 18:55:40 GMT
Connection: keep-alive
x-amzn-RequestId: 6c805e32-58dd-4ae8-afa4-c16d601c462c
x-amz-apigw-id: CsSywHrNyK4EaXg=
X-Amzn-Trace-Id: Root=1-675b31aa-56eac95f5211ccd24b503ffb

{
  "token": "<snip>"
}

And the Replace Authorization in Settings is populated with the token I just got back. It's getting it properly injected into the other requests where it's giving me agita.

Well then it sounds there's a problem on the other end... (the server)

+++ Rick ---

I wouldn't be surprised. My devops lead is currently excoriating the devs for not following standards in the json formatting of the response. Which I know nothing about. To quote him:

but I do think the response structure should be as concise and machine-readable as possible.

with machine readable seeming to be his key concern. I am concerned that the humans can read it. 😃

The one other bit I noted is I've got the Websurge settings using Http protocol 1.2 and the response comes back as 1.1? But that's probably irrelevant. I'm just playing "one of these things is not like the other".

As always, thanks for playing along. 😃

but I do think the response structure should be as concise and machine-readable as possible.

Whatever the hell that means. JSON is by definition machine readable...

As to the token you can always hardcode a token into the request and send exactly like you want it:

• Run the auth request manually (without the WebSurge auto-injection)
• Grab the token
• Add the Authorization: Bearer <token> header manually to the request
• (remove the Session Authorization setting as that will override the per request token)

There's no such thing as Http 1.2. There's 1.1 and 2.0 and you should match that to what the server uses most likely 1.1. If the server supports 2.0 you can get better performance with 2.0 but every server supports 1.1 not every server supports 2.0. I think the Http handshake figures out which to use so even if you specify 2.0 it'll use 1.1 if the server doesn't support 2.0.

+++ Rick ---

That was my reaction. He seemed to be disturbed by the data/string shown in the message property in the response. I dunno... 😃

The 1.2 ref is my dumb setting change. Duly noted and corrected.

Finally figured it out. (This is restating somewhat what you said earlier about the more manual process, and much like some of your blog posts I'm writing this all down for my future reference.)

• Use the capture and set directive to get the token into the settings.

WebSurge-Request-CaptureAndSetAuthorizationBearerTokenFromJson: token

• Remove the word Bearer from the captured token value.

• Add the empty bearer token to the requests.

Authorization: Bearer

The empty Bearer directive appears to allow Websurge to simply swap out everything after Authorization: with the Replace Authorization settings value. Once I do that I can run all my test requests without having to manipulate each one.

Some feedback on this topic in the FM - https://websurge.west-wind.com/docs/_67p0uypk0.htm

Important: In order for this to work you have to provide an Authorization: header. It can be empty or contain any content, but the Authorization: bit must be present.

The above paragraph says the "Authorization: bit" can be empty but my experience is you must have the word Bearer there in order for the magic token swap to occur. In my environment, when I only include Authorization: in the header by itself, that's when I get the CheckSite Error: The format of value '' is invalid response. And being literal in nature I assumed empty meant empty. 😃

Maybe my devops guy will have something to say about how the auth header is handled if the word "Bearer" is the standard. If I raise the question... 😃 Bookmarking this post so that when I run into this again in 6 months I'll just have to remember I bookmarked this post.

It depends on what you're doing - there are multiple replacement strategies that can be applied.

• Bearer Token injection
  This is direct token injection and for this to work you need to have Bearer in place as it's explicitly designed for bearer tokens. This concerns WebSurge-Request-InjectJsonBearerToken specifically.

• Session Configuration Authorization Replacement
  This captures the token and sets the Session Configuration Authorization header. This settings usess the authorization header override which can also be set from WebSurge-Request-CaptureAndSetAuthorizationBearerTokenFromJson. This replaces any authorization header Bearer or otherwise as long as the Authorization: header is present.

+++ Rick ---