FoxPro and .NET Interop
Issue with integers parsing wwjsonserializer
Gravatar is a globally recognized avatar based on your email address. Issue with integers parsing wwjsonserializer
  Darragh McGuire
  All
  Oct 19, 2020 @ 04:19am

Hi Rick,

Updated below. Thanks, Darragh

Gravatar is a globally recognized avatar based on your email address. re: Issue with integers parsing wwjsonserializer
  Darragh McGuire
  Darragh McGuire
  Oct 19, 2020 @ 10:43am

Have purchased latest version of west wind client tools, hopefully will resolve the issue..

Gravatar is a globally recognized avatar based on your email address. re: Issue with integers parsing wwjsonserializer
  Darragh McGuire
  Darragh McGuire
  Oct 19, 2020 @ 11:16am

Have tested this with the latest version and still an issue.

Heres a simplified version of this problem:

TEXT TO lcJson NOSHOW 
{
	"id": 504457428912,
	"message": "Whooo hooo",
	"status": 10,
	"entered": "2018-11-21T23:10:10Z",
	"address": {
		"street": "123 Timber Ln",
		"number": 32
	}
}
ENDTEXT 
loSer = CREATEOBJECT("wwJsonSerializer")
loResult = loSer.DeserializeJson(lcJson)
MESSAGEBOX(loResult.id)

Edit: Looking at this issue further, it looks like in the background its storing the value as an Integer and Integer is generating a numeric overflow, i did a test creating a table and if i add a column n(20), it doesn't generate a numeric overflow error...

Can this be resolved Rick?

Gravatar is a globally recognized avatar based on your email address. re: Issue with integers parsing wwjsonserializer
  Rick Strahl
  Darragh McGuire
  Oct 19, 2020 @ 01:04pm

Please edit your message(s) and format your code as described in the big banner above the Post button.

Thank you.

+++ Rick ---

Gravatar is a globally recognized avatar based on your email address. re: Issue with integers parsing wwjsonserializer
  Rick Strahl
  Darragh McGuire
  Oct 19, 2020 @ 02:28pm

What exactly are you seeing? The code you have shows a ***.**** in the MessageBox but that's a display issue in FoxPro, not an actual value issue.

If you use:

? loResult.Id

instead of MessageBox - I get a valid value. You can use an explicit TRANSFORM() formatting value.

String formatting in FoxPro has always been screwed up. Check this thread:

https://support.west-wind.com/Thread5VC0ULUX2.wwt

The additional issue is that FoxPro runs into rounding errors with very large numbers when trying to convert the numbers into strings. The underlying value is fine (you should be able to see the 'real' value in the debugger), but the display thereof may not be due to apparent bugs in the string formatting routines like TRANS() and any auto conversions like the one used in MESSAGEBOX.

+++ Rick ---

Gravatar is a globally recognized avatar based on your email address. re: Issue with integers parsing wwjsonserializer
  Darragh McGuire
  Rick Strahl
  Oct 19, 2020 @ 02:41pm

Hi Rick,

The problem is we are storing the json string to a .json file we are receiving a collection of orders in json from a web service, we are then iterating through each order and storing each order to a json text file

In the text file it shows **. And when we are running our other app processing each order file it crashes due to this... If you add code below to above you will see what I mean:

lcJson = loSer.serialize(loResult)
strtofile(lcjson,"c:\test\ouotput.json")
Gravatar is a globally recognized avatar based on your email address. re: Issue with integers parsing wwjsonserializer
  Rick Strahl
  Darragh McGuire
  Oct 19, 2020 @ 05:46pm

I don't understand. If you're receiving the JSON data from a service, the data will be a string - there's no translation.

You're not making yourself very clear what your actual problem is. The original problem you posted is not a problem of the data, it's a problem of displaying it and not properly formatting it.

Beyond that I can't see what you're talking about here.

+++ Rick ---

Gravatar is a globally recognized avatar based on your email address. re: Issue with integers parsing wwjsonserializer
  Darragh McGuire
  Darragh McGuire
  Oct 20, 2020 @ 01:05am

Hi Rick,

It doesn't matter even if it's just one order, the issue is still there.

As mentioned on my last post The post back data from the web service is a collection of orders.

We parse the json orders which is one string to a json object, this has the issue..

We then run a foreach loop through each order and parse each order back to a json file like follows:

For each loOrder in loOrders
lcJson = loSer.serialize(loOrder)
FileToStr(lcJson,GetServer() + "PendingOrders\"+transform(loOrder.id))  && this file has the ****** instead of the numeric value that should be passed..
Next 

And as mentioned on my last post if you added the 2 lines of code I added and viewed the text file, you should see the issue... Here is the full code (I was simplifying it, just so you could see the issue...):

TEXT TO lcJson NOSHOW 
{
	"id": 504457428912,
	"message": "Whooo hooo",
	"status": 10,
	"entered": "2018-11-21T23:10:10Z",
	"address": {
		"street": "123 Timber Ln",
		"number": 32
	}
}
ENDTEXT 
loSer = CREATEOBJECT("wwJsonSerializer")
loResult = loSer.DeserializeJson(lcJson)
lcJson = loSer.serialize(loResult)
IF !Directory("c:\test\")
MKDIR("c:\test\")
ENDIF
strtofile(lcjson,"c:\test\ouotput.json")  && Open the text file and you will see id is ********.******

Hope this clarifys my issue, if you need more info let me know.. Thanks, Darragh

Gravatar is a globally recognized avatar based on your email address. re: Issue with integers parsing wwjsonserializer
  Tore Bleken
  Darragh McGuire
  Oct 20, 2020 @ 01:41am

I guess you have to store the id as a string.

Gravatar is a globally recognized avatar based on your email address. re: Issue with integers parsing wwjsonserializer
  Darragh McGuire
  Tore Bleken
  Oct 20, 2020 @ 01:47am

Hi Tore,

Thanks for your reply.

The issue with storing it as a string is the web service (external company) is giving us a load of fields in the json file.

We need wwjsonserialzer to store it as a string for us.

If we use the code on the following link: http://fox.wikis.com/wc.dll?Wiki~JSONParser

It works correct, but unfortunately with that code it doesn't create members on the object for null values like Ricks one does.

Gravatar is a globally recognized avatar based on your email address. re: Issue with integers parsing wwjsonserializer
  Tore Bleken
  Darragh McGuire
  Oct 20, 2020 @ 01:52am

I understand, I hope Rick will take a look. Unfortunately his local time is very different from ours.

Gravatar is a globally recognized avatar based on your email address. re: Issue with integers parsing wwjsonserializer
  Richard Kaye
  Tore Bleken
  Oct 20, 2020 @ 01:56pm

Put an explicit picture clause on your TRANSFORM function call with enough nines to handle the 12 digit number in your ID kvp. See if that helps you, Darragh.

FileToStr(lcJson,GetServer() + "PendingOrders\"+transform(loOrder.id,"999999999999"))

Gravatar is a globally recognized avatar based on your email address. re: Issue with integers parsing wwjsonserializer
  Rick Strahl
  Darragh McGuire
  Oct 20, 2020 @ 02:39pm

The problem with the JSON string is that the value for the ID is not an integer - it exceeds the integer dimensions, so it's actually a long. wwJsonSerializer turns .NET Long values into a FoxPro currency - Y type, which introduces the extra decimal points. The value that comes back is actually correct, but because the value is so large and has so many decimals it fails to write out the string - boom it fails.

All the JSON integer values are longs so basically any integer is converted to decimal. It works fine as long as the numbers are not too long (ie. don't exceed integer numbering of 2 billion). But once you get values that get really large the standard string formatting in FoxPro fails.

The problem in the parser is, there's no good way to know what type the number is. FoxPro gets the number as N which could be either integer, decimal, float or double. No way to know. Formatting for currency works:

TRANSFORM(lvValue, "@N")

and that's probably the solution here. The serializer does this already for explicitly typed currency fields, but unfortunately the values returned from the .NET parser don't show the exact type. I may be able to fix that though because each value is wrapped into a type wrapper. I may be able to tweak that and check the result type when it comes back to FoxPro (at the cost of some parsing overhead).

+++ Rick ---

If you run this you can see the actual problem:

CLEAR
DO wwJsonSerializer

TEXT TO lcJson NOSHOW 
{
	"id": 504457428912,
	"message": "Whooo hooo",
	"status": 10,
	"entered": "2018-11-21T23:10:10Z",
	"address": {
		"street": "123 Timber Ln",
		"number": 32
	}
}
ENDTEXT 

loSer = CREATEOBJECT("wwJsonSerializer")
loResult = loSer.DeserializeJson(lcJson)

*** Output the result value - 
? loResult.id                 &&  valid number but with 9 decimals
? TRANSFORM(loResult.id)      &&  fails
? TRANSFORM(loResult.id,"@N") &&  works (2 decimals) and some crazy padding
? TRANSFORM(INT(loResult.id)) &&  works

*** Produces JSON with ****.***** value
lcJson = loSer.serialize(loResult)     && add .T. parm when valid to see formatted JSON

*** Generic display for Json
ShowText(lcJson,"c:\temp\test.json")
Gravatar is a globally recognized avatar based on your email address. re: Issue with integers parsing wwjsonserializer
  Darragh McGuire
  Rick Strahl
  Oct 20, 2020 @ 02:51pm

Hi Rick

Thanks for getting back to me, The annoying thing I was having with it was it was failing to parse back in the individual orders due to the ***'s.

I worked around the issue by adding:

lcJson = STRTRAN(lcJson,"***********.**********","0")

So at least I got it working and close to getting project completed 😃

Thanks again, Darragh

Gravatar is a globally recognized avatar based on your email address. re: Issue with integers parsing wwjsonserializer
  Rick Strahl
  Darragh McGuire
  Oct 20, 2020 @ 03:00pm

So I think I was able to make this work, although this is not what I would call a foolproof solution, but it'll likely work for the vast majority of cases, and certainly a lot better than it worked before at the cost of some extra overhead for number processing.

Basically, inside of .NET json parsing I can detect when a long is converted to a decimal, and set the type flag accordingly. Then the FoxPro code can check for that and round the number down to the SET DECIMAL setting which makes your example work.

************************************************************************
*  ParseValueJson
****************************************
PROTECTED FUNCTION ParseValueJson(loRes)
LOCAL lvValue
lvValue =  this.oBridge.GetProperty(loRes,"Value")

if(loRes.Type == "Y")
     lvValue = ROUND(lvValue,SET("DECIMAL"))
ENDIF
         
*!*	? lvValue
*!*	? " - " + VARTYPE(lvValue) +  " - " + loRes.Type

RETURN lvValue
ENDFUNC
*   ParseValueJson

I think this actually fixes 2 issues with one stone:

  • Fixes the Display issue due to too many decimal places
  • Fixes the rounding errors for extremely large numbers

The latter I think is the reason we're seeing these problems in the first place. Even though the display values are showing all 0 after the decimals the fact that FoxPro is displaying those decimals is that there's some rounding happening. Somewhere at the end of that sequence of zero's there's a 1 that we can't see and that's what's causing the 0's to display. Calling ROUND() removes those rounding artifacts and shrinks down the number of decimals to the point that they can be accurately printed.

In your example code this works and produces this output:

{
    "address": {
        "number": 32,
        "street": "123 Timber Ln"
    },
    "entered": "2018-11-21T23:10:10Z",
    "id": 504457428912,
    "message": "Whooo hooo",
    "status": 10
}

Note that the number correctly displays without any decimals. ROUND() produces a FoxPro N type and I guess it's smart enough to know when to display decimals and when not.

Gravatar is a globally recognized avatar based on your email address. re: Issue with integers parsing wwjsonserializer
  Darragh McGuire
  Rick Strahl
  Oct 20, 2020 @ 03:29pm

Thanks Rick,

Will look at these and let you know.

Regards Darragh

© 1996-2024