Hello friends,

It's been 28 years since we started writing our payroll app, which is specialized for the labor and tax laws of Panama. It was written in FoxPro DOS 2.6 and later migrated to VFP. It is very well adapted to local conditions and still the leading payroll program for this market, but we have decided to finally rewrite it for the next leg.

So, the first question we ask ourselves and you all is, what languages, tools and approach should we choose? Hopefully they will offer enough power and flexibility to help us expand, adapt and innovate for another bunch of years. While we may see things like AI in the future, I am talking about the the basics here. Blocking and tackling first.

I would appreciate all suggestions this crowd may give.

Thank you very much,

Alex

Hi Alex,

I think before you completely do a re-write consider this first. VFP Advanced (version 10) is now 64-bit. https://www.youtube.com/watch?v=iQqoLBOCTrI

Why re-invent if you can now port it to 64-bit?

That's my 2-cents. Best regards, Joy

Joy, what is this fascination with 64 bit?

64 bit doesn't solve any of the big ticket issues that FoxPro has:

• No Unicode support
• Flaky Windows UI behavior (for GUI applications)
• Terrible string processing performance (for parsing tasks)
• Limitations of the DBF engine (corruption, data management, 2gb limit) (not an issue if you use a SQL backend)

64 bit introduces as many problems as it fixes:

• Interfacing with 32 bit libraries (ie. wwipstuff, wwDotnetBridge)
• ActiveX controls - none of the standard ActiveX controls (ComCtl) are 64 bit
• Existing DLL access has to be reviewed for 32-64 bit

The 32 bit DLL issues are more of an issue than you might think. Web Connection for example won't run in 64 bit due to the dependence on 32 bit libraries. Any UI application that uses Common Control ActiveX controls like the TreeView, ListView, Rich Text, Progress bar, Animation control, WebBrowser control - none of that works in 64 bit. And good luck finding 64 bit ActiveX replacements (mainstream 64 bit arrived after ActiveX's heyday in the late 90's).

I can't see too many FoxPro applications that would get a benefit from 64 bit - which mainly means more memory to access. Do you have or know of any apps that are running into ~2gb of memory limit? I can't imagine apps that do. 2gb of memory is a lot of memory, especially for Win32 app! I have one (.NET) app where I benefitted from 64 bit and that's WebSurge which keeps its test run logs entirely in memory. But even then it took some serious doing (with 50,000+ requests a second) to bump up against the 2gb limits.

+++ Rick ---

Hello Alejandro,

Choosing a language comes long after major architectural decisions:

1. Target platform(s): desktop and/or browser and/or handheld
2. Based on (1), inventory of modules: Functional blocks, forms/pages, reports, transactions
3. Migration strategy: whether you target a ‘big bang’ approach, where the old system is totally replaced by the new one overnight, or a gradual shift of functional blocks, old and new systems working side by side.

The choice of language is heavily influenced by your responses to these questions, and a mixture of productivity expectations, and developers’ preferences/bias/religion.

Could you shed some light on these concerns? Answering these questions will probably help your decision.

Hi Alejandro

Did you test X-SHARP ? X-SHARP

I have already started to test it ..

It’s promising!

Compile in .NET, use Visual Sudio, integrated VFP command (s), read / write in DBFs and DBCs, etc.

Regards Gilles

Hi Thierry,

Agree, I am really asking about "architectural" issues, especially those that relate to the web interface with which we don't have much experience.

Our current app specializes in payroll for Panama labor and tax laws. The goal is to make a new app that works correctly in different legal jurisdictions, a tricky thing. The old data will be imported and accessible, but the database structure will be different. We want to keep the program organization and feel that users like, call it a similar menu and philosophy. I presume some clients will never to change to the new app, but we have to try.

As far as languages and tools, my instinct is to go mainly dotNet because of the richness of the ecosystem and because it's free or affordable. I assume the front end will be based on html, css and javascript (TypeScript form). Is this a fair assumption?

I understand that there is no inheritance available for an html based interface. So, what is the smart way to design and program to add flexibility for future "redesigns" or modifications should we find those necessary.

Thank you very much.

Alex

Alejandro,

Your Web UI will indeed be in HTML/CSS/JS regardless of the language you choose for the back end; it's the only decision you'll don't need to make.

Otherwise, if you decide to also change the database structure, you implicitly go for a 'big bang' migration.

Changing everything at once (DB model, UI tech, code tech and code) seems crazy to me, I would never start any project like this unless I work for Exon Mobil or any other Fortune 100.

I would suggest you 2 things:

1. Read Joel Spolsky's excellent articles about software engineering in general, and project management: eg. Things You Should Never Do, Evidence Based Scheduling
2. consider a more gradual approach

I can make suggestions on (2) if you desire.

I think it really depends on your code base.

I am a big fan of doing layered upgrades - picking out sub-components and building them with new technology and integrating them into an existing solution. This gives you the ability to see how the new technology works without betting the farm on it working out for the entire huge project.

But, one problem especially for many FoxPro projects is that many are very, very old and are generally not designed well to be modular. Most 'legacy' FoxPro apps that are still around today and haven't been upgraded to something else (or even to newer Fox code) are that way because they have so much baggage attached to them.

I've seen some amazing FoxPro applications in terms of functionality and features and what they do. But then you look under the hood and you see an incredibly byzantine development process that isn't portable in any way. IOW, some very smart and creative people built that software, but weren't thinking about the implications for application maintainability and integration many years later.

Systems like that often are much better off literally starting over in whatever technology is chosen (and again that can be FoxPro with things like Web Connection or FoxInCloud). But even then if possible, picking out manageable chunks instead of the entire project is key both for a reality check with chosen technology but also to keep the scope manageable for developers involved.

Over the years I've been involved in countless migration projects, and a lot of them failed because people underestimate what's involved in re-writing, but also because they underestimate the ability of the old applications to integrate with something new. Software 'migrations' or 're-writes' (even if re-writing in Fox) are always a tough sell - you go from something that works, to something that takes a time and money and effort to hopefully arrive roughly at the same point you started. More maintainable and hopefully some improvements, but overall it's a zero-sum process that sinks time and money. It's not a great selling point especially for management.

If you go down any re-write you have to be able to make hard decisions and stick to the plan. Being wishy washy about - oh this is too hard, or it's easier to fix the old code and changing direction constantly because you see some new technology - is what usually ends up failing this process.

+++ Rick ---

I understand that there is no inheritance available for an html based interface. So, what is the smart way to design and program to add flexibility for future "redesigns" or modifications should we find those necessary.

We VFP developers assimilate flexibility to inheritance.

In HTML, a similar flexibility is implemented by a special type of CSS selectors called combinators.

Using combinators, you can adapt the behavior of an HTML element according to the context where it's used; eg.

<--! form1.html -->
<div id="form1">
…
<input type="text" class="someClass" value=""> <--! pulled from input.someClass.html when building form1.html -->
…
</div>

<--! form2.html -->
<div id="form2">
…
<input type="text" class="someClass" value=""> <--! pulled from input.someClass.html when building form2.html -->
…
</div>

/* yourApp.css */
#form1 .someClass {color: black;}
#form2 .someClass {color: red;}

// yourApp.js
jQuery(".someClass").on(input, function(evt){…}); /* generic behavior */
jQuery("#form1 .someClass").on(input, function(evt){…}); /* specific additional behavior when used in #form1 */
jQuery("#form2 .someClass").on(input, function(evt){…}); /* specific additional behavior when used in #form2 */
// As handlers execute in the same order as they are registered, you can decide whether the specific behavior executes before or after the standard behavior.
// Using HTML extensibility, you can set 'properties' within the element to communicate between event handlers:
/* handler 1 */
jQuery(this).data('flag', true);
…
/* handler 2 */
…
if (jQuery(this).data('flag')) return;
…

This gives about the same flexibility as VFP classes except:

• you need to decide once for all when comes dodefault(): either before or after specific behavior; you can’t call if in the middle nor get a returned value
• while VFP visual classes point to the parent class address, here everything relies on id and class naming; on large application you absolutely need a strong and simple naming scheme.

I would bet that frameworks like React, Ember and Angular provide similar mechanisms.

CSS selectors are the keystone of web dev; understand them is, in my opinion, the very first and most important task.

Because it happens once in a life time, re-writing a full application is something of which, almost by definition, no-one has experience, especially in the VFP world where development has been continuous improvement and extensions along 30 years.

Given that evidence, the best you can do is rely on people who have a larger experience in this area, and Joel Spolsky is, IMO, probably one of the best, for several reasons:

• he was first an employee in a big company (tech lead on MS Office automation), then a successful entrepreneur (founder of stackoverflow), and still in charge of a software company (Fog Creek Software), and probably still developing. He's one of the very rare manager AND developer around (with Paul Graham and probably others)
• he has taken the initiative of writing a practical blog on stinking questions that very few people have written about before, except professors in Harvard that no-one can understand. All his articles are crystal clear, lively and can be understood by anyone in charge of a software house.
• he has a proven track record as a very successful entrepreneur and developer (when I first saw stackoverflow, I found it just breathtaking).

Using his evidence-based scheduling methodology, we can now meet our schedule and budget within a ± 15% error margin, which I never could achieve before.

This methodology teaches us that the only way you can control your time and budget is to break down your project into pieces that you can almost develop by thought. I know it seems counter intuitive, that the time you spend doing that seems lost for 'real' development, however believe him (and me), you'll save far more time on later testing, debugging and fixing 'random' errors.

Re-writing (or migrating) a software is like this character crossing a river in the movie 'Into the Wild'; either you jump into the chilling water and hope you can make it to the other bank without drowning or drifting too much downstream, or you prefer build a bridge that'll take you safely across. You know how long it'll take to build a bridge, much longer than you think you can swim across, at the expense of hazards that you can never account for.

Thank you for your thoughtful answers Rick and Thierry. I am at the beach with connectivity problems. Will answer more fully later. (The water's fine though).

Thank you for your thoughtful answers Rick and Thierry. I am at the beach with connectivity problems. Will answer more fully later. (The water's fine though).

Hi Rick,

Thanks a lot you for your comments.

You mentioned that migrations often go from something that works to roughly the same point where you started. In this migration we want to go from something that works for the laws of one country to something that works for the laws of several countries. Since some of our customers operate in several countries and have expressed a need for just that, we hope that having that capability would buy us enough time to get our act together. Old customers would likely stay where they are until the product catches up on the essentials and offers new advantages.

Your layered upgrade suggestion sounds interesting. I don't think you mean to ship product after some of the VFP code and DBF data is replaced with new code and SQL server data. How do you envision a layered upgrade?

Thanks,

Alex

Hi Thierry,

I will get back to you after I understand better something Rick has suggested.

Cheers,

Alex

Hi Alejandro, 1842 / 5000 Résultats de traduction From my own experience, migrating data in VFP format to an SQL server (personally I chose PostGreSQL) consists of carrying out three phases:

First phase (the longest) no longer working directly on DBF tables:

• Add an autoincrement primary key to the DBF tables
• Replace all X-BASE data update commands (replace, append blank, etc ...) by their equivalence in SQL commands (SELECT, UPDATE, INSERT, etc.)
• No longer working directly on DBF tables, but using datasets in cursors, parameterized views, etc.
• Modify all the procedures using the VFP record positioning commands (locate, seek, goto, etc.) by SQL procedures performing an extraction in cursors or arrays or CursorAdaptors of the sets of records corresponding to the search / desired positioning
• Use these views or cursors or CursorAdaptor in all your procedures, methods, forms, etc.

Second phase, use the remote database:

• Create in the SQL server all the tables necessary for your application from the structure of the DBF tables
• Create remote views on tables in a VFP (DBC)
• Replace in the procedures, methods etc. of your applications the use of DBF tables by the use of remote SQL views with connection and disconnection to the SQL server

Third phase, develop functions and procedures on the SQL server side to replace VFP functions and methods, leaving only the business layer on the VFP side:

• Replace when possible and useful VFP functions or procedures by functions / procedures executed directly by the SQL server

I took less than 3 man months to do this in my BioSIL application..

Regards Gilles

Thank you Giles for sharing your experience.

Alex

Rick,

You mention, "Flaky Windows UI behavior (for GUI applications)." This has been driving me nuts for years.

Can anything be done about it?

Bruce

Can anything be done about it?

Not that I know about it using the native FoxPro controls.

+++ Rick ---

Hi Rick,
Can you give an example or two on this?

"Flaky Windows UI behavior (for GUI applications)."

One I can think of is when Labels turn bold upon refresh. Very annoying.

TIA,
Steve

Hi Thierry and all,

I've been thinking a lot whether it's possible to change our software in layered upgrades as Rick suggests or if we need to go to a 'big bang' as Thierry calls it, but I'm afraid it is going to have to be the second case.

We see an opportunity if we can evolve our product into one that can be customized for several countries because several of our multinational customers who like our product and operate in Central America and the Caribbean have asked us if we have versions for other countries.

We have made these choices so far:

• C# and .NET
• SQL Server
• Web delivery through the browser
• Entity Framework Power Tools in order to design tables in SQL Management Studio and go from there to "Code First"

Still searching for tools to maximize developer productivity

• Aids to internationalize the user interface since three languages are used in the region.
• Data driven tools that give good results.

Should we consider Blazor as one of the tools?

I would appreciate suggestions very much.

Thank you,

Alex

Hi Alejandro,

Software is made of soft and ware.

the ware part is the set of tools used to base the system on; you've devoted a lot of interest on this part and probably did a fine job picking the items you mentioned. Each tool has its pro and cons, almost all of them are closely comparable within a thin margin, you've made a choice and that's probably fine.

the soft part is the human component; any Information System is the combination of:

• designers and developers (humans)
• programs (tools and infrastructure)
• users and customers (humans)

From this simple list you see intuitively that the soft part is far more important than the ware part. Picking new tools by is far easier than changing (adapting) the humans around them.

Let's first consider the client and users side.

In the 80's-90's, probably due to the heavy changes in the information technologies, and because systems were still relatively simple, management and users accepted to get involved in building Information Systems together with consultants and developers; they would validate functional specs, understand the tedious details about validating and testing, and so forth.
Since roughly y2k, software is generally considered as a commodity: buy, install and it works. No user today accepts to do any testing, give feedback, understand the details, in a word co-operate with the software companies; not even professional developers, and we have examples on this forum almost every day. The only answer you'll get is: it doesn't work.

If you go for a big bang strategy, you will say to your customer and users "we're working on a brand new version, just wait a couple of years and it'll be a brand new world".

The only little problem is that you're completely unable to have the slightest idea of how long it'll take you to get to this new version. And you'll postpone once, twice, etc., and at some point you'll say "OK, even if we're not completely done, let's take the risk and move forward and try the new version". In the best possible scenario you'll have a data migration tool -- but frankly I doubt very hard because it's one of the most difficult part of the project, especially is your current database is not normalized with PKs and FKs everywhere and relational integrity -- but you can be sure that you'll have to start without real testing, and you'll inevitably fail.
Even the best software teams like jQuery and others, with a complete set of test routines, need a large number of beta testers. And even that way they have accidents when releasing even minor versions, that require patches.

Then you'll rollback to the good old desktop software and your users will have to redo the work lost with the new version and believe me, at this point the relation with your customers will degrade very rapidly.
They may follow you on one, maybe 2 'big bangs' attempts, and then look for an alternative solution from a large software house, even if it cost 10 times your price. Management seek personal security first, social quietness, regardless of the price it costs to the company, and in that occurrence the price of software is completely negligible.

If you now look at the developer side, you really need to consider the huge technical and cultural chock between an integrated environment like VFP and (1) the ± 10 layers of independent technologies in the tech stack you've selected, (2) the giant leap in abstraction between a procedural language and the heavily object-oriented techs of the Web world.

The greatest interest of a forum like this one is to share experience on these soft aspects of systems development rather than tech tools that personal preferences influence so much without a real advantage for such or such tool.

Many participants in this forum are former VFP developers who have, to some extend, moved to the web; they probably can share their experience on addressing these soft issues, on both the developer and user/customer sides.

As a conclusion I would say that, at least, you should migrate the database prior to the whole software. This will give you a first milestone with your customers and a mainstay on which you can progressively release new web-based modules co-existing with the desktop application.
Definitely flee the big bang migration.

Thank you for your comments Thierry.

The "ware" comments are encouraging and your warning about current customers' probable response is on target.

We have to keep thinking.

Thank you.

Alex

That's a great analysis, Thierry,

In my experience the soft part is 90% of the failures I've seen and it has mainly to do with lack of commitment or - on the other hand - lack of the ability to make hard choices.

Technology is important and it interacts with people using the technology and there can be a lot of friction there. Friction can come from a lot of different places: It can be inadequacy of legacy tools for modern requirements, it can be complexity of new tooling, it can be that the developer base is not capable of adjusting to something new etc. etc. These aspects will vary from company to company, but they have arguably the largest effect on the outcome.

Taking on a large project that can't organically grow requires strong leadership and someone to guide it with a vision and the ability to make hard choices. That person should be somebody who knows both the business and the technology or at least can speak intelligently about the technology. Sounds obvious, but this is one of the key failures I've seen where the expectation of the technology vs. reality of can be reasonably achieved is out of sync.

The other problem I've seen is that people are paralyzed by too many choices. They look at every possible technology and see the 'cool' things it can do but they look at every possible solution and never make a choice or dive in.

Most tools and tech has strong points and you can bet that those trying to sell (directly or indirectly) that technology are not very keen to point out the problems with the technology which you otherwise don't find until you're perhaps too far on your way to turn back. The key is to get your hands dirty as soon as you have a candidate technology and build something. Don't build the huge project, but build some adjunct feature to give it a (realistic) test run and see how it fits and see where if at all it breaks. If it works out - great you've spent some time building something you can use and you've gained valuable experience. If it doesn't - well you found out that it doesn't rather than ending up in a dead end with a committed project. Win - Win.

This one of the things that help make good decisions that can lead to better outcomes than the over 50% average failure rate of IT projects.

In short, there are no one size fits all solutions. Each company has different prerogatives on what's needed and how to get there.

+++ Rick ---

I like your suggestion Rick: "The key is to get your hands dirty as soon as you have a candidate technology and build something. Don't build a huge project but an adjunct feature to give it a realistic test run and see how it fits and if at all and where it breaks"

One more thing. I see Blazor promoted for PWAs, not for large projects. For us, not needing to be proficient in javascript would be a helpful shortcut. What do you think?

Thanks for the comments,

Alex