I'm still seeing this behavior more often than not. Open a project, make no edits, close the project, get prompted to save the current topic. I may have found a clue. Using git to diff the repo, it appears that a timestamp value is getting updated in the root json file simply by opening the project.

And here's another clue; that timestamp is getting changed by bringing up the git dialog in DM.

Correlation does not imply causation 😄

The timestamp update occurs after the file is saved. The issue is related to change detection which then causes the file to be saved when you move of the topic.

I think the issue is related to file encoding (Utf-8 vs. Utf-8 BOM) because when you look in Git you don't see the change and Git ignores those. What's strange is that it's intermittent. Sometimes it happens other times it doesn't.

I'll take another look - again 😄

+++ Rick ---

No argument there. 😃 And I did think it could be related to handling of line endings, too.

I can reproduce this behavior. Here's a blow-by-blow description, but let me know if you want to see it with your own eyes.

• In a shell I made sure my local repo was clean.
• I fired up DM which righteously opened up the last project used.
• Checked git status again and verified it detected no changes.
• Loaded the git form in DM
• Change detected in the primary file

Ran a git diff and the only change is the aforementioned timestamp.

Well the JSON file change makes perfect sense because that happens once DM sees a change it writes out the updated timestamp.

Most of the time this is 'invisible' - you only see this behavior when you quit and you sit on a topic that hasn't been changed and most likely the original topic you loaded when you opened DM.

I just can't figure out what triggers the change state and why it's intermittent. On my machine most of the time I can start DM and then exit without a change notification. At other times I do exactly the same and it comes up right away with the * in the filename for change detected. I can't make sense of it. I know where this happens but it's difficult to debug because there are many places that call into that so breakpoint debugging is difficult here.

I'll take another look today - it's been long enough that I maybe have another perspective.

+++ Rick ---

I've confused 2 things here. Let me see if I can do better.

I was originally thinking of the behavior you're mentioning in this thread where DM prompts to save changes when nothing was edited. That still happens but inconsistently.

But the thing I'm trying to highlight now is git-related. Simply invoking the git form in DM is writing to the main project file. So while DM may or may not prompt to save changes, git says there are changes to commit.

I'm not doing anything particularly complicated at the moment so if you want to jump on a zoom I'm happy to do that.

Yeah, but that is the same issue. The difference is that DM saves files silently if there's a change. Whenever you move off a topic and there's a change (or 'perceived change') the topic is saved which updates the timestamp in the JSON file with the meta data.

So while that may seem like it's a different issue it's not. The underlying problem is that DM sees a change when there isn't one, which might be for the same reason that sometimes you see changes in Git when you check the Diff and there's 0 changes.

What DM does is the following:

• Checks the current content in the editor
• Loads the file's current content into a variable
• Compares the two

If not equal that's when the change shows.

I think what I might do is set up a new project and set a few very small topics then dump out the content on each access. The problem with usually is that there's too much text to really see what's different with just a Console dump of both texts.

+++ Rick --

The issue is related also as to what Git sees along the same lines:

Notice that according to git the files are binary the same but yet it somehow sees a change.

I did find one more issue that might be contributing which is that if the app is shut down, the project is not properly shut down which means potentially topics didn't get updated right and last topic selection was off by one (ie. next to last topic rather than last topic).

Playing around with this now again and I don't see any weird arbitrary dirty topics but I've been here before.

+++ Rick ---

Odd. My git repos do not treat markdown files as binaries. OTOH I do use git lfs for tracking binaries.

My simplified troubleshooting approach to this was to run a git status after each DM action. That's how I saw that the primary json file for my project had that one timestamp change apparently resulting from invoking the DM git form.

Nah that's just Beyond Compare saying that the files on disk are exactly byte for byte identical - not just the content. I believe Git actually works of file content not of the actual physical bytes for known types of files (depending on settings). Beyond Compare just lets you know if there's literally no difference in the before and after files.

Why it shows up in Git as changed though is a mystery - either way you look at it there are no changes. And it's not just DM that shows the changed Git item, it's also showing up in SmartGit's change log (also with no visible changes), and you can't Discard those changes either. Very odd.

I've seen that in other projects (.NET mainly) as well.

+++ Rick ---

But there consistently is a change to the updated key value of the json index that happens when the git dialog is loaded. I will put together a video recording. Sometime over the long weekend. 😃

Your confusing cause and effect again. The problem as I mentioned earlier is that the change detection is not working properly and detecting a change when there is none and once detected it updates the timestamp.

The problem is that I see not just the index updated, but also the topic with no changes as shown when that happens. There's something that is triggering the change detection and I can't quite figure out what it is because it doesn't happen all the time. Most commonly I see it when first opening a project and I think it might be due to some timing issue where a check occurs looking and no previous topic (null) and the new topic. there's a check for that but it still happens at some point.

I haven't see this happen in a while - it's very random.

+++ Rick ---

Just trying to share some empirical observations, not identify root cause for you. You don't really need my help with that. 😃

To recap verbally, I load the project in DM; no other actions taken in DM.
Run git status in an external powershell instance and it's clean; no changes to any tracked files.
Click the Git button in DM to load its form.
Run git status and there is one tracked file changed; the main project json index file with the one change being the timestamp on the updated kvp in the json as shown in my earlier screenshots.

Clearly whatever happens when I click that button is internal plumbing. If it's useful/helpful to keep this thread going just let me know.

Ok after some more spelunking I can see similar behavior - sometimes. File unchanged, but updated stamp changed in the meta data file.

+++ Rick ---

Another possibly related issue I can identify: It looks like the Git Repo holds some files that are encoded with UTF-8 with BOM while DM encodes everything without a BOM. These files have no actual changes other than the encoding:

DM changed the default encoding to use UTF-8 without BOM a bit ago, so this would make sense against files that were saved and stored in Git a while ago before the change, but I feel like I'm seeing some files show up that are newer as well.

I did a recent export of my main HB project and a start from scratch approach last week. If there's something I can check here that could confirm the encoding update as a potential root cause just let me know what you want me to review.

The HB import should generate the files with the correct encoding. DM should write all files with the correct encoding, but external editing or Git might change that.

It'd be interesting if you run into file diffs again to double check the encoding and see if that's the difference. Not the project file since that updates all the time as it keeps a running total of the last topic used (that probably shouldn't be in there to avoid the frequent churn, but then we loose per project topic remembering.

Can you grab v0.4 and see if you see any improvements?

I've found several instances of topics being saved where it's not necessary, plus I added a sync lock to saving and loading topics to avoid potential race conditions. Also in one place when multiple tabs are open (from file explorer) there was a potential for the wrong file to get saved altogether.

I've had a couple of reproducible failures with empty topics that seem now resolved so this is encouraging.

+++ Rick ---

Will do. Should I start from scratch again? IOW new project with new import and new repo?

Hmmm... That's unsettling. I just opened the project with 0.4. Again no other actions beyond simply starting DM. It opened up to the first topic which I feel like is not where I last left it. Git status indicated that a topic (md) page had changed this time as opposed to the index. But the rendered page in the UI is not showing the text.

I moved the cursor around a bit to sese if the preview pane would re-render but it did not. The I decided to take a look at the file system and noticed two MD files that are 0 byte.

I'm sorry I don't recall looking previously so can't speak to what it was prior to me running DM just now.

As none of this matters at the moment I'm going to start from scratch in 0.4 with this project and will pay more attention before hooking up to git.

The 0 byte files are folders - there may just be no data in those topics. In some cases when importing those folders are empty to begin with - and they should show all the child topics which is the default behavior.

+++ Rick ---

But the topic shown in the screenshot has a couple of short paragraphs while the preview window does not? And if those are supposed to be folders, not files, Windows doesn't seem to understand that.

I pulled up DM today (haven't reimported yet, etc). The contents of that page are gone. I never explicitly saved.

Just did a fresh import with 0.4. Preview pane is coming up completely blank.

Looking at the file system and those same 2 MD files are zero byte.

These are unrelated issues but after some experimenting I can duplicate what you're seeing.

Two issues:

• Root Topic Empties Out
  This appears to be specific to the root topic, when you import and close without saving or navigating off. In that case for some reason the root topic is not saved and then comes up empty. Oddly other topics that does not occur. First thought was that it's due to the import but it seems that I can make that happen even after navigating through a few times. Other topics seem Ok though.

• Preview not Opening when Project Is Closed then Opening another Project (or same)
  This doesn't happen every time but I see it about 2 out of 3. Toggling the preview brings the preview back. Suspect that's a race condition related to the preview being not visible on an empty project without a flag being properly updated, so the preview doesn't completely refresh as it should.

Give v4.1.1 a try...

I think I found the issue related to topics being overwritten on startup. Due to a race condition where the editor is not ready and we're reading the content from the editor to display the preview and in the process get empty content from the editor that is not ready. It's a race condition, so some of the time it works, some of the time it doesn't. Slower load (slower machine, larger project file) made that issue more prominent.

That's been fixed.

I've also moved around the load order for the previewer, using a later event to load the previewer which should ensure that a topic is loaded by the time the preview gets initially rendered. As a bonus it looks like the separation into two separate events offsets the initial load of editor and preview enough to actual speed up the load of the two controls.

The new load scenario will ensure the preview renders after the editor is loaded which should provide reliable results and no empty previews in addition to better performance.

To be clear I was able to duplicate what you saw previously: Preview starting up blank and the initially loaded topic getting cleared and saved. With these changes I can now no longer duplicate that in any way. Additionally it seems that the initial render, and especially a project load after DM is started is quite a bit faster as the two controls are not competing for resources as much at load time (both first time and repeat loads).

Let's see how you fare for this - since you are Break Master General (much appreciate BTW)...

Rick

I shall do that and let you know. I did try changing the size of the preview pane and DM but not the toggle. Of course, fired it up this morning and the preview worked. OTOH it also showed the main topic as changed, etc.

IAC I will try the 0.4.11 and let you know what happens.

It is a gift and I accept my new title. 😃 I have a member of my team who always seems to be able to break stuff. She is highly valued.

Imported to a new DM project and the top-level topic appears correct. However, the second 0-byte topic I highlighted is still missing text. Perhaps related to the topic type in HB being HEADER?

Here's the json from the export file:

  {
    "pk": "_5UT0OGD83",
    "parentpk": "",
    "viewmode": 2,
    "topic": "FAQs",
    "type": "HEADER",
    "external": false,
    "body": "This section contains answers to many frequently asked questions.\r\n\r\n<%= ChildTopicsList() %>\r\n\r\n",
    "endbody": "",
    "class": "",
    "method": "",
    "scope": "",
    "syntax": "",
    "parameters": "",
    "returns": "",
    "remarks": "",
    "example": "",
    "seealso": "",
    "keywords": "",
    "properties": "",
    "methods": "",
    "events": "",
    "expanded": true,
    "helpid": 76,
    "nocontent": false,
    "sortorder": 40,
    "updated": "2023-10-03T22:30:49Z",
    "followup": false,
    "signature": "",
    "assembly": "",
    "namespace": "",
    "inherits": "",
    "inh_tree": "",
    "implements": "",
    "exceptions": "",
    "contract": "",
    "static": false
  },

Also, is ChildTopicList supported in DM? I searched the DM docs but came up empty. That's the other piece that's missing from my FAQs topic.

The topic type dropdown does not appear to be populated.

When you do the import can you do the following:

• Import
• Check that topic on disk, and see if there is content in there (FAQ)
• Immediate check into Git (before navigating or anything else)

The way the importer works is that it pulls in topics and immediately at the time of import writes the content out to disk. So the file should be there - unless there's a problem with the topic import specifically for header topics (which I don't see here but I have to check closer).

I suspect the problem is - as before - a situation where the topics gets cleared on navigation somehow.

Make sure you are on 0.4.1 and you're not still running 0.4.0... (you can turn on the version in the title bar in settings to make this easier).

+++ Rick ---

Hey, could you please break out these bug reports into separate messages. Like the drop down issue is a separate issue as is the single topic update as is the empty topic issue.

It's really hard to discuss 5 different issues on one thread.

+++ Rick ---

I'm aware of the Dropdown issue after creating a new project or importing. Reloading the project fixes that (ie. close and re-open). I have an internal issue for trying to track that down - I had changed the logic to completely reload the project after import but apparently even that is not fixing the issue.

+++ Rick ---

Yep, that's what I did in 0.41, and why I posted the 2 screenshots, and the json from the index for reference. But I can do it again.

My apologies; you are right, of course. I will start separate threads as needed. Of perhaps the issue tracker in GH is better than the board?

Reloading the project brought the topic type list back as you said.

Reloading the project brought the topic type list back as you said.

Found and I think fixed that issue - the bindings were not refreshing when a new project was loaded or assigned. That should be fixed for the next update.

OK, here goes.

In version 0.4.1

Went to Tools->Import, and did what was needed:

Ran the import, and the file system looks like this:

As you can see the FAQs.md file is 0-byte.

Brought up the DM Git dialog and did a local commit.

Checked the file system and the FAQs.md file is still 0-byte.

Let me know if you want me to try something else.

So did the topic import at all? When you first imported and before you did anything else is that FAQ file 0 bytes?

Is it only that 'folder' or any folder - looks like only that one.

I've pushed up v0.4.2 which has a few fixes for the topic list drop down, class imports and a few other things.

Unfortunately I just ran into another instance of the root topic getting wiped out. Damn it I can't get a reliable repro for this. It works fine 99 times and then fails once.

+++ Rick ---

Can you send me your Help Builder Project and Export file? I'd like to import the whole thing and see what I can see. I'm doing my own here but everything that I can see is working fine with my old projects...

+++ Rick ---

Do you still have the FTP account I setup for you? I'll zip it up and drop it there. BTW the FAQs folder is properly populated. It's the md file should should consist of a single sentence and the output of the ChildTopicsList helper.

No - don't have it. Can you send a link privately?

+++ Rick ---

I sent you something. Just let me know you got it when convenient.

Thanks for the fun time behind the curtain this afternoon.

I'll let you know if I come up with anything interesting on the FTP publishing. Thanks to git diffing I actually found a 2nd topic in that API project that was messed up. I was able to recover what I needed from git itself and am chalking up the issues to the pre-beta version of DM I was working with at the time.