"Out" file overwrite causes mixed results

MegaZine3 ForumCategory: Technical"Out" file overwrite causes mixed results
donflinn asked 12 months ago

The folder named, _mz3Data, is created in the Convert stage of producing a flipbook in the directory location named in the “Out” field of the input data. If a folder of the same name from a previous Convert is still in the same location, the new data is mixed in with the old data, and some of the filenames and data remain from the previous Convert.

I would suggest a warning in the Convert code to the user that the _mz3Data folder already exists, and a choice to save it in a different location or to delete the old folder.

hnuecke replied 12 months ago

Don, I got your comment, which I think was meant as reply to this post. But somehow it ended up as a comment to the logo you attached. I have no idea what you did, and only can find your comment in the admin area, So I paste it here for reference:

comment from donflinn:
In developing my flipbook, I go through many incremental changes to the pdf file, such as changing wording, pictures, etc. At each stage I test it with mz3Tool. Sometimes I will also change the name of the pdf to indicate a new Version number. .

If I do not change the Out address when testing each stage, and do not delete the _mz3Data folder, the folders and files in the _mz3Data file have mixed “Date modified” values, some being the original value, and some being the recent value. This is confusing.

If I change the version number in the name of the pdf, and do not delete the _mz3Data folder, the same Date modified confusion occurs. In addition the folder names and filenames in _mz3Data remain as they originally were, and do not reflect the new version number of the “In” pdf file. When this happens, I am not clear as to whether the generated flipbook reflects the new update or the original file.

Why don’t I just change the Out address when I change the version number? One reason is that I really don’t want a bunch of locations with output data. Also, I guess I didn’t know I had to, in order to be sure to get clean output. A reminder warning to delete the old _mz3Data folder would tell me that I should.

Why doesn’t the program delete the old _mz3Data folder each time. Maybe the user wants to keep the old file. A reminder warning to give the user the option to move or delete the old _mz3Data folder would tell the user to make a choice.

You might say that I do not follow correct developer procedures, and I would agree. As a user, not a developer, I don’t always take time to read the instructions or watch the video and subtitles. I tend to learn from my experience and mistakes. That’s why I need better guidance by the program to do things correctly.

hnuecke replied 12 months ago

The “_mz3Data” folder is the working area for all books with the same path, as defined in the “OUT” field.
Book specific data is separated by book, general settings are shared between all books using the same OUT path.
This allows defining e.g. a common favicon icon, the plugins used, … (all what is configured in the “Main” panel) between different “books”. Helpful e.g. in case of a magazine, where all issues should look the same.

If that “feature” is not requested, a different OUT path should be chosen. That can be done in placing the PDF file in a different folder, since by default the OUT path is set to the folder the PDF is located in. Or in defining a “custom” OUT path.

If changes are made to an existing book, new data overwrites old data. What should be the expected behaviour. Can you please be more specific in what file names and data causes confusion and are “mixed”?

Or in other words: What do you think can go wrong, which justified a warning?

6 Answers
hnuecke answered 12 months ago

Still not sure what is confusing you 😉

If you start working on a new PDF file (a new PDF file is any file with a different name), that will be kept separate; in separate sub folders within _mz3Data. And you have to make a new CONVERSION.

All "books" share some files in common, like all MegaZine3 files and folders in "onlinePackage" and the config files in "configData"; with e.g. info about your license key so you do not have to enter that with every new book.

But all your book specific data is kept separate using the PDF file name.

I do not see how that will cause confusion. If you know the name of the PDF file you want to check, you’ll find the relevant data in folders named after that PDF file name.

If the name does not change and you provide a new PDF file with the same name, that will overwrite the old PDF file. But as long as you do not "use" that file, i.e. make a new CONVERSION, the old files will be used.

So it is up to you to start using the new PDF or still check the old version.

Why do you need the modification dates? What do you think is going wrong and not working as expected (and explained above)?

Why do you think the dates are incorrect or the wrong files are used? There are different dates available when browsing, and had remember I was confused myself because the nameing was misleading.

Or do you talk about the dates reported by mz3Tool?

Those dates are the dates mz3Tool reads from the actual PDF file.

Or do you not see changes you made in your PDF file (using the same PDF file name)?

Did you start a new CONVERSION after you provided the new PDF file?

Otherwise the old, available data is shown.

How did you "provide" the updated PDF file (i.e. the PDF file with the same name but modified content)?

Easiest is dragging it again to the IN field.

If the file name did not change, it also is sufficient to click into the IN field and hit .

This also will refresh the PDFinfo and prepare for using the modified file.

But in any case: You need to click "CONVERT" again to use that file.

mz3Tool does a new "CREATE" automatically, so changed page sizes or page count are considered. But without a new CREATION the existing, old "pages" (i.e. converted files: swf, jpg, png images) will be used.

Main reason for that approach:

Dealing with big books takes some time doing a conversion. Especially if you use different qualities and image types (e.g. swf AND jpg AND png). And a conversion run cannot be aborted.

Therefore the decision when to make a new conversion, and under what conditions (number of pages, page range, image types, scale levels, book type: single/spread pages, …) is your decision.

You just have to remember to do it 😉

Working with the dropdown list sould make it easy finding the book you’d like to work on.

You can sort for date, file name and path; and even destination (the related OUT path):

At the end there shouldn’t be no need working with the mz3Data folder directly; or deleting them.

The mz3Data folder is important for mz3Tool to work correctly; it’s kind of "database"; keeping all the work.

I guess you are used to work differently, as you indicated ("I’m a user’). And that’s OK. We are trying to give all users the choice to work the way they want. That’s why you e.g. can select a different OUT path than the one proposed by mz3Tool.

But it is always a compromise between comfort based on standard "procedures", freedom of choice (adjustments, customization) and ease of use.

I’m sure once I understand better how you’d like to work, and can help you in giving guidance how to achieve that. There will be ways 😉

Hopefully some of my hints help to address areas of confusion/misunderstanding!

Don Flinn answered 11 months ago

Correct me, if I am wrong, but these are my understandings from your forum responses and previous emails.

  1. mz3Viewer searches myDocuments (in Win 10, Documents) folder for folders containing mz3 files. Not clear, is the question of whether this is only for the first level of folders or for folders within folders within folders…
  2. It is not necessary to specify any “Out” file location before doing a Convert in mz3Tool, because the “_mz3Data” output folder will, by default, be placed in the “In” folder where the pdf file is located.
  3. However, if an “Out” folder is specified, the “_mz3Data” folder will be placed in the designated location.
  4. If an “ _mz3Data” folder from a previous Convert is in the location where the new Convert output folder is to be placed, the old folder will be overwritten with the new data. However, if there are no changes in the existing folders and/or files within the “_mz3Data” folder, the original folder and/or file will not be replaced, as can be verified by the value in the “Date modified” column.
  5. If any change is made to the name of the input pdf file, it must be put in a different “In” folder in order to keep from mixing new output data with old output data in the “_mz3Data” folder, if the default location or a common specified location is used. However, if a new “Out” location is specified, there would be no mixing of data.
  6. There is no facility within mz3Tool to remove a previously specified “Out” location. Up to 20 locations can be specified before replacement begins.

Again, correct me if I am wrong, but I believe these are possible problems that can occur because of these specifications.

  1. If mz3Viewer finds more than one folder with mz3 files, it will include all of them in the viewer. Instead, the user should have control over which files(s) to include.
  2. Since there is no facility to remove a previously specified “Out” file, it is not possible to use the default “In” folder after an “Out” folder is specified.
  3. When the “_mz3Data” is overwritten, the folders and files within it which are not changed retain the old “Date modified” value. This can be confusing to a new user.
  4. If any change is made to a pdf file name, such as a Version number, the pdf file must be put in a new folder in order to avoid mixing new and old data files. In the case of many Versions required to develop a large flipbook, a proliferation of similar folders and files throughout the system would be produced, and confusion would occur.

My recommendations to alleviate these problems.

  1. There MUST be a facility within mz3Tool to remove the “Out” location(s).
  2. There MUST be an option to use the default “Out” location even when there are other “Out” locations in the list.
  3. Overwriting of a previous “_mz3Data” folder should include ALL folders and files within, not just the ones that change.
  4. When there is no change in the pdf name, the user should be warned if a previous “_mz3Data” folder exists in the “Out” location, and given the option of moving it or overwriting it.
  5. If the “In” pdf contains a modified name, the program should completely overwrite any previous “_mz3Data” folder in the "Out" location.

BTW, the text format changes above are caused by the _ character followed by any character other than a space, as in _character. I cannot prevent this.

hnuecke answered 11 months ago

Wow, thank you for the detailed and well organized analysis!

Let me address your points one by one.

There are two questions on mz3Viewer (1.1 and 2.1)

1.1 Search stratagy mz3Viewer

To make it most comfortable for the user a combination of search strategies is used. The goal is to find a good compromise between performance (fast response, short search times) and number of hits.

Search depth

The search depth is limited to one. Exception: the folder name contains "mz3". In such case also sub folders are searched.

Search pathes

You are right, a former version of mz3Viewer searched in myDocuments, desktop and other folders; with search depths of 1…3; depending on the path. That was far too complex and could take pretty long if the user had much data. Hard to predict, several disadvantages with no real value. So that was abandoned

Actually only the documents/mz3Viewer/ path is searched. All Mz3 Titles that are entered as zipped Offline package will end up in that folder. And if you plan to collect Mz3 Titles for viewing with mz3Viewer, that would be the best location. Those Mz3 Titles will be found automatically.

To help during the "development process", i.e. while working with mz3Tool (or mz3Demo), also the remembered "destination" pathes are searched. Or in other words: Every path that shows up in the "Recent destination" drop downlist is searched also. With that you will find all Mz3 Titles you worked on recently also in the mz3Viewer list.

If you start mz3Viewer with a file (an mz3 or mz3z or zip file), also that path will be added to the list.

You can do so in double clicking on such file, if that extension is assigned to mz3Viewer. Or in "opening with" such file and selcting mz3Viewer.

You also can drag&drop such file on mz3Viewer (anywhere in the window) or the program icon of mz3Viewer.

Updates and Maintenance

Updates and Maintenance

If mz3Viewer cannot find a saved path during start, that path will be deleted in the list automatically.

Summary:

This strategy should help seeing Mz3 Titles of interest in most cases. You can sort each column to quickly find the Mz3 Title you are interested in. You even could enter some characters of the name and mz3Viewer will show the first Mz3 Title containing that string.

If mz3Viewer does not find what you expected, simply drag&drop your file on mz3Viewer and next time it’ll be in the list.

I do not see any reason qhy the user should restrict that automatically built list. And as explained: he could in simply deleting or renaming folders he does not want to see in the list.

This should help most users finding "their" titles quickly and easily, and still give full control.

First paragraph:

1.2 … 1.6 Mostly correct; and all that is on purpose

"mixing" of Mz3 Titles does not harm, but offers the option to share certain genral configuration settings (as explained in my previous answer). Once you create an "Offline package", the Mz3 Titles are serparated and no more mixed.

It is by intention that existing data is not deleted. Only that way you can convert/exchange a selected range of pages.

Helpful in case of Mz3 Titles with hundreds of pages and a big time saver.

Also needed if you do not convert again but just do changes that only require a new "Create". Like adding video or sound, changing the kind of pages (stiff, flexible), change other book attributes, …

Also in case of multilingual Mz3 Titles (more than one language supported in the book) a "mixing" is needed.

As you correctly realized, if that default behaviour is not what you need/like, you can defnie a different OUT (destination) path or place your PDF file in a different location; i.e. not "mixing" your PDFs. Or in other words:

mixed in -> mixed out 😉

mz3Tool relies on finding an _mz3Data folder. Therefore that folder always is created. And you never should rename it! Otherwise that folder will be created again and all former data is not seen/found any longer.

See also the blog post MegaZine3 Variants and Folder Structure where in the last paragraph that is explained in more detail.

And finally on your point 1.6: It is possible to delete entries in the list; in two ways:

  • indirectly in deleting, moving or renaming the respective (real) path. During start mz3Tool like mz3Viewer checks if the pathes in the list do exist. In case they don’t, the reference is taken off the list.
  • there is a button now that allows to delete the complete list. Especially if the list is long this can be the faster, more comfortable approch.

Second paragraph

Most points should be addressed already.

On your remark on versioning:

If you do not like the "automatic" handling (or "mixing" as you call it; which should not harm and is similar as "mixing" PDF files in having them in the same folder?!?):

Why don’t you manage your versions in putting them in sub folders named after the version?

This is a standard approach and does not clutter but organize. And then all data is separated in _mz3Data sub folders ot the respective version folders

Third paragraph

3.1 It is (see above: either use the delete option (trash icon) or hide the path (delete/move/rename)

3.2 don’t understand; you can define any path as OUT destination, also another "default" path

3.3 not sure I understand fully; Not touching (overwriting) data of other files is the expected and desired behaviour, I think.

3.4. That kind of warning already is given: if data exists, i.e. if the PDF file was converted before, the Mz3 Title will be shown; without conversion. A conversion always only happens if initiated by the user.

If no Mz3 title is shown: it most probably is a new file name or it was never converted before.

If a book shows up after a PDF file was entered/selected, it is a known name and the PDF file was converted before.

3.5 Same as 3.3? No, although data of different PDF files is kept in the same folder ("mixed"), this does not mean that changing one file should delelte all others (most users would kill me if that happened ;-).

But you always have the option doing so:

Everything is transparent, the folder structure is know and you in many cases can open the desired folder with a click on the "…" button or the "open folder icon".

Summary

you might have a special use case with many versions (and not many different Mz3 Titles).

mz3Tool is not a version management tool. That is up to you and you have all means doing so. And I hope I could show you some options for doing so.

hnuecke answered 11 months ago

Thank you for the detailed "analysis" 😉

Answering all those points here would become a very lengthy, hard to write and read effort. Therefore I combined answers and hints in a blog post. This offers the additional option of a translation.

Readers of the blog that do not yet pay attention to this forum will get the information easier, too.

In short: Most points are either already available and possible, or can be addressed with kind of change in structure and organization.

I hope you will find answers on all your points in this new blog post

If still something is unclear, misunderstand or not answered, please let me know (again)

hnuecke answered 11 months ago

I created a new blog with all answers and hints.

This will help more users and offers th option of a translation.

I hope I addressed all your points. As you will see most of your requested "features" are available already or canb be solved with an adjested fodler structure

I also provided some details about the "search strategy" of mz3Viewer. YOu are completely right: That not only was missing but also had changed throughout the last versions ("learning curve").

If there’s still something not answered or misunderstood, please let me know!

hnuecke answered 11 months ago

Thank you for the detailed analysis!

I created a new blog with all answers and hints.

This will help more users and offers th option of a translation.

I hope I addressed all your points. As you will see most of your requested "features" are available already or canb be solved with an adjested fodler structure

I also provided some details about the "search strategy" of mz3Viewer. YOu are completely right: That not only was missing but also had changed throughout the last versions ("learning curve").

If there’s still something not answered or misunderstood, please let me know!

hnuecke answered 11 months ago

Thank you for your answer!
I created a new blog with all answers and hints.
This will help more users and offers th option of a translation.
I hope I addressed all your points. As you will see most of your requested "features" are available already or canb be solved with an adjested fodler structure
I also provided some details about the "search strategy" of mz3Viewer. YOu are completely right: That not only was missing but also had changed throughout the last versions ("learning curve").
If there’s still something not answered or misunderstood, please let me know!

hnuecke answered 11 months ago

Thank you for your answer!

I created a new blog with all answers and hints.

This will help more users and offers th option of a translation.

I hope I addressed all your points. As you will see most of your requested "features" are available already or canb be solved with an adjested fodler structure

I also provided some details about the "search strategy" of mz3Viewer. YOu are completely right: That not only was missing but also had changed throughout the last versions ("learning curve").

If there’s still something not answered or misunderstood, please let me know!

hnuecke answered 11 months ago

This will help more users and offers th option of a translation.

I hope I addressed all your points. As you will see most of your requested "features" are available already or canb be solved with an adjested fodler structure

I also provided some details about the "search strategy" of mz3Viewer. YOu are completely right: That not only was missing but also had changed throughout the last versions ("learning curve").

If there’s still something not answered or misunderstood, please let me know!

Don Flinn answered 11 months ago

The Second-Most Unread type of document is the User Guide for computer programs. After installing a new program most users select the option, "Start Program." Users first want to see what the program will do before they take time to read the directions contained in the User Guide. Besides, most of the jargon in the User Guide is not understood until after the user has played with the program. If the program is well documented internally, the User Guide may never be read. Users, like me, depend more on Help files, Google Searches for similar problems, and Chat sessions for guidance. I admit I have never before seen or read a blog recommendation on how to run a program. As a result , I seem to have gotten off on the wrong foot with mz3Tool.

Your recommendation for structuring files with a separate folder for each new named pdf version of the same project was not followed, and the fact that the output would be put in the same folder as the pdf by default was not obvious to me. I thought it was necessary to specify the output folder, and used the same output folder for each named version. I did not need to keep the earlier version of the output, because I could always rerun the pdf. I expected the old output file would be completely overwritten, and I could move the new one to a safe place if I wanted to keep it. That’s where things got mixed up since only the existing output files actually changed would be overwritten. So I wound up with a mix of 2 versions of some of the files in the output mz4Data file. This precipitated this whole discussion.

That’s when I suggested a warning message prior to running a CONVERT if there was an existing mz3Data file in the output location. Such as:

Want to save your changes to "Filepath_mz3Data"?

SAVE DON’T SAVE CANCEL

If this warning was in the program, AND the program deleted the existing file before doing the CONVERT, I would be able to recognize that I had to move the existing file or it would be deleted. I would be happy, and anyone doing the same thing in the future would be happy. It would not affect those who used the recommended folder structure.

BTW, the Most Unread type of document is the New Car Owner’s Manual. Did you read yours completely before starting your car?

hnuecke answered 11 months ago

Hi Don, I love this (final?) answer 😉

I coudn’t agree more: Most people do not follow the "RTFM" (read the f…ing manual) thing 😉

And that’s the reason why we gave up writing and updating user manuals (we had some in the past ;-).

I also agree that a program should "behave as expected". The problem is: expectations are different! I hope you recognize and accept that we made big progress over the past year, and many things became very simple!

And we will never stop improving (and listening).

But as said: expectations (and people) are soooo different! At the end I like that; makes our life interesting. But it also means that before just following one advice (or an own idea!), it is better to either test and watch or think about it deeply or talk with several people with different background knowledge about it… Or best: Do all of that.

I believeyou and I are finally in synch. And I will think about your "proposal", and we might implement something like that.

And we will continue "training" our customers day by day in

  • writing regular blogs with Tips&Tricks,
  • sharing User Experiences,
  • presenting "real" Projects, showing details of Customer Solutions
  • providing Technical Details

This way everybody can pick what is of interest. Or later use the search function , the categories or tag cloud to find answers.

And for those who do not like reading, we offer a bunch of videos. All just running for a couple of minutes. Where you can look "across my shoulders" (see the answeri in FAQs to "are there videos?" for a list with links).

I admit: We are not there yet, and probably never will be. But we want to learn and improve, and need your input for that.

Thank you for being that patient with me and making your points. And please follow us and raise your voice whenever possible/necessary 😉

And before I forget… Believe it or not, I read the user manual of any new car completely!

Although I know that brand, and often even the model, since > 30 years.

And I always detect something new and interesting. Admitted: I quickly flip over the "common things", but nevertheless… 😉

Don Flinn answered 11 months ago

‘ figures 🙂

Your Answer

Question Categories: Technical.
Question Tags: _mz3Data.