1 2 Previous Next 26 Replies Latest reply on Apr 27, 2010 1:32 PM by Cable2001

    Data loss bug : Spontaneous and erroneous import matching of new fields in specified imports !

    Vincent_L

      Summary

      Data loss bug : Spontaneous and erroneous import matching of new fields in specified imports !

      Description of the issue

      Hi I've reported this more than one year ago (15 april 2008) to FileMaker Italia Reply [Case ID #: 907402], that was escalated to the US : "Regarding the technical case open the US team told me that the issue is still under investigation but unfortunately we don't have any update at the moment."  I gave a sample file demonstrating teh problem and even made a video about it (too bad we can't put attachment in here) Waht's the bug is about ? everytime I'm adding a field in my database, I've to recheck every specified import whose target is this table, to uncheck the matching that FMP automatically and erroneously created itself to that new field. Yes you heard it right, filemaker will automatically add a field import in saved imporrts ! As I've just spend two hours once again, to tediously fix FMP mistakes, I decided to repost it.    --- Original Message ---Hi,I think I've discovered a bug that destroyed a lot of user inputted data inmy solution for several times.New created fields have their data replaced by scripted specified XML/XSLTUpdate matching records imports that were created before the new fields. Asif someone would add the new fields as targets by hand.So every time I add a new field in my solution, I have to change all myscripted specified XML/XSLT Update matching records imports to remove thenew fields that Filemaker has, by itself, set as targets !!!The scenario is this one.I've a product catalog where user input new products, their sku, theirdescription etc.Every morning the inventory is imported via a scheduled XML with XSLT importthat gets the skus and their inventory count, as well as some otherinformation.I edited my solution to put a new field that user needs to input, like"complementary information" for instance.The users filled that extra field for all day.The next morning they complained because all the data they've inputted wasgone, replaced by some other values.So I looked at the import script steps (specified) and noticed thatFilemaker added by itselfBefore my import wasSKU = SKUQty -> Invetory countD1 - Product nameD2 - I've added in database definition a new field called "new complementaryinformation" and then Filemaker changed my import script steps toSKU = SKUQty -> Invetory countD1 - Product nameD2 -> new complementary informationI've more complex solution where it is like that :SKU = SKUQty -> Invetory countD1 - Product nameD2 - info 1- info 2- info 3-> new complementary informationSo Filemaker decided by itself to put the data of D2 in my new field, ofcourse overwriting all the data that was inputted !I tried to reproduce the bug and wasn't able to reproduce it withtab-tab-return imput file, nor from Filemaker file, nor from XML withoutXSLT as the inputI guess this only occurs with XSLT, and specifically the msdso_attrib.xsltexample xslt Filemaker ships with as an example.To help you figure out the problem, I've made a movie using the "XMLExamples" folders that is installed with Filemaker advancedThis is very serious and I've no workaround I need to use this XML/XSLTimport 

        • 1. Re: Data loss bug : Spontaneous and erroneous import matching of new fields in specified imports !
          Vincent_L
            

          Thanks LaRetta for weighting in !

           

          So Mr Pm, since you read everything, I'd really, really ave your take about this issue, I'm thinking about you everytime I loose 2 hours of my life thanks to this mess ! 

          • 2. Re: Data loss bug : Spontaneous and erroneous import matching of new fields in specified imports !
            Vincent_L
              

             

            Here's a SERIOUS offer for Mr PM.

             

            I know that bitching is easy, fixing is more complicated. But I'll do my part. 

             

            I'm willing to take a week of holidays to go to FM Inc US Headquarters and explain face to face the issues we have and provide you ideas (often very simple ones) on how to improve the product and address the many hurdles we're facing as FMP developers.

             

            I would compile the most frequently raised issues by users.

             

            We would sat down with the teams to see how and when to fix things, and IF YOU WANT, I'll then publish a report for the FMP community.

             

            I'll even do this at my own expense, that means FREE for FM Inc (considering I'm in France that's not a slouch offer)

             

            • 3. Re: Data loss bug : Spontaneous and erroneous import matching of new fields in specified imports !
              MartinBrändle
                

              I agree partially with LaRetta, especially with the reports of developers constantly trying to raise awareness of bugs and not getting enough feedback.

               

              The FileMaker suite probably got so complex and also open (think of what we could do a few years ago with FM; think of the extensibility that we have now) that it gets very difficult for the SE and QA people of FM Inc. to have an idea of what developers do or are trying to do with FM.

               

              At least we have this forum now and know which bugs may exist. Before we had no exchange and discussion between developers, and also no confirmation of FM Inc. that a bug could be reproduced. This forum was a step forward.

               

              A next step would be something like Bugzilla.

               

              Another step would be to extend ETS by a beta testing phase for a broader public (the TechNet members) and also have enough time scheduled for corrections before the next version is released. This could be organized similary as Apple does with its developer versions for the next OS releases. A NDA still may prohibit that too much gets known publicly.

              • 4. Re: Data loss bug : Spontaneous and erroneous import matching of new fields in specified imports !
                comment_1
                  

                Martin Brändle wrote:
                it gets very difficult for the SE and QA people of FM Inc. to have an idea of what developers do or are trying to do with FM.

                That's what product managers are for, I think.

                 

                I agree that listening to users is not as simple as it may seem; you have to be very careful WHICH users you listen to. This is true for any application - most users know only what they do and need, and have no way of judging how mainstream or bizarre their outlook may be.

                 

                It is even more problematic with Filemaker, because most users do not truly understand the application and its underlying philosophy. Thus script triggers were for a long time the top wished-for feature - spurred, in 99% of the cases, by the "need" to overcome an inadequate data model.

                • 5. Re: Data loss bug : Spontaneous and erroneous import matching of new fields in specified imports !
                  TSGal

                  Thank you all for you comments.

                   

                  Nothing I can say will provide any comfort, but here is what I do know.

                   

                  Yes, the import problem has been around for some time.  Yes, our Developers and Testers are aware of the problem.

                   

                  Internally, we do have a system similar to Bugzilla (a bug tracking system).  From this system, I am able to enter information and receive confirmation from Development and/or Testing.

                   

                  The Product Manager has read this thread.

                   

                  Our Sales Engineers deal with customers every day, and they are very familiar what customers are trying to accomplish.  Their feedback is essential for Product Development.

                   

                  Our yearly Developer's Conference is attended by many of our Developers, and they also get to see what our customers are doing with FileMaker.

                   

                  This forum has only been active since last October, and this has been a learning experience for both myself and FileMaker, Inc.  Having a "Report a Bug" board has been very successful in reporting issues.  In the past, you could report a problem via a web form using limited space, and you would not receive any feedback.  Now, you are able to explain in greater detail and get verification.  Since this is now public, other users have also chimed in, and this provides more information to the issue.  Hopefully, this will continue to evolve/expand.

                   

                  We have never viewed the customer as the problem.  When a problem in the software is discovered, it is entered into the database, and it remains there.  It never gets removed.

                   

                  There have been many suggestions how to improve this forum.  Some of these suggestions have been implemented, but there are many more but they require time and resources.

                   

                  TSGal

                  FileMaker, Inc. 

                  • 6. Re: Data loss bug : Spontaneous and erroneous import matching of new fields in specified imports !
                    comment_1
                      

                    TSGal wrote:
                    When a problem in the software is discovered, it is entered into the database, and it remains there.  It never gets removed.

                    That's the gist of the complaint, I think.

                    • 7. Re: Data loss bug : Spontaneous and erroneous import matching of new fields in specified imports !
                      Vincent_L
                        
                      First Thanks TGsal for your answer,
                      Let's not forget The thread started about a DATA LOSS PRONE BUG, and then evolved about the lack of proper response from FM Inc.

                       

                      Yes, the import problem has been around for some time.  Yes, our Developers and Testers are aware of the problem.

                       


                      I'm AMAZED that a DATA LOSS (and huge timeloss) BUG, is know by your team and not fixed after 3 full releases. It's a DATA LOSS BUG = D A T A  L O S S   B U G, for god's sake. What can be more important to fix it ASAP in a DATAbase. At the very least a WARNING in the documentation would be mandatory.
                      How can we trust, how can our customers, trust a company that leaves a DATA LOSS BUG for more than 3 years without doing ANY action about it !
                      I can't imagine why those were more important, than a DATA LOSS BUG:
                      - Autosort, and intern cool idea,  that pisses off a lot of disgrunted customers, who can't understand why there list view moves itself. I'm not agianst teh  feature, I'm agains an unilatteral change of behaviour without any clue about the consequences.
                      -  a Default script menu, not customisable of course, that NO ONE uses
                       etc… 
                      As Filemaker developpers we put a lot of energy in our solution, and we have too answer to our users/customers about the shortcomings. We have to be advocates of FM Inc, to justify our technical choice. It's VERY hard when Because FM Inc makes BUGS, doesn't fixe them after 3 years, the user loose DATA, or has his solution changing behaviour. 
                      I'm tired to have to justify FMP to my users 
                      Finally, Mr PM, I think that having TGsal playing the role of a fuse, when your role is clearly exposed, is not nice. I'd like to hear from YOU. Adobe sucks, but its seems to do it's homework. Take cues from them. at leats John Nack, PS's PM talks.

                       

                      • 8. Re: Data loss bug : Spontaneous and erroneous import matching of new fields in specified imports !
                        TSGal

                        comment:

                         

                        Sorry I wasn't clear in my last post.  All problems are reported in the database.  When a problem is fixed, it is marked as "fixed".  If a problem cannot be reproduced, it is marked as "cannot reproduce".   For the latter case, if another customer runs into the same problem, we try to get as much information possible to try and replicate the problem.

                         

                        LaRetta:

                         

                        It's my fault for not making sure these issues get into the Knowledge Base.  I will begin working on it later this week when I have some time.

                         

                        On a side note, when a search is executed on the Knowledge Base, a search is also executed on the forum.  The result is a listing of relevant articles from the Knowledge Base, and below that, relevant postings from the forum.  That may be why people are finding their way to the forum.

                         

                        I don't have an answer why this wasn't fixed in a previous release.

                         

                        TSGal

                        FileMaker, Inc. 

                        • 9. Re: Data loss bug : Spontaneous and erroneous import matching of new fields in specified imports !
                          Vincent_L
                            


                          I don't have an answer why this wasn't fixed in a previous release.

                           

                          TSGal

                          FileMaker, Inc. 


                           
                          If you don't know, who knows, I think the issue is serious enough that we need a clear explanation on why this is not fixed. And to know how many sundays afternoon I should plan to fix the consequences of this bug. Because, contrary to all FM Inc employees, It's not my job to fix FMP bug. My Job is to deliver a database solution to my users, these users, or those customers for professionnal FMP developers, are willing to understand that The FMP dev use his billing time, his working hours to improve the solution. They don't understand we'd use some of this time to workaround FMP Bugs. We have to hide those from them because otherwise they'd tell us to ditch FMP and they'd blame us for using a buggy tool. So we do those workaround for free, in our FREE time. 
                          The contrast : having to fix things in our scarce free time, because FM Inc (a compagny that lives because we pay their solution and because we prescribe it to enterprises), on its working ours and whose only reason to live is to produce software, is failing to fix reported ad nauseam bugs, is unbearable.
                          TGsal, Mr PM, you're reading me at your working hours it seems. I'm writing this from France, it's midnight, I should watch a movie not complaining about your bugs ! 
                           

                           


                          • 10. Re: Data loss bug : Spontaneous and erroneous import matching of new fields in specified imports !
                            comment_1
                              

                            TSGal wrote:

                            comment:

                             

                            Sorry I wasn't clear in my last post.


                            You were perfectly clear.  It was just the way you put it, it was an irresistible setup for a punchline. Now don't make me complain that FMI has no sense of humor either...


                            • 11. Re: Data loss bug : Spontaneous and erroneous import matching of new fields in specified imports !
                              Vincent_L
                                 Another week passed, One more our spent during the weekend to fix this bug mess in my numerous import scripts, and yet, still not any news on this DATA LOSS prone issue !
                              • 13. Re: Data loss bug : Spontaneous and erroneous import matching of new fields in specified imports !
                                Vincent_L
                                  

                                My point is not to let go the thread and let this fade away, I want FM Inc to get a sting every week till that's not fixed. I want every new people redaing this forum to stumble on it, I want PM to see it every day.

                                 

                                Because it bugs me every day ! 

                                • 14. Re: Data loss bug : Spontaneous and erroneous import matching of new fields in specified imports !
                                  hschlossberg
                                    

                                  Isn't the workaround rather simple?  Just refrain from adding or deleting fields, or else just make sure your import source and target fields all have matching names.  Just takes some advanced planning ;-) 

                                  1 2 Previous Next