1 2 Previous Next 19 Replies Latest reply on Apr 24, 2016 3:59 PM by beverly

    Integration of XML via SFTP

    mchancevet@gmail.com

      Hello,

      I'm researching integrating a parcel service's ordering API. The service accepts parcel consignment orders as XML via SFTP.

      I'm new to XML and SFTP. SO get ready for some stupid questions.

       

      I've accessed raw XML as a text string from a Google API (I think I used 'Insert from URL', ... just checking, yep that's how I got it.)

       

      If I need to send a parcel manifest (effectively an order for service) as an XML via SFTP:

      • Can I generate text (via calculation) and save as file type ' .xml' in some temporary folder.
      • Then can I use Base Elements Plugin 'BE_FTP_Upload' to upload the XML.
      • Is that too simple?
      • AM I missing something?
      • This is very different to a curl 'POST' right OR is it?
      • Can I use a 'Post' instead and still use SFTP?
      • Is a 'Post' a 'file transfer', I'm thinking of it more like a request.
      • This same solution is receiving orders as 'Posts' which it has to generate 'responses' to. (it's doing this via a PHP layer on a dedicated order received website) The incoming order posts contain JSON strings which seem to fulfil an equivalent function as the XML in the parcel integration, i.e. define the order for a product or service.
      • How are these two systems different or the same?

      Thanks,

      Morgan

        • 1. Re: Integration of XML via SFTP
          wimdecorte

          The SFTP is indeed different than doing an HTTP or HTTPS POST.  Different protocol.  Whether you can use HTTP POST depends on whether the service allows it.

           

          Yes: XML is just text so you can construct it any which way you want.  Including using FM's built-in XML export (you will need an XSLT stylesheet to transform FM's XML into the XML grammar that the parcel service expects).

           

          mchancevet@gmail.com wrote:

           

          • This same solution is receiving orders as 'Posts' which it has to generate 'responses' to.

           

          Not sure I follow this: are you talking about the FM solution or the parcel service?

           

          If you are describing the FM solution then your question "How are these two systems different": because they are two different systems... each receiving input for their own purposes.  One uses XML as its input, the other JSON, probably very different structures and data elements required.

          • 2. Re: Integration of XML via SFTP
            beverly

            sendreceive is not 'natively' possible with FM (as a single process). Plug-ins can facilitate this type of cycle (including requestacknowledgerequestresult cycles).

             

            and the plug-ins can also assist with the SFTP (secure FTP)

            beverly

            • 3. Re: Integration of XML via SFTP
              DonCollier

              I think I have this right and if i do the process involved in this is:

               

              Create an XML files as Wim says using an xslt so it is generated as the recipient requires and use SFTP to upload it ot the address provided by the recipient..  They will periodically sweep that point and upload all the xml dumped there into their system.

               

              The biggest problem have had with this type of arrangement is getting the right information back from the test uploads so i can get the XML absolutely correct.

               

               

              So you will need a plugin for the SFTP and an xslt to generate the XML  

              • 4. Re: Integration of XML via SFTP
                beverly

                Well yes, the XSLT can make the correct XML format for them. See if they can give you links to documentation of the schema they use. Sometimes there are restrictions (like validation of field lengths and types, for example).

                 

                Then SFTP (via plug-in). You may be able to retrieve the results that way as well.

                 

                You can use XSLT to parse the results as needed to push back into FileMaker as needed.

                beverly

                • 5. Re: Integration of XML via SFTP
                  monkeybreadsoftware

                  You can do SFTP upload with MBS Plugin.

                  As well as FTP, FTPS, HTTP and HTTPS.

                   

                  With MBS Plugin it is similar to post, just with different data and URL.

                  Maybe you try our examples?

                  • 6. Re: Integration of XML via SFTP
                    mchancevet@gmail.com

                    Hi,

                    Thanks very much everyone. I learn so much by asking Qs here.

                    Lets see if I've made any progress. I'm going to try and check my understanding.

                     

                    In terms of my 2 examples above (JSON in a Post, XML in a SFTP upload):

                     

                    1. There are syntax and format differences b/w JSON and XML
                    2. An XML file is text in the appropriate format developed according to the syntax of XML and saved as a ".xml"
                    3. Is JSON handled the same way (format and syntax aside) . Does it get saved as a ".json"?
                    4. If I have this right the XML file is being uploaded as just that, a file, which as Dan indicates will be stored until processing (swept) - No real-time 2 way dialogue.
                    5. The JSON post is not a 'File' which is transferred but rather the actual text 'string' of the body of the post, formatted and developed according to JSON syntax. So unless some process explicitly captures it, it is not stored in some dedicated location.
                    6. A Post is like a statement or question in a dialogue. These must be processed and addressed (responded to) in real time .

                     

                    Wimdecorte you were correct. I was referring to the FM solution as having two data integration points.

                    • Orders in as JSON posts
                    • Parcel data out as XML uploads

                     

                    Thanks again,

                    • 7. Re: Integration of XML via SFTP
                      wimdecorte

                      mchancevet@gmail.com wrote:

                       

                       

                      1. Is JSON handled the same way (format and syntax aside) . Does it get saved as a ".json"?

                       

                      Yes.

                       

                       

                      The JSON post is not a 'File' which is transferred but rather the actual text 'string' of the body of the post, formatted and developed according to JSON syntax. So unless some process explicitly captures it, it is not stored in some dedicated location.

                       

                      There is no functional difference in sending a json file or an xml file.  They are basically both just large text strings in a POST or both text files (with the proper extension) in a SFTP transaction.

                       

                       

                      A Post is like a statement or question in a dialogue. These must be processed and addressed (responded to) in real time .

                       

                      Not sure what you are expecting here.  An HTTP POST will get you a response but what it will be depends on the API that the web service you are talking to provides.  You have no control over that, you have to consume what they give you.

                      An SFTP "send" will also give you a response but is more limited typically in that it is strictly a 'file transport protocol'; it will tell you if the file was sent correctly.

                      • 8. Re: Integration of XML via SFTP
                        mchancevet@gmail.com

                        Ah ,

                        thanks for the clarification.

                        That helps a lot.

                        In terms of the MBS plugin - if sending from within FM using a Post or Upload function,

                        Where would  the response  to that go? How could it be accessed in FM?


                        An SFTP "send" will also give you a response but is more limited typically in that it is strictly a 'file transport protocol'; it will tell you if the file was sent correctly.

                        Would a web viewer capture the response if it was pointed at the URL referenced in the post or upload function?

                        Can you use plugins like MBS to set up the 2 way request-response process? i.e. generate a post or upload and then capture the response?

                         

                        An HTTP POST will get you a response but what it will be depends on the API that the web service you are talking to provides.

                        So presumably depending on their function some APIs may not respond at all?

                         

                         

                         

                        Apologies for naivety of questions.

                        • 9. Re: Integration of XML via SFTP
                          ch0c0halic

                          The basic functionality you will probably be using inside of the MBS is cURL. So all the operations available through cURL are available through the MBS.

                           

                          You can build an extremely robust query response system.

                          • 10. Re: Integration of XML via SFTP
                            wimdecorte

                             


                            An SFTP "send" will also give you a response but is more limited typically in that it is strictly a 'file transport protocol'; it will tell you if the file was sent correctly.

                            Would a web viewer capture the response if it was pointed at the URL referenced in the post or upload function?

                             

                            No.  Because an SFTP transaction is not meant to be a full-blown conversation.  You send a file and you will get an acknowledgement that the file has been received.  You will NOT get an updated file back if that is what you are expecting.  That is not what the SFTP protocol is about.

                             

                            But again, this should not be an abstract conversation: the company that you are sending a file to will probably have an API that will give you what you want.  We can't answer that for you; that is a conversation that you need to have with them: what do they give you to achieve what you want?

                            • 11. Re: Integration of XML via SFTP
                              wimdecorte

                               

                              An HTTP POST will get you a response but what it will be depends on the API that the web service you are talking to provides.

                              So presumably depending on their function some APIs may not respond at all?

                               

                               

                              All APIs that I have ever worked with have always sent a response.  But you never have a control over what is in the response.  It is what they offer.

                               

                              Depending on your negotiating power you may or may not have control over what is in the response.

                              • 12. Re: Integration of XML via SFTP
                                mchancevet@gmail.com

                                Hi,

                                thanks all for sticking with me.

                                I've been reviewing the MBS curl functions and I think I understand a little better now.

                                 

                                • I need to open up a curl session
                                • I send and receive using curl functions (MBS plugin functions)
                                • I close the session
                                • 13. Re: Integration of XML via SFTP
                                  mchancevet@gmail.com

                                  Hi Wimdecorte,

                                  But again, this should not be an abstract conversation: the company that you are sending a file to will probably have an API that will give you what you want.  We can't answer that for you; that is a conversation that you need to have with them: what do they give you to achieve what you want?

                                  Yes,

                                  This line of enquiry is an abstraction catalysed by a specific problem.

                                  My key objective is to get my head around the various transaction mechanisms and the structures required to support them.

                                  I've certainly learned a lot so far.

                                  I have a reference with plenty of info on the XML requirements. I'm happy I should be able to construct the XML OK.

                                  I don't need to extract any info from the response to the SFTP upload. I just wanted to know where such a response would go and how I would access it if I did need it.

                                  My real issue is getting my head around the processes of transactions and what FM needs to have, do and talk to to support this activity generally.

                                  MBS looks like the goer here.

                                  I wonder if FM 15 will have some of this built in?

                                   

                                  In terms of the web viewer - I expect that if the calculation for the web viewer referenced an appropriate set of curl functions (like MBS ones) it could be harnessed to some extent in terms of capturing some of the activity in a curl session?

                                   

                                  Thanks again,

                                  • 14. Re: Integration of XML via SFTP
                                    beverly

                                    If you use a plug-in, you should have a method to get a response (an place result in variable and/or field), as well as the actual upload of the file.

                                    I suggest you research SFTP and FileMaker Plug-in, there are several.

                                     

                                    The 'transmission' is via plug-in, unless you use Web Published FileMaker. The plug-in should be able to handle any authentication and any other needs.

                                     

                                    beverly

                                    1 2 Previous Next