Re: Scheduled jobs weird behavior

[Nicolas Malin-2]

Hi Giulio,

When we realized ftp transfer, we process as follow :
1. create a content linked to the OFBiz element. In you case we would be
created an OrderContent-Content with all OrderHeader present in the file.
2. Initialize a communicationEvent with the content, a type FTP_TRANSFER
and a contactMechIdTo with the destination ftp location
3. run a batch that transfer all ftp communication
With this process we haven't lost data and when a problem come it's
easily tracking.

Gil Portenseigne create an issue [1] with the code base improvement with
own customer site feedback

Nicolas

[1] https://issues.apache.org/jira/browse/OFBIZ-10245 File transfer
management with communicationEvent and new contactMech FTP_ADDRESS

On 02/04/2018 09:54, Rishi Solanki wrote:

> Hi Giulio,
>
> +1 for Michael suggestion, you should try to backup the file locally before
> transferring.
>
> Another item I could see in your code (thanks for sharing) that, it just
> connect and transfer the file to ftp location. I see many methods with
> FTPClient class. One is to set the buffer size.
> FTPClient.setBufferSize(int), which set the internal buffer size for
> buffered data streams.
>
> As you mentioned initially mostly you got the successful transfer, and
> problem occurs very few time. Please consider the size of the file to
> transfer and increase its buffer size should resolve your problem. Also
> there are many other methods which allows you to get/set the transfer
> status and mode etc should help.
>
> Reference:
> https://commons.apache.org/proper/commons-net/apidocs/org/apache/commons/net/ftp/FTPClient.html
>
> HTH!
>
>
>
> Rishi Solanki
> Sr Manager, Enterprise Software Development
> HotWax Systems Pvt. Ltd.
> Direct: [phone]
> http://www.hotwaxsystems.com
> www.hotwax.co
>
> On Fri, Mar 30, 2018 at 10:42 PM, Michael Brohl <[hidden email]>
> wrote:
>
>> Hi Giulio,
>>
>> It seems you directly store the generated file to the ftp server, correct?
>>
>> In such cases, we store the file to be transferred locally in an archive
>> directory and then transfer it to the ftp server.
>>
>> You'll see if the file was written and you have a backup in case there was
>> something wrong with the ftp transfer.
>>
>> Best regards,
>>
>> Michael
>>
>>
>> Am 30.03.18 um 16:16 schrieb Giulio Speri - MpStyle Srl:
>>
>> Hi Rishi,
>>> thanks for the reply!
>>>
>>> The api used for the ftp transfer are the apache FTP api, group
>>> "commons-net" (version 3.5);
>>> The code used to connect and store file via fftp is the following:
>>>
>>> ftp = new FTPClient()
>>> ftp.connect "xxx.yyyyyy.ww"
>>> ftp.enterLocalPassiveMode()
>>> ftp.login "username", "password"
>>> ftp.changeWorkingDirectory "INPUT_DIRECTORY"
>>> //put
>>> InputStream file_put= new FileInputStream(directoryZip);
>>> ftp.setFileType(FTP.BINARY_FILE_TYPE);
>>> ftp.storeFile(zipName, file_put);
>>> file_put.close()
>>> ftp.logout()
>>> ftp.disconnect()
>>>
>>> Shouldn't Groovy automatically raise exception in case of errors?
>>>
>>>
>>>
>>>
>>> 2018-03-30 15:07 GMT+02:00 Rishi Solanki <[hidden email]>:
>>>
>>> Giulio,
>>>> In addition to what Nicolas suggested please share or look into the
>>>> specific code which logs the files and export the orders at ftp locaion.
>>>> Probable cause for such error writer create the file at particular
>>>> location
>>>> but could not push the data/file due to some reason but send no
>>>> exception/failure.
>>>>
>>>> My guess problem should be related to pushing the exported data and logs
>>>> to
>>>> ftp location and is giving success but not pushing. Kind of case when we
>>>> tell the writer object to write and close. Sometime writer.flush() is
>>>> also
>>>> required.
>>>>
>>>> Also if possible at your end then share the relevant code and api you are
>>>> using, to better understand the problem.
>>>>
>>>> HTH!
>>>>
>>>>
>>>> Rishi Solanki
>>>> Sr Manager, Enterprise Software Development
>>>> HotWax Systems Pvt. Ltd.
>>>> Direct: [phone]
>>>> http://www.hotwaxsystems.com
>>>> www.hotwax.co
>>>>
>>>> On Fri, Mar 30, 2018 at 5:48 PM, Nicolas Malin <[hidden email]
>>>> wrote:
>>>>
>>>> Hello Giulio,
>>>>> On 30/03/2018 14:05, Giulio Speri - MpStyle Srl wrote:
>>>>>
>>>>> Hi all,
>>>>>> [...] Now, here is the issue; sometimes (runtime in which happen seems
>>>>>> random), I
>>>>>> notice that:
>>>>>> - the service in the Job List screen, has the state of "Finished" (no
>>>>>> errors);
>>>>>> - No log file nor zip file is written;
>>>>>> - the flag on the OrderHeader entity has been set to "Y" (by default is
>>>>>> null this field);
>>>>>> - no error or exception found in ofbiz/error log files;
>>>>>>
>>>>>> The Job Poller on the second instance of ofbiz is disabled
>>>>>> ("poll-enable=false" in serviceengine.xml), so that all the scheduled
>>>>>>
>>>>> jobs
>>>>> are run only on the first instance.
>>>>>> I am quite confused by this situation and I really don't know what and
>>>>>> where to look, to understand what happens.
>>>>>>
>>>>>> Does anyone ever experienced a similar issue?
>>>>>>
>>>>>> I worked several time with service engine and I interpreted your
>>>>> feedback
>>>>> as follow :
>>>>> * the service in the Job List screen, has the state of "Finished" -> Ok
>>>>> for OFBiz the service return a success
>>>>> * the flag on the OrderHeader entity has been set to "Y" (by default is
>>>>> null this field); -> If your service run under transaction, no reason
>>>>>
>>>> that
>>>>
>>>>> your service correctly
>>>>> * No log file nor zip file is written
>>>>>     ** I see different case, if you are sure that your code works
>>>>> properly
>>>>> check if you didn't manage some silent exception ( as
>>>>> try{...}catch(Exception){ }) and is it's the case review you exception
>>>>> management.
>>>>>     ** Increase your code with more log with good log4j configuration for
>>>>> track your service.
>>>>>     ** Check if you haven't external element that can be delete your file
>>>>>
>>>> by
>>>>
>>>>> inadvertence
>>>>>     ** a doubt, send an email with the generate file to your developer
>>>>> team
>>>>> in // to the storage.
>>>>>     ** no solution -> Use the remote debugger
>>>>>
>>>>> I hope you can find a solution to your problem
>>>>> Nicolas
>>>>>
>>>>> Thanks in advance for your help.
>>>>>
>>>>>> Giulio
>>>>>>
>>>>>>
>>>>>>
>>>
>>