Strange behavior with Entity Sync
Locked
Strange behavior with Entity Sync
 
|
We are using entity sync to keep POS terminals and a central server synchronized. We are going direct from POS to MCS, no Per Store Server involved. We have a pull sync (MCS to POS) and a push sync (POS to MCS.) Some data such as inventory is going both ways.
A few issues: 1) Records deleted on MCS are not deleted on POS. 2) Pull sync is not getting complete records in some cases. Two entities we see this consistenly is on Facility and InventoryItem. We get partial records sometimes. Note that this only seems to occur on the initial pull when we are first populating a new POS terminal from the central server. Once we identify a record, such as a facility, that didn't come down completely we can update the timestamp on the server and the next pull gets the complete record. 3) Related to item 2. If a push occurs from the POS terminal BEFORE we get the initial data correct, it overwrites the data on MCS. For example, if a facility record was pulled down and the store id is missing then when the terminal push occurs it trashes the facility record on the server. Strange thing is we aren't pushing facility data, only pulling. Note that the server time is different than the terminals (different timezones.) Vince Clark Global Era The Freedom of Open Source [hidden email] (303) 493-6723 |
Re: Strange behavior with Entity Sync
|
|
ignorant question
does not entity sync mess up the accounting. how are the ecas that are suppose to fire based on certain services, that do CRUD of the entity suppose to fire? am I missing some documentation on how this works? Vince M. Clark sent the following on 12/6/2007 1:58 PM: > We are using entity sync to keep POS terminals and a central server synchronized. We are going direct from POS to MCS, no Per Store Server involved. We have a pull sync (MCS to POS) and a push sync (POS to MCS.) Some data such as inventory is going both ways. > > A few issues: > 1) Records deleted on MCS are not deleted on POS. > 2) Pull sync is not getting complete records in some cases. Two entities we see this consistenly is on Facility and InventoryItem. We get partial records sometimes. Note that this only seems to occur on the initial pull when we are first populating a new POS terminal from the central server. Once we identify a record, such as a facility, that didn't come down completely we can update the timestamp on the server and the next pull gets the complete record. > 3) Related to item 2. If a push occurs from the POS terminal BEFORE we get the initial data correct, it overwrites the data on MCS. For example, if a facility record was pulled down and the store id is missing then when the terminal push occurs it trashes the facility record on the server. Strange thing is we aren't pushing facility data, only pulling. > > Note that the server time is different than the terminals (different timezones.) > > Vince Clark > Global Era > The Freedom of Open Source > [hidden email] > (303) 493-6723 > |
Re: Strange behavior with Entity Sync
|
|
BJ - your question is not ignorant, but I have no idea how to answer. We'll have to depend on others in the community much smarter than us.
Entity Sync is a feature available in the system. We are using it and I can only assume that it works if configured properly. I have figured out the basics of push and pull synchronization. That is about as much intelligence as I can claim. I can get into the details of what I loaded and where if necessary, such as EnityGroup, EnittySync, RecurrenceRule, etc. ----- Original Message ----- From: "BJ Freeman" <[hidden email]> To: [hidden email] Sent: Thursday, December 6, 2007 4:35:09 PM (GMT-0700) America/Denver Subject: Re: Strange behavior with Entity Sync ignorant question does not entity sync mess up the accounting. how are the ecas that are suppose to fire based on certain services, that do CRUD of the entity suppose to fire? am I missing some documentation on how this works? Vince M. Clark sent the following on 12/6/2007 1:58 PM: > We are using entity sync to keep POS terminals and a central server synchronized. We are going direct from POS to MCS, no Per Store Server involved. We have a pull sync (MCS to POS) and a push sync (POS to MCS.) Some data such as inventory is going both ways. > > A few issues: > 1) Records deleted on MCS are not deleted on POS. > 2) Pull sync is not getting complete records in some cases. Two entities we see this consistenly is on Facility and InventoryItem. We get partial records sometimes. Note that this only seems to occur on the initial pull when we are first populating a new POS terminal from the central server. Once we identify a record, such as a facility, that didn't come down completely we can update the timestamp on the server and the next pull gets the complete record. > 3) Related to item 2. If a push occurs from the POS terminal BEFORE we get the initial data correct, it overwrites the data on MCS. For example, if a facility record was pulled down and the store id is missing then when the terminal push occurs it trashes the facility record on the server. Strange thing is we aren't pushing facility data, only pulling. > > Note that the server time is different than the terminals (different timezones.) > > Vince Clark > Global Era > The Freedom of Open Source > [hidden email] > (303) 493-6723 > |
Re: Strange behavior with Entity Sync
|
|
I read
http://ofbizwiki.go-integral.com/Wiki.jsp?page=UsingEntitySynchronization http://www.opentaps.org/javadocs/version-1.0/framework/api/org/ofbiz/entityext/synchronization/EntitySyncServices.html http://docs.ofbiz.org/display/OFBTECH/Apache+OFBiz+Technical+Production+Setup+Guide /specialpurpose/pos/src/org.ofbiz.pos.rmi.adapter &event I like my method better, but it takes more programming. Vince M. Clark sent the following on 12/6/2007 3:33 PM: > BJ - your question is not ignorant, but I have no idea how to answer. We'll have to depend on others in the community much smarter than us. > > Entity Sync is a feature available in the system. We are using it and I can only assume that it works if configured properly. I have figured out the basics of push and pull synchronization. That is about as much intelligence as I can claim. > > I can get into the details of what I loaded and where if necessary, such as EnityGroup, EnittySync, RecurrenceRule, etc. > > ----- Original Message ----- > From: "BJ Freeman" <[hidden email]> > To: [hidden email] > Sent: Thursday, December 6, 2007 4:35:09 PM (GMT-0700) America/Denver > Subject: Re: Strange behavior with Entity Sync > > ignorant question > does not entity sync mess up the accounting. > how are the ecas that are suppose to fire based on certain services, > that do CRUD of the entity suppose to fire? > > am I missing some documentation on how this works? > > > Vince M. Clark sent the following on 12/6/2007 1:58 PM: >> We are using entity sync to keep POS terminals and a central server synchronized. We are going direct from POS to MCS, no Per Store Server involved. We have a pull sync (MCS to POS) and a push sync (POS to MCS.) Some data such as inventory is going both ways. >> >> A few issues: >> 1) Records deleted on MCS are not deleted on POS. >> 2) Pull sync is not getting complete records in some cases. Two entities we see this consistenly is on Facility and InventoryItem. We get partial records sometimes. Note that this only seems to occur on the initial pull when we are first populating a new POS terminal from the central server. Once we identify a record, such as a facility, that didn't come down completely we can update the timestamp on the server and the next pull gets the complete record. >> 3) Related to item 2. If a push occurs from the POS terminal BEFORE we get the initial data correct, it overwrites the data on MCS. For example, if a facility record was pulled down and the store id is missing then when the terminal push occurs it trashes the facility record on the server. Strange thing is we aren't pushing facility data, only pulling. >> >> Note that the server time is different than the terminals (different timezones.) >> >> Vince Clark >> Global Era >> The Freedom of Open Source >> [hidden email] >> (303) 493-6723 >> > |
Re: Strange behavior with Entity Sync
|
|
In reply to this post by Vince Clark
Just a stab in the dark but do these issues arise if the server and terminal
are in the same timezone? Regards Scott On 07/12/2007, Vince M. Clark <[hidden email]> wrote: > > BJ - your question is not ignorant, but I have no idea how to answer. > We'll have to depend on others in the community much smarter than us. > > Entity Sync is a feature available in the system. We are using it and I > can only assume that it works if configured properly. I have figured out the > basics of push and pull synchronization. That is about as much intelligence > as I can claim. > > I can get into the details of what I loaded and where if necessary, such > as EnityGroup, EnittySync, RecurrenceRule, etc. > > ----- Original Message ----- > From: "BJ Freeman" <[hidden email]> > To: [hidden email] > Sent: Thursday, December 6, 2007 4:35:09 PM (GMT-0700) America/Denver > Subject: Re: Strange behavior with Entity Sync > > ignorant question > does not entity sync mess up the accounting. > how are the ecas that are suppose to fire based on certain services, > that do CRUD of the entity suppose to fire? > > am I missing some documentation on how this works? > > > Vince M. Clark sent the following on 12/6/2007 1:58 PM: > > We are using entity sync to keep POS terminals and a central server > synchronized. We are going direct from POS to MCS, no Per Store Server > involved. We have a pull sync (MCS to POS) and a push sync (POS to MCS.) > Some data such as inventory is going both ways. > > > > A few issues: > > 1) Records deleted on MCS are not deleted on POS. > > 2) Pull sync is not getting complete records in some cases. Two entities > we see this consistenly is on Facility and InventoryItem. We get partial > records sometimes. Note that this only seems to occur on the initial pull > when we are first populating a new POS terminal from the central server. > Once we identify a record, such as a facility, that didn't come down > completely we can update the timestamp on the server and the next pull gets > the complete record. > > 3) Related to item 2. If a push occurs from the POS terminal BEFORE we > get the initial data correct, it overwrites the data on MCS. For example, if > a facility record was pulled down and the store id is missing then when the > terminal push occurs it trashes the facility record on the server. Strange > thing is we aren't pushing facility data, only pulling. > > > > Note that the server time is different than the terminals (different > timezones.) > > > > Vince Clark > > Global Era > > The Freedom of Open Source > > [hidden email] > > (303) 493-6723 > > > |
Re: Strange behavior with Entity Sync
|
|
It could be a timezone issue. We will test a bit more and try to nail down a pattern. We can isolate the problem to the same set of records that always get pulled incomplete.
The server is set to GMT and all the POS terminals are mountain. When first setting up sync I had some difficulty getting the PULL to start running and suspected it was due to the timezone being in the future on the server. But we got past that. Given that we will have terminals in all US timezones it is not really a good option to set them all to the same time as the server. ----- Original Message ----- From: "Scott Gray" <[hidden email]> To: [hidden email] Sent: Thursday, December 6, 2007 11:32:11 PM (GMT-0700) America/Denver Subject: Re: Strange behavior with Entity Sync Just a stab in the dark but do these issues arise if the server and terminal are in the same timezone? Regards Scott On 07/12/2007, Vince M. Clark <[hidden email]> wrote: > > BJ - your question is not ignorant, but I have no idea how to answer. > We'll have to depend on others in the community much smarter than us. > > Entity Sync is a feature available in the system. We are using it and I > can only assume that it works if configured properly. I have figured out the > basics of push and pull synchronization. That is about as much intelligence > as I can claim. > > I can get into the details of what I loaded and where if necessary, such > as EnityGroup, EnittySync, RecurrenceRule, etc. > > ----- Original Message ----- > From: "BJ Freeman" <[hidden email]> > To: [hidden email] > Sent: Thursday, December 6, 2007 4:35:09 PM (GMT-0700) America/Denver > Subject: Re: Strange behavior with Entity Sync > > ignorant question > does not entity sync mess up the accounting. > how are the ecas that are suppose to fire based on certain services, > that do CRUD of the entity suppose to fire? > > am I missing some documentation on how this works? > > > Vince M. Clark sent the following on 12/6/2007 1:58 PM: > > We are using entity sync to keep POS terminals and a central server > synchronized. We are going direct from POS to MCS, no Per Store Server > involved. We have a pull sync (MCS to POS) and a push sync (POS to MCS.) > Some data such as inventory is going both ways. > > > > A few issues: > > 1) Records deleted on MCS are not deleted on POS. > > 2) Pull sync is not getting complete records in some cases. Two entities > we see this consistenly is on Facility and InventoryItem. We get partial > records sometimes. Note that this only seems to occur on the initial pull > when we are first populating a new POS terminal from the central server. > Once we identify a record, such as a facility, that didn't come down > completely we can update the timestamp on the server and the next pull gets > the complete record. > > 3) Related to item 2. If a push occurs from the POS terminal BEFORE we > get the initial data correct, it overwrites the data on MCS. For example, if > a facility record was pulled down and the store id is missing then when the > terminal push occurs it trashes the facility record on the server. Strange > thing is we aren't pushing facility data, only pulling. > > > > Note that the server time is different than the terminals (different > timezones.) > > > > Vince Clark > > Global Era > > The Freedom of Open Source > > [hidden email] > > (303) 493-6723 > > > |
Re: Strange behavior with Entity Sync
|
|
Hi Vince,
are you running a recent OFBiz revision? I guess that Scott asked you about the timezones because we have recently refactored the way datetimes are treated in OFBiz: Adrian Crum worked on this, and he could better explain, but the idea is that when a date field is edited it is converted from the user session locale to the system locale. I think that Scott could be right... Adrian, do you think that your work could have caused some side effects on the entity synch stuff? Jacopo Vince M. Clark wrote: > It could be a timezone issue. We will test a bit more and try to nail down a pattern. We can isolate the problem to the same set of records that always get pulled incomplete. > > The server is set to GMT and all the POS terminals are mountain. When first setting up sync I had some difficulty getting the PULL to start running and suspected it was due to the timezone being in the future on the server. But we got past that. > > Given that we will have terminals in all US timezones it is not really a good option to set them all to the same time as the server. > > ----- Original Message ----- > From: "Scott Gray" <[hidden email]> > To: [hidden email] > Sent: Thursday, December 6, 2007 11:32:11 PM (GMT-0700) America/Denver > Subject: Re: Strange behavior with Entity Sync > > Just a stab in the dark but do these issues arise if the server and terminal > are in the same timezone? > > Regards > Scott > > On 07/12/2007, Vince M. Clark <[hidden email]> wrote: >> BJ - your question is not ignorant, but I have no idea how to answer. >> We'll have to depend on others in the community much smarter than us. >> >> Entity Sync is a feature available in the system. We are using it and I >> can only assume that it works if configured properly. I have figured out the >> basics of push and pull synchronization. That is about as much intelligence >> as I can claim. >> >> I can get into the details of what I loaded and where if necessary, such >> as EnityGroup, EnittySync, RecurrenceRule, etc. >> >> ----- Original Message ----- >> From: "BJ Freeman" <[hidden email]> >> To: [hidden email] >> Sent: Thursday, December 6, 2007 4:35:09 PM (GMT-0700) America/Denver >> Subject: Re: Strange behavior with Entity Sync >> >> ignorant question >> does not entity sync mess up the accounting. >> how are the ecas that are suppose to fire based on certain services, >> that do CRUD of the entity suppose to fire? >> >> am I missing some documentation on how this works? >> >> >> Vince M. Clark sent the following on 12/6/2007 1:58 PM: >>> We are using entity sync to keep POS terminals and a central server >> synchronized. We are going direct from POS to MCS, no Per Store Server >> involved. We have a pull sync (MCS to POS) and a push sync (POS to MCS.) >> Some data such as inventory is going both ways. >>> A few issues: >>> 1) Records deleted on MCS are not deleted on POS. >>> 2) Pull sync is not getting complete records in some cases. Two entities >> we see this consistenly is on Facility and InventoryItem. We get partial >> records sometimes. Note that this only seems to occur on the initial pull >> when we are first populating a new POS terminal from the central server. >> Once we identify a record, such as a facility, that didn't come down >> completely we can update the timestamp on the server and the next pull gets >> the complete record. >>> 3) Related to item 2. If a push occurs from the POS terminal BEFORE we >> get the initial data correct, it overwrites the data on MCS. For example, if >> a facility record was pulled down and the store id is missing then when the >> terminal push occurs it trashes the facility record on the server. Strange >> thing is we aren't pushing facility data, only pulling. >>> Note that the server time is different than the terminals (different >> timezones.) >>> Vince Clark >>> Global Era >>> The Freedom of Open Source >>> [hidden email] >>> (303) 493-6723 >>> > |
Re: Strange behavior with Entity Sync
|
|
Yes. I pulled it from trunk about a week ago.
----- Original Message ----- From: "Jacopo Cappellato" <[hidden email]> To: [hidden email] Sent: Friday, December 7, 2007 8:36:18 AM (GMT-0700) America/Denver Subject: Re: Strange behavior with Entity Sync Hi Vince, are you running a recent OFBiz revision? I guess that Scott asked you about the timezones because we have recently refactored the way datetimes are treated in OFBiz: Adrian Crum worked on this, and he could better explain, but the idea is that when a date field is edited it is converted from the user session locale to the system locale. I think that Scott could be right... Adrian, do you think that your work could have caused some side effects on the entity synch stuff? Jacopo Vince M. Clark wrote: > It could be a timezone issue. We will test a bit more and try to nail down a pattern. We can isolate the problem to the same set of records that always get pulled incomplete. > > The server is set to GMT and all the POS terminals are mountain. When first setting up sync I had some difficulty getting the PULL to start running and suspected it was due to the timezone being in the future on the server. But we got past that. > > Given that we will have terminals in all US timezones it is not really a good option to set them all to the same time as the server. > > ----- Original Message ----- > From: "Scott Gray" <[hidden email]> > To: [hidden email] > Sent: Thursday, December 6, 2007 11:32:11 PM (GMT-0700) America/Denver > Subject: Re: Strange behavior with Entity Sync > > Just a stab in the dark but do these issues arise if the server and terminal > are in the same timezone? > > Regards > Scott > > On 07/12/2007, Vince M. Clark <[hidden email]> wrote: >> BJ - your question is not ignorant, but I have no idea how to answer. >> We'll have to depend on others in the community much smarter than us. >> >> Entity Sync is a feature available in the system. We are using it and I >> can only assume that it works if configured properly. I have figured out the >> basics of push and pull synchronization. That is about as much intelligence >> as I can claim. >> >> I can get into the details of what I loaded and where if necessary, such >> as EnityGroup, EnittySync, RecurrenceRule, etc. >> >> ----- Original Message ----- >> From: "BJ Freeman" <[hidden email]> >> To: [hidden email] >> Sent: Thursday, December 6, 2007 4:35:09 PM (GMT-0700) America/Denver >> Subject: Re: Strange behavior with Entity Sync >> >> ignorant question >> does not entity sync mess up the accounting. >> how are the ecas that are suppose to fire based on certain services, >> that do CRUD of the entity suppose to fire? >> >> am I missing some documentation on how this works? >> >> >> Vince M. Clark sent the following on 12/6/2007 1:58 PM: >>> We are using entity sync to keep POS terminals and a central server >> synchronized. We are going direct from POS to MCS, no Per Store Server >> involved. We have a pull sync (MCS to POS) and a push sync (POS to MCS.) >> Some data such as inventory is going both ways. >>> A few issues: >>> 1) Records deleted on MCS are not deleted on POS. >>> 2) Pull sync is not getting complete records in some cases. Two entities >> we see this consistenly is on Facility and InventoryItem. We get partial >> records sometimes. Note that this only seems to occur on the initial pull >> when we are first populating a new POS terminal from the central server. >> Once we identify a record, such as a facility, that didn't come down >> completely we can update the timestamp on the server and the next pull gets >> the complete record. >>> 3) Related to item 2. If a push occurs from the POS terminal BEFORE we >> get the initial data correct, it overwrites the data on MCS. For example, if >> a facility record was pulled down and the store id is missing then when the >> terminal push occurs it trashes the facility record on the server. Strange >> thing is we aren't pushing facility data, only pulling. >>> Note that the server time is different than the terminals (different >> timezones.) >>> Vince Clark >>> Global Era >>> The Freedom of Open Source >>> [hidden email] >>> (303) 493-6723 >>> > |
Re: Strange behavior with Entity Sync
|
|
I don't know if the recent time zone work would have any affect on this. It's a possibility.
I'm not familiar with POS and entity sync, so I'll describe the changes I made and maybe others who are more familiar with Vince's setup can figure it out. The basic concept is: 1. Date/time String in UI (user's time zone) gets converted to TimeStamp (which is referenced to GMT), then it is stored. 2. TimeStamp in storage (GMT) gets converted to date/time String for UI (user's time zone). 3. Only the screen widget and Freemarker template code was changed. You shouldn't have to set all of your terminals to the same time zone as the server. If the problem is in OFBiz, we should fix it. -Adrian Vince M. Clark wrote: > Yes. I pulled it from trunk about a week ago. > > ----- Original Message ----- > From: "Jacopo Cappellato" <[hidden email]> > To: [hidden email] > Sent: Friday, December 7, 2007 8:36:18 AM (GMT-0700) America/Denver > Subject: Re: Strange behavior with Entity Sync > > Hi Vince, > > are you running a recent OFBiz revision? > I guess that Scott asked you about the timezones because we have > recently refactored the way datetimes are treated in OFBiz: Adrian Crum > worked on this, and he could better explain, but the idea is that when a > date field is edited it is converted from the user session locale to the > system locale. > I think that Scott could be right... Adrian, do you think that your work > could have caused some side effects on the entity synch stuff? > > Jacopo > > > > Vince M. Clark wrote: > >>It could be a timezone issue. We will test a bit more and try to nail down a pattern. We can isolate the problem to the same set of records that always get pulled incomplete. >> >>The server is set to GMT and all the POS terminals are mountain. When first setting up sync I had some difficulty getting the PULL to start running and suspected it was due to the timezone being in the future on the server. But we got past that. >> >>Given that we will have terminals in all US timezones it is not really a good option to set them all to the same time as the server. >> >>----- Original Message ----- >>From: "Scott Gray" <[hidden email]> >>To: [hidden email] >>Sent: Thursday, December 6, 2007 11:32:11 PM (GMT-0700) America/Denver >>Subject: Re: Strange behavior with Entity Sync >> >>Just a stab in the dark but do these issues arise if the server and terminal >>are in the same timezone? >> >>Regards >>Scott >> >>On 07/12/2007, Vince M. Clark <[hidden email]> wrote: >> >>>BJ - your question is not ignorant, but I have no idea how to answer. >>>We'll have to depend on others in the community much smarter than us. >>> >>>Entity Sync is a feature available in the system. We are using it and I >>>can only assume that it works if configured properly. I have figured out the >>>basics of push and pull synchronization. That is about as much intelligence >>>as I can claim. >>> >>>I can get into the details of what I loaded and where if necessary, such >>>as EnityGroup, EnittySync, RecurrenceRule, etc. >>> >>>----- Original Message ----- >>>From: "BJ Freeman" <[hidden email]> >>>To: [hidden email] >>>Sent: Thursday, December 6, 2007 4:35:09 PM (GMT-0700) America/Denver >>>Subject: Re: Strange behavior with Entity Sync >>> >>>ignorant question >>>does not entity sync mess up the accounting. >>>how are the ecas that are suppose to fire based on certain services, >>>that do CRUD of the entity suppose to fire? >>> >>>am I missing some documentation on how this works? >>> >>> >>>Vince M. Clark sent the following on 12/6/2007 1:58 PM: >>> >>>>We are using entity sync to keep POS terminals and a central server >>> >>>synchronized. We are going direct from POS to MCS, no Per Store Server >>>involved. We have a pull sync (MCS to POS) and a push sync (POS to MCS.) >>>Some data such as inventory is going both ways. >>> >>>>A few issues: >>>>1) Records deleted on MCS are not deleted on POS. >>>>2) Pull sync is not getting complete records in some cases. Two entities >>> >>>we see this consistenly is on Facility and InventoryItem. We get partial >>>records sometimes. Note that this only seems to occur on the initial pull >>>when we are first populating a new POS terminal from the central server. >>>Once we identify a record, such as a facility, that didn't come down >>>completely we can update the timestamp on the server and the next pull gets >>>the complete record. >>> >>>>3) Related to item 2. If a push occurs from the POS terminal BEFORE we >>> >>>get the initial data correct, it overwrites the data on MCS. For example, if >>>a facility record was pulled down and the store id is missing then when the >>>terminal push occurs it trashes the facility record on the server. Strange >>>thing is we aren't pushing facility data, only pulling. >>> >>>>Note that the server time is different than the terminals (different >>> >>>timezones.) >>> >>>>Vince Clark >>>>Global Era >>>>The Freedom of Open Source >>>>[hidden email] >>>>(303) 493-6723 >>>> >> > > |
Re: Strange behavior with Entity Sync
|
|
I don't know for sure that we are dealing with a timezone issue. That is just one theory as we try to iron out some problems with entity sync. The fact that the same records always seem to be problematic gives us a pattern to pursue. I'll update the thread as we research the problem.
----- Original Message ----- From: "Adrian Crum" <[hidden email]> To: [hidden email] Sent: Friday, December 7, 2007 10:33:19 AM (GMT-0700) America/Denver Subject: Re: Strange behavior with Entity Sync I don't know if the recent time zone work would have any affect on this. It's a possibility. I'm not familiar with POS and entity sync, so I'll describe the changes I made and maybe others who are more familiar with Vince's setup can figure it out. The basic concept is: 1. Date/time String in UI (user's time zone) gets converted to TimeStamp (which is referenced to GMT), then it is stored. 2. TimeStamp in storage (GMT) gets converted to date/time String for UI (user's time zone). 3. Only the screen widget and Freemarker template code was changed. You shouldn't have to set all of your terminals to the same time zone as the server. If the problem is in OFBiz, we should fix it. -Adrian Vince M. Clark wrote: > Yes. I pulled it from trunk about a week ago. > > ----- Original Message ----- > From: "Jacopo Cappellato" <[hidden email]> > To: [hidden email] > Sent: Friday, December 7, 2007 8:36:18 AM (GMT-0700) America/Denver > Subject: Re: Strange behavior with Entity Sync > > Hi Vince, > > are you running a recent OFBiz revision? > I guess that Scott asked you about the timezones because we have > recently refactored the way datetimes are treated in OFBiz: Adrian Crum > worked on this, and he could better explain, but the idea is that when a > date field is edited it is converted from the user session locale to the > system locale. > I think that Scott could be right... Adrian, do you think that your work > could have caused some side effects on the entity synch stuff? > > Jacopo > > > > Vince M. Clark wrote: > >>It could be a timezone issue. We will test a bit more and try to nail down a pattern. We can isolate the problem to the same set of records that always get pulled incomplete. >> >>The server is set to GMT and all the POS terminals are mountain. When first setting up sync I had some difficulty getting the PULL to start running and suspected it was due to the timezone being in the future on the server. But we got past that. >> >>Given that we will have terminals in all US timezones it is not really a good option to set them all to the same time as the server. >> >>----- Original Message ----- >>From: "Scott Gray" <[hidden email]> >>To: [hidden email] >>Sent: Thursday, December 6, 2007 11:32:11 PM (GMT-0700) America/Denver >>Subject: Re: Strange behavior with Entity Sync >> >>Just a stab in the dark but do these issues arise if the server and terminal >>are in the same timezone? >> >>Regards >>Scott >> >>On 07/12/2007, Vince M. Clark <[hidden email]> wrote: >> >>>BJ - your question is not ignorant, but I have no idea how to answer. >>>We'll have to depend on others in the community much smarter than us. >>> >>>Entity Sync is a feature available in the system. We are using it and I >>>can only assume that it works if configured properly. I have figured out the >>>basics of push and pull synchronization. That is about as much intelligence >>>as I can claim. >>> >>>I can get into the details of what I loaded and where if necessary, such >>>as EnityGroup, EnittySync, RecurrenceRule, etc. >>> >>>----- Original Message ----- >>>From: "BJ Freeman" <[hidden email]> >>>To: [hidden email] >>>Sent: Thursday, December 6, 2007 4:35:09 PM (GMT-0700) America/Denver >>>Subject: Re: Strange behavior with Entity Sync >>> >>>ignorant question >>>does not entity sync mess up the accounting. >>>how are the ecas that are suppose to fire based on certain services, >>>that do CRUD of the entity suppose to fire? >>> >>>am I missing some documentation on how this works? >>> >>> >>>Vince M. Clark sent the following on 12/6/2007 1:58 PM: >>> >>>>We are using entity sync to keep POS terminals and a central server >>> >>>synchronized. We are going direct from POS to MCS, no Per Store Server >>>involved. We have a pull sync (MCS to POS) and a push sync (POS to MCS.) >>>Some data such as inventory is going both ways. >>> >>>>A few issues: >>>>1) Records deleted on MCS are not deleted on POS. >>>>2) Pull sync is not getting complete records in some cases. Two entities >>> >>>we see this consistenly is on Facility and InventoryItem. We get partial >>>records sometimes. Note that this only seems to occur on the initial pull >>>when we are first populating a new POS terminal from the central server. >>>Once we identify a record, such as a facility, that didn't come down >>>completely we can update the timestamp on the server and the next pull gets >>>the complete record. >>> >>>>3) Related to item 2. If a push occurs from the POS terminal BEFORE we >>> >>>get the initial data correct, it overwrites the data on MCS. For example, if >>>a facility record was pulled down and the store id is missing then when the >>>terminal push occurs it trashes the facility record on the server. Strange >>>thing is we aren't pushing facility data, only pulling. >>> >>>>Note that the server time is different than the terminals (different >>> >>>timezones.) >>> >>>>Vince Clark >>>>Global Era >>>>The Freedom of Open Source >>>>[hidden email] >>>>(303) 493-6723 >>>> >> > > |
Re: Strange behavior with Entity Sync
|
|
Yeah, it doesn't sound like it's a time zone problem. As long at the timestamps in the databases are all GMT then it shouldn't be a bit deal. There could be other things going on though, like the last updated stamp for a certain record on the target server could be more recent than the one on the source server (if seed/initial data was loaded on the source and then on the target, even if the source was updated before the target was loaded) then it would not write the record. Of course, if that happens it will tell you that it did that and you'll at least see it in the summary numbers that come back from the remote call. It sounds like you have things far enough along that the only way to progress is with elbow grease. In other words pick a couple of records that you think should be transfered over but that are not getting copied, and then look at the data (on both source and target if applicable) and read through the code to see exactly what it would do with that data. -David On Dec 7, 2007, at 10:27 AM, Vince M. Clark wrote: > I don't know for sure that we are dealing with a timezone issue. > That is just one theory as we try to iron out some problems with > entity sync. The fact that the same records always seem to be > problematic gives us a pattern to pursue. I'll update the thread as > we research the problem. > > ----- Original Message ----- > From: "Adrian Crum" <[hidden email]> > To: [hidden email] > Sent: Friday, December 7, 2007 10:33:19 AM (GMT-0700) America/Denver > Subject: Re: Strange behavior with Entity Sync > > I don't know if the recent time zone work would have any affect on > this. It's a possibility. > > I'm not familiar with POS and entity sync, so I'll describe the > changes I made and maybe others who > are more familiar with Vince's setup can figure it out. > > The basic concept is: > > 1. Date/time String in UI (user's time zone) gets converted to > TimeStamp (which is referenced to > GMT), then it is stored. > 2. TimeStamp in storage (GMT) gets converted to date/time String for > UI (user's time zone). > 3. Only the screen widget and Freemarker template code was changed. > > You shouldn't have to set all of your terminals to the same time > zone as the server. If the problem > is in OFBiz, we should fix it. > > -Adrian > > > Vince M. Clark wrote: > >> Yes. I pulled it from trunk about a week ago. >> >> ----- Original Message ----- >> From: "Jacopo Cappellato" <[hidden email]> >> To: [hidden email] >> Sent: Friday, December 7, 2007 8:36:18 AM (GMT-0700) America/Denver >> Subject: Re: Strange behavior with Entity Sync >> >> Hi Vince, >> >> are you running a recent OFBiz revision? >> I guess that Scott asked you about the timezones because we have >> recently refactored the way datetimes are treated in OFBiz: Adrian >> Crum >> worked on this, and he could better explain, but the idea is that >> when a >> date field is edited it is converted from the user session locale >> to the >> system locale. >> I think that Scott could be right... Adrian, do you think that your >> work >> could have caused some side effects on the entity synch stuff? >> >> Jacopo >> >> >> >> Vince M. Clark wrote: >> >>> It could be a timezone issue. We will test a bit more and try to >>> nail down a pattern. We can isolate the problem to the same set of >>> records that always get pulled incomplete. >>> >>> The server is set to GMT and all the POS terminals are mountain. >>> When first setting up sync I had some difficulty getting the PULL >>> to start running and suspected it was due to the timezone being in >>> the future on the server. But we got past that. >>> >>> Given that we will have terminals in all US timezones it is not >>> really a good option to set them all to the same time as the server. >>> >>> ----- Original Message ----- >>> From: "Scott Gray" <[hidden email]> >>> To: [hidden email] >>> Sent: Thursday, December 6, 2007 11:32:11 PM (GMT-0700) America/ >>> Denver >>> Subject: Re: Strange behavior with Entity Sync >>> >>> Just a stab in the dark but do these issues arise if the server >>> and terminal >>> are in the same timezone? >>> >>> Regards >>> Scott >>> >>> On 07/12/2007, Vince M. Clark <[hidden email]> wrote: >>> >>>> BJ - your question is not ignorant, but I have no idea how to >>>> answer. >>>> We'll have to depend on others in the community much smarter than >>>> us. >>>> >>>> Entity Sync is a feature available in the system. We are using it >>>> and I >>>> can only assume that it works if configured properly. I have >>>> figured out the >>>> basics of push and pull synchronization. That is about as much >>>> intelligence >>>> as I can claim. >>>> >>>> I can get into the details of what I loaded and where if >>>> necessary, such >>>> as EnityGroup, EnittySync, RecurrenceRule, etc. >>>> >>>> ----- Original Message ----- >>>> From: "BJ Freeman" <[hidden email]> >>>> To: [hidden email] >>>> Sent: Thursday, December 6, 2007 4:35:09 PM (GMT-0700) America/ >>>> Denver >>>> Subject: Re: Strange behavior with Entity Sync >>>> >>>> ignorant question >>>> does not entity sync mess up the accounting. >>>> how are the ecas that are suppose to fire based on certain >>>> services, >>>> that do CRUD of the entity suppose to fire? >>>> >>>> am I missing some documentation on how this works? >>>> >>>> >>>> Vince M. Clark sent the following on 12/6/2007 1:58 PM: >>>> >>>>> We are using entity sync to keep POS terminals and a central >>>>> server >>>> >>>> synchronized. We are going direct from POS to MCS, no Per Store >>>> Server >>>> involved. We have a pull sync (MCS to POS) and a push sync (POS >>>> to MCS.) >>>> Some data such as inventory is going both ways. >>>> >>>>> A few issues: >>>>> 1) Records deleted on MCS are not deleted on POS. >>>>> 2) Pull sync is not getting complete records in some cases. Two >>>>> entities >>>> >>>> we see this consistenly is on Facility and InventoryItem. We get >>>> partial >>>> records sometimes. Note that this only seems to occur on the >>>> initial pull >>>> when we are first populating a new POS terminal from the central >>>> server. >>>> Once we identify a record, such as a facility, that didn't come >>>> down >>>> completely we can update the timestamp on the server and the next >>>> pull gets >>>> the complete record. >>>> >>>>> 3) Related to item 2. If a push occurs from the POS terminal >>>>> BEFORE we >>>> >>>> get the initial data correct, it overwrites the data on MCS. For >>>> example, if >>>> a facility record was pulled down and the store id is missing >>>> then when the >>>> terminal push occurs it trashes the facility record on the >>>> server. Strange >>>> thing is we aren't pushing facility data, only pulling. >>>> >>>>> Note that the server time is different than the terminals >>>>> (different >>>> >>>> timezones.) >>>> >>>>> Vince Clark >>>>> Global Era >>>>> The Freedom of Open Source >>>>> [hidden email] >>>>> (303) 493-6723 >>>>> >>> >> >> > |
smime.p7s (3K)
Re: Strange behavior with Entity Sync
|
|
Clarification. The timestamps on the central server are GMT. POS terminals are all in their local timezones. So we do get into a bit of trickiness around timestamps I think.
Here is the summary of what I'm doing: To setup the pull I create an entity sync record on the source (MCS) I load the recurrence rule and job on the target (POS) The pull launches, and sucks everything down from the server based on the EntityGroup defined on the MCS. During this process it appears as though the records are written to POS with the server timestamp. The sync process does not seem to change the timestamp to the local time on the POS terminal. This seems to be OK for data that is only pulled, because the next job that runs a pull will depend on logic running on MCS to determine what records to send down to POS. But then when we trigger the push, some entities are synched both ways, such as inventory. We did have one situation where inventory was overwritten on MCS with bad data. My theory is that the POS terminal pulled that data from the server initially with the server timestamp. Then on the push, since the server timestamp was in the future, the push process picked up those records as having been changed. This might explain why our initiial push takes forever. The pos terminals may be sending everything back to the server. I'll try setting all machines to the same timezone and see if it stabilizes the process. ----- Original Message ----- From: "David E Jones" <[hidden email]> To: [hidden email] Sent: Friday, December 7, 2007 10:57:34 AM (GMT-0700) America/Denver Subject: Re: Strange behavior with Entity Sync Yeah, it doesn't sound like it's a time zone problem. As long at the timestamps in the databases are all GMT then it shouldn't be a bit deal. There could be other things going on though, like the last updated stamp for a certain record on the target server could be more recent than the one on the source server (if seed/initial data was loaded on the source and then on the target, even if the source was updated before the target was loaded) then it would not write the record. Of course, if that happens it will tell you that it did that and you'll at least see it in the summary numbers that come back from the remote call. It sounds like you have things far enough along that the only way to progress is with elbow grease. In other words pick a couple of records that you think should be transfered over but that are not getting copied, and then look at the data (on both source and target if applicable) and read through the code to see exactly what it would do with that data. -David On Dec 7, 2007, at 10:27 AM, Vince M. Clark wrote: > I don't know for sure that we are dealing with a timezone issue. > That is just one theory as we try to iron out some problems with > entity sync. The fact that the same records always seem to be > problematic gives us a pattern to pursue. I'll update the thread as > we research the problem. > > ----- Original Message ----- > From: "Adrian Crum" <[hidden email]> > To: [hidden email] > Sent: Friday, December 7, 2007 10:33:19 AM (GMT-0700) America/Denver > Subject: Re: Strange behavior with Entity Sync > > I don't know if the recent time zone work would have any affect on > this. It's a possibility. > > I'm not familiar with POS and entity sync, so I'll describe the > changes I made and maybe others who > are more familiar with Vince's setup can figure it out. > > The basic concept is: > > 1. Date/time String in UI (user's time zone) gets converted to > TimeStamp (which is referenced to > GMT), then it is stored. > 2. TimeStamp in storage (GMT) gets converted to date/time String for > UI (user's time zone). > 3. Only the screen widget and Freemarker template code was changed. > > You shouldn't have to set all of your terminals to the same time > zone as the server. If the problem > is in OFBiz, we should fix it. > > -Adrian > > > Vince M. Clark wrote: > >> Yes. I pulled it from trunk about a week ago. >> >> ----- Original Message ----- >> From: "Jacopo Cappellato" <[hidden email]> >> To: [hidden email] >> Sent: Friday, December 7, 2007 8:36:18 AM (GMT-0700) America/Denver >> Subject: Re: Strange behavior with Entity Sync >> >> Hi Vince, >> >> are you running a recent OFBiz revision? >> I guess that Scott asked you about the timezones because we have >> recently refactored the way datetimes are treated in OFBiz: Adrian >> Crum >> worked on this, and he could better explain, but the idea is that >> when a >> date field is edited it is converted from the user session locale >> to the >> system locale. >> I think that Scott could be right... Adrian, do you think that your >> work >> could have caused some side effects on the entity synch stuff? >> >> Jacopo >> >> >> >> Vince M. Clark wrote: >> >>> It could be a timezone issue. We will test a bit more and try to >>> nail down a pattern. We can isolate the problem to the same set of >>> records that always get pulled incomplete. >>> >>> The server is set to GMT and all the POS terminals are mountain. >>> When first setting up sync I had some difficulty getting the PULL >>> to start running and suspected it was due to the timezone being in >>> the future on the server. But we got past that. >>> >>> Given that we will have terminals in all US timezones it is not >>> really a good option to set them all to the same time as the server. >>> >>> ----- Original Message ----- >>> From: "Scott Gray" <[hidden email]> >>> To: [hidden email] >>> Sent: Thursday, December 6, 2007 11:32:11 PM (GMT-0700) America/ >>> Denver >>> Subject: Re: Strange behavior with Entity Sync >>> >>> Just a stab in the dark but do these issues arise if the server >>> and terminal >>> are in the same timezone? >>> >>> Regards >>> Scott >>> >>> On 07/12/2007, Vince M. Clark <[hidden email]> wrote: >>> >>>> BJ - your question is not ignorant, but I have no idea how to >>>> answer. >>>> We'll have to depend on others in the community much smarter than >>>> us. >>>> >>>> Entity Sync is a feature available in the system. We are using it >>>> and I >>>> can only assume that it works if configured properly. I have >>>> figured out the >>>> basics of push and pull synchronization. That is about as much >>>> intelligence >>>> as I can claim. >>>> >>>> I can get into the details of what I loaded and where if >>>> necessary, such >>>> as EnityGroup, EnittySync, RecurrenceRule, etc. >>>> >>>> ----- Original Message ----- >>>> From: "BJ Freeman" <[hidden email]> >>>> To: [hidden email] >>>> Sent: Thursday, December 6, 2007 4:35:09 PM (GMT-0700) America/ >>>> Denver >>>> Subject: Re: Strange behavior with Entity Sync >>>> >>>> ignorant question >>>> does not entity sync mess up the accounting. >>>> how are the ecas that are suppose to fire based on certain >>>> services, >>>> that do CRUD of the entity suppose to fire? >>>> >>>> am I missing some documentation on how this works? >>>> >>>> >>>> Vince M. Clark sent the following on 12/6/2007 1:58 PM: >>>> >>>>> We are using entity sync to keep POS terminals and a central >>>>> server >>>> >>>> synchronized. We are going direct from POS to MCS, no Per Store >>>> Server >>>> involved. We have a pull sync (MCS to POS) and a push sync (POS >>>> to MCS.) >>>> Some data such as inventory is going both ways. >>>> >>>>> A few issues: >>>>> 1) Records deleted on MCS are not deleted on POS. >>>>> 2) Pull sync is not getting complete records in some cases. Two >>>>> entities >>>> >>>> we see this consistenly is on Facility and InventoryItem. We get >>>> partial >>>> records sometimes. Note that this only seems to occur on the >>>> initial pull >>>> when we are first populating a new POS terminal from the central >>>> server. >>>> Once we identify a record, such as a facility, that didn't come >>>> down >>>> completely we can update the timestamp on the server and the next >>>> pull gets >>>> the complete record. >>>> >>>>> 3) Related to item 2. If a push occurs from the POS terminal >>>>> BEFORE we >>>> >>>> get the initial data correct, it overwrites the data on MCS. For >>>> example, if >>>> a facility record was pulled down and the store id is missing >>>> then when the >>>> terminal push occurs it trashes the facility record on the >>>> server. Strange >>>> thing is we aren't pushing facility data, only pulling. >>>> >>>>> Note that the server time is different than the terminals >>>>> (different >>>> >>>> timezones.) >>>> >>>>> Vince Clark >>>>> Global Era >>>>> The Freedom of Open Source >>>>> [hidden email] >>>>> (303) 493-6723 >>>>> >>> >> >> > |
Re: Strange behavior with Entity Sync
|
|
If in fact timestamps are causing problems it seems to me that a simple fix would be to write the records to the target with the current time on the target rather than inheriting timestamps from the source.
----- Original Message ----- From: "Vince M. Clark" <[hidden email]> To: [hidden email] Sent: Friday, December 7, 2007 11:13:16 AM (GMT-0700) America/Denver Subject: Re: Strange behavior with Entity Sync Clarification. The timestamps on the central server are GMT. POS terminals are all in their local timezones. So we do get into a bit of trickiness around timestamps I think. Here is the summary of what I'm doing: To setup the pull I create an entity sync record on the source (MCS) I load the recurrence rule and job on the target (POS) The pull launches, and sucks everything down from the server based on the EntityGroup defined on the MCS. During this process it appears as though the records are written to POS with the server timestamp. The sync process does not seem to change the timestamp to the local time on the POS terminal. This seems to be OK for data that is only pulled, because the next job that runs a pull will depend on logic running on MCS to determine what records to send down to POS. But then when we trigger the push, some entities are synched both ways, such as inventory. We did have one situation where inventory was overwritten on MCS with bad data. My theory is that the POS terminal pulled that data from the server initially with the server timestamp. Then on the push, since the server timestamp was in the future, the push process picked up those records as having been changed. This might explain why our initiial push takes forever. The pos terminals may be sending everything back to the server. I'll try setting all machines to the same timezone and see if it stabilizes the process. ----- Original Message ----- From: "David E Jones" <[hidden email]> To: [hidden email] Sent: Friday, December 7, 2007 10:57:34 AM (GMT-0700) America/Denver Subject: Re: Strange behavior with Entity Sync Yeah, it doesn't sound like it's a time zone problem. As long at the timestamps in the databases are all GMT then it shouldn't be a bit deal. There could be other things going on though, like the last updated stamp for a certain record on the target server could be more recent than the one on the source server (if seed/initial data was loaded on the source and then on the target, even if the source was updated before the target was loaded) then it would not write the record. Of course, if that happens it will tell you that it did that and you'll at least see it in the summary numbers that come back from the remote call. It sounds like you have things far enough along that the only way to progress is with elbow grease. In other words pick a couple of records that you think should be transfered over but that are not getting copied, and then look at the data (on both source and target if applicable) and read through the code to see exactly what it would do with that data. -David On Dec 7, 2007, at 10:27 AM, Vince M. Clark wrote: > I don't know for sure that we are dealing with a timezone issue. > That is just one theory as we try to iron out some problems with > entity sync. The fact that the same records always seem to be > problematic gives us a pattern to pursue. I'll update the thread as > we research the problem. > > ----- Original Message ----- > From: "Adrian Crum" <[hidden email]> > To: [hidden email] > Sent: Friday, December 7, 2007 10:33:19 AM (GMT-0700) America/Denver > Subject: Re: Strange behavior with Entity Sync > > I don't know if the recent time zone work would have any affect on > this. It's a possibility. > > I'm not familiar with POS and entity sync, so I'll describe the > changes I made and maybe others who > are more familiar with Vince's setup can figure it out. > > The basic concept is: > > 1. Date/time String in UI (user's time zone) gets converted to > TimeStamp (which is referenced to > GMT), then it is stored. > 2. TimeStamp in storage (GMT) gets converted to date/time String for > UI (user's time zone). > 3. Only the screen widget and Freemarker template code was changed. > > You shouldn't have to set all of your terminals to the same time > zone as the server. If the problem > is in OFBiz, we should fix it. > > -Adrian > > > Vince M. Clark wrote: > >> Yes. I pulled it from trunk about a week ago. >> >> ----- Original Message ----- >> From: "Jacopo Cappellato" <[hidden email]> >> To: [hidden email] >> Sent: Friday, December 7, 2007 8:36:18 AM (GMT-0700) America/Denver >> Subject: Re: Strange behavior with Entity Sync >> >> Hi Vince, >> >> are you running a recent OFBiz revision? >> I guess that Scott asked you about the timezones because we have >> recently refactored the way datetimes are treated in OFBiz: Adrian >> Crum >> worked on this, and he could better explain, but the idea is that >> when a >> date field is edited it is converted from the user session locale >> to the >> system locale. >> I think that Scott could be right... Adrian, do you think that your >> work >> could have caused some side effects on the entity synch stuff? >> >> Jacopo >> >> >> >> Vince M. Clark wrote: >> >>> It could be a timezone issue. We will test a bit more and try to >>> nail down a pattern. We can isolate the problem to the same set of >>> records that always get pulled incomplete. >>> >>> The server is set to GMT and all the POS terminals are mountain. >>> When first setting up sync I had some difficulty getting the PULL >>> to start running and suspected it was due to the timezone being in >>> the future on the server. But we got past that. >>> >>> Given that we will have terminals in all US timezones it is not >>> really a good option to set them all to the same time as the server. >>> >>> ----- Original Message ----- >>> From: "Scott Gray" <[hidden email]> >>> To: [hidden email] >>> Sent: Thursday, December 6, 2007 11:32:11 PM (GMT-0700) America/ >>> Denver >>> Subject: Re: Strange behavior with Entity Sync >>> >>> Just a stab in the dark but do these issues arise if the server >>> and terminal >>> are in the same timezone? >>> >>> Regards >>> Scott >>> >>> On 07/12/2007, Vince M. Clark <[hidden email]> wrote: >>> >>>> BJ - your question is not ignorant, but I have no idea how to >>>> answer. >>>> We'll have to depend on others in the community much smarter than >>>> us. >>>> >>>> Entity Sync is a feature available in the system. We are using it >>>> and I >>>> can only assume that it works if configured properly. I have >>>> figured out the >>>> basics of push and pull synchronization. That is about as much >>>> intelligence >>>> as I can claim. >>>> >>>> I can get into the details of what I loaded and where if >>>> necessary, such >>>> as EnityGroup, EnittySync, RecurrenceRule, etc. >>>> >>>> ----- Original Message ----- >>>> From: "BJ Freeman" <[hidden email]> >>>> To: [hidden email] >>>> Sent: Thursday, December 6, 2007 4:35:09 PM (GMT-0700) America/ >>>> Denver >>>> Subject: Re: Strange behavior with Entity Sync >>>> >>>> ignorant question >>>> does not entity sync mess up the accounting. >>>> how are the ecas that are suppose to fire based on certain >>>> services, >>>> that do CRUD of the entity suppose to fire? >>>> >>>> am I missing some documentation on how this works? >>>> >>>> >>>> Vince M. Clark sent the following on 12/6/2007 1:58 PM: >>>> >>>>> We are using entity sync to keep POS terminals and a central >>>>> server >>>> >>>> synchronized. We are going direct from POS to MCS, no Per Store >>>> Server >>>> involved. We have a pull sync (MCS to POS) and a push sync (POS >>>> to MCS.) >>>> Some data such as inventory is going both ways. >>>> >>>>> A few issues: >>>>> 1) Records deleted on MCS are not deleted on POS. >>>>> 2) Pull sync is not getting complete records in some cases. Two >>>>> entities >>>> >>>> we see this consistenly is on Facility and InventoryItem. We get >>>> partial >>>> records sometimes. Note that this only seems to occur on the >>>> initial pull >>>> when we are first populating a new POS terminal from the central >>>> server. >>>> Once we identify a record, such as a facility, that didn't come >>>> down >>>> completely we can update the timestamp on the server and the next >>>> pull gets >>>> the complete record. >>>> >>>>> 3) Related to item 2. If a push occurs from the POS terminal >>>>> BEFORE we >>>> >>>> get the initial data correct, it overwrites the data on MCS. For >>>> example, if >>>> a facility record was pulled down and the store id is missing >>>> then when the >>>> terminal push occurs it trashes the facility record on the >>>> server. Strange >>>> thing is we aren't pushing facility data, only pulling. >>>> >>>>> Note that the server time is different than the terminals >>>>> (different >>>> >>>> timezones.) >>>> >>>>> Vince Clark >>>>> Global Era >>>>> The Freedom of Open Source >>>>> [hidden email] >>>>> (303) 493-6723 >>>>> >>> >> >> > |
Re: Strange behavior with Entity Sync
|
|
The important thing to remember about Timestamps is that they are like constants - a Timestamp
represents a specific moment in time regardless of where the user is. Developers get confused and think they have to "adjust" Timestamps - something you can't do. However, you CAN adjust how the Timestamp is displayed - to present the moment in time in the user's time zone. If I create a record at 5pm my time (PST) and at the exact same time someone on the east coast creates a record (8 pm their time) - both records will have the same Timestamp value. If they aren't, then that's a bug - something went wrong in converting the user's time zone to a Timestamp. -Adrian Vince M. Clark wrote: > If in fact timestamps are causing problems it seems to me that a simple fix would be to write the records to the target with the current time on the target rather than inheriting timestamps from the source. > > ----- Original Message ----- > From: "Vince M. Clark" <[hidden email]> > To: [hidden email] > Sent: Friday, December 7, 2007 11:13:16 AM (GMT-0700) America/Denver > Subject: Re: Strange behavior with Entity Sync > > > Clarification. The timestamps on the central server are GMT. POS terminals are all in their local timezones. So we do get into a bit of trickiness around timestamps I think. > > Here is the summary of what I'm doing: > To setup the pull I create an entity sync record on the source (MCS) > I load the recurrence rule and job on the target (POS) > The pull launches, and sucks everything down from the server based on the EntityGroup defined on the MCS. During this process it appears as though the records are written to POS with the server timestamp. The sync process does not seem to change the timestamp to the local time on the POS terminal. This seems to be OK for data that is only pulled, because the next job that runs a pull will depend on logic running on MCS to determine what records to send down to POS. > But then when we trigger the push, some entities are synched both ways, such as inventory. > We did have one situation where inventory was overwritten on MCS with bad data. My theory is that the POS terminal pulled that data from the server initially with the server timestamp. Then on the push, since the server timestamp was in the future, the push process picked up those records as having been changed. > This might explain why our initiial push takes forever. The pos terminals may be sending everything back to the server. > > I'll try setting all machines to the same timezone and see if it stabilizes the process. > > ----- Original Message ----- > From: "David E Jones" <[hidden email]> > To: [hidden email] > Sent: Friday, December 7, 2007 10:57:34 AM (GMT-0700) America/Denver > Subject: Re: Strange behavior with Entity Sync > > > Yeah, it doesn't sound like it's a time zone problem. As long at the > timestamps in the databases are all GMT then it shouldn't be a bit deal. > > There could be other things going on though, like the last updated > stamp for a certain record on the target server could be more recent > than the one on the source server (if seed/initial data was loaded on > the source and then on the target, even if the source was updated > before the target was loaded) then it would not write the record. Of > course, if that happens it will tell you that it did that and you'll > at least see it in the summary numbers that come back from the remote > call. > > It sounds like you have things far enough along that the only way to > progress is with elbow grease. In other words pick a couple of records > that you think should be transfered over but that are not getting > copied, and then look at the data (on both source and target if > applicable) and read through the code to see exactly what it would do > with that data. > > -David > > > On Dec 7, 2007, at 10:27 AM, Vince M. Clark wrote: > > >>I don't know for sure that we are dealing with a timezone issue. >>That is just one theory as we try to iron out some problems with >>entity sync. The fact that the same records always seem to be >>problematic gives us a pattern to pursue. I'll update the thread as >>we research the problem. >> >>----- Original Message ----- >>From: "Adrian Crum" <[hidden email]> >>To: [hidden email] >>Sent: Friday, December 7, 2007 10:33:19 AM (GMT-0700) America/Denver >>Subject: Re: Strange behavior with Entity Sync >> >>I don't know if the recent time zone work would have any affect on >>this. It's a possibility. >> >>I'm not familiar with POS and entity sync, so I'll describe the >>changes I made and maybe others who >>are more familiar with Vince's setup can figure it out. >> >>The basic concept is: >> >>1. Date/time String in UI (user's time zone) gets converted to >>TimeStamp (which is referenced to >>GMT), then it is stored. >>2. TimeStamp in storage (GMT) gets converted to date/time String for >>UI (user's time zone). >>3. Only the screen widget and Freemarker template code was changed. >> >>You shouldn't have to set all of your terminals to the same time >>zone as the server. If the problem >>is in OFBiz, we should fix it. >> >>-Adrian >> >> >>Vince M. Clark wrote: >> >> >>>Yes. I pulled it from trunk about a week ago. >>> >>>----- Original Message ----- >>>From: "Jacopo Cappellato" <[hidden email]> >>>To: [hidden email] >>>Sent: Friday, December 7, 2007 8:36:18 AM (GMT-0700) America/Denver >>>Subject: Re: Strange behavior with Entity Sync >>> >>>Hi Vince, >>> >>>are you running a recent OFBiz revision? >>>I guess that Scott asked you about the timezones because we have >>>recently refactored the way datetimes are treated in OFBiz: Adrian >>>Crum >>>worked on this, and he could better explain, but the idea is that >>>when a >>>date field is edited it is converted from the user session locale >>>to the >>>system locale. >>>I think that Scott could be right... Adrian, do you think that your >>>work >>>could have caused some side effects on the entity synch stuff? >>> >>>Jacopo >>> >>> >>> >>>Vince M. Clark wrote: >>> >>> >>>>It could be a timezone issue. We will test a bit more and try to >>>>nail down a pattern. We can isolate the problem to the same set of >>>>records that always get pulled incomplete. >>>> >>>>The server is set to GMT and all the POS terminals are mountain. >>>>When first setting up sync I had some difficulty getting the PULL >>>>to start running and suspected it was due to the timezone being in >>>>the future on the server. But we got past that. >>>> >>>>Given that we will have terminals in all US timezones it is not >>>>really a good option to set them all to the same time as the server. >>>> >>>>----- Original Message ----- >>>>From: "Scott Gray" <[hidden email]> >>>>To: [hidden email] >>>>Sent: Thursday, December 6, 2007 11:32:11 PM (GMT-0700) America/ >>>>Denver >>>>Subject: Re: Strange behavior with Entity Sync >>>> >>>>Just a stab in the dark but do these issues arise if the server >>>>and terminal >>>>are in the same timezone? >>>> >>>>Regards >>>>Scott >>>> >>>>On 07/12/2007, Vince M. Clark <[hidden email]> wrote: >>>> >>>> >>>>>BJ - your question is not ignorant, but I have no idea how to >>>>>answer. >>>>>We'll have to depend on others in the community much smarter than >>>>>us. >>>>> >>>>>Entity Sync is a feature available in the system. We are using it >>>>>and I >>>>>can only assume that it works if configured properly. I have >>>>>figured out the >>>>>basics of push and pull synchronization. That is about as much >>>>>intelligence >>>>>as I can claim. >>>>> >>>>>I can get into the details of what I loaded and where if >>>>>necessary, such >>>>>as EnityGroup, EnittySync, RecurrenceRule, etc. >>>>> >>>>>----- Original Message ----- >>>>>From: "BJ Freeman" <[hidden email]> >>>>>To: [hidden email] >>>>>Sent: Thursday, December 6, 2007 4:35:09 PM (GMT-0700) America/ >>>>>Denver >>>>>Subject: Re: Strange behavior with Entity Sync >>>>> >>>>>ignorant question >>>>>does not entity sync mess up the accounting. >>>>>how are the ecas that are suppose to fire based on certain >>>>>services, >>>>>that do CRUD of the entity suppose to fire? >>>>> >>>>>am I missing some documentation on how this works? >>>>> >>>>> >>>>>Vince M. Clark sent the following on 12/6/2007 1:58 PM: >>>>> >>>>> >>>>>>We are using entity sync to keep POS terminals and a central >>>>>>server >>>>> >>>>>synchronized. We are going direct from POS to MCS, no Per Store >>>>>Server >>>>>involved. We have a pull sync (MCS to POS) and a push sync (POS >>>>>to MCS.) >>>>>Some data such as inventory is going both ways. >>>>> >>>>> >>>>>>A few issues: >>>>>>1) Records deleted on MCS are not deleted on POS. >>>>>>2) Pull sync is not getting complete records in some cases. Two >>>>>>entities >>>>> >>>>>we see this consistenly is on Facility and InventoryItem. We get >>>>>partial >>>>>records sometimes. Note that this only seems to occur on the >>>>>initial pull >>>>>when we are first populating a new POS terminal from the central >>>>>server. >>>>>Once we identify a record, such as a facility, that didn't come >>>>>down >>>>>completely we can update the timestamp on the server and the next >>>>>pull gets >>>>>the complete record. >>>>> >>>>> >>>>>>3) Related to item 2. If a push occurs from the POS terminal >>>>>>BEFORE we >>>>> >>>>>get the initial data correct, it overwrites the data on MCS. For >>>>>example, if >>>>>a facility record was pulled down and the store id is missing >>>>>then when the >>>>>terminal push occurs it trashes the facility record on the >>>>>server. Strange >>>>>thing is we aren't pushing facility data, only pulling. >>>>> >>>>> >>>>>>Note that the server time is different than the terminals >>>>>>(different >>>>> >>>>>timezones.) >>>>> >>>>> >>>>>>Vince Clark >>>>>>Global Era >>>>>>The Freedom of Open Source >>>>>>[hidden email] >>>>>>(303) 493-6723 >>>>>> >>>> >>> > > |
Re: Strange behavior with Entity Sync
|
|
Adrian
I would agree if both records are created on the same machine. The timestamps should match. However, when you are doing synchronization the clocks on various machines are different. So going with your scenario: Record is written to central server at 5:00 pm PST, so the timestamp gets written as '2007-12-07 17:00:000' At the same time an order is transacted on a POS terminal on the east coast. The timestamp on that machine is '2007-12-07 20:00:000' When sync occurs (MCS record is pulled by POS, POS order is pushed to central server). The timestamp is carried with the transfer, so the POS order gets written to MCS as '2007-12-07 20:00:000'. So the only way we know when the order was actually placed was to look at what POS terminal took the order and adjust for that timezone. ----- Original Message ----- From: "Adrian Crum" <[hidden email]> To: [hidden email] Sent: Friday, December 7, 2007 11:37:32 AM (GMT-0700) America/Denver Subject: Re: Strange behavior with Entity Sync The important thing to remember about Timestamps is that they are like constants - a Timestamp represents a specific moment in time regardless of where the user is. Developers get confused and think they have to "adjust" Timestamps - something you can't do. However, you CAN adjust how the Timestamp is displayed - to present the moment in time in the user's time zone. If I create a record at 5pm my time (PST) and at the exact same time someone on the east coast creates a record (8 pm their time) - both records will have the same Timestamp value. If they aren't, then that's a bug - something went wrong in converting the user's time zone to a Timestamp. -Adrian Vince M. Clark wrote: > If in fact timestamps are causing problems it seems to me that a simple fix would be to write the records to the target with the current time on the target rather than inheriting timestamps from the source. > > ----- Original Message ----- > From: "Vince M. Clark" <[hidden email]> > To: [hidden email] > Sent: Friday, December 7, 2007 11:13:16 AM (GMT-0700) America/Denver > Subject: Re: Strange behavior with Entity Sync > > > Clarification. The timestamps on the central server are GMT. POS terminals are all in their local timezones. So we do get into a bit of trickiness around timestamps I think. > > Here is the summary of what I'm doing: > To setup the pull I create an entity sync record on the source (MCS) > I load the recurrence rule and job on the target (POS) > The pull launches, and sucks everything down from the server based on the EntityGroup defined on the MCS. During this process it appears as though the records are written to POS with the server timestamp. The sync process does not seem to change the timestamp to the local time on the POS terminal. This seems to be OK for data that is only pulled, because the next job that runs a pull will depend on logic running on MCS to determine what records to send down to POS. > But then when we trigger the push, some entities are synched both ways, such as inventory. > We did have one situation where inventory was overwritten on MCS with bad data. My theory is that the POS terminal pulled that data from the server initially with the server timestamp. Then on the push, since the server timestamp was in the future, the push process picked up those records as having been changed. > This might explain why our initiial push takes forever. The pos terminals may be sending everything back to the server. > > I'll try setting all machines to the same timezone and see if it stabilizes the process. > > ----- Original Message ----- > From: "David E Jones" <[hidden email]> > To: [hidden email] > Sent: Friday, December 7, 2007 10:57:34 AM (GMT-0700) America/Denver > Subject: Re: Strange behavior with Entity Sync > > > Yeah, it doesn't sound like it's a time zone problem. As long at the > timestamps in the databases are all GMT then it shouldn't be a bit deal. > > There could be other things going on though, like the last updated > stamp for a certain record on the target server could be more recent > than the one on the source server (if seed/initial data was loaded on > the source and then on the target, even if the source was updated > before the target was loaded) then it would not write the record. Of > course, if that happens it will tell you that it did that and you'll > at least see it in the summary numbers that come back from the remote > call. > > It sounds like you have things far enough along that the only way to > progress is with elbow grease. In other words pick a couple of records > that you think should be transfered over but that are not getting > copied, and then look at the data (on both source and target if > applicable) and read through the code to see exactly what it would do > with that data. > > -David > > > On Dec 7, 2007, at 10:27 AM, Vince M. Clark wrote: > > >>I don't know for sure that we are dealing with a timezone issue. >>That is just one theory as we try to iron out some problems with >>entity sync. The fact that the same records always seem to be >>problematic gives us a pattern to pursue. I'll update the thread as >>we research the problem. >> >>----- Original Message ----- >>From: "Adrian Crum" <[hidden email]> >>To: [hidden email] >>Sent: Friday, December 7, 2007 10:33:19 AM (GMT-0700) America/Denver >>Subject: Re: Strange behavior with Entity Sync >> >>I don't know if the recent time zone work would have any affect on >>this. It's a possibility. >> >>I'm not familiar with POS and entity sync, so I'll describe the >>changes I made and maybe others who >>are more familiar with Vince's setup can figure it out. >> >>The basic concept is: >> >>1. Date/time String in UI (user's time zone) gets converted to >>TimeStamp (which is referenced to >>GMT), then it is stored. >>2. TimeStamp in storage (GMT) gets converted to date/time String for >>UI (user's time zone). >>3. Only the screen widget and Freemarker template code was changed. >> >>You shouldn't have to set all of your terminals to the same time >>zone as the server. If the problem >>is in OFBiz, we should fix it. >> >>-Adrian >> >> >>Vince M. Clark wrote: >> >> >>>Yes. I pulled it from trunk about a week ago. >>> >>>----- Original Message ----- >>>From: "Jacopo Cappellato" <[hidden email]> >>>To: [hidden email] >>>Sent: Friday, December 7, 2007 8:36:18 AM (GMT-0700) America/Denver >>>Subject: Re: Strange behavior with Entity Sync >>> >>>Hi Vince, >>> >>>are you running a recent OFBiz revision? >>>I guess that Scott asked you about the timezones because we have >>>recently refactored the way datetimes are treated in OFBiz: Adrian >>>Crum >>>worked on this, and he could better explain, but the idea is that >>>when a >>>date field is edited it is converted from the user session locale >>>to the >>>system locale. >>>I think that Scott could be right... Adrian, do you think that your >>>work >>>could have caused some side effects on the entity synch stuff? >>> >>>Jacopo >>> >>> >>> >>>Vince M. Clark wrote: >>> >>> >>>>It could be a timezone issue. We will test a bit more and try to >>>>nail down a pattern. We can isolate the problem to the same set of >>>>records that always get pulled incomplete. >>>> >>>>The server is set to GMT and all the POS terminals are mountain. >>>>When first setting up sync I had some difficulty getting the PULL >>>>to start running and suspected it was due to the timezone being in >>>>the future on the server. But we got past that. >>>> >>>>Given that we will have terminals in all US timezones it is not >>>>really a good option to set them all to the same time as the server. >>>> >>>>----- Original Message ----- >>>>From: "Scott Gray" <[hidden email]> >>>>To: [hidden email] >>>>Sent: Thursday, December 6, 2007 11:32:11 PM (GMT-0700) America/ >>>>Denver >>>>Subject: Re: Strange behavior with Entity Sync >>>> >>>>Just a stab in the dark but do these issues arise if the server >>>>and terminal >>>>are in the same timezone? >>>> >>>>Regards >>>>Scott >>>> >>>>On 07/12/2007, Vince M. Clark <[hidden email]> wrote: >>>> >>>> >>>>>BJ - your question is not ignorant, but I have no idea how to >>>>>answer. >>>>>We'll have to depend on others in the community much smarter than >>>>>us. >>>>> >>>>>Entity Sync is a feature available in the system. We are using it >>>>>and I >>>>>can only assume that it works if configured properly. I have >>>>>figured out the >>>>>basics of push and pull synchronization. That is about as much >>>>>intelligence >>>>>as I can claim. >>>>> >>>>>I can get into the details of what I loaded and where if >>>>>necessary, such >>>>>as EnityGroup, EnittySync, RecurrenceRule, etc. >>>>> >>>>>----- Original Message ----- >>>>>From: "BJ Freeman" <[hidden email]> >>>>>To: [hidden email] >>>>>Sent: Thursday, December 6, 2007 4:35:09 PM (GMT-0700) America/ >>>>>Denver >>>>>Subject: Re: Strange behavior with Entity Sync >>>>> >>>>>ignorant question >>>>>does not entity sync mess up the accounting. >>>>>how are the ecas that are suppose to fire based on certain >>>>>services, >>>>>that do CRUD of the entity suppose to fire? >>>>> >>>>>am I missing some documentation on how this works? >>>>> >>>>> >>>>>Vince M. Clark sent the following on 12/6/2007 1:58 PM: >>>>> >>>>> >>>>>>We are using entity sync to keep POS terminals and a central >>>>>>server >>>>> >>>>>synchronized. We are going direct from POS to MCS, no Per Store >>>>>Server >>>>>involved. We have a pull sync (MCS to POS) and a push sync (POS >>>>>to MCS.) >>>>>Some data such as inventory is going both ways. >>>>> >>>>> >>>>>>A few issues: >>>>>>1) Records deleted on MCS are not deleted on POS. >>>>>>2) Pull sync is not getting complete records in some cases. Two >>>>>>entities >>>>> >>>>>we see this consistenly is on Facility and InventoryItem. We get >>>>>partial >>>>>records sometimes. Note that this only seems to occur on the >>>>>initial pull >>>>>when we are first populating a new POS terminal from the central >>>>>server. >>>>>Once we identify a record, such as a facility, that didn't come >>>>>down >>>>>completely we can update the timestamp on the server and the next >>>>>pull gets >>>>>the complete record. >>>>> >>>>> >>>>>>3) Related to item 2. If a push occurs from the POS terminal >>>>>>BEFORE we >>>>> >>>>>get the initial data correct, it overwrites the data on MCS. For >>>>>example, if >>>>>a facility record was pulled down and the store id is missing >>>>>then when the >>>>>terminal push occurs it trashes the facility record on the >>>>>server. Strange >>>>>thing is we aren't pushing facility data, only pulling. >>>>> >>>>> >>>>>>Note that the server time is different than the terminals >>>>>>(different >>>>> >>>>>timezones.) >>>>> >>>>> >>>>>>Vince Clark >>>>>>Global Era >>>>>>The Freedom of Open Source >>>>>>[hidden email] >>>>>>(303) 493-6723 >>>>>> >>>> >>> > > |
Re: Strange behavior with Entity Sync
|
|
Vince,
You're making the exact mistake I just described. Think about it some more. -Adrian Vince M. Clark wrote: > Adrian > > I would agree if both records are created on the same machine. The timestamps should match. > > However, when you are doing synchronization the clocks on various machines are different. So going with your scenario: > Record is written to central server at 5:00 pm PST, so the timestamp gets written as '2007-12-07 17:00:000' > At the same time an order is transacted on a POS terminal on the east coast. The timestamp on that machine is '2007-12-07 20:00:000' > > When sync occurs (MCS record is pulled by POS, POS order is pushed to central server). The timestamp is carried with the transfer, so the POS order gets written to MCS as '2007-12-07 20:00:000'. > > So the only way we know when the order was actually placed was to look at what POS terminal took the order and adjust for that timezone. > > ----- Original Message ----- > From: "Adrian Crum" <[hidden email]> > To: [hidden email] > Sent: Friday, December 7, 2007 11:37:32 AM (GMT-0700) America/Denver > Subject: Re: Strange behavior with Entity Sync > > The important thing to remember about Timestamps is that they are like constants - a Timestamp > represents a specific moment in time regardless of where the user is. Developers get confused and > think they have to "adjust" Timestamps - something you can't do. However, you CAN adjust how the > Timestamp is displayed - to present the moment in time in the user's time zone. > > If I create a record at 5pm my time (PST) and at the exact same time someone on the east coast > creates a record (8 pm their time) - both records will have the same Timestamp value. If they > aren't, then that's a bug - something went wrong in converting the user's time zone to a Timestamp. > > -Adrian > > Vince M. Clark wrote: > > >>If in fact timestamps are causing problems it seems to me that a simple fix would be to write the records to the target with the current time on the target rather than inheriting timestamps from the source. >> >>----- Original Message ----- >>From: "Vince M. Clark" <[hidden email]> >>To: [hidden email] >>Sent: Friday, December 7, 2007 11:13:16 AM (GMT-0700) America/Denver >>Subject: Re: Strange behavior with Entity Sync >> >> >>Clarification. The timestamps on the central server are GMT. POS terminals are all in their local timezones. So we do get into a bit of trickiness around timestamps I think. >> >>Here is the summary of what I'm doing: >>To setup the pull I create an entity sync record on the source (MCS) >>I load the recurrence rule and job on the target (POS) >>The pull launches, and sucks everything down from the server based on the EntityGroup defined on the MCS. During this process it appears as though the records are written to POS with the server timestamp. The sync process does not seem to change the timestamp to the local time on the POS terminal. This seems to be OK for data that is only pulled, because the next job that runs a pull will depend on logic running on MCS to determine what records to send down to POS. >>But then when we trigger the push, some entities are synched both ways, such as inventory. >>We did have one situation where inventory was overwritten on MCS with bad data. My theory is that the POS terminal pulled that data from the server initially with the server timestamp. Then on the push, since the server timestamp was in the future, the push process picked up those records as having been changed. >>This might explain why our initiial push takes forever. The pos terminals may be sending everything back to the server. >> >>I'll try setting all machines to the same timezone and see if it stabilizes the process. >> >>----- Original Message ----- >>From: "David E Jones" <[hidden email]> >>To: [hidden email] >>Sent: Friday, December 7, 2007 10:57:34 AM (GMT-0700) America/Denver >>Subject: Re: Strange behavior with Entity Sync >> >> >>Yeah, it doesn't sound like it's a time zone problem. As long at the >>timestamps in the databases are all GMT then it shouldn't be a bit deal. >> >>There could be other things going on though, like the last updated >>stamp for a certain record on the target server could be more recent >>than the one on the source server (if seed/initial data was loaded on >>the source and then on the target, even if the source was updated >>before the target was loaded) then it would not write the record. Of >>course, if that happens it will tell you that it did that and you'll >>at least see it in the summary numbers that come back from the remote >>call. >> >>It sounds like you have things far enough along that the only way to >>progress is with elbow grease. In other words pick a couple of records >>that you think should be transfered over but that are not getting >>copied, and then look at the data (on both source and target if >>applicable) and read through the code to see exactly what it would do >>with that data. >> >>-David >> >> >>On Dec 7, 2007, at 10:27 AM, Vince M. Clark wrote: >> >> >> >>>I don't know for sure that we are dealing with a timezone issue. >>>That is just one theory as we try to iron out some problems with >>>entity sync. The fact that the same records always seem to be >>>problematic gives us a pattern to pursue. I'll update the thread as >>>we research the problem. >>> >>>----- Original Message ----- >>>From: "Adrian Crum" <[hidden email]> >>>To: [hidden email] >>>Sent: Friday, December 7, 2007 10:33:19 AM (GMT-0700) America/Denver >>>Subject: Re: Strange behavior with Entity Sync >>> >>>I don't know if the recent time zone work would have any affect on >>>this. It's a possibility. >>> >>>I'm not familiar with POS and entity sync, so I'll describe the >>>changes I made and maybe others who >>>are more familiar with Vince's setup can figure it out. >>> >>>The basic concept is: >>> >>>1. Date/time String in UI (user's time zone) gets converted to >>>TimeStamp (which is referenced to >>>GMT), then it is stored. >>>2. TimeStamp in storage (GMT) gets converted to date/time String for >>>UI (user's time zone). >>>3. Only the screen widget and Freemarker template code was changed. >>> >>>You shouldn't have to set all of your terminals to the same time >>>zone as the server. If the problem >>>is in OFBiz, we should fix it. >>> >>>-Adrian >>> >>> >>>Vince M. Clark wrote: >>> >>> >>> >>>>Yes. I pulled it from trunk about a week ago. >>>> >>>>----- Original Message ----- >>>>From: "Jacopo Cappellato" <[hidden email]> >>>>To: [hidden email] >>>>Sent: Friday, December 7, 2007 8:36:18 AM (GMT-0700) America/Denver >>>>Subject: Re: Strange behavior with Entity Sync >>>> >>>>Hi Vince, >>>> >>>>are you running a recent OFBiz revision? >>>>I guess that Scott asked you about the timezones because we have >>>>recently refactored the way datetimes are treated in OFBiz: Adrian >>>>Crum >>>>worked on this, and he could better explain, but the idea is that >>>>when a >>>>date field is edited it is converted from the user session locale >>>>to the >>>>system locale. >>>>I think that Scott could be right... Adrian, do you think that your >>>>work >>>>could have caused some side effects on the entity synch stuff? >>>> >>>>Jacopo >>>> >>>> >>>> >>>>Vince M. Clark wrote: >>>> >>>> >>>> >>>>>It could be a timezone issue. We will test a bit more and try to >>>>>nail down a pattern. We can isolate the problem to the same set of >>>>>records that always get pulled incomplete. >>>>> >>>>>The server is set to GMT and all the POS terminals are mountain. >>>>>When first setting up sync I had some difficulty getting the PULL >>>>>to start running and suspected it was due to the timezone being in >>>>>the future on the server. But we got past that. >>>>> >>>>>Given that we will have terminals in all US timezones it is not >>>>>really a good option to set them all to the same time as the server. >>>>> >>>>>----- Original Message ----- >>>>>From: "Scott Gray" <[hidden email]> >>>>>To: [hidden email] >>>>>Sent: Thursday, December 6, 2007 11:32:11 PM (GMT-0700) America/ >>>>>Denver >>>>>Subject: Re: Strange behavior with Entity Sync >>>>> >>>>>Just a stab in the dark but do these issues arise if the server >>>>>and terminal >>>>>are in the same timezone? >>>>> >>>>>Regards >>>>>Scott >>>>> >>>>>On 07/12/2007, Vince M. Clark <[hidden email]> wrote: >>>>> >>>>> >>>>> >>>>>>BJ - your question is not ignorant, but I have no idea how to >>>>>>answer. >>>>>>We'll have to depend on others in the community much smarter than >>>>>>us. >>>>>> >>>>>>Entity Sync is a feature available in the system. We are using it >>>>>>and I >>>>>>can only assume that it works if configured properly. I have >>>>>>figured out the >>>>>>basics of push and pull synchronization. That is about as much >>>>>>intelligence >>>>>>as I can claim. >>>>>> >>>>>>I can get into the details of what I loaded and where if >>>>>>necessary, such >>>>>>as EnityGroup, EnittySync, RecurrenceRule, etc. >>>>>> >>>>>>----- Original Message ----- >>>>>>From: "BJ Freeman" <[hidden email]> >>>>>>To: [hidden email] >>>>>>Sent: Thursday, December 6, 2007 4:35:09 PM (GMT-0700) America/ >>>>>>Denver >>>>>>Subject: Re: Strange behavior with Entity Sync >>>>>> >>>>>>ignorant question >>>>>>does not entity sync mess up the accounting. >>>>>>how are the ecas that are suppose to fire based on certain >>>>>>services, >>>>>>that do CRUD of the entity suppose to fire? >>>>>> >>>>>>am I missing some documentation on how this works? >>>>>> >>>>>> >>>>>>Vince M. Clark sent the following on 12/6/2007 1:58 PM: >>>>>> >>>>>> >>>>>> >>>>>>>We are using entity sync to keep POS terminals and a central >>>>>>>server >>>>>> >>>>>>synchronized. We are going direct from POS to MCS, no Per Store >>>>>>Server >>>>>>involved. We have a pull sync (MCS to POS) and a push sync (POS >>>>>>to MCS.) >>>>>>Some data such as inventory is going both ways. >>>>>> >>>>>> >>>>>> >>>>>>>A few issues: >>>>>>>1) Records deleted on MCS are not deleted on POS. >>>>>>>2) Pull sync is not getting complete records in some cases. Two >>>>>>>entities >>>>>> >>>>>>we see this consistenly is on Facility and InventoryItem. We get >>>>>>partial >>>>>>records sometimes. Note that this only seems to occur on the >>>>>>initial pull >>>>>>when we are first populating a new POS terminal from the central >>>>>>server. >>>>>>Once we identify a record, such as a facility, that didn't come >>>>>>down >>>>>>completely we can update the timestamp on the server and the next >>>>>>pull gets >>>>>>the complete record. >>>>>> >>>>>> >>>>>> >>>>>>>3) Related to item 2. If a push occurs from the POS terminal >>>>>>>BEFORE we >>>>>> >>>>>>get the initial data correct, it overwrites the data on MCS. For >>>>>>example, if >>>>>>a facility record was pulled down and the store id is missing >>>>>>then when the >>>>>>terminal push occurs it trashes the facility record on the >>>>>>server. Strange >>>>>>thing is we aren't pushing facility data, only pulling. >>>>>> >>>>>> >>>>>> >>>>>>>Note that the server time is different than the terminals >>>>>>>(different >>>>>> >>>>>>timezones.) >>>>>> >>>>>> >>>>>> >>>>>>>Vince Clark >>>>>>>Global Era >>>>>>>The Freedom of Open Source >>>>>>>[hidden email] >>>>>>>(303) 493-6723 >>>>>>> >>>>> >> > > |
Re: Strange behavior with Entity Sync
|
|
Diverging from the timestamp issue for a moment, I have a question about push vs. pull.
We are setting up sync in two steps: 1) We initiate a "pull" from POS terminal, pulling from central server. This pulls all of our initial data like parties, products, promos, prices, inventory, etc. 2) Then we initiate a "push" so that new orders and all associated transactions (invoices, payments, etc.) are sent back to central server. Both of these processes take quite a long time the first time they run. I would expect that from the pull, as there is a lot of data coming down from the server. However, the push should not take anytime at all since there is no new data. What I suspect is that the initial push is actually sending back all data that was just pulled. I believe this is happening because we had a situation where some records came down as incomplete, and then overwrote what was on the server, rendering the records on the server incomplete as well. If the push is sending all data back the first time it runs I need to prevent this from happening. Would this be possible if I loaded a dummy EntitySyncHistory record when I initially create the job so the first time the job runs it will pick up a timestamp and not grab all the data that was pulled from the server? ----- Original Message ----- From: "Adrian Crum" <[hidden email]> To: [hidden email] Sent: Friday, December 7, 2007 12:00:38 PM (GMT-0700) America/Denver Subject: Re: Strange behavior with Entity Sync Vince, You're making the exact mistake I just described. Think about it some more. -Adrian Vince M. Clark wrote: > Adrian > > I would agree if both records are created on the same machine. The timestamps should match. > > However, when you are doing synchronization the clocks on various machines are different. So going with your scenario: > Record is written to central server at 5:00 pm PST, so the timestamp gets written as '2007-12-07 17:00:000' > At the same time an order is transacted on a POS terminal on the east coast. The timestamp on that machine is '2007-12-07 20:00:000' > > When sync occurs (MCS record is pulled by POS, POS order is pushed to central server). The timestamp is carried with the transfer, so the POS order gets written to MCS as '2007-12-07 20:00:000'. > > So the only way we know when the order was actually placed was to look at what POS terminal took the order and adjust for that timezone. > > ----- Original Message ----- > From: "Adrian Crum" <[hidden email]> > To: [hidden email] > Sent: Friday, December 7, 2007 11:37:32 AM (GMT-0700) America/Denver > Subject: Re: Strange behavior with Entity Sync > > The important thing to remember about Timestamps is that they are like constants - a Timestamp > represents a specific moment in time regardless of where the user is. Developers get confused and > think they have to "adjust" Timestamps - something you can't do. However, you CAN adjust how the > Timestamp is displayed - to present the moment in time in the user's time zone. > > If I create a record at 5pm my time (PST) and at the exact same time someone on the east coast > creates a record (8 pm their time) - both records will have the same Timestamp value. If they > aren't, then that's a bug - something went wrong in converting the user's time zone to a Timestamp. > > -Adrian > > Vince M. Clark wrote: > > >>If in fact timestamps are causing problems it seems to me that a simple fix would be to write the records to the target with the current time on the target rather than inheriting timestamps from the source. >> >>----- Original Message ----- >>From: "Vince M. Clark" <[hidden email]> >>To: [hidden email] >>Sent: Friday, December 7, 2007 11:13:16 AM (GMT-0700) America/Denver >>Subject: Re: Strange behavior with Entity Sync >> >> >>Clarification. The timestamps on the central server are GMT. POS terminals are all in their local timezones. So we do get into a bit of trickiness around timestamps I think. >> >>Here is the summary of what I'm doing: >>To setup the pull I create an entity sync record on the source (MCS) >>I load the recurrence rule and job on the target (POS) >>The pull launches, and sucks everything down from the server based on the EntityGroup defined on the MCS. During this process it appears as though the records are written to POS with the server timestamp. The sync process does not seem to change the timestamp to the local time on the POS terminal. This seems to be OK for data that is only pulled, because the next job that runs a pull will depend on logic running on MCS to determine what records to send down to POS. >>But then when we trigger the push, some entities are synched both ways, such as inventory. >>We did have one situation where inventory was overwritten on MCS with bad data. My theory is that the POS terminal pulled that data from the server initially with the server timestamp. Then on the push, since the server timestamp was in the future, the push process picked up those records as having been changed. >>This might explain why our initiial push takes forever. The pos terminals may be sending everything back to the server. >> >>I'll try setting all machines to the same timezone and see if it stabilizes the process. >> >>----- Original Message ----- >>From: "David E Jones" <[hidden email]> >>To: [hidden email] >>Sent: Friday, December 7, 2007 10:57:34 AM (GMT-0700) America/Denver >>Subject: Re: Strange behavior with Entity Sync >> >> >>Yeah, it doesn't sound like it's a time zone problem. As long at the >>timestamps in the databases are all GMT then it shouldn't be a bit deal. >> >>There could be other things going on though, like the last updated >>stamp for a certain record on the target server could be more recent >>than the one on the source server (if seed/initial data was loaded on >>the source and then on the target, even if the source was updated >>before the target was loaded) then it would not write the record. Of >>course, if that happens it will tell you that it did that and you'll >>at least see it in the summary numbers that come back from the remote >>call. >> >>It sounds like you have things far enough along that the only way to >>progress is with elbow grease. In other words pick a couple of records >>that you think should be transfered over but that are not getting >>copied, and then look at the data (on both source and target if >>applicable) and read through the code to see exactly what it would do >>with that data. >> >>-David >> >> >>On Dec 7, 2007, at 10:27 AM, Vince M. Clark wrote: >> >> >> >>>I don't know for sure that we are dealing with a timezone issue. >>>That is just one theory as we try to iron out some problems with >>>entity sync. The fact that the same records always seem to be >>>problematic gives us a pattern to pursue. I'll update the thread as >>>we research the problem. >>> >>>----- Original Message ----- >>>From: "Adrian Crum" <[hidden email]> >>>To: [hidden email] >>>Sent: Friday, December 7, 2007 10:33:19 AM (GMT-0700) America/Denver >>>Subject: Re: Strange behavior with Entity Sync >>> >>>I don't know if the recent time zone work would have any affect on >>>this. It's a possibility. >>> >>>I'm not familiar with POS and entity sync, so I'll describe the >>>changes I made and maybe others who >>>are more familiar with Vince's setup can figure it out. >>> >>>The basic concept is: >>> >>>1. Date/time String in UI (user's time zone) gets converted to >>>TimeStamp (which is referenced to >>>GMT), then it is stored. >>>2. TimeStamp in storage (GMT) gets converted to date/time String for >>>UI (user's time zone). >>>3. Only the screen widget and Freemarker template code was changed. >>> >>>You shouldn't have to set all of your terminals to the same time >>>zone as the server. If the problem >>>is in OFBiz, we should fix it. >>> >>>-Adrian >>> >>> >>>Vince M. Clark wrote: >>> >>> >>> >>>>Yes. I pulled it from trunk about a week ago. >>>> >>>>----- Original Message ----- >>>>From: "Jacopo Cappellato" <[hidden email]> >>>>To: [hidden email] >>>>Sent: Friday, December 7, 2007 8:36:18 AM (GMT-0700) America/Denver >>>>Subject: Re: Strange behavior with Entity Sync >>>> >>>>Hi Vince, >>>> >>>>are you running a recent OFBiz revision? >>>>I guess that Scott asked you about the timezones because we have >>>>recently refactored the way datetimes are treated in OFBiz: Adrian >>>>Crum >>>>worked on this, and he could better explain, but the idea is that >>>>when a >>>>date field is edited it is converted from the user session locale >>>>to the >>>>system locale. >>>>I think that Scott could be right... Adrian, do you think that your >>>>work >>>>could have caused some side effects on the entity synch stuff? >>>> >>>>Jacopo >>>> >>>> >>>> >>>>Vince M. Clark wrote: >>>> >>>> >>>> >>>>>It could be a timezone issue. We will test a bit more and try to >>>>>nail down a pattern. We can isolate the problem to the same set of >>>>>records that always get pulled incomplete. >>>>> >>>>>The server is set to GMT and all the POS terminals are mountain. >>>>>When first setting up sync I had some difficulty getting the PULL >>>>>to start running and suspected it was due to the timezone being in >>>>>the future on the server. But we got past that. >>>>> >>>>>Given that we will have terminals in all US timezones it is not >>>>>really a good option to set them all to the same time as the server. >>>>> >>>>>----- Original Message ----- >>>>>From: "Scott Gray" <[hidden email]> >>>>>To: [hidden email] >>>>>Sent: Thursday, December 6, 2007 11:32:11 PM (GMT-0700) America/ >>>>>Denver >>>>>Subject: Re: Strange behavior with Entity Sync >>>>> >>>>>Just a stab in the dark but do these issues arise if the server >>>>>and terminal >>>>>are in the same timezone? >>>>> >>>>>Regards >>>>>Scott >>>>> >>>>>On 07/12/2007, Vince M. Clark <[hidden email]> wrote: >>>>> >>>>> >>>>> >>>>>>BJ - your question is not ignorant, but I have no idea how to >>>>>>answer. >>>>>>We'll have to depend on others in the community much smarter than >>>>>>us. >>>>>> >>>>>>Entity Sync is a feature available in the system. We are using it >>>>>>and I >>>>>>can only assume that it works if configured properly. I have >>>>>>figured out the >>>>>>basics of push and pull synchronization. That is about as much >>>>>>intelligence >>>>>>as I can claim. >>>>>> >>>>>>I can get into the details of what I loaded and where if >>>>>>necessary, such >>>>>>as EnityGroup, EnittySync, RecurrenceRule, etc. >>>>>> >>>>>>----- Original Message ----- >>>>>>From: "BJ Freeman" <[hidden email]> >>>>>>To: [hidden email] >>>>>>Sent: Thursday, December 6, 2007 4:35:09 PM (GMT-0700) America/ >>>>>>Denver >>>>>>Subject: Re: Strange behavior with Entity Sync >>>>>> >>>>>>ignorant question >>>>>>does not entity sync mess up the accounting. >>>>>>how are the ecas that are suppose to fire based on certain >>>>>>services, >>>>>>that do CRUD of the entity suppose to fire? >>>>>> >>>>>>am I missing some documentation on how this works? >>>>>> >>>>>> >>>>>>Vince M. Clark sent the following on 12/6/2007 1:58 PM: >>>>>> >>>>>> >>>>>> >>>>>>>We are using entity sync to keep POS terminals and a central >>>>>>>server >>>>>> >>>>>>synchronized. We are going direct from POS to MCS, no Per Store >>>>>>Server >>>>>>involved. We have a pull sync (MCS to POS) and a push sync (POS >>>>>>to MCS.) >>>>>>Some data such as inventory is going both ways. >>>>>> >>>>>> >>>>>> >>>>>>>A few issues: >>>>>>>1) Records deleted on MCS are not deleted on POS. >>>>>>>2) Pull sync is not getting complete records in some cases. Two >>>>>>>entities >>>>>> >>>>>>we see this consistenly is on Facility and InventoryItem. We get >>>>>>partial >>>>>>records sometimes. Note that this only seems to occur on the >>>>>>initial pull >>>>>>when we are first populating a new POS terminal from the central >>>>>>server. >>>>>>Once we identify a record, such as a facility, that didn't come >>>>>>down >>>>>>completely we can update the timestamp on the server and the next >>>>>>pull gets >>>>>>the complete record. >>>>>> >>>>>> >>>>>> >>>>>>>3) Related to item 2. If a push occurs from the POS terminal >>>>>>>BEFORE we >>>>>> >>>>>>get the initial data correct, it overwrites the data on MCS. For >>>>>>example, if >>>>>>a facility record was pulled down and the store id is missing >>>>>>then when the >>>>>>terminal push occurs it trashes the facility record on the >>>>>>server. Strange >>>>>>thing is we aren't pushing facility data, only pulling. >>>>>> >>>>>> >>>>>> >>>>>>>Note that the server time is different than the terminals >>>>>>>(different >>>>>> >>>>>>timezones.) >>>>>> >>>>>> >>>>>> >>>>>>>Vince Clark >>>>>>>Global Era >>>>>>>The Freedom of Open Source >>>>>>>[hidden email] >>>>>>>(303) 493-6723 >>>>>>> >>>>> >> > > |
|
|
In reply to this post by BJ Freeman
Brendan
Hmmm. Well, first, I would suggest that you get one of the very many wonderful Tomcat books. However, in a nutshell, you can run as many instances of Tomcat as your CPU can stomach so long as they are listening on different ports. Ofbiz has an imbedded Tomcat server that by default listens on port 8080 (instead of the port 80 that is the default for IIS and Apache). You can configure your Ofbiz to listen to port 80 if you want (and remove the need to add ":8080" to the url), but that is not generally a good idea unless it is entirely on an internal network because of security concerns. If you leave Ofbiz at 8080, you can run IIS with it (at the same time). Then you would have: http://localhost/MyWebApp1/ecommerce http://localhost/MyWebApp2/ecommerce http://localhost:8080/MyWebApp1/webtools http://localhost:8080/MyWebApp2/webtools assuming that you are running the first two on IIS and the second two on Ofbiz. You can tell Ofbiz where to load your apps with an entry in your custom ofbiz-component.xml like this: <webapp name="accounting" title="Accounting" server="default-server" location="webapp/accounting" base-permission="OFBTOOLS,ACCOUNTING" mount-point="/accounting"/> Its all pretty simple once you get the hang of it. On the other hand, if you haven't got a clue, you are going to struggle endlessly. Take my word for it because I have been through the same drill as has everyone else here. As I get back to work this Monday morning, I have 120+ emails from the Ofbiz list, and perhaps 1/2 are either from you or responses to your questions. The best advice I have heard given is to "read the docs". That is because everything you have asked so far can be found there. I know you don't want to here this, but if you want to be successful with Ofbiz, bite the bullet and get it done. Expect to spend a month on the "engine" portion of things. There are lots of resources on everything except the use of Widgets which is kinda sparce. Fortunately, the basics for Widgets are not too hard. Believe me when I tell you that the time spent reading and experimenting will be time well spent. The Ofbiz entity and service engines are nothing short of extraordinary and useful well beyond the accounting functionality normally associated with Ofbiz. If you intend to implement the business part of Ofbiz, read the Data Model Resource book through so you fully grok it, then keep it handy for a reference for a while. Not what you wanted to hear I expect, but it is the best advice I have. Skip -----Original Message----- From: Brendan Vogt [mailto:[hidden email]] Sent: Sunday, December 09, 2007 7:20 AM To: [hidden email] Subject: Tomcat Help Importance: High Hi Skip, You have helped me the most with OFBiz, and I am seeking personal help with Tomcat. Please can you advise me. You have always explained how to do things in the finest detail. So I appreciate that. I am running it on windows. I have Apache Tomcat 6.0.14 Server installed. I am not familiar with Tomcat at all. I normally run my .NET apps in IIS. I want to have a test OFBiz app where I can test an go crazy. And then I want my own clean app where I install my own data. I have use Eclipse to checkout the OFBiz code to the ofbiz and SAMarketplace directories. I would like to run both apps at the same time so that I can switch between the two. Normally in IIS I would do it like the following: http://localhost/MyWebApp1/ecommerce http://localhost/MyWebApp2/ecommerce http://localhost/MyWebApp1/webtools http://localhost/MyWebApp2/webtools How can I do this? Brendan (South Africa) |
«
Return to OFBiz - User
|
1 view|%1 views
| Free forum by Nabble | Edit this page |