Updates to EntityQuery
Locked
 
|
Hi all
We have internally patched EntityQuery with some additional functionality and before I create a Jira was going to see what the community thinks. New functionality: - New method getFieldMap which returns a Map object of selected fields from GenericValue objects, useful for creating cache map objects for lookup - Support forUpdate - forEach method to accept Consumer We have found these useful and believe the project can benefit, let me know what you think Regards Gareth Carter Software Development Analyst Stannah Management Services Ltd IT Department Ext: 7036 DDI: 01264 364311 [http://logos.stannah.co.uk/stan150.jpg] [http://logos.stannah.co.uk/enviro.jpg]Please consider the environment before printing this email. This email is intended only for the above addressee. It may contain privileged information. If you are not the addressee you must not copy, distribute, disclose or use any of the information in it. If you have received it in error, please delete it and notify the sender. Stannah Lift Holdings Ltd registered No. 686996, Stannah Management Services Ltd registered No. 2483693, Stannah Lift Services Ltd registered No. 1189799, Stannah Microlifts Ltd registered No. 964804, Stannah Lifts Ltd registered No. 1189836, Stannah Stairlifts Ltd registered No. 1401451, Global Upholstery Solutions Ltd registered No. 02452728. All registered offices at Watt Close, East Portway, Andover, Hampshire, SP10 3SD, England. All Registered in England and Wales. This message has been scanned for malware by Websense. www.websense.com |
|
|
Hello Gareth,
Please provide some more details or patch to understand - forUpdate and forEach method utility. getFieldMap method looks good to me. -- Thanks & Regards --- Arun Patidar Manager, Enterprise Software Development HotWax Systems Pvt Ltd. www.hotwaxsystems.com On Tue, Oct 24, 2017 at 9:06 PM, Gareth Carter <[hidden email]> wrote: > Hi all > > We have internally patched EntityQuery with some additional functionality > and before I create a Jira was going to see what the community thinks. > > New functionality: > > - New method getFieldMap which returns a Map object of selected > fields from GenericValue objects, useful for creating cache map objects for > lookup > > - Support forUpdate > > - forEach method to accept Consumer > > We have found these useful and believe the project can benefit, let me > know what you think > > Regards > > Gareth Carter > > > Software Development Analyst > > > Stannah Management Services Ltd > > > IT Department > > > Ext: > > > 7036 > > > DDI: > > > 01264 364311 > > > > > [http://logos.stannah.co.uk/stan150.jpg] > > > [http://logos.stannah.co.uk/enviro.jpg]Please consider the environment > before printing this email. > > > > This email is intended only for the above addressee. It may contain > privileged information. If you are not the addressee you must not copy, > distribute, disclose or use any of the information in it. If you have > received it in error, please delete it and notify the sender. > > Stannah Lift Holdings Ltd registered No. 686996, Stannah Management > Services Ltd registered No. 2483693, Stannah Lift Services Ltd registered > No. 1189799, Stannah Microlifts Ltd registered No. 964804, Stannah Lifts > Ltd registered No. 1189836, Stannah Stairlifts Ltd registered No. 1401451, > Global Upholstery Solutions Ltd registered No. 02452728. > > All registered offices at Watt Close, East Portway, Andover, Hampshire, > SP10 3SD, England. > > All Registered in England and Wales. > > This message has been scanned for malware by Websense. www.websense.com > |
|
|
Hello Gareth,
Looks like a nice addition to EntityQuery API. Thanks for that. As mentioned by Arun, please provide more details to understand about "forUpdate" and "forEach" methods. Regards -- Pritam Kute On Wed, Oct 25, 2017 at 10:18 AM, Arun Patidar < [hidden email]> wrote: > Hello Gareth, > > Please provide some more details or patch to understand - forUpdate > and forEach > method utility. > > getFieldMap method looks good to me. > > > > -- > Thanks & Regards > --- > Arun Patidar > Manager, Enterprise Software Development > > HotWax Systems Pvt Ltd. > > www.hotwaxsystems.com > > > On Tue, Oct 24, 2017 at 9:06 PM, Gareth Carter < > [hidden email]> > wrote: > > > Hi all > > > > We have internally patched EntityQuery with some additional functionality > > and before I create a Jira was going to see what the community thinks. > > > > New functionality: > > > > - New method getFieldMap which returns a Map object of selected > > fields from GenericValue objects, useful for creating cache map objects > for > > lookup > > > > - Support forUpdate > > > > - forEach method to accept Consumer > > > > We have found these useful and believe the project can benefit, let me > > know what you think > > > > Regards > > > > Gareth Carter > > > > > > Software Development Analyst > > > > > > Stannah Management Services Ltd > > > > > > IT Department > > > > > > Ext: > > > > > > 7036 > > > > > > DDI: > > > > > > 01264 364311 > > > > > > > > > > [http://logos.stannah.co.uk/stan150.jpg] > > > > > > [http://logos.stannah.co.uk/enviro.jpg]Please consider the environment > > before printing this email. > > > > > > > > This email is intended only for the above addressee. It may contain > > privileged information. If you are not the addressee you must not copy, > > distribute, disclose or use any of the information in it. If you have > > received it in error, please delete it and notify the sender. > > > > Stannah Lift Holdings Ltd registered No. 686996, Stannah Management > > Services Ltd registered No. 2483693, Stannah Lift Services Ltd registered > > No. 1189799, Stannah Microlifts Ltd registered No. 964804, Stannah Lifts > > Ltd registered No. 1189836, Stannah Stairlifts Ltd registered No. > 1401451, > > Global Upholstery Solutions Ltd registered No. 02452728. > > > > All registered offices at Watt Close, East Portway, Andover, Hampshire, > > SP10 3SD, England. > > > > All Registered in England and Wales. > > > > This message has been scanned for malware by Websense. www.websense.com > > > |
|
|
In reply to this post by Arun Patidar-2
forUpdate
We patched EntityFindOptions with a new field "forUpdate" with shorthand methods in EntityQuery to enable. We then made a change to GenericDAO.selectListIteratorByCondition to add "FOR UPDATE" on the end of the SQL select statement - this allows for DB row locks (we use postgres and works but have not tested other databases). I believe there may have been a discussion about this before forEach on EntityQuery Use Consumer in java and groovy to iterate over a query. This can reduce memory consumption (replacement for queryList()) and boiler plate code (eg queryIterator(), while loop and close) Example: EntityQuery.use(delegator).from("Foobar") .forEach(item -> Debug.logInfo(item.toString(), module); ); A further update could be to provide stream capabilities Hope this helps, I can provide a patch aswell Gareth Carter Software Development Analyst Stannah Management Services Ltd IT Department Ext: 7036 DDI: 01264 364311 Please consider the environment before printing this email. -----Original Message----- From: Arun Patidar [mailto:[hidden email]] Sent: 25 October 2017 5:49 AM To: [hidden email] Cc: [hidden email] Subject: Re: Updates to EntityQuery Hello Gareth, Please provide some more details or patch to understand - forUpdate and forEach method utility. getFieldMap method looks good to me. -- Thanks & Regards --- Arun Patidar Manager, Enterprise Software Development HotWax Systems Pvt Ltd. www.hotwaxsystems.com On Tue, Oct 24, 2017 at 9:06 PM, Gareth Carter <[hidden email]> wrote: > Hi all > > We have internally patched EntityQuery with some additional > functionality and before I create a Jira was going to see what the community thinks. > > New functionality: > > - New method getFieldMap which returns a Map object of selected > fields from GenericValue objects, useful for creating cache map > objects for lookup > > - Support forUpdate > > - forEach method to accept Consumer > > We have found these useful and believe the project can benefit, let me > know what you think > > Regards > > Gareth Carter > > > Software Development Analyst > > > Stannah Management Services Ltd > > > IT Department > > > Ext: > > > 7036 > > > DDI: > > > 01264 364311 > > > > > [http://logos.stannah.co.uk/stan150.jpg] > > > [http://logos.stannah.co.uk/enviro.jpg]Please consider the environment > before printing this email. > > > > This email is intended only for the above addressee. It may contain > privileged information. If you are not the addressee you must not > copy, distribute, disclose or use any of the information in it. If you > have received it in error, please delete it and notify the sender. > > Stannah Lift Holdings Ltd registered No. 686996, Stannah Management > Services Ltd registered No. 2483693, Stannah Lift Services Ltd > registered No. 1189799, Stannah Microlifts Ltd registered No. 964804, > Stannah Lifts Ltd registered No. 1189836, Stannah Stairlifts Ltd > registered No. 1401451, Global Upholstery Solutions Ltd registered No. 02452728. > > All registered offices at Watt Close, East Portway, Andover, > Hampshire, > SP10 3SD, England. > > All Registered in England and Wales. > > This message has been scanned for malware by Websense. > www.websense.com > To report this email as spam, please send the original message, complete with headers to [hidden email] |
|
|
Yes, Usage of the stream will surely enhance performance to a certain
extent and removes ceremony from the code base. I think Parallel streams will add enhancements in many folds as we are using a multicore processor (I have not tested yet), but the Functional approach seems promising. Thanks & Regards, -- *Pradhan Yash Sharma* *HotWax Systems* | www.hotwaxsystems.com On Wed, Oct 25, 2017 at 5:59 PM, Gareth Carter <[hidden email]> wrote: > forUpdate > > We patched EntityFindOptions with a new field "forUpdate" with shorthand > methods in EntityQuery to enable. We then made a change to GenericDAO.selectListIteratorByCondition > to add "FOR UPDATE" on the end of the SQL select statement - this allows > for DB row locks (we use postgres and works but have not tested other > databases). I believe there may have been a discussion about this before > > > forEach on EntityQuery > > Use Consumer in java and groovy to iterate over a query. This can reduce > memory consumption (replacement for queryList()) and boiler plate code (eg > queryIterator(), while loop and close) > > Example: > > EntityQuery.use(delegator).from("Foobar") > .forEach(item -> > Debug.logInfo(item.toString(), module); > ); > > A further update could be to provide stream capabilities > > Hope this helps, I can provide a patch aswell > > Gareth Carter > Software Development Analyst > Stannah Management Services Ltd > IT Department > Ext: > 7036 > DDI: > 01264 364311 > > > Please consider the environment before printing this email. > > -----Original Message----- > From: Arun Patidar [mailto:[hidden email]] > Sent: 25 October 2017 5:49 AM > To: [hidden email] > Cc: [hidden email] > Subject: Re: Updates to EntityQuery > > Hello Gareth, > > Please provide some more details or patch to understand - forUpdate and > forEach method utility. > > getFieldMap method looks good to me. > > > > -- > Thanks & Regards > --- > Arun Patidar > Manager, Enterprise Software Development > > HotWax Systems Pvt Ltd. > > www.hotwaxsystems.com > > > On Tue, Oct 24, 2017 at 9:06 PM, Gareth Carter < > [hidden email]> > wrote: > > > Hi all > > > > We have internally patched EntityQuery with some additional > > functionality and before I create a Jira was going to see what the > community thinks. > > > > New functionality: > > > > - New method getFieldMap which returns a Map object of selected > > fields from GenericValue objects, useful for creating cache map > > objects for lookup > > > > - Support forUpdate > > > > - forEach method to accept Consumer > > > > We have found these useful and believe the project can benefit, let me > > know what you think > > > > Regards > > > > Gareth Carter > > > > > > Software Development Analyst > > > > > > Stannah Management Services Ltd > > > > > > IT Department > > > > > > Ext: > > > > > > 7036 > > > > > > DDI: > > > > > > 01264 364311 > > > > > > > > > > [http://logos.stannah.co.uk/stan150.jpg] > > > > > > [http://logos.stannah.co.uk/enviro.jpg]Please consider the environment > > before printing this email. > > > > > > > > This email is intended only for the above addressee. It may contain > > privileged information. If you are not the addressee you must not > > copy, distribute, disclose or use any of the information in it. If you > > have received it in error, please delete it and notify the sender. > > > > Stannah Lift Holdings Ltd registered No. 686996, Stannah Management > > Services Ltd registered No. 2483693, Stannah Lift Services Ltd > > registered No. 1189799, Stannah Microlifts Ltd registered No. 964804, > > Stannah Lifts Ltd registered No. 1189836, Stannah Stairlifts Ltd > > registered No. 1401451, Global Upholstery Solutions Ltd registered No. > 02452728. > > > > All registered offices at Watt Close, East Portway, Andover, > > Hampshire, > > SP10 3SD, England. > > > > All Registered in England and Wales. > > > > This message has been scanned for malware by Websense. > > www.websense.com > > > > > To report this email as spam, please send the original message, complete > with headers to [hidden email] > |
|
|
I am all up for it, please let me know if I could help.
Thanks & Regards, -- *Pradhan Yash Sharma* *HotWax Systems* | www.hotwaxsystems.com On Wed, Oct 25, 2017 at 6:23 PM, Yash Sharma <[hidden email]> wrote: > Yes, Usage of the stream will surely enhance performance to a certain > extent and removes ceremony from the code base. I think Parallel streams > will add enhancements in many folds as we are using a multicore processor > (I have not tested yet), but the Functional approach seems promising. > > Thanks & Regards, > -- > *Pradhan Yash Sharma* > *HotWax Systems* | www.hotwaxsystems.com > > On Wed, Oct 25, 2017 at 5:59 PM, Gareth Carter < > [hidden email]> wrote: > >> forUpdate >> >> We patched EntityFindOptions with a new field "forUpdate" with shorthand >> methods in EntityQuery to enable. We then made a change to >> GenericDAO.selectListIteratorByCondition to add "FOR UPDATE" on the end >> of the SQL select statement - this allows for DB row locks (we use postgres >> and works but have not tested other databases). I believe there may have >> been a discussion about this before >> >> >> forEach on EntityQuery >> >> Use Consumer in java and groovy to iterate over a query. This can reduce >> memory consumption (replacement for queryList()) and boiler plate code (eg >> queryIterator(), while loop and close) >> >> Example: >> >> EntityQuery.use(delegator).from("Foobar") >> .forEach(item -> >> Debug.logInfo(item.toString(), module); >> ); >> >> A further update could be to provide stream capabilities >> >> Hope this helps, I can provide a patch aswell >> >> Gareth Carter >> Software Development Analyst >> Stannah Management Services Ltd >> IT Department >> Ext: >> 7036 >> DDI: >> 01264 364311 >> >> >> Please consider the environment before printing this email. >> >> -----Original Message----- >> From: Arun Patidar [mailto:[hidden email]] >> Sent: 25 October 2017 5:49 AM >> To: [hidden email] >> Cc: [hidden email] >> Subject: Re: Updates to EntityQuery >> >> Hello Gareth, >> >> Please provide some more details or patch to understand - forUpdate and >> forEach method utility. >> >> getFieldMap method looks good to me. >> >> >> >> -- >> Thanks & Regards >> --- >> Arun Patidar >> Manager, Enterprise Software Development >> >> HotWax Systems Pvt Ltd. >> >> www.hotwaxsystems.com >> >> >> On Tue, Oct 24, 2017 at 9:06 PM, Gareth Carter < >> [hidden email]> >> wrote: >> >> > Hi all >> > >> > We have internally patched EntityQuery with some additional >> > functionality and before I create a Jira was going to see what the >> community thinks. >> > >> > New functionality: >> > >> > - New method getFieldMap which returns a Map object of selected >> > fields from GenericValue objects, useful for creating cache map >> > objects for lookup >> > >> > - Support forUpdate >> > >> > - forEach method to accept Consumer >> > >> > We have found these useful and believe the project can benefit, let me >> > know what you think >> > >> > Regards >> > >> > Gareth Carter >> > >> > >> > Software Development Analyst >> > >> > >> > Stannah Management Services Ltd >> > >> > >> > IT Department >> > >> > >> > Ext: >> > >> > >> > 7036 >> > >> > >> > DDI: >> > >> > >> > 01264 364311 >> > >> > >> > >> > >> > [http://logos.stannah.co.uk/stan150.jpg] >> > >> > >> > [http://logos.stannah.co.uk/enviro.jpg]Please consider the environment >> > before printing this email. >> > >> > >> > >> > This email is intended only for the above addressee. It may contain >> > privileged information. If you are not the addressee you must not >> > copy, distribute, disclose or use any of the information in it. If you >> > have received it in error, please delete it and notify the sender. >> > >> > Stannah Lift Holdings Ltd registered No. 686996, Stannah Management >> > Services Ltd registered No. 2483693, Stannah Lift Services Ltd >> > registered No. 1189799, Stannah Microlifts Ltd registered No. 964804, >> > Stannah Lifts Ltd registered No. 1189836, Stannah Stairlifts Ltd >> > registered No. 1401451, Global Upholstery Solutions Ltd registered No. >> 02452728. >> > >> > All registered offices at Watt Close, East Portway, Andover, >> > Hampshire, >> > SP10 3SD, England. >> > >> > All Registered in England and Wales. >> > >> > This message has been scanned for malware by Websense. >> > www.websense.com >> > >> >> >> To report this email as spam, please send the original message, complete >> with headers to [hidden email] >> > > |
|
|
Looking good to me also. Let me know if I can be of any help.
Thanks & Regards -- Deepak Nigam On Wed, Oct 25, 2017 at 6:26 PM, Yash Sharma <[hidden email]> wrote: > I am all up for it, please let me know if I could help. > > Thanks & Regards, > -- > *Pradhan Yash Sharma* > *HotWax Systems* | www.hotwaxsystems.com > > On Wed, Oct 25, 2017 at 6:23 PM, Yash Sharma < > [hidden email]> > wrote: > > > Yes, Usage of the stream will surely enhance performance to a certain > > extent and removes ceremony from the code base. I think Parallel streams > > will add enhancements in many folds as we are using a multicore processor > > (I have not tested yet), but the Functional approach seems promising. > > > > Thanks & Regards, > > -- > > *Pradhan Yash Sharma* > > *HotWax Systems* | www.hotwaxsystems.com > > > > On Wed, Oct 25, 2017 at 5:59 PM, Gareth Carter < > > [hidden email]> wrote: > > > >> forUpdate > >> > >> We patched EntityFindOptions with a new field "forUpdate" with shorthand > >> methods in EntityQuery to enable. We then made a change to > >> GenericDAO.selectListIteratorByCondition to add "FOR UPDATE" on the end > >> of the SQL select statement - this allows for DB row locks (we use > postgres > >> and works but have not tested other databases). I believe there may have > >> been a discussion about this before > >> > >> > >> forEach on EntityQuery > >> > >> Use Consumer in java and groovy to iterate over a query. This can reduce > >> memory consumption (replacement for queryList()) and boiler plate code > (eg > >> queryIterator(), while loop and close) > >> > >> Example: > >> > >> EntityQuery.use(delegator).from("Foobar") > >> .forEach(item -> > >> Debug.logInfo(item.toString(), module); > >> ); > >> > >> A further update could be to provide stream capabilities > >> > >> Hope this helps, I can provide a patch aswell > >> > >> Gareth Carter > >> Software Development Analyst > >> Stannah Management Services Ltd > >> IT Department > >> Ext: > >> 7036 > >> DDI: > >> 01264 364311 > >> > >> > >> Please consider the environment before printing this email. > >> > >> -----Original Message----- > >> From: Arun Patidar [mailto:[hidden email]] > >> Sent: 25 October 2017 5:49 AM > >> To: [hidden email] > >> Cc: [hidden email] > >> Subject: Re: Updates to EntityQuery > >> > >> Hello Gareth, > >> > >> Please provide some more details or patch to understand - forUpdate and > >> forEach method utility. > >> > >> getFieldMap method looks good to me. > >> > >> > >> > >> -- > >> Thanks & Regards > >> --- > >> Arun Patidar > >> Manager, Enterprise Software Development > >> > >> HotWax Systems Pvt Ltd. > >> > >> www.hotwaxsystems.com > >> > >> > >> On Tue, Oct 24, 2017 at 9:06 PM, Gareth Carter < > >> [hidden email]> > >> wrote: > >> > >> > Hi all > >> > > >> > We have internally patched EntityQuery with some additional > >> > functionality and before I create a Jira was going to see what the > >> community thinks. > >> > > >> > New functionality: > >> > > >> > - New method getFieldMap which returns a Map object of selected > >> > fields from GenericValue objects, useful for creating cache map > >> > objects for lookup > >> > > >> > - Support forUpdate > >> > > >> > - forEach method to accept Consumer > >> > > >> > We have found these useful and believe the project can benefit, let me > >> > know what you think > >> > > >> > Regards > >> > > >> > Gareth Carter > >> > > >> > > >> > Software Development Analyst > >> > > >> > > >> > Stannah Management Services Ltd > >> > > >> > > >> > IT Department > >> > > >> > > >> > Ext: > >> > > >> > > >> > 7036 > >> > > >> > > >> > DDI: > >> > > >> > > >> > 01264 364311 > >> > > >> > > >> > > >> > > >> > [http://logos.stannah.co.uk/stan150.jpg] > >> > > >> > > >> > [http://logos.stannah.co.uk/enviro.jpg]Please consider the > environment > >> > before printing this email. > >> > > >> > > >> > > >> > This email is intended only for the above addressee. It may contain > >> > privileged information. If you are not the addressee you must not > >> > copy, distribute, disclose or use any of the information in it. If you > >> > have received it in error, please delete it and notify the sender. > >> > > >> > Stannah Lift Holdings Ltd registered No. 686996, Stannah Management > >> > Services Ltd registered No. 2483693, Stannah Lift Services Ltd > >> > registered No. 1189799, Stannah Microlifts Ltd registered No. 964804, > >> > Stannah Lifts Ltd registered No. 1189836, Stannah Stairlifts Ltd > >> > registered No. 1401451, Global Upholstery Solutions Ltd registered No. > >> 02452728. > >> > > >> > All registered offices at Watt Close, East Portway, Andover, > >> > Hampshire, > >> > SP10 3SD, England. > >> > > >> > All Registered in England and Wales. > >> > > >> > This message has been scanned for malware by Websense. > >> > www.websense.com > >> > > >> > >> > >> To report this email as spam, please send the original message, complete > >> with headers to [hidden email] > >> > > > > > |
|
|
Ok I will create a Jira with patch for review.
Sent from my Samsung Galaxy smartphone. Gareth Carter Software Development Analyst Stannah Management Services Ltd IT Department Ext: 7036 DDI: 01264 364311 [http://logos.stannah.co.uk/stan150.jpg] [http://logos.stannah.co.uk/enviro.jpg]Please consider the environment before printing this email. -------- Original message -------- From: deepak nigam <[hidden email]> Date: 25/10/2017 14:06 (GMT+00:00) To: [hidden email] Subject: Re: Updates to EntityQuery Looking good to me also. Let me know if I can be of any help. Thanks & Regards -- Deepak Nigam On Wed, Oct 25, 2017 at 6:26 PM, Yash Sharma <[hidden email]> wrote: > I am all up for it, please let me know if I could help. > > Thanks & Regards, > -- > *Pradhan Yash Sharma* > *HotWax Systems* | www.hotwaxsystems.com<http://www.hotwaxsystems.com> > > On Wed, Oct 25, 2017 at 6:23 PM, Yash Sharma < > [hidden email]> > wrote: > > > Yes, Usage of the stream will surely enhance performance to a certain > > extent and removes ceremony from the code base. I think Parallel streams > > will add enhancements in many folds as we are using a multicore processor > > (I have not tested yet), but the Functional approach seems promising. > > > > Thanks & Regards, > > -- > > *Pradhan Yash Sharma* > > *HotWax Systems* | www.hotwaxsystems.com<http://www.hotwaxsystems.com> > > > > On Wed, Oct 25, 2017 at 5:59 PM, Gareth Carter < > > [hidden email]> wrote: > > > >> forUpdate > >> > >> We patched EntityFindOptions with a new field "forUpdate" with shorthand > >> methods in EntityQuery to enable. We then made a change to > >> GenericDAO.selectListIteratorByCondition to add "FOR UPDATE" on the end > >> of the SQL select statement - this allows for DB row locks (we use > postgres > >> and works but have not tested other databases). I believe there may have > >> been a discussion about this before > >> > >> > >> forEach on EntityQuery > >> > >> Use Consumer in java and groovy to iterate over a query. This can reduce > >> memory consumption (replacement for queryList()) and boiler plate code > (eg > >> queryIterator(), while loop and close) > >> > >> Example: > >> > >> EntityQuery.use(delegator).from("Foobar") > >> .forEach(item -> > >> Debug.logInfo(item.toString(), module); > >> ); > >> > >> A further update could be to provide stream capabilities > >> > >> Hope this helps, I can provide a patch aswell > >> > >> Gareth Carter > >> Software Development Analyst > >> Stannah Management Services Ltd > >> IT Department > >> Ext: > >> 7036 > >> DDI: > >> 01264 364311 > >> > >> > >> Please consider the environment before printing this email. > >> > >> -----Original Message----- > >> From: Arun Patidar [mailto:[hidden email]] > >> Sent: 25 October 2017 5:49 AM > >> To: [hidden email] > >> Cc: [hidden email] > >> Subject: Re: Updates to EntityQuery > >> > >> Hello Gareth, > >> > >> Please provide some more details or patch to understand - forUpdate and > >> forEach method utility. > >> > >> getFieldMap method looks good to me. > >> > >> > >> > >> -- > >> Thanks & Regards > >> --- > >> Arun Patidar > >> Manager, Enterprise Software Development > >> > >> HotWax Systems Pvt Ltd. > >> > >> www.hotwaxsystems.com<http://www.hotwaxsystems.com> > >> > >> > >> On Tue, Oct 24, 2017 at 9:06 PM, Gareth Carter < > >> [hidden email]> > >> wrote: > >> > >> > Hi all > >> > > >> > We have internally patched EntityQuery with some additional > >> > functionality and before I create a Jira was going to see what the > >> community thinks. > >> > > >> > New functionality: > >> > > >> > - New method getFieldMap which returns a Map object of selected > >> > fields from GenericValue objects, useful for creating cache map > >> > objects for lookup > >> > > >> > - Support forUpdate > >> > > >> > - forEach method to accept Consumer > >> > > >> > We have found these useful and believe the project can benefit, let me > >> > know what you think > >> > > >> > Regards > >> > > >> > Gareth Carter > >> > > >> > > >> > Software Development Analyst > >> > > >> > > >> > Stannah Management Services Ltd > >> > > >> > > >> > IT Department > >> > > >> > > >> > Ext: > >> > > >> > > >> > 7036 > >> > > >> > > >> > DDI: > >> > > >> > > >> > 01264 364311 > >> > > >> > > >> > > >> > > >> > [http://logos.stannah.co.uk/stan150.jpg] > >> > > >> > > >> > [http://logos.stannah.co.uk/enviro.jpg]Please consider the > environment > >> > before printing this email. > >> > > >> > > >> > > >> > This email is intended only for the above addressee. It may contain > >> > privileged information. If you are not the addressee you must not > >> > copy, distribute, disclose or use any of the information in it. If you > >> > have received it in error, please delete it and notify the sender. > >> > > >> > Stannah Lift Holdings Ltd registered No. 686996, Stannah Management > >> > Services Ltd registered No. 2483693, Stannah Lift Services Ltd > >> > registered No. 1189799, Stannah Microlifts Ltd registered No. 964804, > >> > Stannah Lifts Ltd registered No. 1189836, Stannah Stairlifts Ltd > >> > registered No. 1401451, Global Upholstery Solutions Ltd registered No. > >> 02452728. > >> > > >> > All registered offices at Watt Close, East Portway, Andover, > >> > Hampshire, > >> > SP10 3SD, England. > >> > > >> > All Registered in England and Wales. > >> > > >> > This message has been scanned for malware by Websense. > >> > www.websense.com<http://www.websense.com> > >> > > >> > >> > >> To report this email as spam, please send the original message, complete > >> with headers to [hidden email] > >> > > > > > |
|
Hi Gareth,
FOR UPDATE is standard SQL, but even so SQLite for one doesn't support it: http://sqlite.1065341.n5.nabble.com/SELECT-FOR-UPDATE-td89630.html https://sqlite.org/lang_select.html FOR UPDATE only makes sense for relational databases. Adding assumptions it's available would make it harder to use a NoSQL database in future. In the case of SQLite, in effect it's got serialized isolation anyway, so the FOR UPDATE is not strictly necessary. I suggest OFBiz shouldn't have a method that generates syntactically incorrect SQL for any database we want to support, and we claim to support several. Where there are variations between SQL dialects, we have attributes in the data source, such as "join-style". So think our choices are: 1. Add an attribute to say "SELECT ... FOR UPDATE" is supported by a data source. OFBiz would only add the "FOR UPDATE" if it is supported. In the case of SQLite, I suspect we'd get away without it, but there may be other databases where the absence of "FOR UPDATE" might bring on unexpected behaviour. 2. Don't provide a method to expose FOR UPDATE. Could you achieve the results you want with a higher transaction isolation level like repeatable read, or with optimistic locking (enable-lock)? What do you think? Cheers Paul Foxworthy On 26 October 2017 at 07:50, Gareth Carter <[hidden email]> wrote: > Ok I will create a Jira with patch for review. > > > > Sent from my Samsung Galaxy smartphone. > > > > Gareth Carter > > > Software Development Analyst > > > Stannah Management Services Ltd > > > IT Department > > > Ext: > > > 7036 > > > DDI: > > > 01264 364311 > > > > > [http://logos.stannah.co.uk/stan150.jpg] > > > [http://logos.stannah.co.uk/enviro.jpg]Please consider the environment > before printing this email. > > > -------- Original message -------- > From: deepak nigam <[hidden email]> > Date: 25/10/2017 14:06 (GMT+00:00) > To: [hidden email] > Subject: Re: Updates to EntityQuery > > Looking good to me also. Let me know if I can be of any help. > > > Thanks & Regards > -- > Deepak Nigam > > On Wed, Oct 25, 2017 at 6:26 PM, Yash Sharma < > [hidden email]> > wrote: > > > I am all up for it, please let me know if I could help. > > > > Thanks & Regards, > > -- > > *Pradhan Yash Sharma* > > *HotWax Systems* | www.hotwaxsystems.com<http://www.hotwaxsystems.com> > > > > On Wed, Oct 25, 2017 at 6:23 PM, Yash Sharma < > > [hidden email]> > > wrote: > > > > > Yes, Usage of the stream will surely enhance performance to a certain > > > extent and removes ceremony from the code base. I think Parallel > streams > > > will add enhancements in many folds as we are using a multicore > processor > > > (I have not tested yet), but the Functional approach seems promising. > > > > > > Thanks & Regards, > > > -- > > > *Pradhan Yash Sharma* > > > *HotWax Systems* | www.hotwaxsystems.com<http://www.hotwaxsystems.com> > > > > > > On Wed, Oct 25, 2017 at 5:59 PM, Gareth Carter < > > > [hidden email]> wrote: > > > > > >> forUpdate > > >> > > >> We patched EntityFindOptions with a new field "forUpdate" with > shorthand > > >> methods in EntityQuery to enable. We then made a change to > > >> GenericDAO.selectListIteratorByCondition to add "FOR UPDATE" on the > end > > >> of the SQL select statement - this allows for DB row locks (we use > > postgres > > >> and works but have not tested other databases). I believe there may > have > > >> been a discussion about this before > > >> > > >> > > >> forEach on EntityQuery > > >> > > >> Use Consumer in java and groovy to iterate over a query. This can > reduce > > >> memory consumption (replacement for queryList()) and boiler plate code > > (eg > > >> queryIterator(), while loop and close) > > >> > > >> Example: > > >> > > >> EntityQuery.use(delegator).from("Foobar") > > >> .forEach(item -> > > >> Debug.logInfo(item.toString(), module); > > >> ); > > >> > > >> A further update could be to provide stream capabilities > > >> > > >> Hope this helps, I can provide a patch aswell > > >> > > >> Gareth Carter > > >> Software Development Analyst > > >> Stannah Management Services Ltd > > >> IT Department > > >> Ext: > > >> 7036 > > >> DDI: > > >> 01264 364311 > > >> > > >> > > >> Please consider the environment before printing this email. > > >> > > >> -----Original Message----- > > >> From: Arun Patidar [mailto:[hidden email]] > > >> Sent: 25 October 2017 5:49 AM > > >> To: [hidden email] > > >> Cc: [hidden email] > > >> Subject: Re: Updates to EntityQuery > > >> > > >> Hello Gareth, > > >> > > >> Please provide some more details or patch to understand - forUpdate > and > > >> forEach method utility. > > >> > > >> getFieldMap method looks good to me. > > >> > > >> > > >> > > >> -- > > >> Thanks & Regards > > >> --- > > >> Arun Patidar > > >> Manager, Enterprise Software Development > > >> > > >> HotWax Systems Pvt Ltd. > > >> > > >> www.hotwaxsystems.com<http://www.hotwaxsystems.com> > > >> > > >> > > >> On Tue, Oct 24, 2017 at 9:06 PM, Gareth Carter < > > >> [hidden email]> > > >> wrote: > > >> > > >> > Hi all > > >> > > > >> > We have internally patched EntityQuery with some additional > > >> > functionality and before I create a Jira was going to see what the > > >> community thinks. > > >> > > > >> > New functionality: > > >> > > > >> > - New method getFieldMap which returns a Map object of > selected > > >> > fields from GenericValue objects, useful for creating cache map > > >> > objects for lookup > > >> > > > >> > - Support forUpdate > > >> > > > >> > - forEach method to accept Consumer > > >> > > > >> > We have found these useful and believe the project can benefit, let > me > > >> > know what you think > > >> > > > >> > Regards > > >> > > > >> > Gareth Carter > > >> > > > >> > > > >> > Software Development Analyst > > >> > > > >> > > > >> > Stannah Management Services Ltd > > >> > > > >> > > > >> > IT Department > > >> > > > >> > > > >> > Ext: > > >> > > > >> > > > >> > 7036 > > >> > > > >> > > > >> > DDI: > > >> > > > >> > > > >> > 01264 364311 > > >> > > > >> > > > >> > > > >> > > > >> > [http://logos.stannah.co.uk/stan150.jpg] > > >> > > > >> > > > >> > [http://logos.stannah.co.uk/enviro.jpg]Please consider the > > environment > > >> > before printing this email. > > >> > > > >> > > > >> > > > >> > This email is intended only for the above addressee. It may contain > > >> > privileged information. If you are not the addressee you must not > > >> > copy, distribute, disclose or use any of the information in it. If > you > > >> > have received it in error, please delete it and notify the sender. > > >> > > > >> > Stannah Lift Holdings Ltd registered No. 686996, Stannah Management > > >> > Services Ltd registered No. 2483693, Stannah Lift Services Ltd > > >> > registered No. 1189799, Stannah Microlifts Ltd registered No. > 964804, > > >> > Stannah Lifts Ltd registered No. 1189836, Stannah Stairlifts Ltd > > >> > registered No. 1401451, Global Upholstery Solutions Ltd registered > No. > > >> 02452728. > > >> > > > >> > All registered offices at Watt Close, East Portway, Andover, > > >> > Hampshire, > > >> > SP10 3SD, England. > > >> > > > >> > All Registered in England and Wales. > > >> > > > >> > This message has been scanned for malware by Websense. > > >> > www.websense.com<http://www.websense.com> > > >> > > > >> > > >> > > >> To report this email as spam, please send the original message, > complete > > >> with headers to [hidden email] > > >> > > > > > > > > > -- Coherent Software Australia Pty Ltd PO Box 2773 Cheltenham Vic 3192 Australia Phone: +61 3 9585 6788 Web: http://www.coherentsoftware.com.au/ Email: [hidden email]
--
Coherent Software Australia Pty Ltd http://www.coherentsoftware.com.au/ Bonsai ERP, the all-inclusive ERP system http://www.bonsaierp.com.au/ |
|
|
I think it's better to have for-update available for use for the majority
of use-cases than it is to exclude it altogether. I'd rather lose the databases that don't support it than constrain the ones that do and are used most by pretty much everyone. It would be nice to hear from anyone using a database that doesn't support it since I'm just guessing most of us use postgres or mysql. Has anyone actually looked at how NoSQL databases might work with the OFBiz entity engine or are the concepts too far removed from SQL for it to even be workable? If we did want to support for-update globally, we could always have the delegator simulate it by issuing an UPDATE to lock the rows prior to performing the SELECT. Although it might not work perfectly in all cases, it could be a good compromise. e.g. UPDATE table SET lastUpdatedTxStamp = NOW() WHERE ...conditions -- lock the rows SELECT * FROM table WHERE ...conditions -- select the rows Regards Scott On 26 October 2017 at 13:45, Paul Foxworthy <[hidden email]> wrote: > Hi Gareth, > > FOR UPDATE is standard SQL, but even so SQLite for one doesn't support it: > > http://sqlite.1065341.n5.nabble.com/SELECT-FOR-UPDATE-td89630.html > https://sqlite.org/lang_select.html > > FOR UPDATE only makes sense for relational databases. Adding assumptions > it's available would make it harder to use a NoSQL database in future. > > In the case of SQLite, in effect it's got serialized isolation anyway, so > the FOR UPDATE is not strictly necessary. > > I suggest OFBiz shouldn't have a method that generates syntactically > incorrect SQL for any database we want to support, and we claim to support > several. Where there are variations between SQL dialects, we have > attributes in the data source, such as "join-style". > > So think our choices are: > > 1. Add an attribute to say "SELECT ... FOR UPDATE" is supported by a data > source. OFBiz would only add the "FOR UPDATE" if it is supported. In the > case of SQLite, I suspect we'd get away without it, but there may be other > databases where the absence of "FOR UPDATE" might bring on unexpected > behaviour. > > 2. Don't provide a method to expose FOR UPDATE. > > Could you achieve the results you want with a higher transaction isolation > level like repeatable read, or with optimistic locking (enable-lock)? > > What do you think? > > Cheers > > Paul Foxworthy > > > On 26 October 2017 at 07:50, Gareth Carter <[hidden email]> > wrote: > > > Ok I will create a Jira with patch for review. > > > > > > > > Sent from my Samsung Galaxy smartphone. > > > > > > > > Gareth Carter > > > > > > Software Development Analyst > > > > > > Stannah Management Services Ltd > > > > > > IT Department > > > > > > Ext: > > > > > > 7036 > > > > > > DDI: > > > > > > 01264 364311 > > > > > > > > > > [http://logos.stannah.co.uk/stan150.jpg] > > > > > > [http://logos.stannah.co.uk/enviro.jpg]Please consider the environment > > before printing this email. > > > > > > -------- Original message -------- > > From: deepak nigam <[hidden email]> > > Date: 25/10/2017 14:06 (GMT+00:00) > > To: [hidden email] > > Subject: Re: Updates to EntityQuery > > > > Looking good to me also. Let me know if I can be of any help. > > > > > > Thanks & Regards > > -- > > Deepak Nigam > > > > On Wed, Oct 25, 2017 at 6:26 PM, Yash Sharma < > > [hidden email]> > > wrote: > > > > > I am all up for it, please let me know if I could help. > > > > > > Thanks & Regards, > > > -- > > > *Pradhan Yash Sharma* > > > *HotWax Systems* | www.hotwaxsystems.com<http://www.hotwaxsystems.com> > > > > > > On Wed, Oct 25, 2017 at 6:23 PM, Yash Sharma < > > > [hidden email]> > > > wrote: > > > > > > > Yes, Usage of the stream will surely enhance performance to a certain > > > > extent and removes ceremony from the code base. I think Parallel > > streams > > > > will add enhancements in many folds as we are using a multicore > > processor > > > > (I have not tested yet), but the Functional approach seems promising. > > > > > > > > Thanks & Regards, > > > > -- > > > > *Pradhan Yash Sharma* > > > > *HotWax Systems* | www.hotwaxsystems.com<http:// > www.hotwaxsystems.com> > > > > > > > > On Wed, Oct 25, 2017 at 5:59 PM, Gareth Carter < > > > > [hidden email]> wrote: > > > > > > > >> forUpdate > > > >> > > > >> We patched EntityFindOptions with a new field "forUpdate" with > > shorthand > > > >> methods in EntityQuery to enable. We then made a change to > > > >> GenericDAO.selectListIteratorByCondition to add "FOR UPDATE" on the > > end > > > >> of the SQL select statement - this allows for DB row locks (we use > > > postgres > > > >> and works but have not tested other databases). I believe there may > > have > > > >> been a discussion about this before > > > >> > > > >> > > > >> forEach on EntityQuery > > > >> > > > >> Use Consumer in java and groovy to iterate over a query. This can > > reduce > > > >> memory consumption (replacement for queryList()) and boiler plate > code > > > (eg > > > >> queryIterator(), while loop and close) > > > >> > > > >> Example: > > > >> > > > >> EntityQuery.use(delegator).from("Foobar") > > > >> .forEach(item -> > > > >> Debug.logInfo(item.toString(), module); > > > >> ); > > > >> > > > >> A further update could be to provide stream capabilities > > > >> > > > >> Hope this helps, I can provide a patch aswell > > > >> > > > >> Gareth Carter > > > >> Software Development Analyst > > > >> Stannah Management Services Ltd > > > >> IT Department > > > >> Ext: > > > >> 7036 > > > >> DDI: > > > >> 01264 364311 > > > >> > > > >> > > > >> Please consider the environment before printing this email. > > > >> > > > >> -----Original Message----- > > > >> From: Arun Patidar [mailto:[hidden email]] > > > >> Sent: 25 October 2017 5:49 AM > > > >> To: [hidden email] > > > >> Cc: [hidden email] > > > >> Subject: Re: Updates to EntityQuery > > > >> > > > >> Hello Gareth, > > > >> > > > >> Please provide some more details or patch to understand - forUpdate > > and > > > >> forEach method utility. > > > >> > > > >> getFieldMap method looks good to me. > > > >> > > > >> > > > >> > > > >> -- > > > >> Thanks & Regards > > > >> --- > > > >> Arun Patidar > > > >> Manager, Enterprise Software Development > > > >> > > > >> HotWax Systems Pvt Ltd. > > > >> > > > >> www.hotwaxsystems.com<http://www.hotwaxsystems.com> > > > >> > > > >> > > > >> On Tue, Oct 24, 2017 at 9:06 PM, Gareth Carter < > > > >> [hidden email]> > > > >> wrote: > > > >> > > > >> > Hi all > > > >> > > > > >> > We have internally patched EntityQuery with some additional > > > >> > functionality and before I create a Jira was going to see what the > > > >> community thinks. > > > >> > > > > >> > New functionality: > > > >> > > > > >> > - New method getFieldMap which returns a Map object of > > selected > > > >> > fields from GenericValue objects, useful for creating cache map > > > >> > objects for lookup > > > >> > > > > >> > - Support forUpdate > > > >> > > > > >> > - forEach method to accept Consumer > > > >> > > > > >> > We have found these useful and believe the project can benefit, > let > > me > > > >> > know what you think > > > >> > > > > >> > Regards > > > >> > > > > >> > Gareth Carter > > > >> > > > > >> > > > > >> > Software Development Analyst > > > >> > > > > >> > > > > >> > Stannah Management Services Ltd > > > >> > > > > >> > > > > >> > IT Department > > > >> > > > > >> > > > > >> > Ext: > > > >> > > > > >> > > > > >> > 7036 > > > >> > > > > >> > > > > >> > DDI: > > > >> > > > > >> > > > > >> > 01264 364311 > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > [http://logos.stannah.co.uk/stan150.jpg] > > > >> > > > > >> > > > > >> > [http://logos.stannah.co.uk/enviro.jpg]Please consider the > > > environment > > > >> > before printing this email. > > > >> > > > > >> > > > > >> > > > > >> > This email is intended only for the above addressee. It may > contain > > > >> > privileged information. If you are not the addressee you must not > > > >> > copy, distribute, disclose or use any of the information in it. If > > you > > > >> > have received it in error, please delete it and notify the sender. > > > >> > > > > >> > Stannah Lift Holdings Ltd registered No. 686996, Stannah > Management > > > >> > Services Ltd registered No. 2483693, Stannah Lift Services Ltd > > > >> > registered No. 1189799, Stannah Microlifts Ltd registered No. > > 964804, > > > >> > Stannah Lifts Ltd registered No. 1189836, Stannah Stairlifts Ltd > > > >> > registered No. 1401451, Global Upholstery Solutions Ltd registered > > No. > > > >> 02452728. > > > >> > > > > >> > All registered offices at Watt Close, East Portway, Andover, > > > >> > Hampshire, > > > >> > SP10 3SD, England. > > > >> > > > > >> > All Registered in England and Wales. > > > >> > > > > >> > This message has been scanned for malware by Websense. > > > >> > www.websense.com<http://www.websense.com> > > > >> > > > > >> > > > >> > > > >> To report this email as spam, please send the original message, > > complete > > > >> with headers to [hidden email] > > > >> > > > > > > > > > > > > > > > > > -- > Coherent Software Australia Pty Ltd > PO Box 2773 > Cheltenham Vic 3192 > Australia > > Phone: +61 3 9585 6788 > Web: http://www.coherentsoftware.com.au/ > Email: [hidden email] > |
|
Hmmm usually my first gut reaction would be "keep things consistent",
however Scott made a good argument for supporting non-common DB features. So my suggestion would be to use something like the widget's <platform-specific ...> tag (or equivalent in code) for any features not supported in all DBMSs. By labeling it as such the user is aware that the DB selection is restricted. Now with regards to your comment about NoSQL, such databases are fundamentally different, and act more like a key-value based entities without a schema or the resulting ACID properties. It is possible to make it work with the entity engine, but then you'd lose the flexibility it gives you because you want to restrict yourself to the relational model that we apply in OFBiz. The idea of NoSQL is to think at a different conceptual level of your domain model. On Thu, Oct 26, 2017 at 4:33 AM, Scott Gray <[hidden email]> wrote: > I think it's better to have for-update available for use for the majority > of use-cases than it is to exclude it altogether. I'd rather lose the > databases that don't support it than constrain the ones that do and are > used most by pretty much everyone. It would be nice to hear from anyone > using a database that doesn't support it since I'm just guessing most of us > use postgres or mysql. > > Has anyone actually looked at how NoSQL databases might work with the OFBiz > entity engine or are the concepts too far removed from SQL for it to even > be workable? > > If we did want to support for-update globally, we could always have the > delegator simulate it by issuing an UPDATE to lock the rows prior to > performing the SELECT. Although it might not work perfectly in all cases, > it could be a good compromise. e.g. > UPDATE table SET lastUpdatedTxStamp = NOW() WHERE ...conditions -- lock the > rows > SELECT * FROM table WHERE ...conditions -- select the rows > > Regards > Scott > > > > On 26 October 2017 at 13:45, Paul Foxworthy <[hidden email]> wrote: > >> Hi Gareth, >> >> FOR UPDATE is standard SQL, but even so SQLite for one doesn't support it: >> >> http://sqlite.1065341.n5.nabble.com/SELECT-FOR-UPDATE-td89630.html >> https://sqlite.org/lang_select.html >> >> FOR UPDATE only makes sense for relational databases. Adding assumptions >> it's available would make it harder to use a NoSQL database in future. >> >> In the case of SQLite, in effect it's got serialized isolation anyway, so >> the FOR UPDATE is not strictly necessary. >> >> I suggest OFBiz shouldn't have a method that generates syntactically >> incorrect SQL for any database we want to support, and we claim to support >> several. Where there are variations between SQL dialects, we have >> attributes in the data source, such as "join-style". >> >> So think our choices are: >> >> 1. Add an attribute to say "SELECT ... FOR UPDATE" is supported by a data >> source. OFBiz would only add the "FOR UPDATE" if it is supported. In the >> case of SQLite, I suspect we'd get away without it, but there may be other >> databases where the absence of "FOR UPDATE" might bring on unexpected >> behaviour. >> >> 2. Don't provide a method to expose FOR UPDATE. >> >> Could you achieve the results you want with a higher transaction isolation >> level like repeatable read, or with optimistic locking (enable-lock)? >> >> What do you think? >> >> Cheers >> >> Paul Foxworthy >> >> >> On 26 October 2017 at 07:50, Gareth Carter <[hidden email]> >> wrote: >> >> > Ok I will create a Jira with patch for review. >> > >> > >> > >> > Sent from my Samsung Galaxy smartphone. >> > >> > >> > >> > Gareth Carter >> > >> > >> > Software Development Analyst >> > >> > >> > Stannah Management Services Ltd >> > >> > >> > IT Department >> > >> > >> > Ext: >> > >> > >> > 7036 >> > >> > >> > DDI: >> > >> > >> > 01264 364311 >> > >> > >> > >> > >> > [http://logos.stannah.co.uk/stan150.jpg] >> > >> > >> > [http://logos.stannah.co.uk/enviro.jpg]Please consider the environment >> > before printing this email. >> > >> > >> > -------- Original message -------- >> > From: deepak nigam <[hidden email]> >> > Date: 25/10/2017 14:06 (GMT+00:00) >> > To: [hidden email] >> > Subject: Re: Updates to EntityQuery >> > >> > Looking good to me also. Let me know if I can be of any help. >> > >> > >> > Thanks & Regards >> > -- >> > Deepak Nigam >> > >> > On Wed, Oct 25, 2017 at 6:26 PM, Yash Sharma < >> > [hidden email]> >> > wrote: >> > >> > > I am all up for it, please let me know if I could help. >> > > >> > > Thanks & Regards, >> > > -- >> > > *Pradhan Yash Sharma* >> > > *HotWax Systems* | www.hotwaxsystems.com<http://www.hotwaxsystems.com> >> > > >> > > On Wed, Oct 25, 2017 at 6:23 PM, Yash Sharma < >> > > [hidden email]> >> > > wrote: >> > > >> > > > Yes, Usage of the stream will surely enhance performance to a certain >> > > > extent and removes ceremony from the code base. I think Parallel >> > streams >> > > > will add enhancements in many folds as we are using a multicore >> > processor >> > > > (I have not tested yet), but the Functional approach seems promising. >> > > > >> > > > Thanks & Regards, >> > > > -- >> > > > *Pradhan Yash Sharma* >> > > > *HotWax Systems* | www.hotwaxsystems.com<http:// >> www.hotwaxsystems.com> >> > > > >> > > > On Wed, Oct 25, 2017 at 5:59 PM, Gareth Carter < >> > > > [hidden email]> wrote: >> > > > >> > > >> forUpdate >> > > >> >> > > >> We patched EntityFindOptions with a new field "forUpdate" with >> > shorthand >> > > >> methods in EntityQuery to enable. We then made a change to >> > > >> GenericDAO.selectListIteratorByCondition to add "FOR UPDATE" on the >> > end >> > > >> of the SQL select statement - this allows for DB row locks (we use >> > > postgres >> > > >> and works but have not tested other databases). I believe there may >> > have >> > > >> been a discussion about this before >> > > >> >> > > >> >> > > >> forEach on EntityQuery >> > > >> >> > > >> Use Consumer in java and groovy to iterate over a query. This can >> > reduce >> > > >> memory consumption (replacement for queryList()) and boiler plate >> code >> > > (eg >> > > >> queryIterator(), while loop and close) >> > > >> >> > > >> Example: >> > > >> >> > > >> EntityQuery.use(delegator).from("Foobar") >> > > >> .forEach(item -> >> > > >> Debug.logInfo(item.toString(), module); >> > > >> ); >> > > >> >> > > >> A further update could be to provide stream capabilities >> > > >> >> > > >> Hope this helps, I can provide a patch aswell >> > > >> >> > > >> Gareth Carter >> > > >> Software Development Analyst >> > > >> Stannah Management Services Ltd >> > > >> IT Department >> > > >> Ext: >> > > >> 7036 >> > > >> DDI: >> > > >> 01264 364311 >> > > >> >> > > >> >> > > >> Please consider the environment before printing this email. >> > > >> >> > > >> -----Original Message----- >> > > >> From: Arun Patidar [mailto:[hidden email]] >> > > >> Sent: 25 October 2017 5:49 AM >> > > >> To: [hidden email] >> > > >> Cc: [hidden email] >> > > >> Subject: Re: Updates to EntityQuery >> > > >> >> > > >> Hello Gareth, >> > > >> >> > > >> Please provide some more details or patch to understand - forUpdate >> > and >> > > >> forEach method utility. >> > > >> >> > > >> getFieldMap method looks good to me. >> > > >> >> > > >> >> > > >> >> > > >> -- >> > > >> Thanks & Regards >> > > >> --- >> > > >> Arun Patidar >> > > >> Manager, Enterprise Software Development >> > > >> >> > > >> HotWax Systems Pvt Ltd. >> > > >> >> > > >> www.hotwaxsystems.com<http://www.hotwaxsystems.com> >> > > >> >> > > >> >> > > >> On Tue, Oct 24, 2017 at 9:06 PM, Gareth Carter < >> > > >> [hidden email]> >> > > >> wrote: >> > > >> >> > > >> > Hi all >> > > >> > >> > > >> > We have internally patched EntityQuery with some additional >> > > >> > functionality and before I create a Jira was going to see what the >> > > >> community thinks. >> > > >> > >> > > >> > New functionality: >> > > >> > >> > > >> > - New method getFieldMap which returns a Map object of >> > selected >> > > >> > fields from GenericValue objects, useful for creating cache map >> > > >> > objects for lookup >> > > >> > >> > > >> > - Support forUpdate >> > > >> > >> > > >> > - forEach method to accept Consumer >> > > >> > >> > > >> > We have found these useful and believe the project can benefit, >> let >> > me >> > > >> > know what you think >> > > >> > >> > > >> > Regards >> > > >> > >> > > >> > Gareth Carter >> > > >> > >> > > >> > >> > > >> > Software Development Analyst >> > > >> > >> > > >> > >> > > >> > Stannah Management Services Ltd >> > > >> > >> > > >> > >> > > >> > IT Department >> > > >> > >> > > >> > >> > > >> > Ext: >> > > >> > >> > > >> > >> > > >> > 7036 >> > > >> > >> > > >> > >> > > >> > DDI: >> > > >> > >> > > >> > >> > > >> > 01264 364311 >> > > >> > >> > > >> > >> > > >> > >> > > >> > >> > > >> > [http://logos.stannah.co.uk/stan150.jpg] >> > > >> > >> > > >> > >> > > >> > [http://logos.stannah.co.uk/enviro.jpg]Please consider the >> > > environment >> > > >> > before printing this email. >> > > >> > >> > > >> > >> > > >> > >> > > >> > This email is intended only for the above addressee. It may >> contain >> > > >> > privileged information. If you are not the addressee you must not >> > > >> > copy, distribute, disclose or use any of the information in it. If >> > you >> > > >> > have received it in error, please delete it and notify the sender. >> > > >> > >> > > >> > Stannah Lift Holdings Ltd registered No. 686996, Stannah >> Management >> > > >> > Services Ltd registered No. 2483693, Stannah Lift Services Ltd >> > > >> > registered No. 1189799, Stannah Microlifts Ltd registered No. >> > 964804, >> > > >> > Stannah Lifts Ltd registered No. 1189836, Stannah Stairlifts Ltd >> > > >> > registered No. 1401451, Global Upholstery Solutions Ltd registered >> > No. >> > > >> 02452728. >> > > >> > >> > > >> > All registered offices at Watt Close, East Portway, Andover, >> > > >> > Hampshire, >> > > >> > SP10 3SD, England. >> > > >> > >> > > >> > All Registered in England and Wales. >> > > >> > >> > > >> > This message has been scanned for malware by Websense. >> > > >> > www.websense.com<http://www.websense.com> >> > > >> > >> > > >> >> > > >> >> > > >> To report this email as spam, please send the original message, >> > complete >> > > >> with headers to [hidden email] >> > > >> >> > > > >> > > > >> > > >> > >> >> >> >> -- >> Coherent Software Australia Pty Ltd >> PO Box 2773 >> Cheltenham Vic 3192 >> Australia >> >> Phone: +61 3 9585 6788 >> Web: http://www.coherentsoftware.com.au/ >> Email: [hidden email] >> |
|
|
HI all
Thank for the response It could be considered a hint, if the underlying db does not support it then don't use it. Adding an attribute on the datasource could be the way to go but I also believe we need methods to tell the system when to use it, you don't need it all the time only in selective cases. Until support is added I see ofbiz as purely RDMS. I don't believe ofbiz should restrict itself on the assumption it might support different databases such as nosql. There could be an alternative to forUpdate, anyone considered implementing a distributed locking mechanism? The whole point for using forUpdate is to prevent one or more code blocks changing the same database records concurrently, db row locks help but are not supported by all databases but an external locking mechanism may provide a generic way of dealing with this. OFBiz already provides a semaphore on services but I believe you need something at entity level Gareth Carter Software Development Analyst Stannah Management Services Ltd IT Department Ext: 7036 DDI: 01264 364311 Please consider the environment before printing this email. -----Original Message----- From: Taher Alkhateeb [mailto:[hidden email]] Sent: 26 October 2017 6:13 AM To: OFBIZ Development Mailing List <[hidden email]> Subject: Re: Updates to EntityQuery Hmmm usually my first gut reaction would be "keep things consistent", however Scott made a good argument for supporting non-common DB features. So my suggestion would be to use something like the widget's <platform-specific ...> tag (or equivalent in code) for any features not supported in all DBMSs. By labeling it as such the user is aware that the DB selection is restricted. Now with regards to your comment about NoSQL, such databases are fundamentally different, and act more like a key-value based entities without a schema or the resulting ACID properties. It is possible to make it work with the entity engine, but then you'd lose the flexibility it gives you because you want to restrict yourself to the relational model that we apply in OFBiz. The idea of NoSQL is to think at a different conceptual level of your domain model. On Thu, Oct 26, 2017 at 4:33 AM, Scott Gray <[hidden email]> wrote: > I think it's better to have for-update available for use for the > majority of use-cases than it is to exclude it altogether. I'd rather > lose the databases that don't support it than constrain the ones that > do and are used most by pretty much everyone. It would be nice to > hear from anyone using a database that doesn't support it since I'm > just guessing most of us use postgres or mysql. > > Has anyone actually looked at how NoSQL databases might work with the > OFBiz entity engine or are the concepts too far removed from SQL for > it to even be workable? > > If we did want to support for-update globally, we could always have > the delegator simulate it by issuing an UPDATE to lock the rows prior > to performing the SELECT. Although it might not work perfectly in all > cases, it could be a good compromise. e.g. > UPDATE table SET lastUpdatedTxStamp = NOW() WHERE ...conditions -- > lock the rows SELECT * FROM table WHERE ...conditions -- select the > rows > > Regards > Scott > > > > On 26 October 2017 at 13:45, Paul Foxworthy <[hidden email]> wrote: > >> Hi Gareth, >> >> FOR UPDATE is standard SQL, but even so SQLite for one doesn't support it: >> >> http://sqlite.1065341.n5.nabble.com/SELECT-FOR-UPDATE-td89630.html >> https://sqlite.org/lang_select.html >> >> FOR UPDATE only makes sense for relational databases. Adding >> assumptions it's available would make it harder to use a NoSQL database in future. >> >> In the case of SQLite, in effect it's got serialized isolation >> anyway, so the FOR UPDATE is not strictly necessary. >> >> I suggest OFBiz shouldn't have a method that generates syntactically >> incorrect SQL for any database we want to support, and we claim to >> support several. Where there are variations between SQL dialects, we >> have attributes in the data source, such as "join-style". >> >> So think our choices are: >> >> 1. Add an attribute to say "SELECT ... FOR UPDATE" is supported by a >> data source. OFBiz would only add the "FOR UPDATE" if it is >> supported. In the case of SQLite, I suspect we'd get away without it, >> but there may be other databases where the absence of "FOR UPDATE" >> might bring on unexpected behaviour. >> >> 2. Don't provide a method to expose FOR UPDATE. >> >> Could you achieve the results you want with a higher transaction >> isolation level like repeatable read, or with optimistic locking (enable-lock)? >> >> What do you think? >> >> Cheers >> >> Paul Foxworthy >> >> >> On 26 October 2017 at 07:50, Gareth Carter >> <[hidden email]> >> wrote: >> >> > Ok I will create a Jira with patch for review. >> > >> > >> > >> > Sent from my Samsung Galaxy smartphone. >> > >> > >> > >> > Gareth Carter >> > >> > >> > Software Development Analyst >> > >> > >> > Stannah Management Services Ltd >> > >> > >> > IT Department >> > >> > >> > Ext: >> > >> > >> > 7036 >> > >> > >> > DDI: >> > >> > >> > 01264 364311 >> > >> > >> > >> > >> > [http://logos.stannah.co.uk/stan150.jpg] >> > >> > >> > [http://logos.stannah.co.uk/enviro.jpg]Please consider the >> > environment before printing this email. >> > >> > >> > -------- Original message -------- >> > From: deepak nigam <[hidden email]> >> > Date: 25/10/2017 14:06 (GMT+00:00) >> > To: [hidden email] >> > Subject: Re: Updates to EntityQuery >> > >> > Looking good to me also. Let me know if I can be of any help. >> > >> > >> > Thanks & Regards >> > -- >> > Deepak Nigam >> > >> > On Wed, Oct 25, 2017 at 6:26 PM, Yash Sharma < >> > [hidden email]> >> > wrote: >> > >> > > I am all up for it, please let me know if I could help. >> > > >> > > Thanks & Regards, >> > > -- >> > > *Pradhan Yash Sharma* >> > > *HotWax Systems* | >> > > www.hotwaxsystems.com<http://www.hotwaxsystems.com> >> > > >> > > On Wed, Oct 25, 2017 at 6:23 PM, Yash Sharma < >> > > [hidden email]> >> > > wrote: >> > > >> > > > Yes, Usage of the stream will surely enhance performance to a >> > > > certain extent and removes ceremony from the code base. I think >> > > > Parallel >> > streams >> > > > will add enhancements in many folds as we are using a multicore >> > processor >> > > > (I have not tested yet), but the Functional approach seems promising. >> > > > >> > > > Thanks & Regards, >> > > > -- >> > > > *Pradhan Yash Sharma* >> > > > *HotWax Systems* | www.hotwaxsystems.com<http:// >> www.hotwaxsystems.com> >> > > > >> > > > On Wed, Oct 25, 2017 at 5:59 PM, Gareth Carter < >> > > > [hidden email]> wrote: >> > > > >> > > >> forUpdate >> > > >> >> > > >> We patched EntityFindOptions with a new field "forUpdate" with >> > shorthand >> > > >> methods in EntityQuery to enable. We then made a change to >> > > >> GenericDAO.selectListIteratorByCondition to add "FOR UPDATE" >> > > >> on the >> > end >> > > >> of the SQL select statement - this allows for DB row locks (we >> > > >> use >> > > postgres >> > > >> and works but have not tested other databases). I believe >> > > >> there may >> > have >> > > >> been a discussion about this before >> > > >> >> > > >> >> > > >> forEach on EntityQuery >> > > >> >> > > >> Use Consumer in java and groovy to iterate over a query. This >> > > >> can >> > reduce >> > > >> memory consumption (replacement for queryList()) and boiler >> > > >> plate >> code >> > > (eg >> > > >> queryIterator(), while loop and close) >> > > >> >> > > >> Example: >> > > >> >> > > >> EntityQuery.use(delegator).from("Foobar") >> > > >> .forEach(item -> >> > > >> Debug.logInfo(item.toString(), module); ); >> > > >> >> > > >> A further update could be to provide stream capabilities >> > > >> >> > > >> Hope this helps, I can provide a patch aswell >> > > >> >> > > >> Gareth Carter >> > > >> Software Development Analyst >> > > >> Stannah Management Services Ltd IT Department >> > > >> Ext: >> > > >> 7036 >> > > >> DDI: >> > > >> 01264 364311 >> > > >> >> > > >> >> > > >> Please consider the environment before printing this email. >> > > >> >> > > >> -----Original Message----- >> > > >> From: Arun Patidar [mailto:[hidden email]] >> > > >> Sent: 25 October 2017 5:49 AM >> > > >> To: [hidden email] >> > > >> Cc: [hidden email] >> > > >> Subject: Re: Updates to EntityQuery >> > > >> >> > > >> Hello Gareth, >> > > >> >> > > >> Please provide some more details or patch to understand - forUpdate >> > and >> > > >> forEach method utility. >> > > >> >> > > >> getFieldMap method looks good to me. >> > > >> >> > > >> >> > > >> >> > > >> -- >> > > >> Thanks & Regards >> > > >> --- >> > > >> Arun Patidar >> > > >> Manager, Enterprise Software Development >> > > >> >> > > >> HotWax Systems Pvt Ltd. >> > > >> >> > > >> www.hotwaxsystems.com<http://www.hotwaxsystems.com> >> > > >> >> > > >> >> > > >> On Tue, Oct 24, 2017 at 9:06 PM, Gareth Carter < >> > > >> [hidden email]> >> > > >> wrote: >> > > >> >> > > >> > Hi all >> > > >> > >> > > >> > We have internally patched EntityQuery with some additional >> > > >> > functionality and before I create a Jira was going to see what the >> > > >> community thinks. >> > > >> > >> > > >> > New functionality: >> > > >> > >> > > >> > - New method getFieldMap which returns a Map object of >> > selected >> > > >> > fields from GenericValue objects, useful for creating cache map >> > > >> > objects for lookup >> > > >> > >> > > >> > - Support forUpdate >> > > >> > >> > > >> > - forEach method to accept Consumer >> > > >> > >> > > >> > We have found these useful and believe the project can benefit, >> let >> > me >> > > >> > know what you think >> > > >> > >> > > >> > Regards >> > > >> > >> > > >> > Gareth Carter >> > > >> > >> > > >> > >> > > >> > Software Development Analyst >> > > >> > >> > > >> > >> > > >> > Stannah Management Services Ltd >> > > >> > >> > > >> > >> > > >> > IT Department >> > > >> > >> > > >> > >> > > >> > Ext: >> > > >> > >> > > >> > >> > > >> > 7036 >> > > >> > >> > > >> > >> > > >> > DDI: >> > > >> > >> > > >> > >> > > >> > 01264 364311 >> > > >> > >> > > >> > >> > > >> > >> > > >> > >> > > >> > [http://logos.stannah.co.uk/stan150.jpg] >> > > >> > >> > > >> > >> > > >> > [http://logos.stannah.co.uk/enviro.jpg]Please consider the >> > > environment >> > > >> > before printing this email. >> > > >> > >> > > >> > >> > > >> > >> > > >> > This email is intended only for the above addressee. It may >> contain >> > > >> > privileged information. If you are not the addressee you must not >> > > >> > copy, distribute, disclose or use any of the information in it. If >> > you >> > > >> > have received it in error, please delete it and notify the sender. >> > > >> > >> > > >> > Stannah Lift Holdings Ltd registered No. 686996, Stannah >> Management >> > > >> > Services Ltd registered No. 2483693, Stannah Lift Services Ltd >> > > >> > registered No. 1189799, Stannah Microlifts Ltd registered No. >> > 964804, >> > > >> > Stannah Lifts Ltd registered No. 1189836, Stannah Stairlifts Ltd >> > > >> > registered No. 1401451, Global Upholstery Solutions Ltd registered >> > No. >> > > >> 02452728. >> > > >> > >> > > >> > All registered offices at Watt Close, East Portway, Andover, >> > > >> > Hampshire, >> > > >> > SP10 3SD, England. >> > > >> > >> > > >> > All Registered in England and Wales. >> > > >> > >> > > >> > This message has been scanned for malware by Websense. >> > > >> > www.websense.com<http://www.websense.com> >> > > >> > >> > > >> >> > > >> >> > > >> To report this email as spam, please send the original message, >> > complete >> > > >> with headers to [hidden email] >> > > >> >> > > > >> > > > >> > > >> > >> >> >> >> -- >> Coherent Software Australia Pty Ltd >> PO Box 2773 >> Cheltenham Vic 3192 >> Australia >> >> Phone: +61 3 9585 6788 >> Web: http://webdefence.global.blackspider.com/urlwrap/?q=AXicFYxNCsIwEEa_E02K-NPqRlDs2t5gKJGIaVIm0wYPVnDr0rt4COP28d5Di8MHWN6A-OeqZkoy08B338egEj31ccDcXDfn7tJV63q3bdCyWHV0YlErx6QcArsi0vSAUx33xuScC3BWbNAUb5pL8l8RTwbA9wX8ANMdKWg&Z >> Email: [hidden email] >> |
Administrator
|
I agree with Scott and Taher,
About your question Gareth, we once had a hard coded "for update" somewhere in OFBiz code (in SequenceUtil.SequenceBank.fillBank() ). I put it in. Then Jacopo crossed an issue in a cluster and rightly changed it to something better See https://issues.apache.org/jira/browse/OFBIZ-2353 for more where Adrian gave some directions also. Since I did not read all in the thread and there is no patch yet I don't know if something alike applies to EntityQuery API. Putting a for update is certainly much simpler, so just saying for now. Jacques Le 26/10/2017 à 10:46, Gareth Carter a écrit : > HI all > > Thank for the response > > It could be considered a hint, if the underlying db does not support it then don't use it. Adding an attribute on the datasource could be the way to go but I also believe we need methods to tell the system when to use it, you don't need it all the time only in selective cases. > > Until support is added I see ofbiz as purely RDMS. I don't believe ofbiz should restrict itself on the assumption it might support different databases such as nosql. > > There could be an alternative to forUpdate, anyone considered implementing a distributed locking mechanism? The whole point for using forUpdate is to prevent one or more code blocks changing the same database records concurrently, db row locks help but are not supported by all databases but an external locking mechanism may provide a generic way of dealing with this. OFBiz already provides a semaphore on services but I believe you need something at entity level > Gareth Carter > Software Development Analyst > Stannah Management Services Ltd > IT Department > Ext: > 7036 > DDI: > 01264 364311 > > > Please consider the environment before printing this email. > > -----Original Message----- > From: Taher Alkhateeb [mailto:[hidden email]] > Sent: 26 October 2017 6:13 AM > To: OFBIZ Development Mailing List <[hidden email]> > Subject: Re: Updates to EntityQuery > > Hmmm usually my first gut reaction would be "keep things consistent", however Scott made a good argument for supporting non-common DB features. So my suggestion would be to use something like the widget's <platform-specific ...> tag (or equivalent in code) for any features not supported in all DBMSs. By labeling it as such the user is aware that the DB selection is restricted. > > Now with regards to your comment about NoSQL, such databases are fundamentally different, and act more like a key-value based entities without a schema or the resulting ACID properties. It is possible to make it work with the entity engine, but then you'd lose the flexibility it gives you because you want to restrict yourself to the relational model that we apply in OFBiz. The idea of NoSQL is to think at a different conceptual level of your domain model. > > On Thu, Oct 26, 2017 at 4:33 AM, Scott Gray <[hidden email]> wrote: >> I think it's better to have for-update available for use for the >> majority of use-cases than it is to exclude it altogether. I'd rather >> lose the databases that don't support it than constrain the ones that >> do and are used most by pretty much everyone. It would be nice to >> hear from anyone using a database that doesn't support it since I'm >> just guessing most of us use postgres or mysql. >> >> Has anyone actually looked at how NoSQL databases might work with the >> OFBiz entity engine or are the concepts too far removed from SQL for >> it to even be workable? >> >> If we did want to support for-update globally, we could always have >> the delegator simulate it by issuing an UPDATE to lock the rows prior >> to performing the SELECT. Although it might not work perfectly in all >> cases, it could be a good compromise. e.g. >> UPDATE table SET lastUpdatedTxStamp = NOW() WHERE ...conditions -- >> lock the rows SELECT * FROM table WHERE ...conditions -- select the >> rows >> >> Regards >> Scott >> >> >> >> On 26 October 2017 at 13:45, Paul Foxworthy <[hidden email]> wrote: >> >>> Hi Gareth, >>> >>> FOR UPDATE is standard SQL, but even so SQLite for one doesn't support it: >>> >>> http://sqlite.1065341.n5.nabble.com/SELECT-FOR-UPDATE-td89630.html >>> https://sqlite.org/lang_select.html >>> >>> FOR UPDATE only makes sense for relational databases. Adding >>> assumptions it's available would make it harder to use a NoSQL database in future. >>> >>> In the case of SQLite, in effect it's got serialized isolation >>> anyway, so the FOR UPDATE is not strictly necessary. >>> >>> I suggest OFBiz shouldn't have a method that generates syntactically >>> incorrect SQL for any database we want to support, and we claim to >>> support several. Where there are variations between SQL dialects, we >>> have attributes in the data source, such as "join-style". >>> >>> So think our choices are: >>> >>> 1. Add an attribute to say "SELECT ... FOR UPDATE" is supported by a >>> data source. OFBiz would only add the "FOR UPDATE" if it is >>> supported. In the case of SQLite, I suspect we'd get away without it, >>> but there may be other databases where the absence of "FOR UPDATE" >>> might bring on unexpected behaviour. >>> >>> 2. Don't provide a method to expose FOR UPDATE. >>> >>> Could you achieve the results you want with a higher transaction >>> isolation level like repeatable read, or with optimistic locking (enable-lock)? >>> >>> What do you think? >>> >>> Cheers >>> >>> Paul Foxworthy >>> >>> >>> On 26 October 2017 at 07:50, Gareth Carter >>> <[hidden email]> >>> wrote: >>> >>>> Ok I will create a Jira with patch for review. >>>> >>>> >>>> >>>> Sent from my Samsung Galaxy smartphone. >>>> >>>> >>>> >>>> Gareth Carter >>>> >>>> >>>> Software Development Analyst >>>> >>>> >>>> Stannah Management Services Ltd >>>> >>>> >>>> IT Department >>>> >>>> >>>> Ext: >>>> >>>> >>>> 7036 >>>> >>>> >>>> DDI: >>>> >>>> >>>> 01264 364311 >>>> >>>> >>>> >>>> >>>> [http://logos.stannah.co.uk/stan150.jpg] >>>> >>>> >>>> [http://logos.stannah.co.uk/enviro.jpg]Please consider the >>>> environment before printing this email. >>>> >>>> >>>> -------- Original message -------- >>>> From: deepak nigam <[hidden email]> >>>> Date: 25/10/2017 14:06 (GMT+00:00) >>>> To: [hidden email] >>>> Subject: Re: Updates to EntityQuery >>>> >>>> Looking good to me also. Let me know if I can be of any help. >>>> >>>> >>>> Thanks & Regards >>>> -- >>>> Deepak Nigam >>>> >>>> On Wed, Oct 25, 2017 at 6:26 PM, Yash Sharma < >>>> [hidden email]> >>>> wrote: >>>> >>>>> I am all up for it, please let me know if I could help. >>>>> >>>>> Thanks & Regards, >>>>> -- >>>>> *Pradhan Yash Sharma* >>>>> *HotWax Systems* | >>>>> www.hotwaxsystems.com<http://www.hotwaxsystems.com> >>>>> >>>>> On Wed, Oct 25, 2017 at 6:23 PM, Yash Sharma < >>>>> [hidden email]> >>>>> wrote: >>>>> >>>>>> Yes, Usage of the stream will surely enhance performance to a >>>>>> certain extent and removes ceremony from the code base. I think >>>>>> Parallel >>>> streams >>>>>> will add enhancements in many folds as we are using a multicore >>>> processor >>>>>> (I have not tested yet), but the Functional approach seems promising. >>>>>> >>>>>> Thanks & Regards, >>>>>> -- >>>>>> *Pradhan Yash Sharma* >>>>>> *HotWax Systems* | www.hotwaxsystems.com<http:// >>> www.hotwaxsystems.com> >>>>>> On Wed, Oct 25, 2017 at 5:59 PM, Gareth Carter < >>>>>> [hidden email]> wrote: >>>>>> >>>>>>> forUpdate >>>>>>> >>>>>>> We patched EntityFindOptions with a new field "forUpdate" with >>>> shorthand >>>>>>> methods in EntityQuery to enable. We then made a change to >>>>>>> GenericDAO.selectListIteratorByCondition to add "FOR UPDATE" >>>>>>> on the >>>> end >>>>>>> of the SQL select statement - this allows for DB row locks (we >>>>>>> use >>>>> postgres >>>>>>> and works but have not tested other databases). I believe >>>>>>> there may >>>> have >>>>>>> been a discussion about this before >>>>>>> >>>>>>> >>>>>>> forEach on EntityQuery >>>>>>> >>>>>>> Use Consumer in java and groovy to iterate over a query. This >>>>>>> can >>>> reduce >>>>>>> memory consumption (replacement for queryList()) and boiler >>>>>>> plate >>> code >>>>> (eg >>>>>>> queryIterator(), while loop and close) >>>>>>> >>>>>>> Example: >>>>>>> >>>>>>> EntityQuery.use(delegator).from("Foobar") >>>>>>> .forEach(item -> >>>>>>> Debug.logInfo(item.toString(), module); ); >>>>>>> >>>>>>> A further update could be to provide stream capabilities >>>>>>> >>>>>>> Hope this helps, I can provide a patch aswell >>>>>>> >>>>>>> Gareth Carter >>>>>>> Software Development Analyst >>>>>>> Stannah Management Services Ltd IT Department >>>>>>> Ext: >>>>>>> 7036 >>>>>>> DDI: >>>>>>> 01264 364311 >>>>>>> >>>>>>> >>>>>>> Please consider the environment before printing this email. >>>>>>> >>>>>>> -----Original Message----- >>>>>>> From: Arun Patidar [mailto:[hidden email]] >>>>>>> Sent: 25 October 2017 5:49 AM >>>>>>> To: [hidden email] >>>>>>> Cc: [hidden email] >>>>>>> Subject: Re: Updates to EntityQuery >>>>>>> >>>>>>> Hello Gareth, >>>>>>> >>>>>>> Please provide some more details or patch to understand - forUpdate >>>> and >>>>>>> forEach method utility. >>>>>>> >>>>>>> getFieldMap method looks good to me. >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Thanks & Regards >>>>>>> --- >>>>>>> Arun Patidar >>>>>>> Manager, Enterprise Software Development >>>>>>> >>>>>>> HotWax Systems Pvt Ltd. >>>>>>> >>>>>>> www.hotwaxsystems.com<http://www.hotwaxsystems.com> >>>>>>> >>>>>>> >>>>>>> On Tue, Oct 24, 2017 at 9:06 PM, Gareth Carter < >>>>>>> [hidden email]> >>>>>>> wrote: >>>>>>> >>>>>>>> Hi all >>>>>>>> >>>>>>>> We have internally patched EntityQuery with some additional >>>>>>>> functionality and before I create a Jira was going to see what the >>>>>>> community thinks. >>>>>>>> New functionality: >>>>>>>> >>>>>>>> - New method getFieldMap which returns a Map object of >>>> selected >>>>>>>> fields from GenericValue objects, useful for creating cache map >>>>>>>> objects for lookup >>>>>>>> >>>>>>>> - Support forUpdate >>>>>>>> >>>>>>>> - forEach method to accept Consumer >>>>>>>> >>>>>>>> We have found these useful and believe the project can benefit, >>> let >>>> me >>>>>>>> know what you think >>>>>>>> >>>>>>>> Regards >>>>>>>> >>>>>>>> Gareth Carter >>>>>>>> >>>>>>>> >>>>>>>> Software Development Analyst >>>>>>>> >>>>>>>> >>>>>>>> Stannah Management Services Ltd >>>>>>>> >>>>>>>> >>>>>>>> IT Department >>>>>>>> >>>>>>>> >>>>>>>> Ext: >>>>>>>> >>>>>>>> >>>>>>>> 7036 >>>>>>>> >>>>>>>> >>>>>>>> DDI: >>>>>>>> >>>>>>>> >>>>>>>> 01264 364311 >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> [http://logos.stannah.co.uk/stan150.jpg] >>>>>>>> >>>>>>>> >>>>>>>> [http://logos.stannah.co.uk/enviro.jpg]Please consider the >>>>> environment >>>>>>>> before printing this email. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> This email is intended only for the above addressee. It may >>> contain >>>>>>>> privileged information. If you are not the addressee you must not >>>>>>>> copy, distribute, disclose or use any of the information in it. If >>>> you >>>>>>>> have received it in error, please delete it and notify the sender. >>>>>>>> >>>>>>>> Stannah Lift Holdings Ltd registered No. 686996, Stannah >>> Management >>>>>>>> Services Ltd registered No. 2483693, Stannah Lift Services Ltd >>>>>>>> registered No. 1189799, Stannah Microlifts Ltd registered No. >>>> 964804, >>>>>>>> Stannah Lifts Ltd registered No. 1189836, Stannah Stairlifts Ltd >>>>>>>> registered No. 1401451, Global Upholstery Solutions Ltd registered >>>> No. >>>>>>> 02452728. >>>>>>>> All registered offices at Watt Close, East Portway, Andover, >>>>>>>> Hampshire, >>>>>>>> SP10 3SD, England. >>>>>>>> >>>>>>>> All Registered in England and Wales. >>>>>>>> >>>>>>>> This message has been scanned for malware by Websense. >>>>>>>> www.websense.com<http://www.websense.com> >>>>>>>> >>>>>>> >>>>>>> To report this email as spam, please send the original message, >>>> complete >>>>>>> with headers to [hidden email] >>>>>>> >>>>>> >>> >>> >>> -- >>> Coherent Software Australia Pty Ltd >>> PO Box 2773 >>> Cheltenham Vic 3192 >>> Australia >>> >>> Phone: +61 3 9585 6788 >>> Web: http://webdefence.global.blackspider.com/urlwrap/?q=AXicFYxNCsIwEEa_E02K-NPqRlDs2t5gKJGIaVIm0wYPVnDr0rt4COP28d5Di8MHWN6A-OeqZkoy08B338egEj31ccDcXDfn7tJV63q3bdCyWHV0YlErx6QcArsi0vSAUx33xuScC3BWbNAUb5pL8l8RTwbA9wX8ANMdKWg&Z >>> Email: [hidden email] >>> |
|
|
Hi all,
As far as I know, FOR UPDATE would only break SQLite, and I can live with that. Gareth, would optimistic instead of pessimistic work for your situation? The database does less of the work and it's more scalable. I haven't tested it out, but OFBiz provides optimistic locking if you turn on enable-lock. Cheers Paul Foxworthy On 27 October 2017 at 03:49, Jacques Le Roux <[hidden email]> wrote: > I agree with Scott and Taher, > > About your question Gareth, we once had a hard coded "for update" > somewhere in OFBiz code (in SequenceUtil.SequenceBank.fillBank() ). > I put it in. Then Jacopo crossed an issue in a cluster and rightly changed > it to something better > See https://issues.apache.org/jira/browse/OFBIZ-2353 for more where > Adrian gave some directions also. > > Since I did not read all in the thread and there is no patch yet I don't > know if something alike applies to EntityQuery API. Putting a for update is > certainly much simpler, so just saying for now. > > Jacques > > > > Le 26/10/2017 à 10:46, Gareth Carter a écrit : > >> HI all >> >> Thank for the response >> >> It could be considered a hint, if the underlying db does not support it >> then don't use it. Adding an attribute on the datasource could be the way >> to go but I also believe we need methods to tell the system when to use it, >> you don't need it all the time only in selective cases. >> >> Until support is added I see ofbiz as purely RDMS. I don't believe ofbiz >> should restrict itself on the assumption it might support different >> databases such as nosql. >> >> There could be an alternative to forUpdate, anyone considered >> implementing a distributed locking mechanism? The whole point for using >> forUpdate is to prevent one or more code blocks changing the same database >> records concurrently, db row locks help but are not supported by all >> databases but an external locking mechanism may provide a generic way of >> dealing with this. OFBiz already provides a semaphore on services but I >> believe you need something at entity level >> Gareth Carter >> Software Development Analyst >> Stannah Management Services Ltd >> IT Department >> Ext: >> 7036 >> DDI: >> 01264 364311 >> >> >> Please consider the environment before printing this email. >> >> -----Original Message----- >> From: Taher Alkhateeb [mailto:[hidden email]] >> Sent: 26 October 2017 6:13 AM >> To: OFBIZ Development Mailing List <[hidden email]> >> Subject: Re: Updates to EntityQuery >> >> Hmmm usually my first gut reaction would be "keep things consistent", >> however Scott made a good argument for supporting non-common DB features. >> So my suggestion would be to use something like the widget's >> <platform-specific ...> tag (or equivalent in code) for any features not >> supported in all DBMSs. By labeling it as such the user is aware that the >> DB selection is restricted. >> >> Now with regards to your comment about NoSQL, such databases are >> fundamentally different, and act more like a key-value based entities >> without a schema or the resulting ACID properties. It is possible to make >> it work with the entity engine, but then you'd lose the flexibility it >> gives you because you want to restrict yourself to the relational model >> that we apply in OFBiz. The idea of NoSQL is to think at a different >> conceptual level of your domain model. >> >> On Thu, Oct 26, 2017 at 4:33 AM, Scott Gray <[hidden email]> >> wrote: >> >>> I think it's better to have for-update available for use for the >>> majority of use-cases than it is to exclude it altogether. I'd rather >>> lose the databases that don't support it than constrain the ones that >>> do and are used most by pretty much everyone. It would be nice to >>> hear from anyone using a database that doesn't support it since I'm >>> just guessing most of us use postgres or mysql. >>> >>> Has anyone actually looked at how NoSQL databases might work with the >>> OFBiz entity engine or are the concepts too far removed from SQL for >>> it to even be workable? >>> >>> If we did want to support for-update globally, we could always have >>> the delegator simulate it by issuing an UPDATE to lock the rows prior >>> to performing the SELECT. Although it might not work perfectly in all >>> cases, it could be a good compromise. e.g. >>> UPDATE table SET lastUpdatedTxStamp = NOW() WHERE ...conditions -- >>> lock the rows SELECT * FROM table WHERE ...conditions -- select the >>> rows >>> >>> Regards >>> Scott >>> >>> >>> >>> On 26 October 2017 at 13:45, Paul Foxworthy <[hidden email]> wrote: >>> >>> Hi Gareth, >>>> >>>> FOR UPDATE is standard SQL, but even so SQLite for one doesn't support >>>> it: >>>> >>>> http://sqlite.1065341.n5.nabble.com/SELECT-FOR-UPDATE-td89630.html >>>> https://sqlite.org/lang_select.html >>>> >>>> FOR UPDATE only makes sense for relational databases. Adding >>>> assumptions it's available would make it harder to use a NoSQL database >>>> in future. >>>> >>>> In the case of SQLite, in effect it's got serialized isolation >>>> anyway, so the FOR UPDATE is not strictly necessary. >>>> >>>> I suggest OFBiz shouldn't have a method that generates syntactically >>>> incorrect SQL for any database we want to support, and we claim to >>>> support several. Where there are variations between SQL dialects, we >>>> have attributes in the data source, such as "join-style". >>>> >>>> So think our choices are: >>>> >>>> 1. Add an attribute to say "SELECT ... FOR UPDATE" is supported by a >>>> data source. OFBiz would only add the "FOR UPDATE" if it is >>>> supported. In the case of SQLite, I suspect we'd get away without it, >>>> but there may be other databases where the absence of "FOR UPDATE" >>>> might bring on unexpected behaviour. >>>> >>>> 2. Don't provide a method to expose FOR UPDATE. >>>> >>>> Could you achieve the results you want with a higher transaction >>>> isolation level like repeatable read, or with optimistic locking >>>> (enable-lock)? >>>> >>>> What do you think? >>>> >>>> Cheers >>>> >>>> Paul Foxworthy >>>> >>>> >>>> On 26 October 2017 at 07:50, Gareth Carter >>>> <[hidden email]> >>>> wrote: >>>> >>>> Ok I will create a Jira with patch for review. >>>>> >>>>> >>>>> >>>>> Sent from my Samsung Galaxy smartphone. >>>>> >>>>> >>>>> >>>>> Gareth Carter >>>>> >>>>> >>>>> Software Development Analyst >>>>> >>>>> >>>>> Stannah Management Services Ltd >>>>> >>>>> >>>>> IT Department >>>>> >>>>> >>>>> Ext: >>>>> >>>>> >>>>> 7036 >>>>> >>>>> >>>>> DDI: >>>>> >>>>> >>>>> 01264 364311 >>>>> >>>>> >>>>> >>>>> >>>>> [http://logos.stannah.co.uk/stan150.jpg] >>>>> >>>>> >>>>> [http://logos.stannah.co.uk/enviro.jpg]Please consider the >>>>> environment before printing this email. >>>>> >>>>> >>>>> -------- Original message -------- >>>>> From: deepak nigam <[hidden email]> >>>>> Date: 25/10/2017 14:06 (GMT+00:00) >>>>> To: [hidden email] >>>>> Subject: Re: Updates to EntityQuery >>>>> >>>>> Looking good to me also. Let me know if I can be of any help. >>>>> >>>>> >>>>> Thanks & Regards >>>>> -- >>>>> Deepak Nigam >>>>> >>>>> On Wed, Oct 25, 2017 at 6:26 PM, Yash Sharma < >>>>> [hidden email]> >>>>> wrote: >>>>> >>>>> I am all up for it, please let me know if I could help. >>>>>> >>>>>> Thanks & Regards, >>>>>> -- >>>>>> *Pradhan Yash Sharma* >>>>>> *HotWax Systems* | >>>>>> www.hotwaxsystems.com<http://www.hotwaxsystems.com> >>>>>> >>>>>> On Wed, Oct 25, 2017 at 6:23 PM, Yash Sharma < >>>>>> [hidden email]> >>>>>> wrote: >>>>>> >>>>>> Yes, Usage of the stream will surely enhance performance to a >>>>>>> certain extent and removes ceremony from the code base. I think >>>>>>> Parallel >>>>>>> >>>>>> streams >>>>> >>>>>> will add enhancements in many folds as we are using a multicore >>>>>>> >>>>>> processor >>>>> >>>>>> (I have not tested yet), but the Functional approach seems promising. >>>>>>> >>>>>>> Thanks & Regards, >>>>>>> -- >>>>>>> *Pradhan Yash Sharma* >>>>>>> *HotWax Systems* | www.hotwaxsystems.com<http:// >>>>>>> >>>>>> www.hotwaxsystems.com> >>>> >>>>> On Wed, Oct 25, 2017 at 5:59 PM, Gareth Carter < >>>>>>> [hidden email]> wrote: >>>>>>> >>>>>>> forUpdate >>>>>>>> >>>>>>>> We patched EntityFindOptions with a new field "forUpdate" with >>>>>>>> >>>>>>> shorthand >>>>> >>>>>> methods in EntityQuery to enable. We then made a change to >>>>>>>> GenericDAO.selectListIteratorByCondition to add "FOR UPDATE" >>>>>>>> on the >>>>>>>> >>>>>>> end >>>>> >>>>>> of the SQL select statement - this allows for DB row locks (we >>>>>>>> use >>>>>>>> >>>>>>> postgres >>>>>> >>>>>>> and works but have not tested other databases). I believe >>>>>>>> there may >>>>>>>> >>>>>>> have >>>>> >>>>>> been a discussion about this before >>>>>>>> >>>>>>>> >>>>>>>> forEach on EntityQuery >>>>>>>> >>>>>>>> Use Consumer in java and groovy to iterate over a query. This >>>>>>>> can >>>>>>>> >>>>>>> reduce >>>>> >>>>>> memory consumption (replacement for queryList()) and boiler >>>>>>>> plate >>>>>>>> >>>>>>> code >>>> >>>>> (eg >>>>>> >>>>>>> queryIterator(), while loop and close) >>>>>>>> >>>>>>>> Example: >>>>>>>> >>>>>>>> EntityQuery.use(delegator).from("Foobar") >>>>>>>> .forEach(item -> >>>>>>>> Debug.logInfo(item.toString(), module); ); >>>>>>>> >>>>>>>> A further update could be to provide stream capabilities >>>>>>>> >>>>>>>> Hope this helps, I can provide a patch aswell >>>>>>>> >>>>>>>> Gareth Carter >>>>>>>> Software Development Analyst >>>>>>>> Stannah Management Services Ltd IT Department >>>>>>>> Ext: >>>>>>>> 7036 >>>>>>>> DDI: >>>>>>>> 01264 364311 >>>>>>>> >>>>>>>> >>>>>>>> Please consider the environment before printing this email. >>>>>>>> >>>>>>>> -----Original Message----- >>>>>>>> From: Arun Patidar [mailto:[hidden email]] >>>>>>>> Sent: 25 October 2017 5:49 AM >>>>>>>> To: [hidden email] >>>>>>>> Cc: [hidden email] >>>>>>>> Subject: Re: Updates to EntityQuery >>>>>>>> >>>>>>>> Hello Gareth, >>>>>>>> >>>>>>>> Please provide some more details or patch to understand - forUpdate >>>>>>>> >>>>>>> and >>>>> >>>>>> forEach method utility. >>>>>>>> >>>>>>>> getFieldMap method looks good to me. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Thanks & Regards >>>>>>>> --- >>>>>>>> Arun Patidar >>>>>>>> Manager, Enterprise Software Development >>>>>>>> >>>>>>>> HotWax Systems Pvt Ltd. >>>>>>>> >>>>>>>> www.hotwaxsystems.com<http://www.hotwaxsystems.com> >>>>>>>> >>>>>>>> >>>>>>>> On Tue, Oct 24, 2017 at 9:06 PM, Gareth Carter < >>>>>>>> [hidden email]> >>>>>>>> wrote: >>>>>>>> >>>>>>>> Hi all >>>>>>>>> >>>>>>>>> We have internally patched EntityQuery with some additional >>>>>>>>> functionality and before I create a Jira was going to see what the >>>>>>>>> >>>>>>>> community thinks. >>>>>>>> >>>>>>>>> New functionality: >>>>>>>>> >>>>>>>>> - New method getFieldMap which returns a Map object of >>>>>>>>> >>>>>>>> selected >>>>> >>>>>> fields from GenericValue objects, useful for creating cache map >>>>>>>>> objects for lookup >>>>>>>>> >>>>>>>>> - Support forUpdate >>>>>>>>> >>>>>>>>> - forEach method to accept Consumer >>>>>>>>> >>>>>>>>> We have found these useful and believe the project can benefit, >>>>>>>>> >>>>>>>> let >>>> >>>>> me >>>>> >>>>>> know what you think >>>>>>>>> >>>>>>>>> Regards >>>>>>>>> >>>>>>>>> Gareth Carter >>>>>>>>> >>>>>>>>> >>>>>>>>> Software Development Analyst >>>>>>>>> >>>>>>>>> >>>>>>>>> Stannah Management Services Ltd >>>>>>>>> >>>>>>>>> >>>>>>>>> IT Department >>>>>>>>> >>>>>>>>> >>>>>>>>> Ext: >>>>>>>>> >>>>>>>>> >>>>>>>>> 7036 >>>>>>>>> >>>>>>>>> >>>>>>>>> DDI: >>>>>>>>> >>>>>>>>> >>>>>>>>> 01264 364311 >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> [http://logos.stannah.co.uk/stan150.jpg] >>>>>>>>> >>>>>>>>> >>>>>>>>> [http://logos.stannah.co.uk/enviro.jpg]Please consider the >>>>>>>>> >>>>>>>> environment >>>>>> >>>>>>> before printing this email. >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> This email is intended only for the above addressee. It may >>>>>>>>> >>>>>>>> contain >>>> >>>>> privileged information. If you are not the addressee you must not >>>>>>>>> copy, distribute, disclose or use any of the information in it. If >>>>>>>>> >>>>>>>> you >>>>> >>>>>> have received it in error, please delete it and notify the sender. >>>>>>>>> >>>>>>>>> Stannah Lift Holdings Ltd registered No. 686996, Stannah >>>>>>>>> >>>>>>>> Management >>>> >>>>> Services Ltd registered No. 2483693, Stannah Lift Services Ltd >>>>>>>>> registered No. 1189799, Stannah Microlifts Ltd registered No. >>>>>>>>> >>>>>>>> 964804, >>>>> >>>>>> Stannah Lifts Ltd registered No. 1189836, Stannah Stairlifts Ltd >>>>>>>>> registered No. 1401451, Global Upholstery Solutions Ltd registered >>>>>>>>> >>>>>>>> No. >>>>> >>>>>> 02452728. >>>>>>>> >>>>>>>>> All registered offices at Watt Close, East Portway, Andover, >>>>>>>>> Hampshire, >>>>>>>>> SP10 3SD, England. >>>>>>>>> >>>>>>>>> All Registered in England and Wales. >>>>>>>>> >>>>>>>>> This message has been scanned for malware by Websense. >>>>>>>>> www.websense.com<http://www.websense.com> >>>>>>>>> >>>>>>>>> >>>>>>>> To report this email as spam, please send the original message, >>>>>>>> >>>>>>> complete >>>>> >>>>>> with headers to [hidden email] >>>>>>>> >>>>>>>> >>>>>>> >>>> >>>> -- >>>> Coherent Software Australia Pty Ltd >>>> PO Box 2773 >>>> Cheltenham Vic 3192 >>>> Australia >>>> >>>> Phone: +61 3 9585 6788 >>>> Web: http://webdefence.global.blackspider.com/urlwrap/?q=AXicFYxN >>>> CsIwEEa_E02K-NPqRlDs2t5gKJGIaVIm0wYPVnDr0rt4COP28d5Di8MHWN6A >>>> -OeqZkoy08B338egEj31ccDcXDfn7tJV63q3bdCyWHV0YlErx6QcArsi0vSA >>>> Ux33xuScC3BWbNAUb5pL8l8RTwbA9wX8ANMdKWg&Z >>>> Email: [hidden email] >>>> >>>> > -- Coherent Software Australia Pty Ltd PO Box 2773 Cheltenham Vic 3192 Australia Phone: +61 3 9585 6788 Web: http://www.coherentsoftware.com.au/ Email: [hidden email]
--
Coherent Software Australia Pty Ltd http://www.coherentsoftware.com.au/ Bonsai ERP, the all-inclusive ERP system http://www.bonsaierp.com.au/ |
Re: Updates to EntityQuery
|
|
In reply to this post by taher
On Thu, Oct 26, 2017 at 7:13 AM, Taher Alkhateeb <[hidden email]
> wrote: > [...] The idea of NoSQL is to think at a different conceptual level of > your domain model. > I agree! In order to leverage the power of NoSQL databases you have to write your code accordingly and apply patterns that are fundamentally different from the ones adopted with relational databases; also, NoSQL is a rather large group that contains different families of products that require different handling. For all of these reasons I don't think that porting the entity engine to a NoSQL database, while technically possible, would be actually useful. Jacopo |
|
|
In reply to this post by Paul Foxworthy
Hi Paul
Problem with that, it's all or nothing. We only need this for specific use cases. Having forUpdate in EntityQuery gives us that flexibility. It could be changed to use either method which depends on the scenario you use it for Gareth Carter Software Development Analyst Stannah Management Services Ltd IT Department Ext: 7036 DDI: 01264 364311 Please consider the environment before printing this email. -----Original Message----- From: Paul Foxworthy [mailto:[hidden email]] Sent: 27 October 2017 12:42 AM To: [hidden email] Subject: Re: Updates to EntityQuery Hi all, As far as I know, FOR UPDATE would only break SQLite, and I can live with that. Gareth, would optimistic instead of pessimistic work for your situation? The database does less of the work and it's more scalable. I haven't tested it out, but OFBiz provides optimistic locking if you turn on enable-lock. Cheers Paul Foxworthy On 27 October 2017 at 03:49, Jacques Le Roux <[hidden email]> wrote: > I agree with Scott and Taher, > > About your question Gareth, we once had a hard coded "for update" > somewhere in OFBiz code (in SequenceUtil.SequenceBank.fillBank() ). > I put it in. Then Jacopo crossed an issue in a cluster and rightly > changed it to something better See > https://issues.apache.org/jira/browse/OFBIZ-2353 for more where Adrian > gave some directions also. > > Since I did not read all in the thread and there is no patch yet I > don't know if something alike applies to EntityQuery API. Putting a > for update is certainly much simpler, so just saying for now. > > Jacques > > > > Le 26/10/2017 à 10:46, Gareth Carter a écrit : > >> HI all >> >> Thank for the response >> >> It could be considered a hint, if the underlying db does not support >> it then don't use it. Adding an attribute on the datasource could be >> the way to go but I also believe we need methods to tell the system >> when to use it, you don't need it all the time only in selective cases. >> >> Until support is added I see ofbiz as purely RDMS. I don't believe >> ofbiz should restrict itself on the assumption it might support >> different databases such as nosql. >> >> There could be an alternative to forUpdate, anyone considered >> implementing a distributed locking mechanism? The whole point for >> using forUpdate is to prevent one or more code blocks changing the >> same database records concurrently, db row locks help but are not >> supported by all databases but an external locking mechanism may >> provide a generic way of dealing with this. OFBiz already provides a >> semaphore on services but I believe you need something at entity >> level Gareth Carter Software Development Analyst Stannah Management >> Services Ltd IT Department >> Ext: >> 7036 >> DDI: >> 01264 364311 >> >> >> Please consider the environment before printing this email. >> >> -----Original Message----- >> From: Taher Alkhateeb [mailto:[hidden email]] >> Sent: 26 October 2017 6:13 AM >> To: OFBIZ Development Mailing List <[hidden email]> >> Subject: Re: Updates to EntityQuery >> >> Hmmm usually my first gut reaction would be "keep things consistent", >> however Scott made a good argument for supporting non-common DB features. >> So my suggestion would be to use something like the widget's >> <platform-specific ...> tag (or equivalent in code) for any features >> not supported in all DBMSs. By labeling it as such the user is aware >> that the DB selection is restricted. >> >> Now with regards to your comment about NoSQL, such databases are >> fundamentally different, and act more like a key-value based entities >> without a schema or the resulting ACID properties. It is possible to >> make it work with the entity engine, but then you'd lose the >> flexibility it gives you because you want to restrict yourself to the >> relational model that we apply in OFBiz. The idea of NoSQL is to >> think at a different conceptual level of your domain model. >> >> On Thu, Oct 26, 2017 at 4:33 AM, Scott Gray >> <[hidden email]> >> wrote: >> >>> I think it's better to have for-update available for use for the >>> majority of use-cases than it is to exclude it altogether. I'd >>> rather lose the databases that don't support it than constrain the >>> ones that do and are used most by pretty much everyone. It would be >>> nice to hear from anyone using a database that doesn't support it >>> since I'm just guessing most of us use postgres or mysql. >>> >>> Has anyone actually looked at how NoSQL databases might work with >>> the OFBiz entity engine or are the concepts too far removed from SQL >>> for it to even be workable? >>> >>> If we did want to support for-update globally, we could always have >>> the delegator simulate it by issuing an UPDATE to lock the rows >>> prior to performing the SELECT. Although it might not work >>> perfectly in all cases, it could be a good compromise. e.g. >>> UPDATE table SET lastUpdatedTxStamp = NOW() WHERE ...conditions -- >>> lock the rows SELECT * FROM table WHERE ...conditions -- select the >>> rows >>> >>> Regards >>> Scott >>> >>> >>> >>> On 26 October 2017 at 13:45, Paul Foxworthy <[hidden email]> wrote: >>> >>> Hi Gareth, >>>> >>>> FOR UPDATE is standard SQL, but even so SQLite for one doesn't >>>> support >>>> it: >>>> >>>> http://sqlite.1065341.n5.nabble.com/SELECT-FOR-UPDATE-td89630.html >>>> https://sqlite.org/lang_select.html >>>> >>>> FOR UPDATE only makes sense for relational databases. Adding >>>> assumptions it's available would make it harder to use a NoSQL >>>> database in future. >>>> >>>> In the case of SQLite, in effect it's got serialized isolation >>>> anyway, so the FOR UPDATE is not strictly necessary. >>>> >>>> I suggest OFBiz shouldn't have a method that generates >>>> syntactically incorrect SQL for any database we want to support, >>>> and we claim to support several. Where there are variations between >>>> SQL dialects, we have attributes in the data source, such as "join-style". >>>> >>>> So think our choices are: >>>> >>>> 1. Add an attribute to say "SELECT ... FOR UPDATE" is supported by >>>> a data source. OFBiz would only add the "FOR UPDATE" if it is >>>> supported. In the case of SQLite, I suspect we'd get away without >>>> it, but there may be other databases where the absence of "FOR UPDATE" >>>> might bring on unexpected behaviour. >>>> >>>> 2. Don't provide a method to expose FOR UPDATE. >>>> >>>> Could you achieve the results you want with a higher transaction >>>> isolation level like repeatable read, or with optimistic locking >>>> (enable-lock)? >>>> >>>> What do you think? >>>> >>>> Cheers >>>> >>>> Paul Foxworthy >>>> >>>> >>>> On 26 October 2017 at 07:50, Gareth Carter >>>> <[hidden email]> >>>> wrote: >>>> >>>> Ok I will create a Jira with patch for review. >>>>> >>>>> >>>>> >>>>> Sent from my Samsung Galaxy smartphone. >>>>> >>>>> >>>>> >>>>> Gareth Carter >>>>> >>>>> >>>>> Software Development Analyst >>>>> >>>>> >>>>> Stannah Management Services Ltd >>>>> >>>>> >>>>> IT Department >>>>> >>>>> >>>>> Ext: >>>>> >>>>> >>>>> 7036 >>>>> >>>>> >>>>> DDI: >>>>> >>>>> >>>>> 01264 364311 >>>>> >>>>> >>>>> >>>>> >>>>> [http://logos.stannah.co.uk/stan150.jpg] >>>>> >>>>> >>>>> [http://logos.stannah.co.uk/enviro.jpg]Please consider the >>>>> environment before printing this email. >>>>> >>>>> >>>>> -------- Original message -------- >>>>> From: deepak nigam <[hidden email]> >>>>> Date: 25/10/2017 14:06 (GMT+00:00) >>>>> To: [hidden email] >>>>> Subject: Re: Updates to EntityQuery >>>>> >>>>> Looking good to me also. Let me know if I can be of any help. >>>>> >>>>> >>>>> Thanks & Regards >>>>> -- >>>>> Deepak Nigam >>>>> >>>>> On Wed, Oct 25, 2017 at 6:26 PM, Yash Sharma < >>>>> [hidden email]> >>>>> wrote: >>>>> >>>>> I am all up for it, please let me know if I could help. >>>>>> >>>>>> Thanks & Regards, >>>>>> -- >>>>>> *Pradhan Yash Sharma* >>>>>> *HotWax Systems* | >>>>>> www.hotwaxsystems.com<http://www.hotwaxsystems.com> >>>>>> >>>>>> On Wed, Oct 25, 2017 at 6:23 PM, Yash Sharma < >>>>>> [hidden email]> >>>>>> wrote: >>>>>> >>>>>> Yes, Usage of the stream will surely enhance performance to a >>>>>>> certain extent and removes ceremony from the code base. I think >>>>>>> Parallel >>>>>>> >>>>>> streams >>>>> >>>>>> will add enhancements in many folds as we are using a multicore >>>>>>> >>>>>> processor >>>>> >>>>>> (I have not tested yet), but the Functional approach seems promising. >>>>>>> >>>>>>> Thanks & Regards, >>>>>>> -- >>>>>>> *Pradhan Yash Sharma* >>>>>>> *HotWax Systems* | www.hotwaxsystems.com<http:// >>>>>>> >>>>>> www.hotwaxsystems.com> >>>> >>>>> On Wed, Oct 25, 2017 at 5:59 PM, Gareth Carter < >>>>>>> [hidden email]> wrote: >>>>>>> >>>>>>> forUpdate >>>>>>>> >>>>>>>> We patched EntityFindOptions with a new field "forUpdate" with >>>>>>>> >>>>>>> shorthand >>>>> >>>>>> methods in EntityQuery to enable. We then made a change to >>>>>>>> GenericDAO.selectListIteratorByCondition to add "FOR UPDATE" >>>>>>>> on the >>>>>>>> >>>>>>> end >>>>> >>>>>> of the SQL select statement - this allows for DB row locks (we >>>>>>>> use >>>>>>>> >>>>>>> postgres >>>>>> >>>>>>> and works but have not tested other databases). I believe >>>>>>>> there may >>>>>>>> >>>>>>> have >>>>> >>>>>> been a discussion about this before >>>>>>>> >>>>>>>> >>>>>>>> forEach on EntityQuery >>>>>>>> >>>>>>>> Use Consumer in java and groovy to iterate over a query. This >>>>>>>> can >>>>>>>> >>>>>>> reduce >>>>> >>>>>> memory consumption (replacement for queryList()) and boiler >>>>>>>> plate >>>>>>>> >>>>>>> code >>>> >>>>> (eg >>>>>> >>>>>>> queryIterator(), while loop and close) >>>>>>>> >>>>>>>> Example: >>>>>>>> >>>>>>>> EntityQuery.use(delegator).from("Foobar") >>>>>>>> .forEach(item -> >>>>>>>> Debug.logInfo(item.toString(), module); ); >>>>>>>> >>>>>>>> A further update could be to provide stream capabilities >>>>>>>> >>>>>>>> Hope this helps, I can provide a patch aswell >>>>>>>> >>>>>>>> Gareth Carter >>>>>>>> Software Development Analyst >>>>>>>> Stannah Management Services Ltd IT Department >>>>>>>> Ext: >>>>>>>> 7036 >>>>>>>> DDI: >>>>>>>> 01264 364311 >>>>>>>> >>>>>>>> >>>>>>>> Please consider the environment before printing this email. >>>>>>>> >>>>>>>> -----Original Message----- >>>>>>>> From: Arun Patidar [mailto:[hidden email]] >>>>>>>> Sent: 25 October 2017 5:49 AM >>>>>>>> To: [hidden email] >>>>>>>> Cc: [hidden email] >>>>>>>> Subject: Re: Updates to EntityQuery >>>>>>>> >>>>>>>> Hello Gareth, >>>>>>>> >>>>>>>> Please provide some more details or patch to understand - >>>>>>>> forUpdate >>>>>>>> >>>>>>> and >>>>> >>>>>> forEach method utility. >>>>>>>> >>>>>>>> getFieldMap method looks good to me. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Thanks & Regards >>>>>>>> --- >>>>>>>> Arun Patidar >>>>>>>> Manager, Enterprise Software Development >>>>>>>> >>>>>>>> HotWax Systems Pvt Ltd. >>>>>>>> >>>>>>>> www.hotwaxsystems.com<http://www.hotwaxsystems.com> >>>>>>>> >>>>>>>> >>>>>>>> On Tue, Oct 24, 2017 at 9:06 PM, Gareth Carter < >>>>>>>> [hidden email]> >>>>>>>> wrote: >>>>>>>> >>>>>>>> Hi all >>>>>>>>> >>>>>>>>> We have internally patched EntityQuery with some additional >>>>>>>>> functionality and before I create a Jira was going to see what >>>>>>>>> the >>>>>>>>> >>>>>>>> community thinks. >>>>>>>> >>>>>>>>> New functionality: >>>>>>>>> >>>>>>>>> - New method getFieldMap which returns a Map object of >>>>>>>>> >>>>>>>> selected >>>>> >>>>>> fields from GenericValue objects, useful for creating cache map >>>>>>>>> objects for lookup >>>>>>>>> >>>>>>>>> - Support forUpdate >>>>>>>>> >>>>>>>>> - forEach method to accept Consumer >>>>>>>>> >>>>>>>>> We have found these useful and believe the project can >>>>>>>>> benefit, >>>>>>>>> >>>>>>>> let >>>> >>>>> me >>>>> >>>>>> know what you think >>>>>>>>> >>>>>>>>> Regards >>>>>>>>> >>>>>>>>> Gareth Carter >>>>>>>>> >>>>>>>>> >>>>>>>>> Software Development Analyst >>>>>>>>> >>>>>>>>> >>>>>>>>> Stannah Management Services Ltd >>>>>>>>> >>>>>>>>> >>>>>>>>> IT Department >>>>>>>>> >>>>>>>>> >>>>>>>>> Ext: >>>>>>>>> >>>>>>>>> >>>>>>>>> 7036 >>>>>>>>> >>>>>>>>> >>>>>>>>> DDI: >>>>>>>>> >>>>>>>>> >>>>>>>>> 01264 364311 >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> [http://logos.stannah.co.uk/stan150.jpg] >>>>>>>>> >>>>>>>>> >>>>>>>>> [http://logos.stannah.co.uk/enviro.jpg]Please consider the >>>>>>>>> >>>>>>>> environment >>>>>> >>>>>>> before printing this email. >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> This email is intended only for the above addressee. It may >>>>>>>>> >>>>>>>> contain >>>> >>>>> privileged information. If you are not the addressee you must not >>>>>>>>> copy, distribute, disclose or use any of the information in >>>>>>>>> it. If >>>>>>>>> >>>>>>>> you >>>>> >>>>>> have received it in error, please delete it and notify the sender. >>>>>>>>> >>>>>>>>> Stannah Lift Holdings Ltd registered No. 686996, Stannah >>>>>>>>> >>>>>>>> Management >>>> >>>>> Services Ltd registered No. 2483693, Stannah Lift Services Ltd >>>>>>>>> registered No. 1189799, Stannah Microlifts Ltd registered No. >>>>>>>>> >>>>>>>> 964804, >>>>> >>>>>> Stannah Lifts Ltd registered No. 1189836, Stannah Stairlifts Ltd >>>>>>>>> registered No. 1401451, Global Upholstery Solutions Ltd >>>>>>>>> registered >>>>>>>>> >>>>>>>> No. >>>>> >>>>>> 02452728. >>>>>>>> >>>>>>>>> All registered offices at Watt Close, East Portway, Andover, >>>>>>>>> Hampshire, >>>>>>>>> SP10 3SD, England. >>>>>>>>> >>>>>>>>> All Registered in England and Wales. >>>>>>>>> >>>>>>>>> This message has been scanned for malware by Websense. >>>>>>>>> www.websense.com<http://www.websense.com> >>>>>>>>> >>>>>>>>> >>>>>>>> To report this email as spam, please send the original message, >>>>>>>> >>>>>>> complete >>>>> >>>>>> with headers to [hidden email] >>>>>>>> >>>>>>>> >>>>>>> >>>> >>>> -- >>>> Coherent Software Australia Pty Ltd PO Box 2773 Cheltenham Vic 3192 >>>> Australia >>>> >>>> Phone: +61 3 9585 6788 >>>> Web: http://webdefence.global.blackspider.com/urlwrap/?q=AXicFYxN >>>> CsIwEEa_E02K-NPqRlDs2t5gKJGIaVIm0wYPVnDr0rt4COP28d5Di8MHWN6A >>>> -OeqZkoy08B338egEj31ccDcXDfn7tJV63q3bdCyWHV0YlErx6QcArsi0vSA >>>> Ux33xuScC3BWbNAUb5pL8l8RTwbA9wX8ANMdKWg&Z >>>> Email: [hidden email] >>>> >>>> > -- Coherent Software Australia Pty Ltd PO Box 2773 Cheltenham Vic 3192 Australia Phone: +61 3 9585 6788 Web: http://webdefence.global.blackspider.com/urlwrap/?q=AXicY2BnsD7JwLD1IANDUU6lkXGiXnFRmV5uYmZOcn5eSVF-jl5yfi5DmWWgX3pwiLmBgZmhhQmDe2JRakmGnnNiUUlqkUNxSWJeXmIGUKFeaTZDRklJgZW-fnl5OVAgI7UoNa-kOD-tpByoBWSUXmKpPgMDw5sdDAwAvaYpRA&Z Email: [hidden email] |
|
Administrator
|
In reply to this post by Yash Sharma
Hi Yash, Gareth,
Not something obvious, but as explained in https://dzone.com/articles/whats-wrong-java-8-part-iii hoping to *blindly *use Parallel Streams for intensive computation tasks is not a good idea :) Jacques Le 25/10/2017 à 14:53, Yash Sharma a écrit : > Yes, Usage of the stream will surely enhance performance to a certain > extent and removes ceremony from the code base. I think Parallel streams > will add enhancements in many folds as we are using a multicore processor > (I have not tested yet), but the Functional approach seems promising. > > Thanks & Regards, > -- > *Pradhan Yash Sharma* > *HotWax Systems* | www.hotwaxsystems.com > > On Wed, Oct 25, 2017 at 5:59 PM, Gareth Carter <[hidden email]> > wrote: > >> forUpdate >> >> We patched EntityFindOptions with a new field "forUpdate" with shorthand >> methods in EntityQuery to enable. We then made a change to GenericDAO.selectListIteratorByCondition >> to add "FOR UPDATE" on the end of the SQL select statement - this allows >> for DB row locks (we use postgres and works but have not tested other >> databases). I believe there may have been a discussion about this before >> >> >> forEach on EntityQuery >> >> Use Consumer in java and groovy to iterate over a query. This can reduce >> memory consumption (replacement for queryList()) and boiler plate code (eg >> queryIterator(), while loop and close) >> >> Example: >> >> EntityQuery.use(delegator).from("Foobar") >> .forEach(item -> >> Debug.logInfo(item.toString(), module); >> ); >> >> A further update could be to provide stream capabilities >> >> Hope this helps, I can provide a patch aswell >> >> Gareth Carter >> Software Development Analyst >> Stannah Management Services Ltd >> IT Department >> Ext: >> 7036 >> DDI: >> 01264 364311 >> >> >> Please consider the environment before printing this email. >> >> -----Original Message----- >> From: Arun Patidar [mailto:[hidden email]] >> Sent: 25 October 2017 5:49 AM >> To: [hidden email] >> Cc: [hidden email] >> Subject: Re: Updates to EntityQuery >> >> Hello Gareth, >> >> Please provide some more details or patch to understand - forUpdate and >> forEach method utility. >> >> getFieldMap method looks good to me. >> >> >> >> -- >> Thanks & Regards >> --- >> Arun Patidar >> Manager, Enterprise Software Development >> >> HotWax Systems Pvt Ltd. >> >> www.hotwaxsystems.com >> >> >> On Tue, Oct 24, 2017 at 9:06 PM, Gareth Carter < >> [hidden email]> >> wrote: >> >>> Hi all >>> >>> We have internally patched EntityQuery with some additional >>> functionality and before I create a Jira was going to see what the >> community thinks. >>> New functionality: >>> >>> - New method getFieldMap which returns a Map object of selected >>> fields from GenericValue objects, useful for creating cache map >>> objects for lookup >>> >>> - Support forUpdate >>> >>> - forEach method to accept Consumer >>> >>> We have found these useful and believe the project can benefit, let me >>> know what you think >>> >>> Regards >>> >>> Gareth Carter >>> >>> >>> Software Development Analyst >>> >>> >>> Stannah Management Services Ltd >>> >>> >>> IT Department >>> >>> >>> Ext: >>> >>> >>> 7036 >>> >>> >>> DDI: >>> >>> >>> 01264 364311 >>> >>> >>> >>> >>> [http://logos.stannah.co.uk/stan150.jpg] >>> >>> >>> [http://logos.stannah.co.uk/enviro.jpg]Please consider the environment >>> before printing this email. >>> >>> >>> >>> This email is intended only for the above addressee. It may contain >>> privileged information. If you are not the addressee you must not >>> copy, distribute, disclose or use any of the information in it. If you >>> have received it in error, please delete it and notify the sender. >>> >>> Stannah Lift Holdings Ltd registered No. 686996, Stannah Management >>> Services Ltd registered No. 2483693, Stannah Lift Services Ltd >>> registered No. 1189799, Stannah Microlifts Ltd registered No. 964804, >>> Stannah Lifts Ltd registered No. 1189836, Stannah Stairlifts Ltd >>> registered No. 1401451, Global Upholstery Solutions Ltd registered No. >> 02452728. >>> All registered offices at Watt Close, East Portway, Andover, >>> Hampshire, >>> SP10 3SD, England. >>> >>> All Registered in England and Wales. >>> >>> This message has been scanned for malware by Websense. >>> www.websense.com >>> >> >> To report this email as spam, please send the original message, complete >> with headers to [hidden email] >> |
|
Administrator
|
In reply to this post by Jacopo Cappellato-5
Le 27/10/2017 à 09:50, Jacopo Cappellato a écrit :
> On Thu, Oct 26, 2017 at 7:13 AM, Taher Alkhateeb <[hidden email] >> wrote: >> [...] The idea of NoSQL is to think at a different conceptual level of >> your domain model. >> > I agree! In order to leverage the power of NoSQL databases you have to > write your code accordingly and apply patterns that are fundamentally > different from the ones adopted with relational databases; also, NoSQL is a > rather large group that contains different families of products that > require different handling. > For all of these reasons I don't think that porting the entity engine to a > NoSQL database, while technically possible, would be actually useful. > > Jacopo > There are several TLPs which are *more or less* leveraging NoSQL databases with SQL, at least: Apache Drill, Apache Hive, Apache Impala They have JDBC drivers, but I don't know how much of SQL they support. Of course there are also other not Apache solutions: Presto (Facebook), Dremel (Google), ... One of them, Trafodion (initially from HP), is in incubation for 2+ years: http://trafodion.apache.org/ Here is a simple, clear answer about its possible usage https://mail-archives.apache.org/mod_mbox/incubator-trafodion-dev/201711.mbox/browser There are JDBC and ODBC drivers for Trafodion which is based on HBase http://trafodion.apache.org/architecture-overview.html http://trafodion.apache.org/faq.html Jacques |
«
Return to OFBiz - Dev
|
1 view|%1 views
| Free forum by Nabble | Edit this page |