[jira] [Commented] (OFBIZ-11200) Refactoring old job process

[Nicolas Malin (Jira)]

    [ https://issues.apache.org/jira/browse/OFBIZ-11200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17042655#comment-17042655 ]

ASF subversion and git services commented on OFBIZ-11200:
---------------------------------------------------------

Commit dff226ea601958608926d5623c2940232a707a93 in ofbiz-framework's branch refs/heads/trunk from Nicolas Malin
[ https://gitbox.apache.org/repos/asf?p=ofbiz-framework.git;h=dff226e ]

Improved: Refactoring old job process
(OFBIZ-11200)

Before this commit, to purge old job we use the thread job manager to check
if some job need to be run and if the pool is empty, check the old job to purge.

I detected a problem when you have many job's server and more than one pool.
When you have a first pool that run regular job and a second that receive
huge asynchrone services (by persist call), the server who manage the second pool
have rarely the possibility to purge their jobs that, by the way, increase JobSandbox table.

The Server who manage the first pool, call often the purge process, but the table size
generate a long query time to purge few element. Long query time, run often,
this purge process load uselessly the database.

After analyze, I refactoring these following items :

 * When you call JobSandbox table for a purge, use a limit on the query
   with the max thread pool because when you have the database return,
   you purge only with this limit so this help the database to not scan
   the whole table. Other small improvement, do not sort the result
   (no functional gain).

 * Purge by the pool thread is nice, each server can only purge their
   own job (filtered on run_by_instance_id), to help an overloaded server,
   I was rehability purgeOldJobs service that it can run with specific
    parameter to make assistance.

 * Last, all job services are historical on class org.apache.ofbiz.service.ServiceUtil,
   I moved them all in a new dedicate class org.apache.ofbiz.service.job.JobServices

> Refactoring old job process
> ---------------------------
>
>                 Key: OFBIZ-11200
>                 URL: https://issues.apache.org/jira/browse/OFBIZ-11200
>             Project: OFBiz
>          Issue Type: Improvement
>          Components: framework
>    Affects Versions: Trunk
>            Reporter: Nicolas Malin
>            Assignee: Nicolas Malin
>            Priority: Minor
>              Labels: job, service
>         Attachments: OFBIZ-11200.patch
>
>
> Currently to purge old job, we use the thread job manager to check if some job need to be run and if the pool is empty, check the old job to purge.
> I detected a problem when you have many job's server and two pools.
> In my case, I have a first pool that run regular job and a second that receive huge asynchrone services (by persist call).
> Server who manage the second pool have rarely the possibility to purge their jobs that, by the way, increase JobSandbox table
> Server who manage the first pool, call often the purge process, but the table size generate a long query time to purge few element. Long query time, run often, this purge process consume 15% of the database load (with correct reindex)
> After analyze, I propose this refactoring :
> * When you call JobSandbox table for a purge, use a limit on the query with the max thread pool because when you have the database return, you purge only with this limit so this help the database to not scan the whole table. Other small improvement, do not sort the result (no functional  gain). With this each query pass to 3.5s at 0.30s (on postgres).
> * Purge by the pool thread is nice, each server can only purge their own job (filtered on run_by_instance_id), to help an overloaded server, I was rehabiliting service purgeOldJobs that you can run with specific parameter to make assistance.
> * Last, all job services are historical on class org.apache.ofbiz.service.ServiceUtil, I moved them all in a new dedicate class org.apache.ofbiz.service.job.JobServices

--
This message was sent by Atlassian Jira
(v8.3.4#803005)