Users - Debugging Scheduled Jobs Tip

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

Users - Debugging Scheduled Jobs Tip

Lon F. Binder-2
All -
 
After spending a few hours working through an issue, I'd like to provide an interesting (albeit obvious in hindsight) tip.
 
We had two different OFBiz installations sharing the same database.  One was a dev environment, the other was a QA environment (for the same target application).  This led to different versions of the same codebase across the two environments.  All in all, this was a fine approach for sharing the general data (store, catalog, parties, etc.).
 
However, a consequence was overlapping job queue workers.  Depending on the timing jobs would be worked on by the two applications running.  And because the code base was slightly different (due to ongoing development), sometimes the jobs would work and sometimes they wouldn't.
 
TIP: Don't let multiple OFBiz application environments (dev, qa, etc.) share a single OFBiz database, at least while you're testing/developing scheduled jobs.
 
 - Lon
 

 
_______________________________________________
Users mailing list
[hidden email]
http://lists.ofbiz.org/mailman/listinfo/users
Reply | Threaded
Open this post in threaded view
|

Re: Users - Debugging Scheduled Jobs Tip

Florin T.PATRASCU (work)
Hi,

The way to avoid this is to use the service engine *pool* feature that
allows each service engine instance to specify which pool it "publishes"
to by default, and which pool(s) it "subscribes" to.

excerpt from serviceengine.xml installed on a Test server:

   <!-- Thread pool configuration (max/min threads, uses to live and
time to live) -->
    <thread-pool send-to-pool="hobbit1"
                 purge-job-days="4"
....
                 poll-db-millis="20000">
        <run-from-pool name="hobbit1"/>
    </thread-pool>


while the same serviceengine config file but on a Production machine
will look different a bit:

    <thread-pool send-to-pool="everest01"
....
                 poll-db-millis="20000">
        <run-from-pool name="everest01"/>
    </thread-pool>

Just make sure they have different names for your development servers
and your production servers and this problem should go away. We
encountered this issue in the past. So you can still share your db and
get advantages from that;)

I hope it helps.

-florin

PS
Sorry for my English if it is confusing :( but I believe is better than
the one Andrew Dupa is "using" ;)

Lon F. Binder wrote:

> All -
>  
> After spending a few hours working through an issue, I'd like to
> provide an interesting (albeit obvious in hindsight) tip.
>  
> We had two different OFBiz installations sharing the same database.  
> One was a dev environment, the other was a QA environment (for the
> same target application).  This led to different versions of the same
> codebase across the two environments.  All in all, this was a fine
> approach for sharing the general data (store, catalog, parties, etc.).
>  
> However, a consequence was overlapping job queue workers.  Depending
> on the timing jobs would be worked on by the two applications
> running.  And because the code base was slightly different (due to
> ongoing development), sometimes the jobs would work and sometimes they
> wouldn't.
>  
> TIP: Don't let multiple OFBiz application environments (dev, qa, etc.)
> share a single OFBiz database, at least while you're
> testing/developing scheduled jobs.
>  
>  - Lon
>  
> ------------------------------------------------------------------------
>
>  
> _______________________________________________
> Users mailing list
> [hidden email]
> http://lists.ofbiz.org/mailman/listinfo/users


 
_______________________________________________
Users mailing list
[hidden email]
http://lists.ofbiz.org/mailman/listinfo/users
Reply | Threaded
Open this post in threaded view
|

Re: Users - Debugging Scheduled Jobs Tip

David E. Jones

Yes, this is the way to go...

We usually recommend keeping a local repository with a base set of  
code and configuration files for a "development" mode, and then  
keeping a directory with patches (in the repository) for the various  
servers. When OFBiz is deployed on each server (test, staging,  
production, perhaps multiple of each) you just check out the rev you  
want to deploy, apply the patch for that server, and build and run.  
This makes things more easy to reproduce and allows for configuration  
variations between servers.

-David


On Feb 16, 2006, at 8:58 AM, Florin T.PATRASCU (work) wrote:

> Hi,
>
> The way to avoid this is to use the service engine *pool* feature that
> allows each service engine instance to specify which pool it  
> "publishes"
> to by default, and which pool(s) it "subscribes" to.
>
> excerpt from serviceengine.xml installed on a Test server:
>
>    <!-- Thread pool configuration (max/min threads, uses to live and
> time to live) -->
>     <thread-pool send-to-pool="hobbit1"
>                  purge-job-days="4"
> ....
>                  poll-db-millis="20000">
>         <run-from-pool name="hobbit1"/>
>     </thread-pool>
>
>
> while the same serviceengine config file but on a Production machine
> will look different a bit:
>
>     <thread-pool send-to-pool="everest01"
> ....
>                  poll-db-millis="20000">
>         <run-from-pool name="everest01"/>
>     </thread-pool>
>
> Just make sure they have different names for your development servers
> and your production servers and this problem should go away. We
> encountered this issue in the past. So you can still share your db and
> get advantages from that;)
>
> I hope it helps.
>
> -florin
>
> PS
> Sorry for my English if it is confusing :( but I believe is better  
> than
> the one Andrew Dupa is "using" ;)
>
> Lon F. Binder wrote:
>> All -
>>
>> After spending a few hours working through an issue, I'd like to
>> provide an interesting (albeit obvious in hindsight) tip.
>>
>> We had two different OFBiz installations sharing the same database.
>> One was a dev environment, the other was a QA environment (for the
>> same target application).  This led to different versions of the same
>> codebase across the two environments.  All in all, this was a fine
>> approach for sharing the general data (store, catalog, parties,  
>> etc.).
>>
>> However, a consequence was overlapping job queue workers.  Depending
>> on the timing jobs would be worked on by the two applications
>> running.  And because the code base was slightly different (due to
>> ongoing development), sometimes the jobs would work and sometimes  
>> they
>> wouldn't.
>>
>> TIP: Don't let multiple OFBiz application environments (dev, qa,  
>> etc.)
>> share a single OFBiz database, at least while you're
>> testing/developing scheduled jobs.
>>
>>  - Lon
>>
>> ---------------------------------------------------------------------
>> ---
>>
>>
>> _______________________________________________
>> Users mailing list
>> [hidden email]
>> http://lists.ofbiz.org/mailman/listinfo/users
>
>
>
> _______________________________________________
> Users mailing list
> [hidden email]
> http://lists.ofbiz.org/mailman/listinfo/users

 
_______________________________________________
Users mailing list
[hidden email]
http://lists.ofbiz.org/mailman/listinfo/users
Reply | Threaded
Open this post in threaded view
|

Re: Users - Debugging Scheduled Jobs Tip

Lon F. Binder-2
David (and everyone),

That's a good idea.  Alternatively, this is bigger-picture-approach, perhaps
we should alter the build file to include build targets by environment, that
way we can standardize ofbiz projects on a file extension that can be picked
up and removed per target.

Using Florin's great tip as an example, I might have:
  serviceengine.xml@Lon
  serviceengine.xml@B1
  serviceengine.xml@production
Where the String following the '@' is free form.

Then we could add a build target that accepts a parameter.  When files end
in "@${param}" that extension is removed locally.

So if I did a "build B1" upon build it would choose "servicengine.xml@B1" as
listed above, and create a copy in the same folder called
"serviceengine.xml".

The goal being to use the same freeform extension for whatever means across
the entire fileset for environment-sensitive files.  Having a freeform
extension would allow for unlimited environments (which is particularly
helpful on projects with many devs).  And doesn't require any special
repository work to maintain.

Thoughts?

 - Lon

-----Original Message-----
From: [hidden email] [mailto:[hidden email]]
On Behalf Of David E. Jones
Sent: Thursday, February 16, 2006 11:42 AM
To: OFBiz Users / Usage Discussion
Subject: Re: [OFBiz] Users - Debugging Scheduled Jobs Tip


Yes, this is the way to go...

We usually recommend keeping a local repository with a base set of code and
configuration files for a "development" mode, and then keeping a directory
with patches (in the repository) for the various servers. When OFBiz is
deployed on each server (test, staging, production, perhaps multiple of
each) you just check out the rev you want to deploy, apply the patch for
that server, and build and run.  
This makes things more easy to reproduce and allows for configuration
variations between servers.

-David


On Feb 16, 2006, at 8:58 AM, Florin T.PATRASCU (work) wrote:

> Hi,
>
> The way to avoid this is to use the service engine *pool* feature that
> allows each service engine instance to specify which pool it
> "publishes"
> to by default, and which pool(s) it "subscribes" to.
>
> excerpt from serviceengine.xml installed on a Test server:
>
>    <!-- Thread pool configuration (max/min threads, uses to live and
> time to live) -->
>     <thread-pool send-to-pool="hobbit1"
>                  purge-job-days="4"
> ....
>                  poll-db-millis="20000">
>         <run-from-pool name="hobbit1"/>
>     </thread-pool>
>
>
> while the same serviceengine config file but on a Production machine
> will look different a bit:
>
>     <thread-pool send-to-pool="everest01"
> ....
>                  poll-db-millis="20000">
>         <run-from-pool name="everest01"/>
>     </thread-pool>
>
> Just make sure they have different names for your development servers
> and your production servers and this problem should go away. We
> encountered this issue in the past. So you can still share your db and
> get advantages from that;)
>
> I hope it helps.
>
> -florin
>
> PS
> Sorry for my English if it is confusing :( but I believe is better
> than the one Andrew Dupa is "using" ;)
>
> Lon F. Binder wrote:
>> All -
>>
>> After spending a few hours working through an issue, I'd like to
>> provide an interesting (albeit obvious in hindsight) tip.
>>
>> We had two different OFBiz installations sharing the same database.
>> One was a dev environment, the other was a QA environment (for the
>> same target application).  This led to different versions of the same
>> codebase across the two environments.  All in all, this was a fine
>> approach for sharing the general data (store, catalog, parties,
>> etc.).
>>
>> However, a consequence was overlapping job queue workers.  Depending
>> on the timing jobs would be worked on by the two applications
>> running.  And because the code base was slightly different (due to
>> ongoing development), sometimes the jobs would work and sometimes
>> they wouldn't.
>>
>> TIP: Don't let multiple OFBiz application environments (dev, qa,
>> etc.)
>> share a single OFBiz database, at least while you're
>> testing/developing scheduled jobs.
>>
>>  - Lon
>>
>> ---------------------------------------------------------------------
>> ---
>>
>>
>> _______________________________________________
>> Users mailing list
>> [hidden email]
>> http://lists.ofbiz.org/mailman/listinfo/users
>
>
>
> _______________________________________________
> Users mailing list
> [hidden email]
> http://lists.ofbiz.org/mailman/listinfo/users

 
_______________________________________________
Users mailing list
[hidden email]
http://lists.ofbiz.org/mailman/listinfo/users


 
_______________________________________________
Users mailing list
[hidden email]
http://lists.ofbiz.org/mailman/listinfo/users
Reply | Threaded
Open this post in threaded view
|

Re: Users - Debugging Scheduled Jobs Tip

David E. Jones
In reply to this post by David E. Jones

Patches are generally easier to maintain...

They can also be applied using ant if you create sets of build targets.

-David


On Feb 16, 2006, at 10:20 AM, Lon F. Binder wrote:

> David (and everyone),
>
> That's a good idea.  Alternatively, this is bigger-picture-
> approach, perhaps
> we should alter the build file to include build targets by  
> environment, that
> way we can standardize ofbiz projects on a file extension that can  
> be picked
> up and removed per target.
>
> Using Florin's great tip as an example, I might have:
>   serviceengine.xml@Lon
>   serviceengine.xml@B1
>   serviceengine.xml@production
> Where the String following the '@' is free form.
>
> Then we could add a build target that accepts a parameter.  When  
> files end
> in "@${param}" that extension is removed locally.
>
> So if I did a "build B1" upon build it would choose  
> "servicengine.xml@B1" as
> listed above, and create a copy in the same folder called
> "serviceengine.xml".
>
> The goal being to use the same freeform extension for whatever  
> means across
> the entire fileset for environment-sensitive files.  Having a freeform
> extension would allow for unlimited environments (which is  
> particularly
> helpful on projects with many devs).  And doesn't require any special
> repository work to maintain.
>
> Thoughts?
>
>  - Lon
>
> -----Original Message-----
> From: [hidden email] [mailto:users-
> [hidden email]]
> On Behalf Of David E. Jones
> Sent: Thursday, February 16, 2006 11:42 AM
> To: OFBiz Users / Usage Discussion
> Subject: Re: [OFBiz] Users - Debugging Scheduled Jobs Tip
>
>
> Yes, this is the way to go...
>
> We usually recommend keeping a local repository with a base set of  
> code and
> configuration files for a "development" mode, and then keeping a  
> directory
> with patches (in the repository) for the various servers. When  
> OFBiz is
> deployed on each server (test, staging, production, perhaps  
> multiple of
> each) you just check out the rev you want to deploy, apply the  
> patch for
> that server, and build and run.
> This makes things more easy to reproduce and allows for configuration
> variations between servers.
>
> -David
>
>
> On Feb 16, 2006, at 8:58 AM, Florin T.PATRASCU (work) wrote:
>
>> Hi,
>>
>> The way to avoid this is to use the service engine *pool* feature  
>> that
>> allows each service engine instance to specify which pool it
>> "publishes"
>> to by default, and which pool(s) it "subscribes" to.
>>
>> excerpt from serviceengine.xml installed on a Test server:
>>
>>    <!-- Thread pool configuration (max/min threads, uses to live and
>> time to live) -->
>>     <thread-pool send-to-pool="hobbit1"
>>                  purge-job-days="4"
>> ....
>>                  poll-db-millis="20000">
>>         <run-from-pool name="hobbit1"/>
>>     </thread-pool>
>>
>>
>> while the same serviceengine config file but on a Production machine
>> will look different a bit:
>>
>>     <thread-pool send-to-pool="everest01"
>> ....
>>                  poll-db-millis="20000">
>>         <run-from-pool name="everest01"/>
>>     </thread-pool>
>>
>> Just make sure they have different names for your development servers
>> and your production servers and this problem should go away. We
>> encountered this issue in the past. So you can still share your db  
>> and
>> get advantages from that;)
>>
>> I hope it helps.
>>
>> -florin
>>
>> PS
>> Sorry for my English if it is confusing :( but I believe is better
>> than the one Andrew Dupa is "using" ;)
>>
>> Lon F. Binder wrote:
>>> All -
>>>
>>> After spending a few hours working through an issue, I'd like to
>>> provide an interesting (albeit obvious in hindsight) tip.
>>>
>>> We had two different OFBiz installations sharing the same database.
>>> One was a dev environment, the other was a QA environment (for the
>>> same target application).  This led to different versions of the  
>>> same
>>> codebase across the two environments.  All in all, this was a fine
>>> approach for sharing the general data (store, catalog, parties,
>>> etc.).
>>>
>>> However, a consequence was overlapping job queue workers.  Depending
>>> on the timing jobs would be worked on by the two applications
>>> running.  And because the code base was slightly different (due to
>>> ongoing development), sometimes the jobs would work and sometimes
>>> they wouldn't.
>>>
>>> TIP: Don't let multiple OFBiz application environments (dev, qa,
>>> etc.)
>>> share a single OFBiz database, at least while you're
>>> testing/developing scheduled jobs.
>>>
>>>  - Lon
>>>
>>> --------------------------------------------------------------------
>>> -
>>> ---
>>>
>>>
>>> _______________________________________________
>>> Users mailing list
>>> [hidden email]
>>> http://lists.ofbiz.org/mailman/listinfo/users
>>
>>
>>
>> _______________________________________________
>> Users mailing list
>> [hidden email]
>> http://lists.ofbiz.org/mailman/listinfo/users
>
>
> _______________________________________________
> Users mailing list
> [hidden email]
> http://lists.ofbiz.org/mailman/listinfo/users
>
>
>
> _______________________________________________
> Users mailing list
> [hidden email]
> http://lists.ofbiz.org/mailman/listinfo/users

 
_______________________________________________
Users mailing list
[hidden email]
http://lists.ofbiz.org/mailman/listinfo/users

smime.p7s (3K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Users - Debugging Scheduled Jobs Tip

Florin T.PATRASCU (work)
Guys, I don't know if this sounds alien, but David, how difficult will
be to improve the UtilXml.getXmlRootElement(...) (or a better place?) to
"decorate" the loaded xml config files with parameters defined in a
properties file or coming from the System properties, classpath, JVM
params or etc...??

In our case this will look like this:

local property file defining:
send-to-pool=devServerA

And then in the serviceengine.xml we could simply have:
...
  <thread-pool send-to-pool="${send-to-pool}"
                 purge-job-days="4"
                 failed-retry-min="3"
 
...

And then the developer only has to tweak his local config file.

I don't know how realistic my proposal is, but I was just thinking
about, anyway please consider it my 2 CAD cents intervention to this
thread ;)

-florin


David E. Jones wrote:

>
> Patches are generally easier to maintain...
>
> They can also be applied using ant if you create sets of build targets.
>
> -David
>
>
> On Feb 16, 2006, at 10:20 AM, Lon F. Binder wrote:
>
>> David (and everyone),
>>
>> That's a good idea.  Alternatively, this is bigger-picture-approach,
>> perhaps
>> we should alter the build file to include build targets by
>> environment, that
>> way we can standardize ofbiz projects on a file extension that can be
>> picked
>> up and removed per target.
>>
>> Using Florin's great tip as an example, I might have:
>>   serviceengine.xml@Lon
>>   serviceengine.xml@B1
>>   serviceengine.xml@production
>> Where the String following the '@' is free form.
>>
>> Then we could add a build target that accepts a parameter.  When
>> files end
>> in "@${param}" that extension is removed locally.
>>
>> So if I did a "build B1" upon build it would choose
>> "servicengine.xml@B1" as
>> listed above, and create a copy in the same folder called
>> "serviceengine.xml".
>>
>> The goal being to use the same freeform extension for whatever means
>> across
>> the entire fileset for environment-sensitive files.  Having a freeform
>> extension would allow for unlimited environments (which is particularly
>> helpful on projects with many devs).  And doesn't require any special
>> repository work to maintain.
>>
>> Thoughts?
>>
>>  - Lon
>>
>> -----Original Message-----
>> From: [hidden email]
>> [mailto:[hidden email]]
>> On Behalf Of David E. Jones
>> Sent: Thursday, February 16, 2006 11:42 AM
>> To: OFBiz Users / Usage Discussion
>> Subject: Re: [OFBiz] Users - Debugging Scheduled Jobs Tip
>>
>>
>> Yes, this is the way to go...
>>
>> We usually recommend keeping a local repository with a base set of
>> code and
>> configuration files for a "development" mode, and then keeping a
>> directory
>> with patches (in the repository) for the various servers. When OFBiz is
>> deployed on each server (test, staging, production, perhaps multiple of
>> each) you just check out the rev you want to deploy, apply the patch for
>> that server, and build and run.
>> This makes things more easy to reproduce and allows for configuration
>> variations between servers.
>>
>> -David
>>
>>
>> On Feb 16, 2006, at 8:58 AM, Florin T.PATRASCU (work) wrote:
>>
>>> Hi,
>>>
>>> The way to avoid this is to use the service engine *pool* feature that
>>> allows each service engine instance to specify which pool it
>>> "publishes"
>>> to by default, and which pool(s) it "subscribes" to.
>>>
>>> excerpt from serviceengine.xml installed on a Test server:
>>>
>>>    <!-- Thread pool configuration (max/min threads, uses to live and
>>> time to live) -->
>>>     <thread-pool send-to-pool="hobbit1"
>>>                  purge-job-days="4"
>>> ....
>>>                  poll-db-millis="20000">
>>>         <run-from-pool name="hobbit1"/>
>>>     </thread-pool>
>>>
>>>
>>> while the same serviceengine config file but on a Production machine
>>> will look different a bit:
>>>
>>>     <thread-pool send-to-pool="everest01"
>>> ....
>>>                  poll-db-millis="20000">
>>>         <run-from-pool name="everest01"/>
>>>     </thread-pool>
>>>
>>> Just make sure they have different names for your development servers
>>> and your production servers and this problem should go away. We
>>> encountered this issue in the past. So you can still share your db and
>>> get advantages from that;)
>>>
>>> I hope it helps.
>>>
>>> -florin
>>>
>>> PS
>>> Sorry for my English if it is confusing :( but I believe is better
>>> than the one Andrew Dupa is "using" ;)
>>>
>>> Lon F. Binder wrote:
>>>> All -
>>>>
>>>> After spending a few hours working through an issue, I'd like to
>>>> provide an interesting (albeit obvious in hindsight) tip.
>>>>
>>>> We had two different OFBiz installations sharing the same database.
>>>> One was a dev environment, the other was a QA environment (for the
>>>> same target application).  This led to different versions of the same
>>>> codebase across the two environments.  All in all, this was a fine
>>>> approach for sharing the general data (store, catalog, parties,
>>>> etc.).
>>>>
>>>> However, a consequence was overlapping job queue workers.  Depending
>>>> on the timing jobs would be worked on by the two applications
>>>> running.  And because the code base was slightly different (due to
>>>> ongoing development), sometimes the jobs would work and sometimes
>>>> they wouldn't.
>>>>
>>>> TIP: Don't let multiple OFBiz application environments (dev, qa,
>>>> etc.)
>>>> share a single OFBiz database, at least while you're
>>>> testing/developing scheduled jobs.
>>>>
>>>>  - Lon
>>>>
>>>> ---------------------------------------------------------------------
>>>> ---
>>>>
>>>>
>>>> _______________________________________________
>>>> Users mailing list
>>>> [hidden email]
>>>> http://lists.ofbiz.org/mailman/listinfo/users
>>>
>>>
>>>
>>> _______________________________________________
>>> Users mailing list
>>> [hidden email]
>>> http://lists.ofbiz.org/mailman/listinfo/users
>>
>>
>> _______________________________________________
>> Users mailing list
>> [hidden email]
>> http://lists.ofbiz.org/mailman/listinfo/users
>>
>>
>>
>> _______________________________________________
>> Users mailing list
>> [hidden email]
>> http://lists.ofbiz.org/mailman/listinfo/users
>
> ------------------------------------------------------------------------
>
>  
> _______________________________________________
> Users mailing list
> [hidden email]
> http://lists.ofbiz.org/mailman/listinfo/users


 
_______________________________________________
Users mailing list
[hidden email]
http://lists.ofbiz.org/mailman/listinfo/users
Reply | Threaded
Open this post in threaded view
|

Re: Users - Debugging Scheduled Jobs Tip

David E. Jones

Hmmmm... I always considered the serviceengine.xml file to _be_ a  
"local config" file....

-David


On Feb 16, 2006, at 10:47 AM, Florin T.PATRASCU (work) wrote:

> Guys, I don't know if this sounds alien, but David, how difficult will
> be to improve the UtilXml.getXmlRootElement(...) (or a better  
> place?) to
> "decorate" the loaded xml config files with parameters defined in a
> properties file or coming from the System properties, classpath, JVM
> params or etc...??
>
> In our case this will look like this:
>
> local property file defining:
> send-to-pool=devServerA
>
> And then in the serviceengine.xml we could simply have:
> ...
>   <thread-pool send-to-pool="${send-to-pool}"
>                  purge-job-days="4"
>                  failed-retry-min="3"
>
> ...
>
> And then the developer only has to tweak his local config file.
>
> I don't know how realistic my proposal is, but I was just thinking
> about, anyway please consider it my 2 CAD cents intervention to this
> thread ;)
>
> -florin
>
>
> David E. Jones wrote:
>>
>> Patches are generally easier to maintain...
>>
>> They can also be applied using ant if you create sets of build  
>> targets.
>>
>> -David
>>
>>
>> On Feb 16, 2006, at 10:20 AM, Lon F. Binder wrote:
>>
>>> David (and everyone),
>>>
>>> That's a good idea.  Alternatively, this is bigger-picture-approach,
>>> perhaps
>>> we should alter the build file to include build targets by
>>> environment, that
>>> way we can standardize ofbiz projects on a file extension that  
>>> can be
>>> picked
>>> up and removed per target.
>>>
>>> Using Florin's great tip as an example, I might have:
>>>   serviceengine.xml@Lon
>>>   serviceengine.xml@B1
>>>   serviceengine.xml@production
>>> Where the String following the '@' is free form.
>>>
>>> Then we could add a build target that accepts a parameter.  When
>>> files end
>>> in "@${param}" that extension is removed locally.
>>>
>>> So if I did a "build B1" upon build it would choose
>>> "servicengine.xml@B1" as
>>> listed above, and create a copy in the same folder called
>>> "serviceengine.xml".
>>>
>>> The goal being to use the same freeform extension for whatever means
>>> across
>>> the entire fileset for environment-sensitive files.  Having a  
>>> freeform
>>> extension would allow for unlimited environments (which is  
>>> particularly
>>> helpful on projects with many devs).  And doesn't require any  
>>> special
>>> repository work to maintain.
>>>
>>> Thoughts?
>>>
>>>  - Lon
>>>
>>> -----Original Message-----
>>> From: [hidden email]
>>> [mailto:[hidden email]]
>>> On Behalf Of David E. Jones
>>> Sent: Thursday, February 16, 2006 11:42 AM
>>> To: OFBiz Users / Usage Discussion
>>> Subject: Re: [OFBiz] Users - Debugging Scheduled Jobs Tip
>>>
>>>
>>> Yes, this is the way to go...
>>>
>>> We usually recommend keeping a local repository with a base set of
>>> code and
>>> configuration files for a "development" mode, and then keeping a
>>> directory
>>> with patches (in the repository) for the various servers. When  
>>> OFBiz is
>>> deployed on each server (test, staging, production, perhaps  
>>> multiple of
>>> each) you just check out the rev you want to deploy, apply the  
>>> patch for
>>> that server, and build and run.
>>> This makes things more easy to reproduce and allows for  
>>> configuration
>>> variations between servers.
>>>
>>> -David
>>>
>>>
>>> On Feb 16, 2006, at 8:58 AM, Florin T.PATRASCU (work) wrote:
>>>
>>>> Hi,
>>>>
>>>> The way to avoid this is to use the service engine *pool*  
>>>> feature that
>>>> allows each service engine instance to specify which pool it
>>>> "publishes"
>>>> to by default, and which pool(s) it "subscribes" to.
>>>>
>>>> excerpt from serviceengine.xml installed on a Test server:
>>>>
>>>>    <!-- Thread pool configuration (max/min threads, uses to live  
>>>> and
>>>> time to live) -->
>>>>     <thread-pool send-to-pool="hobbit1"
>>>>                  purge-job-days="4"
>>>> ....
>>>>                  poll-db-millis="20000">
>>>>         <run-from-pool name="hobbit1"/>
>>>>     </thread-pool>
>>>>
>>>>
>>>> while the same serviceengine config file but on a Production  
>>>> machine
>>>> will look different a bit:
>>>>
>>>>     <thread-pool send-to-pool="everest01"
>>>> ....
>>>>                  poll-db-millis="20000">
>>>>         <run-from-pool name="everest01"/>
>>>>     </thread-pool>
>>>>
>>>> Just make sure they have different names for your development  
>>>> servers
>>>> and your production servers and this problem should go away. We
>>>> encountered this issue in the past. So you can still share your  
>>>> db and
>>>> get advantages from that;)
>>>>
>>>> I hope it helps.
>>>>
>>>> -florin
>>>>
>>>> PS
>>>> Sorry for my English if it is confusing :( but I believe is better
>>>> than the one Andrew Dupa is "using" ;)
>>>>
>>>> Lon F. Binder wrote:
>>>>> All -
>>>>>
>>>>> After spending a few hours working through an issue, I'd like to
>>>>> provide an interesting (albeit obvious in hindsight) tip.
>>>>>
>>>>> We had two different OFBiz installations sharing the same  
>>>>> database.
>>>>> One was a dev environment, the other was a QA environment (for the
>>>>> same target application).  This led to different versions of  
>>>>> the same
>>>>> codebase across the two environments.  All in all, this was a fine
>>>>> approach for sharing the general data (store, catalog, parties,
>>>>> etc.).
>>>>>
>>>>> However, a consequence was overlapping job queue workers.  
>>>>> Depending
>>>>> on the timing jobs would be worked on by the two applications
>>>>> running.  And because the code base was slightly different (due to
>>>>> ongoing development), sometimes the jobs would work and sometimes
>>>>> they wouldn't.
>>>>>
>>>>> TIP: Don't let multiple OFBiz application environments (dev, qa,
>>>>> etc.)
>>>>> share a single OFBiz database, at least while you're
>>>>> testing/developing scheduled jobs.
>>>>>
>>>>>  - Lon
>>>>>
>>>>> ------------------------------------------------------------------
>>>>> ---
>>>>> ---
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Users mailing list
>>>>> [hidden email]
>>>>> http://lists.ofbiz.org/mailman/listinfo/users
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Users mailing list
>>>> [hidden email]
>>>> http://lists.ofbiz.org/mailman/listinfo/users
>>>
>>>
>>> _______________________________________________
>>> Users mailing list
>>> [hidden email]
>>> http://lists.ofbiz.org/mailman/listinfo/users
>>>
>>>
>>>
>>> _______________________________________________
>>> Users mailing list
>>> [hidden email]
>>> http://lists.ofbiz.org/mailman/listinfo/users
>>
>> ---------------------------------------------------------------------
>> ---
>>
>>
>> _______________________________________________
>> Users mailing list
>> [hidden email]
>> http://lists.ofbiz.org/mailman/listinfo/users
>
>
>
> _______________________________________________
> Users mailing list
> [hidden email]
> http://lists.ofbiz.org/mailman/listinfo/users

 
_______________________________________________
Users mailing list
[hidden email]
http://lists.ofbiz.org/mailman/listinfo/users

smime.p7s (3K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Users - Debugging Scheduled Jobs Tip

Florin T.PATRASCU (work)
true, when you don't have scenarios like the ones we encountered, where
the "local config" file is shared among Developers or different machines
and such.

-florin


David E. Jones wrote:

>
> Hmmmm... I always considered the serviceengine.xml file to _be_ a
> "local config" file....
>
> -David
>
>
> On Feb 16, 2006, at 10:47 AM, Florin T.PATRASCU (work) wrote:
>
>> Guys, I don't know if this sounds alien, but David, how difficult will
>> be to improve the UtilXml.getXmlRootElement(...) (or a better place?) to
>> "decorate" the loaded xml config files with parameters defined in a
>> properties file or coming from the System properties, classpath, JVM
>> params or etc...??
>>
>> In our case this will look like this:
>>
>> local property file defining:
>> send-to-pool=devServerA
>>
>> And then in the serviceengine.xml we could simply have:
>> ...
>>   <thread-pool send-to-pool="${send-to-pool}"
>>                  purge-job-days="4"
>>                  failed-retry-min="3"
>>
>> ...
>>
>> And then the developer only has to tweak his local config file.
>>
>> I don't know how realistic my proposal is, but I was just thinking
>> about, anyway please consider it my 2 CAD cents intervention to this
>> thread ;)
>>
>> -florin
>>
>>
>> David E. Jones wrote:
>>>
>>> Patches are generally easier to maintain...
>>>
>>> They can also be applied using ant if you create sets of build targets.
>>>
>>> -David
>>>
>>>
>>> On Feb 16, 2006, at 10:20 AM, Lon F. Binder wrote:
>>>
>>>> David (and everyone),
>>>>
>>>> That's a good idea.  Alternatively, this is bigger-picture-approach,
>>>> perhaps
>>>> we should alter the build file to include build targets by
>>>> environment, that
>>>> way we can standardize ofbiz projects on a file extension that can be
>>>> picked
>>>> up and removed per target.
>>>>
>>>> Using Florin's great tip as an example, I might have:
>>>>   serviceengine.xml@Lon
>>>>   serviceengine.xml@B1
>>>>   serviceengine.xml@production
>>>> Where the String following the '@' is free form.
>>>>
>>>> Then we could add a build target that accepts a parameter.  When
>>>> files end
>>>> in "@${param}" that extension is removed locally.
>>>>
>>>> So if I did a "build B1" upon build it would choose
>>>> "servicengine.xml@B1" as
>>>> listed above, and create a copy in the same folder called
>>>> "serviceengine.xml".
>>>>
>>>> The goal being to use the same freeform extension for whatever means
>>>> across
>>>> the entire fileset for environment-sensitive files.  Having a freeform
>>>> extension would allow for unlimited environments (which is
>>>> particularly
>>>> helpful on projects with many devs).  And doesn't require any special
>>>> repository work to maintain.
>>>>
>>>> Thoughts?
>>>>
>>>>  - Lon
>>>>
>>>> -----Original Message-----
>>>> From: [hidden email]
>>>> [mailto:[hidden email]]
>>>> On Behalf Of David E. Jones
>>>> Sent: Thursday, February 16, 2006 11:42 AM
>>>> To: OFBiz Users / Usage Discussion
>>>> Subject: Re: [OFBiz] Users - Debugging Scheduled Jobs Tip
>>>>
>>>>
>>>> Yes, this is the way to go...
>>>>
>>>> We usually recommend keeping a local repository with a base set of
>>>> code and
>>>> configuration files for a "development" mode, and then keeping a
>>>> directory
>>>> with patches (in the repository) for the various servers. When
>>>> OFBiz is
>>>> deployed on each server (test, staging, production, perhaps
>>>> multiple of
>>>> each) you just check out the rev you want to deploy, apply the
>>>> patch for
>>>> that server, and build and run.
>>>> This makes things more easy to reproduce and allows for configuration
>>>> variations between servers.
>>>>
>>>> -David
>>>>
>>>>
>>>> On Feb 16, 2006, at 8:58 AM, Florin T.PATRASCU (work) wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> The way to avoid this is to use the service engine *pool* feature
>>>>> that
>>>>> allows each service engine instance to specify which pool it
>>>>> "publishes"
>>>>> to by default, and which pool(s) it "subscribes" to.
>>>>>
>>>>> excerpt from serviceengine.xml installed on a Test server:
>>>>>
>>>>>    <!-- Thread pool configuration (max/min threads, uses to live and
>>>>> time to live) -->
>>>>>     <thread-pool send-to-pool="hobbit1"
>>>>>                  purge-job-days="4"
>>>>> ....
>>>>>                  poll-db-millis="20000">
>>>>>         <run-from-pool name="hobbit1"/>
>>>>>     </thread-pool>
>>>>>
>>>>>
>>>>> while the same serviceengine config file but on a Production machine
>>>>> will look different a bit:
>>>>>
>>>>>     <thread-pool send-to-pool="everest01"
>>>>> ....
>>>>>                  poll-db-millis="20000">
>>>>>         <run-from-pool name="everest01"/>
>>>>>     </thread-pool>
>>>>>
>>>>> Just make sure they have different names for your development servers
>>>>> and your production servers and this problem should go away. We
>>>>> encountered this issue in the past. So you can still share your db
>>>>> and
>>>>> get advantages from that;)
>>>>>
>>>>> I hope it helps.
>>>>>
>>>>> -florin
>>>>>
>>>>> PS
>>>>> Sorry for my English if it is confusing :( but I believe is better
>>>>> than the one Andrew Dupa is "using" ;)
>>>>>
>>>>> Lon F. Binder wrote:
>>>>>> All -
>>>>>>
>>>>>> After spending a few hours working through an issue, I'd like to
>>>>>> provide an interesting (albeit obvious in hindsight) tip.
>>>>>>
>>>>>> We had two different OFBiz installations sharing the same database.
>>>>>> One was a dev environment, the other was a QA environment (for the
>>>>>> same target application).  This led to different versions of the
>>>>>> same
>>>>>> codebase across the two environments.  All in all, this was a fine
>>>>>> approach for sharing the general data (store, catalog, parties,
>>>>>> etc.).
>>>>>>
>>>>>> However, a consequence was overlapping job queue workers.  Depending
>>>>>> on the timing jobs would be worked on by the two applications
>>>>>> running.  And because the code base was slightly different (due to
>>>>>> ongoing development), sometimes the jobs would work and sometimes
>>>>>> they wouldn't.
>>>>>>
>>>>>> TIP: Don't let multiple OFBiz application environments (dev, qa,
>>>>>> etc.)
>>>>>> share a single OFBiz database, at least while you're
>>>>>> testing/developing scheduled jobs.
>>>>>>
>>>>>>  - Lon
>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>>
>>>>>> ---
>


 
_______________________________________________
Users mailing list
[hidden email]
http://lists.ofbiz.org/mailman/listinfo/users
Reply | Threaded
Open this post in threaded view
|

Re: Users - Debugging Scheduled Jobs Tip

Lon F. Binder-2
David is correct-- there shouldn't be "meta config" files for environments.
The standard approach is to version the individual files.

Whether it's in the repository (as David suggested), by file extension (as I
mentioned), or by subdirectory (as Joe mentioned).  I think the file
extension approach is the most flexible.

We are about to start a new OFBiz project in shop, I will probably employ
this approach (by file extension) and see how it goes.  If it works, I'll
gladly share.

 - Lon

-----Original Message-----
From: [hidden email] [mailto:[hidden email]]
On Behalf Of Florin T.PATRASCU (work)
Sent: Thursday, February 16, 2006 1:00 PM
To: OFBiz Users / Usage Discussion
Subject: Re: [OFBiz] Users - Debugging Scheduled Jobs Tip

true, when you don't have scenarios like the ones we encountered, where the
"local config" file is shared among Developers or different machines and
such.

-florin


David E. Jones wrote:

>
> Hmmmm... I always considered the serviceengine.xml file to _be_ a
> "local config" file....
>
> -David
>
>
> On Feb 16, 2006, at 10:47 AM, Florin T.PATRASCU (work) wrote:
>
>> Guys, I don't know if this sounds alien, but David, how difficult
>> will be to improve the UtilXml.getXmlRootElement(...) (or a better
>> place?) to "decorate" the loaded xml config files with parameters
>> defined in a properties file or coming from the System properties,
>> classpath, JVM params or etc...??
>>
>> In our case this will look like this:
>>
>> local property file defining:
>> send-to-pool=devServerA
>>
>> And then in the serviceengine.xml we could simply have:
>> ...
>>   <thread-pool send-to-pool="${send-to-pool}"
>>                  purge-job-days="4"
>>                  failed-retry-min="3"
>>
>> ...
>>
>> And then the developer only has to tweak his local config file.
>>
>> I don't know how realistic my proposal is, but I was just thinking
>> about, anyway please consider it my 2 CAD cents intervention to this
>> thread ;)
>>
>> -florin
>>
>>
>> David E. Jones wrote:
>>>
>>> Patches are generally easier to maintain...
>>>
>>> They can also be applied using ant if you create sets of build targets.
>>>
>>> -David
>>>
>>>
>>> On Feb 16, 2006, at 10:20 AM, Lon F. Binder wrote:
>>>
>>>> David (and everyone),
>>>>
>>>> That's a good idea.  Alternatively, this is
>>>> bigger-picture-approach, perhaps we should alter the build file to
>>>> include build targets by environment, that way we can standardize
>>>> ofbiz projects on a file extension that can be picked up and
>>>> removed per target.
>>>>
>>>> Using Florin's great tip as an example, I might have:
>>>>   serviceengine.xml@Lon
>>>>   serviceengine.xml@B1
>>>>   serviceengine.xml@production
>>>> Where the String following the '@' is free form.
>>>>
>>>> Then we could add a build target that accepts a parameter.  When
>>>> files end in "@${param}" that extension is removed locally.
>>>>
>>>> So if I did a "build B1" upon build it would choose
>>>> "servicengine.xml@B1" as listed above, and create a copy in the
>>>> same folder called "serviceengine.xml".
>>>>
>>>> The goal being to use the same freeform extension for whatever
>>>> means across the entire fileset for environment-sensitive files.  
>>>> Having a freeform extension would allow for unlimited environments
>>>> (which is particularly helpful on projects with many devs).  And
>>>> doesn't require any special repository work to maintain.
>>>>
>>>> Thoughts?
>>>>
>>>>  - Lon
>>>>
>>>> -----Original Message-----
>>>> From: [hidden email]
>>>> [mailto:[hidden email]]
>>>> On Behalf Of David E. Jones
>>>> Sent: Thursday, February 16, 2006 11:42 AM
>>>> To: OFBiz Users / Usage Discussion
>>>> Subject: Re: [OFBiz] Users - Debugging Scheduled Jobs Tip
>>>>
>>>>
>>>> Yes, this is the way to go...
>>>>
>>>> We usually recommend keeping a local repository with a base set of
>>>> code and configuration files for a "development" mode, and then
>>>> keeping a directory with patches (in the repository) for the
>>>> various servers. When OFBiz is deployed on each server (test,
>>>> staging, production, perhaps multiple of
>>>> each) you just check out the rev you want to deploy, apply the
>>>> patch for that server, and build and run.
>>>> This makes things more easy to reproduce and allows for
>>>> configuration variations between servers.
>>>>
>>>> -David
>>>>
>>>>
>>>> On Feb 16, 2006, at 8:58 AM, Florin T.PATRASCU (work) wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> The way to avoid this is to use the service engine *pool* feature
>>>>> that allows each service engine instance to specify which pool it
>>>>> "publishes"
>>>>> to by default, and which pool(s) it "subscribes" to.
>>>>>
>>>>> excerpt from serviceengine.xml installed on a Test server:
>>>>>
>>>>>    <!-- Thread pool configuration (max/min threads, uses to live
>>>>> and time to live) -->
>>>>>     <thread-pool send-to-pool="hobbit1"
>>>>>                  purge-job-days="4"
>>>>> ....
>>>>>                  poll-db-millis="20000">
>>>>>         <run-from-pool name="hobbit1"/>
>>>>>     </thread-pool>
>>>>>
>>>>>
>>>>> while the same serviceengine config file but on a Production
>>>>> machine will look different a bit:
>>>>>
>>>>>     <thread-pool send-to-pool="everest01"
>>>>> ....
>>>>>                  poll-db-millis="20000">
>>>>>         <run-from-pool name="everest01"/>
>>>>>     </thread-pool>
>>>>>
>>>>> Just make sure they have different names for your development
>>>>> servers and your production servers and this problem should go
>>>>> away. We encountered this issue in the past. So you can still
>>>>> share your db and get advantages from that;)
>>>>>
>>>>> I hope it helps.
>>>>>
>>>>> -florin
>>>>>
>>>>> PS
>>>>> Sorry for my English if it is confusing :( but I believe is better
>>>>> than the one Andrew Dupa is "using" ;)
>>>>>
>>>>> Lon F. Binder wrote:
>>>>>> All -
>>>>>>
>>>>>> After spending a few hours working through an issue, I'd like to
>>>>>> provide an interesting (albeit obvious in hindsight) tip.
>>>>>>
>>>>>> We had two different OFBiz installations sharing the same database.
>>>>>> One was a dev environment, the other was a QA environment (for
>>>>>> the same target application).  This led to different versions of
>>>>>> the same codebase across the two environments.  All in all, this
>>>>>> was a fine approach for sharing the general data (store, catalog,
>>>>>> parties, etc.).
>>>>>>
>>>>>> However, a consequence was overlapping job queue workers.  
>>>>>> Depending on the timing jobs would be worked on by the two
>>>>>> applications running.  And because the code base was slightly
>>>>>> different (due to ongoing development), sometimes the jobs would
>>>>>> work and sometimes they wouldn't.
>>>>>>
>>>>>> TIP: Don't let multiple OFBiz application environments (dev, qa,
>>>>>> etc.)
>>>>>> share a single OFBiz database, at least while you're
>>>>>> testing/developing scheduled jobs.
>>>>>>
>>>>>>  - Lon
>>>>>>
>>>>>> -----------------------------------------------------------------
>>>>>> ----
>>>>>>
>>>>>> ---
>


 
_______________________________________________
Users mailing list
[hidden email]
http://lists.ofbiz.org/mailman/listinfo/users


 
_______________________________________________
Users mailing list
[hidden email]
http://lists.ofbiz.org/mailman/listinfo/users
Reply | Threaded
Open this post in threaded view
|

Re: Users - Debugging Scheduled Jobs Tip

Andrew Sykes
In reply to this post by Lon F. Binder-2
Lon,

We tried something similar to this a while back, the big problem with
this is you end up having to maintain a whole file when perhaps all you
want to do is adjust a single line in that file.

You end up with a load of extra work, because your version of the file
can quickly go stale.

We have chosen therefore to use patches instead, the advantage here is
that if someone enhances a part of the file you don't care about, you
still get the benefit. Occasionally your patches fail if a file has
changed significantly, but this too can be a good thing, because it
flags up that something you rely on has changed. Imagine how much can
slip through the net if you are effectively just overwriting the latest
copy of the file with a version you made 500 commits ago!
--
Kind Regards
Andrew Sykes <[hidden email]>
Sykes Development Ltd
http://www.sykesdevelopment.com

 
_______________________________________________
Users mailing list
[hidden email]
http://lists.ofbiz.org/mailman/listinfo/users
Reply | Threaded
Open this post in threaded view
|

Re: Users - Debugging Scheduled Jobs Tip

Lon F. Binder-2
Andrew,

That is a good point.  However, I'm just hesitant to use the repository
patching for this purpose.  Here is another approach, intermingling some of
the suggestions given thus far.

Maintain the config files, again serviceengine.xml as an example. For
settings within this file that are environment- sensitive, assign ant
replacement variables, such as @poolname@.  Then create environment files
such as servicengine.xml.env-production, etc.  That only have mappinsg of
the replacement variables.

The result here is self-documenting configuration files that indicate
environment-sensitive settings.  And you get the benefit of a single file
and no patching issues later!

Thoughts?

 - Lon

-----Original Message-----
From: [hidden email] [mailto:[hidden email]]
On Behalf Of Andrew Sykes
Sent: Thursday, February 16, 2006 1:25 PM
To: OFBiz Users / Usage Discussion
Subject: Re: [OFBiz] Users - Debugging Scheduled Jobs Tip

Lon,

We tried something similar to this a while back, the big problem with this
is you end up having to maintain a whole file when perhaps all you want to
do is adjust a single line in that file.

You end up with a load of extra work, because your version of the file can
quickly go stale.

We have chosen therefore to use patches instead, the advantage here is that
if someone enhances a part of the file you don't care about, you still get
the benefit. Occasionally your patches fail if a file has changed
significantly, but this too can be a good thing, because it flags up that
something you rely on has changed. Imagine how much can slip through the net
if you are effectively just overwriting the latest copy of the file with a
version you made 500 commits ago!
--
Kind Regards
Andrew Sykes <[hidden email]> Sykes Development Ltd
http://www.sykesdevelopment.com

 
_______________________________________________
Users mailing list
[hidden email]
http://lists.ofbiz.org/mailman/listinfo/users


 
_______________________________________________
Users mailing list
[hidden email]
http://lists.ofbiz.org/mailman/listinfo/users
Reply | Threaded
Open this post in threaded view
|

Re: Users - Debugging Scheduled Jobs Tip

Andrew Sykes
In reply to this post by Lon F. Binder-2
Lon,

I have tried this, but it pretty much leads to the same problems as
replacing whole files.

If it wasn't for the sheer flexibility of OFBiz config, I'd suggest this
as something to add to the trunk, but it would be so difficult to
maintain the hundreds of variables involved. You also have to remember
that while most setups involve just tweaking the files, for others it's
a matter of writing whole new blocks of XML. So some setups just need to
replace an attribute value whereas others require adding line upon line.

I think the only way this can work is if the trunk follows this pattern,
but this would make life so much more difficult for the unfortunate few
that I don't really think it's practical.

One solution you may want to consider (though I've never tried this).
Perhaps you could create a set of patches and place your @variables@ in
the patch files. That way you'd only have a single set of patches, but
could customise them for your various environments.
--
Kind Regards
Andrew Sykes <[hidden email]>
Sykes Development Ltd
http://www.sykesdevelopment.com

 
_______________________________________________
Users mailing list
[hidden email]
http://lists.ofbiz.org/mailman/listinfo/users