How to deploy OFBiz without Gradle

[taher]

Yeah well your input is definitely informative and useful. I think we
probably face this issue because of switching to remote libraries more than
changing the build system.

FYI I have my entire infrastructure managed by docker images through
Ansible. I almost never SSH to any server anymore. So if you want to do any
kind of checking or maintenance on your nodes you would just issue one
command or one script against all the nodes in a certain group. In fact you
can delegate the entire build process to the nodes as this would probably
be faster than one Master server publishing everywhere, especially when you
use the parallel options. So if you haven't already then I highly recommend
trying ansible. You can also try chef or puppet but I think they are a bit
more complex with a steeper learning curve.

On Aug 31, 2016 8:59 AM, "Scott Gray" <[hidden email]> wrote:

> To be honest I really don't have much input.  Because I pretty much only
> work with OFBiz and the ant builds always just worked, I really don't have
> much experience in dealing with build technologies.
>
> I was mostly just writing the email to highlight how one large user/company
> of OFBiz deploys currently.  The advantage for them is that the deployment
> itself is very quick because the build has already happened on another
> server earlier, but I guess it's not that big of a deal if it only happens
> on the first compile.  Their approach does also help with consistency as we
> had an issue for a while where random compilations would cause a runtime
> error in a particular groovy script.  Rebuilding would fix the issue, but
> if we had to check every node for it individually because of separate
> compilations then it would have been a much bigger PITA.
>
> So I guess for them, they could continue to use a build server to deploy to
> nodes from and then they'd just have to run a separate gradle command on
> the production node to download any missing dependencies before starting
> OFBiz.  So all in all, I don't think it's a big deal and not something I'm
> going to complain about.
>
> Regards
> Scott
>
> On 31 August 2016 at 17:28, Taher Alkhateeb <[hidden email]>
> wrote:
>
> > Hi Scott,.
> >
> > I will explain the build process to analyze the problem to try and come
> up
> > with a solution:
> > - You run ./gradlew build
> > - Gradle downloads all jars
> > - Compile everything
> > - Create a jar that embeds inside its manifest the classpath of all the
> > jars (that are sitting in cache) So the location of the jars is hardcoded
> > into the created jar. This makes for a far cleaner executable than
> calling
> > the classpath in the java command because then the java command would be
> > tousands of characters long and you will not see it on the screen.
> >
> > Let's say you created OFBiz with your modifications, database and
> > everything ready to deploy and your server has an internet connection.
> The
> > moment you deploy it and run in the server ./gradlew "ofbiz --start" it
> > will download the missing jars for you and start the server. So you do
> not
> > need to necessarily do anything
> >
> > To create a self contained binary and continue using gradle to control
> the
> > server means multiple changes as follows:
> > - Must copy all libs inside OFBiz
> > - Alter the jar build to point the classpath to embedded location (/libs
> > for example)
> > - Everytime the dependencies change for whatever reason (if you for
> example
> > disable a component with its dependencies you might get new dependencies
> > exposed), you have to recopy from the cache. I think we can do that
> > programmatically through the gradle event model.
> >
> > So we need a solution that fits everyone. I think copying the
> dependencies
> > by default is not the best solution because you waste space and
> complicate
> > the build script every time there is a change in the dependencies. I list
> > below some solutions. Not sure which one is best but at least this
> > kickstarts the conversation:
> >
> > - One solution is to create a rule-based task (similar to ofbiz,
> > ofbizDebug, ofbizBackground) and maybe call it (ofbizBinary or
> > ofbizContained or something like that). This task would contain the
> custom
> > logic such that upon any change in dependencies it downloads the changes
> > and recopies the cache, and then create the jar holding the proper
> > classpath. However we need to decide whether to copy runtime or
> everything
> > (compile time)
> >
> > - Another solution is similar to the above, but without the logic to
> detect
> > dependency change assuming the deployment environment does not change. So
> > you create and copy the dependencies once.
> >
> > - Yet another solution is similar to what Pierre did, just create a task
> > to copy the dependencies and you run java manually against them. Again we
> > need to decide on dependencies (I think it should be everything not just
> > runtime), however this means stripping away gradle which adds complexity
> to
> > less experienced users because they have to add jvm args and run the
> > commands properly.
> >
> > - And for completeness my original proposal, just let Gradle handle it
> > because:
> >   - You will consume bandwidth either way (server to server or jcenter to
> > server)
> >   - The build script will be simpler and cleaner
> >   - The deployed system will be open to change in dependencies and
> > automatically handle it
> >   - Externalizing dependencies is not uncommon at all. It is the default
> > with Django, Rails, Node.js, and even Java (inside .m2 directory). People
> > usually do not want to deal with the dependency headache directly in many
> > newer systems.
> >
> > If you have a preference for one of the above solutions then we can start
> > exploring it in more depth and enhancing it, or if you have other
> > suggestions and ideas please fire away.
> >
> > Regards,
> >
> > Taher Alkhateeb
> >
> > On Tuesday, 30 August 2016, Scott Gray <[hidden email]>
> > wrote:
> >
> > > I just want to add that I don't think it is an uncommon scenario to
> > desire
> > > a self-contained ofbiz build.  The project I'm currently working on
> uses
> > > build servers to compile and deploy to test instances and to
> production.
> > > A production deployment for us means deploying to 10+ nodes and we do
> > this
> > > by building once and then pushing that 'binary' version out to the
> > various
> > > nodes.  I guess (if we were ever to upgrade to gradle ofbiz) we could
> > > figure out how to push the gradle jar cache as well so it isn't the end
> > of
> > > the world, but it was nice being able to push a single self contained
> > unit.
> > >
> > > Regards
> > > Scott
> > >
> > > On 31 August 2016 at 01:28, Taher Alkhateeb <
> [hidden email]
> > > <javascript:;>>
> > > wrote:
> > >
> > > > Yeah I'm not sure why it increased, dependency management is a
> > difficult
> > > > task if you have a good tool, and an almost impossible task if done
> > > > manually. Just type ./gradlew dependencies and observe the absolutely
> > > > insane dependency tree. So I can't tell if the size increased because
> > we
> > > > pulled more libraries unnecessarily while converting or if the reason
> > is
> > > > wrong dependencies in the past, maybe probably both!
> > > >
> > > > Anyway, whatever the correct size is, it is still "too much". We
> depend
> > > way
> > > > on too many libraries, which adds complexity and this scary
> dependency
> > > > tree, and so I think the right solution is to start cutting out
> > unneeded
> > > > libraries. I already did multiple rounds in build.gradle and
> identified
> > > > libraries that I think are not needed. Also, if we decide to disable
> > the
> > > > plugins by default (especially BIRT) then that would also
> substantially
> > > > reduce the size. This is one area that needs a lot of cleanup.
> > > >
> > > > On Tue, Aug 30, 2016 at 3:51 PM, Jacques Le Roux <
> > > > [hidden email] <javascript:;>> wrote:
> > > >
> > > > > I double checked all these jars comes with the birt component. I
> was
> > > also
> > > > > surprised that we had eclipse jars in the Gradle caches!
> > > > >
> > > > > So not a question, and we have effectively 350MB of dependencies
> > > instead
> > > > > of 150MB before.
> > > > >
> > > > > Jacques
> > > > >
> > > > >
> > > > >
> > > > > Le 30/08/2016 à 13:36, Taher Alkhateeb a écrit :
> > > > >
> > > > >> Be careful, those could be just regular OFBiz requirements. For
> > > example,
> > > > >> take a look at the below outputs from ./gradlew dependencies. As
> you
> > > can
> > > > >> see there is the eclipse compiler as a requirement from tomcat.
> Just
> > > > >> because it has the word eclipse does not mean it came from
> eclipse.
> > > > >>
> > > > >> +--- org.apache.tomcat:tomcat-jasper:8.0.36
> > > > >> |    +--- org.apache.tomcat:tomcat-servlet-api:8.0.36
> > > > >> |    +--- org.apache.tomcat:tomcat-juli:8.0.36
> > > > >> |    +--- org.apache.tomcat:tomcat-jsp-api:8.0.36 (*)
> > > > >> |    +--- org.apache.tomcat:tomcat-el-api:8.0.36
> > > > >> |    +--- org.eclipse.jdt.core.compiler:ecj:4.5
> > > > >>
> > > > >>
> > > > >> On Tue, Aug 30, 2016 at 2:30 PM, Jacques Le Roux <
> > > > >> [hidden email] <javascript:;>> wrote:
> > > > >>
> > > > >> I mean those using Intellij don't run gradlew eclipse, so have
> less
> > > > stuff
> > > > >>> in cache, no?
> > > > >>>
> > > > >>> OK, I just answered myself :) There are 30 MB of *eclipse*.jar in
> > the
> > > > >>> Gradle caches, so it's only 320 MB of dependencies related to
> only
> > > > OFBiz
> > > > >>>
> > > > >>> Jacques
> > > > >>>
> > > > >>>
> > > > >>>
> > > > >>> Le 30/08/2016 à 13:18, Taher Alkhateeb a écrit :
> > > > >>>
> > > > >>> Hmmm, not sure if Intellij or eclipse would make a difference in
> > the
> > > > >>>> cache
> > > > >>>> size? It is gradle that is downloading, not eclipse nor
> intellij.
> > > > >>>>
> > > > >>>> On Tue, Aug 30, 2016 at 2:02 PM, Jacques Le Roux <
> > > > >>>> [hidden email] <javascript:;>> wrote:
> > > > >>>>
> > > > >>>> Le 30/08/2016 à 12:43, Taher Alkhateeb a écrit :
> > > > >>>>
> > > > >>>>> Hi Jacques,
> > > > >>>>>
> > > > >>>>>> I think to get the absolute minimum cache size you do the
> > > following:
> > > > >>>>>> - delete .gradle
> > > > >>>>>> - ./gradlew cleanAll build (do not run eclipse so that you do
> > not
> > > > >>>>>> download
> > > > >>>>>> the source dependencies)
> > > > >>>>>>
> > > > >>>>>> This would give you the minimum cache needed for OFBiz to run.
> > > > >>>>>>
> > > > >>>>>> Yep, sure if someone using IntelIJ could share the burden that
> > > would
> > > > >>>>>> be
> > > > >>>>>>
> > > > >>>>> easier for me ;)
> > > > >>>>> Not a big deal anyway
> > > > >>>>>
> > > > >>>>> Jacques
> > > > >>>>>
> > > > >>>>>
> > > > >>>>> Taher Alkhateeb
> > > > >>>>>
> > > > >>>>> On Tue, Aug 30, 2016 at 1:38 PM, Jacques Le Roux <
> > > > >>>>>> [hidden email] <javascript:;>> wrote:
> > > > >>>>>>
> > > > >>>>>> Le 30/08/2016 à 11:25, Taher Alkhateeb a écrit :
> > > > >>>>>>
> > > > >>>>>> Hi Jacques,
> > > > >>>>>>>
> > > > >>>>>>> I know you probably meant this as a friendly joke but this is
> > our
> > > > >>>>>>>> official
> > > > >>>>>>>> ML, so I have to state for the record that I did not play a
> > > > "trick",
> > > > >>>>>>>> my
> > > > >>>>>>>> objective was to say that you changed the topic and
> therefore
> > we
> > > > >>>>>>>> need
> > > > >>>>>>>> to
> > > > >>>>>>>> start a new thread. Choosing the new topic is entirely
> within
> > > your
> > > > >>>>>>>> control.
> > > > >>>>>>>>
> > > > >>>>>>>> Yes of course, only a friendly joke :)
> > > > >>>>>>>>
> > > > >>>>>>>> Now as you probably already know Gradle has different
> > > dependencies
> > > > >>>>>>> such
> > > > >>>>>>> as
> > > > >>>>>>>
> > > > >>>>>>> compile and runtime. Part of the difference in size could be
> > due
> > > to
> > > > >>>>>>>
> > > > >>>>>>>> copying
> > > > >>>>>>>> only one of these dependencies and not all of them. For
> > example
> > > > the
> > > > >>>>>>>> copy
> > > > >>>>>>>> libs task (discussed earlier) only copied runtime
> > dependencies.
> > > > But
> > > > >>>>>>>> is
> > > > >>>>>>>> this
> > > > >>>>>>>> the right thing to do? are you not going to compile anything
> > in
> > > > the
> > > > >>>>>>>> future
> > > > >>>>>>>> in the production environment? Maybe yes maybe no it depends
> > > > doesn't
> > > > >>>>>>>> it?
> > > > >>>>>>>> It's different from one deployment to another. Therefore it
> is
> > > > >>>>>>>> specific
> > > > >>>>>>>> to
> > > > >>>>>>>> each user in their own environment.
> > > > >>>>>>>>
> > > > >>>>>>>> Yes, that's why I did not continue this way. I have though
> > still
> > > > to
> > > > >>>>>>>>
> > > > >>>>>>>> find a
> > > > >>>>>>> right solution for OWASP-DC
> > > > >>>>>>> I mean https://issues.apache.org/jira/browse/OFBIZ-7930
> > > > >>>>>>>
> > > > >>>>>>> Another reason could be that the development machine contains
> > > > >>>>>>> additional
> > > > >>>>>>>
> > > > >>>>>>> unneeded dependencies. So if you try to delete the cache
> folder
> > > and
> > > > >>>>>>> run
> > > > >>>>>>>
> > > > >>>>>>>> the
> > > > >>>>>>>> build again you might get a smaller size upon copying.
> > > > >>>>>>>>
> > > > >>>>>>>> I just tried, after downloading the Internet again
> > (kiiiidiiing
> > > > ;))
> > > > >>>>>>>> it's
> > > > >>>>>>>>
> > > > >>>>>>>> indeed much smaller (Eclipse included) it's only 350 MB, a
> > good
> > > > >>>>>>> news!
> > > > >>>>>>>
> > > > >>>>>>> BTW handling (copying, deleting, moving) the caches on
> Windows
> > is
> > > > "a
> > > > >>>>>>> bit"
> > > > >>>>>>> long. Because Windows does not handle well a folder with
> plenty
> > > of
> > > > >>>>>>> files
> > > > >>>>>>> (I
> > > > >>>>>>> guess some are small did not check).
> > > > >>>>>>> Not a big deal since most of the time (if not all the time)
> > > Windows
> > > > >>>>>>> is
> > > > >>>>>>> not
> > > > >>>>>>> used as a server.
> > > > >>>>>>>
> > > > >>>>>>> Also for the record, if no internet connection is a
> substantial
> > > > >>>>>>> enough
> > > > >>>>>>>
> > > > >>>>>>> problem and multiple people are facing it then it makes sense
> > > that
> > > > we
> > > > >>>>>>>
> > > > >>>>>>>> help
> > > > >>>>>>>> our users, but this needs to be discussed thoroughly in ML
> to
> > > come
> > > > >>>>>>>> up
> > > > >>>>>>>> with
> > > > >>>>>>>> a good clean solution before starting multiple jiras like
> the
> > > ones
> > > > >>>>>>>> mentioned earlier. To me personally I don't think this is a
> > big
> > > > >>>>>>>> issue
> > > > >>>>>>>> but
> > > > >>>>>>>> I
> > > > >>>>>>>> could be wrong so I leave it to others to have their say.
> > > > >>>>>>>>
> > > > >>>>>>>> I agree a plethora of Jira is not good. I think we have
> > > discussed
> > > > >>>>>>>> this
> > > > >>>>>>>>
> > > > >>>>>>>> enough, now we need to continue to update the documentation.
> > > > >>>>>>> For that you need 1st to know what you are talking about,
> hence
> > > > this
> > > > >>>>>>> discussion indeed.
> > > > >>>>>>>
> > > > >>>>>>> I wonder about the dependencies introduced in Gradle cache by
> > > > Eclipse
> > > > >>>>>>> Could someone using IntelliJ confirm it has much less than
> 350
> > MB
> > > > in
> > > > >>>>>>> cache?
> > > > >>>>>>>
> > > > >>>>>>> Thanks
> > > > >>>>>>>
> > > > >>>>>>> Jacques
> > > > >>>>>>>
> > > > >>>>>>>
> > > > >>>>>>>
> > > > >>>>>>> Regards,
> > > > >>>>>>>
> > > > >>>>>>> Taher Alkhateeb
> > > > >>>>>>>>
> > > > >>>>>>>> On Aug 30, 2016 11:50 AM, "Jacques Le Roux" <
> > > > >>>>>>>> [hidden email] <javascript:;>
> > > > >>>>>>>> wrote:
> > > > >>>>>>>>
> > > > >>>>>>>> Le 30/08/2016 à 09:21, Taher Alkhateeb a écrit :
> > > > >>>>>>>>
> > > > >>>>>>>> Hi Jacques, All,
> > > > >>>>>>>>
> > > > >>>>>>>>> Okay just to put things into context, the reason that
> > > > kick-started
> > > > >>>>>>>>>
> > > > >>>>>>>>>> this
> > > > >>>>>>>>>> discussion (multiple times so far) is that Pierre Smits
> had
> > > > >>>>>>>>>> deployment
> > > > >>>>>>>>>> requirements in which Gradle is not allowed as detailed in
> > > this
> > > > >>>>>>>>>> thread:
> > > > >>>>>>>>>> http://markmail.org/message/dzq3e55n6z4cwmre
> > > > >>>>>>>>>>
> > > > >>>>>>>>>> To make things short, I was of the opinion that Pierre's
> > > request
> > > > >>>>>>>>>> represents
> > > > >>>>>>>>>> unusual and specific deployment requirements and that it
> > > should
> > > > >>>>>>>>>> not
> > > > >>>>>>>>>> be
> > > > >>>>>>>>>> the
> > > > >>>>>>>>>> default way to deploy OFBiz because people would lose all
> > the
> > > > >>>>>>>>>> power
> > > > >>>>>>>>>> and
> > > > >>>>>>>>>> advantages from having a powerful build system automating
> > > > >>>>>>>>>> everything.
> > > > >>>>>>>>>> The
> > > > >>>>>>>>>> discussions were had between myself, Pierre, Jacques with
> > > > >>>>>>>>>> occasional
> > > > >>>>>>>>>> input
> > > > >>>>>>>>>> from others.
> > > > >>>>>>>>>>
> > > > >>>>>>>>>> Despite my above position, the following JIRAs were
> created
> > > and
> > > > >>>>>>>>>> had
> > > > >>>>>>>>>> patches
> > > > >>>>>>>>>> / initiatives, all of which were very specific to the
> > > deployment
> > > > >>>>>>>>>> requirements of Pierre:
> > > > >>>>>>>>>>
> > > > >>>>>>>>>> - https://issues.apache.org/jira/browse/OFBIZ-7782 -> The
> > > JIRA
> > > > >>>>>>>>>> had
> > > > >>>>>>>>>> a
> > > > >>>>>>>>>> patch
> > > > >>>>>>>>>> to create a task to copy all OFBiz runtime libraries to a
> > > > certain
> > > > >>>>>>>>>> folder.
> > > > >>>>>>>>>> I
> > > > >>>>>>>>>> objected to it as being too specific to the deployment
> > > > >>>>>>>>>> requirements
> > > > >>>>>>>>>> of
> > > > >>>>>>>>>> Pierre.
> > > > >>>>>>>>>> - https://issues.apache.org/jira/browse/OFBIZ-7893 ->
> > > > >>>>>>>>>> Reintroduce a
> > > > >>>>>>>>>> task
> > > > >>>>>>>>>> to
> > > > >>>>>>>>>> copy all OFBiz runtime libraries to a certain folder and
> > > remove
> > > > >>>>>>>>>> gradle
> > > > >>>>>>>>>> from
> > > > >>>>>>>>>> the deployment scripts in /tools. I objected saying this
> is
> > > the
> > > > >>>>>>>>>> wrong
> > > > >>>>>>>>>> this
> > > > >>>>>>>>>> to do and that I already objected in OFBIZ-7782
> > > > >>>>>>>>>> - https://issues.apache.org/jira/browse/OFBIZ-7796 ->
> > > Attempted
> > > > >>>>>>>>>> to
> > > > >>>>>>>>>> strip
> > > > >>>>>>>>>> away gradle from the deployment scripts and replace it
> with
> > > > Java.
> > > > >>>>>>>>>> Again
> > > > >>>>>>>>>> I
> > > > >>>>>>>>>> objected to it for being too specific to the deployment
> > > > >>>>>>>>>> requirements
> > > > >>>>>>>>>> of
> > > > >>>>>>>>>> Pierre.
> > > > >>>>>>>>>>
> > > > >>>>>>>>>> Sorry for the long introduction but the context is
> important
> > > for
> > > > >>>>>>>>>> people
> > > > >>>>>>>>>> to
> > > > >>>>>>>>>> know where we stand exactly. Now to your question of how
> to
> > > > deploy
> > > > >>>>>>>>>> OFBiz
> > > > >>>>>>>>>> without Gradle, I would like to reply in two parts:
> > > > >>>>>>>>>>
> > > > >>>>>>>>>> OK you played a trick here Taher by suggesting me the
> title
> > :D
> > > > >>>>>>>>>>
> > > > >>>>>>>>>> The problem I tried to tackle is not "How to deploy OFBiz
> > > > without
> > > > >>>>>>>>>>
> > > > >>>>>>>>> Gradle"
> > > > >>>>>>>>> but "How to use Gradle w/o Internet connection"
> > > > >>>>>>>>>
> > > > >>>>>>>>> 1- Why without Gradle?
> > > > >>>>>>>>>
> > > > >>>>>>>>> 2- Available Options
> > > > >>>>>>>>>
> > > > >>>>>>>>> Why without Gradle?
> > > > >>>>>>>>>> ---------------------------
> > > > >>>>>>>>>> Imagine if I tell you that I want to deploy a
> ruby-on-rails
> > > > >>>>>>>>>> application
> > > > >>>>>>>>>> without rails. Or I want to deploy OFBiz without the
> widget
> > > > >>>>>>>>>> engine.
> > > > >>>>>>>>>> Is
> > > > >>>>>>>>>> that
> > > > >>>>>>>>>> a normal requirement? No, it is not a normal requirement,
> it
> > > is
> > > > a
> > > > >>>>>>>>>> requirement specific to my needs.
> > > > >>>>>>>>>>
> > > > >>>>>>>>>> Gradle is a fundamental part of the OFBiz solution.
> > Stripping
> > > it
> > > > >>>>>>>>>> away
> > > > >>>>>>>>>> is
> > > > >>>>>>>>>> like stripping away a core component. It is already
> holding
> > a
> > > > lot
> > > > >>>>>>>>>> of
> > > > >>>>>>>>>> responsibilities and expected to hold a lot more. You must
> > > have
> > > > a
> > > > >>>>>>>>>> good
> > > > >>>>>>>>>> reason to want to remove it, and you should have the
> > technical
> > > > >>>>>>>>>> capabilities
> > > > >>>>>>>>>> to do such a thing.
> > > > >>>>>>>>>>
> > > > >>>>>>>>>> So, it is in my opinion unreasonable to require the code
> > base
> > > to
> > > > >>>>>>>>>> have
> > > > >>>>>>>>>> a
> > > > >>>>>>>>>> solution to remove Gradle as an option (A task defined
> > inside
> > > > >>>>>>>>>> Gradle
> > > > >>>>>>>>>> to
> > > > >>>>>>>>>> copy libs so you can later run java -jar). Because if that
> > is
> > > > >>>>>>>>>> proper
> > > > >>>>>>>>>> then
> > > > >>>>>>>>>> we should have such options in the system to remove the
> > > service
> > > > >>>>>>>>>> engine,
> > > > >>>>>>>>>> the
> > > > >>>>>>>>>> entity engine, the widgets, and many other fundamental
> > > > components.
> > > > >>>>>>>>>>
> > > > >>>>>>>>>> Available Options
> > > > >>>>>>>>>> -----------------------
> > > > >>>>>>>>>> Taking into consideration that this is an _advanced_ topic
> > and
> > > > >>>>>>>>>> those
> > > > >>>>>>>>>> involved _should_ have the technical capacity to do it
> > > > themselves;
> > > > >>>>>>>>>> there
> > > > >>>>>>>>>> are different scenarios we are discussing here and I
> > provide a
> > > > >>>>>>>>>> sample
> > > > >>>>>>>>>> solution for each:
> > > > >>>>>>>>>>
> > > > >>>>>>>>>> 1- No internet connection: One solution is to copy the
> > .gradle
> > > > >>>>>>>>>> directory
> > > > >>>>>>>>>> to
> > > > >>>>>>>>>> the deployment server and run all gradle commands with
> > > --offline
> > > > >>>>>>>>>>
> > > > >>>>>>>>>> That's what I did above but only with the caches (and the
> > > needed
> > > > >>>>>>>>>>
> > > > >>>>>>>>>> wrapper).
> > > > >>>>>>>>>>
> > > > >>>>>>>>> It works but needs 10 times more disk space than before.
> > > > >>>>>>>>> Since it's local and servers are supposed to have
> sufficient
> > > disk
> > > > >>>>>>>>> spaces,
> > > > >>>>>>>>> it's only a problem of bandwidth to copy the Gradle cache
> > each
> > > > time
> > > > >>>>>>>>> it's
> > > > >>>>>>>>> changed
> > > > >>>>>>>>> To clarify the OFBiz compiled running code is all
> > > > >>>>>>>>> build\libs\ofbiz.jar
> > > > >>>>>>>>> and
> > > > >>>>>>>>> only the external dependencies are in the the Gradle cache,
> > > > right?
> > > > >>>>>>>>>
> > > > >>>>>>>>> BTW I also tried with the whole .gradle directory (the one
> in
> > > > your
> > > > >>>>>>>>> user
> > > > >>>>>>>>> folder) but got issues because I guess I tried only on
> > Windows
> > > > (too
> > > > >>>>>>>>> long
> > > > >>>>>>>>> file paths).
> > > > >>>>>>>>> I see no reasons it would not work, since by simply copying
> > the
> > > > >>>>>>>>> caches
> > > > >>>>>>>>> and
> > > > >>>>>>>>> the wrapper it worked for me (only slighter less space:
> 10MB)
> > > > >>>>>>>>>
> > > > >>>>>>>>> 2- Gradle not allowed: One solution is to copy the .gradle
> > > > >>>>>>>>> directory
> > > > >>>>>>>>> to
> > > > >>>>>>>>> the
> > > > >>>>>>>>>
> > > > >>>>>>>>> deployment server and run java -jar build/ofbiz.jar (make
> > sure
> > > to
> > > > >>>>>>>>> put
> > > > >>>>>>>>>
> > > > >>>>>>>>> the
> > > > >>>>>>>>>> correct JVM arguments)
> > > > >>>>>>>>>>
> > > > >>>>>>>>>> Yep, that's also the reason I kept and updated this
> > > information
> > > > in
> > > > >>>>>>>>>>
> > > > >>>>>>>>>> https://cwiki.apache.org/confluence/display/OFBIZ/Apache+
> > > > >>>>>>>>>>
> > > > >>>>>>>>> OFBiz+Technical+Production+Setup+Guide, despite your
> > > reluctance
> > > > ;)
> > > > >>>>>>>>> I still need to complete it...
> > > > >>>>>>>>>
> > > > >>>>>>>>> Of course the option proposed by Pierre and yourself by
> > having
> > > a
> > > > >>>>>>>>> gradle
> > > > >>>>>>>>>
> > > > >>>>>>>>> task to copy the runtime libraries to some location and
> then
> > > run
> > > > >>>>>>>>> java
> > > > >>>>>>>>>
> > > > >>>>>>>>> -jar
> > > > >>>>>>>>>> adding that folder to the classpath works. But it is too
> > > > specific
> > > > >>>>>>>>>> and
> > > > >>>>>>>>>> awkward.
> > > > >>>>>>>>>>
> > > > >>>>>>>>>> The only reason is you have 10(!) times less data to move
> > > > between
> > > > >>>>>>>>>>
> > > > >>>>>>>>>> machines...
> > > > >>>>>>>>>>
> > > > >>>>>>>>> If you want to customize things to your liking, it is
> > extremely
> > > > >>>>>>>>>
> > > > >>>>>>>>> easy to create a gradle subproject (a file somewhere) and
> put
> > > all
> > > > >>>>>>>>> the
> > > > >>>>>>>>>
> > > > >>>>>>>>> custom deployment logic in it, and then just create a patch
> > > that
> > > > >>>>>>>>>> adds
> > > > >>>>>>>>>> "
> > > > >>>>>>>>>> apply from: 'foo/bar/my-custom-build-script.gradle' "
> > > > >>>>>>>>>>
> > > > >>>>>>>>>> OK, I'll consider it (as I did with the 1st post of this
> > > commit)
> > > > >>>>>>>>>> before
> > > > >>>>>>>>>>
> > > > >>>>>>>>>> adding it in the doc
> > > > >>>>>>>>>>
> > > > >>>>>>>>> As you can see, I do that to document the possibilities for
> > our
> > > > >>>>>>>>> users.
> > > > >>>>>>>>> Because I know it will happen and it's good for adoption.
> > > > >>>>>>>>>
> > > > >>>>>>>>> So to conclude, if we succumb to every deployment scenario
> > for
> > > > >>>>>>>>> every
> > > > >>>>>>>>>
> > > > >>>>>>>>> person, we would indeed have a very big code base. I
> suggest
> > to
> > > > >>>>>>>>> keep
> > > > >>>>>>>>>
> > > > >>>>>>>>> things
> > > > >>>>>>>>>> lean and clean, and to avoid adding anything to the code
> > base
> > > > for
> > > > >>>>>>>>>> the
> > > > >>>>>>>>>> specific exceptional deployment requirements of some
> > > > individuals.
> > > > >>>>>>>>>>
> > > > >>>>>>>>>> I totally agree on that!
> > > > >>>>>>>>>>
> > > > >>>>>>>>>> Jacques
> > > > >>>>>>>>>>
> > > > >>>>>>>>>
> > > > >>>>>>>>> Regards,
> > > > >>>>>>>>>
> > > > >>>>>>>>> Taher Alkhateeb.
> > > > >>>>>>>>>
> > > > >>>>>>>>>> In case of no internet connection, there are multiple
> > options:
> > > > >>>>>>>>>> - Copy the .gradle directory to the server and run all
> > gradle
> > > > >>>>>>>>>> commands
> > > > >>>>>>>>>> with
> > > > >>>>>>>>>> --offline
> > > > >>>>>>>>>> -
> > > > >>>>>>>>>>
> > > > >>>>>>>>>> On Tue, Aug 30, 2016 at 9:21 AM, Jacques Le Roux <
> > > > >>>>>>>>>> [hidden email] <javascript:;>> wrote:
> > > > >>>>>>>>>>
> > > > >>>>>>>>>> Hi,
> > > > >>>>>>>>>>
> > > > >>>>>>>>>> I just tried something which seemed very simple after
> > reading
> > > > >>>>>>>>>> Taher's
> > > > >>>>>>>>>>
> > > > >>>>>>>>>> suggestion
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>> in OFBIZ-7783
> > > > >>>>>>>>>>> "For example if your problem is simply that you cannot
> > build
> > > > on a
> > > > >>>>>>>>>>> disconnected computer even though the gradle cache is
> > > available
> > > > >>>>>>>>>>> then
> > > > >>>>>>>>>>> the
> > > > >>>>>>>>>>> solution is to issue the command ./gradlew --offline
> > > > >>>>>>>>>>> tasks-in-here.
> > > > >>>>>>>>>>> So
> > > > >>>>>>>>>>> one
> > > > >>>>>>>>>>> solution is to simply archive the gradle cache and
> restore
> > > it."
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>> in the "Should we do binary releases?" thread
> > > > >>>>>>>>>>> "You can also copy the .gradle cache from another
> computer
> > > and
> > > > >>>>>>>>>>> start
> > > > >>>>>>>>>>> using
> > > > >>>>>>>>>>> it with the --offline flag"
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>> My idea was to mix --offline with --gradle-user-home
> Gradle
> > > > >>>>>>>>>>> commands,
> > > > >>>>>>>>>>> because I wanted to do this on the same machine and did
> not
> > > > know
> > > > >>>>>>>>>>> where
> > > > >>>>>>>>>>> to
> > > > >>>>>>>>>>> copy the .gradle cache (where the dependencies are)
> > > > >>>>>>>>>>> -g, --gradle-user-home  <->   Specifies the Gradle user
> > home
> > > > >>>>>>>>>>> directory.
> > > > >>>>>>>>>>> The default is the .gradle directory in the user's home
> > > > >>>>>>>>>>> directory.
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>> So, *on the same machine*, I copied the cache (830 MB)
> from
> > > > >>>>>>>>>>> .gradle
> > > > >>>>>>>>>>> directory to another place (I picked the shortest one, ie
> > > > >>>>>>>>>>> c:\gradle). I
> > > > >>>>>>>>>>> got
> > > > >>>>>>>>>>> 9 issues with file names too long (was surprised about
> that
> > > > since
> > > > >>>>>>>>>>> from
> > > > >>>>>>>>>>> the
> > > > >>>>>>>>>>> user's home directory they should be longer)
> > > > >>>>>>>>>>> Then tried to use "gradlew --offline -g c:\gradle ofbiz"
> > but
> > > > got
> > > > >>>>>>>>>>> a
> > > > >>>>>>>>>>> syntax
> > > > >>>>>>>>>>> error and Gradle began to download the Internet again:
> > > > >>>>>>>>>>> La syntaxe du nom de fichier, de répertoire ou de volume
> > est
> > > > >>>>>>>>>>> incorrecte.
> > > > >>>>>>>>>>> "Downloading https://services.gradle.org/di
> > > > >>>>>>>>>>> stributions/gradle-2.13-bin.zip
> > > > >>>>>>>>>>> "
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>> I stopped. I guess I missed something, and rather decided
> > to
> > > > set
> > > > >>>>>>>>>>> the
> > > > >>>>>>>>>>> set
> > > > >>>>>>>>>>> GRADLE_USER_HOME to c:\gradle in the gradlew.bat script.
> > but
> > > > then
> > > > >>>>>>>>>>> got
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>> Exception in thread "main" java.io.FileNotFoundException:
> > > > >>>>>>>>>>> "c:\gradle"\wrapper\dists\gradle-2.13-bin\
> > 4xsgxlfjcxvrea7akf
> > > > >>>>>>>>>>> 941nvc7\
> > > > >>>>>>>>>>> gradle-2.13-bin.zip.lck (La syntaxe du nom de fichier, de
> > > > >>>>>>>>>>> répertoire
> > > > >>>>>>>>>>> ou
> > > > >>>>>>>>>>> de volumeest incorrecte)
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>> I then copied the missing wrapper folder in c:\gradle
> (400
> > > MB).
> > > > >>>>>>>>>>> Despite
> > > > >>>>>>>>>>> Windows and its damned limitation on paths names,  it
> then
> > > > worked
> > > > >>>>>>>>>>> perfectly
> > > > >>>>>>>>>>> well.
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>> But it's still disappointing because you have to copy 1,2
> > GB
> > > > >>>>>>>>>>> instead
> > > > >>>>>>>>>>> of
> > > > >>>>>>>>>>> 150 MB (160MB including OFBiz jars)
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>> So my next question: should we use that as an advice to
> our
> > > > users
> > > > >>>>>>>>>>> who
> > > > >>>>>>>>>>> can't use Gradle on their production and alike machines,
> or
> > > > >>>>>>>>>>> should
> > > > >>>>>>>>>>> we
> > > > >>>>>>>>>>> try
> > > > >>>>>>>>>>> to refine and find a better option?
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>> Thanks
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>> Jacques
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>>
> > > > >
> > > >
> > >
> >
>

[Jacques Le Roux]

Le 31/08/2016 à 07:28, Taher Alkhateeb a écrit :

> - And for completeness my original proposal, just let Gradle handle it
> because:
>    - You will consume bandwidth either way (server to server or jcenter to
> server)
>    - The build script will be simpler and cleaner
>    - The deployed system will be open to change in dependencies and
> automatically handle it
>    - Externalizing dependencies is not uncommon at all. It is the default
> with Django, Rails, Node.js, and even Java (inside .m2 directory). People
> usually do not want to deal with the dependency headache directly in many
> newer systems.

Like Scott, it's also OK with me. There is though still one thing I slightly worry about, as you said

> People usually do not want to deal with the dependency headache directly in many newer systems.

I can understand that!
But I have to check Gradle is doing a good job concerning disclosed vulnerabilities.
I mean that it always automatically downloads the latest safe external libs versions. I guess it does but I have still to check that. This is the
purpose of OFBIZ-7930

Jacques

[taher]

Hi Jacques, yes security is another topic and we can start a new thread on
that if you wish. Suffice to say in here that dependency management in
gradle although automatic allows you to customize dependencies. You can use
patterns to say give me the latest minor version or latest major version or
exclude this transitive dependency and so on and so forth.

On Aug 31, 2016 2:03 PM, "Jacques Le Roux" <[hidden email]>
wrote:

> Le 31/08/2016 à 07:28, Taher Alkhateeb a écrit :
>
>> - And for completeness my original proposal, just let Gradle handle it
>> because:
>>    - You will consume bandwidth either way (server to server or jcenter to
>> server)
>>    - The build script will be simpler and cleaner
>>    - The deployed system will be open to change in dependencies and
>> automatically handle it
>>    - Externalizing dependencies is not uncommon at all. It is the default
>> with Django, Rails, Node.js, and even Java (inside .m2 directory). People
>> usually do not want to deal with the dependency headache directly in many
>> newer systems.
>>
>
> Like Scott, it's also OK with me. There is though still one thing I
> slightly worry about, as you said
>
> People usually do not want to deal with the dependency headache directly
>> in many newer systems.
>>
>
> I can understand that!
> But I have to check Gradle is doing a good job concerning disclosed
> vulnerabilities.
> I mean that it always automatically downloads the latest safe external
> libs versions. I guess it does but I have still to check that. This is the
> purpose of OFBIZ-7930
>
> Jacques
>
>

[Jacques Le Roux]

Thanks for the tip Taher

Jacques


Le 31/08/2016 à 13:11, Taher Alkhateeb a écrit :

> Hi Jacques, yes security is another topic and we can start a new thread on
> that if you wish. Suffice to say in here that dependency management in
> gradle although automatic allows you to customize dependencies. You can use
> patterns to say give me the latest minor version or latest major version or
> exclude this transitive dependency and so on and so forth.
>
> On Aug 31, 2016 2:03 PM, "Jacques Le Roux" <[hidden email]>
> wrote:
>
>> Le 31/08/2016 à 07:28, Taher Alkhateeb a écrit :
>>
>>> - And for completeness my original proposal, just let Gradle handle it
>>> because:
>>>     - You will consume bandwidth either way (server to server or jcenter to
>>> server)
>>>     - The build script will be simpler and cleaner
>>>     - The deployed system will be open to change in dependencies and
>>> automatically handle it
>>>     - Externalizing dependencies is not uncommon at all. It is the default
>>> with Django, Rails, Node.js, and even Java (inside .m2 directory). People
>>> usually do not want to deal with the dependency headache directly in many
>>> newer systems.
>>>
>> Like Scott, it's also OK with me. There is though still one thing I
>> slightly worry about, as you said
>>
>> People usually do not want to deal with the dependency headache directly
>>> in many newer systems.
>>>
>> I can understand that!
>> But I have to check Gradle is doing a good job concerning disclosed
>> vulnerabilities.
>> I mean that it always automatically downloads the latest safe external
>> libs versions. I guess it does but I have still to check that. This is the
>> purpose of OFBIZ-7930
>>
>> Jacques
>>
>>